09/17/2020

As I mentioned in an earlier post, I wrote a visual editor for building \Psi programs. For most nodes in a \Psi program the inputs/outputs are known ahead of time. For instance, my editor exposes a node for performing a Gaussian blur. The inputs to that node is the image to be blurred. This makes writing the code for that node pretty simple:

var inputStreamName1 = this.inputs[0].Connections[0].Parent.GetOutputStreamName(this.inputs[0].Connections[0]);
Emitter<Shared<Microsoft.Psi.Imaging.Image>> emitter1 = ctx.connectors[inputStreamName1] as Emitter<Shared<Microsoft.Psi.Imaging.Image>>;
var he = Microsoft.Psi.OpenCV.PsiOpenCV.GaussianBlur(emitter1, (int)GetProperty("WindowSize"), (double)GetProperty("Sigma"));
ctx.connectors.Add(GetOutputStreamName(outputs[0]), he.Out);

As can be seen by the bolded text, I know that the emitter has a type Shared<Microsoft.Psi.Imaging.Image>.

However, for some operators, I don't know in advance what the type is. This is where I use C#'s reflection support to call the correct function. For instance, I have a node for \Psi's "return" generator that simply returns an arbitrary (type-wise) object as its output. In order to make this work, I have to run code similar to the following:

object[] args = { ctx.pipeline, value };
var methods = typeof(Generators).GetMethods();
foreach (var method in methods)
{
    if (method.Name == "Return")
    {
        Type[] argTypes = new Type[] { value.GetType() };
        var repeatMethod = method.MakeGenericMethod(argTypes);
        object prod = repeatMethod.Invoke(null, args);
        object outprod = prod.GetType().GetProperty("Out").GetValue(prod);
        ctx.connectors.Add(GetOutputStreamName(outputs[0]), outprod);
        break;
    }

}

As shown the example above, we first need to walk across all the methods (GetMethods()) exposed by type Generators and find the correct method. We then need to make a generic method (MakeGenericMethod) using the types based on the value type. We then have to call the generic method using the arguments we supplied (Invoke()).

This works for the most part. Where this becomes super difficult is when the arguments to the method are multilevel templates and the method we are attempting to call is an extension method. The first problem is finding the correct extension method. We can do this by enumerating over the extension class to find the correct method using something like:

var ms = typeof(Microsoft.Psi.Operators).GetMethods();
foreach (var m in ms)
{
    if (m.Name.Contains("Join"))
    {
        // Possibly use this method
    }
}

One problem with this approach is that there can be many overloads of Join<> (e.g. Join<TPrimary,TSecondary>, Join(TPrimary,TSecondary1,TSecondary2> etc) and we need to find the correct one.

The second problem is that we might have to construct new types to pass to MakeGenericMethod. For instance, here is one of the signatures for one version of Join<>:

public static IProducer<(TPrimary, TSecondary)> Join<TPrimary, TSecondary>(
    this IProducer<TPrimary> primary,
    IProducer<TSecondary> secondary,
    RelativeTimeInterval relativeTimeInterval,
    DeliveryPolicy<TPrimary> primaryDeliveryPolicy = null,
    DeliveryPolicy<TSecondary> secondaryDeliveryPolicy = null)

Notice that the first and second arguments are a template using the TPrimary/TSecondary template specification (i.e. IProducer<TPrimary>). Thus in order to call the generic method we need to do something like:

var q = typeof(IProducer<>);
var i0 = q.MakeGenericType(inputType0);
var i1 = q.MakeGenericType(inputType1);
var qd = typeof(DeliveryPolicy<>);
var q0 = qd.MakeGenericType(inputType0);
var q1 = qd.MakeGenericType(inputType1);
Type[] argTypes = new Type[] { i0, i1, typeof(RelativeTimeInterval), q0, q1};
var repeatMethod = m.MakeGenericMethod(argTypes);

The upshot of all this, is that using reflection to call heavily templatized methods is a royal PITA.

Luckily, there is a much simpler solution. Simpler create a new type! Then instantiate that object, which does all the heavy lifting. This way we avoid all the nested nastiness of types on the method we are trying to call. Below is the implementation for PsiEditorJoin<> using this pattern. This allows us to easily call Join() without worrying about all its various implementations. We let the compiler figure it out.

class PsiEditorJoin<T1,T2>
{
        public void Preview(ref PreviewContext ctx, List<DispConnector> inputs, List<DispConnector> outputs, string outputStreamName)
        {
                var inName0 = inputs[0].Connections[0].Parent.GetOutputStreamName(inputs[0].Connections[0]);
                var emitter0 = ctx.connectors[inName0] as Emitter<T1>;
                var inName1 = inputs[1].Connections[0].Parent.GetOutputStreamName(inputs[1].Connections[0]);
                var emitter1 = ctx.connectors[inName1] as Emitter<T2>;
                var firstArg = emitter0.Join(emitter1, new RelativeTimeInterval(TimeSpan.FromSeconds(-1),         TimeSpan.FromSeconds(1)));
                ctx.connectors.Add(outputStreamName, firstArg.Out);
                inputs[0].ValueType = typeof(T1);
                inputs[1].ValueType = typeof(T2);
                outputs[0].ValueType = typeof((T1, T2));
        }
}

public override void Preview(ref PreviewContext ctx)
{
        if (!visited)
        {
                visited = true;
                base.Preview(ref ctx);
                var inputType0 = this.inputs[0].Connections[0].ValueType;
                var inputType1 = this.inputs[1].Connections[0].ValueType;
                var d1 = typeof(PsiEditorJoin<,>);
                System.Type[] typeArgs = { inputType0, inputType1 };
                var objType = d1.MakeGenericType(typeArgs);
                object o = System.Activator.CreateInstance(objType);

                var previewMethodInfo = objType.GetMethod("Preview");
                object[] parameters = { ctx, this.inputs, this.outputs, GetOutputStreamName(outputs[0]) };
                previewMethodInfo.Invoke(o, parameters);
        }
}

This is way cleaner and simpler to understand.

06/21/2019

After a bunch of rewriting from DX11 to DX12 I finally got my mesh viewer back up and running. Yah!

01/17/2025

FROM:

=====================

Dr. Florent POUXDr. Florent POUX • 3rd+ • 3rd+Founder, Writer, Professor and 3D Innovator. I help Creators build Solutions with 3D and AIFounder, Writer, Professor and 3D Innovator. I help Creators build Solutions with 3D and AI

Visit my website

1d • 1 day ago

Follow

PYTORCH
Conv2d and ConvTranspose2d explained: conv_arithmetic/README.md at master · vdumoulin/conv_arithmetic · GitHub
Convolutions Math 1603.07285

3DGS
VistaDream

CUDA for PyTorch
Pytorch GPU Setup Guide | The MCT Blog (mct-master.github.io)

Unity Related Stuff:
#AltDevBlog » Unity3D coroutines in detail (jahej.com) - Good explanation on how Coroutines probably within Unity

Useful WebRTC links:
https://webrtc.org/
https://webrtc.googlesource.com/src/+/refs/heads/main/docs/native-code/index.md
https://webrtc.github.io/webrtc-org/native-code/development/
https://docs.microsoft.com/en-us/winrtc/getting-started
https://github.com/microsoft/winrtc
https://github.com/sipsorcery-org/sipsorcery/tree/710e3bb3bc9d787df45d97f1f80b4b700e4121f2
https://microsoft.github.io/MixedReality-WebRTC/manual/cs/cs.html
https://github.com/radioman/WebRtc.NET
https://www.c-sharpcorner.com/blogs/webrtc-web-real-time-communication
Really good description of how STUN/TURN works:
2008-08-cluecon-stun-turn-ice.pdf (viagenie.ca)

State of the Art Image Segmentation Survey
https://arxiv.org/pdf/2007.00047.pdf

NeRF rendering
https://www.matthewtancik.com/nerf
2210.13641.pdf (arxiv.org)

Camera Pose Determination
Cameras as Rays: Pose Estimation via Ray Diffusion (jasonyzhang.com)
DUSt3R: Geometric 3D Vision Made Easy (naverlabs.com)

Another great color resource:
Welcome to Bruce Lindbloom's Web Site

Article displaying differences between color distances (CIE1976 vs CIE2000)
https://making.lyst.com/2014/02/22/color-detection/

Good overview of color
Frequently asked questions about color (poynton.ca)

Survery of methods for computing surface normals from point clouds
Comparison of Surface Normal Estimation Methods for Range Sensing Applications

Good overview of PCA
A Step-by-Step Explanation of Principal Component Analysis (PCA) | Built In

ChaiScript: Easily embeddable scripting C-link language
http://chaiscript.com/index.html

Natural Docs (documentation system)
https://naturaldocs.org/

Useful features of C++17
https://www.bfilipek.com/2019/08/17smallercpp17features.html?m=1
https://www.linkedin.com/pulse/21-new-features-modern-c-use-your-project-vishal-chovatiya/
GitHub - hanickadot/compile-time-regular-expressions: Compile Time Regular Expression in C++

C++20 Draft
2019-02 Kona ISO C++ Committee Trip Report

C++ Algorithms
GitHub - HappyCerberus/book-cpp-algorithms: The Standard Algorithms in C++.

Good site for comparing possible software licenses:
https://tldrlegal.com/

Neat little camera
https://openmv.io/

Computing Barycentric coordinates of projected pointhttp://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=ACDF35543780A74C1E36F71B687E2513?doi=10.1.1.165.1208&rep=rep1&type=pdf

Found this useful article on tensor decomposition (and tensors in general):
https://public.ca.sandia.gov/~tgkolda/pubs/bibtgkfiles/SAND2007-6702.pdf

04/29/2019

As of late, I've been doing a lot of work in DirectX 12. Still can't decide if I like it or just flat out hate it. Initially it takes a bit to get around the programming model. I find myself spending an inordinate amount of time tracking down bugs/issues in memory management. So, although I can see wanting to be able to control stuff at a very fine level, I find that for certain tasks where performance isn't such an issue, DX12 just becomes a giant PITA. There are so many ways to shoot yourself in the foot and then squander a couple hours trying to figure out why. Maybe it gets better the more I use it, but so far I'm not holding out a lot of hope.