Is there a way to speedup Inference? (strange that Python code is twice faster than C#)

[dbriard]

Hi all,

I ported an image segmentation code from python to C# but the performance are not equal, python is faster by a factor of 2 (about 10 sec instead of 20 sec).

I can't figure out why??? May be I am wrong, but to me, both python and C# call the same C++ function under the hood?

I am wondering is there is some kind "configuration settings" in torchsharp to tweak the inference speed?

Both codes are using the same torchscript model.
Both are using CPU.
On C# I installed Windows x64 cpu binaries.
I am only measuring the inference (Call() in C#)

Python

with torch.no_grad():
            pred = self.model(x)

C#

using (var d = torch.NewDisposeScope())
{
    using (var noGrad = torch.no_grad())
    {
        // ... prepare input ...
        var sw = Stopwatch.StartNew();
        var result = _module.call(image_tensor);
        sw.Stop();
        Debug.WriteLine($"Call: {sw.ElapsedMilliseconds}ms");
        // ... process output ...
    }
    d.DisposeEverything();
}

Do you know what could explain a such huge speed difference?
Thanks for helping.

[dbriard]

Argh, I think I found the guilty! I have an animated progress bar in C# during the processing... I didn't know that it can slow the app so much !!!