Slow performance for simple image brightness adjustment

[justasxz]

Hi, I am trying to make a very simple blazor maui hybrid app, that would take a picture load it, and adjust the brightness and see it change in real time.

With hd (720p) pictures it works without any issues, but after I bump up the resolution to 4k it becomes a slideshow and I can't even begin to test 8k (8k is the limit I need).

    public async Task<byte[]> AdjustBrightnessWithMagickNet(byte[] inputImageBytes, float brightness)
    {
        try
        {
            OpenCL.IsEnabled = true; // Enable OpenCL for GPU acceleration

            using (var inputStream = new MemoryStream(inputImageBytes))
            {
                // Load the image from the input stream
                using (var image = new MagickImage(inputStream))
                {
                    
                    // Adjust brightness
                    image.BrightnessContrast(new Percentage((brightness - 1) * 100), new Percentage(0));
                    using (var outputStream = new MemoryStream())
                    {
                        // Save the adjusted image to the output stream
                        await image.WriteAsync(outputStream, MagickFormat.Jpeg);

                        return outputStream.ToArray();
                    }
                }
            }
        }
        catch (Exception e)
        {
            Console.WriteLine(e);
            throw;
        }
    }

This is the method, it takes around 300ms to process, which is quick, but not quick enough for real time adjustment.

I noticed that it uses my IGPU instead of dGPU (the IGPU is integrated with ryzen 7700x), and the dGPU is rx 6800 XT

But even when using iGPU it uses around 5-10% of it (I found this out, by opening task manager and dragging the slider to adjust brightness really fast). Also without GPU it takes around 600ms. So it's an improvement but not a huge one.

The code to show the image is:

@if (selectedImage != null)
{
    <img src="@selectedImage" alt="Selected Image" style="width:1024px; height:768px;" />
}

this line takes more than 100ms
using (var image = new MagickImage(inputStream))
so does this
await image.WriteAsync(outputStream, MagickFormat.Jpeg);

I refactored the code, so MagickImage is loaded only once, so I managed to improve the performance quite a bit, but I am not sure, how to improve on WriteAsync

so I assume, this is not reliant on gpu, but maybe there is something that can be done ?

I suppose my main question is how to improve performance of the action.
I suppose it might even be different issue, so I would like someone to advice.

[dlemstra]

You are encoding the image and that probably also takes some time. What are you doing with the byte array that you are returning? And ImageMagick focuses more on quality and less on performance.

[justasxz]

I take that byte array convert to base64 and put it into <img src I know it does not focus on performance, but it feels like the best out of the box tool that has everything we would need. To adjust image in real time with gpu support.

Skia refuses to work on blazor hybrid, ImageSharp has no gpu support and is paid. This was my last option and it looks good, just lacking those last few parts

[dlemstra]

This project also has a build in image.ToBase64(MagickFormat.Jpeg) method. That might be faster?

[justasxz]

It might have shaved a few milliseconds ? Well at least it made the code look nicer. But still a bit slow. I have this code currently:

@page "/ImageCanvas"
@using System.Globalization
@using ImageMagick

Select a Picture

@if (selectedImage != null)
{

}

<input type="range" min="0" max="2" step="0.05" @oninput="OnSliderChange" />

Slider Value: @sliderValue

```

@code {
private string selectedImage;
private float sliderValue = 1f;
private byte[] currentImage;
MagickImage? magickImage;

private async Task HandleFileSelected(InputFileChangeEventArgs e)
{
    try
    {
        var file = e.File;
        //using var stream = file.OpenReadStream();
        using var stream = file.OpenReadStream(maxAllowedSize: 3000 * 2000);

        using var memoryStream = new MemoryStream();
        await stream.CopyToAsync(memoryStream);

        // Convert to base64 string
        memoryStream.Position = 0;
        currentImage = memoryStream.ToArray();
        magickImage = new MagickImage(currentImage);
        selectedImage = $"data:{file.ContentType};base64,{magickImage.ToBase64(MagickFormat.Jpeg)}";

    }
    catch (Exception ex)
    {
        Console.WriteLine(e);
        throw;
    }
}
private async Task OnSliderChange(ChangeEventArgs e)
{
    sliderValue = float.Parse(e.Value.ToString(), CultureInfo.InvariantCulture);
    await HandleAdjustment();
}

private async Task HandleAdjustment()
{
    var adjustedBuffer = await AdjustBrightnessWithMagickNet(currentImage, sliderValue);
    selectedImage = $"data:image/jpeg;base64,{adjustedBuffer}";
}


public async Task<string> AdjustBrightnessWithMagickNet(byte[] inputImageBytes, float brightness)
{
    try
    {
        if (magickImage != null)
        {
            OpenCL.IsEnabled = true; // Enable OpenCL for GPU acceleration
            var adjustedImage = magickImage.Clone();
            adjustedImage.BrightnessContrast(new Percentage((brightness - 1) * 100), new Percentage(0));

            return adjustedImage.ToBase64(MagickFormat.Jpeg);

        }
        else
        {
            return "";
        }


    }
    catch (Exception e)
    {
        Console.WriteLine(e);
        throw;
    }
}

}

return adjustedImage.ToBase64(MagickFormat.Jpeg); - This line takes 100 ms :( It seems slightly faster, but not enough to be smooth. I wonder is there anything else that could be done. (the Image I am using is 4k or 1.6mb, in size).

It might be out of scope for this question, but if you have any other suggestions I would be grateful. 

[justasxz]

Also going back to my previous question, how to enforce gpu usage ? I tried the app on few different computers, and on some it takes much longer, because it does not seem like it uses gpu. Or does it not like using dGPU ?

[dlemstra]

ImageMagick will run a benchmark to check if the CPU is faster than the GPU. But with the help of the OpenCL class you can go through the devices and disable all devices except the ones where DeviceType is OpenCLDeviceType.Gpu.

[justasxz]

ImageMagick will run a benchmark to check if the CPU is faster than the GPU. But with the help of the OpenCL class you can go through the devices and disable all devices except the ones where DeviceType is OpenCLDeviceType.Gpu.

I tried to do so, but I observed a weird occurrence. My iGPU (Which is very weak) was chosen over my dGPU. And I tried disabling it and enabling my dGPU only, but that seemed not to affect which gpu it uses.

And in another computers case OpenCL.Devices, only contains the gpu, but it's disabled, and enabling it does not move the processing to gpu, at least the BrightnessContrast, option does not improve at all and it does not seem that GPU usage rises at all.

[dlemstra]

There could be other reasons why the GPU cannot be used. It might be possible that the GPU refuses to execute the compiled kernel. Our OpenCL implementation is a best effort implementation so it might not work for all cards. And we also don't have the hardware to do a lot of testing.