Using a Video File as Input Video Source

[mariodruiz]

In the Riallto material we make extensive use of the webcam to demonstrate the pre-build functions and we also suggest you use the webcam for some of the user exercises. We recognize that not all Ryzen AI enabled devices have an integrated webcam and also not all users may have access to a (suitable) webcam.

In this blog, we will discuss how to use a video file as source for your video pipelines.

The VideoApplication class

All the pre-build applications derive from the VideoApplication. Let's have a look at the class documentation.

class VideoApplication:
    """Wrapper class that allows to feed and visualize video stream from the NPU

    You must pass an xclbin and optionally the pixel type for the input and
    output images.

    Parameters
    ----------
    filename : str
        Path to the xclbin file
    videosource : [int, str]
        videosource webcam index or path to video file
    pxtype_in : pxtype
        Pixel type for the input image, either pxtype.RGBA or pxtype.GRAY
    pxtype_out : pxtype
        Pixel type for the output image, either pxtype.RGBA or pxtype.GRAY

    Returns
    -------
    VideoApplication
        Object that abstracts away the video handling from a webcam or
        video file to visualization
    """

    def __init__(self, filename, videosource: Union[int, str] = 0,
                 pxtype_in: pxtype = pxtype.RGBA,
                 pxtype_out: pxtype = pxtype.RGBA):
        ...

From the documentation, we can see that the variable videosource supports both int and string arguments, its default value is 0 (webcam with index 0). Alternatively, you can specify a path to a video file. This means that support for video files is already included in Riallto.

Downloading a Video File

You can find a number of video files on the Internet, we are going to use Pixabay, a vibrant community of creatives, sharing copyright free images, videos and music.

From your Ryzen AI enabled device, download a suggested video by visiting this link: https://pixabay.com/videos/mountains-nature-landscape-clouds-48873/, then follow the steps below:

1. Click Free Download button

2. Select 1280x720 (free) or 1920x1080 (you need to register)

3. Click Download

This will start the download of the video on your computer. Once the video is downloaded, drag it to the File Browser on Juptyer Lab and rename it to mountains.mp4

You can search for other videos on Pixabay and follow the same procedure to download them.

Running a pre-built Application using the video file

With everything in place, let's run one of the pre-built applications.

In a new notebook, type the following code and then run:

from npu.lib import ColorDetectVideoProcessing

app = ColorDetectVideoProcessing('mountains.mp4')
app.start()

Note, for the pre-built applications filename (xclbin file) is provided by default.

You can also use videos with 1080p resolution, we have pre-built xclbin for this resolution too.

Multiple apps at the same time

Using a video file can enable you to run multiple applications with the same input video source. Although, you may notice a reduction in FPS. We suggest using different copies of the video file for each concurrent application.

Other Video Resolutions

Riallto includes pre-built xclbins for efficient execution of video resolution of 720p and 1080p. If the video source, namely webcam or file, has different resolution we also have you covered. The video will be automatically resized to the closest resolution and then passed on to be processed by the NPU.