Video models (take 2)

[dreadatour]

Alternative approach to implement video models based on this comment. Looks much cleaner IMO.

New VideoFile model

class VideoFile(File):
    """`DataModel` for reading video files."""

    def get_info(self) -> "Video":
        """Returns video file information."""

    def get_frame(self, frame: int) -> "VideoFrame":
        """
        Returns VideoFrame model for a video frame.

        Args:
            frame (int): Frame number to read.

        Returns:
            VideoFrame: Video frame model.
        """

    def get_frames(
        self,
        start: int = 0,
        end: Optional[int] = None,
        step: int = 1,
    ) -> "Iterator[VideoFrame]":
        """
        Returns VideoFrame models for a video frame.

        Args:
            start (int): Frame number to start reading from (default: 0).
            end (Optional[int]): Frame number to stop reading at, non-inclusive
                                 (default: None, read until the end).
            step (int): Step size for reading frames (default: 1).

        Returns:
            Iterator[VideoFrame]: List of video frame models.

        Note:
            If end is not specified, number of frames will be taken from the video file.
        """

    def get_fragment(self, start: float, end: float) -> "VideoFragment":
        """
        Returns VideoFragment model for a video interval.

        Args:
            start (float): Start time in seconds.
            end (float): End time in seconds.

        Returns:
            VideoFragment: Video fragment model.
        """

    def get_fragments(
        self,
        intervals: list[tuple[float, float]],
    ) -> "Iterator[VideoFragment]":
        """
        Returns VideoFragment models for video intervals.

        Args:
            intervals (list[tuple[float, float]]): List of start and end times
                                                   in seconds.

        Returns:
            Iterator[VideoFragment]: List of video fragment models.
        """

New VideoFrame model

One can create VideoFrame without downloading video file, since it is "virtual" frame: original VideoFile + frame number.

If physical frame image is needed, call save method, which uploads frame image into storage and returns ImageFile new model.

API:

class VideoFrame(VideoFile):
    """`DataModel` for reading video frames."""

    frame: int = Field(default=-1)

    def get_np(self) -> "ndarray":
        """
        Reads video frame from a video file and returns as numpy array.

        Returns:
            ndarray: Video frame.
        """

    def read_bytes(self, format: str = "jpg") -> bytes:
        """
        Reads video frame from a video file and returns as image bytes.

        Args:
            format (str): Image format (default: 'jpg').

        Returns:
            bytes: Video frame image as bytes.
        """

    def save(self, output: str, format: str = "jpg") -> "ImageFile":
        """
        Saves video frame as a new image file. If output is a remote path,
        the image file will be uploaded to the remote storage.

        Args:
            output (str): Output path, can be a local path or a remote path.
            format (str): Image format (default: 'jpg').

        Returns:
            ImageFile: Image file model.
        """

New VideoFragment model

One can create VideoFragment without downloading video file, since it is "virtual" fragment: original video file + start/end timestamp.

If physical fragment video is needed, call save method, which uploads fragment video into storage and returns new VideoFile model.

API:

class VideoFragment(VideoFile):
    """`DataModel` for reading video fragments."""

    start: float = Field(default=-1.0)
    end: float = Field(default=-1.0)

    def save(self, output: str, format: Optional[str] = None) -> "VideoFile":
        """
        Saves video interval as a new video file. If output is a remote path,
        the video file will be uploaded to the remote storage.

        Args:
            output (str): Output path, can be a local path or a remote path.
            format (Optional[str]): Output format (default: None). If not provided,
                                    the format will be inferred from the video fragment
                                    file extension.

        Returns:
            VideoFile: Video fragment model.
        """

New Video model

Video file meta information.

class Video(DataModel):
    """`DataModel` for video file meta information."""

    width: int = Field(default=-1)
    height: int = Field(default=-1)
    fps: float = Field(default=-1.0)
    duration: float = Field(default=-1.0)
    frames: int = Field(default=-1)
    format: str = Field(default="")
    codec: str = Field(default="")

[codecov]

Codecov Report

Attention: Patch coverage is 86.02941% with 19 lines in your changes missing coverage. Please review.

Project coverage is 87.75%. Comparing base (7f757b3) to head (0f2e12c).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
src/datachain/lib/video.py	75.00%	10 Missing and 7 partials ⚠️
src/datachain/lib/file.py	97.05%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #890      +/-   ##
==========================================
+ Coverage   87.74%   87.75%   +0.01%     
==========================================
  Files         129      130       +1     
  Lines       11462    11595     +133     
  Branches     1545     1563      +18     
==========================================
+ Hits        10057    10175     +118     
- Misses       1017     1025       +8     
- Partials      388      395       +7     

Flag	Coverage Δ
datachain	87.67% <86.02%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[mattseddon]

Minor, but should these be to_ methods to match the DataChain class?

[dreadatour]

Minor, but should these be to_ methods to match the DataChain class?

Looks reasonable 🤔 Although it is not a direct conversion ("to"), but rather getting a part of the file into another file, like "get frame from video" looks good to me, but "video to frame" looks odd. What do you think? I don't have strict opinion on this 🤔