Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a sampler that samples only keyframes (I-Frames) #474

Open
ahmadsharif1 opened this issue Jan 23, 2025 · 1 comment
Open

Add a sampler that samples only keyframes (I-Frames) #474

ahmadsharif1 opened this issue Jan 23, 2025 · 1 comment
Labels
enhancement New feature or request

Comments

@ahmadsharif1
Copy link
Contributor

ahmadsharif1 commented Jan 23, 2025

Starting clips on keyframes can potentially be a fast and useful sampler because it can sometimes be faster to seek and decode to a keyframe instead of decoding multiple P frames to get to the sample point.

Ideas:

  1. There can be an API option to to return clips whose first frame is always a keyframe
  2. There can be an API option to have clips start on keyframes if they are "close enough" (in pts) to the sample point that the sampler wanted to choose. This can be used to balance performance and scene diversity
  3. This should work in approximate mode (i.e. it shouldn't require a full scan of the file to do this -- we could read the list of keyframes from the header)
  4. There could potentially be an API call that returns the list of pts to the user, if they want to do something with it (they could look at it and decide whether it's worth doing keyframe only sampling or uniform sampling)
@scotts scotts added the enhancement New feature or request label Jan 23, 2025
@scotts
Copy link
Contributor

scotts commented Jan 23, 2025

@NicolasHug and I have also talked about a similar thing. We'll want to do some research to better understand the use-cases people want to cover. The biggest question to me is, should "key_frame_only" be a kind of seek mode, and then we use the samplers as-is? Or should the sampler API itself be aware of the concept of key frames? If the latter, what do we need to expose in the decoder to enable it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants