Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the vulkan video decoder to the compositor #803

Merged
merged 13 commits into from
Oct 11, 2024

Conversation

jerzywilczek
Copy link
Collaborator

closes #751.

The decoder structure:

  1. data comes into the parser, which turns it into decoder instructions
    • first it aggregates the NAL units into Access Units (basically whole frames)
    • then the hardest part happens: reference management. for each frame that needs to be decoded, a list of necessary reference pictures is generated based on quite complicated conditions
  2. the instructions are fed into the decoder, which executes them, basically decoding the frames
    • the decoder does two things. the first one is managing the state of the structures necessary for decoding. This mostly happens on processing SPS packets.
    • the other thing is just translating the instructions into vulkan structs, which can be fed to the GPU

@jerzywilczek jerzywilczek force-pushed the @jerzywilczek/vk-video branch from f558574 to d8b7720 Compare October 2, 2024 08:43
@jerzywilczek jerzywilczek force-pushed the @jerzywilczek/vk-video branch from d8b7720 to cb8e496 Compare October 2, 2024 08:47
compositor_api/src/types/register_input.rs Outdated Show resolved Hide resolved
Cargo.toml Show resolved Hide resolved

let limits = set_required_wgpu_limits(limits);

match vk_video::VulkanCtx::new(vulkan_features, limits.clone()) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add an example of initiating graphics context directly in a way where users have access to e.g. adapter

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just realized that I need a mac os computer to write the mac os part of the example. I'll have access to a mac next week. Do you think we can marge the example as-is?

compositor_pipeline/src/pipeline.rs Outdated Show resolved Hide resolved
compositor_pipeline/src/pipeline.rs Show resolved Hide resolved
vk-video/src/vulkan_decoder/vulkan_ctx.rs Show resolved Hide resolved
Copy link
Member

@noituri noituri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The parts I understood looked good to me ✅

compositor_api/src/types/from_register_input.rs Outdated Show resolved Hide resolved
compositor_pipeline/src/pipeline.rs Outdated Show resolved Hide resolved
vk-video/src/vulkan_decoder.rs Show resolved Hide resolved
vk-video/src/parser/au_splitter.rs Show resolved Hide resolved
Cargo.toml Show resolved Hide resolved
vk-video/src/vulkan_decoder.rs Outdated Show resolved Hide resolved
Cargo.toml Outdated Show resolved Hide resolved
vk-video/build.rs Show resolved Hide resolved
compositor_api/src/types/from_register_input.rs Outdated Show resolved Hide resolved
vk-video/src/lib.rs Outdated Show resolved Hide resolved
@jerzywilczek jerzywilczek merged commit ea29422 into master Oct 11, 2024
3 checks passed
@jerzywilczek jerzywilczek deleted the @jerzywilczek/vk-video branch October 11, 2024 09:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add the vulkan video-based decoder to the compositor
4 participants