Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable speculative decoding #2777

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
Open

Enable speculative decoding #2777

wants to merge 10 commits into from

Conversation

mzegla
Copy link
Collaborator

@mzegla mzegla commented Oct 30, 2024 •
edited
Loading

🛠 Summary

JIRA/Issue if applicable.
Describe the changes.

🧪 Checklist

  • Unit tests added.
  • The documentation updated.
  • Change follows security best practices.
    ``

@mzegla mzegla force-pushed the speculative_decoding branch 2 times, most recently from 03b8654 to 46acb7d Compare December 12, 2024 09:15
mzegla added 4 commits January 17, 2025 10:25
@mzegla
init
bad618f 
dynamic decoding + separate scheduler
@mzegla
revert patch removal
79839b0
@mzegla
style
9a1f68a
@mzegla
fix
6912a7d
@mzegla mzegla force-pushed the speculative_decoding branch from f07f188 to 6912a7d Compare January 17, 2025 09:34
@mzegla mzegla marked this pull request as ready for review January 22, 2025 13:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dtrawins dtrawins Awaiting requested review from dtrawins

@dkalinowski dkalinowski Awaiting requested review from dkalinowski

@ngrozae ngrozae Awaiting requested review from ngrozae

At least 2 approving reviews are required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@mzegla