Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bugfix][Kernel] FA3 Fix - RuntimeError: This flash attention build only supports pack_gqa (for build size reasons). #12405

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

LucasWilkinson
Copy link
Contributor

@LucasWilkinson LucasWilkinson commented Jan 24, 2025

Based off of #12375 that should be merged first

We thought we could get away with only packed-gqa (is the default for varlen gqa) for build time and size reasons, but fails for true MHA, this turns back on the non packed-gqa kernels.

Tested with facebook/opt-125m

Copy link

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

  • Add ready label to the PR
  • Enable auto-merge.

🚀

@mergify mergify bot added the ci/build label Jan 24, 2025
@LucasWilkinson LucasWilkinson changed the title [Kernel] FA3 Fix - RuntimeError: This flash attention build only supports pack_gqa (for build size reasons). [Bugfix][Kernel] FA3 Fix - RuntimeError: This flash attention build only supports pack_gqa (for build size reasons). Jan 24, 2025
@tlrmchlsmth tlrmchlsmth mentioned this pull request Jan 24, 2025
8 tasks
Copy link

mergify bot commented Jan 24, 2025

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @LucasWilkinson.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify bot added the needs-rebase label Jan 24, 2025
Signed-off-by: Lucas Wilkinson <[email protected]>
@LucasWilkinson LucasWilkinson force-pushed the lwilkinson/re-enable-non-packed-gqa branch from 658544d to f992fef Compare January 24, 2025 15:47
@mergify mergify bot removed the needs-rebase label Jan 24, 2025
@tlrmchlsmth tlrmchlsmth added the ready ONLY add when PR is ready to merge/full CI is needed label Jan 24, 2025
@tlrmchlsmth tlrmchlsmth enabled auto-merge (squash) January 24, 2025 15:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci/build ready ONLY add when PR is ready to merge/full CI is needed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants