Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FA3 does not work with torch.compile #1435

Open
nighting0le01 opened this issue Jan 10, 2025 · 0 comments
Open

FA3 does not work with torch.compile #1435

nighting0le01 opened this issue Jan 10, 2025 · 0 comments

Comments

@nighting0le01
Copy link

FA3 not working with torch.compile
[rank7]: torch._dynamo.exc.Unsupported: Graph break due to unsupported builtin flash_attn_3_cuda.PyCapsule.fwd. This function is either a Python builtin (e.g. _warnings.warn) or a third-party C/C++ Python extension (perhaps created with pybind). If it is a Python builtin, please file an issue on GitHub so the PyTorch team can add support for it and see the next case for a workaround. If it is a third-party C/C++ Python extension, please either wrap it into a PyTorch-understood custom operator (see https://pytorch.org/tutorials/advanced/custom_ops_landing_page.html for more details) or, if it is traceable, use torch.compiler.allow_in_graph.

@nighting0le01 nighting0le01 changed the title FA3 torch.compile FA3 does not work with torch.compile Jan 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant