Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
hotfix: revert torch.library register (#709)
We observe performance degradation for small operations in flashinfer v0.2 because of the overhead of `torch.library.custom_op` introduced in #554. This PR disables torch custom operator registrations first, we can add them back with lightweight registration later: https://github.com/vllm-project/vllm/blob/36e76700453924c8d421db99af70a88a1df835cd/vllm/utils.py#L1660-L1674 cc @zhyncs @abcdabcd987 @youkaichao
- Loading branch information