Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

We still don't use BFDOT on macOS CPU because Apple compiler used for PyTorch wheel is outdated #1444

Open
swolchok opened this issue Dec 27, 2024 · 2 comments
Open

We still don't use BFDOT on macOS CPU because Apple compiler used for PyTorch wheel is outdated #1444

swolchok opened this issue Dec 27, 2024 · 2 comments
Labels
Known Gaps These are known Gaps/Issues/Bug items in torchchat MPS/Metal Issues related to Metal MPS/env set up performance triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@swolchok
Copy link
Contributor

🚀 The feature, motivation and pitch

See pytorch/pytorch#143913 . This is blocking improved CPU performance for bfloat16 decoding on Mac; setting up an issue on torchchat side to track.

Alternatives

robust setup for decoding using accelerators on Mac

Additional context

No response

RFC (Optional)

No response
@Jack-Khuu Jack-Khuu added performance MPS/Metal Issues related to Metal MPS/env set up Known Gaps These are known Gaps/Issues/Bug items in torchchat triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module labels Jan 3, 2025
@Jack-Khuu
Copy link
Contributor

Pulling up context from the chain:

Per @huydhn

We don't plan to take any action for this right now and will wait for Apple to release a new clang/xcode version on MacOS

@huydhn
Copy link

huydhn commented Jan 20, 2025

There is a recent update on this pytorch/pytorch#143913 (comment), it looks like the beta GitHub MacOS 15 runner now has clang 18

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Known Gaps These are known Gaps/Issues/Bug items in torchchat MPS/Metal Issues related to Metal MPS/env set up performance triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@swolchok @huydhn @Jack-Khuu