Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RuntimeError: q must have shape (batch_size, seqlen_q, num_heads, head_size) #2896

Closed
MohamedAliRashad opened this issue Feb 18, 2025 · 1 comment
Closed

RuntimeError: q must have shape (batch_size, seqlen_q, num_heads, head_size) #2896

MohamedAliRashad opened this issue Feb 18, 2025 · 1 comment
Labels
🐛 bug Something isn't working 🏋 SFT Related to SFT

Comments

@MohamedAliRashad
Copy link

I am using SFTTrainer with preprocessed datasets (input_ids, attention_mask, labels) and for some reason:

  1. There is a progress bar for processing that i don't know what is doing exactly
  2. The training is giving me the error in the title of the issue.

I think there should be a flag somewhere to tell the trainer if the data is preprocessed and ready for direct training or automatic preperation is required
@github-actions github-actions bot added 🐛 bug Something isn't working 🏋 SFT Related to SFT labels Feb 18, 2025
@MohamedAliRashad
Copy link
Author

Sorry, this was mistake from my side

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
🐛 bug Something isn't working 🏋 SFT Related to SFT
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@MohamedAliRashad