Skip to content
This repository has been archived by the owner on Nov 1, 2024. It is now read-only.

train opt-125M from scratch #725

Open
emrecanacikgoz opened this issue May 30, 2023 · 2 comments
Open

train opt-125M from scratch #725

emrecanacikgoz opened this issue May 30, 2023 · 2 comments
Labels
question Further information is requested

Comments

@emrecanacikgoz
Copy link

I couldn't find a detailed documentation (or a step-by-step guideline) about pre-training opt125 with exactly the same corpus and model architecture that you used in paper. In short, I would like to reproduce your smallest model results, from scratch.

Could you point out where can find the regarding guideline or provide anything else which can help? Special thanks,
@emrecanacikgoz emrecanacikgoz added the question Further information is requested label May 30, 2023
@Gusicun
Copy link

Gusicun commented Jul 8, 2023

same questions

1 similar comment
@CaptainSlowWZY
Copy link

same questions

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@CaptainSlowWZY @Gusicun @emrecanacikgoz