Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Divanovic/llama tg demo #13105

Merged
merged 3 commits into from
Sep 30, 2024
Merged

Divanovic/llama tg demo #13105

merged 3 commits into from
Sep 30, 2024

Conversation

djordje-tt
Copy link
Contributor

Ticket

#13102
#11730

Problem description

Enable prefill+decode demo

What's changed

  • Added support for prefill+decode demo
  • Removed model_config dependency to t3k model_config and created TG one
  • Separated all component configs from it's files to model_config and made components cleaner and more readable

Checklist

@djordje-tt djordje-tt added prefill LLM models have prefill mode and it's optimization is usually separated from decode mode. llama3 llm_tg labels Sep 30, 2024
@djordje-tt djordje-tt self-assigned this Sep 30, 2024
@djordje-tt djordje-tt force-pushed the divanovic/llama-tg-demo branch from a47d70f to 767ebba Compare September 30, 2024 10:46
@djordje-tt djordje-tt force-pushed the divanovic/llama-tg-demo branch from 767ebba to 4b95982 Compare September 30, 2024 16:35
@djordje-tt
Copy link
Contributor Author

CI frequent test run: https://github.com/tenstorrent/tt-metal/actions/runs/11034614348

@djordje-tt djordje-tt force-pushed the divanovic/llama-tg-demo branch 4 times, most recently from 0503b41 to b38ee94 Compare September 30, 2024 17:13
@djordje-tt djordje-tt force-pushed the divanovic/llama-tg-demo branch 2 times, most recently from dbd354b to b38ee94 Compare September 30, 2024 19:35
@djordje-tt djordje-tt force-pushed the divanovic/llama-tg-demo branch from 52c07af to 52a7562 Compare September 30, 2024 19:42
@djordje-tt djordje-tt merged commit b23d07e into main Sep 30, 2024
6 checks passed
@djordje-tt djordje-tt deleted the divanovic/llama-tg-demo branch September 30, 2024 19:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@johanna-rock-tt johanna-rock-tt johanna-rock-tt left review comments

@cglagovichTT cglagovichTT cglagovichTT approved these changes

@uaydonat uaydonat Awaiting requested review from uaydonat uaydonat is a code owner

@kpaigwar kpaigwar Awaiting requested review from kpaigwar kpaigwar is a code owner

Assignees

@djordje-tt djordje-tt

Labels
llama3 llm_tg prefill LLM models have prefill mode and it's optimization is usually separated from decode mode.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@djordje-tt @cglagovichTT @johanna-rock-tt