Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[meta] Migrate model registry from Github #1017

Open
1 task
eu9ene opened this issue Jan 27, 2025 · 0 comments
Open
1 task

[meta] Migrate model registry from Github #1017

eu9ene opened this issue Jan 27, 2025 · 0 comments
Labels
meta A collection of sub-issues that uses a tasklist

Comments

@eu9ene
Copy link
Collaborator

eu9ene commented Jan 27, 2025

The goal is to consolidate the code in this repo and reuse the infrastructure we're already using to store and evaluate models: Taskcluster, GCP, Weights and Biases. Then we'll be able to retire the current Github project https://github.com/mozilla/firefox-translations-models

This would require:

  • Migrate storage from GitHub LFS to Weights and Biases + GCP
  • Migrate evaluation code to this repo (reuse evals from the training pipeline)
  • Migrate evaluation reports to W&B
  • Implement a Taskcluster action for model deployment (currently done manually)

There is also an option to use Lumigator for the final evaluation TBD

The model artifacts will be integrated with W&B experiments.

Deployment automation

Preview Give feedback
  1. weights and biases
@eu9ene eu9ene added the meta A collection of sub-issues that uses a tasklist label Jan 27, 2025
@eu9ene eu9ene changed the title Migrate model registry from Github [meta] Migrate model registry from Github Jan 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
meta A collection of sub-issues that uses a tasklist
Projects
None yet
Development

No branches or pull requests

1 participant