Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Submission Process Rules #476

Merged
merged 19 commits into from
Nov 3, 2023
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Rename Jury Award
  • Loading branch information
fsschneider committed Oct 10, 2023
commit e7a907cf21770eabf8bc4520983b9c9e2c6c5995
8 changes: 4 additions & 4 deletions SUBMISSION_PROCESS_RULES.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# MLCommons™ AlgoPerf: Submission Process Rules

**Version:** 0.0.2 *(Last updated 03 Oktober 2023)*
**Version:** 0.0.3 *(Last updated 10 Oktober 2023)*

- [Basics](#basics)
- [Schedule](#schedule)
@@ -37,7 +37,7 @@ Three additional documents complement this document:

### Dates

- **Publication of the call for submission: 17. Oktober 2023 (08:00 AM UTC)**
- **Publication of the call for submission: 17. October 2023 (08:00 AM UTC)**
- Registration deadline for submissions: 15. December 2023 (08:00 AM UTC)
- Version freeze for the benchmark codebase: 17. January 2024 (08:00 AM UTC)
- **Submission deadline: 15. February 2024 (08:00 AM UTC)**
@@ -141,9 +141,9 @@ The spirit jury may then hear the justifications of the submitters, inspect the

## Awards and prize money

An awards committee will award a prize for the "*Best Performance*" in each ruleset as well as a "*Jury Award*". The prize for the best-performing submission will take into account the [benchmark score](RULES.md#benchmark-score-using-performance-profiles) on the full benchmark. The "*Jury Award*" will favor more out-of-the-box ideas that show great potential, even though the method may not be of practical value with the current landscape of models, software, etc.
An awards committee will award a prize for the "*Best Performance*" in each ruleset as well as a "*Innovative Submission Award*". The prize for the best-performing submission will take into account the [benchmark score](RULES.md#benchmark-score-using-performance-profiles) on the full benchmark. The "*Innovative Submission Award*" will favor more out-of-the-box ideas that show great potential, even though the method may not be of practical value with the current landscape of models, software, etc.

The prize money for "*Best Performance*" in a ruleset is $20,000 each. The winner of the "*Jury Award*" will be awarded $10,000. We reserve the right to split the prize money and distribute it among multiple submissions.
The prize money for "*Best Performance*" in a ruleset is $20,000 each. The winner of the "*Innovative Submission Award*" will be awarded $10,000. We reserve the right to split the prize money and distribute it among multiple submissions.

If a submission is ineligible to win prize money it can still win an award. The prize money will then go to the highest-ranking eligible submission.