1.31.6

1.31.6 (2025-01-30)

Fix

• fix: Filling missing metadata for leaderboard release (#1895)

• Update ArxivClusteringS2S.py

• fill some metadat for retrieval

• fill in the reste of missing metadata

• fix metadata

• fix climatefever metadata

• fix: Added CQADupstack annotations

• removed annotation for non-exisitant task

• format

• Added financial to other financial dataset

• Moved ArguAna annotation to derivate datasets

---

Co-authored-by: Kenneth Enevoldsen <[email protected]> (938e90f)

Unknown

• Update tasks table (12ad5bd)

• Update tasks table (9076213)

• Update tasks table (4bb4ec6)

• Update tasks table (d510ddb)

• Update tasks table (e35c8dd)

• Update tasks table (9a6275e)

• Update tasks table (d9ab239)

• Update tasks table (21b60f5)

• Update tasks table (c46cb8b)

• Update tasks table (0bbc4c7)

• Update tasks table (3123d1c)

• Update tasks table (f7438b8)

• Update tasks table (51faf65)

• Update tasks table (1b76261)

• Update tasks table (67f8a79)

• Update tasks table (933f4af)

• Update tasks table (599849b)

• Update tasks table (ff4ae8d)

• Update tasks table (780a7d3)

• Update tasks table (c34ef64)

• Update tasks table (b23597c)

• Update tasks table (1030888)

• Update tasks table (913112a)

• Update tasks table (25a6f17)

• Update tasks table (e07ffe8)

• Update tasks table (b78525d)

• Update tasks table (6989fd5)

• Update tasks table (b7e412d)

• Update tasks table (2e817b0)

• Update tasks table (28ad172)

• Update tasks table (2850a97)

• Update tasks table (77681bf)

• Adding a banner to the new MMTEB leaderboard (#1908)

• Adding a banner to the new MMTEB leaderboard

• linting

• Update mteb/leaderboard/app.py

Co-authored-by: Isaac Chung <[email protected]>

• adding reference to mteb arena

---

Co-authored-by: Isaac Chung <[email protected]> (d0bb5b9)

• Update tasks table (f258cfc)

• Update tasks table (6cc0560)

• Update tasks table (7996458)

• Docs: update docs according to current state (#1870)

• update docs

• Apply suggestions from code review

Co-authored-by: Isaac Chung <[email protected]>

• update readme

• Update README.md

Co-authored-by: Isaac Chung <[email protected]>

---

Co-authored-by: Isaac Chung <[email protected]> (7e5d6c8)

• Update tasks table (0a59704)

• Feat: Add FaMTEB (Farsi/Persian Text Embedding Benchmark) (#1843)

• Add Summary Retrieval Task

• Add FaMTEBClassification

• Add FaMTEBClustering

• Add FaMTEBPairClassification

• Add FaMTEBRetrieval and BEIRFA and FaMTEBSTS

• Add FaMTEBSummaryRetrieval

• Add FaMTEB to benchmarks

• fix benchmark names

• temporary fix metadata

• Fix dataset revisions

• Update SummaryRetrievalEvaluator.py

• Update task files

• Update task files

• add data domain and subtask description

• Update AbsTaskSummaryRetrieval and FaMTEBSummaryRetrieval

• Update AbsTaskSummaryRetrieval

• Add mock task

• Update AbsTaskSummaryRetrieval

• Update AbsTaskSummaryRetrieval

• make lint

• Refactor SummaryRetrieval to subclass BitextMining

• Add aggregated datasets

---

Co-authored-by: mehran <[email protected]>
Co-authored-by: e.zeinivand <[email protected]>
Co-authored-by: Erfun76 <[email protected]> (f3404b4)

• Update tasks table (35b2c09)

• Update tasks table (d44f9c3)

• Update tasks table (0a57880)

• Update tasks table (e04218c)