Updated docs

.github/workflows/deploy_pages.yml

@@ -40,10 +40,9 @@ jobs:
       - name: Install dependencies
         run: |
           pip install -r docs/requirements.txt -r requirements.txt -r requirements-optional.txt
-          pip install quadprog==0.1.11
       - name: Sphinx build
         run: |
-          sphinx-build -j auto docs _build
+          python ./utils/args.py && sphinx-build -j auto docs _build
       - name: Setup Pages
         uses: actions/configure-pages@v5
       - name: Upload artifact
@@ -53,4 +52,4 @@ jobs:
           path: '_build'
       - name: Deploy to GitHub Pages
         id: deployment
-        uses: actions/deploy-pages@v4
+        uses: actions/deploy-pages@v4

README.md

@@ -2,19 +2,27 @@
   <img width="230" height="230" src="logo.png" alt="logo">
 </p>
 
+<p align="center">
+  <img alt="GitHub commit activity" src="https://img.shields.io/github/commit-activity/m/aimagelab/mammoth">
+  <a href="https://aimagelab.github.io/mammoth/index.html"><img alt="Static Badge" src="https://img.shields.io/badge/wiki-gray?style=flat&logo=readthedocs&link=https%3A%2F%2Faimagelab.github.io%2Fmammoth%2Findex.html"></a>
+  <img alt="Discord" src="https://img.shields.io/discord/1164956257392799860">
+</p>
+
 # Mammoth - An Extendible (General) Continual Learning Framework for Pytorch
 
-Official repository of [Class-Incremental Continual Learning into the eXtended DER-verse](https://arxiv.org/abs/2201.00766), [Dark Experience for General Continual Learning: a Strong, Simple Baseline](https://papers.nips.cc/paper/2020/hash/b704ea2c39778f07c617f6b7ce480e9e-Abstract.html), and [Semantic Residual Prompts for Continual Learning](https://arxiv.org/abs/2403.06870)
+Official repository of:
+- [Class-Incremental Continual Learning into the eXtended DER-verse](https://arxiv.org/abs/2201.00766)
+- [Dark Experience for General Continual Learning: a Strong, Simple Baseline](https://papers.nips.cc/paper/2020/hash/b704ea2c39778f07c617f6b7ce480e9e-Abstract.html)
+- [Semantic Residual Prompts for Continual Learning](https://arxiv.org/abs/2403.06870)
+- [CLIP with Generative Latent Replay: a Strong Baseline for Incremental Learning](https://arxiv.org/abs/2407.15793)
 
-Mammoth is a framework for continual learning research. With **40 methods and 21 datasets**, it includes the most complete list competitors and benchmarks for research purposes.
+Mammoth is a framework for continual learning research. With **more than 40 methods and 20 datasets**, it includes the most complete list competitors and benchmarks for research purposes.
 
 The core idea of Mammoth is that it is designed to be modular, easy to extend, and - most importantly - _easy to debug_.
 Ideally, all the code necessary to run the experiments is included _in the repository_, without needing to check out other repositories or install additional packages.
 
 With Mammoth, nothing is set in stone. You can easily add new models, datasets, training strategies, or functionalities.
 
-Join our Discord Server for all your Mammoth-related questions → ![Discord Shield](https://discordapp.com/api/guilds/1164956257392799860/widget.png?style=shield)
-
 ## Documentation
 
 ### Check out the official [DOCUMENTATION](https://aimagelab.github.io/mammoth/) for more information on how to use Mammoth!
@@ -37,7 +45,7 @@ Join our Discord Server for all your Mammoth-related questions → ![Discord Shi
 
 ## Models
 
-Mammoth currently supports **42** models, with new releases covering the main competitors in literature.
+Mammoth currently supports **more than 40** models, with new releases covering the main competitors in literature.
 
 - Efficient Lifelong Learning with A-GEM (A-GEM, A-GEM-R - A-GEM with reservoir buffer): `agem`, `agem_r`
 - Bias Correction (BiC): `bic`.
@@ -74,10 +82,10 @@ Mammoth currently supports **42** models, with new releases covering the main co
 ## Datasets
 
 **NOTE**: Datasets are automatically downloaded in `data/`.
-- This can be changes by changing the `base_path` function in `utils/conf.py` or using the `--base_path` argument.
-- The `data/` folder should not tracked by git and is craeted automatically if missing.
+- This can be changed by changing the `base_path` function in `utils/conf.py` or using the `--base_path` argument.
+- The `data/` folder should not be tracked by git and is craeted automatically if missing.
 
-Mammoth includes **21** datasets, covering *toy classification problems* (different versions of MNIST), *standard domains* (CIFAR, Imagenet-R, TinyImagenet, MIT-67), *fine-grained classification domains* (Cars-196, CUB-200), *aerial domains* (EuroSAT-RGB, Resisc45), *medical domains* (CropDisease, ISIC, ChestX).
+Mammoth currently includes **21** datasets, covering *toy classification problems* (different versions of MNIST), *standard domains* (CIFAR, Imagenet-R, TinyImagenet, MIT-67), *fine-grained classification domains* (Cars-196, CUB-200), *aerial domains* (EuroSAT-RGB, Resisc45), *medical domains* (CropDisease, ISIC, ChestX).
 
 - Sequential MNIST (_Class-Il / Task-IL_): `seq-mnist`.
 - Permuted MNIST (_Domain-IL_): `perm-mnist`.
@@ -135,69 +143,56 @@ Mammoth includes **21** datasets, covering *toy classification problems* (differ
 
 ### Our Papers
 
+Expand to see the BibTex!
+
 <ul>
-<li><details><summary>Dark Experience for General Continual Learning: a Strong, Simple Baseline (<b>NeurIPS 2020</b>) <a href=https://arxiv.org/abs/2004.07211>paper</a></summary>
+<li><details><summary>CLIP with Generative Latent Replay: a Strong Baseline for Incremental Learning (<b>BMVC 2024</b>) <a href=https://arxiv.org/abs/2407.15793>paper</a></summary>
 
-<pre><code>@inproceedings{buzzega2020dark,
- author = {Buzzega, Pietro and Boschini, Matteo and Porrello, Angelo and Abati, Davide and Calderara, Simone},
- booktitle = {Advances in Neural Information Processing Systems},
- editor = {H. Larochelle and M. Ranzato and R. Hadsell and M. F. Balcan and H. Lin},
- pages = {15920--15930},
- publisher = {Curran Associates, Inc.},
- title = {Dark Experience for General Continual Learning: a Strong, Simple Baseline},
- volume = {33},
- year = {2020}
+<pre><code>@inproceedings{heng2022enhancing,
+  title={CLIP with Generative Latent Replay: a Strong Baseline for Incremental Learning},
+  author={Frascaroli, Emanuele and Panariello, Aniello and Buzzega, Pietro and Bonicelli, Lorenzo and Porrello, Angelo and Calderara, Simone},
+  booktitle={35th British Machine Vision Conference},
+  year={2024}
 }</code></pre>
 
-</details>
 </li>
-<li><details><summary>Rethinking Experience Replay: a Bag of Tricks for Continual Learning (<b>ICPR 2020</b>) <a href=https://arxiv.org/abs/2010.05595>paper</a> <a href=https://github.com/hastings24/rethinking_er>code</a></summary>
-
-<pre><code>@inproceedings{buzzega2021rethinking,
-  title={Rethinking experience replay: a bag of tricks for continual learning},
-  author={Buzzega, Pietro and Boschini, Matteo and Porrello, Angelo and Calderara, Simone},
-  booktitle={25th International Conference on Pattern Recognition},
-  pages={2180--2187},
-  year={2021},
-  organization={IEEE}
-}</code></pre>
 
-</li>
-<li><details><summary>Class-Incremental Continual Learning into the eXtended DER-verse (<b>TPAMI 2022</b>) <a href=https://arxiv.org/abs/2201.00766>paper</a></summary>
+<li><details><summary>Semantic Residual Prompts for Continual Learning (<b>ECCV 2024</b>) <a href=https://arxiv.org/abs/2403.06870>paper</a></summary>
 
-<pre><code>@article{boschini2022class,
-  title={Class-Incremental Continual Learning into the eXtended DER-verse},
-  author={Boschini, Matteo and Bonicelli, Lorenzo and Buzzega, Pietro and Porrello, Angelo and Calderara, Simone},
-  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
-  year={2022},
-  publisher={IEEE}
+<pre><code>@inproceedings{menabue2024semantic,
+  title={Semantic Residual Prompts for Continual Learning},
+  author={Menabue, Martin and Frascaroli, Emanuele and Boschini, Matteo and Sangineto, Enver and Bonicelli, Lorenzo and Porrello, Angelo and Calderara, Simone},
+  booktitle={18th European Conference on Computer Vision},
+  year={202},
+  organization={Springer}
 }</code></pre>
 
 </li>
-<li><details><summary>Effects of Auxiliary Knowledge on Continual Learning (<b>ICPR 2022</b>) <a href=https://arxiv.org/abs/2206.02577>paper</a></summary>
 
-<pre><code>@inproceedings{bellitto2022effects,
-  title={Effects of auxiliary knowledge on continual learning},
-  author={Bellitto, Giovanni and Pennisi, Matteo and Palazzo, Simone and Bonicelli, Lorenzo and Boschini, Matteo and Calderara, Simone},
-  booktitle={26th International Conference on Pattern Recognition},
-  pages={1357--1363},
-  year={2022},
-  organization={IEEE}
+<li><details><summary>Mask and Compress: Efficient Skeleton-based Action Recognition in Continual Learning (<b>ICPR 2024</b>) <a href=https://arxiv.org/pdf/2407.01397>paper</a> <a href=https://github.com/Sperimental3/CHARON>code</a></summary>
+
+<pre><code>@inproceedings{mosconi2024mask,
+  title={Mask and Compress: Efficient Skeleton-based Action Recognition in Continual Learning},
+  author={Mosconi, Matteo and Sorokin, Andriy and Panariello, Aniello and Porrello, Angelo and Bonato, Jacopo and Cotogni, Marco and Sabetta, Luigi and Calderara, Simone and Cucchiara, Rita},
+  booktitle={International Conference on Pattern Recognition},
+  year={2024}
 }</code></pre>
 
 </li>
-<li><details><summary>Transfer without Forgetting (<b>ECCV 2022</b>) <a href=https://arxiv.org/abs/2206.00388>paper</a> <a href=https://github.com/mbosc/twf>code</a> (Also available here)</summary>
 
-<pre><code>@inproceedings{boschini2022transfer,
-  title={Transfer without forgetting},
-  author={Boschini, Matteo and Bonicelli, Lorenzo and Porrello, Angelo and Bellitto, Giovanni and Pennisi, Matteo and Palazzo, Simone and Spampinato, Concetto and Calderara, Simone},
-  booktitle={17th European Conference on Computer Vision},
-  pages={692--709},
-  year={2022},
-  organization={Springer}
+<li><details><summary>On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning (<b>NeurIPS 2022</b>) <a href=https://arxiv.org/abs/2210.06443>paper</a> <a href=https://github.com/aimagelab/lider>code</a> (Also available here)</summary>
+
+<pre><code>@article{bonicelli2022effectiveness,
+  title={On the effectiveness of lipschitz-driven rehearsal in continual learning},
+  author={Bonicelli, Lorenzo and Boschini, Matteo and Porrello, Angelo and Spampinato, Concetto and Calderara, Simone},
+  journal={Advances in Neural Information Processing Systems},
+  volume={35},
+  pages={31886--31901},
+  year={2022}
 }</code></pre>
 
 </li>
+
 <li><details><summary>Continual semi-supervised learning through contrastive interpolation consistency (<b>PRL 2022</b>) <a href=https://arxiv.org/abs/2108.06552>paper</a> <a href=https://github.com/aimagelab/CSSL>code</a> (Also available here)</summary>
 
 <pre><code>@article{boschini2022continual,
@@ -211,50 +206,73 @@ Mammoth includes **21** datasets, covering *toy classification problems* (differ
 }</code></pre>
 
 </li>
-<li><details><summary>On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning (<b>NeurIPS 2022</b>) <a href=https://arxiv.org/abs/2210.06443>paper</a> <a href=https://github.com/aimagelab/lider>code</a> (Also available here)</summary>
 
-<pre><code>@article{bonicelli2022effectiveness,
-  title={On the effectiveness of lipschitz-driven rehearsal in continual learning},
-  author={Bonicelli, Lorenzo and Boschini, Matteo and Porrello, Angelo and Spampinato, Concetto and Calderara, Simone},
-  journal={Advances in Neural Information Processing Systems},
-  volume={35},
-  pages={31886--31901},
-  year={2022}
+<li><details><summary>Transfer without Forgetting (<b>ECCV 2022</b>) <a href=https://arxiv.org/abs/2206.00388>paper</a> <a href=https://github.com/mbosc/twf>code</a> (Also available here)</summary>
+
+<pre><code>@inproceedings{boschini2022transfer,
+  title={Transfer without forgetting},
+  author={Boschini, Matteo and Bonicelli, Lorenzo and Porrello, Angelo and Bellitto, Giovanni and Pennisi, Matteo and Palazzo, Simone and Spampinato, Concetto and Calderara, Simone},
+  booktitle={17th European Conference on Computer Vision},
+  pages={692--709},
+  year={2022},
+  organization={Springer}
 }</code></pre>
 
 </li>
-<li><details><summary>Mask and Compress: Efficient Skeleton-based Action Recognition in Continual Learning (<b>ICPR 2024</b>) <a href=https://arxiv.org/pdf/2407.01397>paper</a> <a href=https://github.com/Sperimental3/CHARON>code</a></summary>
 
-<pre><code>@inproceedings{mosconi2024mask,
-  title={Mask and Compress: Efficient Skeleton-based Action Recognition in Continual Learning},
-  author={Mosconi, Matteo and Sorokin, Andriy and Panariello, Aniello and Porrello, Angelo and Bonato, Jacopo and Cotogni, Marco and Sabetta, Luigi and Calderara, Simone and Cucchiara, Rita},
-  booktitle={International Conference on Pattern Recognition},
-  year={2024}
+<li><details><summary>Effects of Auxiliary Knowledge on Continual Learning (<b>ICPR 2022</b>) <a href=https://arxiv.org/abs/2206.02577>paper</a></summary>
+
+<pre><code>@inproceedings{bellitto2022effects,
+  title={Effects of auxiliary knowledge on continual learning},
+  author={Bellitto, Giovanni and Pennisi, Matteo and Palazzo, Simone and Bonicelli, Lorenzo and Boschini, Matteo and Calderara, Simone},
+  booktitle={26th International Conference on Pattern Recognition},
+  pages={1357--1363},
+  year={2022},
+  organization={IEEE}
 }</code></pre>
 
 </li>
-<li><details><summary>Semantic Residual Prompts for Continual Learning (<b>ECCV 2024</b>) <a href=https://arxiv.org/abs/2403.06870>paper</a></summary>
 
-<pre><code>@inproceedings{menabue2024semantic,
-  title={Semantic Residual Prompts for Continual Learning},
-  author={Menabue, Martin and Frascaroli, Emanuele and Boschini, Matteo and Sangineto, Enver and Bonicelli, Lorenzo and Porrello, Angelo and Calderara, Simone},
-  booktitle={18th European Conference on Computer Vision},
-  year={202},
-  organization={Springer}
+<li><details><summary>Class-Incremental Continual Learning into the eXtended DER-verse (<b>TPAMI 2022</b>) <a href=https://arxiv.org/abs/2201.00766>paper</a></summary>
+
+<pre><code>@article{boschini2022class,
+  title={Class-Incremental Continual Learning into the eXtended DER-verse},
+  author={Boschini, Matteo and Bonicelli, Lorenzo and Buzzega, Pietro and Porrello, Angelo and Calderara, Simone},
+  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
+  year={2022},
+  publisher={IEEE}
 }</code></pre>
 
 </li>
-<li><details><summary>CLIP with Generative Latent Replay: a Strong Baseline for Incremental Learning (<b>BMVC 2024</b>) <a href=https://arxiv.org/abs/2407.15793>paper</a></summary>
 
-<pre><code>@inproceedings{heng2022enhancing,
-  title={CLIP with Generative Latent Replay: a Strong Baseline for Incremental Learning},
-  author={Frascaroli, Emanuele and Panariello, Aniello and Buzzega, Pietro and Bonicelli, Lorenzo and Porrello, Angelo and Calderara, Simone},
-  booktitle={35th British Machine Vision Conference},
-  year={2024}
+<li><details><summary>Rethinking Experience Replay: a Bag of Tricks for Continual Learning (<b>ICPR 2020</b>) <a href=https://arxiv.org/abs/2010.05595>paper</a> <a href=https://github.com/hastings24/rethinking_er>code</a></summary>
+
+<pre><code>@inproceedings{buzzega2021rethinking,
+  title={Rethinking experience replay: a bag of tricks for continual learning},
+  author={Buzzega, Pietro and Boschini, Matteo and Porrello, Angelo and Calderara, Simone},
+  booktitle={25th International Conference on Pattern Recognition},
+  pages={2180--2187},
+  year={2021},
+  organization={IEEE}
 }</code></pre>
 
 </li>
 
+<li><details><summary>Dark Experience for General Continual Learning: a Strong, Simple Baseline (<b>NeurIPS 2020</b>) <a href=https://arxiv.org/abs/2004.07211>paper</a></summary>
+
+<pre><code>@inproceedings{buzzega2020dark,
+ author = {Buzzega, Pietro and Boschini, Matteo and Porrello, Angelo and Abati, Davide and Calderara, Simone},
+ booktitle = {Advances in Neural Information Processing Systems},
+ editor = {H. Larochelle and M. Ranzato and R. Hadsell and M. F. Balcan and H. Lin},
+ pages = {15920--15930},
+ publisher = {Curran Associates, Inc.},
+ title = {Dark Experience for General Continual Learning: a Strong, Simple Baseline},
+ volume = {33},
+ year = {2020}
+}</code></pre>
+
+</details>
+</li>
 </ul>
 
 ### Other Awesome CL works using Mammoth

docs/getting_started/checkpoints.rst

@@ -1,14 +1,15 @@
 Load and save checkpoints
 =========================
 
-Loading and saving checkpoints is handeled automatically in :ref:`module-training` by supplying the ``--savecheck`` and ``--loadcheck`` arguments. 
+Loading and saving checkpoints is handled automatically in :ref:`module-training` by supplying the ``--savecheck`` and ``--loadcheck`` arguments. 
 
 For example, to save a checkpoint after the end of the last task, simply run the following command:
+
 .. code-block:: python 
 
         python utils/main.py --savecheck=last --model=sgd --dataset=seq-cifar10 --lr=0.1
 
-Other options for ``--savecheck`` are:
+The available options for ``--savecheck`` are:
 
 - ``last``: save the checkpoint after **the last task**.
 - ``task``: save the checkpoint after **each task**.

docs/getting_started/index.rst

@@ -22,7 +22,7 @@ WandB
 For advanced logging, including loss values, metrics, and hyperparameters, you can use `WandB <https://wandb.ai/>`_ by providing both ``--wandb_project`` and ``--wandb_entity`` arguments. If you don't want to use WandB, you can simply omit these arguments.
 
 .. tip::
-    By default, all arguments, loss values, and metrics are logged. Thanks to the **autolog_wandb** (:ref:`module-models`), all the variables created in the **observe** that contain *loss* or start with *_wandb_* will be logged. Thus, in order to loss all the separate loss values, you can simply add ``loss = loss + loss1 + loss2`` to the **observe** function.
+    By default, all arguments, loss values, and metrics are logged. Thanks to the **autolog_wandb** (:ref:`module-models`), all the variables created in the **observe** that contain *loss* or start with *_wandb_* will be logged. Thus, in order to log all the separate loss values, you can simply add ``loss = loss + loss1 + loss2`` to the **observe** function.
 
 Metrics are logged on WandB both in a raw form, separated for each task and class. This allows further analysis (e.g., with the Mammoth :ref:`Parseval <module-parseval>`). To differentiate between raw metrics logged on WandB and other aggregated metrics that may have been logged, all the raw metrics are prefixed with **RESULTS_**. This behavior can be changed by changing the prefix in the **log_accs** function in :ref:`module-loggers`.
 

docs/getting_started/scripts.rst

@@ -8,7 +8,7 @@ Mammoth includes a couple of scripts (under the ``scripts`` folder) to help you
 - ``scripts/local_launcher.py``: this script will launch all the experiments in the ``data/jobs/<experiment_name>.txt`` file in paralel on your local machine. Logs for each experiment will be stored in the ``logs`` folder. It accepts the following arguments:
     - ``--file``: path to the file containing the experiments to run (default: ``data/jobs/<experiment_name>.txt``)
 
-    - ``--redundancy``: number of times each experiment should be repeated (default: 1)
+    - ``--cycles``: number of times each experiment should be repeated (default: 1)
 
     - ``--at_a_time``: number of experiments to run in parallel (default: 1)