Welcome to my STMO project!
I wanted to explore the findings of Pezeshki et al. presented in their publication Gradient Starvation: A Learning Proclivity in Neural Networks. The goal was to explain the problem of gradient starvation and to investigate its effects and a proposed remedy on a simple 2D classification task.
The notebook and the code of my project are separate! You can find them in their respective subfolders notebook
and src
.
All experiments conducted, including the related figures, can be simply repeated and re-created by running include("src\\gradientstarvation.jl")
from a Julia REPL after checking out the project on your local machine.