neural-networks

single-layer neural network

Dataset: cifar-10

Optimization method: mini batch gradient descent

Loss function: cross entropy for softmax; SVM multi-class

Regularization: L2

New features: eg. learning rate decay, Xavier initialization

Test accuracy (highest): 40.66%

double-layer neural network

Dataset: cifar-10

Optimization method: mini batch gradient descent

Loss function: cross entropy

Regularization: L2

New features: eg. cyclical learning rate, ensemble learning, dropout

Test accuracy (highest): 54.84%

multi-layer neural network

Dataset: cifar-10

Optimization method: mini batch gradient descent

Loss function: cross entropy

Regularization: L2

New features: eg. batch normalization, He initialization, data augmentation

Test accuracy (highest): 58.66%

recurrent neural network

Dataset: Text from Harry Potter and the Goblet of Fire

Optimization method: AdaGrad

Loss function: cross entropy

Goal: synthesize text