DL-Brief.md

DL in an hour, a personal perspective

Fundament

1. Fit, aprroximation, interpolation, estimation, predict

2. case in real life, spline
   • https://www.ibiblio.org/e-notes/Splines/nurbs.html
   • https://www.ibiblio.org/e-notes/Splines/bezier3d.html

2. Universal approximation theorem

   • UAT

   

   • spline appr

   

   • multi-diemensions

   

   • image, cases of low dimension embedded

   

3. https://github.com/NirViaje/DeepLearningRobotics/blob/master/NeuronTalk.md

   • Hebbian, naive ANN

     • where the neuron is?

     

     • matrix and non-linear

     

   • XOR, multi-layer

   

   • principle

     • info redundant, constrain (cnn, dimension reduction

   

   • info distill (auto-encoder/LeNet

   

   • localize, convolution, primary visual cortex of mammal, DCT

     

     • huge number of weights/paras

     

     • visual cortex, DCT

     

     • localize, convolution

   • LeNet, AlexNet, VGG, GoogLeNet inception, ResNet

   

   • timeline, go deep, Autoencoder, Hinton 2006

   

   0. LeNet, 1998

   

   1. AlexNet, 2012, ImageNet Challenge

   

   2. Vgg, 2014, Visual Geometry Group - University of Oxford

   

   3. GoogLeNet inception, 2014

   

   4. ResNet, 2015 Dec

   

   • Survey

   

   • build blocks, ReLu 1996/2010, with leakage, pooling, dropout etc

   

   • optimization, training, backpropagation, Andrew Ng's course

   

4. Gu@SUNY, manifold

   

5. what's more

   • detect, track, Yolo/openPose etc
   • LSTM
   • GAN
   • DRL

   

   • connect concrete and abstract
     • Deep Learning and Geometry, by Xianfeng Gu
     • the figure of three stage
   • further methodology
     • non-BP
     • unsupervised
     • etc

6. a little bit more on AGI

   • AIXI
   • 

7. fully DL rendered movie?

   • nVidia be a render company again

HandsOn

1. available workbench in lab
   • 1080Tix3
   • P40x2
2. Deepo
3. git

Several Cases

1. on Aesthetic, eyes movement
2. scratch2code
3. coloring
4. more