Module 4 - Neural Networks.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Module 4 : Neural Networks with TensorFlow\n",
    "\n",
    "이번 module에서는 TF를 이용해서 NN을 만들어보겠습니다. 앞서 module 3에서 estimator.DNNClassifier를 이용해 NN으로 classification 문제를 해결하긴 했는데, 이번에는 조금 더 자세한 부분까지 직접 조절할 수 있는 neural network입니다. \n",
    "\n",
    "이번 module은 다음 문서를 기반으로 만들어졌습니다.\n",
    "\n",
    "https://www.tensorflow.org/tutorials/keras/classification"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "from tensorflow import keras\n",
    "\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Intro to NN\n",
    "\n",
    "본 영상에서는 Keras 및 NN에 대해 꽤 많이 설명을 하고 있습니다. 각 neuron들이 어떻게 연결되어 있는지부터 layers, weight/bias, activation function, backpropagation, loss function, gradient descent, oprimizer까지 간단하게 짚어보고 구현으로 들어갑니다. 해당 내용을 다 싣기에는 현재 시간이 부족해서... 스킵하도록 하겠습니다. 추후 여견이 된다면 추가하도록 하겠습니다. [원작자의 colab notebook 링크](https://colab.research.google.com/drive/1m2cg3D1x3j5vrFc-Cu0gMvc48gWyCOuG#forceEdit=true&sandboxMode=true&scrollTo=-HJV4JF789aC)를 걸어둡니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dataset\n",
    "\n",
    "이번 module에서는 Fashion-MNIST dataset을 사용합니다. 기존 MNIST가 흑백으로 된 숫자(0~9) 손글씨 데이터셋이라면, Fashion-MNIST는 10종류의 옷 이미지로 구성된 데이터셋입니다. Neural network를 이용해서 사진을 입력받아 이 옷이 어떤 종류인지 분류하는 문제를 해결하고자 합니다.\n",
    "\n",
    "Training set에 포함된 이미지는 총 60000개이고, 각 이미지는 28x28 픽셀입니다. Test/validation set에는 10000개의 이미지가 있습니다.\n",
    "\n",
    "텐서플로에서 이 데이터셋을 제공합니다. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset = keras.datasets.fashion_mnist\n",
    "\n",
    "(img_train, label_train), (img_test, label_test) = dataset.load_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Training set에 총 60000장의 이미지가 28x28 사이즈로 존재함을 알 수 있습니다. 각 픽셀값은 0~255 사이의 값이고, RGB 대신 흑백 이미지임을 확인할 수 있습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(60000, 28, 28)\n",
      "[  0   0   0   0   0   0   0   0   0   0   0   0   6   0 102 204 176 134\n",
      " 144 123  23   0   0   0   0  12  10   0]\n"
     ]
    }
   ],
   "source": [
    "print(img_train.shape)\n",
    "print(img_train[0, 5])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Label은 0~9 사이의 숫자로 이루어져 있습니다. 각 숫자가 한 class(옷 종류)를 의미한다고 보면 됩니다. 각 숫자가 어떤 옷을 의미하는지 보기 위해 class 이름을 정의합시다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[9 0 0 3 0]\n"
     ]
    }
   ],
   "source": [
    "print(label_train[:5])\n",
    "\n",
    "class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "마지막으로 실제 이미지가 어떻게 생겼는지 확인해보도록 합시다. 각 픽셀이 가지는 값에 따라 어떤 색을 가지는지 colorbar로 확인할 수 있습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAATEAAAD4CAYAAACE9dGgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAdAklEQVR4nO3dfZBc5XXn8e+Z0YxeRqM3JIQQsgVY2Mj2IlgZZEg52CTmpVwRxMYFlcVyQkXsLmxMij8gbLbMVootymtgnThmIwxrUQVmiYGgEJV5kbExdngRQkYSWiwBMhISegVJSBpppvvsH30n9Gjmnntnume6r/h9qK7puaefvo96Zg73Pvfc5zF3R0SkqFoa3QERkVooiYlIoSmJiUihKYmJSKEpiYlIoY0ayZ2122gfQ8dI7lLkI6WLAxzxw1bLe1z4xQ7fvaeU67Uvv3r4CXe/qJb91aqmJGZmFwHfA1qBH7r7bdHrx9DBOXZBLbsUkcALvqLm99i9p8SLT3ws12tbZ2yYWvMOazTk00kzawX+HrgYmAtcaWZz69UxEWkMB8o5/8tiZrPM7BkzW29m68zsW8n2W8zsHTNbnTwuqWrzV2a20cxeN7MLs/ZRy5HY2cBGd38z2fGDwELgtRreU0QazHG6Pd/pZA49wA3uvsrMOoGXzeypJHanu3+3+sXJgdAVwKeBE4Gnzew09/QO1TKwPxPYXPX9lmRbH2a22MxWmtnKbg7XsDsRGSn1OhJz923uvip5vh9YzwB5ospC4EF3P+zubwEbqRwwpaoliQ00eNjvHiZ3X+Lu8919fhuja9idiIwExyl5vgcwtfcgJXksTntfM5sNnAm8kGy6zsxeNbN7zWxysi3XwVG1WpLYFmBW1fcnAVtreD8RaRJlPNcD2NV7kJI8lgz0fmY2HngYuN7d9wF3AacC84BtwO29Lx2geXiDdy1J7CVgjpmdbGbtVM5jl9XwfiLSBBwo4bkeeZhZG5UEdr+7PwLg7tvdveTuZeBuPjxlHPTB0ZCTmLv3ANcBT1A5z33I3dcN9f1EpHkM4kgsZGYG3AOsd/c7qrbPqHrZZcDa5Pky4AozG21mJwNzgBejfdRUJ+buy4HltbyHiDQXB7rrN0XXecBVwBozW51su5lKSda8ZHebgGsA3H2dmT1EpcqhB7g2ujIJI1yxLyLNzwdxqpj5Xu7PMfA4V+rBj7vfCtyadx9KYiLSl0OpQHOlKomJSB+Viv3iUBITkaMYpQHPAJuTkpiI9FEZ2FcSE5GCqtSJKYmJSIGVdSQmIkWlIzERKTTHKBVo5nolMRHpR6eTIlJYjnHEWxvdjdyUxESkj0qxq04nRaTANLAvzcMyfhlrnK2g9bgpYfy9C09LjU144Pma9p31b7NRbakx7z5S275rlfVzidRvhomUtzdKriMxESmwso7ERKSoKgP7xUkNxempiIwIDeyLSOGVVCcmIkWlin0RKbyyrk6KSFFVbgBXEpMmYa3x7SPe0xPGW+bNDePrrxkftz+UHms7EK5Oz6hD8STJbU+uDOM11YJl1aBlfK5YnARq6ZuNCv5s4x9nLo7RrduORKSo3FGxq4gUmanYVUSKy9GRmIgUnAb2RaSwHNOkiCJSXJUl24qTGorTUxEZIVo8V5pIWFNEdp3Y5gsnhfE/+fwvw/ivdp6SGvvd6BPCtj42DDPqDz4fxk/7wTupsZ5Nb8dvnjFnV9bnlqV18uT0YKkUti3t25cerMNUY85HqGLfzDYB+4ES0OPu8+vRKRFprI/akdgX3X1XHd5HRJqAu310jsRE5NhTGdj/6Nx25MCTZubAP7j7kqNfYGaLgcUAYxhX4+5EZPgVa479Wnt6nrufBVwMXGtmXzj6Be6+xN3nu/v8NkbXuDsRGW6VgX3L9chiZrPM7BkzW29m68zsW8n2KWb2lJltSL5OTrabmf2tmW00s1fN7KysfdSUxNx9a/J1B/AoEE9LICKFUKIl1yOHHuAGdz8dWEDlYGcucBOwwt3nACuS76FyQDQneSwG7srawZCTmJl1mFln73Pgy8Daob6fiDSH3or9ehyJufs2d1+VPN8PrAdmAguBpcnLlgKXJs8XAvd5xfPAJDObEe2jljGx6cCjVpl3aRTwgLv/tIb3k2FQ7uqqqf2RMz8I41+bGM/pNaalOzX2i5Z4vrB3fjYrjJf+Xdy3393RmRorv3Ju2Pa4tXGt1oRXtoXxXV+YGcZ3/vv0gq7pGctxTn76jdSY7anPtbpBLBQy1cyqfwmWDDQ2DmBms4EzgReA6e6+DSqJzsyOT142E9hc1WxLsi31Ax/yv9jd3wTOGGp7EWlO7tBdzp3EduWpDzWz8cDDwPXuvs/SJ50cKBCW8KrEQkT6qJxO1u/qpJm1UUlg97v7I8nm7WY2IzkKmwHsSLZvAaoPwU8CtkbvX5zrqCIyYkrJ/ZNZjyxWOeS6B1jv7ndUhZYBi5Lni4DHqrZ/I7lKuQDY23vamUZHYiLSR2+JRZ2cB1wFrDGz1cm2m4HbgIfM7GrgbeDyJLYcuATYCBwE/jRrB0piInKU+p1OuvtzDDzOBXDBAK934NrB7ENJTET60Rz7MrKi5cUyppT54OsLwvg35v48jL/RPS2Mn9S+JzV2+Ykvh235D3H8+6//fhg/8ObE1FhLR/y5vLsgPhJ5Z2H87/bueKqeyavS//RaFm0P2+47kj69UWlF7XfFVK5OfnTunRSRY4ympxaRwtPppIgUVp2vTg47JTER6UeTIopIYbkbPUpiIlJkOp0UkcLSmJgMXlTnNcwW3PhiGP/i+Ndqev+ZwQQEB7w9bPt+qSOMf3vuv4TxnaelT8WTtTjsDzfEU/V8ENSgAbT2xD/TBX/2Smrsq1NeCtt+5+HPpsZa/EDYNi8lMREpLNWJiUjhqU5MRArLHXryT4rYcEpiItKPTidFpLA0JiYihedKYiJSZBrYl8HJmPNrOG344PgwvnvC+DD+bs+kMH5ca/qyap0th8K2s9t2hfGdpfQ6MIDWtvQl4Y54PF/Wf//0P4fxrtPbwnibxUu+nTsmfe2Ly1/7Rti2gzfDeK3cNSYmIoVmlHR1UkSKTGNiIlJYundSRIrNGzpMO2hKYiLSj65OikhhuQb2RaTodDophTFtdHodF8AY6w7j7Ravr7i1e3JqbMOhT4Ztf7svrmG7aPq6MN4d1IK1BvOcQXad14lt74XxLo/ryKJP9bzpcR3Y6jBaH0W6Opl5zGhm95rZDjNbW7Vtipk9ZWYbkq/pv6kiUijulSSW59EM8pz4/gi46KhtNwEr3H0OsCL5XkSOEWW3XI9mkJnE3P1Z4Oi16BcCS5PnS4FL69wvEWkg93yPZjDUMbHp7r4NwN23mVnq4IWZLQYWA4xh3BB3JyIjxTHKBbo6Oew9dfcl7j7f3ee3MXq4dycideA5H81gqElsu5nNAEi+7qhfl0SkoY7Bgf2BLAMWJc8XAY/Vpzsi0hQKdCiWOSZmZj8GzgemmtkW4NvAbcBDZnY18DZw+XB28piXse6ktcZzX3lPeq1W6+S4+uX3J60J4ztLE8L4+6V4nHNS68HU2P6eMWHbPYfi9/7U6G1hfNXB2amxae1xnVfUb4BNR6aG8Tmj3w3j39l+QWps1pijr6P11XPBF1Jj/sK/hm3zapajrDwyk5i7X5kSSv8piEhhOVAu1yeJmdm9wFeAHe7+mWTbLcCfAzuTl93s7suT2F8BVwMl4C/c/YmsfRTnEoSIjAwH3PI9sv2I/nWmAHe6+7zk0ZvA5gJXAJ9O2vzAzOLTEJTERGQA9aoTS6kzTbMQeNDdD7v7W8BG4OysRkpiItJf/oH9qWa2suqxOOcerjOzV5PbGnsHbmcCm6tesyXZFtIN4CJylEGVT+xy9/mD3MFdwN9QSYN/A9wO/BkMOIlZ5vGejsREpL9hLLFw9+3uXnL3MnA3H54ybgFmVb30JCB9WaiEjsSaQcbggo2Kf0xRicXmq08P235pXLw02a+74qP5aaP2h/FoOpwZo/eGbTund4XxrPKOKaPSpxnaXxobth3XcjiMZ/27z2qPl5v7y6fPSo11fmZ32HZCW3DsUY+Lig5ep6uTAzGzGb23LQKXAb0z5CwDHjCzO4ATgTnAi1nvpyQmIgOoW4nFQHWm55vZPCrHcpuAawDcfZ2ZPQS8BvQA17p7PLEbSmIiMpA6VeOn1JneE7z+VuDWwexDSUxE+muSW4ryUBITkb56i10LQklMRPpplgkP81ASE5H+hvHqZL0piYlIP6YjMRkMa2sP4+WuuF4qMnXNkTC+qxQvLTapJZ6Spj1jabMjQZ3YuVPeCtvuzKjlWnXo5DDe2XooNTatJa7zmtUW12qt6ZoVxpcf+EQYv/orT6fGfrzkD8O27T/9dWrMPP555dJEc4XloSQmIkfJPUNFU1ASE5H+dCQmIoVWbnQH8lMSE5G+VCcmIkWnq5MiUmwFSmKaT0xECq1YR2LB0mY2Kq53staMfN0Sx8tdwfxS5czZQkLeHddy1eJ7//D9ML65Z1IYf7c7jmctbVYKpnR5/tDEsO2Ylu4wPm3UvjC+rxzXmUX2l+Pl5KJ50iC77zcetyE19sjePwjbjgSdTopIcTm67UhECk5HYiJSZDqdFJFiUxITkUJTEhORojLX6aSIFJ2uTg5NLesrZtVaeVy201CHFp4dxjdfGteh/cmZ6UvzvdvTGbZ95eDsMD4xmJMLoCNjfcYuT6/f23pkcmoMsmutonUlAY4P6shKHtcFvtMd9y1LVv3clp5gTcw/iuc6m3TfkLo0KEU6Esus2Deze81sh5mtrdp2i5m9Y2ark8clw9tNERlRw7gCeL3lue3oR8BFA2y/093nJY/l9e2WiDSMfzgulvVoBplJzN2fBfaMQF9EpFkcY0diaa4zs1eT083UAQQzW2xmK81sZTfx+ImINAcr53s0g6EmsbuAU4F5wDbg9rQXuvsSd5/v7vPbGD3E3YmIDGxISczdt7t7yd3LwN1AfHlNRIrlWD+dNLMZVd9eBqxNe62IFEzBBvYz68TM7MfA+cBUM9sCfBs438zmUcnFm4Br6tGZqA6sVqNmnBDGu0+eHsb3nD4uNXbwhLgwcN4l68P4N6f/nzC+szQhjLdZ+ue2ufu4sO2Z4zaF8Z/tnRvGd40aH8ajOrNzO9Ln1AJ4v5z+mQOcOOq9MH7jxq+lxqaPi2uxfvjx+IJ7t8cDQq93x0Mne8vp85H9xdxnwraPMi2M10WTJKg8MpOYu185wOZ7hqEvItIsjqUkJiIfLUbzXHnMQ0lMRPpqovGuPLRQiIj0V6erkym3LU4xs6fMbEPydXKy3czsb81sY1KDelaeriqJiUh/9Sux+BH9b1u8CVjh7nOAFcn3ABcDc5LHYir1qJmUxESkn3qVWKTctrgQWJo8XwpcWrX9Pq94Hph0VDnXgJpqTOzwxZ8L48f/1zdTY/MmbAnbzh37XBjvKsdLvkXTwrx2aGbY9mC5PYxvOBKXf+ztiUsNWoNR2B1H4ql4bn8rXh5sxdn/O4z/9daB5gb4UMvY9N/03aW4POOr4+Ml2SD+mV3zsWdTY6e07wjbPn4g/tvZmjFVz/S2vWF8dtvO1Ngfd/42bHsMlFhMd/dtAO6+zcyOT7bPBDZXvW5Lsm1b9GZNlcREpAn4oK5OTjWzlVXfL3H3JUPc80AFl5npVElMRPrLfyS2y93nD/Ldt5vZjOQobAbQe1i8BZhV9bqTgK1Zb6YxMRHpZ5hvO1oGLEqeLwIeq9r+jeQq5QJgb+9pZ0RHYiLSX53GxFJuW7wNeMjMrgbeBi5PXr4cuATYCBwE/jTPPpTERKSvOs5QkXLbIsAFA7zWgWsHuw8lMRHpwyhWxb6SmIj0oySWxuJl2c75Hy+FzS/oXJcaO+jx1CdZdWBZdT+RiaPi5bkOd8cf847ueKqdLKeNfjc1dtmE1WHbZ79/Thj/va7/Esbf+FI8jdCKQ+lTzuzsif/dV7z1pTC+6u1ZYXzB7LdSY5/tfCdsm1Wb19naFcaj6ZEADpTTf1+f74rr50aEkpiIFJqSmIgUVsFmsVASE5H+lMREpMg0KaKIFJpOJ0WkuJpoObY8lMREpD8lsYF1H9/B1qvS19m9ZeLfhe0f2LMgNTZrzNHzrvX18fZdYfyMsb8L45HOlrhm6JMT4pqhxw+cFMZ//v6nwviMtvdTY788eGrY9sFb/mcY/+Zf3hDGP7/8P4bxfbPT5xjo6Yj/UiacsTuM//WZ/xLG262UGnu/FNeBTRl9IIxPao1rA7NEdY2dLenL3AG0fvITqTHbFM+bl4cq9kWk8KxcnCymJCYifWlMTESKTqeTIlJsSmIiUmQ6EhORYlMSE5HCGtxqRw03okmspRvGbU//dB7fNy9sf8rY9LX6dnXH6ys+8cFnw/hJY98L4xNb02t3PhHM5wWwumtSGP/pzk+H8RPHxusvbu+emBrb3d0Rtj0YzGsFcM+dd4Tx27fH61ZeNmVVauyM9rgO7P1yvI7Naxnrde4vj0mNdXk8v9zejDqyzuD3AaDb4z+tVk//O5jUEteg7fvscamx0vba/6SLVieWudqRmc0ys2fMbL2ZrTOzbyXbp5jZU2a2Ifk69FkFRaS5uOd7NIE8S7b1ADe4++nAAuBaM5sL3ASscPc5wIrkexE5Bgzzkm11lZnE3H2bu69Knu8H1lNZWnwhsDR52VLg0uHqpIiMIB/EowkM6gTazGYDZwIvANN7F7ZMVvI9PqXNYmAxQHuHzjhFiqBIA/u5VwA3s/HAw8D17h6PNFdx9yXuPt/d548aHQ8yi0hzsHK+RzPIlcTMrI1KArvf3R9JNm83sxlJfAawY3i6KCIjyinUwH7m6aSZGXAPsN7dq6+3LwMWUVmSfBHwWNZ7tR4p07n5cGq87Ba2/9mu9Clppo/ZH7ad17k5jL9+ML5cv+bQiamxVaM+FrYd29odxie2x1P5dIxK/8wApral/9tPHh3/vyWargbgpa743/afpv08jL/dkz6E8M8HTgvbvnYw/TMHmJyxVN6afentD/a0h20Pl+I/ja6euGRn4uj4Z/q5KelTP73OjLDtzjOC6Y1+FTbNrVkG7fPIMyZ2HnAVsMbMehcxvJlK8nrIzK4G3gYuH54uisiIO5aSmLs/R6X+bSAX1Lc7ItJoRSt21W1HItKXuyZFFJGCK04OUxITkf50OikixeWATidFpNCKk8NGOIl9cIiWX7ySGv7HJ88Lm/+3hf+YGvtFxrJmj78b1/XsOxJPSTNtXPoSXhOCOi2AKW3x8l8TM+qdxli85Nt7Pel3QhxuiaecKaVeeK5493D6ND8AvyrPCePd5dbU2OEgBtn1dXuOTA3jJ47dmxrb35M+TQ/Apv1TwviuvePDeNe4+E/ruVL6UnoXnbAubDt2R/rPrCX+VclNp5MiUmj1vDppZpuA/UAJ6HH3+WY2Bfi/wGxgE/B1d48n9UuR+95JEfmIGJ5ZLL7o7vPcfX7yfd2m8lISE5E+KsWunutRg7pN5aUkJiL9lXM+YKqZrax6LB7g3Rx40sxeror3mcoLGHAqrzw0JiYi/QziKGtX1SlimvPcfWsy5+BTZvb/autdXzoSE5G+6jwm5u5bk687gEeBs6njVF5KYiJylMq9k3keWcysw8w6e58DXwbW8uFUXpBzKq80TXU6ecqN/xrGf/Dq19Lb/ufXw7YXn7A2jK/aF8+b9XZQN/SbYK4xgLaWeArMcW1HwviYjHqp9tb0OcFaMv53Wc6oE+tojfuWNdfZlNHpNXKdrfGcWy01Th3aGvzbX9w7O2w7fVxc+/eJCbvCeI/Hxwefn/hGauzet84N207/u1+nxjZ5XJOYW/0mPJwOPFqZlpBRwAPu/lMze4k6TeXVVElMRJpAHRfPdfc3gTMG2L6bOk3lpSQmIv01ydTTeSiJiUh/xclhSmIi0p+Vm2QpoxyUxESkL6e3kLUQlMREpA+j5luKRpSSmIj0pyQWaAnmkCrHayBOvP/51Nju++Pd/uSrF4bxc25+KYx/ZfZvUmOfat8etm3LODYfk3E9u6MlruXqCn7hsqqZnzs0K4yXMt7hZ++dHsbf7x6bGtt+cELYti2of8sjWsf0UE88z9reQ/F8Y60t8R9518/juc7eei19/ruJy+PfxRGhJCYihaUxMREpOl2dFJECc51OikiBOUpiIlJwxTmbVBITkf5UJyYixXYsJTEzmwXcB5xA5SBzibt/z8xuAf4c2Jm89GZ3X565x4xasOHS8fALYXztw3H7tZycGrPP/VHY9tAJ6bVSAKN3x3Ny7f943H7CG+lzSLUcjhciLP9mfRjP9kENbfeF0XgWtdq0Z8Sn1byH39b8Dg3jDqXinE/mORLrAW5w91XJDI0vm9lTSexOd//u8HVPRBriWDoSS1Yi6V2VZL+ZrQdmDnfHRKSBCpTEBjXHvpnNBs4Ees/NrjOzV83sXjObnNJmce9yTt3Ep00i0gQcKHu+RxPIncTMbDzwMHC9u+8D7gJOBeZROVK7faB27r7E3ee7+/w2RtehyyIyvBy8nO/RBHJdnTSzNioJ7H53fwTA3bdXxe8GHh+WHorIyHIKNbCfeSRmlWVK7gHWu/sdVdtnVL3sMirLMInIscA936MJ5DkSOw+4ClhjZquTbTcDV5rZPCp5exNwzbD0sAD8pTVhPJ7UJduE9BW6MhXn/6fSVJokQeWR5+rkczDg4oTZNWEiUkDNc5SVhyr2RaQvBzQVj4gUmo7ERKS4jr3bjkTko8TBm6QGLA8lMRHpr0mq8fNQEhOR/jQmJiKF5a6rkyJScDoSE5HicrzUmMlLh0JJTET66p2KpyCUxESkvwKVWAxqUkQROfY54GXP9cjDzC4ys9fNbKOZ3VTv/iqJiUhfXr9JEc2sFfh74GJgLpXZb+bWs7s6nRSRfuo4sH82sNHd3wQwsweBhcBr9drBiCax/by362n/ye+qNk0Fdo1kHwahWfvWrP0C9W2o6tm3j9f6Bvt574mn/SdTc758jJmtrPp+ibsvqfp+JrC56vstwDm19rHaiCYxd++znJ+ZrXT3+SPZh7yatW/N2i9Q34aq2frm7hfV8e0Gmouwrpc+NSYmIsNpCzCr6vuTgK313IGSmIgMp5eAOWZ2spm1A1cAy+q5g0YP7C/JfknDNGvfmrVfoL4NVTP3rSbu3mNm1wFPAK3Ave6+rp77MC/QPVIiIkfT6aSIFJqSmIgUWkOS2HDfhlALM9tkZmvMbPVR9S+N6Mu9ZrbDzNZWbZtiZk+Z2Ybk6+Qm6tstZvZO8tmtNrNLGtS3WWb2jJmtN7N1ZvatZHtDP7ugX03xuRXViI+JJbch/Bb4QyqXX18CrnT3ulXw1sLMNgHz3b3hhZFm9gXgA+A+d/9Msu07wB53vy35H8Bkd7+xSfp2C/CBu393pPtzVN9mADPcfZWZdQIvA5cC36SBn13Qr6/TBJ9bUTXiSOzfbkNw9yNA720IchR3fxbYc9TmhcDS5PlSKn8EIy6lb03B3be5+6rk+X5gPZXK8YZ+dkG/pAaNSGID3YbQTD9IB540s5fNbHGjOzOA6e6+DSp/FMDxDe7P0a4zs1eT082GnOpWM7PZwJnACzTRZ3dUv6DJPrciaUQSG/bbEGp0nrufReWu+2uT0ybJ5y7gVGAesA24vZGdMbPxwMPA9e6+r5F9qTZAv5rqcyuaRiSxYb8NoRbuvjX5ugN4lMrpbzPZnoyt9I6x7Ghwf/6Nu29395JXFi28mwZ+dmbWRiVR3O/ujySbG/7ZDdSvZvrciqgRSWzYb0MYKjPrSAZcMbMO4MvA2rjViFsGLEqeLwIea2Bf+uhNEInLaNBnZ2YG3AOsd/c7qkIN/ezS+tUsn1tRNaRiP7mE/L/48DaEW0e8EwMws1OoHH1B5ZasBxrZNzP7MXA+lalatgPfBv4JeAj4GPA2cLm7j/gAe0rfzqdySuTAJuCa3jGoEe7b7wG/BNYAvTP33Uxl/Klhn13Qrytpgs+tqHTbkYgUmir2RaTQlMREpNCUxESk0JTERKTQlMREpNCUxESk0JTERKTQ/j/oNQwZhzrgvQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow(img_train[0])\n",
    "plt.colorbar()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data preprocessing\n",
    "\n",
    "먼저 데이터 전처리를 진행합시다. 0-255의 값을 그대로 neural network에 집어넣어도 되지만 그 전에 모든 픽셀값을 0-1로 normalize해줍시다. Training set과 test set에 똑같이 적용합니다.\n",
    "\n",
    "왜 이런 작업을 해 주는것이 중요한가...라고 물으신다면 간단하게 학습이 더 빨라지고, 더 괜찮게 converge할 수 있게 해 준다라고 이야기 할 것 같습니다. 지금이야 feature가 한 종류니까 그렇다쳐도 각 input 값들이 다른 scale을 가지게 된다면, 그리고 normalization을 하지 않는다면 특정 feature가 dominant하게 작용할 수도 있기 때문입니다.\n",
    "\n",
    "영상에서는 weight, bias는 random하게 0-1사이로 정해지는데 input value와 mismatch가 일어나면 학습 단계에서 업데이트 시 문제가 생길 수 있다고 합니다.\n",
    "\n",
    "당장 바로 아래 코드를 돌리고 NN 학습 후 결과를 보는 것과, 돌리지 않고 보는 것 사이에 4-5%의 차이가 생기는 것을 확인할 수 있습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "img_train = img_train/255.0\n",
    "img_test = img_test/255.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Building NN model\n",
    "\n",
    "이제 NN 모델을 만들어봅시다. Keras 덕분에 매우 쉽게, 코드 몇 줄 만으로 neural network를 구현할 수 있습니다.\n",
    "\n",
    "Input layer, output layer, 그리고 hidden layer 1개로 구성된 neural network를 만들어봅시다. 이 모델은 fully connected network(FCN)입니다.\n",
    "\n",
    "  - Input layer에서는 28x28 모양의 이미지를 1x784개로 펼칩니다. 그래서 각 픽셀이 neuron 1개에 대응될 수 있도록 해 줍니다. Input layer의 neuron 개수는 784개입니다.\n",
    "  \n",
    "  - Hidden layer의 neuron 개수는 128개입니다. Dense layer이기 때문에 이전 layer의 neuron들이 현재 layer의 neuron들과 모두 연결되어 있습니다. Activation function으로는 ReLU를 사용했습니다.\n",
    "  \n",
    "  - Output layer의 neuron 개수는 10개입니다. 마찬가지로 dense layer이고, 이제 neuron들은 10개의 class와 일대일 대응이 됩니다. Activation function으로 Softmax를 사용했기 때문에 neuron의 값이 해당 class일 확률이라고 생각할 수 있습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = keras.Sequential([\n",
    "    keras.layers.Flatten(input_shape = (28, 28)),  # input layer (1)\n",
    "    keras.layers.Dense(128, activation = 'relu'),  # hidden layer (2)\n",
    "    keras.layers.Dense(10, activation = 'softmax') # output layer (3)\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Compile model\n",
    "\n",
    "마지막으로 loss function과 optimizer, metric을 지정하면서 model을 compile해줍니다. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(optimizer = 'adam', loss = 'sparse_categorical_crossentropy', metrics = ['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training model\n",
    "\n",
    "이제 model을 학습시켜봅시다. model.fit(data, label, epochs)으로 간단하게 모델을 학습시킬 수 있습니다. 매 epoch, 매 순간마다 학습 경과와 loss, accuracy를 출력해줍니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 60000 samples\n",
      "Epoch 1/10\n",
      "60000/60000 [==============================] - 3s 46us/sample - loss: 0.5033 - accuracy: 0.8227\n",
      "Epoch 2/10\n",
      "60000/60000 [==============================] - 3s 42us/sample - loss: 0.3764 - accuracy: 0.8645\n",
      "Epoch 3/10\n",
      "60000/60000 [==============================] - 3s 42us/sample - loss: 0.3382 - accuracy: 0.8773\n",
      "Epoch 4/10\n",
      "60000/60000 [==============================] - 3s 42us/sample - loss: 0.3142 - accuracy: 0.8842\n",
      "Epoch 5/10\n",
      "60000/60000 [==============================] - 2s 41us/sample - loss: 0.2972 - accuracy: 0.8902\n",
      "Epoch 6/10\n",
      "60000/60000 [==============================] - 3s 42us/sample - loss: 0.2819 - accuracy: 0.8965\n",
      "Epoch 7/10\n",
      "60000/60000 [==============================] - 3s 42us/sample - loss: 0.2714 - accuracy: 0.8993\n",
      "Epoch 8/10\n",
      "60000/60000 [==============================] - 3s 42us/sample - loss: 0.2596 - accuracy: 0.9029\n",
      "Epoch 9/10\n",
      "60000/60000 [==============================] - 3s 42us/sample - loss: 0.2516 - accuracy: 0.9062\n",
      "Epoch 10/10\n",
      "60000/60000 [==============================] - 3s 42us/sample - loss: 0.2419 - accuracy: 0.9107\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7f0675771090>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit(img_train, label_train, epochs = 10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Validate model\n",
    "\n",
    "학습시킨 모델이 test set에 대해서 얼마나 정확하게 작동하는지 확인해보도록 하겠습니다. 여기서 verbose라는 parameter가 등장하는데, 위에서 보여준 학습 과정 등을 출력할 지 말 지를 정하는 파라미터입니다. 0일 경우에 아무것도 출력하지 않고, 1일 경우에 위와 같이 progress bar를 보여줍니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test accuracy: 0.8847\n"
     ]
    }
   ],
   "source": [
    "loss, accuracy = model.evaluate(img_test, label_test, verbose=0) \n",
    "\n",
    "print('Test accuracy:', accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "약 88%의 정확도를 보여주네요. Training 과정에서의 정확도보다 test 과정에서의 정확도가 더 낮은데 이런 현상을 overfitting 현상이라고 합니다. 최악의 경우에는, 모델이 training set만 잘 맞추고 test set은 거의 맞추지 못하는 상황도 발생합니다. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Making prediction model\n",
    "\n",
    "이제 test set에 있는 이미지를 예측하는 모델을 만들어봅시다. 0~1000 사이의 번호를 입력하면 test set에서 입력한 숫자에 해당하는 index의 이미지를 출력하면서 모델이 예측한 label과 실제 label을 출력합니다.\n",
    "\n",
    "*plt.title, plt.xlabel이 출력되어야 하는데 현재 출력이 되지 않습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pick a number: 123\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAS4AAAEWCAYAAADYaXqDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dfbxVVb3v8c8P5ElAFEEPAgopZsBV0J3a1Y6WVug9Ny0zJfViaXTPKyuro8fKo+bp3GtWeupoFibXh3zIMpPrpbQ8cqw0Az0qIHUiROXh8qgBqQH6O3+MuWXuvdccY+291l5rTfi+X6/1Yq35mw9jzc3+7THGHHNMc3dERMqkT7MLICLSXUpcIlI6SlwiUjpKXCJSOkpcIlI6SlwiUjpKXLIcOLFBx5oHnN+DWMw4wIHdelQiKSUlrsqWA68CW3Kv65pZoALHAyuacNwrCMniyCYcu5EcOKjZhZCu9Feq2H8HftHsQrQgA84BNgIzgN82tziyK1KNq/tuAH6U+/xV4CHCLzTAKcBTwCbgj8C0bPk84H8TftH/BNwHDM/t52jgUeBl4GlCbardcOD/AKuAl4CfAIOBnwL7saNWuB/hZ3pJduwNwN2djnMO8HwW+1I3vzvAO7PjfAY4E+ifi50L/Ar4elbO54CTCvYzCngG+LuC+MeAJdl+HgAOSJTrY4Tzsxr4fG75AOCfs9iq7P2AXPzjwFJCIp6TfTeAR7J/nyac2zMSx5dGcne9ur6Wu/uJBbHd3f0/3P1cd3+nu6939zFZ7Eh3/5O7v8fd+7j7aHc/JIvNc/eV7j7Z3Qe7+z3u/v0sNtrdN7j7ydl278k+j8zi/8/df+Due7l7P3c/Llt+vLuv6FS+C939N1mZBrj7d939ziw20d23uPtfZ7Fr3H177rse6+4vJ87NTe5+d1aODe7+wVzsXHff5u4fd/e+7v637r7K3S13Ds5393HZOZyZ27Y9hruf6u5L3f1t7r6bu1/q7o8WlGecB3dm5/W/uPu63He6Mjsf+2Tn81F3/8cs9u7s53d4dj7+xd0fye3b3f2gxPnQqwmvphegRV/LPfyCv5x7fTwXP9LdN7r78+4+Pbf8u+5+bcE+57n7VbnPE919q4df8L9399s6rf+Au89w91Hu/oaHpNV5n8d718S1xN1PyH0e5SGZ7Obul7n7XbnY4KwMRUm682t3d9/kIbG0f9/7cvFzPSSc/Pru7n+VOwfXZOd3eqd9z/Mdieun7n5eLtbH3V9x9wMqlGlcdoxDcsuu9pBgcfc/eviD0B57X3Z8snWuzsWGZOdqXPbZXYmrJV9qKhY7Fdgz97oxF/stsIzQPLw7t3wsoYlW5MXc++eBfsAIQjPodEIzsf11LKE5NZbQjHmpynIfANyb288S4HVgX0IzKF+GPxOajNX6ALAdmJt9vp3QFByZW+f/596/kv07JLfsLGAlHZvblb7DN3PfYSPhXI+ObNP53LY3+fbLPlcT20I4H7HjSAtQ4uqZTxL6SVYBF+eWvwgcGNlubO79/sA2YH223W10TJSDgauy2PBsWWeVpvZ4kZBM8vsaSEgWqzuVYXdg70h5O5tBSEIvEBLUDwnJd3o39nEF4TvfAfQtWOdF4BN0/A6DCH2ARTqf21XZ+1V07B+LxQYTzsfKxHeQJlPi6r6Dga8AZxM6ui8GpmSxm4CPAicQzu1o4JDctmcDEwkJ40pCreN14PuEq5jvI/wyDyR0zo8hJJufAt8G9iIkir/O9reG8Is2LHeM7wD/xI5fyJGECwZkx/sbQm2uf1aGav8PjM6+199k33cKcBjh4sSMKvcBIVmfTkgStxUc/zvAF4BJ2edh2TYx/0A4r5MIP4MfZMvvBC4lnIcRwGWE8w0heX40+y4DgP8FPE4YDgPh/L6l2i8mjaPEVez/0nEc172E4SPfJ/yyPg38Afgi4RdwAKEJ+VHgWsKVw3+j41/024CbCbWVgcCns+UvEpLLF4F12eeL2PHzOYfwC/87YC1wYbb8d4RfzGWEJtV+hCbWHOBBYDPwG+CobP3FhNriHYSE+BIdx4G9M/uulZxDuFr6YFb+9te3gEOByQXbVbIV+CCwDzCbrv8P7yWc47sIV2cXUXx1st2/Ea4OPkS4qvlgtvwrwALCFcyFwJPZMrJ1/wG4h3A+DiRcKW13BXAL4dx+uPqvJ73N3DWRYIPMIyS97zW5HCKlpxqXiJSOEpeI9Bozm21ma81sUUHczOxbZrbUzJ4xs8Or2a8SV+Mcj5qJsuu5mR13j1RyEjAhe80k3JmSpMQlIr3G3R8hjMMrcgpwazbY9zfAnmY2KrXfht5kPWLECB83blwjDymyS1m+fDnr16+39JrFzKw7V+wWA6/lPs9y91nd2H40HQcPr8iWrY5tVFPiMrNphMvvfYHvuftVsfXHjRvHggULajmkiES0tbU1+pCvuXstB62UZJOJs8dNRTPrC1xPaKNOBKab2cSe7k9EWoeZVfWqgxV0vOthDDvubChUSx/XkcBSd1/m7lsJgwVPSWwjIiXQp0+fql51MAf4H9nVxaOBP7l7tJkItTUVK7VNj+q8kpnNJFwtYP/996/hcCLSKHWqTWFmdxKuqI8wsxXA5YTb1nD37xBu2D+ZcNfDK4Q7T5JqSVxVtU2zjrpZAG1tbRqmL9Li6tgMxN2jN+B7uHXnk93dby2Jq0dtUxFpffVKXL2llkbqfGCCmY03s/6Em1Pn1KdYItJMDeyc75Ee17jcfbuZXUCYD7wvMNvdF9etZCLSNK1e46ppHJe7z2XHbJgishMws3pdMew1ejyZiHSxU9e4RGTnpMQlIqWjxCUipaPEJSKlos55ESkl1bhEpHSUuESkdJS4RKRUmn07TzWUuESkCyUuESkdXVUUkdJRjUtESkV9XCJSSkpcIlI6SlwiUjrqnBeRUlEfl4iUkhKXiJSOEpeIlI4Sl4iUjhKXiJSKJhIUkVJSjUtESkeJS0RKR4lLREpFA1BFpJSUuESkdHRVUURKRzUuKTV3j8Zb/T94b0mdl23btkXj/fv3L4ytW7cuuu3IkSOj8Vrt9H1cZrYc2Ay8Dmx397Z6FEpEmqvVE1c9GrLvcvcpSloiO4/2WlfqVeW+ppnZ781sqZldUiG+v5k9bGb/bmbPmNnJqX2qqSgiXdSrc97M+gLXA+8BVgDzzWyOuz+bW+1S4G53v8HMJgJzgXHR8tVYLgceNLMnzGxmQcFnmtkCM1uQaruLSPNVW9uqssZ1JLDU3Ze5+1bgLuCUTus4sEf2fhiwKrXTWmtcx7j7KjPbB/i5mf3O3R/pUCL3WcAsgLa2tniPpoi0hG70cY0wswW5z7Oy3/l2o4EXc59XAEd12scVhArQp4DBwImpg9aUuNx9VfbvWjO7l5BdH4lvJSKtrhuJa32if7vSjjpXYKYDN7v7N8zsHcBtZjbZ3d8o2mmPm4pmNtjMhra/B94LLOrp/kSkddSxqbgCGJv7PIauTcHzgLsB3P0xYCAwIrbTWmpc+wL3ZoXfDbjD3X9Ww/6kCXp7nNYjjxRXwCdPnhzddvjw4TUd+403Cv9g19z5nDovsXFaAIsWFf+Nv+iii6LbnnXWWYWxjRs3RretVh2HQ8wHJpjZeGAlcCbwkU7rvACcANxsZm8jJK5oh3iPE5e7LwMO6+n2ItKa6jmRoLtvN7MLgAeAvsBsd19sZlcCC9x9DvB54EYz+yyhGXmuJ/6iajiEiHRRzwGo7j6XMMQhv+yy3PtngWO6s08lLhHpotVHzitxiUgXSlwiUio7/U3WIrJzUuKSllbL9CsAzzzzTDT+oQ99qDA2dOjQ6LZf/vKXo/Gzzz47Gm/mZHgLFy6Mxn/xi18Uxk477bTotkcffXRhbPDgwfGCVUkTCYpI6ajGJSKloj4uESklJS4RKR0lLhEpHXXOi0ipqI9LREpJiUtaWmqcVsq1114bjQ8aNKjHx/7qV78aja9evToa33vvvQtjhx0Wn9hk//33j8YPPfTQaHzq1KnR+IYNGwpjBx54YHTb888/vzA2YMCA6LbVUuISkdJR4hKR0lHiEpFSqedEgr1FiUtEulCNS0RKR4lLREpHiUtESkUDUKXpYo/ogtpv7bjnnnui8dGjRxfGUmXbvn17NH755ZdH46+++mo0XovUOK+U2Hc/4ogjatp3PShxiUjp6KqiiJSKmooiUkpKXCJSOkpcIlI6SlwiUiq65UdESkk1LmmqWv8D/uxnP4vGR4wYEY3vsccehbFNmzZFt+3bt280PmXKlGg8Vmv485//HN329ddfj8ZffvnlaPyxxx7r8fapOc4aodUTV7I+aGazzWytmS3KLRtuZj83sz9k/+7Vu8UUkUZqHxKRejVLNQ3Zm4FpnZZdAjzk7hOAh7LPIrKTKH3icvdHgI2dFp8C3JK9vwU4tc7lEpEmqTZpNTNx9bSPa193Xw3g7qvNbJ+iFc1sJjATar+/S0Qao9WvKvZ66dx9lru3uXvbyJEje/twIlIHrV7j6mniWmNmowCyf9fWr0gi0mz1TFxmNs3Mfm9mS82sYn+4mX3YzJ41s8Vmdkdqnz1NXHOAGdn7GcB9PdyPiLSYevZxmVlf4HrgJGAiMN3MJnZaZwLwBeAYd58EXJjab7KPy8zuBI4HRpjZCuBy4CrgbjM7D3gBOD35DaQpaq3OX3311dF46jl+r732WmFs8+bN0W333HPPaHzLli3ReGzOq9QYsW3bttUUHzZsWDQee6bkxIkTC2ONUsdm4JHAUndflu33LsLFvWdz63wcuN7dXwJw92QLLpm43H16QeiE1LYiUk7d6JwfYWYLcp9nufus3OfRwIu5zyuAozrt42AAM/s10Be4wt2jI581cl5EuuhGjWu9u7fFdlVhmXf6vBswgdCyGwP80swmu3vh7QWtfc1TRBquzuO4VgBjc5/HAKsqrHOfu29z9+eA3xMSWSElLhHpoo6Jaz4wwczGm1l/4EzCxb28nwDvyo47gtB0XBbbqZqKItJFvTrn3X27mV0APEDov5rt7ovN7EpggbvPyWLvNbNngdeBi9x9Q2y/Slwi0kU9B5e6+1xgbqdll+XeO/C57FUVJa4WkJpCJXXpPvzcK0v9B1y4cGE0/vDDD0fjkyZNisY3bux8m+sO/fr1i26b+t4pse1T+049Gi31aLXYMBCID4cYMmRIdNveZppIUETKqNXn41LiEpEulLhEpHSUuESkdJS4RKRUmj1lTTWUuESkC11VFJHSUY1LklJ/3Wod5xVz3HHHReOHHHJINJ6aWiZWttijy6oRG7+WkjqnKYMGDYrGU7/4sceTrVy5Mrrt+PHjo/F6UOISkVJRH5eIlJISl4iUjjrnRaR0VOMSkVJRH5eIlJISl4iUjhKXJKXGFO22W89/TFOnTo3GU3M/peK1jONKjcNKzdeVmhMrFk9tm5Kar2vw4MHReOxnfv/990e3/dSnPhWN14MSl4iUiiYSFJFSUo1LREpHiUtESkeJS0RKR4lLREpFA1BFpJR0VXEXkBoTlPpPUMs4LYjPqbV48eLotocffng0vmFD9IHCNf1lTn3vWsdaxcaJpX4mqfi2bdui8dR5GThwYGHs1ltvjW6rcVyQTKtmNtvM1prZotyyK8xspZk9lb1O7t1iikgjtTcXU69mqaY+eDMwrcLya919SvaaWyEuIiVUbdJqZuJKtlHc/REzG9f7RRGRVlH6pmLEBWb2TNaU3KtoJTObaWYLzGzBunXrajiciDRKnz59qno1rXw93O4G4EBgCrAa+EbRiu4+y93b3L1t5MiRPTyciDRS6ZuKlbj7mvb3ZnYjEL+dXURKo9lJqRo9qnGZ2ajcxw8Ai4rWFZHyKX2Ny8zuBI4HRpjZCuBy4HgzmwI4sBz4RC+W8U21PEev1n3H2vO1tvVTx540aVI0vmbNmsLYYYcdFt1248aN0fhf/vKXaLyWObFSY6F6c5xX6meWelZl//79o/GtW7dG47Hv9vTTT0e3bYRWr3FVc1VxeoXFN/VCWUSkRZQ+cYnIrqUMEwm2dulEpCnq2cdlZtPM7PdmttTMLoms9yEzczNrS+1TiUtEuqhX4jKzvsD1wEnARGC6mU2ssN5Q4NPA49WUT4lLRLqoY43rSGCpuy9z963AXcApFdb7R+Bq4LVqdqrEJSJddCNxjWi/MyZ7zey0q9HAi7nPK7Jl+WNNBca6e9XjQVuqcz71yKdap3+J6c2rKNddd100/tnPfjYaHzRoUDQ+fvz4wtjmzZuj26bOeWrIQWq4xJ577lkYS/08U/tObV9LB3PqvKQeKZca4hIbTpEaJvLcc88VxlLnrBrdHKO13t1jfVKVdvTmyTGzPsC1wLlVF5AWS1wi0hrqeFVxBTA293kMsCr3eSgwGZiXJcu/AuaY2fvdfUHRTpW4RKSLOrZA5gMTzGw8sBI4E/hIe9Dd/wSMyB13HvB3saQF6uMSkQrq1Tnv7tuBC4AHgCXA3e6+2MyuNLP397R8qnGJSAf1vg8xm2h0bqdllxWse3w1+1TiEpEudMuPiJROq9/yo8QlIh00e8qaarRU4urNcVopS5cujcZnz55dGEs9TmrlypXR+MEHHxyNp8ZSvfrqq4Wx1PQqqX2nxiul/oOnpoeJqXXqmddeKx6EnfreqfNWa42klvOyYsWKwlhqDFi1lLhEpHSUuESkdJS4RKR0lLhEpFTKMJGgEpeIdKEal4iUjhKXiJSOElc33H9/fB6xb3/724WxLVu2RLd96qmnovHUvFVjxowpjA0ZMiS6bWy+LEiPGUrNsRTrj6hl22qkto8dv9YxYqmxWLFxXCmpfafKlvqZxsZxDR06NLpt7JzW8si2dhqAKiKlpM55ESkd1bhEpHSUuESkVNTHJSKlpMQlIqWjxCUipVP6q4pmNha4lfDYoDeAWe7+TTMbDvwAGAcsBz7s7i/F9rV9+3Y2bNhQGL/44oujZYnNOxV7Th3AfvvtF42nflCx5+SlxpC98sor0XhqbqZUPHZe+vXrF9029Zd19913j8ZTzx+MjTmKlRvSZUud102bNhXGBgwYEN221mc6psb2xaTGFI4dO7Ywlvo9qEYZ+riqSavbgc+7+9uAo4FPmtlE4BLgIXefADyUfRaRnUC9nvLTW5KJy91Xu/uT2fvNhEcMjQZOAW7JVrsFOLW3CikijdXqiatbfVxmNg6YCjwO7OvuqyEkNzPbp+6lE5GmaPWmYtWJy8yGAPcAF7r7pmq/mJnNBGZC/H4/EWkdrZ64qrp0YGb9CEnrdnf/cbZ4jZmNyuKjgLWVtnX3We7e5u5te++9dz3KLCK9qH0iwWpezZI8soXUexOwxN2vyYXmADOy9zOA++pfPBFphp2hj+sY4BxgoZm1zw3zReAq4G4zOw94ATg9taPNmzczb968wnjq8nZsSEJqGpHUZeLUsWPThaSGKwwePDgaT6lluETqvMTOKcDLL78cjaempokNx1i/fn2PtwV461vfGo0PHz68MJZ6jNfAgQOj8dRwitRwiD322KMwtnZtxcbLm4YNG1YYq+WxZ3mt3lRMJi53/xVQ9C1OqG9xRKQVlD5xiciupdnNwGoocYlIF6W/5UdEdj2qcYlI6ShxiUipqI9LREpJiStnr7324rTTTiuMp6ZImT17dmFs/vz50W1T45FS45l6U+o/SW+WLTYmCOCQQw6Jxo855pho/O1vf3th7KCDDopue8QRR0TjN9xwQzS+cePGwti+++4b3XbNmjXReGq8VOr/ciy+atWq6LYvvVQ8e1RqXF216pm4zGwa8E2gL/A9d7+qU/xzwPmEmWjWAR9z9+dj+2ztSwci0hT1uuXHzPoC1wMnAROB6dm0WHn/DrS5+6HAj4Crk+Xr9jcSkZ1atbf7VFkrOxJY6u7L3H0rcBdhSqw3ufvD7t5+68pvgORsDOrjEpEuutFUHGFmC3KfZ7n7rNzn0cCLuc8rgKMi+zsP+GnqoEpcItJFNxLXendvi+2qwrKKnbZmdjbQBhyXOqgSl4h0UcfO+RVAfpL8MUCXqw9mdiLwJeA4d49P+I8Sl4hUUMfENR+YYGbjgZXAmcBHOh1rKvBdYJq7x6fGyChxiUgH7RMJ1oO7bzezC4AHCMMhZrv7YjO7Eljg7nOArwFDgB9mCfMFd39/bL8tlbjOOOOMHsdT82k9+eST0XjsUVYQf5RW6nFSqcdwpeIpBxxwQGEsNWfV5MmTazp2M40bNy4aj43FSj1+LDUXWOrxZKn532Lzgb373e+ObjthwoTCWGqesGrVcxyXu88F5nZadlnu/Ynd3WdLJS4RaQ0aOS8ipaPEJSKlopusRaSUNJGgiJSOalwiUjpKXCJSKurjaqDdd989Gj/22GMbVBJplJNPPrmm+M6oXglHiUtESkeJS0RKR1cVRaRU1MclIqWkxCUipaPEJSKlo8QlIqXT6okreenAzMaa2cNmtsTMFpvZZ7LlV5jZSjN7KnvteoNmRHZC7RMJ1uPxZL2lmhrXduDz7v6kmQ0FnjCzn2exa939671XPBFphlavcSUTl7uvBlZn7zeb2RLCI4dEZCfV6omrW3U9MxsHTAUezxZdYGbPmNlsM9urYJuZZrbAzBasW7eupsKKSGPU8YGwvaLqxGVmQ4B7gAvdfRNwA3AgMIVQI/tGpe3cfZa7t7l728iRI+tQZBHpTXV+knWvqOqqopn1IySt2939xwDuviYXvxG4v1dKKCIN1+q3/FRzVdGAm4Al7n5Nbvmo3GofABbVv3gi0gw7Q43rGOAcYKGZPZUt+yIw3cymEB6nvRz4RK+UUEQartU756u5qvgroNK3mFthmYiUXLNrU9XQyHkR6UKJS0RKR4lLREql/ZafVqbEJSJdqMYlIqWjxCUipaPEJSKlo8QlIqWicVwiUkq6qigipaMal4iUTqsnrtauD4pIw9V7Pi4zm2ZmvzezpWZ2SYX4ADP7QRZ/PJuwNEqJS0S6qFfiMrO+wPXAScBEwqwyEzutdh7wkrsfBFwLfDW1XyUuEemijk/5ORJY6u7L3H0rcBdwSqd1TgFuyd7/CDjBElmxoX1cTzzxxHozez63aASwvpFl6IZWLVurlgtUtp6qZ9kOqHUHTzzxxANmNqLK1Qea2YLc51nuPiv3eTTwYu7zCuCoTvt4cx13325mfwL2JnJOGpq43L3DpPNmtsDd2xpZhmq1atlatVygsvVUq5XN3afVcXeVak7eg3U6UFNRRHrTCmBs7vMYYFXROma2GzAM2BjbqRKXiPSm+cAEMxtvZv2BM4E5ndaZA8zI3n8I+Fd3j9a4mj2Oa1Z6laZp1bK1arlAZeupVi5bTbI+qwuAB4C+wGx3X2xmVwIL3H0O4WE8t5nZUkJN68zUfi2R2EREWo6aiiJSOkpcIlI6TUlcqVsAmsnMlpvZQjN7qtP4lGaUZbaZrTWzRbllw83s52b2h+zfvVqobFeY2crs3D1lZic3qWxjzexhM1tiZovN7DPZ8qaeu0i5WuK8lUnD+7iyWwD+A3gP4TLofGC6uz/b0IIUMLPlQJu7N32wopn9NbAFuNXdJ2fLrgY2uvtVWdLfy93/vkXKdgWwxd2/3ujydCrbKGCUuz9pZkOBJ4BTgXNp4rmLlOvDtMB5K5Nm1LiquQVAAHd/hK7jWfK3R9xC+I/fcAVlawnuvtrdn8zebwaWEEZnN/XcRcol3dSMxFXpFoBW+uE58KCZPWFmM5tdmAr2dffVEH4RgH2aXJ7OLjCzZ7KmZFOasXnZTANTgcdpoXPXqVzQYuet1TUjcXV7eH+DHePuhxPuZv9k1iSS6twAHAhMAVYD32hmYcxsCHAPcKG7b2pmWfIqlKulzlsZNCNxVXMLQNO4+6rs37XAvYSmbStZk/WVtPeZrG1yed7k7mvc/XV3fwO4kSaeOzPrR0gOt7v7j7PFTT93lcrVSuetLJqRuKq5BaApzGxw1mmKmQ0G3gssim/VcPnbI2YA9zWxLB20J4XMB2jSucumRLkJWOLu1+RCTT13ReVqlfNWJk0ZOZ9d7v1ndtwC8E8NL0QFZvYWQi0Lwu1QdzSzbGZ2J3A8YdqTNcDlwE+Au4H9gReA09294Z3kBWU7ntDccWA58In2PqUGl+1Y4JfAQuCNbPEXCf1JTTt3kXJNpwXOW5nolh8RKR2NnBeR0lHiEpHSUeISkdJR4hKR0lHiEpHSUeJqrH2BO4BlhBtsHyOM22m2qYRL8e+rcv15QKWHOywnDI+oVnfXL3Iq4Zl9sotQ4mocI4zBegR4C3AEYfDtmGYWKjMd+FX2bxkpce1ilLga593AVuA7uWXPA/+SvT8XuC4Xu58woBPCCP7HgCeBHwJDsuVXAc8CzwDtU6KcThh5/TQhSaYY4QEF52bHGZgtH0eYveBGYDHwIDCo07Z9CLMsfKXCfs8Gfgs8BXyXMNi4kouy9X4LHJQtOwB4KPteDxEGjBYt/6/A+4GvZcc6MPF9ZSegxNU4kwiJp7tGAJcCJwKHAwuAzwHDCc3MScCh7EgelxGafIcRfqEB9gPmFuz/GOA54I+EJmB+ErsJhMenTwJeBk7LxXYDbifMrXZpp32+DTgj2/cU4HXgrILjbyLcm3cd4W4Ksve3Zt/rduBbkeWPEm7luSg71h8LjiM7ESWu5rmeUCuan1jvaEIz6NeEGsUMQs1jE/Aa8D3gg8Ar2fq/Bm4GPs6OWs4qOiakvOmEOdHI/s03F5/LjgmhT25cLvZdQs2u0i1RJxCawvOz7U8gNI8ruTP37zuy9+8g9AUC3AYcm1guu5hmP55sV7KYjjWWTxJqU+3TQ2+n4x+S9iabAT+ncv/TkYSkcCZwAaE5+j8Jjzj/b4SkMQXYUFCmvlmZ3g98KTvW3sDQLP6X3Lqv07Gp+CjwLsIULK912q8RmpBfKDhunhe8L1qnmuWyk1ONq3H+lZCM/ja3bPfc++WEJNOHMO1P+9QmvyE0uQ7KbXMwoZ9rGKEJeGG2LYQ+nscJTcb1dJxCqLMTCbW+sYTa1AGEKVeqmRn0puzYP6TrH8CHCP1m7RP1Dc/2XckZuX8fy94/yo5n651FuHAQW76ZHclWdgGqcTWOExLCtcDFwH5ZWGwAAACrSURBVDrgz0D7nOe/JjTNFhKaYO39YesIHed3AgOyZZcSflnvIyRDAz6bxb5G6JsyQgJ5mtDH9T26Nhens2M2jHb3EJLrL6v4TtcQkudtdOzDejYr44OERLyNUMN8vsI+BhASbR921Co/Dcwm9FutAz6aWH4X4SLCpwkJU/1cOznNDiEipaOmooiUjhKXiJSOEpeIlI4Sl4iUjhKXiJSOEpeIlI4Sl4iUzn8CRKzulKAch04AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "COLOR = 'white'\n",
    "plt.rcParams['text.color'] = COLOR\n",
    "plt.rcParams['axes.labelcolor'] = COLOR\n",
    "\n",
    "def predict(model, image, correct_label):\n",
    "    class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat',\n",
    "               'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']\n",
    "    prediction = model.predict(np.array([image]))\n",
    "    predicted_class = class_names[np.argmax(prediction)]\n",
    "\n",
    "    show_image(image, class_names[correct_label], predicted_class)\n",
    "\n",
    "\n",
    "def show_image(img, label, guess):\n",
    "    plt.figure()\n",
    "    plt.imshow(img, cmap=plt.cm.binary)\n",
    "    plt.colorbar()\n",
    "    plt.grid(False)\n",
    "    plt.title(\"Excpected: \" + label)\n",
    "    plt.xlabel(\"Guess: \" + guess)\n",
    "    plt.show()   \n",
    "\n",
    "def get_number():\n",
    "    while True:\n",
    "        num = input(\"Pick a number: \")\n",
    "        if num.isdigit():\n",
    "            num = int(num)\n",
    "            if 0 <= num <= 1000:\n",
    "                return int(num)\n",
    "        else:\n",
    "            print(\"Try again...\")\n",
    "\n",
    "num = get_number()\n",
    "image = img_test[num]\n",
    "label = label_test[num]\n",
    "predict(model, image, label)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "\n",
    "매우 간단하게 neural network을 만들어보고, 이미지를 분류해보는 작업을 했습니다. 다음 module에서는 DNN보다 이미지 처리에 더 적합한 Convolutional neural network(CNN)에 대해 알아보도록 하겠습니다."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}