02 梯度下降.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$cost=\\dfrac{1}{N}\\sum ^{N}_{n=1}\\left( \\widehat{y}_{n}-y_{n}\\right) ^{2}$$\n",
    "\n",
    "三、[梯度下降](https://so.csdn.net/so/search?q=%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D&spm=1001.2101.3001.7020)\n",
    "------------------------------------------------------------------------------------------------------\n",
    "\n",
    "类似[牛顿迭代法](https://so.csdn.net/so/search?q=%E7%89%9B%E9%A1%BF%E8%BF%AD%E4%BB%A3%E6%B3%95&spm=1001.2101.3001.7020)/ 二分法，对cost func 求导 , 利用偏导进行迭代，使得cost func 达到最小值。\n",
    "\n",
    "![](https://img-blog.csdnimg.cn/img_convert/9b8fe9d396ca643d6b62693bf627323c.png)\n",
    "\n",
    "具体的求导过程\n",
    "\n",
    "![](https://img-blog.csdnimg.cn/img_convert/f16d0ff50dd6280d501a1cd928571a45.png)\n",
    "\n",
    "关键迭代式：\n",
    "\n",
    "![](https://img-blog.csdnimg.cn/img_convert/8821b49fbfe7e8853d1e1e82a73a521e.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 梯度下降法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "w =  2.0 epoch =  189\n",
      "After train forward(4) =  7.99999996372013\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEJCAYAAACZjSCSAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAdRklEQVR4nO3df5RfdX3n8edrfv9MJskMJOQHSQAVofKjMcWilrVogSJxLWps/bF0PTl4cFdre1qtq1a73V3b6q6Ia5pdUWipv6rY1BNq0QWV9iAkIcFAEAKIGQjJ5Hcmv2fmvX/cO5NvvvnO5Esyd+43c1+Pc75n7vfez9x5585kXvO5n3s/VxGBmZkVV13eBZiZWb4cBGZmBecgMDMrOAeBmVnBOQjMzArOQWBmVnCZB4GkekmPSPpehW2SdKukTZIelXR51vWYmdnxJqJH8EFg4yjbrgUuSF/LgC9NQD1mZlaiIcudS5oD/DbwF8CHKzRZAtwZyV1tD0rqkjQrIraMts/u7u6YP39+JvWamU1Wa9as2R4RPZW2ZRoEwP8C/hjoHGX7bGBzyfvedN2oQTB//nxWr149XvWZmRWCpOdG25bZqSFJ1wPbImLNWM0qrDthzgtJyyStlrS6r69v3Go0M7NsxwiuBG6Q9Avg68AbJP1dWZteYG7J+znAC+U7iogVEbEoIhb19FTs2ZiZ2SnKLAgi4qMRMSci5gNLgf8XEe8qa7YSeE969dAVwJ6xxgfMzGz8ZT1GcAJJNwNExHJgFXAdsAk4ANw00fWYmRXdhARBRNwP3J8uLy9ZH8AtE1GDmZlV5juLzcwKzkFgZlZwhQmCJ17cy19//+fs3H8k71LMzGpKYYLg2b793HbfJrbuPZR3KWZmNaUwQdDWnIyLHzgykHMlZma1pThB0FQPwIEjgzlXYmZWWwoXBPsPOwjMzEoVJgjam3xqyMysksIEQVtz2iPwqSEzs+MUJghGegSH3SMwMytVmCBobfRgsZlZJYUJgro60dpY7zECM7MyhQkCgPbmeo8RmJmVKVQQtDU1eIzAzKxMwYKg3mMEZmZlHARmZgVXqCBob25gvweLzcyOk1kQSGqR9JCk9ZIek/SpCm2ukrRH0rr09Yms6oG0R+ApJszMjpPloyoPA2+IiH5JjcADku6JiAfL2v0kIq7PsI4R7U0NHDjqHoGZWanMgiB9HnF/+rYxfUVWX68abc3uEZiZlct0jEBSvaR1wDbg3oj4aYVmr0lPH90j6aIs62lr8hiBmVm5TIMgIgYj4lJgDrBY0sVlTdYC50bEJcAXgO9W2o+kZZJWS1rd19d3yvW0NdVz6OgQg0O5dkzMzGrKhFw1FBG7gfuBa8rW742I/nR5FdAoqbvC56+IiEURsainp+eU6/BU1GZmJ8ryqqEeSV3pcitwNfBEWZuZkpQuL07r2ZFVTcNTUR/0vQRmZiOyvGpoFnCHpHqSX/DfjIjvSboZICKWAzcC75c0ABwElqaDzJkY7hF4viEzs2OyvGroUeCyCuuXlyzfBtyWVQ3lWkceV+lTQ2Zmw4p1Z/HIGIF7BGZmwwoVBMNjBB4sNjM7plBB4B6BmdmJChUEbR4jMDM7QaGCoL3ZPQIzs3KFCoKRHoHHCMzMRhQqCJob6qiTbygzMytVqCCQRHtTA/s9A6mZ2YhCBQGkU1H71JCZ2YjCBUF7U4OnmDAzK1G4IGhtquegewRmZiMKFwQeIzAzO17hgqCtud6Xj5qZlShcEHQ0N9B/yEFgZjascEHQ2dLIPk8xYWY2ooBB4B6BmVmpwgVBR3MDB48OcnRwKO9SzMxqQpbPLG6R9JCk9ZIek/SpCm0k6VZJmyQ9KunyrOoZ1tmSPq7Sp4fMzIBsewSHgTdExCXApcA1kq4oa3MtcEH6WgZ8KcN6gKRHALDPp4fMzIAMgyAS/enbxvRV/mD6JcCdadsHgS5Js7KqCY71CBwEZmaJTMcIJNVLWgdsA+6NiJ+WNZkNbC5535uuK9/PMkmrJa3u6+s7rZo6mhsB6PepITMzIOMgiIjBiLgUmAMslnRxWRNV+rQK+1kREYsiYlFPT89p1XSsR3D0tPZjZjZZTMhVQxGxG7gfuKZsUy8wt+T9HOCFLGvpSIPAPQIzs0SWVw31SOpKl1uBq4EnypqtBN6TXj10BbAnIrZkVRNApweLzcyO05DhvmcBd0iqJwmcb0bE9yTdDBARy4FVwHXAJuAAcFOG9QDJncXgIDAzG5ZZEETEo8BlFdYvL1kO4JasaqikpbGO+jrRf9hjBGZmUMA7iyV54jkzsxKFCwJIrhzyqSEzs0Qhg6CjucEzkJqZpQoZBJ6B1MzsmIIGQSP7PFhsZgYUNAg8WGxmdkwxg6ClwXcWm5mlChkEnS0N7HWPwMwMKGoQNDdwZGCIwwODeZdiZpa7QgbB8MNpPE5gZlbUIGjxMwnMzIYVMgj8lDIzs2OKGQSeitrMbEQhg8APpzEzO6aQQdA5Mkbgu4vNzAoZBMNXDe096B6BmVmWj6qcK+k+SRslPSbpgxXaXCVpj6R16esTWdVTampr0iPYc9A9AjOzLB9VOQD8YUSsldQJrJF0b0Q8XtbuJxFxfYZ1nKCpoY62pnoHgZkZGfYIImJLRKxNl/cBG4HZWX29l6qrtZHdBxwEZmYTMkYgaT7J84t/WmHzayStl3SPpIsmoh6AKa2N7hGYmZHtqSEAJHUA3wY+FBF7yzavBc6NiH5J1wHfBS6osI9lwDKAefPmjUtdXW2N7Dl4ZFz2ZWZ2Jsu0RyCpkSQE7oqI75Rvj4i9EdGfLq8CGiV1V2i3IiIWRcSinp6ecaltqnsEZmZAtlcNCfgysDEiPjdKm5lpOyQtTuvZkVVNpRwEZmaJLE8NXQm8G/iZpHXpuj8F5gFExHLgRuD9kgaAg8DSiIgMaxrR1dbkwWIzMzIMgoh4ANBJ2twG3JZVDWOZ2trI4YEhDh0dpKWxPo8SzMxqQiHvLAbfVGZmNsxB4CAws4IrbBB0tSVB4HECMyu6wgaBewRmZonCBkFXaxPgIDAzK2wQDPcIdh/w3cVmVmyFDYLOlgYk2OsegZkVXGGDoK5OTGlpZLeDwMwKrrBBAJ5mwswMCh4EyQykDgIzK7ZCB8FUP5zGzMxB4MFiMyu6wgeBB4vNrOgKHQTDYwQTNPO1mVlNKnYQtDYxOBTsPTSQdylmZrkpdBBMb0+mmdi533cXm1lxFToIZnQMB8HhnCsxM8tPVUEg6W3VrCvbPlfSfZI2SnpM0gcrtJGkWyVtkvSopMurL/30zWhvBmBHv3sEZlZc1fYIPlrlulIDwB9GxIXAFcAtkl5Z1uZa4IL0tQz4UpX1jIvpHT41ZGY25jOLJV0LXAfMlnRryaYpJL/oRxURW4At6fI+SRuB2cDjJc2WAHemD6x/UFKXpFnp52ZuRjpGsMNBYGYFdrKH178ArAZuANaUrN8H/EG1X0TSfOAy4Kdlm2YDm0ve96brJiQIWhrraW+q96khMyu0MYMgItYD6yX9fUQcBZA0DZgbEbuq+QKSOoBvAx+KiL3lmyt92Qr7WEZy6oh58+ZV82WrNr2jyYPFZlZo1Y4R3CtpiqTpwHrgK5I+d7JPktRIEgJ3RcR3KjTpBeaWvJ9D0gs5TkSsiIhFEbGop6enypKrM7292aeGzKzQqg2Cqelf828FvhIRvwpcPdYnSBLwZWBjRIwWGiuB96RXD10B7Jmo8YFhM9qbPFhsZoV2sjGCkXaSZgFvBz5W5edcCbwb+Jmkdem6PwXmAUTEcmAVyWD0JuAAcFOV+x4309ub2Lil/IyVmVlxVBsEnwa+D/xrRDwsaSHw1FifEBEPUHkMoLRNALdUWUMmZnQ0saP/CBFB0okxMyuWqoIgIr4FfKvk/TPA72RV1ESa0d7EkcEh+g8P0NnSmHc5ZmYTrto7i+dIulvSNklbJX1b0pysi5sI09O7iz1OYGZFVe1g8VdIBnbPIbnO/5/SdWc831RmZkVXbRD0RMRXImIgfX0VGN/rOHMyMvGcbyozs4KqNgi2S3qXpPr09S5gR5aFTZTpIz0C31RmZsVUbRD8Psmloy+STP9wIzlc6pmFkRlIfWrIzAqq2stH/xx47/C0Eukdxn9NEhBntNamelob631qyMwKq9oewatK5xaKiJ0kk8hNCjM6mtwjMLPCqjYI6tLJ5oCRHkG1vYma19PZTN8+jxGYWTFV+8v8s8C/SfoHktlB3w78RWZVTbCzO1t4uq8/7zLMzHJRVY8gIu4kuZN4K9AHvDUi/jbLwibS2VOa2br3UN5lmJnlourTOxHxOMc/XWzSOGtKC3sPDXDo6CAtjfV5l2NmNqGqHSOY1M7qTC4h3bbX4wRmVjwOAuDsKS0AbN3n00NmVjwOAkqCwOMEZlZADgKSwWKArT41ZGYF5CAAprY20tRQxzb3CMysgDILAkm3p88v2DDK9qsk7ZG0Ln19IqtaTkYSZ3U2s803lZlZAWV5d/BXgduAO8do85OIuD7DGqp29pQWjxGYWSFl1iOIiB8DO7Pa/3jzTWVmVlR5jxG8RtJ6SfdIuijPQs7qbPF9BGZWSHkGwVrg3Ii4BPgC8N3RGkpaJmm1pNV9fX2ZFHPWlGb2HR7gwJGBTPZvZlarcguCiNgbEf3p8iqgUVL3KG1XRMSiiFjU05PNEzLP7kzuJXCvwMyKJrcgkDRTktLlxWktuT3+cvimshc9TmBmBZPZVUOSvgZcBXRL6gU+CTQCRMRyksddvl/SAHAQWBoRkVU9JzOrKwmCF3YfzKsEM7NcZBYEEfHOk2y/jeTy0powu6sVgN5dDgIzK5a8rxqqGS2N9XR3NPO8g8DMCsZBUGLOtFae96khMysYB0GJ2dNa6d11IO8yzMwmlIOgxJxprbyw+xBDQ7mNWZuZTTgHQYk5Xa0cGRyir9/3EphZcTgISsyZ1gb4yiEzKxYHQYnZ05JLSD1gbGZF4iAocexeAg8Ym1lxOAhKtDc3MK2t0fcSmFmhOAjKzPa9BGZWMA6CMnO62ti806eGzKw4HARlzp3RxuZdBxn0vQRmVhAOgjILuts5MjDkWUjNrDAcBGUWdLcD8Oz2/TlXYmY2MRwEZRb0OAjMrFgcBGV6OprpaG7gmb7+vEsxM5sQDoIykljY084z7hGYWUFkFgSSbpe0TdKGUbZL0q2SNkl6VNLlWdXyUi3obvepITMrjCx7BF8Frhlj+7XABelrGfClDGt5SRZ0t/P87oMcOjqYdylmZpnLLAgi4sfAzjGaLAHujMSDQJekWVnV81Is6G4nAn7pG8vMrADyHCOYDWwued+brjuBpGWSVkta3dfXl3lhC7s7ADxgbGaFkGcQqMK6irfzRsSKiFgUEYt6enoyLgvmdyfPJXi6z+MEZjb55RkEvcDckvdzgBdyquU4nS2NnDO1hSe37su7FDOzzOUZBCuB96RXD10B7ImILTnWc5wLZ01h45a9eZdhZpa5hqx2LOlrwFVAt6Re4JNAI0BELAdWAdcBm4ADwE1Z1XIqXjGrk/uf7OPwwCDNDfV5l2NmlpnMgiAi3nmS7QHcktXXP10XzprC4FCwaVs/F50zNe9yzMwy4zuLR/GKmVMA2LjF4wRmNrk5CEYxf0YbzQ11POFxAjOb5BwEo2ior+PlMzvZ+KKDwMwmNwfBGF4xs5ONW/aRDGeYmU1ODoIxvGLmFHbuP8K2fYfzLsXMLDMOgjFcMje5WuiRX+7OtxAzsww5CMZw0TlTaawXj2zelXcpZmaZcRCMoaWxnovOmcojz+3OuxQzs8w4CE7isnldPPr8bo4ODuVdiplZJhwEJ3H5vGkcOjrEE76xzMwmKQfBSVw2rwvA4wRmNmk5CE5idlcrZ3U2s/Y5B4GZTU4OgpOQxKsXTOfBZ3b6xjIzm5QcBFV47fndvLj3EE/70ZVmNgk5CKrw2vO7AfjJU9tzrsTMbPw5CKowd3ob585o4wEHgZlNQg6CKl15fjcPPrPD9xOY2aSTaRBIukbSzyVtkvSRCtuvkrRH0rr09Yks6zkdrzu/m/1HBlm3eXfepZiZjassn1lcD3wReCPQCzwsaWVEPF7W9CcRcX1WdYyXXz+/m4Y68YPHt/Lq+dPzLsfMbNxk2SNYDGyKiGci4gjwdWBJhl8vU1NbG7ny/G5Wbdjiy0jNbFLJMghmA5tL3vem68q9RtJ6SfdIuqjSjiQtk7Ra0uq+vr4saq3KtRfPZPPOgzz2gp9aZmaTR5ZBoArryv+UXgucGxGXAF8AvltpRxGxIiIWRcSinp6e8a3yJXjTRTOprxP3bNiSWw1mZuMtyyDoBeaWvJ8DvFDaICL2RkR/urwKaJTUnWFNp2V6exNXLJzOqp+96NNDZjZpZBkEDwMXSFogqQlYCqwsbSBppiSly4vTenZkWNNpe/OrzuHZ7ftZ+0vPPWRmk0NmQRARA8AHgO8DG4FvRsRjkm6WdHPa7EZgg6T1wK3A0qjxP7XffMk5tDfV87WHNp+8sZnZGSCzy0dh5HTPqrJ1y0uWbwNuy7KG8dbe3MANl87m7kd6+fj1r2Rqa2PeJZmZnRbfWXwKfnfxPA4dHeIf1z2fdylmZqfNQXAKfmXOVC6Z28WXH3iWAU85YWZnOAfBKbrlqvN4bscBvveoLyU1szObg+AUXX3h2bz87E6+eN8mhoZqenzbzGxMDoJTVFcnbnnD+Ty1rZ+7H/FYgZmduRwEp+H6X5nFpXO7+O/3PMG+Q0fzLsfM7JQ4CE5DXZ341A0Xsb3/MJ//wVN5l2NmdkocBKfpkrldvHPxXG7/12d56NmdeZdjZvaSOQjGwcd++5XMnd7GH3xjHXsO+hSRmZ1ZHATjoKO5gc8vvYytew/xn7/2iB9naWZnFAfBOLl0bhf/9S0X86Mn+/j4dzd4dlIzO2NkOtdQ0SxdPI/Nuw7wxfuepq5O/PmSi6mvq/RYBjOz2uEgGGd/9KaXEwH/+/6n2bX/CH/1tkvoaPZhNrPa5d9Q40wSf3zNK5je3sR/W7WRJ7fu4/NLL+Pi2VPzLs3MrCKPEWTkfa9byF3vu4J9hwa44bYH+PQ/Pc6u/UfyLsvM7AQOggy95rwZ3Pvh3+Adr57HV/7tWV7/l/fxmX9+gi17DuZdmpnZCGV5dYuka4DPA/XA/42I/1G2Xen264ADwH+IiLVj7XPRokWxevXqjCrOzpNb9/E/732S7z/2IgCvnj+day+eyW9dPJNZU1tzrs7MJjtJayJiUcVtWQWBpHrgSeCNJA+yfxh4Z0Q8XtLmOuA/kQTBrwGfj4hfG2u/Z2oQDNu88wDfWtPLP2/YwpNb+wFY2N3OpXO7uGxeFxfOmsKC7namtzeRPs7ZzOy0jRUEWQ4WLwY2RcQzaRFfB5YAj5e0WQLcmT6n+EFJXZJmRcSkneR/7vQ2PvzGl/HhN76Mp/v6uffxrax5bhc/fmo73ymZxXRKSwPzZrTR3dE88urpbKartZH25nramhqOfWxqoLWpnsZ60VBfR0OdaKgT9XVymJjZSWUZBLOB0ie895L81X+yNrOBSRsEpc7r6eC83+gAICJ4fvdBntraz7Pb9/Ps9v307jrA9v4jPLFlH9v7DzNwCs89aKgTDfWioa6OhnpRL5FkQ/JRwHBW6Lh1ycqRbTpx+0jEpOts/DnIs3MmHtl3vHou73vdwnHfb5ZBUOk4l/8mq6YNkpYBywDmzZt3+pXVIEnMmdbGnGlt/LsK24eGgj0Hj7L30FH2Hx7kwJEB9h8Z5MDhAQ4cSd4fHQwGh4KjQ0MMDAYDQ8HA4FCybjAYGEqWA0jOCAbDZwYjINL3UfI+2Ti8Lkq2DX9enPgNs/HhA5uZM/WntrujOZP9ZhkEvcDckvdzgBdOoQ0RsQJYAckYwfiWeWaoqxPT2puY1t6UdylmNslkefnow8AFkhZIagKWAivL2qwE3qPEFcCeyTw+YGZWizLrEUTEgKQPAN8nuXz09oh4TNLN6fblwCqSK4Y2kVw+elNW9ZiZWWWZTjEREatIftmXrlteshzALVnWYGZmY/OdxWZmBecgMDMrOAeBmVnBOQjMzArOQWBmVnCZzj6aBUl9wHOn+OndwPZxLCcLtV5jrdcHtV9jrdcHtV9jrdcHtVfjuRHRU2nDGRcEp0PS6tFm36sVtV5jrdcHtV9jrdcHtV9jrdcHZ0aNw3xqyMys4BwEZmYFV7QgWJF3AVWo9RprvT6o/RprvT6o/RprvT44M2oECjZGYGZmJypaj8DMzMoUJggkXSPp55I2SfpIDdQzV9J9kjZKekzSB9P1fybpeUnr0td1Odf5C0k/S2tZna6bLuleSU+lH6flVNvLS47TOkl7JX0o72Mo6XZJ2yRtKFk36jGT9NH05/Lnkn4rp/r+StITkh6VdLekrnT9fEkHS47l8lF3nH2No35fa+QYfqOktl9IWpeuz+UYviQRMelfJNNgPw0sBJqA9cArc65pFnB5utwJPAm8Evgz4I/yPmYldf4C6C5b95fAR9LljwCfqYE664EXgXPzPobA64HLgQ0nO2bp93w90AwsSH9O63Oo701AQ7r8mZL65pe2y/kYVvy+1soxLNv+WeATeR7Dl/IqSo9gMbApIp6JiCPA14EleRYUEVsiYm26vA/YSPK85jPBEuCOdPkO4C35lTLiN4GnI+JUbzYcNxHxY2Bn2erRjtkS4OsRcTginiV5Nsfiia4vIv4lIgbStw+SPC0wN6Mcw9HUxDEcpuRB028HvpZlDeOpKEEwG9hc8r6XGvqlK2k+cBnw03TVB9Iu+u15nXYpEcC/SFqTPjsa4OxInySXfjwrt+qOWcrx//Fq6RjC6MesFn82fx+4p+T9AkmPSPqRpNflVVSq0ve11o7h64CtEfFUybpaOoYnKEoQqMK6mrhcSlIH8G3gQxGxF/gScB5wKbCFpIuZpysj4nLgWuAWSa/PuZ4TpI9CvQH4Vrqq1o7hWGrqZ1PSx4AB4K501RZgXkRcBnwY+HtJU3Iqb7Tva00dQ+CdHP9HSS0dw4qKEgS9wNyS93OAF3KqZYSkRpIQuCsivgMQEVsjYjAihoD/Q8Zd3JOJiBfSj9uAu9N6tkqaBZB+3JZfhUASUmsjYivU3jFMjXbMauZnU9J7geuB34v05HZ6umVHuryG5Pz7y/Kob4zvay0dwwbgrcA3htfV0jEcTVGC4GHgAkkL0r8elwIr8ywoPY/4ZWBjRHyuZP2skmb/HthQ/rkTRVK7pM7hZZIBxQ0kx+69abP3Av+YT4UjjvsLrJaOYYnRjtlKYKmkZkkLgAuAhya6OEnXAH8C3BARB0rW90iqT5cXpvU9M9H1pV9/tO9rTRzD1NXAExHRO7yilo7hqPIerZ6oF3AdyZU5TwMfq4F6XkvSfX0UWJe+rgP+FvhZun4lMCvHGheSXI2xHnhs+LgBM4AfAk+lH6fnWGMbsAOYWrIu12NIEkpbgKMkf63+x7GOGfCx9Ofy58C1OdW3ieQ8+/DP4vK07e+k3/v1wFrgzTkew1G/r7VwDNP1XwVuLmubyzF8KS/fWWxmVnBFOTVkZmajcBCYmRWcg8DMrOAcBGZmBecgMDMrOAeB2QSSdJWk7+Vdh1kpB4GZWcE5CMwqkPQuSQ+l88f/jaR6Sf2SPitpraQfSupJ214q6cGSufynpevPl/QDSevTzzkv3X2HpH9I5/+/K73L3Cw3DgKzMpIuBN5BMuHepcAg8HtAO8mcRpcDPwI+mX7KncCfRMSrSO58HV5/F/DFiLgE+HWSO1EhmWn2QyTz6C8Ersz4n2Q2poa8CzCrQb8J/CrwcPrHeivJJHFDHJtM7O+A70iaCnRFxI/S9XcA30rnaJodEXcDRMQhgHR/D0U6F036FKv5wAOZ/6vMRuEgMDuRgDsi4qPHrZQ+XtZurPlZxjrdc7hkeRD/P7Sc+dSQ2Yl+CNwo6SwYed7wuST/X25M2/wu8EBE7AF2lTxs5N3AjyJ5tkSvpLek+2iW1DaR/wizavkvEbMyEfG4pP9C8mS2OpIZJm8B9gMXSVoD7CEZR4BkWunl6S/6Z4Cb0vXvBv5G0qfTfbxtAv8ZZlXz7KNmVZLUHxEdeddhNt58asjMrODcIzAzKzj3CMzMCs5BYGZWcA4CM7OCcxCYmRWcg8DMrOAcBGZmBff/AdHI7fNkPdPYAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "def function(x, w):         #数学模型\n",
    "    return x * w \n",
    "\n",
    "def cost(x, y, w):          #损失函数\n",
    "    cost = 0.0\n",
    "    for x_val, y_val in zip(x,y):\n",
    "        y_pred = function(x_val, w)\n",
    "        cost += (y_pred - y_val) ** 2\n",
    "    return cost/len(x)\n",
    "\n",
    "def gradient(x, y, w):      #梯度函数\n",
    "    grad = 0.0\n",
    "    for x_val, y_val in zip(x, y):\n",
    "        grad += x_val*(x_val*w-y_val)\n",
    "    return (2*grad)/len(x)\n",
    "\n",
    "x_data = [1.0, 2.0, 3.0]\n",
    "y_data = [2.0, 4.0, 6.0]\n",
    "alpha = 0.01\n",
    "w = 1.0\n",
    "epoch_list = []\n",
    "cost_list = []\n",
    "epoch = 0\n",
    "last_cost = cost(x_data, y_data, w)\n",
    "\n",
    "while True:\n",
    "    grad_val = gradient(x_data, y_data, w)\n",
    "    w -= alpha*grad_val\n",
    "    this_cost = cost(x_data, y_data, w)\n",
    "    # print('epoch:', epoch, 'w=', w, 'loss=', this_cost)\n",
    "    epoch_list.append(epoch)\n",
    "    cost_list.append(this_cost)\n",
    "    epoch += 1\n",
    "    if((last_cost - this_cost) < 1e-16):\n",
    "        break\n",
    "    last_cost = this_cost\n",
    "    \n",
    "print(\"w = \",\"%.1f\"%w,\"epoch = \",epoch)\n",
    "print(\"After train forward(4) = \",function(4, w))\n",
    "plt.plot(epoch_list,cost_list)\n",
    "plt.ylabel('cost')\n",
    "plt.xlabel('epoch')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "w =  2.0 b =  0.0 epoch =  6063\n",
      "After train forward(4) =  7.999999713401953\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEGCAYAAAB/+QKOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAf+klEQVR4nO3de3RdZ33m8e+jIx3rassXOXYsEdvgJrgh5CKMA5S2CUxtYMUsoIzTgTAMXV6ZkkJaWEwynZZpu9q5UQoZMjEmJCVDIAMhgGEZQriEW3Ox4jgXxzFRTBIrtmMlju8XSdZv/thbzvHxsX1ka+tIOs9nrbO8z7vffc7vTWw92rd3KyIwMzMrVlPpAszMbGxyQJiZWUkOCDMzK8kBYWZmJTkgzMyspNpKFzCSZsyYEXPnzq10GWZm48ZDDz30YkS0lVo3oQJi7ty5dHV1VboMM7NxQ9KzJ1rnQ0xmZlaSA8LMzEpyQJiZWUkOCDMzK8kBYWZmJTkgzMysJAeEmZmV5IAA/vdPnuLnv+mtdBlmZmNKpgEhaYmkTZK6JV1XYv15ku6TdFjSJ4vWtUq6U9KTkjZKujSrOm/6+dP86ikHhJlZoczupJaUA24E3g70AGslrY6IJwq67QQ+Bry7xEd8HvhhRLxPUh5ozKrWXI0YGPSDk8zMCmW5B7EI6I6IzRHRB9wBLCvsEBE7ImIt0F/YLmky8Fbgy2m/vojYlVWhtTVi4IgDwsysUJYBMQfYUvC+J20rx3ygF7hV0sOSbpbUVKqjpBWSuiR19fae3mGi2lyN9yDMzIpkGRAq0VbuT+Fa4GLgpoi4CNgPHHcOAyAiVkVEZ0R0trWVnJDw1F9WIwaODJ7WtmZmE1WWAdEDdBS8bwe2DmPbnoh4IH1/J0lgZCJXI454D8LM7BhZBsRaYIGkeelJ5uXA6nI2jIjtwBZJ56ZNlwNPnGSTM1LnQ0xmZsfJ7CqmiBiQdA1wN5ADbomIDZKuTtevlDQL6AImA4OSrgUWRsQe4M+B29Nw2Qx8OKtak6uYfIjJzKxQpg8Miog1wJqitpUFy9tJDj2V2nY90JllfUN8FZOZ2fF8JzVQm/M5CDOzYg4IIFdTQ78DwszsGA4IkkNMR3wOwszsGA4IfA7CzKwUBwTJOQhf5mpmdiwHBMk5CAeEmdmxHBBAnc9BmJkdxwFBeqOcz0GYmR3DAUEy1Ua/J+szMzuGAwLI19bQ54AwMzuGAwKor8txsM8BYWZWyAEBNNTlONR/pNJlmJmNKQ4IoCFfw8H+I0T4RLWZ2RAHBMkexJHBoN9XMpmZHeWAIDkHAXDQh5nMzI7KNCAkLZG0SVK3pOOeKS3pPEn3STos6ZMl1uckPSzp+1nW2ZBPAsLnIczMXpFZQEjKATcCS4GFwJWSFhZ12wl8DPjMCT7m48DGrGoc0jC0B9HngDAzG5LlHsQioDsiNkdEH3AHsKywQ0TsiIi1QH/xxpLagXcCN2dYI1AQEN6DMDM7KsuAmANsKXjfk7aV63PAp4DMb1CozzsgzMyKZRkQKtFW1mVCkt4F7IiIh8rou0JSl6Su3t7e4dYIQGO6B3HgsAPCzGxIlgHRA3QUvG8Htpa57ZuBKyQ9Q3Jo6jJJXy3VMSJWRURnRHS2tbWdVqEt9XUA7D103JEuM7OqlWVArAUWSJonKQ8sB1aXs2FEXB8R7RExN93upxHxgawKndxQC8DeQwNZfYWZ2bhTm9UHR8SApGuAu4EccEtEbJB0dbp+paRZQBcwGRiUdC2wMCL2ZFVXKUN7EHu8B2FmdlRmAQEQEWuANUVtKwuWt5McejrZZ9wL3JtBeUc1T0r+M+zxHoSZ2VG+k5rkgUEtk2p9DsLMrIADItVSX+tzEGZmBRwQqZb6OvYc9B6EmdkQB0RqcoP3IMzMCjkgUi31dew97D0IM7MhDohUS30tew56D8LMbIgDIjW5vs5XMZmZFXBApIauYvJjR83MEg6IVEt9HQOD4RldzcxSDohUa2My3cZuX+pqZgY4II5qbUgCYtcBB4SZGTggjprS6IAwMyvkgEi1NuQB2H2wr8KVmJmNDQ6IVKv3IMzMjuGASE1tTPYgdvkktZkZ4IA4qr6uhnxtDS8f8CEmMzPIOCAkLZG0SVK3pOtKrD9P0n2SDkv6ZEF7h6SfSdooaYOkj2dZZ/qdtDbUsduHmMzMgAyfKCcpB9wIvB3oAdZKWh0RTxR02wl8DHh30eYDwCciYp2kFuAhSfcUbTviWhvrfA7CzCyV5R7EIqA7IjZHRB9wB7CssENE7IiItUB/Ufu2iFiXLu8FNgJzMqwVSK5k2uWrmMzMgGwDYg6wpeB9D6fxQ17SXOAi4IETrF8hqUtSV29v7+nUedQU70GYmR2VZUCoRNuwZsKT1Ax8C7g2IvaU6hMRqyKiMyI629raTqPMV7Q21HmqDTOzVJYB0QN0FLxvB7aWu7GkOpJwuD0i7hrh2kryOQgzs1dkGRBrgQWS5knKA8uB1eVsKEnAl4GNEfHZDGs8RmtjnoP9RzjkGV3NzLK7iikiBiRdA9wN5IBbImKDpKvT9SslzQK6gMnAoKRrgYXABcAHgcckrU8/8j9HxJqs6gWYkk7Yt+dgP/V1uSy/ysxszMssIADSH+hritpWFixvJzn0VOxXlD6Hkamj020c7Gfm5PrR/nozszHFd1IXGJqw7+X9vtTVzMwBUaBwD8LMrNo5IAocfaqcr2QyM3NAFGo9OqOrDzGZmTkgCjTlc9TWiJe9B2Fm5oAoJInWxjy7POW3mZkDotj0pjwv7XNAmJk5IIpMa8qz05e5mpk5IIpNa3ZAmJmBA+I405vyvOSAMDNzQBSb1pRn98F++o8MVroUM7OKckAUmd6UTrfhK5nMrMo5IIpMa5oE4PMQZlb1HBBFpqV7EDt9qauZVTkHRJHpzUlA+ES1mVW7TANC0hJJmyR1S7quxPrzJN0n6bCkTw5n26xMTedj8iEmM6t2mQWEpBxwI7CU5ClxV0paWNRtJ/Ax4DOnsW0mpqYzunoPwsyqXZZ7EIuA7ojYHBF9wB3AssIOEbEjItYCxbPjnXLbrNTmamhtrGPn/sOj8XVmZmNWlgExB9hS8L4nbRvRbSWtkNQlqau3t/e0Ci3m6TbMzLINiFLPlI6R3jYiVkVEZ0R0trW1lV3cyXjCPjOzbAOiB+goeN8ObB2Fbc+Y9yDMzLINiLXAAknzJOWB5cDqUdj2jE1rmuSAMLOqV5vVB0fEgKRrgLuBHHBLRGyQdHW6fqWkWUAXMBkYlHQtsDAi9pTaNqtai01vyvPygT4GB4OamlJHu8zMJr7MAgIgItYAa4raVhYsbyc5fFTWtqNlWlOewYBdB/uP3lltZlZtfCd1CTNakvmYXtznS13NrHo5IEqYmQZE714HhJlVLwdECW1pQOzYe6jClZiZVY4DogTvQZiZOSBKap5US31dDTv2OCDMrHo5IEqQxMyWenp9ktrMqpgD4gTaWib5EJOZVTUHxAm0NU9ihwPCzKqYA+IEZk72HoSZVbeyAkLSH5fTNpG0NU9i98F+DvUfqXQpZmYVUe4exPVltk0YMyf7bmozq24nnYtJ0lLgHcAcSTcUrJoMDGRZWKW9crPcYdqnNla4GjOz0Xeqyfq2ksy2egXwUEH7XuAvsipqLJjZUg/4Zjkzq14nDYiIeAR4RNLXIqIfQNJUoCMiXh6NAiulcA/CzKwalXsO4h5JkyVNAx4BbpX02QzrqrjpTXkk70GYWfUqNyCmRMQe4D3ArRFxCfC2U20kaYmkTZK6JV1XYr0k3ZCuf1TSxQXr/kLSBkmPS/q6pPpyBzUSanM1TG/KOyDMrGqVGxC1kmYD7we+X84GknLAjcBSYCFwpaSFRd2WAgvS1wrgpnTbOcDHgM6IOJ/kqXLLy6x1xMxsqeeFPZ7R1cyqU7kB8Xckj/98OiLWSpoPPHWKbRYB3RGxOSL6gDuAZUV9lgG3ReJ+oDUNIkjOjzRIqgUaSU6Yj6rZU+rZttsBYWbVqayAiIhvRsQFEfEf0/ebI+K9p9hsDrCl4H1P2nbKPhHxPPAZ4DlgG7A7In5UTq0jadaUerbvPjjaX2tmNiaUeyd1u6RvS9oh6QVJ35JU8lnShZuVaIty+qRXSi0D5gFnA02SPnCC2lZI6pLU1dvbe6qhDMvZrQ28fMB3U5tZdSr3ENOtwGqSH9ZzgO+lbSfTA3QUvG/n+MNEJ+rzNuC3EdGbXl57F/CmUl8SEasiojMiOtva2socTnlmTU7Oi/swk5lVo3IDoi0ibo2IgfT1L8CpfhqvBRZImicpT3KSeXVRn9XAVenVTItJDiVtIzm0tFhSoyQBlwMbyx3USJndOhQQPsxkZtXnVHdSD3kxPcTz9fT9lcBLJ9sgIgYkXUNycjsH3BIRGyRdna5fCawhmcqjGzgAfDhd94CkO4F1JFN6PAysGs7ARsLsKQ0AbNvlPQgzqz7lBsR/AL4A/DPJeYR/Jf1hfjIRsYYkBArbVhYsB/DRE2z7aeDTZdaXidlTkj2I7b7U1cyqULkB8ffAh4am10jvqP4MSXBMWPV1OaY21rF1lw8xmVn1KfccxAWFcy9FxE7gomxKGltmTWlgu09Sm1kVKjcgatJLT4GjexDl7n2Ma2dPqWerA8LMqlC5P+T/CfjX9MRxkEy58Q+ZVTWGzJpSz7rnJvTEtWZmJZUVEBFxm6Qu4DKSm9veExFPZFrZGFF4s1x9Xa7S5ZiZjZqyDxOlgVAVoVCo8Ga5eTOaKlyNmdnoKfccRNU6uzW5F8JXMplZtXFAnEL71CQgtuw8UOFKzMxGlwPiFGZPqSdXI7a87IAws+rigDiF2lwNc1ob2LLTh5jMrLo4IMrQMa3BexBmVnUcEGXomNrocxBmVnUcEGXomNbIi/v6ONA3UOlSzMxGjQOiDENXMvW87PMQZlY9HBBl6JjWCPhSVzOrLg6IMnRMdUCYWfXJNCAkLZG0SVK3pOtKrJekG9L1j0q6uGBdq6Q7JT0paaOkS7Os9WRmNOdpqMuxxYeYzKyKZBYQknLAjcBSYCFwpaSFRd2WAgvS1wrgpoJ1nwd+GBHnAa+nAs+kHiKJ9qkN3oMws6qS5R7EIqA7IjZHRB9wB7CsqM8y4LZI3A+0SpotaTLwVuDLABHRFxG7Mqz1lM6Z3shzDggzqyJZBsQcYEvB+560rZw+84Fe4FZJD0u6WVLJqVQlrZDUJamrt7d35KovMm9GE799cT+Dg5HZd5iZjSVZBoRKtBX/dD1Rn1rgYuCmiLgI2A8cdw4DICJWRURnRHS2tbWdSb0nNW9GM4cHBtm62+chzKw6ZBkQPUBHwft2YGuZfXqAnoh4IG2/kyQwKmZ+W7ID89sX91eyDDOzUZNlQKwFFkiaJykPLAdWF/VZDVyVXs20GNgdEdsiYjuwRdK5ab/LqfDDiuanDwva3OuAMLPqUPYT5YYrIgYkXQPcDeSAWyJig6Sr0/UrgTXAO4Bu4ADw4YKP+HPg9jRcNhetG3VtLZNonlTrPQgzqxqZBQRARKwhCYHCtpUFywF89ATbrgc6s6xvOCQxb0YTT/fuq3QpZmajwndSD8P8tibvQZhZ1XBADMO8GU08v+sgh/qPVLoUM7PMOSCGYX5bMxHw7Eu+Yc7MJj4HxDAMXcnk8xBmVg0cEMPw6rZmJNi0fW+lSzEzy5wDYhga8jnmTm9yQJhZVXBADNO5Z7Ww6QUHhJlNfA6IYTp3VgvPvLTfVzKZ2YTngBimc2e1EAFPveAT1WY2sTkghuncWS0APLl9T4UrMTPLlgNimOZOb2JSbY1PVJvZhOeAGKZcjVhwVrNPVJvZhOeAOA3nnjWZjdv2ksw1aGY2MTkgTsPr5kzmxX2HeWHP4UqXYmaWGQfEaXhdeysAj/bsqmgdZmZZyjQgJC2RtElSt6TjnimdPknuhnT9o5IuLlqfk/SwpO9nWedwLZw9mVyNeOz53ZUuxcwsM5kFhKQccCOwFFgIXClpYVG3pcCC9LUCuKlo/ceBjVnVeLoa8jkWzGzm0R4HhJlNXFnuQSwCuiNic0T0AXcAy4r6LANui8T9QKuk2QCS2oF3AjdnWONpu6B9Co/27PKJajObsLIMiDnAloL3PWlbuX0+B3wKGDzZl0haIalLUldvb+8ZFTwcr2tv5eUD/fS8fHDUvtPMbDRlGRAq0Vb863bJPpLeBeyIiIdO9SURsSoiOiOis62t7XTqPC2vb58C4PMQZjZhZRkQPUBHwft2YGuZfd4MXCHpGZJDU5dJ+mp2pQ7fubNayNfWsO7ZlytdiplZJrIMiLXAAknzJOWB5cDqoj6rgavSq5kWA7sjYltEXB8R7RExN93upxHxgQxrHbZJtTkubG9lrQPCzCaozAIiIgaAa4C7Sa5E+kZEbJB0taSr025rgM1AN/Al4M+yqicLnXOnsuH53RzoG6h0KWZmI642yw+PiDUkIVDYtrJgOYCPnuIz7gXuzaC8M/aGedP4P/c+zfrndvGm18yodDlmZiPKd1KfgUvOmYoEDz6zs9KlmJmNOAfEGZhcX8d5syaz1gFhZhOQA+IMLZo7lXXP7qJv4KS3a5iZjTsOiDN06atncLD/CA8/56uZzGxicUCcoTe9Zjq5GvGLp0bvLm4zs9HggDhDk+vruKijlV8+9WKlSzEzG1EOiBHw1t9p47Hnd7Nzf1+lSzEzGzEOiBHwewtmEAG/6vZehJlNHA6IEXBBeyutjXX87MkdlS7FzGzEOCBGQK5GvO21Z/HjjS/4clczmzAcECNkye/OYu+hAe7b/FKlSzEzGxEOiBHylgUzaMrn+OHj2ypdipnZiHBAjJD6uhx/eN5MfrThBY4M+jGkZjb+OSBG0NLzZ/PS/j7ue9qHmcxs/HNAjKDLXzuTlvpavrWup9KlmJmdsUwDQtISSZskdUu6rsR6SbohXf+opIvT9g5JP5O0UdIGSR/Pss6RUl+X44rXn80PHt/GnkP9lS7HzOyMZBYQknLAjcBSYCFwpaSFRd2WAgvS1wrgprR9APhERLwWWAx8tMS2Y9L7LmnnUP8gax71yWozG9+y3INYBHRHxOaI6APuAJYV9VkG3BaJ+4FWSbPT51KvA4iIvSSPLJ2TYa0j5sKOVl4zs5k71m6pdClmZmcky4CYAxT+lOzh+B/yp+wjaS5wEfDAyJc48iTxgTe+ivVbdnkKcDMb17IMCJVoK77+86R9JDUD3wKujYg9Jb9EWiGpS1JXb+/YmHL7fZ0dtEyq5ZZfP1PpUszMTluWAdEDdBS8bwe2lttHUh1JONweEXed6EsiYlVEdEZEZ1tb24gUfqaaJ9WyfFEHax7bxtZdBytdjpnZackyINYCCyTNk5QHlgOri/qsBq5Kr2ZaDOyOiG2SBHwZ2BgRn82wxsx86E1zAVj1i82VLcTM7DRlFhARMQBcA9xNcpL5GxGxQdLVkq5Ou60BNgPdwJeAP0vb3wx8ELhM0vr09Y6sas1C+9RG3t/ZztceeI7nvRdhZuOQIibOtBCdnZ3R1dVV6TKOen7XQf7wf93Ley+Zw397zwWVLsfM7DiSHoqIzlLrfCd1hua0NvAnb3wV3+jq4Tcv7K10OWZmw+KAyNjHLl9AS30tf/2dx5lIe2tmNvE5IDI2rSnPp/7oPB747U6+s/75SpdjZlY2B8QoWP6GDi7saOVvv/cE23cfqnQ5ZmZlcUCMgpoa8dn3v57D/YN84pvrGfTzIsxsHHBAjJL5bc381ysW8uvul7jhp09Vuhwzs1NyQIyi93d28N6L2/ncj59i9SPFN5WbmY0ttZUuoJpI4h/fcz5bdh7gk998hGmNed6yYEalyzIzK8l7EKNsUm2OlR+8hPkzmvjIV9byy6fGxgSDZmbFHBAVMK0pz+1/+kbmzWjiI//SxV1+RKmZjUEOiAqZ3jyJO1Ys5pJzpvKX33iEf1yzkb6BwUqXZWZ2lAOiglob89z2kUV8cPE5rPrFZt594695cnvJx16YmY06B0SF1eVq+Pt3n8+Xrupkx95DvPOGX/E3332cl/YdrnRpZlblfBXTGPH2hWdxyTm/zz/f8xtuf+A57lr3PMvf0MGH3zKPOa0NlS7PzKqQp/seg7p37OULP+3me49uA+D3f6eNZReezdsXnkVj3pluZiPnZNN9OyDGsOd3HeT/3vcs313/PNt2H2JSbQ2L5k3j9xbM4NL5Mzh3Vgv5Wh8lNLPTV7GAkLQE+DyQA26OiP9etF7p+ncAB4B/HxHrytm2lIkWEEMGB4MHn9nJPU+8wC+f6uU3L+wDIJ+r4dxZLZw/ZzLzZzQzd0YTc6c30jGtkfq6XIWrNrPx4GQBkdnxCkk54Ebg7UAPsFbS6oh4oqDbUmBB+nojcBPwxjK3rRo1NWLx/Oksnj8dgO27D9H17E4ee343j/Xs5gePb2fXgf5jtpnSUMeM5jxtLZNoa6lnelOe5km1NNfX0jyplpb65NWYryVfW0M+V8Ok2ppkOX0/tFxXU4OU3AluZtUjywPai4DuiNgMIOkOYBlQ+EN+GXBbJLsx90tqlTQbmFvGtlVr1pR63nXB2bzrgrOPtu060MczLx3gmRf3s2XnAXr3HaZ372Fe3HeYx3p2sXN/H/sOD3AmE8nWCGqk5FWTLOckJMjVJO2SyNW80u9EClcds4xKtifrCrdRyXZUctHhZhPatMY837j60hH/3CwDYg6wpeB9D8lewqn6zClzWwAkrQBWALzqVa86s4rHsdbGPBc25rmwo/WEfSKCg/1H2HdogL2HB9h7aID9hwfoGxjk8MAg/UcG6RsYpG/oz3R54EhwJIKIYDCCI4PJZx0ZDAYDBtP24nVxzHcXLHPMm1KLxz1978SfVXqbY7aeOKfZzEpqqc/mR3mWAVHqV7bif6on6lPOtkljxCpgFSTnIIZTYLWRRGM+Oaw0s9LFmNmYl2VA9AAdBe/bgeI5rk/UJ1/GtmZmlqEsr5FcCyyQNE9SHlgOrC7qsxq4SonFwO6I2FbmtmZmlqHM9iAiYkDSNcDdJJeq3hIRGyRdna5fCawhucS1m+Qy1w+fbNusajUzs+P5Rjkzsyp2svsgfBuumZmV5IAwM7OSHBBmZlaSA8LMzEqaUCepJfUCz57m5jOAF0ewnErxOMaWiTIOmDhj8TiOdU5EtJVaMaEC4kxI6jrRmfzxxOMYWybKOGDijMXjKJ8PMZmZWUkOCDMzK8kB8YpVlS5ghHgcY8tEGQdMnLF4HGXyOQgzMyvJexBmZlaSA8LMzEqq+oCQtETSJkndkq6rdD3FJN0iaYekxwvapkm6R9JT6Z9TC9Zdn45lk6Q/Kmi/RNJj6bobNMrP4JTUIelnkjZK2iDp4+NxLJLqJT0o6ZF0HH87HsdRUENO0sOSvj/Ox/FMWsN6SV3jdSxKHrt8p6Qn038rl1Z0HJE+SrIaXyRTiT8NzCd5SNEjwMJK11VU41uBi4HHC9r+J3Bdunwd8D/S5YXpGCYB89Kx5dJ1DwKXkjyt7wfA0lEex2zg4nS5BfhNWu+4Gkv6nc3pch3wALB4vI2jYDx/CXwN+P54/buV1vAMMKOobdyNBfgK8Kfpch5oreQ4RvV/4lh7pf8B7y54fz1wfaXrKlHnXI4NiE3A7HR5NrCpVP0kz9O4NO3zZEH7lcAXKzym7wJvH89jARqBdSTPSx934yB5UuNPgMt4JSDG3TjS732G4wNiXI0FmAz8lvTiobEwjmo/xDQH2FLwvidtG+vOiuTJe6R/Dj1i+kTjmZMuF7dXhKS5wEUkv32Pu7Gkh2XWAzuAeyJiXI4D+BzwKWCwoG08jgOSZ9b/SNJDklakbeNtLPOBXuDW9LDfzZKaqOA4qj0gSh2XG8/X/Z5oPGNmnJKagW8B10bEnpN1LdE2JsYSEUci4kKS38AXSTr/JN3H5DgkvQvYEREPlbtJibaKj6PAmyPiYmAp8FFJbz1J37E6llqSw8k3RcRFwH6SQ0onkvk4qj0geoCOgvftwNYK1TIcL0iaDZD+uSNtP9F4etLl4vZRJamOJBxuj4i70uZxORaAiNgF3AssYfyN483AFZKeAe4ALpP0VcbfOACIiK3pnzuAbwOLGH9j6QF60j1SgDtJAqNi46j2gFgLLJA0T1IeWA6srnBN5VgNfChd/hDJ8fyh9uWSJkmaBywAHkx3S/dKWpxezXBVwTajIv3eLwMbI+KzBavG1VgktUlqTZcbgLcBT463cUTE9RHRHhFzSf7e/zQiPjDexgEgqUlSy9Ay8G+AxxlnY4mI7cAWSeemTZcDT1R0HKN9MmmsvYB3kFxR8zTwV5Wup0R9Xwe2Af0kvxl8BJhOcnLxqfTPaQX9/yodyyYKrlwAOkn+0TwNfIGiE2GjMI63kOzmPgqsT1/vGG9jAS4AHk7H8TjwN2n7uBpH0Zj+gFdOUo+7cZAcu38kfW0Y+nc8TsdyIdCV/v36DjC1kuPwVBtmZlZStR9iMjOzE3BAmJlZSQ4IMzMryQFhZmYlOSDMzKwkB4TZGCDpD4ZmVDUbKxwQZmZWkgPCbBgkfUDJ8yDWS/piOnHfPkn/JGmdpJ9Iakv7XijpfkmPSvr20Dz+kl4j6cdKnimxTtKr049vLngWwO2j/SwCs2IOCLMySXot8G9JJoa7EDgC/DugCVgXyWRxPwc+nW5yG/CfIuIC4LGC9tuBGyPi9cCbSO6Uh2SG22tJ5vmfTzJfklnF1Fa6ALNx5HLgEmBt+st9A8nEaYPA/0v7fBW4S9IUoDUifp62fwX4Zjpn0JyI+DZARBwCSD/vwYjoSd+vJ3kOyK8yH5XZCTggzMon4CsRcf0xjdJfF/U72fw1JztsdLhg+Qj+92kV5kNMZuX7CfA+STPh6DOPzyH5d/S+tM+fAL+KiN3Ay5J+L23/IPDzSJ6B0SPp3elnTJLUOJqDMCuXf0MxK1NEPCHpv5A8uayGZIbdj5I82OV3JT0E7CY5TwHJ1Mwr0wDYDHw4bf8g8EVJf5d+xh+P4jDMyubZXM3OkKR9EdFc6TrMRpoPMZmZWUnegzAzs5K8B2FmZiU5IMzMrCQHhJmZleSAMDOzkhwQZmZW0v8H13hCDDLvHH4AAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "def function(x, w, b):         #数学模型\n",
    "    return x * w + b \n",
    "\n",
    "def cost(x, y, w, b):          #损失函数\n",
    "    cost = 0.0\n",
    "    for x_val, y_val in zip(x,y):\n",
    "        y_pred = function(x_val, w, b)\n",
    "        cost += (y_pred - y_val) ** 2.0\n",
    "    return cost/len(x)\n",
    "\n",
    "def gradient_w(x, y, w,b):      #梯度函数\n",
    "    grad = 0.0\n",
    "    for x_val, y_val in zip(x, y):\n",
    "        grad += x_val*(x_val*w+b-y_val)\n",
    "    return (2*grad)/len(x)\n",
    "\n",
    "def gradient_b(x, y, w, b): #\n",
    "    grad = 0.0\n",
    "    for x_val, y_val in zip(x, y):\n",
    "        grad += x_val*w+b-y_val\n",
    "    return (2*grad)/len(x)\n",
    "\n",
    "x_data = [1.0, 2.0, 3.0]\n",
    "y_data = [2.0, 4.0, 6.0]\n",
    "\n",
    "alpha = 0.01\n",
    "w = 1.5\n",
    "b = 1.0\n",
    "\n",
    "epoch_list = []\n",
    "cost_list = []\n",
    "epoch = 0\n",
    "\n",
    "last_cost = cost(x_data, y_data, w, b)\n",
    "while True:\n",
    "    grad_val_w = gradient_w(x_data, y_data, w, b)\n",
    "    w -= alpha*grad_val_w\n",
    "    grad_val_b = gradient_b(x_data, y_data, w, b)\n",
    "    b -= alpha*grad_val_b\n",
    "    this_cost = cost(x_data, y_data, w, b)\n",
    "    # print('epoch:', epoch, 'w=', w, 'loss=', this_cost)\n",
    "    epoch_list.append(epoch)\n",
    "    cost_list.append(this_cost)\n",
    "    epoch += 1\n",
    "    if((abs(last_cost) - abs(this_cost)) < 1e-16):\n",
    "        break\n",
    "    last_cost = this_cost\n",
    "    \n",
    "print(\"w = \",\"%.1f\"%w,\"b = \",\"%.1f\"%b,\"epoch = \",epoch)\n",
    "print(\"After train forward(4) = \",function(4, w, b))\n",
    "plt.plot(epoch_list,cost_list)\n",
    "plt.ylabel('cost')\n",
    "plt.xlabel('epoch')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 随机梯度下降"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "predict (before training) 4 4.0\n",
      "predict (after training) 4 7.9999999999996945\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAEGCAYAAABvtY4XAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAVHUlEQVR4nO3de4zdZ33n8ff3zDmeM3Y8di7T4NgODpSltLRxwpQ2m23LEpYFihoqLgm7oaiLZG1L1VChso1otWq1f6y0uwgqdVmsQJsuWS6FpEUR16YlNBIkGQcn5EK45AJOHDwh+JLEHo9nvvvH+Z3xsbGTie3fHM/ze7+kkc85c+Y8zyPbHz/+/p7f80RmIkkqT2vYHZAk1cOAl6RCGfCSVCgDXpIKZcBLUqHaw+7AoHPOOSc3bdo07G5I0rKxbdu2JzJz4ljfO60CftOmTUxNTQ27G5K0bETEI8f7niUaSSqUAS9Jhaq1RBMRDwP7gDngUGZO1tmeJOmwpajB/9vMfGIJ2pEkDbBEI0mFqjvgE/hyRGyLiC3HekNEbImIqYiYmp6errk7ktQcdQf8pZl5MfB64N0R8etHvyEzt2bmZGZOTkwccymnJOkE1BrwmflY9esu4EbglXW085c3f5dbvuPsX5IG1RbwEbEqIlb3HwOvBe6po62P3PJ9/sWAl6Qj1LmK5lzgxojot/P/MvOLdTQ02hnhwKG5Oj5akpat2gI+Mx8ELqzr8wd12y0OzM4vRVOStGwUsUyy2xlh5pABL0mDigj40c4IB2Yt0UjSoCICvttpGfCSdJQiAn603WLGGrwkHaGIgO/V4J3BS9KgMgK+PeIqGkk6ShkB32m5Dl6SjlJEwI+2XUUjSUcrIuB7q2gs0UjSoEIC3ousknS0IgK+d6PTPJk57K5I0mmjiIDvdnrDcLsCSTqsiIAfbY8AeLOTJA0oIuD7M3iXSkrSYWUEvDN4SfopZQR8pxfwzuAl6bAiAn60XZVovNlJkhYUEfALM3hLNJK0oJCA7y+TdAYvSX2FBLwzeEk6WhEBbw1ekn5aEQF/eAZvwEtSXxEBP7pwo5MlGknqKyLg+zP4GWfwkrSgjIDv38nqDF6SFhQR8J2RIMIavCQNKiLgI6I6eNuAl6S+IgIePLZPko5WUMB7bJ8kDSom4EfbzuAlaVAxAd/tWIOXpEG1B3xEjETENyPipjrbGe2MeKOTJA1Yihn81cD9dTfSbbe80UmSBtQa8BGxAfhN4No624GqROMMXpIW1D2D/yDwPuC4yRsRWyJiKiKmpqenT7ihUWfwknSE2gI+It4I7MrMbc/2vszcmpmTmTk5MTFxwu15kVWSjlTnDP5S4Lci4mHgk8CrI+LjdTXmjU6SdKTaAj4zr8nMDZm5CbgS+KfMvKqu9rzRSZKOVMw6eG90kqQjtZeikcz8KvDVOtvoraKZIzOJiDqbkqRloZgZfLczQiYcnHMWL0lQUMAfPnjbgJckKCjgF47t80KrJAEFBXx/Bj/jDF6SgIICvj+D92YnSeopMOCdwUsSFBXwVYnGGrwkAQUF/GjbGbwkDSom4PszeGvwktRTUMBXM3hLNJIElBTwlmgk6QjFBPyoF1kl6QjFBLwzeEk6UjEBP+pFVkk6QjkB324RgeeySlKlmICPiN6hH4cs0UgSFBTw0LvZyRm8JPUUFfAevC1JhxUW8CPe6CRJlbICvj3iKhpJqhQV8KOdFjNeZJUkoLCAdwYvSYcVFfCjXmSVpAVFBXy34wxekvqKC3hr8JLUU1TAj7Zb3ugkSZWiAr7bcasCSeorK+BdRSNJC8oK+Ooia2YOuyuSNHRFBfxou8V8wuycAS9JRQV8/+Btj+2TpBoDPiK6EXF7RNwVEfdGxJ/X1VZfd+FUJy+0SlK7xs+eAV6dmU9FRAe4NSK+kJnfqKvB0U7/XFZn8JJUW8Bn70rnU9XTTvVVa3F8tN2bwVuikaSaa/ARMRIR24FdwFcy87ZjvGdLRExFxNT09PRJtdddmMFbopGkWgM+M+cyczOwAXhlRLz8GO/ZmpmTmTk5MTFxUu15kVWSDluSVTSZuRv4KvC6Otvptr3IKkl9da6imYiItdXjMeA1wLfrag8GSzTO4CWpzlU064DrImKE3j8kn87Mm2psj1GXSUrSgjpX0dwNXFTX5x9Lt20NXpL6iryT1Rm8JBUX8P0SjTN4SSoq4EerEs0BSzSSVFrAe5FVkvqKCvhWK1jRbnmRVZIoLOChd7PTjDN4SSow4Dse2ydJUGDAnzHa5qmZQ8PuhiQNXXEBv3qsw94DBrwkFRfw4902e/fPDrsbkjR0iwr4iLg6Isaj56MRcWdEvLbuzp2I8bEOew8Y8JK02Bn8f8rMvcBrgQngd4H/XluvTsJ4t8Pe/ZZoJGmxAR/Vr28A/joz7xp47bQyPtZ2Bi9JLD7gt0XEl+kF/JciYjVwWi42H+92OHho3qWSkhpvsdsFvwvYDDyYmc9ExFn0yjSnnfGxDgB7988u7C4pSU202Bn8JcADmbk7Iq4C/hTYU1+3TtyafsBbppHUcIsN+A8Dz0TEhcD7gEeAv62tVydhvNv7T8keL7RKarjFBvyhzEzgcuBDmfkhYHV93Tpx487gJQlYfA1+X0RcA7wD+LXqnNVOfd06cePdwzV4SWqyxc7grwBm6K2HfxxYD/yP2np1EsbHev9muV2BpKZbVMBXoX49sCYi3ggcyMzTtAbvDF6SYPFbFbwNuB14K/A24LaIeEudHTtR3c4IK9ota/CSGm+xNfj3A7+cmbsAImIC+EfgM3V17GS4XYEkLb4G3+qHe+XHz+Nnl5zbFUjS4mfwX4yILwGfqJ5fAXy+ni6dvN4M3oCX1GyLCvjM/OOIeDNwKb1NxrZm5o219uwkjI912GPAS2q4xc7gyczPAp+tsS+nzHi3zY4nnxl2NyRpqJ414CNiH5DH+haQmTleS69Okod+SNJzBHxmnpbbETyXNWO9VTSZScRpuW29JNXutF0JczLGux0Ozs0zc+i03LJekpZEmQHf367AC62SGqy2gI+IjRHxzxFxf0TcGxFX19XW0frbFbiSRlKTLXoVzQk4BLw3M++sjvjbFhFfycz7amwTcMtgSYIaZ/CZuTMz76we7wPup7cLZe36h364XYGkJluSGnxEbAIuAm5bivacwUvSEgR8RJxB7wap92Tm3mN8f0tETEXE1PT09Clp0y2DJanmgI+IDr1wvz4zbzjWezJza2ZOZubkxMTEKWl3dddDPySpzlU0AXwUuD8zP1BXO8fS7Yww2m45g5fUaHXO4C+ld4brqyNie/X1hhrbO4LbFUhqutqWSWbmrfT2rBmK8W7bVTSSGq3IO1nBGbwkFRvwvQ3HDHhJzVVswI93O66ikdRo5Qb8WNsZvKRGKzfgu70afOaxziuRpPKVG/BjHWbnkgOz7gkvqZnKDfiu+9FIarZyA95DPyQ1XLkB7wxeUsOVG/D9LYO9m1VSQ5Ub8NWOkh7bJ6mpyg14D/2Q1HDFBvzCnvDO4CU1VLEBP9oeodtpuV2BpMYqNuDBDcckNVvRAT/e7XiRVVJjFR3w55wxyq59M8PuhiQNRdEBv25tl5279w+7G5I0FEUH/Pq1Y/xo3wyH5txwTFLzFB3w69aMMTeflmkkNVLRAX/e2i4AO/dYppHUPIUH/BgAj+4+MOSeSNLSKzrg162pZvBeaJXUQEUH/Opuh9XdNjv3OIOX1DxFBzzAeWvGeNQZvKQGKj7g163tepFVUiMVH/DnrR3jMS+ySmqg8gN+TZcnnz7Igdm5YXdFkpZU8QG/bk1vqaQXWiU1TfEB318L/5gXWiU1TAMCvrcW3oCX1DTFB/wL1vQD3hKNpGapLeAj4mMRsSsi7qmrjcUYbY9wzhmjLpWU1Dh1zuD/BnhdjZ+/aOet7Xqzk6TGqS3gM/NrwJN1ff7zcd6aMVfRSGqcodfgI2JLRExFxNT09HQtbfRPdsrMWj5fkk5HQw/4zNyamZOZOTkxMVFLG+etGePpg3Ps3X+ols+XpNPR0AN+KSyshfdCq6QGaUTAr3MtvKQGqnOZ5CeArwMvjYgdEfGuutp6LusXZvBeaJXUHO26Pjgz317XZz9f55wxSrsVnuwkqVEaUaIZaQXnjnct0UhqlEYEPPTKNDt+YsBLao7GBPzL1q3mvp17mZt3LbykZmhMwF+4cS3PHJzje7ueGnZXJGlJNCrgAe764e6h9kOSlkpjAv6Cs1exuttm+47dw+6KJC2JxgR8qxVcuGEtdxvwkhqiMQEP8Esb1vDtnfs8gFtSIzQq4C/cuJZD88m9j+0ddlckqXaNCvjNXmiV1CCNCvhzx7u8YLzLXdbhJTVAowIeenX4u3fsGXY3JKl2jQv4Czeu5aEnnmb3MweH3RVJqlXjAr5fh3cWL6l0jQv4X9ywBvBCq6TyNS7gx7sdXjyxygutkorXuIAHmHzhWXzjwSfZf9AbniSVq5EB/9sXr+epmUN84Z6dw+6KJNWmkQH/KxecxaazV/KpO3447K5IUm0aGfARwVsnN3LbQ0/y8BNPD7s7klSLRgY8wJsv3kAr4O+2OYuXVKbGBvwL1nT5jX81wWe27fAYP0lFamzAA7xtciM/2jvD1747PeyuSNIp1+iAv+xl53LWqhV82outkgrU6IBf0W7xtsmNfPHex/n693887O5I0inV6IAH+MPLfpZNZ6/ivZ/ezp79s8PujiSdMo0P+JUr2nzwis38aN8Mf/b39wy7O5J0yjQ+4KG3hfB7LnsJn7vrMf5h+6PD7o4knRIGfOX3XvViLj5/Ldfc8C0+/y23MJC0/BnwlfZIiw9f9Qpe+oLV/P71d/LfbrqP2bn5YXdLkk6YAT/g3PEun9pyCb9zyQu59taHuHLrN7jlO9PMeyOUpGWo1oCPiNdFxAMR8b2I+JM62zpVVrRb/MXlL+eDV2zmkR8/zTs/djuv+cAtXPsvD/LA4/sMe0nLRmTWE1gRMQJ8B/h3wA7gDuDtmXnf8X5mcnIyp6amaunPiZg5NMcXvvU41339Yb75g90AjHfbbD7/TF50zio2nDnGhjPHOHPlCtauXMGasQ4rR0cY64zQGfE/R5LqFxHbMnPyWN9r19juK4HvZeaDVSc+CVwOHDfgTzej7RHedNF63nTRen7w42e44+EnmXrkSbb/cA93PvITnpo5dNyfHWkFnZGg02rRabdoRTDSgpEIIoJWC1oRRPX+GHjcfxADnxcx+Oy5Pb93SxqmM1eu4NP/+ZJT/rl1Bvx6YHAPgB3Arxz9pojYAmwBOP/882vszsk5/+yVnH/2St78ig0AZCZ79s/y6O797H5mtve1/yD7D85xYHaO/bNzzM4ls3PzzM7NMzcP8/PJXCbzmZAwV/3vqXq68Llw+PlPP3lu+Xx/QNJQjXc7tXxunQF/rEnkTyVPZm4FtkKvRFNjf06piGBtVZqRpNNRnYXiHcDGgecbgMdqbE+SNKDOgL8DeElEXBARK4Argc/V2J4kaUBtJZrMPBQRfwB8CRgBPpaZ99bVniTpSHXW4MnMzwOfr7MNSdKxuVhbkgplwEtSoQx4SSqUAS9JhaptL5oTERHTwCMn+OPnAE+cwu4sB00cMzRz3E0cMzRz3M93zC/MzIljfeO0CviTERFTx9twp1RNHDM0c9xNHDM0c9yncsyWaCSpUAa8JBWqpIDfOuwODEETxwzNHHcTxwzNHPcpG3MxNXhJ0pFKmsFLkgYY8JJUqGUf8MvxYO8TEREbI+KfI+L+iLg3Iq6uXj8rIr4SEd+tfj1z2H091SJiJCK+GRE3Vc+bMOa1EfGZiPh29Xt+Senjjog/qv5s3xMRn4iIboljjoiPRcSuiLhn4LXjjjMirqny7YGI+PfPp61lHfDVwd5/Bbwe+Hng7RHx88PtVW0OAe/NzJcBvwq8uxrrnwA3Z+ZLgJur56W5Grh/4HkTxvwh4IuZ+XPAhfTGX+y4I2I98IfAZGa+nN4W41dS5pj/BnjdUa8dc5zV3/ErgV+ofuZ/V7m3KMs64Bk42DszDwL9g72Lk5k7M/PO6vE+en/h19Mb73XV264D3jSUDtYkIjYAvwlcO/By6WMeB34d+ChAZh7MzN0UPm5625ePRUQbWEnvBLjixpyZXwOePOrl443zcuCTmTmTmQ8B36OXe4uy3AP+WAd7rx9SX5ZMRGwCLgJuA87NzJ3Q+0cA+Jkhdq0OHwTeB8wPvFb6mF8ETAN/XZWmro2IVRQ87sx8FPifwA+AncCezPwyBY/5KMcb50ll3HIP+EUd7F2SiDgD+CzwnszcO+z+1Cki3gjsysxtw+7LEmsDFwMfzsyLgKcpozRxXFXN+XLgAuA8YFVEXDXcXp0WTirjlnvAN+pg74jo0Av36zPzhurlH0XEuur764Bdw+pfDS4FfisiHqZXfnt1RHycsscMvT/XOzLztur5Z+gFfsnjfg3wUGZOZ+YscAPwryl7zIOON86TyrjlHvCNOdg7IoJeTfb+zPzAwLc+B7yzevxO4B+Wum91ycxrMnNDZm6i93v7T5l5FQWPGSAzHwd+GBEvrV66DLiPssf9A+BXI2Jl9Wf9MnrXmUoe86DjjfNzwJURMRoRFwAvAW5f9Kdm5rL+At4AfAf4PvD+YfenxnH+G3r/Nbsb2F59vQE4m95V9+9Wv5417L7WNP5XATdVj4sfM7AZmKp+v/8eOLP0cQN/DnwbuAf4v8BoiWMGPkHvOsMsvRn6u55tnMD7q3x7AHj982nLrQokqVDLvUQjSToOA16SCmXAS1KhDHhJKpQBL0mFMuClUyAiXtXf7VI6XRjwklQoA16NEhFXRcTtEbE9Ij5S7TX/VET8r4i4MyJujoiJ6r2bI+IbEXF3RNzY36M7In42Iv4xIu6qfubF1cefMbCH+/XVHZnS0BjwaoyIeBlwBXBpZm4G5oD/CKwC7szMi4FbgP9a/cjfAv8lM38J+NbA69cDf5WZF9LbL2Vn9fpFwHvonU3wInp76UhD0x52B6QldBnwCuCOanI9Rm9Tp3ngU9V7Pg7cEBFrgLWZeUv1+nXA30XEamB9Zt4IkJkHAKrPuz0zd1TPtwObgFtrH5V0HAa8miSA6zLzmiNejPizo973bPt3PFvZZWbg8Rz+/dKQWaJRk9wMvCUifgYWzsF8Ib2/B2+p3vMfgFszcw/wk4j4ter1dwC3ZG8P/h0R8abqM0YjYuVSDkJaLGcYaozMvC8i/hT4ckS06O3m9256B2r8QkRsA/bQq9NDb9vW/1MF+IPA71avvwP4SET8RfUZb13CYUiL5m6SaryIeCozzxh2P6RTzRKNJBXKGbwkFcoZvCQVyoCXpEIZ8JJUKANekgplwEtSof4/QRxwCjhfgOcAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    " \n",
    "x_data = [1.0, 2.0, 3.0]\n",
    "y_data = [2.0, 4.0, 6.0]\n",
    " \n",
    "w = 1.0\n",
    " \n",
    "def forward(x):\n",
    "    return x*w\n",
    " \n",
    "# calculate loss function\n",
    "def loss(x, y):\n",
    "    y_pred = forward(x)\n",
    "    return (y_pred - y)**2\n",
    " \n",
    "# define the gradient function  sgd\n",
    "def gradient(x, y):\n",
    "    return 2*x*(x*w - y)\n",
    " \n",
    "epoch_list = []\n",
    "loss_list = []\n",
    "print('predict (before training)', 4, forward(4))\n",
    "for epoch in range(100):\n",
    "    for x,y in zip(x_data, y_data):\n",
    "        grad = gradient(x,y)\n",
    "        w = w - 0.01*grad    # update weight by every grad of sample of training set\n",
    "        # print(\"\\tgrad:\", x, y,grad)\n",
    "        l = loss(x,y)\n",
    "    # print(\"progress:\",epoch,\"w=\",w,\"loss=\",l)\n",
    "    epoch_list.append(epoch)\n",
    "    loss_list.append(l)\n",
    " \n",
    "print('predict (after training)', 4, forward(4))\n",
    "plt.plot(epoch_list,loss_list)\n",
    "plt.ylabel('loss')\n",
    "plt.xlabel('epoch')\n",
    "plt.show() "
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "44143c5ac5f3ceb8e37c69c3af73325ae55d21292b2c7b54871fd886482dde4c"
  },
  "kernelspec": {
   "display_name": "Python 3.6.13 ('pytorch')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.13"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}