homework/Day8_homework_fakenews_imagenet-vgg16.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 作业1：Fake News Detection\n",
    "\n",
    "Develop a machine learning program to identify when an article might be fake news. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "\n",
    "- train.csv: A full training dataset with the following attributes:\n",
    "    - id: unique id for a news article\n",
    "    - title: the title of a news article\n",
    "    - author: author of the news article\n",
    "    - text: the text of the article; could be incomplete\n",
    "    - label: a label that marks the article as potentially unreliable\n",
    "        - 1: unreliable\n",
    "        - 0: reliable\n",
    "- test.csv: A testing training dataset with all the same attributes at train.csv without the label.\n",
    "\n",
    "- submit.csv: A sample submission that you can"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-13T05:33:07.402868Z",
     "start_time": "2020-08-13T05:33:07.395862Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/Users/datalab/bigdata/css_homework/Day7_kaggle_fakenews\n"
     ]
    }
   ],
   "source": [
    "cd 'Day7_kaggle_fakenews'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-13T05:33:12.355323Z",
     "start_time": "2020-08-13T05:33:12.228628Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[31mtest.csv\u001b[m\u001b[m*  \u001b[31mtrain.csv\u001b[m\u001b[m*\r\n"
     ]
    }
   ],
   "source": [
    "ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-13T05:33:17.473916Z",
     "start_time": "2020-08-13T05:33:16.209487Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'0.22.1'"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import sklearn\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "sklearn.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-13T05:33:23.950849Z",
     "start_time": "2020-08-13T05:33:23.228395Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>title</th>\n",
       "      <th>author</th>\n",
       "      <th>text</th>\n",
       "      <th>label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>House Dem Aide: We Didn’t Even See Comey’s Let...</td>\n",
       "      <td>Darrell Lucus</td>\n",
       "      <td>House Dem Aide: We Didn’t Even See Comey’s Let...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>FLYNN: Hillary Clinton, Big Woman on Campus - ...</td>\n",
       "      <td>Daniel J. Flynn</td>\n",
       "      <td>Ever get the feeling your life circles the rou...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>Why the Truth Might Get You Fired</td>\n",
       "      <td>Consortiumnews.com</td>\n",
       "      <td>Why the Truth Might Get You Fired October 29, ...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>15 Civilians Killed In Single US Airstrike Hav...</td>\n",
       "      <td>Jessica Purkiss</td>\n",
       "      <td>Videos 15 Civilians Killed In Single US Airstr...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>Iranian woman jailed for fictional unpublished...</td>\n",
       "      <td>Howard Portnoy</td>\n",
       "      <td>Print \\nAn Iranian woman has been sentenced to...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id                                              title              author  \\\n",
       "0   0  House Dem Aide: We Didn’t Even See Comey’s Let...       Darrell Lucus   \n",
       "1   1  FLYNN: Hillary Clinton, Big Woman on Campus - ...     Daniel J. Flynn   \n",
       "2   2                  Why the Truth Might Get You Fired  Consortiumnews.com   \n",
       "3   3  15 Civilians Killed In Single US Airstrike Hav...     Jessica Purkiss   \n",
       "4   4  Iranian woman jailed for fictional unpublished...      Howard Portnoy   \n",
       "\n",
       "                                                text  label  \n",
       "0  House Dem Aide: We Didn’t Even See Comey’s Let...      1  \n",
       "1  Ever get the feeling your life circles the rou...      0  \n",
       "2  Why the Truth Might Get You Fired October 29, ...      1  \n",
       "3  Videos 15 Civilians Killed In Single US Airstr...      1  \n",
       "4  Print \\nAn Iranian woman has been sentenced to...      1  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('train.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-13T05:33:37.424371Z",
     "start_time": "2020-08-13T05:33:37.419187Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'House Dem Aide: We Didn’t Even See Comey’s Letter Until Jason Chaffetz Tweeted It'"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.iloc[0]['title']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-13T05:33:38.999504Z",
     "start_time": "2020-08-13T05:33:38.995505Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(20800, 5)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-13T05:33:40.801837Z",
     "start_time": "2020-08-13T05:33:40.796256Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0   -- >  House Dem Aide: We Didn’t Even See Comey’s Letter Until Jason Chaffetz Tweeted It\n",
      "1   -- >  FLYNN: Hillary Clinton, Big Woman on Campus - Breitbart\n",
      "2   -- >  Why the Truth Might Get You Fired\n",
      "3   -- >  15 Civilians Killed In Single US Airstrike Have Been Identified\n",
      "4   -- >  Iranian woman jailed for fictional unpublished story about woman stoned to death for adultery\n"
     ]
    }
   ],
   "source": [
    "for k, i in enumerate(df['title'][:5]):\n",
    "    print(k,'  -- > ',  i)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-13T05:33:42.112911Z",
     "start_time": "2020-08-13T05:33:42.108036Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    1\n",
       "1    0\n",
       "2    1\n",
       "3    1\n",
       "4    1\n",
       "Name: label, dtype: int64"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['label'][:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-13T05:33:43.178498Z",
     "start_time": "2020-08-13T05:33:43.171460Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'House Dem Aide: We Didn’t Even See Comey’s Letter Until Jason Chaffetz Tweeted It By Darrell Lucus on October 30, 2016 Subscribe Jason Chaffetz on the stump in American Fork, Utah ( image courtesy Michael Jolley, available under a Creative Commons-BY license) \\nWith apologies to Keith Olbermann, there is no doubt who the Worst Person in The World is this week–FBI Director James Comey. But according to a House Democratic aide, it looks like we also know who the second-worst person is as well. It turns out that when Comey sent his now-infamous letter announcing that the FBI was looking into emails that may be related to Hillary Clinton’s email server, the ranking Democrats on the relevant committees didn’t hear about it from Comey. They found out via a tweet from one of the Republican committee chairmen. \\nAs we now know, Comey notified the Republican chairmen and Democratic ranking members of the House Intelligence, Judiciary, and Oversight committees that his agency was reviewing emails it had recently discovered in order to see if they contained classified information. Not long after this letter went out, Oversight Committee Chairman Jason Chaffetz set the political world ablaze with this tweet. FBI Dir just informed me, \"The FBI has learned of the existence of emails that appear to be pertinent to the investigation.\" Case reopened \\n— Jason Chaffetz (@jasoninthehouse) October 28, 2016 \\nOf course, we now know that this was not the case . Comey was actually saying that it was reviewing the emails in light of “an unrelated case”–which we now know to be Anthony Weiner’s sexting with a teenager. But apparently such little things as facts didn’t matter to Chaffetz. The Utah Republican had already vowed to initiate a raft of investigations if Hillary wins–at least two years’ worth, and possibly an entire term’s worth of them. Apparently Chaffetz thought the FBI was already doing his work for him–resulting in a tweet that briefly roiled the nation before cooler heads realized it was a dud. \\nBut according to a senior House Democratic aide, misreading that letter may have been the least of Chaffetz’ sins. That aide told Shareblue that his boss and other Democrats didn’t even know about Comey’s letter at the time–and only found out when they checked Twitter. “Democratic Ranking Members on the relevant committees didn’t receive Comey’s letter until after the Republican Chairmen. In fact, the Democratic Ranking Members didn’ receive it until after the Chairman of the Oversight and Government Reform Committee, Jason Chaffetz, tweeted it out and made it public.” \\nSo let’s see if we’ve got this right. The FBI director tells Chaffetz and other GOP committee chairmen about a major development in a potentially politically explosive investigation, and neither Chaffetz nor his other colleagues had the courtesy to let their Democratic counterparts know about it. Instead, according to this aide, he made them find out about it on Twitter. \\nThere has already been talk on Daily Kos that Comey himself provided advance notice of this letter to Chaffetz and other Republicans, giving them time to turn on the spin machine. That may make for good theater, but there is nothing so far that even suggests this is the case. After all, there is nothing so far that suggests that Comey was anything other than grossly incompetent and tone-deaf. \\nWhat it does suggest, however, is that Chaffetz is acting in a way that makes Dan Burton and Darrell Issa look like models of responsibility and bipartisanship. He didn’t even have the decency to notify ranking member Elijah Cummings about something this explosive. If that doesn’t trample on basic standards of fairness, I don’t know what does. \\nGranted, it’s not likely that Chaffetz will have to answer for this. He sits in a ridiculously Republican district anchored in Provo and Orem; it has a Cook Partisan Voting Index of R+25, and gave Mitt Romney a punishing 78 percent of the vote in 2012. Moreover, the Republican House leadership has given its full support to Chaffetz’ planned fishing expedition. But that doesn’t mean we can’t turn the hot lights on him. After all, he is a textbook example of what the House has become under Republican control. And he is also the Second Worst Person in the World. About Darrell Lucus \\nDarrell is a 30-something graduate of the University of North Carolina who considers himself a journalist of the old school. An attempt to turn him into a member of the religious right in college only succeeded in turning him into the religious right\\'s worst nightmare--a charismatic Christian who is an unapologetic liberal. His desire to stand up for those who have been scared into silence only increased when he survived an abusive three-year marriage. You may know him on Daily Kos as Christian Dem in NC . Follow him on Twitter @DarrellLucus or connect with him on Facebook . Click here to buy Darrell a Mello Yello. Connect'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['text'][0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Machine Learning\n",
    "\n",
    "### Feature Extraction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-13T05:34:09.700420Z",
     "start_time": "2020-08-13T05:33:58.746429Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import TfidfTransformer, CountVectorizer\n",
    "\n",
    "df=df.fillna(' ')\n",
    "df['total']=df['title']+' '+df['author']+df['text']\n",
    "transformer = TfidfTransformer(smooth_idf=False)\n",
    "count_vectorizer = CountVectorizer(ngram_range=(1,1), \n",
    "                                   stop_words = 'english',\n",
    "                                  max_features = 10000)\n",
    "counts = count_vectorizer.fit_transform(df['total'].values)\n",
    "tfidf = transformer.fit_transform(counts)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-13T05:34:20.675138Z",
     "start_time": "2020-08-13T05:34:20.671308Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10000"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "diction  = count_vectorizer.get_feature_names()\n",
    "\n",
    "len(diction)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-13T05:34:39.739233Z",
     "start_time": "2020-08-13T05:34:37.736992Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[array([0., 0., 0., ..., 0., 0., 0.]),\n",
       " array([0., 0., 0., ..., 0., 0., 0.]),\n",
       " array([0., 0., 0., ..., 0., 0., 0.])]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tfidf = [i.toarray()[0] for i in tfidf]\n",
    "tfidf[:3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-13T05:34:42.466267Z",
     "start_time": "2020-08-13T05:34:42.463667Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "y = df['label']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-13T05:34:46.792366Z",
     "start_time": "2020-08-13T05:34:46.759095Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "Xtrain, Xtest, ytrain, ytest = train_test_split(tfidf, y, random_state=1, train_size = 0.8)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Neural Network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-13T05:35:45.818587Z",
     "start_time": "2020-08-13T05:35:45.815249Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "#打乱所有的数据顺序，形成数据集\n",
    "# indices为所有数据下标的一个全排列\n",
    "indices = np.random.permutation(len(tfidf))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-13T05:35:47.144133Z",
     "start_time": "2020-08-13T05:35:46.884307Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "#重新根据打乱的下标生成数据集dataset，标签集labels，以及对应的原始句子sentences\n",
    "dataset = [tfidf[i] for i in indices]\n",
    "labels = [y[i] for i in indices]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-13T05:37:07.583526Z",
     "start_time": "2020-08-13T05:37:07.578929Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "#对整个数据集进行划分，分为：训练集、校准集和测试集，\n",
    "# 其中校准和测试集合的长度都是整个数据集的10分之一\n",
    "test_size = len(dataset) // 10\n",
    "\n",
    "train_data = dataset[2 * test_size :]\n",
    "train_label = labels[2 * test_size :]\n",
    "\n",
    "valid_data = dataset[: test_size]\n",
    "valid_label = labels[: test_size]\n",
    "\n",
    "test_data = dataset[test_size : 2 * test_size]\n",
    "test_label = labels[test_size : 2 * test_size]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-13T05:37:10.606071Z",
     "start_time": "2020-08-13T05:37:10.194229Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "#PyTorch用的包\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.optim\n",
    "from torch.autograd import Variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-13T05:37:29.997983Z",
     "start_time": "2020-08-13T05:37:29.991739Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# 一个简单的前馈神经网络，三层，第一层线性层，加一个非线性ReLU，第二层线性层，中间有30个隐含层神经元\n",
    "# 输入维度为词典的大小：每一段评论的词袋模型\n",
    "model = nn.Sequential(\n",
    "    nn.Linear(), # change here!! # 第一层线性层\n",
    "    nn.ReLU(), # 加一个非线性ReLU\n",
    "    nn.Linear( ), #change here!! # 第二层线性层\n",
    "    nn.LogSoftmax(dim = 1),\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-13T05:40:33.837668Z",
     "start_time": "2020-08-13T05:40:33.833965Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "def accuracy(predictions, labels):\n",
    "    \"\"\"计算预测错误率的函数，其中predictions是模型给出的一组预测结果，\n",
    "    batch_size行num_classes列的矩阵，labels是数据之中的正确答案\"\"\"\n",
    "    pred = torch.max(predictions.data, 1)[1] # 对于任意一行（一个样本）的输出值的第1个维度，求最大，得到每一行的最大元素的下标\n",
    "    rights = pred.eq(labels.data.view_as(pred)).sum() #将下标与labels中包含的类别进行比较，并累计得到比较正确的数量\n",
    "    return rights, len(labels) #返回正确的数量和这一次一共比较了多少元素"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-13T05:38:04.174948Z",
     "start_time": "2020-08-13T05:38:04.171441Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# 损失函数为交叉熵NLLLoss\n",
    "cost =      # change here\n",
    "# 优化算法为Adam，可以自动调节学习率\n",
    "optimizer =    # change here"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-13T05:48:53.621133Z",
     "start_time": "2020-08-13T05:45:37.354192Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "第0轮，训练损失：0.73, 校验损失：0.69, 校验准确率: 0.49\n",
      "第0轮，训练损失：0.24, 校验损失：0.17, 校验准确率: 0.94\n",
      "第0轮，训练损失：0.20, 校验损失：0.14, 校验准确率: 0.95\n",
      "第0轮，训练损失：0.18, 校验损失：0.12, 校验准确率: 0.95\n",
      "第0轮，训练损失：0.16, 校验损失：0.13, 校验准确率: 0.95\n",
      "第0轮，训练损失：0.15, 校验损失：0.11, 校验准确率: 0.96\n",
      "第1轮，训练损失：0.15, 校验损失：0.10, 校验准确率: 0.96\n",
      "第1轮，训练损失：0.13, 校验损失：0.15, 校验准确率: 0.96\n",
      "第1轮，训练损失：0.12, 校验损失：0.16, 校验准确率: 0.95\n",
      "第1轮，训练损失：0.11, 校验损失：0.19, 校验准确率: 0.96\n",
      "第1轮，训练损失：0.10, 校验损失：0.19, 校验准确率: 0.96\n",
      "第1轮，训练损失：0.09, 校验损失：0.23, 校验准确率: 0.95\n",
      "第2轮，训练损失：0.09, 校验损失：0.16, 校验准确率: 0.96\n",
      "第2轮，训练损失：0.09, 校验损失：0.28, 校验准确率: 0.96\n",
      "第2轮，训练损失：0.08, 校验损失：0.24, 校验准确率: 0.96\n",
      "第2轮，训练损失：0.08, 校验损失：0.25, 校验准确率: 0.96\n",
      "第2轮，训练损失：0.07, 校验损失：0.33, 校验准确率: 0.96\n",
      "第2轮，训练损失：0.07, 校验损失：0.38, 校验准确率: 0.95\n"
     ]
    }
   ],
   "source": [
    "from collections import Counter #搜集器，可以让统计词频更简单\n",
    "\n",
    "#循环3个Epoch\n",
    "records = []\n",
    "losses = []\n",
    "for epoch in range(3):\n",
    "    for i, data in enumerate(zip(train_data, train_label)):\n",
    "        x, y = data\n",
    "        \n",
    "        # 需要将输入的数据进行适当的变形，主要是要多出一个batch_size的维度，也即第一个为1的维度\n",
    "        x = Variable(torch.FloatTensor(x).view(1,-1))\n",
    "        # x的尺寸：batch_size=1, len_dictionary\n",
    "        # 标签也要加一层外衣以变成1*1的张量\n",
    "        y = Variable(torch.LongTensor(np.array([y])))\n",
    "        # y的尺寸：batch_size=1, 1\n",
    "        \n",
    "        # 清空梯度\n",
    "                    # change here\n",
    "        # 模型预测\n",
    "        y_pred =    # change here\n",
    "        # 计算损失函数\n",
    "        loss =      # change here\n",
    "        # 将损失函数数值加入到列表中\n",
    "        losses.append( ) # change here\n",
    "        # 开始进行梯度反传\n",
    "                         # change here\n",
    "        # 开始对参数进行一步优化\n",
    "        optimizer.step()\n",
    "        \n",
    "        # 每隔3000步，跑一下校验数据集的数据，输出临时结果\n",
    "        if i % 3000 == 0:\n",
    "            val_losses = []\n",
    "            rights = []\n",
    "            # 在所有校验数据集上实验\n",
    "            for j, val in enumerate(zip(valid_data, valid_label)):\n",
    "                x, y = val\n",
    "                x = Variable(torch.FloatTensor(x).view(1,-1))\n",
    "                y = Variable(torch.LongTensor(np.array([y])))\n",
    "                predict = model(x)\n",
    "                # 调用rightness函数计算准确度\n",
    "                right = accuracy(predict, y)\n",
    "                rights.append(right)\n",
    "                loss = cost(predict, y)\n",
    "                val_losses.append(loss.item())\n",
    "                \n",
    "            # 将校验集合上面的平均准确度计算出来\n",
    "            right_ratio = 1.0 * np.sum([i[0] for i in rights]) / np.sum([i[1] for i in rights])\n",
    "            print('第{}轮，训练损失：{:.2f}, 校验损失：{:.2f}, 校验准确率: {:.2f}'.format(epoch, np.mean(losses),\n",
    "                                                                        np.mean(val_losses), right_ratio))\n",
    "            records.append([np.mean(losses), np.mean(val_losses), right_ratio])\n",
    "            "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-13T05:49:12.046206Z",
     "start_time": "2020-08-13T05:49:11.874669Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAENCAYAAAAL98L+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dd3xb1fn48c+j5b3lkWknsR3CyIAQNoSSQiht2ITVMlpW6WD1Rwtt6YKWErr4djBLCxTKLLTMBggjNJAQRgjgkcQZTuLY8R6SZev8/riyIzl2rDjWsP28Xy+97pYe3Tj30bnn3HPEGINSSinVwxbrAJRSSsUXTQxKKaVCaGJQSikVQhODUkqpEJoYlFJKhdDEoJRSKkTUEoOIJIlIabQ+Tyml1NBEPDGISLqI/AuoAf5fP9sPFJGPRGSjiNwlIlqKUUqpGHJE4TP8wF3Af4DD+9n+J+D7wCvAa8Ai4F8DvZnb7TZFRUXDH6VSSo1i77//fp0xJjecfSOeGIwxrcCrInJx320ikgtMMca8GFh+BFjIHhJDUVERq1atilC0Sik1OonIxnD3jfVtm4nApqDlLcC4vjuJyOUiskpEVtXW1kYtOKWUGotinRhcWLeaeviB7r47GWPuMcbMNcbMzc0NqySklFJqiGKdGLYBE4KWJwKbYxSLUkopYpwYjDGbgDYRmS8iduCrwBOxjEkppca6iFc+i0ga8AGQBiSKyHzge8A0Y8wS4CLgb0Am8KAx5u1Ix6SUUmpg0WiV1AIU72H7auCgSMehlFIqPLGuY1BKKRVnovGAm1JqlDLGUO+pp95TT5orjXRXOkmOJEQk1qFFjDEGb7cXT5eHjq4OOro76Ojq2LXcZ76jq4MEewIFKQUUpBSQn5xPbnIuTpsz1l9lQJoYVFiMMXi6PTR7m2nubKalsyVk2rO+ubOZjq4Ouv3ddJkuuvxddPu78fl9dJtuazkw7X0Za5+e+Z71DpuDVGcqKc4UUpwppDpTSXYmh65zpZLsSCbVFbpf8Hw0L1R+46els6X3YtngaQiZNnob8fl9GGMwGPzGj8H0v2wMfvxgwI/fWja7WndnJWaRm5RLXnIe7iQ3ecl55CbnkpeUR0ZCxrB8Z2MMOz07qW6tZlvrNqpbq9naupXqNmu6rXUbnm5PyDFOm5OMhAwyXBnWtOflyiAzMZN0V3rouoRMMhIyev+djDG0d7XT0tlCS2cLrb5Wa9rZ2jsfst7XSmtnKy0+a5+Wzha83V5sYsMmNuxi331q2319f/sYY0Iu8J4uD55uT8i/w1DYxIY70W0lipR88pPzexNHQUoBBckFuJPc2G32ffqcoZKRNubz3LlzjT75vDtjDJ3+TrzdXjq7O3tfvcuBbb5uH95urzXv94Xs39HV0Xtxb+5spsXbsmu+swWf37fHGFKdqaS50kh2JOOwObDb7DhsDhzisJbF3rveaXPuWg5Me19i7dPl76LN10abr41WX2vvfO9yZxtdpmvQcyMISY4kkp3JJDuShzRNciTR7munwdvQe5HvueA3eBqo99ZT32Fd+LvNbo/i9J6fzIRMXHYXtkCXYD0XJEEQEQTpd7lnXxHBhg0/fho8Dexo30FzZ/Nun+W0OclNyrUSRXDi6FmXZCWRNFcaOzt29l7wt7ZttaatW61k0LYNb7c35L0zEjIYnzKeCakTGJ86nvGp48lJzKHN10ajt5Gmziaavc00eZto6myy1nmtdX2TSDCHzUGSPYm2rrZBL7wOcZDqSu39m0tzpZHqTCXVZS0n2BMwxtBtuvEb/+5T/wDrg6d+PzaxkeRIIsmRRKIjMWQa/Eq0J5LkDEz7bnMk4unysL1tO9vbt1PTVsP29u3Wctt2atpr2N62nY6ujpDvaBc7ucm5FCTvShizcmexoHDBHs/NQETkfWPM3LD21cQQX4wxtPhaaPQ00uBtoMnbRIOngUZvI43expD5nn1aO1vp9Hfu82fbxd57O6Bnmp6Qvttyz3yGK6N3PtWVisMW3QJoTzJs7Wyl3ddu/XL07ZrvSSDtvnbau9p7px2+jt7ltq62kO0DXdT7k+ZMIzspm6yELLISs8hOzCY7MZusxMByQnbIdpfdFZHz4OnyUNtRS11HHTvad1DbXsuODmta215LbYc1bfG17HasIBhCrwFZCVm9F/zxKdY0OAmkOFP2KdbmTitpNHobrQTS2WQlEW8Tnm4PKc4U0pxp1oXflUqaM3DhD8ynulJJtCeOqttVxhiaO5tDEkXwfM/0xMITue2Y24b0GZoY4pAxhiZvE1tat7C5ZTObWzaztXVr78W+ydvUmwgGujg5xEFmYiaZCdYrKzGLzITM3l9ILrsLl821a94eOt+zzWl3WuttrpD9nDbnqPrPtreMMfj8vt0SSZuvjfaudpIcSeQk5lgX/oQsnPb4vUfcn3Zfe2/y6Jk2ehvJS84LSQLJzuRYh6r60fNDKMGeMKTj9yYxaB3DMOr2d7OjfUfvhT/4taVly26/2Hp+YWYmZDI1c2rvBb/nop+RkEFWQlZvMkh1po7pC3ekiUhvoswkM9bhDLtkZzKTnZOZnD451qGoIRCRISeFvaWJYQg2Nm9kQ9OG3S781a3VIffhHeJgfOp4JqVNYmbuTCalTep9TUidoL/MlFJxSRPDXnpv23t8/ZWv9y6nOFOYlDaJkqwSjp98fMjFPz85P+r33ZVSal/pVWsvfVj7IQAPLnyQKRlTyErI0ts7SqlRRRPDXiqrL2Ni6kQOyT8k1qEopVREaJcYe6m8oZzp2dNjHYZSSkWMJoa90NHVwaaWTUzP0sSglBq9NDHshcqGSvzGT2l2aaxDUUqpiNHEsBfKGsoAKM3SxKCUGr00MeyF8oZyUpwpTEidMPjOSik1Qmli2Atl9WWUZpX2dmqmlFKjkV7hwmSMoaKhQm8jKaVGPU0MYdrWto0WX4smBqXUqKeJIUxl9VbFsz7DoJQa7TQxhKmsoQxBKMksiXUoSikVUZoYwlTeUM6ktEnaI6pSatTTxBAm7QpDKTVWaGIIQ7uvnU3Nm7TiWSk1JmhiCENFYwUGo4lBKTUmaGIIQ3lDOaAtkpRSY4MmhjCU1ZeR6kxlfMr4WIeilFIRp4khDOUN5ZRmlepIbUqpMUETwyCMMb2JQSmlxgJNDIOobq2mzdem9QtKqTFDE8MgdAwGpdRYo4lhEOUN5QhCcWZxrENRSqmo0MQwiPL6cgrTC7UrDKXUmKGJYRBlDWV6G0kpNaZoYtiDNl8bm1s2a2JQSo0pUUkMInKOiGwQkUoRubTPtgUisiaw/QERsUcjpnBUNFQA+sSzUmpsiXhiEJE04E7g6MDrNhHJDdrlXuA8oBgoBU6MdEzh6u0KI0sTg1Jq7IhGieEk4A1jTLUxZjvwGnBC0HZv0HwCsCMKMYWlrL6MNFcaBSkFsQ5FKaWixhGFz5gEbAxa3gKMC1q+EPgn0AE8ZIx5v+8biMjlwOUAkydPjlykfWhXGEqpsSgaJQYX4A9a9gPdQcuXAX8GvgOcKSK7XfmNMfcYY+YaY+bm5ub23RwRfuO3BufR20hKqTEmGolhGzAhaHkisBlARPYHDjHG/J8x5h3gX8DVUYhpUNUt1bR3tWuLJKXUmBONxPAycJKI5IlIAXAk8EpgmxeYLCL5ImID5gANUYhpUDoGg1JqrIp4HYMxpkZEbgb+F1h1PXCiiEwzxiwRkV8DK7GSxErgD5GOKRxlDWXYxKZdYSilxpxoVD5jjHkQeHCAbUuAJdGIY2+U1ZdRmF5IoiMx1qEopVRU6ZPPA9CuMJRSY5Umhn60drZS3VqtLZKUUmOSJoZ+VDRqVxhKqbFLE0M/yup1cB6l1NiliaEf5Q3lpLvSyU/Oj3UoSikVdZoY+lHWUMb07OnaFYZSakzSxNCH3/ipaKjQ20hKqTFLE0MfW1q20NHVoS2SlFJjliaGPsoaAhXP2VpiUEqNTWMmMVTuaOHKh97n8+3Ne9yvrF67wlBKjW1jJjGICC+t3c6aLU173K+soYyi9CIS7AlRikwppeLLmEkMhdnJuOw2Kna07nG/ioYKrV9QSo1pYyYxOOw2puamUFHTMuA+LZ0tVLdWa/2CUmpMGzOJAaA0P43ymoFLDD1jMGhTVaXUWDamEkNJXirVjR20ebv63d47OI/eSlJKjWFjKzHkpwEMWM9QVl9GZkImecl50QxLKaXiyphKDKX5qQAD1jOUN5QzPUu7wlBKjW1jJzHUfErRfxYz27Gx3xJDt7+bioYKSrJKYhCcUkrFj7GTGBwJ2Da+zTFp2ynvp8SwuWUznm6PjsGglBrzwkoMIuKOdCARl1kIdhczk3ZQ0U/LpJ6uMLTiWSk11oVbYlgpIi+KyNdEJC2iEUWK3QHZ05gmW6lu7KC1T8uksvoy7GJnaubUGAWolFLxIazEYIyZAvwY2A94U0SeFJEzRGRk9RvhLiavcxMAlX3qGSoaKpiSMUW7wlBKjXlh1zEYY1YaY24CLga8wMNAmYj8SUSyIxTf8HKXktK2GQddu9UzlDWU6YNtSilF+HUMC0TkLhGpAH4FvAIUAKXAJ8CLkQtxGLlLEX8X0xx1IU1Wm7xNbGvbpolBKaUAR5j73QL8A/iZMaa2z7Y/icj8YY0qUtxWU9QjM+pDmqxWNFQAaIskpZQi/FtJy4DHe5KCiIwTket7NhpjzolAbMMvx0oMc5JDWyZpiySllNol3MRwvjFmZ8+CMWYbcEVkQoqgxHRILaDEti2kZVJ5QznZidm4k0Z+q1yllNpX4SaGVhEp6lkQkXHAyOw3wl3CON9mYFfXGGX1ZZRklWhXGEopRfiJ4WasZqq/EZE7gOXAbZELK4LcpaS1VQGGih2tdPu7qWys1NtISikVEFblszHmPyKyBlgIJAKnGmPWRDSySHGXYvc2Ms7RSkVNCxtbNuLt9mrFs1JKBYTbKgnAAB9j3UJKE5EjjTHvRCasCAq0TDoms4HymlbK660H3rSpqlKD8/l8bNmyBY/HE+tQ1AASExOZOHEiTqdzyO8RVmIQkTuBc4AU4FNgJrAGOGrInxwrgcRwcGotf6hpobyhHIc4mJqhXWEoNZgtW7aQlpZGUVGR1snFIWMMO3fuZMuWLUyZMmXI7xNuHcMZQDHwEnAWMBvYNuRPjaX0ieBIYj/7drY2eVhb9xlTMqfgsrtiHZlScc/j8ZCTk6NJIU6JCDk5Oftcogs3MXgAP1Yp4RhgKyOxtABgs4G7mAndWwD4vF67wlBqb2hSiG/D8e8TbmL4CVYp4e/AHcAW4Nl9/vRYcZeS2V4FtnbqvbXaIkmpMeKss86ioaEh1mHEvXATQ3OgE71qYBow3RhzZbgfIiLniMgGEakUkUv72f4TEdksIlUicmS47ztk7lIczZtIS7FKDZoYlBoZTj/9dIqLi0lPT6egoIDi4mKWLFkS9vFPPvkkWVlZYe8/f/583n777aGEOqKF2yrp9wQ6yjPGdAM797z7LoHxG+4EDge6gQ9F5N9B3WtcCszF6pDPA0S+32t3CYJhalYV64HSbL2VpNRI8MwzzwBw8cUXs2DBAi688MLd9jHG6O2ufRRuiWGpiLwgIpeLyPk9rzCPPQl4wxhTbYzZDrwGnBC0/VrgGmNMh7FEvh1coM+kpMTNSHeqdoWh1Aj34IMPcvrpp3P44Ydz8803s2bNGubOncuUKVNYsGABra1W32hFRUVs2bKFqqoqiouLufbaa5k8eTILFiygo6MjrM/auHEjCxcupLi4mNmzZ7N8+XIA3n33XQ466CCKior48Y9/jDGGq6++msmTJzN16lRqa/v2Pxq/wk0MSUANcATwxcBrQZjHTgI2Bi1vAcYBiIgTq/vuS0WkTESeEZGcvm8QSEirRGTVsJzcnGIAWm01+DrG0eLx7ft7KqViasWKFTz//PPceuutJCQk8Pzzz7NhwwYyMzN58sknd9t/w4YNnHHGGVRVVeHz+XpLI4P52te+xgUXXEBlZSX33Xcf559/Pl6vl9tuu42bb76ZqqoqrrnmGj788EOWLVtGVVUVH3/8MRkZGcP9lSMm3CefL9mHz3BhtWjq4ce6pQTgBrKA17G63fhdYHpdn8+/B7gHYO7cuWYfYglElExXxmS20oLfO5uKHa0cPDn8+45KKfjpv9fy6dbmYX3P/cenc8tXDhjSsccddxw5OdbvykmTJvHggw+ycuVKVq9ezZw5c3bbf/z48RxzzDEAHH300WzcuHG3ffpqa2tj7dq1fPWrXwVg7ty5jB8/nrKyMo4++mh+/etfk5mZycKFCxER2tvb+eEPf8i1115LamrqkL5XLIQ7UM89/b3C/IxtwISg5YnA5sB8HdBqjPmvMcZgtXSKSk3wxpzJ+MTQ7SmgsqZ18AOUUnEt+MJ7ySWXUFVVxU033cTZZ5+NdXkJlZCwqzrT6XTS3d292z59dXV17bbOGIPdbud73/sef/jDH7jtttv49re/TVZWFh999BGpqanMnj2bdevWDfGbRV+4lc/Lg+adwBeAcK+mLwO/FJE8rER0JIEuu40xPhF5V0QWGmNeAr4MrAzzffdJeVoONG/C0TV+t2E+lVKDG+ov+2j45JNP+O53v8uECRN49dVXOe2004blfTMyMpg6dSqPPPIIF1xwAatXr6ahoYHS0lLKy8s5+uij+d3vfsell15KbW0tTqeTm266iRUrVrBmzRqmTZs2LHFEWri3kv7WZ9V9IvLPMI+tEZGbgf8FVl0PnCgi04wxS4CrgIdE5P+wksKPwgt935Q57TiMYVZ6OuU7tMSg1Ghy3XXXsWjRIoqLi5kxY8Y+vdfJJ5+M3W4H4KWXXuKRRx7hG9/4Brfccgu5ubk89thjOJ1Ofvvb3/LCCy+Qnp7O7bffTk1NDYsWLcLhcHDwwQezcOHC4fhqUSH9FbEGPUjkIOAxY0zUfzLMnTvXrFq1ap/f56rnFrNj+4ecYL+Bh3ZM5X8/OGHwg5Qa4z777LN9vtCqyOvv30lE3jfGzA3n+HA70fNh9a4KVuXxJuAHexFn3Cnv2MFhnT4OyKxlW9N4Wjw+0hKH3huhUkqNFuHeShpVV8xGTyM7PHVM99uYQjUwS1smKaVUQLitkv4b/HyBiIwTkccjF1ZklTWUAVCaXECu12qiVqEV0EopBYT/gNtUY0xvNxjGmG3AwZEJKfLK6gOJIbOExKb1JDhslGuTVaWUAsJPDBtF5JSeBRE5BhjeJ1uiqLyhHHeSm5y8A5Dmag7KtVGhLZOUUgoI/zmGq4CnROTnQBeQByyOWFQRVt5QbvWo6rY6zzsio5Ent+3NKKdKKTV6hVViMMaUYQ3neR5wGVBqjHk3koFFis/vo7Kx0hqcJ5AYZibtYFuTh2btM0kppcKvfAayjDFlxpiPgJyRWvm8sWkjPr/P6mo7ewqInWlijVJaofUMSo06PWMqdHR0sGjRIvx+/277/OQnP+EXv/hFDKKLT2Ou8rmnRdL0rOngSICsIvI7NwFQuUNbJikVz0444QSeeOKJkHXf+973+NnPfjbosUlJSTz33HPYbOFe9qxxHx5++OG9jnOkG3OVz2UNZThtTooyiqwV7hKSm9eR6NSWSUrFu8WLF/PUU0+FrHvqqac499xzYxTR6BRuYrgSuF1EVovISuAhrArpEae8oZxpmdNw2gLP7LlLkJ3rKM1N0s70lIpzZ555Jq+88goejzWe16pVq8jOzqa0tJTq6mqOPfZYpk2bxrx586iurt7teIdjVyOTW2+9lSlTpnDYYYfx6aefhh1DfX09ixcvpqSkhBkzZvDss88CUFlZyWGHHUZRURGXXmqNYPyLX/yCwsJCCgsL+fjjj/flq0dVuJXP5eyqfP4G1jCcH0Uwrogpry+3Kp57uEuh28uhWR1ax6BUnMvJyWHevHm8/PLLADzxxBOcd955AIgIDzzwAOvWrePYY4/l3nvvHfB9li5dytNPP80nn3zC0qVLKSsrCzuGa6+9lgMPPJCKigpefPFFrr76ampqarjrrrs4/fTTqaqqYsmSJTQ0NLBkyRLKysqoqKhg6tSp+/bloyjsNprGGD9QJiInYA2aswjIjlRgkVDvqae2o9aqX+gRaJk0J3kH9ze7afb4SNc+k5Qa3Ivfh+1rhvc9Cw6Ck3+1x10WL17Mk08+yamnnsrTTz/N66+/DkB+fj6PPPIId9xxB++88w6HH374gO/x8ssvc9FFF5GSkgLAV77ylbBDfPHFF6msrASsoUKPOeYY3n33XY488kh++MMfMmXKlN4xIAoLC7nmmmv4wQ9+QGFhYdifEWvhtkqaJSJLRKQaeBEoAw6KaGQR0PvEc3afEgNQatOWSUqNBKeffjqvvPIKK1asYPz48UycOBGAm2++mVdffZVrrrmGK6+8st/BeXp4PB6czl0/AL1eb9if39XVFVKB3TNQT0/9x6OPPsqpp56K3W7nvffeY+bMmRx11FG9Y0OPBHssMYjIjcCFgcUngBOBpcaYX0Y6sEgobygHCC0xJGdDUjbjujYDB1FR08IhhdqZnlKDGuSXfaRkZmYyb948brzxxt7bSGANznPBBRdQWlrKjTfeSF5e3oDvcdhhh3H33Xdz6aWX0tzczNNPP80ll4Q3gvGCBQv4/e9/z80338ymTZt49913+dOf/kRFRQUzZ87k/vvvp7i4mLa2NpqamvjmN7/J559/zsqVKznqqKP2+ftHw2AlhuuARqzBc243xqxlV/fbI055Qzl5SXlkJfa58LtLSW1Zry2TlBohFi9ezDvvvMNZZ53Vu+7qq6/muuuu4+CDD2bChAl7OBrOPfdcSktLmTZtGmeeeSZf/OIXB9z3iiuuIDMzk8zMTB5//HHuuusu3n33XaZNm8app57KvffeS3Z2Nk888QSTJk3i6KOP5je/+Q3t7e0cf/zxlJSUUFZW1jtO9Eiwx4F6RMQOnAR8FTgeeB2r1FBgjInJY8L7MlDPWc+dhTvZzV8W/CV0w7PfgvKX+XLiA2Qlu3jo64cNQ6RKjT46UM/IsK8D9eyxxGCM6TbGvGCMOQ8oxhq/+QNgm4gMXOUfh3x+H+ua1oXeRurhLoW2HczK0ToGpZQK+xFAY0yrMeZBY8wCYBZQHrmwht+Gpg10+bsGTgzAISl1bG/20NShfSYppcau8J8ND2KMqTbG3DHcwURSmjONK2ddyay8WbtvdJcAsJ9zO6BdYyilxrYhJYaRaFzqOK6efTUTUvuplMosBJuTCV1bAL2dpJQa28ZMYtgjuwNyppHetp4kp11bJimlxrRBE4OI2EVknogUBa2zBR52Gz3cJcjOSorzUqnQW0lKqTEsnC4xMrCeZ8gXkRTg94AX2BrJwKLOXQplLzJ9eiJvrWuMdTRKKRUzg5YYjDH1xphzjTHHA2cCdwD3A6dFOriocpeCv4tD0pqoafZqyySlRgkdqGfv7TExiMjpIvKRiNwjIqlY3W3fgdWJ3tejEWDU5Fgtk/Z3acskpeJVtAfq6TFnzhyuuOKKvT5upBrsDN2OVTJ4FHgbeMIY81vgF8ClEY4tutzFABT6raoTrYBWKv7EYqCetWvX4nA4ePbZZ+ns7IzY58STwRKDDdgJfApkYt1CwhjTBCRHNrQoS8yA1AIy2qsCLZO0xKBUvInFQD0PPfQQ5513HgcffDDPP/987/qtW7eyaNEipk2bxvTp0wdcV1RUxJYtVlP4ZcuWsWDBAsC6fXXRRRdx4IEH8uc//5nXXnuNgw46iKKiIhYvXtx7y2vFihXMmzePqVOnctFFF3HhhRdyzz339MZx1FFHsXr16iGdz4EMlhh+AXwCPIWVFP4oIiUici0wzB2xxwF3CVJXQXFeKpU7tMSgVLyJ9kA9xhgeffRRzjrrLM4555yQ8Z8vueQSvvjFL7Ju3TrefvvtAdftyQcffMCqVau48soryc7OZvny5axfv56qqirefPNNmpubOfvss/njH//I+vXrWbJkCeeddx7PPfccANu3b6e+vp6DDz44vBMYpj22SjLGPAg82LMsItdjJYlNwGXDGkk8cJfCJ09RMiWF5et2xjoapeLa7e/dzuf1nw/re+6XvR83zrtxj/tEc6CeZcuWUVBQwOTJkznttNP4zne+Q2NjIwkJCaxcuZKXXnoJgNzcXDo6OnZbN5gvfelLJCYmAlBYWMi9997LRx99xKZNm6iursbj8XDIIYdw6KGH9r7niSeeyBVXXEF7ezvPPvssixcvHvRz9tZe1cIYY+40xsw0xnzZGLN+2KOJNXcJeBqZmd2lLZOUilPRHKjn4Ycf5rPPPqOgoID99tsPr9fLE088gdfrxWazISIh79F3HVi3r7q7uwHw+UKvKampqb3zJ598Mna7nZ///OfMnz8fYwwejyfk9heA0+nklFNO4ZVXXuGZZ56JSP1K2EN7jgmBPpNmJtQAQkVNC3OLRtTopUpFzWC/7CMlWgP1eDwennnmGT777LPe8R0eeeQR7rnnHi677DIKCwt5+OGHufDCC9m6dSvjx4/vd11RUREffvghhYWFvPbaawPG9Mknn7B48WJcLhfLly/nlFNO4YgjjuDKK6/s7Ua75z3PO+88/vjHP9LY2Mh+++23j2d0d9olRrBAL6tFWJVWFVrPoFRcisZAPc899xxz5swJea8zzjiDNWvWsHHjRv7xj39w9913M2XKFM444wyAftfdcsst3HTTTZx55pm0tbUNGNONN97IzJkzOeecc5g1y+rsMz8/n/vuu48zzjiDadOm8etf/xqAY489lv/973+ceeaZYZ6xvbPHgXr6PcAqJ6UHWiZF3b4M1DMovx9uG4+Zeyn7Lz+Wc+dN4pavHBCZz1JqBNKBeuLHgQceyAsvvMDkyZN32xbRgXqC3vBlEckUERfwPlApIv8vnGNHFJsN3MXIzgpK8lO1l1WlVFx64YUXmDhxYr9JYTiEeyup1BjTCFwErAbGAeGNnA2IyDkiskFEKkWk3wfjROR+EakM9z0jJqcE6sopyUvTZxmUUnHn+uuv57rrruOuu+6K2GeEmxgqReRPwM3AzwE/kLrnQywikgbcCRwdeN0mIrl99jkeKAg36Ihyl0LDRvZzO9nR4oh6u3kAACAASURBVKWpXVsmKaXix5133snnn39OSUlJxD4j3MRwLtZQnhcYYzYChVhJIhwnAW8ERn3bDrwGnNCzUUQSsZLNj8KOOpLcJYBhZnIdgHbBrZQac8JNDIcADxhjlovImcC3gTfDPHYSsDFoeQvWragePwb+DNSH+X6RFWiZNE2szvS0zySlQu1tgxUVXcPx7xNuYrjHGNMsIvsDvwY2A38P81gX1q2nHn6gG0BEDgJmGWMe2dMbiMjlIrJKRFbV1taG+bFDlGN1ppfdUUWyS/tMUipYYmIiO3fu1OQQp4wx7Ny5s/dp6qEK9wG3bhHJA34C/NAY86iIXBXmsduA+UHLE4F3A/MXAcUi8iFWApkkIv80xoQ8422MuQerq2/mzp0b2b9IVzJkTMa2s4LivCO1zySlgkycOJEtW7YQ8R9oasgSExN7nwYfqnATw21YdQxvBpJCCdAe5rEvA78MJBYbcCRwBYAx5gbgBoDA0KFL+yaFmHAX97ZMeqtC/wMo1cPpdDJlypRYh6EiLKxbScaY+40xmcaYRYHlCmPM7DCPrcGqqP4fsBy4HjhRRG4YYsyR5y6FnZWU5qVoyySlVHzo8oKnOSofFe4DbgUi8riI1IrIdhH5m4iE3YmQMeZBY8y0wOuZwGtJn32qjDHFe/sFIsJdAp2tHJhuFYrKtWWSUiqWNr4DfzkaXvp+VD4u3MrnB7B+7U/Eaqq6Crg7UkHFXKBlUol9G4A+Aa2Uio2OBnjuO/DXk8HngQNOj8rHhlvHUGKM+VLQ8l0i8q1IBBQXAonB7dlIsmuytkxSSkWXMbD2aXjx+9C+E478Nsz/AbhSovLx4SaGZhEpNcaUAwQqn0fv4Kep+eBKw7azgpK8/fUhN6VU9DRshOevh8r/wrjZcOGTMG5WVEMINzFcDywVkfcAg9Wy6OsRiyrWRKx6hrpySvLTeKNcWyYppSKsuwtW/AmW/RIQWPgrmHc52OxRDyWsxGCMWSYis4AjAsd80xgzuq+W7lKoeovSuak8+f4WGts7yUx2xToqpdRoVL0a/v1d2P4xlJ4MX7oDMifFLJywR3AzxjQAL/Qsi8gOY8zAQySNdO4S+Pgx9suy6ucrdrRyqI7mppQaTt5WeP1WePcvkJIH5/wdZiyy7lrE0L4M7ekcfJcRLFABPd1VA0B5TYsmBqXU8Cl7EZ6/AZqrYe6lsOAWSMyIdVTAviWG0d1ZSmD85zzvJlJc6dpkVSk1PJq3wUs3wqfPQt7+cPZfYdK8WEcVYo+JQUR89J8AhNE+XnT2VBAbUldBcf58bZmklNo3fj+8/wAs/an1FPMJP4Yjvg2O+Ku73GNiMMaM7ttFe+JIgKyiQJ9JX9aWSUqpoav5FP5zDWx+F6YcB1/+LeRMi3VUAxrdv/r3VU+fSfmp1LZ4aWwfvY9uKKUiwBh46064+xioq4DT/gJfezaukwJoYtgzdwnsrKQkNxnQQXuUUnvB74cXboBXf2a1NPrWKph9XsxbHIVDE8OeuEuhy8OM5CZAh/lUSoWpqxOe/gasvA+O/A6c9QCk5MQ6qrDtS6uk0S/HapmU791EisuuLZOUUoPrbIPHvwaVS2HBT+Hoa2Id0V7TxLAngWcZZGcFxfmztTM9pdSetdfDPxZD9SpYdBcc/LVYRzQkeitpT1JyICkb6sopzUvVOgal1MCat8GDp8C2D+Hsv43YpACaGAbnLoW6Skrz06hr9dLQpi2TlFJ97FwHD5wIjZvggidh/0WxjmifaGIYTKCX1eL8VMDqM0kppXptXwMPLLTqFi76N0w9LtYR7TNNDINxl0DbDqZndANoPYNSapeN78BfTwG7Cy55CSYcHOuIhoUmhsEEKqDH+TaTmuCgQhODUgqg7CV46HRIzYNLX4Lc0lhHNGw0MQymp2VSXQXFWgGtlAL46DF47HzIm2ElhRiOnRAJmhgGk1kINifsrKAkL1XrGJSKlDVPwl+Ohrd+A607Yh3NwFb8GZ65AoqOsuoUUtyxjmjYaWIYjN1h9WtSV6Etk5SKlJ3r4LnvQPNWePWn8JsZ1kNi6163upaIB8bAa7fCS9+HGV+B85+AhLRYRxUR+oBbOHKKrV5W51gtk8prWjhs6sh5vF2puNbVCU993ep++oq3wNcO7z8IH/7DGrMgawocchHMvhBSc2MTo78bXvgerLof5nwVvvL7mIzFHC1aYgiHuxTq11PqTgSgXG8nKTV8Xr8Vtn5gPSmcMcFqCXjSrXDdZ3DGfZA+AZb+JFCKuAjWL4tuKaKrE576hpUUjrrGinMUJwXQEkN43KXg72KcfzupCQ4qtWWSUsNj/Ruw/PdwyMXW7ZlgzkSYebb1qi23ShEf/QM+/Zc1kNbBF8HsCyJbiuhsg39eCOtegy/+DI76buQ+K45oiSEcvX0mVWrLJKWGS3u9VYnrLoGTbtvzvrmlsPA2uO5zOONeSC2ApbdYpYgnLrYSjBmm0YaNAW+LNX7C30+1SiiL/m/MJAXQEkN43MXWtK6c0vzjee3zOG4xodRIYAw8921oq4Pz/wmulPCOcybCzHOsV23ZrrqItc9A9rRAXcQFoS2F/H7wNEL7Tuvz2uuCpjtDl9vrrflur3Ws3QXn/H330swop4khHIkZ1i+UugpK8xfx+KotbK5vZ1J2cqwjU2pkev+v8Pl/4MRbYdysob1H7nRY+Es44Rarkvr9v8J/fwyv/QLGzwFP866Lvenu/z1caVZnmcluqy6jYFZgObBuwiGQt9/Qv+cIpYkhXIE+k46Yl4PdJiz4zRuce+gkLj9uGhMyk2IdnVIjR20ZvHQTTPsCHP7NfX8/ZyLMWmy9dnxulSK2f2w1M598mHWBT3Fb0+TsoPkc61i1G00M4XKXwCdPc8C4dJZedxx/XlbJI+9u4h/vbeKMORO5av40itxhFoeVGqu6vPDk161bR6f9BWzDXM2Ztx+c/Kvhfc8xSCufw+Uute5TttUxxZ3Cr8+axbLvzee8eZN55sNqvnDnMr7z6AeUbdcWS2oEaKuDl2+G3+xvPXEcLUt/CjVr4NQ/Qlp+9D5X7RUtMYTLbQ3zyc6K3uZxE7OS+dmpB/KtLxRz/1sbeHjFRp77aCsn7p/Pt75QzMyJmTEMWKl+eJrgnf+DFX+yHiTLLLQeLqsrh/k/iOxA9RVLYcUfYd7lMH1h5D5H7TNNDOEKNFmlrhwKjwzZlJeWyA++NIOr5k/jr8ur+OvyDbzyaQ3HlLj51vHF+pS0ir3Odnjvbnj7d1bJd//T4PibIKsI/nMtvHG79bd92p/BGYE6s9Za+NdVkLe/9TyAimuaGMKVPhEcSVbb5gFkJru49oulfOOYKTy8YhP3v72exfesYF5RNld/oZhjS9xIJH+RKdVXVyes/hu8eQe01kDxF+ELP4Txs3ftc+ofrRY+/70FGjbCeY9CWsHwxWAMPPtNq7TytWcjk3jUsNLEEC6brbfPpMGkJTq5av40Lj6yiH+u3MTdb67nogfeY+bEDK4+vpgvzsjHZtMEoSKouws+/ics+xU0bYLCo6xxiAuP2H1fEevhrZxieOoyuOd4OP+xoTcj7eu9e6DiFTj5Dsjff3jeU0VUVCqfReQcEdkgIpUicmmfbd8Vkc9EpEpEHhKR+E1WgSar4Upy2bn4qCm88b3j+dUZB9HU4eOKh95n4e/f5NkPq+nqjpNeI9Xo4fdbD3v9+QjrV3pyNlz4NFz8fP9JIdh+p8DXXwaxWUNVfvbvfY+nZi288iMoOQnmXbbv76eiIuKJQUTSgDuBowOv20QkuHOTFmAWUAzkA4sjHdOQuUutwb59nr06zOWwce68ybx63XH8/tzZGAPffexD5i9Zxk+eW8urn9XQ5u2KUNBqTDAGyl+Be46zuogQGyx+GC5fBsUnhF+pXHAQXPaaVRfwzwutsRGG2tWEr8NqmpqUCaf9KbIV22pYRePX+UnAG8aYagAReQ04AXgMwBjzQM+OIvIxkB2FmIbGXQLGD/Xrh1QkdthtnDp7Al+ZOZ5XPq3hsZWbeGzlJh58pwqnXTh4chbHluZybEkuB4xP19tNKjxVy+HVn8HmFVZl8ul3w0FnD70H0LR8uPg/8Oy3rLER6sqtbqYdCXv3Pq/8CGo/s0oso3Awm9EsGolhErAxaHkLMK7vTiKSDJwCnNzPtsuBywEmT54cmSjD0dNkta58n+6V2mzCwgMLWHhgAR5fN6s3NvBGRS1vlddxx8tl3PFyGVnJTo4uyeWYEjfHluRSkKFPaI4KXV6rb39Hwr533Vy9Gl77udXzZ9o4OOU31lgBDte+x+lMgjPvs0rJy26D+g1w7iPhX+DLXoSV98IR37JKLGpEiUZicAHBN9P9QEjHJSJiA/4O3GWMqer7BsaYe4B7AObOnTtMXSgOQU5PZ3oDt0zaW4lOO0cWuzmy2M0PTobaFi/LK+t4s7yWNyvq+PdHWwEozU/lmECiOGxKDkmu0d0f/KjiaYLPX7Du/a97Dfw+a73YrE7a7C6wO8GeEJgG1jlcfbYHLXfUW++VlA0n/gIO/cbwt/YRgfk3Wj+I/nUV3Hs8nPfPwX8UtWyHZ6+2bkud8OPhjUlFRTQSwzZgftDyRODdngWx2m/eB3xqjPlTFOIZOlcKZEzaqwrovZWblsBpcyZw2pwJGGP4fHsLb1XU8lZFHQ+t2Mj9b2/A5bAxryibY0rcHFOSy4xxadoMNt54mqH8JSsZVC6F7k7rb2fe5dYDkt0+a12Xd9d8d2dgvp913pbQfQHm3wSHXwWJ6ZH9LgeeAVmF8Oj5cP+JcNYDUHpi//v6/fDMldZzE2c+sPe3n1RcEDNcfZgP9AEi+cBqYA5WZfc7wEHGmLbA9j8DDcaYm8J5v7lz55pVq1ZFKtzBPXS61VvjFW9E/aM9vm7e3VDPW+VWoigLDBiUk+JizuRM5kzOYs7kTGZNzCQlIX4bd41a3tZdyaDiv9YFPn2C9TDZAafDxLkjuwK2qRoePRdqPrF6RT38qt2/zzt3wSs/hC//DuZeEps4Vb9E5H1jzNyw9o10YgAQkYuBHwUWbwhMpwErgDeB9UG7/8gY8+hA7xXzxPDijfDBw/CDLTH/T17T7OGtijr+t24nH2xuYH1tGwA2gdL8NA4uzGLOJCthTHWnaGV2JHS2QfnLgWTwCnR5rPv9vcng0OHvKC6WOtvg6cutLrMPuRi+tMS6zQWw7SO49wQoPclqETWSk+AoFHeJYTjFPDGsvA+ev976w5/+pbga+7WxvZMPNzeyelMjH2xq4MPNjbR4rGawGUlOZk/K7C1ZzJ6YSUayM8YRj1Cd7VD5X/jkaSspdHVAaj7sf6qVDCYdPrqSQV9+v1Xp/fZvYMqx1oNzjgS4+zgrcVy13Hp+QsUVTQyRVL8B7ltgDQCSNs5qFjj7fMibEbuYBuD3G9bXtbJ6YyMfbG7gg02NlNW09DZLL85L7S1RzJmcSWl+GnYtVfTP12HVFax9BspeAl8bpOTCjEXWPfjJR8TVj4So+OgxaxS2jEmQf4D1QNxFz1nJQsUdTQyR1uW17iV/+Kj1y9HfZXUfMOt8OOisuG6z3eLx8fGWJj7YZCWKDzY3Ut9mVWa67DbGZyYyMSuZCZlJTMxKYmJ2Uu9yfnri6E0cnW3QtAUaN1tdSPTOb7amLVutZ1iSc6xkcMDpVjcT9jFel7NpBTx2gfVD6ehrYcFPYh2RGoAmhmhqrYVPnrTGnd3+MdgcUHIizDrPutca560yjDFs3NnO6k0NlNW0UN3QwZbAq67VG7KvwyaMz0zalTSykpmYlcSELGu5ID0Rhz0Ob6EYYzUY2O2Cv8maNm2xxgMOZnNA+njr13DGJMicZPWqW3SsJoO+GjbC589bTWaH4xkKFRGaGGKlZi189Ch8/LjVk2VSFhx4ppUkJhwy4irjPL5uqht7EkV7UNJoZ0tDBztaQhOH3SYUpCeSl55AWqKTtEQHaQkOa5roJLV33tG73VpnzSc4bMPb7Hb7Gnj5Jtiyyhp7IJgzedcFv3c6GTImWvNp48berSE1qmliiLXuLli/zEoSn//HaqmSUwKzzrVeGRNjHeGw8Pi62dbkCUkamxva2dnaSYvHR4u3ixZPF62eLjp8AwzGHsRpl5AEMj4zif3HpXPghAwOGJ/OuIzE8BKHtwVevw3e/Yv1ANjMcyAzcNHPmGTNJ2WNuESt1L7QxBBPPE2w9l9WRd2mdwCxKudmnQczvgIJqbGOMCp83X7aAonCevmspOHt2i2J9Gyr2tnG+rq23sryrGQnB4y3ksT+49M5YHwGU9wpu+o9jLEqh1++yXr6du4l8IUfaQsZpdDEEL/qN1h95H/0KDRUgSMRJs2DomOg6GjrdlOc10lEW3tnF59ta+HTrU2s3drM2q3NlG1voTPQZXmS086McWkck9PMubV/YFzdO/gLZmL78m+tB8qUUoAmhvhnjNWa47PnoOot2P4JYKxEMfHQXYli4lxNFP3wdfup3NHK2q3NfL55B9Mr7mNR6+N4cbCk6xweMycyNc8qURw4IZ3pBWkU56aSm5agXYeoMUsTw0jTXg+b/gdVb1uv7WvYPVEcBRPmglN7We1VsRReuAEaNmAOPJsth97Ex01JrA0qXQS3rEpLdDAtN5XivNSgaQqTs5PjszWVUsNIE8NI19EAG3sSxVu7EoU9IXDr6ejAradBEoXPYzXDDHnVD7CuDsRuPY8xfk7gNTs+n8lo3govfR8+fdbq8faUO2Hq/H533dHsobymlXW1rVTu2DUNblHltAtFOSm9CWNaXgrFuWlMzU3RPqfUqKGJYbQJThQb34ZtH9ObKCYeaj117W3edaFvC0x9bQO/Z1KW9bBW7yvbSiTbPoKdQd2KZ0yyEkRPshg3O3aVud1dVkujZb+0Hio89gY48jtDut3W7PGxbkcr62rbehPGuh2tbKxvp9u/6//E+IxEpgUSRmFO8q6H/7KTSE/ULkXUyKGJYbTraLDqKHpKFDvXQ3LPhd4derEPufgHXklZe35Iy9NkJZ+tH8C2D61pfVA/h5mFQaWKOVYpIykzst950wqrj6qaT6wHCE/+NWRPGfaP6ezys6m+J1mEJo22ztAmt2mJjl0P+fU+9Lfrwb+MJKfWaai4oYlBDb+OBqs0sTWQKLZ+AI1BA/NlT92VKNylgQQVSEwJaUN/ZqBtJyz9sdWjbfpEOPl2a9D6KF9wjTHUt3WypaEj8NDfrmc3eh4CbO0zbneKy26VMAIJw0oeyRRkJFKQkUhuagIuh9ZtqOjQxKCio71+V4li6wdW0mjavPt+NmefEkw/JZmk7ND1zmT44CFYeov1wNoRV8Ox/y9un/swxtDU4evtTqQneWxp6AgkkHaaPV27HedOdZGfntj7KkhPpCAjgbye+fREMpO15KH2nSYGFTttddbzGh0DVXIHzXfUWx3T9cfmsOoRJh9pVS7vwxjb8aLZ46O6oYPtzR5qmjzWtNlLTbOH7U0eapo97Ax0aBjM5bCRn55AQZ8EkpuWgDs1AXeaC3dqAlnJrtHbyaHaZ3uTGLTJhRpeKe7wWzL5/eBpHKCl1E4omGn1VjtKfi2nJzpJH+dkxriBh+Ls7PKzo8UTSBZW0qhptpLI9iYPa7c28+pnO/rtYsQmkJ2SgDvV1Zs0rKmVOHpfaS5yUhI0iagBaWJQsWOzBW4fZQPFsY4mLrgctkDldfKA+xhjaPZ0Udfqpa7FS11rpzUfeNW2dFLb6mV9bRt1rV68XbuXykQgO9lFTqqLZJeDJKedRKeNJJedRKedpMAr0WkPWZfotFlTV+g+yS47WSkuUlx2ve01CmhiUGqEEREykpxkJDmZlrvnOhdjDK3erl3JoyWQPALL9a2dtPu68XR2U9faicfXTYev25p2WvP+vbjb7LLbyEpxkpXsIjvFRVayi6wUJ9nJLrJSdq3LTgksJ7tIcmkvtvFGE4NSo5iIBLo1dzLFnbLXxxtj8HWbkGTh6dqVNDy+bjw+P63eLhrbO6lv89HQ1klDu/X6fHszDe0+Gto7Gag6M9Fp600cWcku0pMcZCQ5rVtvSYFXooP0QDJMTwxMkxwkODSpRIImBqXUgEQEl0NwOWxkJA39gb5uv6G5w0d9eycNbZ3UB5JHfZsvMO3sTSjbmz00dfho7vD1exssWEIgrp7ksWveuWsckEQH6QOMA5Ka4NC6ln5oYlBKRZzdJlaJIMUFueEf5/F10+zx0dzRRbPH15swmjt8NHu6aO4IrAvsU9fayfq6Npo7rK7bu8K4D9Yz/kdqQmjyCE4iKQnWoFMpCQ5SEx2kJthJTXCSkmAnLTAdTf1taWJQSsWtxEDldl7a3h9rjMHj84eM99E7DojHSjTB44O0BvZpbO9kc7313Emr14fHt+dSy65YbaQm7EokPfOpiQ6SXQ6SXXZSXHaSAvNJLqvSPtllJ8npICUhMO9ykByo9B/2UQ3DpIlBKTUqiQhJgQtw3j68T1e3nzZvN62dVkJp9fpo9XbT6umyBp/yWtOexNIz3+rtYluTh9YdXbR3dtHe2U175+AjGQazCaS4HL1JZMGMfH745cg/06OJQSml9sBht5GRbCMjed87TewpxQQnivbOLjp65n3dtHutbR2+7t79erYXZESn231NDEopFSXBpZicWAezB6OntkQppdSw0MSglFIqhCYGpZRSITQxKKWUCqGJQSmlVAhNDEoppUJoYlBKKRVCE4NSSqkQI25oTxGpBTYOuuPA3EDdMIUTDSMtXtCYo2WkxTzS4oXRFXOhMSasLgxHXGLYVyKyKtxxT+PBSIsXNOZoGWkxj7R4YezGrLeSlFJKhdDEoJRSKsRYTAz3xDqAvTTS4gWNOVpGWswjLV4YozGPuToGpZRSezYWSwxKKaX2QBODUnFKRJJEpDTWcewNjTk6Ih3zqE0MInKOiGwQkUoRubTPtgNF5CMR2Sgid4lIzM+DiCSKyD0iUhaI69o+2x8UkerA96kUkcmxijWYiFQFxfRWn21xdZ5F5PtBsVaKiEdEvhS0PS7OsYiki8i/gBrg/wWt/66IbAr8jZzcz3HHiUh54O/+5ljHLCI5IvJPEakQkXUicm4/xy3r8zdkj2XMgfW+oHge6+e4eDvPd/X5u+4Wkf37HLd359kYM+peQBqwGZgAFADbgdyg7W8CJwN24A3gtDiIOQc4ExCsB1RqgElB2x8E5sc6zn7irtrDtrg7z0GxZQDrAUe8nWMgFTgB+AZwX2DdNKA88Le9P7AVcAYdI0AFMBNICew7O8Yx79dzPoFioDE45sD6ZUBRvJznwPqqPRwTd+e5z/YSYFU/6/fqPMf8l3KEnAS8YYypNsZsB17DOpmISC4wxRjzojGmG3gEWBi7UC3GmJ3GmKeMpQ4rsWXGOq6hitfzHOQC4EljTFesA+nLGNNqjHkVCI7tdOBxY0yLMeZToAo4JGj7IUCNMeZjY0wb8CRRPN/9xWyM+dwYsywwXwn4gKRoxTSYAc7zYOLuPPdxGfDAvn7OaE0MkwjtNmMLMC4wPxHYNMC2uCAiBwKJwCdBq33A30RkrYhcH5vI+tURuE2wQkROClof7+f56+z+HyhezzHs+W86nO0xFbj1tdoY09xnkxd4XUQ+EJELYxBaf3ICf9Ovi0jfJ4jj9jyLiBM4B/hHP5v36jw7hju4OOEC/EHLfqA7jG0xJyJu4CHgEhMoAwIYYy4LbJ8E/FdEPjLGLI1RmL2MMTMAROQY4BkRKTbGNBLH51lEDgE8xpjPg9fH6zkOGOx8xvP5LgbuAL7cd5sx5qTAPvsDr4rISmNMWZRD7BtTWiCms4FnsJJBj7g9z8AiYHng/1+IvT3Po7XEsA2rfqHHRKxbM4NtiykRyQL+DdxkjFnZ3z7GmM3Af4ADoxnbYIwxb2Hd3igKrIrb84xV3L5/oI1xeo4HO59xeb5FpBDrdsvXjDFVA+0XuD22HJgRpdAGZYx5AkgSkeBbunF5ngP2+HcN4Z/n0ZoYXgZOEpE8ESkAjgReATDGbALaRGR+oGb+q8ATsQvVIiLpwHPArcaYF/vZXhyY5mDd0+w3cUSTiKSIyLjA/BysInUFxPV5TgG+Ajzez7a4O8dBngfOFZFkEZkBZAMfBm1fAUwXkemB73gG8HQM4uwlIhMCMVxmjFk9wD4957wQOIzQ7xR1IuLuSQSB21/1fX6Bx915ht7zVwy8PsD2vTvP0apNj/YLuBhYF3idHnjdENh2MLAGK9P/PNaxBmL6IdAGVAa9rg+K+QWsX+RlwLdiHW8gplysVhnrgNXA8SPgPF8K3B+0fHq8nWOslkeVWC3TmgLzxwM3ARuAz4CjAvt+CzgvML8w8O9RBVwVBzGvC5rvebn6xLwm8J3WAmfHQczfDsSzDngbmDUCzvPxwM+AH/XZd8jnWbvEUEopFWK03kpSSik1RJoYlFJKhdDEoJRSKoQmBqWUUiE0MSillAqhiUEppVQITQxKKaVCaGJQag9E5JpAp3obRWS1iKSJyE9jHZdSkaQPuCk1ABE5EvgdcJwxpkNEigKblhpjimMWmFIRpiUGpQaWjdV7Zk/f9zuwBjyZHBgFa0ZglLJnA6OqvRromwuxRof7XmD0sjUiclBg/ddFZL1YI7HF0/gUSvXSxKDUwF7GGintAxFZaIxpB+YDm4wxxcaYz4DfAn8xxkzH6iTwB4FjXUCrMaYEWALcFVj/G6xOHYuwOmRTKu7orSSlBiEii4DbsXqu/DVBt5JEpBbYGdjVAXxojDlLRPxAkjHGGxhAZacxJl1E/gN4gFuMMWuj/mWUCoOWGJQahDHmOeBQrIFmkvtsdgLzjDH7BUoRZ/Ucxq5bUAlAR2B+EdbgL/8ODASjVNzRxKDUAETkoKAKZwE6sbpGzxARh4gIVtfMVwX2zwuMkAXW/60zA/NXAK8G5ouNMY8A/wccFfEvodQQcYF0AwAAAI5JREFUaGJQamBZWEN8VgHvAL8yxmzEGgGrCpgOfAf4cmCfl7HqFsAa7vEQESkHTgFuCKx/QkQqsQZ4+UN0voZSe0frGJSKABHpMsaM1jHV1SinJQallFIhNDEopZQKobeSlFJKhdASg1JKqRCaGJRSSoXQxKCUUiqEJgallFIhNDEopZQKoYlBKaVUiP8PdDVuAAKvRL0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 绘制误差曲线\n",
    "a = [i[0] for i in records]\n",
    "b = [i[1] for i in records]\n",
    "c = [i[2] for i in records]\n",
    "plt.plot(a, label = 'Train Loss')\n",
    "plt.plot(b, label = 'Valid Loss')\n",
    "plt.plot(c, label = 'Valid Accuracy')\n",
    "plt.xlabel('Steps')\n",
    "plt.ylabel('Loss & Accuracy')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-13T05:55:40.375965Z",
     "start_time": "2020-08-13T05:55:39.993871Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# 对测试集数据test_data进行预测\n",
    "test_pred = "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-13T05:55:43.571410Z",
     "start_time": "2020-08-13T05:55:43.548555Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.958173076923077"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.metrics import accuracy_score, roc_auc_score,  roc_curve, auc\n",
    "# 使用测试集数据衡量模型准确度\n",
    "accuracy_score( )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-13T05:56:10.278771Z",
     "start_time": "2020-08-13T05:56:10.265676Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9582704597905201"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 使用测试集数据衡量模型AUC Score\n",
    "roc_auc_score(  )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-08-13T05:56:27.861088Z",
     "start_time": "2020-08-13T05:56:27.849327Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9582704597905201"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fpr, tpr, thresholds = roc_curve(  )\n",
    "auc(fpr, tpr) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 作业2： 使用Imagenet的VGG16预训练模型识别动物（可选非必选）\n",
    "\n",
    "- 运行这里的代码https://chengjunwang.com/mybook/09-16-pytorch_vgg_pretrained.html\n",
    "- 尝试找一个金鱼（goldfish）的图片替代狗的图片，并识别其效果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  },
  "latex_envs": {
   "LaTeX_envs_menu_present": true,
   "autoclose": false,
   "autocomplete": true,
   "bibliofile": "biblio.bib",
   "cite_by": "apalike",
   "current_citInitial": 1,
   "eqLabelWithNumbers": true,
   "eqNumInitial": 1,
   "hotkeys": {
    "equation": "Ctrl-E",
    "itemize": "Ctrl-I"
   },
   "labels_anchors": false,
   "latex_user_defs": false,
   "report_style_numbering": false,
   "user_envs_cfg": false
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "165px"
   },
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}