main.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction\n",
    "This jupyter notebook is accompanied with [MBTI Twitter personality classsification](https://www.github.com/sahilsehwag/twitter-mbti-personality-classifier) project. This notebook is created to easily explore our dataset, perform analysis and perform modelling. To use the final classifier refer to [README.md](https://www.github.com/sahilsehwag/twitter-mbti-personality-classifier) file of this project. Use [main.py](https://github.com/sahilsehwag/twitter-mbti-personality-classifier/blob/master/main.py) to perform personality classification on any twitter user.\n",
    "<br>\n",
    "### Dataset\n",
    "The [dataset](https://github.com/sahilsehwag/twitter-mbti-personality-classifier/blob/master/personality-test.csv) on which this classifier is trained, contains around 50 posts per user about 8000 users with their *MBTI*\n",
    "personality type known. Dataset is provided in the repo itself [personality-test.csv](https://github.com/sahilsehwag/twitter-mbti-personality-classifier/blob/master/personality-test.csv).\n",
    "<br>\n",
    "Few features of dataset are:\n",
    "* Posts of more than 8000 users.\n",
    "* Last 50 posts per user, each entry is separated by '|||'.\n",
    "\n",
    "### Challenges\n",
    "The main challenge with this particular dataset is the imbalance in dataset. The dataset is hugely imbalanced. I have tackled this problem by training a model for each characteristic pair. There is imbalance in different characteristic pair like IE pair. Different solutions to this problem are stated in below sections."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Setup"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Helper Methods"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-06T18:46:21.754319Z",
     "start_time": "2018-04-06T18:46:21.734133Z"
    }
   },
   "outputs": [],
   "source": [
    "#printing on single line\n",
    "def printl(msg):\n",
    "    sys.stdout.write(\"\\r\" + str(msg))\n",
    "    sys.stdout.flush()\n",
    "\n",
    "\n",
    "#pickler\n",
    "def save(filepath, obj):\n",
    "    with open(filepath, 'wb') as file:\n",
    "        pickle.dump(obj, file, pickle.HIGHEST_PROTOCOL)\n",
    "\n",
    "\n",
    "#unpickler\n",
    "def load(filepath):\n",
    "    return pickle.load(filepath)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-06T18:46:26.417553Z",
     "start_time": "2018-04-06T18:46:21.757793Z"
    }
   },
   "outputs": [],
   "source": [
    "import sys\n",
    "import re\n",
    "import string\n",
    "import pickle\n",
    "import os\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from nltk.corpus import stopwords\n",
    "from nltk.stem import WordNetLemmatizer\n",
    "from nltk.stem import PorterStemmer\n",
    "\n",
    "from sklearn.feature_extraction.text import CountVectorizer, TfidfTransformer\n",
    "from sklearn.metrics import confusion_matrix, accuracy_score, classification_report, recall_score, f1_score\n",
    "from sklearn.preprocessing import LabelEncoder, OneHotEncoder\n",
    "from sklearn.model_selection import train_test_split, cross_val_score\n",
    "from sklearn.externals import joblib\n",
    "\n",
    "from sklearn.naive_bayes import MultinomialNB\n",
    "from sklearn.ensemble import RandomForestClassifier, ExtraTreesClassifier\n",
    "from sklearn.svm import SVC\n",
    "\n",
    "import tweepy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loading Dataset\n",
    "I will add one column for each MBTI characteristic pair, since we will be training independent classifier model \n",
    "for each pair independently. The reason for this is because of imbalance present in our dataset. For example posts of \n",
    "introverts are very large as compared to extroverts. To understand and visualize this go to [**EDA**](#EDA-(Exploratory-Data-Analysis&#41;) section."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-06T18:46:32.142890Z",
     "start_time": "2018-04-06T18:46:26.421099Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 4.89 s, sys: 158 ms, total: 5.05 s\n",
      "Wall time: 5.63 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "df = pd.read_csv('personality-test.csv')\n",
    "df['ie'] = df.type\n",
    "df['ns'] = df.type\n",
    "df['ft'] = df.type\n",
    "df['pj'] = df.type\n",
    "\n",
    "for i, t in enumerate(df.type):\n",
    "    if 'I' in t:\n",
    "        df.ie[i] = 'I'\n",
    "    elif 'E' in t:\n",
    "        df.ie[i] = 'E'\n",
    "        \n",
    "    if 'N' in t:\n",
    "        df.ns[i] = 'N'\n",
    "    elif 'S' in t:\n",
    "        df.ns[i] = 'S'\n",
    "        \n",
    "    if 'F' in t:\n",
    "        df.ft[i] = 'F'\n",
    "    elif 'T' in t:\n",
    "        df.ft[i] = 'T'\n",
    "        \n",
    "    if 'P' in t:\n",
    "        df.pj[i] = 'P'\n",
    "    elif 'J' in t:\n",
    "        df.pj[i] = 'J'\n",
    "\n",
    "\n",
    "posts = df.posts.values\n",
    "yIE = df.ie.values\n",
    "yNS = df.ns.values\n",
    "yFT = df.ft.values\n",
    "yPJ = df.pj.values\n",
    "y = df.type"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preprocessing\n",
    "NOTE: this phase might take some time to execute due to computationaly expensive operations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-06T18:46:32.399181Z",
     "start_time": "2018-04-06T18:46:32.146330Z"
    }
   },
   "outputs": [],
   "source": [
    "#regular expressions for tokenization\n",
    "regexes = [\n",
    "    #urls\n",
    "    #r'http[s]?://(?:[a-z]|[0-9]|[$-_@.&amp;+]|[!*\\(\\),]|(?:%[0-9a-f][0-9a-f]))+',\n",
    "    \n",
    "    #html\n",
    "    #r'<[^>]+>',\n",
    "    \n",
    "    #punctuation\n",
    "    r'(?:(\\w+)\\'s)',\n",
    "    \n",
    "    r'(?:\\s(\\w+)\\.+\\s)',\n",
    "    r'(?:\\s(\\w+),+\\s)',\n",
    "    r'(?:\\s(\\w+)\\?+\\s)',\n",
    "    r'(?:\\s(\\w+)!+\\s)',\n",
    "    \n",
    "    r'(?:\\'+(\\w+)\\'+)',\n",
    "    r'(?:\"+(\\w+)\"+)',\n",
    "    r'(?:\\[+(\\w+)\\]+)',\n",
    "    r'(?:{+(\\w+)}+)',\n",
    "    r'(?:\\(+(\\w+))',\n",
    "    r'(?:(\\w+)\\)+)',\n",
    "\n",
    "    #words containing numbers & special characters & punctuation\n",
    "    r'(?:(?:(?:[a-zA-Z])*(?:[0-9!\"#$%&\\'()*+,\\-./:;<=>?@\\[\\\\\\]^_`{|}~])+(?:[a-zA-Z])*)+)',\n",
    "    \n",
    "    #pure words\n",
    "    r'([a-zA-Z]+)',\n",
    "    \n",
    "    #numbers\n",
    "    #r'(?:(?:\\d+,?)+(?:\\.?\\d+)?)',\n",
    "\n",
    "    #emoticons\n",
    "    #r\"\"\"(?:[:=;][oO\\-]?[D\\)\\]\\(\\]/\\\\OpP])\"\"\",\n",
    "\n",
    "    #other words\n",
    "    #r'(?:[\\w_]+)',\n",
    "\n",
    "    #anything else\n",
    "    #r'(?:\\S)'\n",
    "]\n",
    "\n",
    "#compiling regular expression\n",
    "regex = re.compile(r'(?:'+'|'.join(regexes)+')', re.VERBOSE | re.IGNORECASE)\n",
    "\n",
    "\n",
    "def preprocess(documents):\n",
    "    lemmatizer = WordNetLemmatizer()\n",
    "    stemmer = PorterStemmer()\n",
    "    \n",
    "    #fetching list of stopwords\n",
    "    punctuation = list(string.punctuation)\n",
    "    swords = stopwords.words('english') + ['amp'] + ['monday', 'tuesday', 'wednesday', 'thursday', 'friday', 'saturday', 'sunday', 'january', 'feburary', 'march', 'april', 'may', 'june', 'july', 'august', 'september', 'october', 'november', 'december',  'mon', 'tue', 'wed', 'thu', 'fri', 'sat', 'sun',  'jan', 'feb', 'mar', 'apr', 'may', 'jun' 'jul', 'aug', 'sep', 'oct', 'nov', 'dec', 'tommorow', 'today', 'yesterday'] + ['mr', 'mrs']\n",
    "\n",
    "\n",
    "    processed_documents = []\n",
    "    for i,document in enumerate(documents):\n",
    "        printl('{0}/{1}'.format(i+1, len(documents)))\n",
    "        \n",
    "        #tokenization\n",
    "        tokens = regex.findall(document)\n",
    "\n",
    "        #skipping useless tokens\n",
    "        t_regex = re.compile(r\"[^a-zA-Z]\")\n",
    "        document = []\n",
    "        \n",
    "        for token in tokens:\n",
    "            token = np.array(token)\n",
    "            token = np.unique(token[token != ''])\n",
    "            \n",
    "            if len(token) > 0:\n",
    "                token = token[0].lower()\n",
    "            else:\n",
    "                continue\n",
    "                \n",
    "            if re.search(t_regex, token) == None and token not in swords:\n",
    "                token = lemmatizer.lemmatize(token)\n",
    "                document.append(token)\n",
    "                \n",
    "        document = ' '.join(document)\n",
    "\n",
    "        #skipping\n",
    "        if len(document) > 0:\n",
    "            processed_documents.append(document)\n",
    "\n",
    "    print()\n",
    "    return np.array(processed_documents)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-06T18:46:32.418856Z",
     "start_time": "2018-04-06T18:46:32.402911Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 3 µs, sys: 1 µs, total: 4 µs\n",
      "Wall time: 7.87 µs\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "# posts = preprocess(posts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Processing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I am only using document matrix, using TF-IDF weighing scheme will actually result in model's performance degradation. I think the reason is because our document are relatively small in nature due to small size nature of tweets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-06T18:46:54.932980Z",
     "start_time": "2018-04-06T18:46:32.422438Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 21.5 s, sys: 395 ms, total: 21.9 s\n",
      "Wall time: 22.5 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "# creating document frequency matrix\n",
    "cv = CountVectorizer().fit(posts)\n",
    "X = cv.transform(posts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Splitting Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-06T18:46:55.694833Z",
     "start_time": "2018-04-06T18:46:54.936135Z"
    }
   },
   "outputs": [],
   "source": [
    "# splitting dataset into training and testing dataset\n",
    "xIETrain, xIETest, yIETrain, yIETest = train_test_split(X, yIE)\n",
    "xNSTrain, xNSTest, yNSTrain, yNSTest = train_test_split(X, yNS)\n",
    "xFTTrain, xFTTest, yFTTrain, yFTTest = train_test_split(X, yFT)\n",
    "xPJTrain, xPJTest, yPJTrain, yPJTest = train_test_split(X, yPJ)\n",
    "xTrain, xTest, yTrain, yTest = train_test_split(X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# EDA (Exploratory Data Analysis)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see that our dataset has disproportionate data. Due to this **heavy unbalance** in our dataset we will\n",
    "be training our classifiers independently for each characteristic pair. This unbalance makes it very difficult \n",
    "to work with different algorithms, specially probability based models. To look at this, refer to cells below in this section. For analysing the effect of this balance in machine learning algorithms using conventional approaches look at [result analysis section](#Analysing-our-Models-Results).\n",
    "\n",
    "This problem is called **Imbalanced Classification Problem**. Generally there are two approaches to tackle this kind of problem:\n",
    "* **Sampling**<br>\n",
    "    There are different sampling techniques to tackle this problem, we will be discussing few of them.\n",
    "    * *Random Sampling*\n",
    "    * *SMOTE*\n",
    "    * *Modified SMOTE*\n",
    "* **Ensemble Methods**\n",
    "\n",
    "I will be using type of *random sampling* technique called **undersampling**, In this approach we will randomly remove our sample instances from majority class until the size of majority class instances are equal to minority class instances."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-06T18:46:56.364334Z",
     "start_time": "2018-04-06T18:46:55.697058Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFP    1832\n",
      "INFJ    1470\n",
      "INTP    1304\n",
      "INTJ    1091\n",
      "ENTP     685\n",
      "ENFP     675\n",
      "ISTP     337\n",
      "ISFP     271\n",
      "ENTJ     231\n",
      "ISTJ     205\n",
      "ENFJ     190\n",
      "ISFJ     166\n",
      "ESTP      89\n",
      "ESFP      48\n",
      "ESFJ      42\n",
      "ESTJ      39\n",
      "Name: type, dtype: int64\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEMCAYAAADNtWEcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAHM9JREFUeJzt3X+wXGWd5/H3l0SRIQhxkFsxsCaz\nk50VjIDciVhTU94LI4noDDA7zMIwCIqT2SpwrZrMYrBWcUR+zKxoqYPWhiIDjDh34qpFhKwYkeC4\nKkKQIYQMS8QogWwiBiMBRAOf/aOfG5tL3+7T3c+5v57Pq6rrnvOccz799I/b3z7POd0dkjAzs/Ic\nMNkdMDOzyeECYGZWKBcAM7NCuQCYmRXKBcDMrFAuAGZmhXIBMDMrlAuAmVmhXADMzArlAmBmVqjZ\nk92Bdg4//HAtWLCg5+2ffvppDj744Gz9mep5dWSWlldH5lTPqyOztLw6MvvJ27hx4xOSXt1xRUlt\nL8ArgO8B/wpsBv4mtS8E7gIeBv4ZeHlqPzDNb03LFzRlXZLaHwKWdrruE044Qf244447+tp+uuXV\nkVlaXh2ZUz2vjszS8urI7CcPuEcdXl8lVRoCeg44SdKxwHHAsog4Efhb4BOSFgFPAhek9S8AnpT0\n28An0npExNHAWcAxwDLgMxExq8L1m5lZDToWgFRQ9qbZl6WLgJOA/5XabwBOT9OnpXnS8pMjIlL7\niKTnJP2Qxp7Akiy3wszMulbpIHBEzIqI+4BdwHrgB8DPJO1Lq2wH5qfp+cCjAGn5HuA3m9tbbGNm\nZhMs1MXvAUTEYcCXgQ8B/5CGeYiIo4B1khZHxGYa4/vb07If0Hin/xHgO5I+l9qvS9t8ccx1LAeW\nAwwMDJwwMjLS843bu3cvc+bM6Xn76ZZXR2ZpeXVkTvW8OjJLy6sjs5+84eHhjZIGO65Y5UBB8wW4\nFPhvwBPA7NT2ZuC2NH0b8OY0PTutFzQOAF/SlLN/vfEuPgg8+Zml5dWROdXz6sgsLa+OzClxEDgi\nXp3e+RMRBwF/AGwB7gD+JK12HnBzml6b5knLv5E6tBY4KyIOjIiFwCIaZxeZmdkkqPI5gHnADemM\nnQOANZJuiYgHgZGI+CjwfeC6tP51wD9GxFZgN40zf5C0OSLWAA8C+4ALJT2f9+aYmVlVHQuApPuB\n41u0P0KLs3gk/QI4c5ysy4HLu++mmZnlNqU/CWxmL7XpsT2cv/LWrJkrFu+rlLntqrdnvV6bXP4u\nIDOzQrkAmJkVygXAzKxQLgBmZoVyATAzK5QLgJlZoVwAzMwK5QJgZlYoFwAzs0K5AJiZFcoFwMys\nUC4AZmaFcgEwMyuUC4CZWaFcAMzMCuUCYGZWKBcAM7NCuQCYmRXKBcDMrFAuAGZmhXIBMDMrlAuA\nmVmhXADMzArlAmBmVigXADOzQnUsABFxVETcERFbImJzRLwvtX84Ih6LiPvS5dSmbS6JiK0R8VBE\nLG1qX5batkbEynpukpmZVTG7wjr7gBWS7o2IQ4CNEbE+LfuEpI81rxwRRwNnAccArwG+HhH/IS2+\nBngrsB24OyLWSnowxw0xM7PudCwAknYAO9L0UxGxBZjfZpPTgBFJzwE/jIitwJK0bKukRwAiYiSt\n6wJgZjYJujoGEBELgOOBu1LTRRFxf0Ssjoi5qW0+8GjTZttT23jtZmY2CUJStRUj5gB3ApdL+lJE\nDABPAAIuA+ZJendEXAN8R9Ln0nbXAetoFJulkt6T2s8Flkh675jrWQ4sBxgYGDhhZGSk5xu3d+9e\n5syZ0/P20y2vjszS8urIzJ23a/cedj6bLQ6AgYOolLl4/qGV8qb6fTjTnzfDw8MbJQ12Wq/KMQAi\n4mXAF4GbJH0JQNLOpuXXArek2e3AUU2bHwk8nqbHa99P0ipgFcDg4KCGhoaqdLGlDRs20M/20y2v\njszS8urIzJ336Ztu5upNlf51K1uxeF+lzG3nDFXKm+r3YYnPm1aqnAUUwHXAFkkfb2qf17TaGcAD\naXotcFZEHBgRC4FFwPeAu4FFEbEwIl5O40Dx2jw3w8zMulXlbcTvAecCmyLivtT2AeDsiDiOxhDQ\nNuAvASRtjog1NA7u7gMulPQ8QERcBNwGzAJWS9qc8baYmVkXqpwF9C0gWixa12aby4HLW7Sva7ed\nmZlNHH8S2MysUC4AZmaFcgEwMyuUC4CZWaFcAMzMCuUCYGZWKBcAM7NCuQCYmRXKBcDMrFAuAGZm\nhXIBMDMrlAuAmVmhXADMzArlAmBmVigXADOzQrkAmJkVygXAzKxQLgBmZoVyATAzK5QLgJlZoVwA\nzMwK5QJgZlYoFwAzs0K5AJiZFcoFwMysUC4AZmaF6lgAIuKoiLgjIrZExOaIeF9qf1VErI+Ih9Pf\nuak9IuJTEbE1Iu6PiDc2ZZ2X1n84Is6r72aZmVknVfYA9gErJL0OOBG4MCKOBlYCt0taBNye5gHe\nBixKl+XAZ6FRMIBLgTcBS4BLR4uGmZlNvI4FQNIOSfem6aeALcB84DTghrTaDcDpafo04EY1fBc4\nLCLmAUuB9ZJ2S3oSWA8sy3przMyssq6OAUTEAuB44C5gQNIOaBQJ4Ii02nzg0abNtqe28drNzGwS\nhKRqK0bMAe4ELpf0pYj4maTDmpY/KWluRNwKXCnpW6n9duBi4CTgQEkfTe0fBJ6RdPWY61lOY+iI\ngYGBE0ZGRnq+cXv37mXOnDk9bz/d8urILC2vjszcebt272Hns9niABg4iEqZi+cfWilvqt+HM/15\nMzw8vFHSYKf1ZlcJi4iXAV8EbpL0pdS8MyLmSdqRhnh2pfbtwFFNmx8JPJ7ah8a0bxh7XZJWAasA\nBgcHNTQ0NHaVyjZs2EA/20+3vDoyS8urIzN33qdvupmrN1X6161sxeJ9lTK3nTNUKW+q34clPm9a\nqXIWUADXAVskfbxp0Vpg9Eye84Cbm9rfmc4GOhHYk4aIbgNOiYi56eDvKanNzMwmQZW3Eb8HnAts\nioj7UtsHgKuANRFxAfBj4My0bB1wKrAVeAZ4F4Ck3RFxGXB3Wu8jknZnuRVmZta1jgUgjeXHOItP\nbrG+gAvHyVoNrO6mg2ZmVg9/EtjMrFAuAGZmhXIBMDMrlAuAmVmhXADMzArlAmBmVigXADOzQrkA\nmJkVygXAzKxQLgBmZoVyATAzK5QLgJlZoVwAzMwK5QJgZlYoFwAzs0K5AJiZFcoFwMysUC4AZmaF\ncgEwMyuUC4CZWaFcAMzMCuUCYGZWKBcAM7NCuQCYmRXKBcDMrFAuAGZmhXIBMDMrVMcCEBGrI2JX\nRDzQ1PbhiHgsIu5Ll1Obll0SEVsj4qGIWNrUviy1bY2IlflvipmZdaPKHsD1wLIW7Z+QdFy6rAOI\niKOBs4Bj0jafiYhZETELuAZ4G3A0cHZa18zMJsnsTitI+mZELKiYdxowIuk54IcRsRVYkpZtlfQI\nQESMpHUf7LrHZmaWRT/HAC6KiPvTENHc1DYfeLRpne2pbbx2MzObJCGp80qNPYBbJL0+zQ8ATwAC\nLgPmSXp3RFwDfEfS59J61wHraBSapZLek9rPBZZIem+L61oOLAcYGBg4YWRkpOcbt3fvXubMmdPz\n9tMtr47M0vLqyMydt2v3HnY+my0OgIGDqJS5eP6hlfKm+n040583w8PDGyUNdlqv4xBQK5J2jk5H\nxLXALWl2O3BU06pHAo+n6fHax2avAlYBDA4OamhoqJcuArBhwwb62X665dWRWVpeHZm58z59081c\nvamnf91xrVi8r1LmtnOGKuVN9fuwxOdNKz09iyJinqQdafYMYPQMobXA5yPi48BrgEXA94AAFkXE\nQuAxGgeK/6yfjptNtgUrb6203orF+zi/4rrV8rJFWeE6FoCI+CdgCDg8IrYDlwJDEXEcjSGgbcBf\nAkjaHBFraBzc3QdcKOn5lHMRcBswC1gtaXP2W2NmZpVVOQvo7BbN17VZ/3Lg8hbt62gcDzAzsynA\nnwQ2MyuUC4CZWaFcAMzMCuUCYGZWKBcAM7NCuQCYmRXKBcDMrFAuAGZmhXIBMDMrVN5vlDKzGW2y\nvv/o+mUHZ8uyX/MegJlZoVwAzMwK5QJgZlYoFwAzs0K5AJiZFcoFwMysUC4AZmaFcgEwMyuUC4CZ\nWaFcAMzMCuUCYGZWKBcAM7NCuQCYmRXKBcDMrFAuAGZmhXIBMDMrVMcCEBGrI2JXRDzQ1PaqiFgf\nEQ+nv3NTe0TEpyJia0TcHxFvbNrmvLT+wxFxXj03x8zMqqqyB3A9sGxM20rgdkmLgNvTPMDbgEXp\nshz4LDQKBnAp8CZgCXDpaNEwM7PJ0bEASPomsHtM82nADWn6BuD0pvYb1fBd4LCImAcsBdZL2i3p\nSWA9Ly0qZmY2gXo9BjAgaQdA+ntEap8PPNq03vbUNl67mZlNkpDUeaWIBcAtkl6f5n8m6bCm5U9K\nmhsRtwJXSvpWar8duBg4CThQ0kdT+weBZyRd3eK6ltMYPmJgYOCEkZGRnm/c3r17mTNnTs/bT7e8\nOjJLy+smc9NjeyrlDRwEO5/tt1f15dWRmTtv4aGzZszzZiLyhoeHN0oa7LTe7J7SYWdEzJO0Iw3x\n7Ert24GjmtY7Eng8tQ+Nad/QKljSKmAVwODgoIaGhlqtVsmGDRvoZ/vplldHZml53WSev/LWSnkr\nFu/j6k29/qvVn1dHZu6865cdPGOeN5OV10qvQ0BrgdEzec4Dbm5qf2c6G+hEYE8aIroNOCUi5qaD\nv6ekNjMzmyQdS3RE/BONd++HR8R2GmfzXAWsiYgLgB8DZ6bV1wGnAluBZ4B3AUjaHRGXAXen9T4i\naeyBZTMzm0AdC4Cks8dZdHKLdQVcOE7OamB1V70zM7Pa+JPAZmaFcgEwMyuUC4CZWaFcAMzMCuUC\nYGZWqLyfJpliNj22p/KHdapYsXhfpbxtV70923WamdXFewBmZoVyATAzK5QLgJlZoVwAzMwK5QJg\nZlYoFwAzs0K5AJiZFcoFwMysUC4AZmaFmtGfBC6RP/1sZlV5D8DMrFAuAGZmhXIBMDMrlAuAmVmh\nXADMzArlAmBmVigXADOzQrkAmJkVygXAzKxQLgBmZoVyATAzK1RfBSAitkXEpoi4LyLuSW2vioj1\nEfFw+js3tUdEfCoitkbE/RHxxhw3wMzMepPjy+CGJT3RNL8SuF3SVRGxMs2/H3gbsChd3gR8Nv01\nM5uyFlT8csWqX5xY1fXLDs6WNZ46hoBOA25I0zcApze136iG7wKHRcS8Gq7fzMwq6LcACPhaRGyM\niOWpbUDSDoD094jUPh94tGnb7anNzMwmQUjqfeOI10h6PCKOANYD7wXWSjqsaZ0nJc2NiFuBKyV9\nK7XfDlwsaeOYzOXAcoCBgYETRkZGeu7frt172Plsz5u/xMBBVMpbPP/QSnl79+5lzpw5ffbqxUq7\nzXXch1UzNz22p1Je1fuwqtx5dWTmzlt46KxJe95M1uPcz20eHh7eKGmw03p9HQOQ9Hj6uysivgws\nAXZGxDxJO9IQz660+nbgqKbNjwQeb5G5ClgFMDg4qKGhoZ779+mbbubqTfl+82bF4n2V8radM1Qp\nb8OGDfRz+1op7TbXcR9Wzaw63lv1Pqwqd14dmbnzrl928KQ9bybrcc59m1vpeQgoIg6OiENGp4FT\ngAeAtcB5abXzgJvT9FrgnelsoBOBPaNDRWZmNvH6KVcDwJcjYjTn85K+GhF3A2si4gLgx8CZaf11\nwKnAVuAZ4F19XLeZmfWp5wIg6RHg2BbtPwVObtEu4MJer8/MyjVZv3U90/mTwGZmhXIBMDMrlAuA\nmVmhXADMzArlAmBmVigXADOzQrkAmJkVygXAzKxQLgBmZoVyATAzK5QLgJlZoVwAzMwK5QJgZlYo\nFwAzs0Ll/VkhA2BBF78glPsraVcszhpnZjOY9wDMzArlAmBmVigXADOzQrkAmJkVygXAzKxQLgBm\nZoXyaaA2rW16bE8Np9LmPz3XbCryHoCZWaFcAMzMCuUCYGZWKBcAM7NCTXgBiIhlEfFQRGyNiJUT\nff1mZtYwoWcBRcQs4BrgrcB24O6IWCvpwYnsh+U3WV+A5y+/M+vdRO8BLAG2SnpE0i+BEeC0Ce6D\nmZkx8QVgPvBo0/z21GZmZhMsJE3clUWcCSyV9J40fy6wRNJ7m9ZZDixPs78DPNTHVR4OPNHH9tMt\nr47M0vLqyJzqeXVklpZXR2Y/ea+V9OpOK030J4G3A0c1zR8JPN68gqRVwKocVxYR90gazJE1HfLq\nyCwtr47MqZ5XR2ZpeXVk1tHHsSZ6COhuYFFELIyIlwNnAWsnuA9mZsYE7wFI2hcRFwG3AbOA1ZI2\nT2QfzMysYcK/DE7SOmDdBF1dlqGkaZRXR2ZpeXVkTvW8OjJLy6sjs44+vsiEHgQ2M7Opw18FYWZW\nKBcAM7NCuQCYmRXKvwjWQUQcLin3B0ayiYjjgX8PbJa0pc+sN7ZZ/BzwY0lPdZG3CPhY6t8m4K8l\nPdZPH+uS63GOiFe1WfycpKe7zDsC+ADw2zTuwysl/byPLhIRf9xm8XPAI/0+l6aaiDiddB9Kui1D\n3pR/nCtd70w5CBwRf9Vm8XPAD4CvSXqhYt4fAquBfcDzwJ9K+nYf/fsKMN6dPdq/ayQ9Os46rTI/\nBPw5sBF4E40nzbV99PGONotnA/8u9fHvKub9C3Aj8E3gj4A3S2r34tMpL+tjnDJzP84/pPE4R4vF\no2+4Vkq6qWLeV2k8vt8E3gEcIun8XvuXMv+hzeLZwOuAb0v6rxXz3kTjjJXRQn9BP1/wGBGb6Py/\ncqWkf62Y9xngGODbwMnAVyRd1mv/UuaUf5wrXe8MKgCXtlk8m8YTYJ+kP62Ydz+NF4N/S0/wv5P0\nlj76127b0f6dLenNXWRuBn5X0jMR8ZvAVyX9bq99rHB9BwLfl3R0xfXvk3Rc0/y9ktrtZXTKy/oY\np8ysj3OF63s1cOdk3Ycp448lfanN8gNovFM+pmLePcAl/LrQv0fS0j7699o2i2cDrwc+LOn4inkP\nAMdKej4ifgP4F0kn9Nq/0T5K+lGb5ZP+OFcxY4aAJP1Np3XSP3tV+yT9W8q+KyIO6blzDe/qUNFv\nj4g3dJn5C0nPAEj6afrH7VlEXCHpA2n6rZLWNy+X9Fz6/qaqXpGGqEbfJR3UPC/p3i67+FNJf99u\nhS4fY8j8OEfERaN9jIhjxn7QUdJPIuL93UXGXH59H85qnpe0u4du/ndg3AIg6YWI+IMu8g5oeq58\nISIu6aFPzeZJ+m6b5T/oMFw51i8lPQ+Q3iy1etferS8D4/ZhijzOna90Bu0BfE3SKWn6EklX9pm3\nHfh4U9NfNc9L+vhLNmqfl72iR8TPaLzrgsYT5feb5pH0R13m7e9jpneaGxh/V16STuq1f7nU+Thn\nug+3AS/QeqhBkn6rh8ys92NEPAL8dVPTx5rn2+1tjJPXfB9+p5u94nHyngG2js7SGKramqYlqds3\nXkTE96vugVTM20bmx7mKGbMHADR/892ZQF8FALgWOKTNfLd+Y8y74Rfp4d0wvPS3FD7WQ0ZtJA1N\ndh8qyP04N+v7naakBRn6MdZ/HGdPqdcXxDuBPxxnXrTZ2xhH8/32ii63beV1GTLGmh8RnxpvYdXj\nJ03e0m5IqS4zqQBk3ZWpMqTUpfnA1YxT4YGu3g0nnYaVunVEOtAaTdP79fBuuO2QUg/eEBGtzowY\nfeF6ZQ+ZHYeVunRYRJxB4xTrV44946aHd8Nth5R69ENe/ILdr690e7s6OCANfxzQNL3//6aH4ZBO\nQ0q9eJbGQdtc2g4p1WUmDQGNDoe8ZCgEehoOyT2klHWXMWXm3pVvd5C166JYw3DIdLgP251hI0nv\n7jIv632YcnIPX+S+D7eRcTgk95DS2Mwc6nhuVzGT9gCah0NyDIXkHlKqQ9ZhpRr2eooj6V01xuc4\neAnwfzLl1KKGYa/cQ0oAv8yUMyr3kFIlM6YASLozd2TmvIsz50HmYaWI+Atgg6SH05kS1wH/CfgR\ncJ6k73fZv6xDSsAXuly/iqzDShHxzjaLJekfu+pd5iGl5O52/ZR0Y5d5WY8ppNNAfyZpT5ofBk4H\nttH4HEq3L765h5QAzoqIQ1v08UfA3/fQx9xDSpXMpCGgO2h/xsnJXeblHlLK2r+UmXtX/gHgeEm/\niog/A1YApwDHA5dK+v0u83IPKY0tUKtpFKhtwPm9HEiv4T78dKtmGmPu8yV19aYr95BSyszdx83A\nqeMt7/bgZkTcBZwh6fGIOA74Oo098DcAv1L6Sdku8raR/0yq3H2ckPP+x5oxewC8+DS0USfSeOe9\nq4e83ENKuftXh32SfpWm3wHcKOmnwNcjotKnf5vVMKT0PuD6NH02jX+2hTQK1CdpFOpJpRf/vnUA\n5wDvB74LXN5DXvYhpdx9pHGefc4zWA6SNPpTsX9O44ejrk6fc7mv27CazqTK2kfyDylVMmMKgKT9\nu0/R+NTtB4EDgf8i6X/3kJd1SCl3/5Lcw0ovRMQ84EkaH5lvfjE4qNuwGoaUshaoJPuwUkTMBs6n\nsQd1F/Ankh7qMSv3kNJobrY+kv+YQvM79ZNofMp49ANq3YflH1LK3kfyDylVMmMKAEBELKXxwvoL\n4HJJ7b7bplNWHUM22fqXfKDNpy576eOHgHto/Fzn2tFTDlPBeqSH/o19x34s8Fs03rF/iu7fsWct\nUMlPImJRrmGliLiQxu2+HViW4Z1xq6/22D9cA3RdAGroY+5jCt+IiDXADmAu8A2A9Nj38kK4BjgD\n2JOGa75AY7jmOOAzQFfDNTX18Z/H6eOxffSxo5l0DOBuGmfu/A/gO2OX9/CP3Oq7QvYP2ajL79zJ\n3b86+pgyZ9P4Iqonm9oOpvFc2dtl1v7vN4mIzwN3Sfpkmu96zDMi3gH8TxoF6iuS/iK1vwW4WNLb\nu8lL2+Y+7vECjSG9n/DiNxA9f+q0Kbt5uOZBGm8iuv3qi+x9rOGYQgD/GZgHrFH6Btl0xtsR6vLb\nPCPi/tHbFBEfA16QdPHocE0vj8l06GOl651BBWADGb92YEx285DNFb0M2dTZv5Sfo48XK33TZ0Sc\nKekLTcv2f6iri7x7gbfTeMf+I+Ckpr2KLZK6/oRmzgKVts1dpNp9kVnXB0RT5tjhmiv7GK6ppY9N\n2VmKVE4RsUnS4jR9L3DJ6At08wvvZJq0PkryZZwLsBT4Fo0j/MOT3Z+6+wjc22q61XzFvHcAjwH/\nD7i2qf0twK095F3cNH3mmGVX9HqbabyLewWwEzimadmWKfD4Xgj8X+CzwGsnuz9t+jmbxjDFFhrD\nfr/TR9ZTwM9bXJ4Cft5D3idpDAN9ksanoF+W2ucB98zUPla5zKQ9gNzvXnMPKWXtX0193H9K5NjT\nI3s9XTLzkNK4n4rt9TS63MNKEfEUrff0ev1cQfYhpRr62HxM4SpNwnfatJN7uKYOk9XHmVQAsr44\n5B6yqenFa0r3sYainL1ApW2zDivlVOdwTS51Hvewes2ks4BinOlW8x0p/zdZZu0f1NLHY6Pxqdig\n8d39o5+QDXr7CP1ZwOjpmZfw4lMul9H4CbxuaJzpVvOVNBWpJ5uLlKSnI+KKHvqY1VR4ga9g4WR3\noJ3cezx1mKw+zqQfhc/64hARFzdNnzlm2RXd5lHTi1fTdN99lDRL0islHSJpdpoenX9ZL10cZ7rV\nfBXHRsTP0z/LG9L06PziHvKgUaRGjT2ldlmPmdlExFNNt7P58lS0/gqLCSfpR+0uU6B/hzQ9l185\n5nk96S/+MHl9nEkFIPeLQ+4XhuJevMhc9GooUFDDnllO0+HFazoUKWttxgwBSZqVOTL3kFLu/o3t\nx5R78SL/kFIdsu+ZlUZSrh/QsQk2YwpADabDC8OU7mNNRS+36VCkzGoxY84Cyi0ingeeJr0wAM+M\nLgJe0ceQQzbToY9mNnW5AJiZFWomHQQ2M7MuuACYmRXKBcDMrFAuAGZmhXIBMDMr1P8HqAwEn4jf\nTwYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a0d725cf8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(df.type.value_counts())\n",
    "df.type.hist(xrot=90)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In cell below we can see that there is great **unbalance** in **Introvert/Extrovert** and **Intuition/Sensing** pairs. Whereas **Feeling/Thinking** and **Perception/Judgment** pairs are quite *balanced*. Although I have created trained models for each pair. Only last 2 pairs are somewhat reliable in predicting MBTI type. So it is\n",
    "not advised to depend on first 2 pairs i.e. **IE** and **NS** pairs. For analysis of results of various trained models is present below in [result analysis section](#Analysing-our-Models-Results)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-06T18:46:57.071416Z",
     "start_time": "2018-04-06T18:46:56.368635Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I    6676\n",
      "E    1999\n",
      "Name: ie, dtype: int64\n",
      "\n",
      "N    7478\n",
      "S    1197\n",
      "Name: ns, dtype: int64\n",
      "\n",
      "F    4694\n",
      "T    3981\n",
      "Name: ft, dtype: int64\n",
      "\n",
      "P    5241\n",
      "J    3434\n",
      "Name: pj, dtype: int64\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAEaZJREFUeJzt3V+MnfV95/H3p7i0WbONTSgjy7YW\nVrW6pUJJ2RGwirQawq4x7KrmomiJosUgS+4FG7USUuvsDVpoJCKlmw3RNpJVvJiKLUXpRrYKCjsi\nOap6ASE0FEpo5ClN8axdvO04dCdsUzn73Yv5uTkMM8wZz8yZiN/7JY3O83zP9/f8uXk+5/md59ip\nKiRJ/fmxzT4ASdLmMAAkqVMGgCR1ygCQpE4ZAJLUKQNAkjq1YgAk+dkkLw39/W2SX01yeZLpJCfb\n6/bWnyQPJ5lJ8nKS64a2daD1n0xyYCNPTJL03rKa3wEkuQT4X8ANwL3AXFU9lOQwsL2qfj3JbcAn\ngdta3+er6oYklwPfACaBAl4E/nlVnVvXM5IkjWS1U0A3A39eVX8J7AeOtfox4Pa2vB94rBY8B2xL\nsgO4BZiuqrl20Z8G9q35DCRJF2XLKvvvBH63LU9U1RmAqjqT5MpW3wmcGhoz22rL1Zd1xRVX1FVX\nXbXKQ/yh733ve2zduvWix0vSZlnL9evFF1/866r66ZX6Rg6AJJcCvwh8aqXWJWr1HvXF+zkEHAKY\nmJjgs5/97KiH+C7z8/NcdtllFz1ekjbLWq5fN91001+O0reaO4BbgT+uqjfb+ptJdrRP/zuAs60+\nC+weGrcLON3qU4vqg8U7qaojwBGAycnJmpqaWtwyssFgwFrGS9JmGcf1azXfAXycH07/AJwALjzJ\ncwA4PlS/qz0NdCPwVpsqegbYm2R7e2Job6tJkjbBSHcASf4R8K+BXx4qPwQ8meQg8AZwR6s/zcIT\nQDPA28A9AFU1l+RB4IXW90BVza35DCRJF2WkAKiqt4EPLar9DQtPBS3uLRYeEV1qO0eBo6s/TEnS\nevOXwJLUKQNAkjplAEhSpwwASeqUASBJnVrtPwUhSd246vBTm7bvR/dt/D9j4x2AJHXKAJCkThkA\nktQpA0CSOmUASFKnDABJ6pQBIEmdMgAkqVMGgCR1ygCQpE4ZAJLUKQNAkjplAEhSpwwASeqUASBJ\nnTIAJKlTIwVAkm1JvpTkz5K8luRfJLk8yXSSk+11e+tNkoeTzCR5Ocl1Q9s50PpPJjmwUSclSVrZ\nqHcAnwe+UlX/DPgw8BpwGHi2qvYAz7Z1gFuBPe3vEPBFgCSXA/cDNwDXA/dfCA1J0vitGABJfgr4\nl8AjAFX191X1XWA/cKy1HQNub8v7gcdqwXPAtiQ7gFuA6aqaq6pzwDSwb13PRpI0slHuAP4p8L+B\n/5bkm0l+O8lWYKKqzgC01ytb/07g1ND42VZbri5J2gSj/KfwW4DrgE9W1fNJPs8Pp3uWkiVq9R71\ndw5ODrEwdcTExASDwWCEQ1za/Pz8msZL6tt9157ftH2P4/o1SgDMArNV9Xxb/xILAfBmkh1VdaZN\n8Zwd6t89NH4XcLrVpxbVB4t3VlVHgCMAk5OTNTU1tbhlZIPBgLWMl9S3uw8/tWn7fnTf1g2/fq04\nBVRVfwWcSvKzrXQz8C3gBHDhSZ4DwPG2fAK4qz0NdCPwVpsiegbYm2R7+/J3b6tJkjbBKHcAAJ8E\nHk9yKfA6cA8L4fFkkoPAG8Adrfdp4DZgBni79VJVc0keBF5ofQ9U1dy6nIUkadVGCoCqegmYXOKt\nm5foLeDeZbZzFDi6mgOUJG0MfwksSZ0yACSpUwaAJHXKAJCkThkAktQpA0CSOmUASFKnDABJ6pQB\nIEmdMgAkqVMGgCR1ygCQpE4ZAJLUKQNAkjplAEhSpwwASeqUASBJnTIAJKlTBoAkdcoAkKROGQCS\n1CkDQJI6NVIAJPlOkleSvJTkG612eZLpJCfb6/ZWT5KHk8wkeTnJdUPbOdD6TyY5sDGnJEkaxWru\nAG6qqo9U1WRbPww8W1V7gGfbOsCtwJ72dwj4IiwEBnA/cANwPXD/hdCQJI3fWqaA9gPH2vIx4Pah\n+mO14DlgW5IdwC3AdFXNVdU5YBrYt4b9S5LWYNQAKOB/JnkxyaFWm6iqMwDt9cpW3wmcGho722rL\n1SVJm2DLiH0frarTSa4EppP82Xv0ZolavUf9nYMXAuYQwMTEBIPBYMRDfLf5+fk1jZfUt/uuPb9p\n+x7H9WukAKiq0+31bJIvszCH/2aSHVV1pk3xnG3ts8DuoeG7gNOtPrWoPlhiX0eAIwCTk5M1NTW1\nuGVkg8GAtYyX1Le7Dz+1aft+dN/WDb9+rTgFlGRrkn98YRnYC/wpcAK48CTPAeB4Wz4B3NWeBroR\neKtNET0D7E2yvX35u7fVJEmbYJQ7gAngy0ku9P/3qvpKkheAJ5McBN4A7mj9TwO3ATPA28A9AFU1\nl+RB4IXW90BVza3bmUiSVmXFAKiq14EPL1H/G+DmJeoF3LvMto4CR1d/mJKk9eYvgSWpUwaAJHXK\nAJCkThkAktQpA0CSOmUASFKnDABJ6pQBIEmdMgAkqVMGgCR1ygCQpE4ZAJLUKQNAkjplAEhSpwwA\nSeqUASBJnTIAJKlTBoAkdcoAkKROGQCS1CkDQJI6ZQBIUqdGDoAklyT5ZpI/aOtXJ3k+yckkv5fk\n0lb/ibY+096/amgbn2r1bye5Zb1PRpI0utXcAfwK8NrQ+meAz1XVHuAccLDVDwLnqupngM+1PpJc\nA9wJ/DywD/itJJes7fAlSRdrpABIsgv4N8Bvt/UAHwO+1FqOAbe35f1tnfb+za1/P/BEVX2/qv4C\nmAGuX4+TkCSt3qh3AP8F+DXg/7X1DwHfrarzbX0W2NmWdwKnANr7b7X+f6gvMUaSNGZbVmpI8m+B\ns1X1YpKpC+UlWmuF995rzPD+DgGHACYmJhgMBisd4rLm5+fXNF5S3+679vzKTRtkHNevFQMA+Cjw\ni0luA34S+CkW7gi2JdnSPuXvAk63/llgNzCbZAvwQWBuqH7B8Jh/UFVHgCMAk5OTNTU1dRGntWAw\nGLCW8ZL6dvfhpzZt34/u27rh168Vp4Cq6lNVtauqrmLhS9yvVtUngK8Bv9TaDgDH2/KJtk57/6tV\nVa1+Z3tK6GpgD/D1dTsTSdKqjHIHsJxfB55I8hvAN4FHWv0R4HeSzLDwyf9OgKp6NcmTwLeA88C9\nVfWDNexfkrQGqwqAqhoAg7b8Oks8xVNVfwfcscz4TwOfXu1BSpLWn78ElqROGQCS1CkDQJI6ZQBI\nUqcMAEnqlAEgSZ0yACSpUwaAJHXKAJCkThkAktQpA0CSOmUASFKnDABJ6pQBIEmdMgAkqVMGgCR1\nygCQpE4ZAJLUKQNAkjplAEhSpwwASeqUASBJnVoxAJL8ZJKvJ/mTJK8m+U+tfnWS55OcTPJ7SS5t\n9Z9o6zPt/auGtvWpVv92kls26qQkSSsb5Q7g+8DHqurDwEeAfUluBD4DfK6q9gDngIOt/yBwrqp+\nBvhc6yPJNcCdwM8D+4DfSnLJep6MJGl0KwZALZhvqz/e/gr4GPClVj8G3N6W97d12vs3J0mrP1FV\n36+qvwBmgOvX5SwkSas20ncASS5J8hJwFpgG/hz4blWdby2zwM62vBM4BdDefwv40HB9iTGSpDHb\nMkpTVf0A+EiSbcCXgZ9bqq29Zpn3lqu/Q5JDwCGAiYkJBoPBKIe4pPn5+TWNl9S3+649v3LTBhnH\n9WukALigqr6bZADcCGxLsqV9yt8FnG5ts8BuYDbJFuCDwNxQ/YLhMcP7OAIcAZicnKypqanVHOI7\nDAYD1jJeUt/uPvzUpu370X1bN/z6NcpTQD/dPvmT5APAvwJeA74G/FJrOwAcb8sn2jrt/a9WVbX6\nne0poauBPcDX1+tEJEmrM8odwA7gWHti58eAJ6vqD5J8C3giyW8A3wQeaf2PAL+TZIaFT/53AlTV\nq0meBL4FnAfubVNLkqRNsGIAVNXLwC8sUX+dJZ7iqaq/A+5YZlufBj69+sOUJK03fwksSZ0yACSp\nUwaAJHXKAJCkThkAktQpA0CSOmUASFKnDABJ6pQBIEmdMgAkqVMGgCR1ygCQpE4ZAJLUKQNAkjpl\nAEhSpwwASeqUASBJnTIAJKlTBoAkdcoAkKROGQCS1CkDQJI6tWIAJNmd5GtJXkvyapJfafXLk0wn\nOdlet7d6kjycZCbJy0muG9rWgdZ/MsmBjTstSdJKRrkDOA/cV1U/B9wI3JvkGuAw8GxV7QGebesA\ntwJ72t8h4IuwEBjA/cANwPXA/RdCQ5I0fisGQFWdqao/bsv/B3gN2AnsB461tmPA7W15P/BYLXgO\n2JZkB3ALMF1Vc1V1DpgG9q3r2UiSRraq7wCSXAX8AvA8MFFVZ2AhJIArW9tO4NTQsNlWW64uSdoE\nW0ZtTHIZ8PvAr1bV3yZZtnWJWr1HffF+DrEwdcTExASDwWDUQ3yXs3Nv8YXHj1/0+It17c4Pjn2f\nktbffdee37R9z8/Pr+n6N4qRAiDJj7Nw8X+8qv5HK7+ZZEdVnWlTPGdbfRbYPTR8F3C61acW1QeL\n91VVR4AjAJOTkzU1NbW4ZWRfePw4v/nKyBm3br7ziamx71PS+rv78FObtu9H921lLde/UYzyFFCA\nR4DXquo/D711ArjwJM8B4PhQ/a72NNCNwFttiugZYG+S7e3L372tJknaBKN8PP4o8O+BV5K81Gr/\nEXgIeDLJQeAN4I723tPAbcAM8DZwD0BVzSV5EHih9T1QVXPrchaSpFVbMQCq6o9Yev4e4OYl+gu4\nd5ltHQWOruYAJUkbw18CS1KnDABJ6pQBIEmdMgAkqVMGgCR1ygCQpE4ZAJLUKQNAkjplAEhSpwwA\nSeqUASBJnTIAJKlTBoAkdcoAkKROGQCS1CkDQJI6ZQBIUqcMAEnqlAEgSZ0yACSpUwaAJHXKAJCk\nTq0YAEmOJjmb5E+HapcnmU5ysr1ub/UkeTjJTJKXk1w3NOZA6z+Z5MDGnI4kaVSj3AE8CuxbVDsM\nPFtVe4Bn2zrArcCe9ncI+CIsBAZwP3ADcD1w/4XQkCRtjhUDoKr+EJhbVN4PHGvLx4Dbh+qP1YLn\ngG1JdgC3ANNVNVdV54Bp3h0qkqQxutjvACaq6gxAe72y1XcCp4b6ZlttubokaZNsWeftZYlavUf9\n3RtIDrEwfcTExASDweCiD2biA3DftecvevzFWssxS/rRsRnXjwvm5+c3/FpysQHwZpIdVXWmTfGc\nbfVZYPdQ3y7gdKtPLaoPltpwVR0BjgBMTk7W1NTUUm0j+cLjx/nNV9Y741b2nU9MjX2fktbf3Yef\n2rR9P7pvK2u5/o3iYqeATgAXnuQ5ABwfqt/Vnga6EXirTRE9A+xNsr19+bu31SRJm2TFj8dJfpeF\nT+9XJJll4Wmeh4AnkxwE3gDuaO1PA7cBM8DbwD0AVTWX5EHghdb3QFUt/mJZkjRGKwZAVX18mbdu\nXqK3gHuX2c5R4Oiqjk6StGH8JbAkdcoAkKROGQCS1CkDQJI6ZQBIUqcMAEnqlAEgSZ0yACSpUwaA\nJHXKAJCkThkAktQpA0CSOmUASFKnDABJ6pQBIEmdMgAkqVMGgCR1ygCQpE4ZAJLUKQNAkjplAEhS\npwwASerU2AMgyb4k304yk+TwuPcvSVow1gBIcgnwX4FbgWuAjye5ZpzHIElaMO47gOuBmap6var+\nHngC2D/mY5AkMf4A2AmcGlqfbTVJ0phtGfP+skSt3tGQHAIOtdX5JN9ew/6uAP56DeMvSj4z7j1K\ner+56TNrun79k1Gaxh0As8DuofVdwOnhhqo6AhxZj50l+UZVTa7HtiRpnMZx/Rr3FNALwJ4kVye5\nFLgTODHmY5AkMeY7gKo6n+Q/AM8AlwBHq+rVcR6DJGnBuKeAqKqngafHtLt1mUqSpE2w4devVNXK\nXZKk9x3/KQhJ6tTYp4A2WpIfAK8MlZ6oqoc263gkabWSzFfVZRu9n/ddAAD/t6o+stkHIUk/6pwC\nkqROvR8D4ANJXhr6+3ebfUCS9KPIKSBJ6tT78Q5AkjQCA0CSOvV+nAL6QJKXhta/UlX+z2OStIi/\nBJakTjkFJEmdMgAkqVMGgCR1ygCQpE4ZAJLUKQNAkjplAEhSpwwASerU/wcWetcNFhv2hAAAAABJ\nRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a0d76f780>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAEotJREFUeJzt3X+s3fV93/Hnq7i0jKyxCeXKsq2Z\nqVZWKhZKr4Aq0nYbOmNoV6MpSERoOMiS9wft2olpI9sf7qCRkj9YVtCCZhW3pqIlKFtkq0WllpOj\naX9AIAnDJZTZpQTu7EFTE7IbVCp37/1xPm4O5tr33F/nRv48H9LV+X7f5/39fj/ff76vcz7ne85N\nVSFJ6s8PrfUAJElrwwCQpE4ZAJLUKQNAkjplAEhSpwwASeqUASBJnTIAJKlTBoAkdWrdWg/gfC6/\n/PLaunXrkrf/3ve+x6WXXrpyA5KkCVnO9etrX/vat6vqxxfq+4EOgK1bt/Lcc88tefvBYMDMzMzK\nDUiSJmQ5168k3xqnzykgSeqUASBJnTIAJKlTBoAkdcoAkKROGQCS1CkDQJI6ZQBIUqcMAEnq1A/0\nN4GX6+j/fptP3vuHEz/uq5/5hYkfU5IWy3cAktQpA0CSOmUASFKnDABJ6pQBIEmdMgAkqVMGgCR1\nygCQpE4tGABJPpzk+ZG/7yb5tSSXJTmc5Fh73ND6k+TBJMeTvJDk2pF97Wr9x5LsWs0TkySd34IB\nUFUvV9U1VXUN8DPAO8CXgHuBI1W1DTjS1gFuBra1vz3AwwBJLgP2AtcD1wF7z4SGJGnyFjsFdCPw\nZ1X1LWAncKDVDwC3tuWdwKM19DSwPslG4CbgcFWdqqq3gMPAjmWfgSRpSRYbALcDv9+Wp6rqJEB7\nvKLVNwGvj2wz22rnqkuS1sDYPwaX5GLgl4BPLdQ6T63OUz/7OHsYTh0xNTXFYDAYd4jvM3UJ3HP1\n6SVvv1TLGbMkAczNza36tWQxvwZ6M/D1qnqjrb+RZGNVnWxTPG+2+iywZWS7zcCJVp85qz44+yBV\ntQ/YBzA9PV0zMzNnt4ztoccO8sDRyf/g6at3zEz8mJIuLIPBgOVc/8axmCmgT/D96R+AQ8CZO3l2\nAQdH6ne2u4FuAN5uU0RPAduTbGgf/m5vNUnSGhjr5XGSvwP8E+BfjJQ/AzyRZDfwGnBbqz8J3AIc\nZ3jH0F0AVXUqyf3As63vvqo6tewzkCQtyVgBUFXvAB86q/aXDO8KOru3gLvPsZ/9wP7FD1OStNL8\nJrAkdcoAkKROGQCS1CkDQJI6ZQBIUqcMAEnqlAEgSZ0yACSpUwaAJHXKAJCkThkAktQpA0CSOmUA\nSFKnDABJ6pQBIEmdMgAkqVMGgCR1ygCQpE4ZAJLUqbECIMn6JF9M8qdJXkrys0kuS3I4ybH2uKH1\nJsmDSY4neSHJtSP72dX6jyXZtVonJUla2LjvAH4T+KOq+gfAR4CXgHuBI1W1DTjS1gFuBra1vz3A\nwwBJLgP2AtcD1wF7z4SGJGnyFgyAJD8G/CPgEYCq+uuq+g6wEzjQ2g4At7blncCjNfQ0sD7JRuAm\n4HBVnaqqt4DDwI4VPRtJ0tjGeQfw94G/AH47yTeS/FaSS4GpqjoJ0B6vaP2bgNdHtp9ttXPVJUlr\nYN2YPdcCv1JVzyT5Tb4/3TOfzFOr89Tfu3Gyh+HUEVNTUwwGgzGGOL+pS+Ceq08vefulWs6YJQlg\nbm5u1a8l4wTALDBbVc+09S8yDIA3kmysqpNtiufNkf4tI9tvBk60+sxZ9cHZB6uqfcA+gOnp6ZqZ\nmTm7ZWwPPXaQB46Oc4or69U7ZiZ+TEkXlsFgwHKuf+NYcAqoqv4P8HqSD7fSjcA3gUPAmTt5dgEH\n2/Ih4M52N9ANwNttiugpYHuSDe3D3+2tJklaA+O+PP4V4LEkFwOvAHcxDI8nkuwGXgNua71PArcA\nx4F3Wi9VdSrJ/cCzre++qjq1ImchSVq0sQKgqp4Hpud56sZ5egu4+xz72Q/sX8wAJUmrw28CS1Kn\nDABJ6pQBIEmdMgAkqVMGgCR1ygCQpE4ZAJLUKQNAkjplAEhSpwwASeqUASBJnTIAJKlTBoAkdcoA\nkKROGQCS1CkDQJI6ZQBIUqcMAEnqlAEgSZ0aKwCSvJrkaJLnkzzXapclOZzkWHvc0OpJ8mCS40le\nSHLtyH52tf5jSXatzilJksaxmHcAP1dV11TVmX8Ofy9wpKq2AUfaOsDNwLb2twd4GIaBAewFrgeu\nA/aeCQ1J0uQtZwpoJ3CgLR8Abh2pP1pDTwPrk2wEbgIOV9WpqnoLOAzsWMbxJUnLsG7MvgL+OEkB\n/6Wq9gFTVXUSoKpOJrmi9W4CXh/ZdrbVzlV/jyR7GL5zYGpqisFgMP7ZnGXqErjn6tNL3n6pljNm\nSQKYm5tb9WvJuAHw0ao60S7yh5P86Xl6M0+tzlN/b2EYLvsApqena2ZmZswhvt9Djx3kgaPjnuLK\nefWOmYkfU9KFZTAYsJzr3zjGmgKqqhPt8U3gSwzn8N9oUzu0xzdb+yywZWTzzcCJ89QlSWtgwQBI\ncmmSv3tmGdgO/AlwCDhzJ88u4GBbPgTc2e4GugF4u00VPQVsT7Khffi7vdUkSWtgnPmRKeBLSc70\n/15V/VGSZ4EnkuwGXgNua/1PArcAx4F3gLsAqupUkvuBZ1vffVV1asXORJK0KAsGQFW9Anxknvpf\nAjfOUy/g7nPsaz+wf/HDlCStNL8JLEmdMgAkqVMGgCR1ygCQpE4ZAJLUKQNAkjplAEhSpwwASeqU\nASBJnTIAJKlTBoAkdcoAkKROGQCS1CkDQJI6ZQBIUqcMAEnqlAEgSZ0yACSpU2MHQJKLknwjyR+0\n9SuTPJPkWJIvJLm41X+krR9vz28d2cenWv3lJDet9MlIksa3mHcAvwq8NLL+WeBzVbUNeAvY3eq7\ngbeq6ieAz7U+klwF3A78FLAD+HySi5Y3fEnSUo0VAEk2A78A/FZbD/Ax4Iut5QBwa1ve2dZpz9/Y\n+ncCj1fVu1X158Bx4LqVOAlJ0uKN+w7gPwH/Bvh/bf1DwHeq6nRbnwU2teVNwOsA7fm3W//f1ufZ\nRpI0YesWakjyi8CbVfW1JDNnyvO01gLPnW+b0ePtAfYATE1NMRgMFhriOU1dAvdcfXrhxhW2nDFL\nEsDc3NyqX0sWDADgo8AvJbkF+FHgxxi+I1ifZF17lb8ZONH6Z4EtwGySdcAHgVMj9TNGt/lbVbUP\n2AcwPT1dMzMzSzitoYceO8gDR8c5xZX16h0zEz+mpAvLYDBgOde/cSw4BVRVn6qqzVW1leGHuF+u\nqjuArwAfb227gINt+VBbpz3/5aqqVr+93SV0JbAN+OqKnYkkaVGW8/L43wKPJ/kN4BvAI63+CPC7\nSY4zfOV/O0BVvZjkCeCbwGng7qr6m2UcX5K0DIsKgKoaAIO2/Arz3MVTVX8F3HaO7T8NfHqxg5Qk\nrTy/CSxJnTIAJKlTBoAkdcoAkKROGQCS1CkDQJI6ZQBIUqcMAEnqlAEgSZ0yACSpUwaAJHXKAJCk\nThkAktQpA0CSOmUASFKnDABJ6pQBIEmdMgAkqVMGgCR1asEASPKjSb6a5H8meTHJf2j1K5M8k+RY\nki8kubjVf6StH2/Pbx3Z16da/eUkN63WSUmSFjbOO4B3gY9V1UeAa4AdSW4APgt8rqq2AW8Bu1v/\nbuCtqvoJ4HOtjyRXAbcDPwXsAD6f5KKVPBlJ0vgWDIAammurP9z+CvgY8MVWPwDc2pZ3tnXa8zcm\nSas/XlXvVtWfA8eB61bkLCRJizbWZwBJLkryPPAmcBj4M+A7VXW6tcwCm9ryJuB1gPb828CHRuvz\nbCNJmrB14zRV1d8A1yRZD3wJ+Mn52tpjzvHcuervkWQPsAdgamqKwWAwzhDnNXUJ3HP16YUbV9hy\nxixJAHNzc6t+LRkrAM6oqu8kGQA3AOuTrGuv8jcDJ1rbLLAFmE2yDvggcGqkfsboNqPH2AfsA5ie\nnq6ZmZnFDPE9HnrsIA8cXdQprohX75iZ+DElXVgGgwHLuf6NY5y7gH68vfInySXAzwMvAV8BPt7a\ndgEH2/Khtk57/stVVa1+e7tL6EpgG/DVlToRSdLijPPyeCNwoN2x80PAE1X1B0m+CTye5DeAbwCP\ntP5HgN9NcpzhK//bAarqxSRPAN8ETgN3t6klSdIaWDAAquoF4Kfnqb/CPHfxVNVfAbedY1+fBj69\n+GFKklaa3wSWpE4ZAJLUKQNAkjplAEhSpwwASeqUASBJnTIAJKlTBoAkdcoAkKROGQCS1CkDQJI6\nZQBIUqcMAEnqlAEgSZ0yACSpUwaAJHXKAJCkThkAktQpA0CSOrVgACTZkuQrSV5K8mKSX231y5Ic\nTnKsPW5o9SR5MMnxJC8kuXZkX7ta/7Eku1bvtCRJCxnnHcBp4J6q+kngBuDuJFcB9wJHqmobcKSt\nA9wMbGt/e4CHYRgYwF7geob/TH7vmdCQJE3eggFQVSer6utt+f8CLwGbgJ3AgdZ2ALi1Le8EHq2h\np4H1STYCNwGHq+pUVb0FHAZ2rOjZSJLGtm4xzUm2Aj8NPANMVdVJGIZEkita2ybg9ZHNZlvtXPWz\nj7GH4TsHpqamGAwGixnie0xdAvdcfXrJ2y/VcsYsSQBzc3Orfi0ZOwCSfAD4r8CvVdV3k5yzdZ5a\nnaf+3kLVPmAfwPT0dM3MzIw7xPd56LGDPHB0URm3Il69Y2bix5R0YRkMBizn+jeOse4CSvLDDC/+\nj1XVf2vlN9rUDu3xzVafBbaMbL4ZOHGeuiRpDYxzF1CAR4CXquo/jjx1CDhzJ88u4OBI/c52N9AN\nwNttqugpYHuSDe3D3+2tJklaA+PMj3wU+OfA0STPt9q/Az4DPJFkN/AacFt77kngFuA48A5wF0BV\nnUpyP/Bs67uvqk6tyFlIkhZtwQCoqv/B/PP3ADfO01/A3efY135g/2IGKElaHX4TWJI6ZQBIUqcM\nAEnqlAEgSZ0yACSpUwaAJHXKAJCkThkAktQpA0CSOmUASFKnDABJ6pQBIEmdMgAkqVMGgCR1ygCQ\npE4ZAJLUKQNAkjo1zr+ElKQubb33D9fs2L+z49JVP8Y4/xR+f5I3k/zJSO2yJIeTHGuPG1o9SR5M\ncjzJC0muHdlmV+s/lmTXfMeSJE3OOFNAvwPsOKt2L3CkqrYBR9o6wM3Atva3B3gYhoEB7AWuB64D\n9p4JDUnS2lgwAKrqvwOnzirvBA605QPArSP1R2voaWB9ko3ATcDhqjpVVW8Bh3l/qEiSJmipHwJP\nVdVJgPZ4RatvAl4f6ZtttXPVJUlrZKU/BM48tTpP/f07SPYwnD5iamqKwWCw5MFMXQL3XH16ydsv\n1XLGLOkHx1pcP86Ym5tb9WvJUgPgjSQbq+pkm+J5s9VngS0jfZuBE60+c1Z9MN+Oq2ofsA9genq6\nZmZm5msby0OPHeSBo5O/0enVO2YmfkxJK++Ta3wX0HKuf+NY6hTQIeDMnTy7gIMj9Tvb3UA3AG+3\nKaKngO1JNrQPf7e3miRpjSz48jjJ7zN89X55klmGd/N8BngiyW7gNeC21v4kcAtwHHgHuAugqk4l\nuR94tvXdV1Vnf7AsSZqgBQOgqj5xjqdunKe3gLvPsZ/9wP5FjU6StGr8KQhJ6pQBIEmdMgAkqVMG\ngCR1ygCQpE4ZAJLUKQNAkjplAEhSpwwASeqUASBJnTIAJKlTBoAkdcoAkKROGQCS1CkDQJI6ZQBI\nUqcMAEnqlAEgSZ0yACSpUxMPgCQ7kryc5HiSeyd9fEnS0EQDIMlFwH8GbgauAj6R5KpJjkGSNDTp\ndwDXAcer6pWq+mvgcWDnhMcgSWLyAbAJeH1kfbbVJEkTtm7Cx8s8tXpPQ7IH2NNW55K8vIzjXQ58\nexnbL0k+O+kjSrrQ/Nxnl3X9+nvjNE06AGaBLSPrm4ETow1VtQ/YtxIHS/JcVU2vxL4kaZImcf2a\n9BTQs8C2JFcmuRi4HTg04TFIkpjwO4CqOp3kl4GngIuA/VX14iTHIEkamvQUEFX1JPDkhA63IlNJ\nkrQGVv36lapauEuSdMHxpyAkqVMXXAAkqSQPjKz/6yS/voZDkqSxJfn3SV5M8kKS55Ncv1rHuuAC\nAHgX+GdJLl/rgUjSYiT5WeAXgWur6h8CP897vzy7oi7EADjN8MOTf7XWA5GkRdoIfLuq3gWoqm9X\n1YkFtlmyCzEAYPiDc3ck+eBaD0SSFuGPgS1J/leSzyf5x6t5sAsyAKrqu8CjwL9c67FI0riqag74\nGYY/h/MXwBeSfHK1jnfB3QaaZK6qPpDkMuDrwG8zPM9fX9uRSdLiJPk4sKuq/ulq7P+CfAcAUFWn\ngCeA3Ws9FkkaR5IPJ9k2UroG+NZqHW/i3wSesAeAX17rQUjSmD4APJRkPcMbWo7z/V9HXnEX3BSQ\nJGk8F+wUkCTp/AwASeqUASBJnTIAJKlTBoAkdcoAkKROGQCS1CkDQJI69f8BZkQE4b+ePawAAAAA\nSUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1077418d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAADRZJREFUeJzt3F+IXOd5gPHnjRUnQm5jx062QhJd\ngXURBVMnLIohN1O72LJTKhViUBCNHAS6USEFQyv3xm0Sgw11XWKSgKiElWDqmLRFIjIYIXsJvfA/\n1alVWRhtbTXe2sQNkt2u07iseXuxn9qpvOuZ3Z2dEXqfH4id88135pxzc56ZM2cUmYkkqZ6PjHoH\nJEmjYQAkqSgDIElFGQBJKsoASFJRBkCSijIAklSUAZCkogyAJBW1atQ78GGuu+66HB8fX/L67777\nLmvWrBncDknSkCzn/HXixIlfZOanes27pAMwPj7OCy+8sOT1Jycn6XQ6g9shSRqS5Zy/IuJf+5nn\nJSBJKsoASFJRBkCSijIAklSUAZCkogyAJBVlACSpKAMgSUUZAEkq6pL+JfBynfy3d7hr39Ghb/fs\n/V8a+jYlabH8BCBJRRkASSrKAEhSUQZAkooyAJJUlAGQpKIMgCQVZQAkqSgDIElFGQBJKsoASFJR\nBkCSijIAklSUAZCkogyAJBVlACSpKAMgSUUZAEkqygBIUlEGQJKKMgCSVJQBkKSiDIAkFWUAJKko\nAyBJRRkASSqq7wBExBUR8WJE/Lgtb4yIZyPiTET8MCKubOMfa8tT7fnxrte4p42/EhG3DfpgJEn9\nW8wngK8Dp7uWHwAeysxNwHlgdxvfDZzPzOuBh9o8ImIzsAP4LLAV+G5EXLG83ZckLVVfAYiI9cCX\ngL9uywHcDPyoTTkEbG+Pt7Vl2vO3tPnbgMcy873MfA2YArYM4iAkSYu3qs95fwX8MfBrbfla4O3M\nnG3L08C69ngd8DpAZs5GxDtt/jrgma7X7F7nf0XEHmAPwNjYGJOTk/0eyweMrYa7b5jtPXHAlrPP\nkgQwMzOz4ueSngGIiN8F3srMExHRuTA8z9Ts8dyHrfN/A5n7gf0AExMT2el0Lp7St4cfPcyDJ/tt\n3OCc3dkZ+jYlDd74vqMj2/YjW69iOee/fvRzdvwi8HsRcQfwceDXmftEcHVErGqfAtYDb7T508AG\nYDoiVgGfAM51jV/QvY4kach6fgeQmfdk5vrMHGfuS9ynMnMn8DTw5TZtF3C4PT7SlmnPP5WZ2cZ3\ntLuENgKbgOcGdiSSpEVZzvWRPwEei4hvAS8CB9r4AeAHETHF3Dv/HQCZeSoiHgdeBmaBvZn5/jK2\nL0lahkUFIDMngcn2+FXmuYsnM38F3LnA+vcB9y12JyVJg+cvgSWpKAMgSUUZAEkqygBIUlEGQJKK\nMgCSVJQBkKSiDIAkFWUAJKkoAyBJRRkASSrKAEhSUQZAkooyAJJUlAGQpKIMgCQVZQAkqSgDIElF\nGQBJKsoASFJRBkCSijIAklSUAZCkogyAJBVlACSpKAMgSUUZAEkqygBIUlEGQJKKMgCSVJQBkKSi\nDIAkFWUAJKkoAyBJRRkASSqqZwAi4uMR8VxE/FNEnIqIP2/jGyPi2Yg4ExE/jIgr2/jH2vJUe368\n67XuaeOvRMRtK3VQkqTe+vkE8B5wc2b+FnAjsDUibgIeAB7KzE3AeWB3m78bOJ+Z1wMPtXlExGZg\nB/BZYCvw3Yi4YpAHI0nqX88A5JyZtvjR9i+Bm4EftfFDwPb2eFtbpj1/S0REG38sM9/LzNeAKWDL\nQI5CkrRoq/qZ1N6pnwCuB74D/AvwdmbOtinTwLr2eB3wOkBmzkbEO8C1bfyZrpftXqd7W3uAPQBj\nY2NMTk4u7oi6jK2Gu2+Y7T1xwJazz5IuHaM4f1wwMzOz4ueSvgKQme8DN0bE1cDfA5+Zb1r7Gws8\nt9D4xdvaD+wHmJiYyE6n088uzuvhRw/z4Mm+DnGgzu7sDH2bkgbvrn1HR7btR7auYTnnv34s6i6g\nzHwbmARuAq6OiAtn1/XAG+3xNLABoD3/CeBc9/g860iShqyfu4A+1d75ExGrgd8BTgNPA19u03YB\nh9vjI22Z9vxTmZltfEe7S2gjsAl4blAHIklanH6uj6wFDrXvAT4CPJ6ZP46Il4HHIuJbwIvAgTb/\nAPCDiJhi7p3/DoDMPBURjwMvA7PA3nZpSZI0Aj0DkJkvAZ+bZ/xV5rmLJzN/Bdy5wGvdB9y3+N2U\nJA2avwSWpKIMgCQVZQAkqSgDIElFGQBJKsoASFJRBkCSijIAklSUAZCkogyAJBVlACSpKAMgSUUZ\nAEkqygBIUlEGQJKKMgCSVJQBkKSiDIAkFWUAJKkoAyBJRRkASSrKAEhSUQZAkooyAJJUlAGQpKIM\ngCQVZQAkqSgDIElFGQBJKsoASFJRBkCSijIAklSUAZCkogyAJBVlACSpqJ4BiIgNEfF0RJyOiFMR\n8fU2/smIOBYRZ9rfa9p4RMS3I2IqIl6KiM93vdauNv9MROxaucOSJPXSzyeAWeDuzPwMcBOwNyI2\nA/uA45m5CTjelgFuBza1f3uA78FcMIB7gS8AW4B7L0RDkjR8PQOQmW9m5j+2x/8JnAbWAduAQ23a\nIWB7e7wN+H7OeQa4OiLWArcBxzLzXGaeB44BWwd6NJKkvi3qO4CIGAc+BzwLjGXmmzAXCeDTbdo6\n4PWu1abb2ELjkqQRWNXvxIi4Cvhb4I8y8z8iYsGp84zlh4xfvJ09zF06YmxsjMnJyX538QPGVsPd\nN8wuef2lWs4+S7p0jOL8ccHMzMyKn0v6CkBEfJS5k/+jmfl3bfjnEbE2M99sl3jeauPTwIau1dcD\nb7TxzkXjkxdvKzP3A/sBJiYmstPpXDylbw8/epgHT/bduIE5u7Mz9G1KGry79h0d2bYf2bqG5Zz/\n+tHPXUABHABOZ+Zfdj11BLhwJ88u4HDX+Ffb3UA3Ae+0S0RPArdGxDXty99b25gkaQT6eXv8ReAP\ngJMR8dM29qfA/cDjEbEb+BlwZ3vuCeAOYAr4JfA1gMw8FxHfBJ5v876RmecGchSSpEXrGYDM/Afm\nv34PcMs88xPYu8BrHQQOLmYHJUkrw18CS1JRBkCSijIAklSUAZCkogyAJBVlACSpKAMgSUUZAEkq\nygBIUlEGQJKKMgCSVJQBkKSiDIAkFWUAJKkoAyBJRRkASSrKAEhSUQZAkooyAJJUlAGQpKIMgCQV\nZQAkqSgDIElFGQBJKsoASFJRBkCSijIAklSUAZCkogyAJBVlACSpKAMgSUUZAEkqygBIUlEGQJKK\nMgCSVFTPAETEwYh4KyL+uWvskxFxLCLOtL/XtPGIiG9HxFREvBQRn+9aZ1ebfyYidq3M4UiS+tXP\nJ4BHgK0Xje0DjmfmJuB4Wwa4HdjU/u0BvgdzwQDuBb4AbAHuvRANSdJo9AxAZv4EOHfR8DbgUHt8\nCNjeNf79nPMMcHVErAVuA45l5rnMPA8c44NRkSQN0VK/AxjLzDcB2t9Pt/F1wOtd86bb2ELjkqQR\nWTXg14t5xvJDxj/4AhF7mLt8xNjYGJOTk0vembHVcPcNs0tef6mWs8+SLh2jOH9cMDMzs+LnkqUG\n4OcRsTYz32yXeN5q49PAhq5564E32njnovHJ+V44M/cD+wEmJiay0+nMN60vDz96mAdPDrpxvZ3d\n2Rn6NiUN3l37jo5s249sXcNyzn/9WOoloCPAhTt5dgGHu8a/2u4Gugl4p10iehK4NSKuaV/+3trG\nJEkj0vPtcUT8DXPv3q+LiGnm7ua5H3g8InYDPwPubNOfAO4ApoBfAl8DyMxzEfFN4Pk27xuZefEX\ny5KkIeoZgMz8ygJP3TLP3AT2LvA6B4GDi9o7SdKK8ZfAklSUAZCkogyAJBVlACSpKAMgSUUZAEkq\nygBIUlEGQJKKMgCSVJQBkKSiDIAkFWUAJKkoAyBJRRkASSrKAEhSUQZAkooyAJJUlAGQpKIMgCQV\nZQAkqSgDIElFGQBJKsoASFJRBkCSijIAklSUAZCkogyAJBVlACSpKAMgSUUZAEkqygBIUlEGQJKK\nMgCSVJQBkKSiDIAkFTX0AETE1oh4JSKmImLfsLcvSZoz1ABExBXAd4Dbgc3AVyJi8zD3QZI0Z9if\nALYAU5n5amb+N/AYsG3I+yBJYvgBWAe83rU83cYkSUO2asjbi3nG8v9NiNgD7GmLMxHxyjK2dx3w\ni2WsvyTxwLC3KOly89sPLOv89Zv9TBp2AKaBDV3L64E3uidk5n5g/yA2FhEvZObEIF5LkoZpGOev\nYV8Ceh7YFBEbI+JKYAdwZMj7IEliyJ8AMnM2Iv4QeBK4AjiYmaeGuQ+SpDnDvgREZj4BPDGkzQ3k\nUpIkjcCKn78iM3vPkiRddvyvICSpqKFfAhqGiHgfONk1tD0zz45odySpLxFxLXC8Lf4G8D7w7215\nS/sB7eC2dzleAoqImcy8atT7IUlLFRF/Bsxk5l+s1Da8BCRJRV2Wl4CA1RHx0/b4tcz8/ZHujSRd\ngi7XAPxXZt446p2QpEuZl4AkqSgDIElFGQBJKuqyvA1UktSbnwAkqSgDIElFGQBJKsoASFJRBkCS\nijIAklSUAZCkogyAJBX1P/849cQThSMWAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a0cc71e80>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAADrxJREFUeJzt3V2MXVd5xvH/Q8xHaloSCIwi26oj\nxReERgQ0CpG4GUiVOKGqc9FIRlFxkCULKZWoFKk1VaWoQCRyQVOBANVqrBiUElJaZItETa3AqPQi\nkKSkMSGNPISUuLaIqE3a4SPI9O3FrEEHZ8Zz5usc1+v/k0Zn73evvdfaN/s5e5195qSqkCT151Xj\nHoAkaTwMAEnqlAEgSZ0yACSpUwaAJHXKAJCkThkAktQpA0CSOmUASFKnNox7AGdzySWX1NatW1e8\n/09+8hM2bty4dgOSpBFZzfXriSee+FFVvXmpdud0AGzdupXHH398xftPT08zNTW1dgOSpBFZzfUr\nyX8M084pIEnqlAEgSZ0yACSpUwaAJHXKAJCkThkAktQpA0CSOmUASFKnDABJ6tQ5/U1gSRqnrXsf\nHFvf925f/39j4x2AJHXKAJCkThkAktQpA0CSOmUASFKnhgqAJM8nOZLkySSPt9obkxxOcrS9Xtzq\nSfKpJDNJnkryzoHj7GrtjybZtT6nJEkaxnLuAN5TVVdV1WRb3ws8UlXbgEfaOsANwLb2twf4HMwF\nBnAH8C7gauCO+dCQJI3eaqaAdgAH2vIB4KaB+udrzqPARUkuBa4HDlfVyao6BRwGtq+if0nSKgz7\nRbAC/ilJAX9dVfuAiao6AVBVJ5K8pbXdBLwwsO+xVlus/muS7GHuzoGJiQmmp6eHP5szzM7Ormp/\nSX27/crTY+t7FNevYQPg3VV1vF3kDyf597O0zQK1Okv91wtz4bIPYHJyslbzm77+JrCk1bh1zN8E\nXu/r11BTQFV1vL2+CHyFuTn8H7apHdrri635MWDLwO6bgeNnqUuSxmDJAEiyMclvzi8D1wHfAQ4B\n80/y7AIOtuVDwAfa00DXAC+1qaKHgeuSXNw+/L2u1SRJYzDMFNAE8JUk8+3/tqr+McljwANJdgM/\nAG5u7R8CbgRmgJ8CHwSoqpNJPgY81tp9tKpOrtmZSJKWZckAqKrngLcvUP8v4NoF6gXctsix9gP7\nlz9MSdJa85vAktQpA0CSOmUASFKnDABJ6pQBIEmdMgAkqVMGgCR1ygCQpE4ZAJLUKQNAkjplAEhS\npwwASeqUASBJnTIAJKlTBoAkdcoAkKROGQCS1CkDQJI6ZQBIUqcMAEnqlAEgSZ0yACSpUwaAJHXK\nAJCkThkAktQpA0CSOmUASFKnDABJ6pQBIEmdMgAkqVNDB0CSC5J8O8lX2/plSb6Z5GiSLyV5Tau/\ntq3PtO1bB47xkVZ/Nsn1a30ykqThLecO4MPAMwPrdwF3V9U24BSwu9V3A6eq6nLg7taOJFcAO4G3\nAduBzya5YHXDlySt1FABkGQz8D7gb9p6gPcCX25NDgA3teUdbZ22/drWfgdwf1W9XFXfB2aAq9fi\nJCRJyzfsHcBfAX8C/G9bfxPw46o63daPAZva8ibgBYC2/aXW/lf1BfaRJI3YhqUaJPk94MWqeiLJ\n1Hx5gaa1xLaz7TPY3x5gD8DExATT09NLDXFRs7Ozq9pfUt9uv/L00o3WySiuX0sGAPBu4PeT3Ai8\nDvgt5u4ILkqyob3L3wwcb+2PAVuAY0k2AG8ATg7U5w3u8ytVtQ/YBzA5OVlTU1MrOK0509PTrGZ/\nSX27de+DY+v73u0b1/36teQUUFV9pKo2V9VW5j7E/VpV3QJ8HfiD1mwXcLAtH2rrtO1fq6pq9Z3t\nKaHLgG3At9bsTCRJyzLMHcBi/hS4P8nHgW8D97T6PcAXksww985/J0BVPZ3kAeC7wGngtqr65Sr6\nlyStwrICoKqmgem2/BwLPMVTVT8Hbl5k/zuBO5c7SEnS2vObwJLUKQNAkjplAEhSpwwASeqUASBJ\nnTIAJKlTBoAkdcoAkKROGQCS1CkDQJI6ZQBIUqcMAEnqlAEgSZ0yACSpUwaAJHXKAJCkThkAktSp\n1fwk5DnvyH++NJYfdX7+E+8beZ+StFzeAUhSpwwASeqUASBJnTIAJKlTBoAkdcoAkKROGQCS1CkD\nQJI6ZQBIUqcMAEnqlAEgSZ0yACSpUwaAJHVqyQBI8rok30ryb0meTvIXrX5Zkm8mOZrkS0le0+qv\nbeszbfvWgWN9pNWfTXL9ep2UJGlpw9wBvAy8t6reDlwFbE9yDXAXcHdVbQNOAbtb+93Aqaq6HLi7\ntSPJFcBO4G3AduCzSS5Yy5ORJA1vyQCoObNt9dXtr4D3Al9u9QPATW15R1unbb82SVr9/qp6uaq+\nD8wAV6/JWUiSlm2oH4Rp79SfAC4HPgN8D/hxVZ1uTY4Bm9ryJuAFgKo6neQl4E2t/ujAYQf3Gexr\nD7AHYGJigunp6eWd0YCJC+H2K08v3XCNrWbMks4d47h+zJudnV33a8lQAVBVvwSuSnIR8BXgrQs1\na69ZZNti9TP72gfsA5icnKypqalhhrigT993kE8eGf2Pnj1/y9TI+5S09sbxi4Lz7t2+kdVc/4ax\nrKeAqurHwDRwDXBRkvmr62bgeFs+BmwBaNvfAJwcrC+wjyRpxIZ5CujN7Z0/SS4Efhd4Bvg68Aet\n2S7gYFs+1NZp279WVdXqO9tTQpcB24BvrdWJSJKWZ5j5kUuBA+1zgFcBD1TVV5N8F7g/yceBbwP3\ntPb3AF9IMsPcO/+dAFX1dJIHgO8Cp4Hb2tSSJGkMlgyAqnoKeMcC9edY4Cmeqvo5cPMix7oTuHP5\nw5QkrTW/CSxJnTIAJKlTBoAkdcoAkKROGQCS1CkDQJI6ZQBIUqcMAEnqlAEgSZ0yACSpUwaAJHXK\nAJCkThkAktQpA0CSOmUASFKnDABJ6pQBIEmdMgAkqVMGgCR1ygCQpE4ZAJLUKQNAkjplAEhSpwwA\nSeqUASBJnTIAJKlTBoAkdcoAkKROGQCS1CkDQJI6tWQAJNmS5OtJnknydJIPt/obkxxOcrS9Xtzq\nSfKpJDNJnkryzoFj7WrtjybZtX6nJUlayjB3AKeB26vqrcA1wG1JrgD2Ao9U1TbgkbYOcAOwrf3t\nAT4Hc4EB3AG8C7gauGM+NCRJo7dkAFTViar617b8P8AzwCZgB3CgNTsA3NSWdwCfrzmPAhcluRS4\nHjhcVSer6hRwGNi+pmcjSRrasj4DSLIVeAfwTWCiqk7AXEgAb2nNNgEvDOx2rNUWq0uSxmDDsA2T\nvB74e+CPq+q/kyzadIFanaV+Zj97mJs6YmJigunp6WGH+AoTF8LtV55e8f4rtZoxSzp3jOP6MW92\ndnbdryVDBUCSVzN38b+vqv6hlX+Y5NKqOtGmeF5s9WPAloHdNwPHW33qjPr0mX1V1T5gH8Dk5GRN\nTU2d2WRon77vIJ88MnTGrZnnb5kaeZ+S1t6tex8cW9/3bt/Iaq5/wxjmKaAA9wDPVNVfDmw6BMw/\nybMLODhQ/0B7Guga4KU2RfQwcF2Si9uHv9e1miRpDIZ5e/xu4A+BI0mebLU/Az4BPJBkN/AD4Oa2\n7SHgRmAG+CnwQYCqOpnkY8Bjrd1Hq+rkmpyFJGnZlgyAqvoXFp6/B7h2gfYF3LbIsfYD+5czQEnS\n+vCbwJLUKQNAkjplAEhSpwwASeqUASBJnTIAJKlTBoAkdcoAkKROGQCS1CkDQJI6ZQBIUqcMAEnq\nlAEgSZ0yACSpUwaAJHXKAJCkThkAktQpA0CSOmUASFKnDABJ6pQBIEmdMgAkqVMGgCR1ygCQpE4Z\nAJLUKQNAkjplAEhSpwwASeqUASBJnTIAJKlTBoAkdWrJAEiyP8mLSb4zUHtjksNJjrbXi1s9ST6V\nZCbJU0neObDPrtb+aJJd63M6kqRhDXMHcC+w/YzaXuCRqtoGPNLWAW4AtrW/PcDnYC4wgDuAdwFX\nA3fMh4YkaTyWDICq+mfg5BnlHcCBtnwAuGmg/vma8yhwUZJLgeuBw1V1sqpOAYd5ZahIkkZowwr3\nm6iqEwBVdSLJW1p9E/DCQLtjrbZY/RWS7GHu7oGJiQmmp6dXOESYuBBuv/L0ivdfqdWMWdK5YxzX\nj3mzs7Prfi1ZaQAsJgvU6iz1Vxar9gH7ACYnJ2tqamrFg/n0fQf55JG1PsWlPX/L1Mj7lLT2bt37\n4Nj6vnf7RlZz/RvGSp8C+mGb2qG9vtjqx4AtA+02A8fPUpckjclKA+AQMP8kzy7g4ED9A+1poGuA\nl9pU0cPAdUkubh/+XtdqkqQxWXJ+JMkXgSngkiTHmHua5xPAA0l2Az8Abm7NHwJuBGaAnwIfBKiq\nk0k+BjzW2n20qs78YFmSNEJLBkBVvX+RTdcu0LaA2xY5zn5g/7JGJ0laN34TWJI6ZQBIUqcMAEnq\nlAEgSZ0yACSpUwaAJHXKAJCkThkAktQpA0CSOmUASFKnDABJ6pQBIEmdMgAkqVMGgCR1ygCQpE4Z\nAJLUKQNAkjplAEhSpwwASeqUASBJnTIAJKlTBoAkdcoAkKROGQCS1CkDQJI6ZQBIUqcMAEnqlAEg\nSZ0yACSpUwaAJHVq5AGQZHuSZ5PMJNk76v4lSXNGGgBJLgA+A9wAXAG8P8kVoxyDJGnOqO8ArgZm\nquq5qvoFcD+wY8RjkCQx+gDYBLwwsH6s1SRJI7ZhxP1lgVr9WoNkD7Cnrc4meXYV/V0C/GgV+69I\n7hp1j5LON++5a1XXr98eptGoA+AYsGVgfTNwfLBBVe0D9q1FZ0ker6rJtTiWJI3SKK5fo54CegzY\nluSyJK8BdgKHRjwGSRIjvgOoqtNJ/gh4GLgA2F9VT49yDJKkOaOeAqKqHgIeGlF3azKVJEljsO7X\nr1TV0q0kSecd/xWEJHXqvA6AJLPjHoMkLUeSXyZ5Msl3kvxdkt9Yr77O6wCQpP+HflZVV1XV7wC/\nAD60Xh0ZAJJ07voGcPl6HdwAkKRzUJINzP3jzCPr1cfIHwOVJJ3VhUmebMvfAO5Zr44MAEk6t/ys\nqq4aRUdOAUlSp87bAGjzZy+PexySdK46bwMAeBvwvXEPQpKWo6peP6q+zssASPIh4IvAn497LJJ0\nrvJ/AUlSp87LOwBJ0tIMAEnqlAEgSZ0yACSpUwaAJHXKAJCkTv0fuJOWK4dgaQkAAAAASUVORK5C\nYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a0cbed3c8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(df.ie.value_counts(), end='\\n\\n')\n",
    "print(df.ns.value_counts(), end='\\n\\n')\n",
    "print(df.ft.value_counts(), end='\\n\\n')\n",
    "print(df.pj.value_counts(), end='\\n\\n')\n",
    "\n",
    "df.ie.hist(); plt.show()\n",
    "df.ns.hist(); plt.show()\n",
    "df.ft.hist(); plt.show()\n",
    "df.pj.hist(); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Modelling"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training our Models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-06T18:46:57.741705Z",
     "start_time": "2018-04-06T18:46:57.074448Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 251 ms, sys: 58.7 ms, total: 310 ms\n",
      "Wall time: 345 ms\n",
      "CPU times: user 61.6 ms, sys: 17.1 ms, total: 78.6 ms\n",
      "Wall time: 81.5 ms\n",
      "CPU times: user 57.3 ms, sys: 8.2 ms, total: 65.5 ms\n",
      "Wall time: 66.8 ms\n",
      "CPU times: user 64.7 ms, sys: 10 ms, total: 74.7 ms\n",
      "Wall time: 75.8 ms\n",
      "CPU times: user 58.8 ms, sys: 11.8 ms, total: 70.5 ms\n",
      "Wall time: 82.8 ms\n",
      "CPU times: user 499 ms, sys: 107 ms, total: 606 ms\n",
      "Wall time: 659 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "%time model = MultinomialNB().fit(xTrain, yTrain)\n",
    "%time ieModel = MultinomialNB().fit(xIETrain, yIETrain)\n",
    "%time nsModel = MultinomialNB().fit(xNSTrain, yNSTrain)\n",
    "%time ftModel = MultinomialNB().fit(xFTTrain, yFTTrain)\n",
    "%time pjModel = MultinomialNB().fit(xPJTrain, yPJTrain)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Testing our Models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see our models performed relatively well despite the weaknesses present in our dataset. Although this accuracy is **superficial** in the cases of first 2 models i.e. **IE** and **NS**. The reason being that these are heavily biased towards a single characteristic value. This will be evident in [result analysis section](#Analysing-our-Models-Results) below.\n",
    "Our accuracy is just high because our most of data is belonging to one instance. Introvert for example in IE model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-06T18:46:57.940134Z",
     "start_time": "2018-04-06T18:46:57.745911Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.345320424159\n",
      "0.787920700784\n",
      "0.833102812356\n",
      "0.792992162287\n",
      "0.713692946058\n"
     ]
    }
   ],
   "source": [
    "print(model.score(xTest, yTest))\n",
    "print(ieModel.score(xIETest, yIETest))\n",
    "print(nsModel.score(xNSTest, yNSTest))\n",
    "print(ftModel.score(xFTTest, yFTTest))\n",
    "print(pjModel.score(xPJTest, yPJTest))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Instead of accuracy, which is a bad metric for imbalance dataset, we will be using **recall** and **f1 score**, to measure our model performance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-06T19:18:40.409649Z",
     "start_time": "2018-04-06T19:18:40.007886Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MINORITY CLASSES:\n",
      "0.157464212679\n",
      "0.0838709677419\n",
      "0.717346938776\n",
      "0.469150174622\n",
      "\n",
      "MAJORITY CLASSES:\n",
      "0.971428571429\n",
      "0.958041958042\n",
      "0.855340622372\n",
      "0.874045801527\n"
     ]
    }
   ],
   "source": [
    "#minority classes\n",
    "print('MINORITY CLASSES:')\n",
    "print(recall_score(yIETest, ieModel.predict(xIETest), pos_label='E'))\n",
    "print(recall_score(yNSTest, nsModel.predict(xNSTest), pos_label='S'))\n",
    "print(recall_score(yFTTest, ftModel.predict(xFTTest), pos_label='T'))\n",
    "print(recall_score(yPJTest, pjModel.predict(xPJTest), pos_label='J'), end='\\n\\n')\n",
    "\n",
    "#majority classes\n",
    "print('MAJORITY CLASSES:')\n",
    "print(recall_score(yIETest, ieModel.predict(xIETest), pos_label='I'))\n",
    "print(recall_score(yNSTest, nsModel.predict(xNSTest), pos_label='N'))\n",
    "print(recall_score(yFTTest, ftModel.predict(xFTTest), pos_label='F'))\n",
    "print(recall_score(yPJTest, pjModel.predict(xPJTest), pos_label='P'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-06T19:19:00.861521Z",
     "start_time": "2018-04-06T19:19:00.434506Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MINORITY CLASSES:\n",
      "0.250814332248\n",
      "0.125603864734\n",
      "0.75795148248\n",
      "0.564821303434\n",
      "\n",
      "MAJORITY CLASSES:\n",
      "0.876476906552\n",
      "0.907747196738\n",
      "0.819170358437\n",
      "0.786671246994\n"
     ]
    }
   ],
   "source": [
    "#minority classes\n",
    "print('MINORITY CLASSES:')\n",
    "print(f1_score(yIETest, ieModel.predict(xIETest), pos_label='E'))\n",
    "print(f1_score(yNSTest, nsModel.predict(xNSTest), pos_label='S'))\n",
    "print(f1_score(yFTTest, ftModel.predict(xFTTest), pos_label='T'))\n",
    "print(f1_score(yPJTest, pjModel.predict(xPJTest), pos_label='J'), end='\\n\\n')\n",
    "\n",
    "#majority classes\n",
    "print('MAJORITY CLASSES:')\n",
    "print(f1_score(yIETest, ieModel.predict(xIETest), pos_label='I'))\n",
    "print(f1_score(yNSTest, nsModel.predict(xNSTest), pos_label='N'))\n",
    "print(f1_score(yFTTest, ftModel.predict(xFTTest), pos_label='F'))\n",
    "print(f1_score(yPJTest, pjModel.predict(xPJTest), pos_label='P'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For better analysis or testing of our model, we will be using classification report to see **recall** which is measure of how much right predictions we make per characteristic. *Accuracy* is bad *classification metric* in our case due to **imbalanced classification problem**. Recall will give us better insight."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-06T18:48:48.986338Z",
     "start_time": "2018-04-06T18:48:48.891359Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      "       ENFJ       0.00      0.00      0.00        50\n",
      "       ENFP       0.50      0.01      0.02       160\n",
      "       ENTJ       0.00      0.00      0.00        55\n",
      "       ENTP       0.40      0.01      0.02       159\n",
      "       ESFJ       0.00      0.00      0.00         9\n",
      "       ESFP       0.00      0.00      0.00        12\n",
      "       ESTJ       0.00      0.00      0.00        12\n",
      "       ESTP       0.00      0.00      0.00        23\n",
      "       INFJ       0.37      0.43      0.40       378\n",
      "       INFP       0.30      0.89      0.45       459\n",
      "       INTJ       0.71      0.11      0.20       283\n",
      "       INTP       0.45      0.43      0.44       328\n",
      "       ISFJ       0.00      0.00      0.00        37\n",
      "       ISFP       0.00      0.00      0.00        73\n",
      "       ISTJ       0.00      0.00      0.00        45\n",
      "       ISTP       0.00      0.00      0.00        86\n",
      "\n",
      "avg / total       0.35      0.35      0.26      2169\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/jaat/anaconda3/lib/python3.6/site-packages/sklearn/metrics/classification.py:1135: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples.\n",
      "  'precision', 'predicted', average, warn_for)\n"
     ]
    }
   ],
   "source": [
    "print(classification_report(yTest, model.predict(xTest)), end='\\n\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As it can be seen for first 2 pairs ie **IE** and **NS** the *recall* score is very low for minority classes. In first pair the recall score for \"extroverts\" is **0.13** which is amazingly bad."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-06T18:48:51.516984Z",
     "start_time": "2018-04-06T18:48:51.366138Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      "          E       0.62      0.16      0.25       489\n",
      "          I       0.80      0.97      0.88      1680\n",
      "\n",
      "avg / total       0.76      0.79      0.74      2169\n",
      "\n",
      "\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "          N       0.86      0.96      0.91      1859\n",
      "          S       0.25      0.08      0.13       310\n",
      "\n",
      "avg / total       0.77      0.83      0.80      2169\n",
      "\n",
      "\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "          F       0.79      0.86      0.82      1189\n",
      "          T       0.80      0.72      0.76       980\n",
      "\n",
      "avg / total       0.79      0.79      0.79      2169\n",
      "\n",
      "\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "          J       0.71      0.47      0.56       859\n",
      "          P       0.72      0.87      0.79      1310\n",
      "\n",
      "avg / total       0.71      0.71      0.70      2169\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(classification_report(yIETest, ieModel.predict(xIETest)), end='\\n\\n')\n",
    "print(classification_report(yNSTest, nsModel.predict(xNSTest)), end='\\n\\n')\n",
    "print(classification_report(yFTTest, ftModel.predict(xFTTest)), end='\\n\\n')\n",
    "print(classification_report(yPJTest, pjModel.predict(xPJTest)), end='\\n\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Cross Validation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To get actual performance of our models, I am using **KFold cross validation** with *k=10* to get actual performance. These values will be **pickled** along with models, so that these can be used in scripts to allow users to see\n",
    "performance and reliability of each model corresponding to their characteristic pair."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-06T19:11:48.277494Z",
     "start_time": "2018-04-06T19:11:34.124013Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.376724638516\n",
      "0.0118203235415\n",
      "\n",
      "0.952665658189\n",
      "0.0105624310118\n",
      "\n",
      "0.119474789916\n",
      "0.0278256538239\n",
      "\n",
      "0.742022140779\n",
      "0.0161272773425\n",
      "\n",
      "0.857852780807\n",
      "0.00990676530416\n",
      "\n"
     ]
    }
   ],
   "source": [
    "scores = []\n",
    "\n",
    "scores.append(cross_val_score(estimator=model, cv=10, X=X, y=y, scoring='accuracy'))\n",
    "scores.append(cross_val_score(estimator=ieModel, cv=10, X=X, y=LabelEncoder().fit_transform(yIE), scoring='recall'))\n",
    "scores.append(cross_val_score(estimator=nsModel, cv=10, X=X, y=LabelEncoder().fit_transform(yNS), scoring='recall'))\n",
    "scores.append(cross_val_score(estimator=ftModel, cv=10, X=X, y=LabelEncoder().fit_transform(yFT), scoring='recall'))\n",
    "scores.append(cross_val_score(estimator=pjModel, cv=10, X=X, y=LabelEncoder().fit_transform(yPJ), scoring='recall'))\n",
    "\n",
    "#prining mean and standard deviations for each model\n",
    "for score in scores:\n",
    "    print(score.mean())\n",
    "    print(score.std(), end='\\n\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Analysing our Models Results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In case of this model where we are predicting MBTI type using all characteristics together. It can be seen heavy bias in results of trained model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-06T18:49:38.960722Z",
     "start_time": "2018-04-06T18:49:38.527257Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFP    459\n",
      "INFJ    378\n",
      "INTP    328\n",
      "INTJ    283\n",
      "ENFP    160\n",
      "ENTP    159\n",
      "ISTP     86\n",
      "ISFP     73\n",
      "ENTJ     55\n",
      "ENFJ     50\n",
      "ISTJ     45\n",
      "ISFJ     37\n",
      "ESTP     23\n",
      "ESTJ     12\n",
      "ESFP     12\n",
      "ESFJ      9\n",
      "Name: type, dtype: int64\n",
      "\n",
      "INFP    1358\n",
      "INFJ     446\n",
      "INTP     311\n",
      "INTJ      45\n",
      "ENTP       5\n",
      "ENFP       4\n",
      "dtype: int64\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEMCAYAAADNtWEcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAHjNJREFUeJzt3XuUZWV95vHvDxoascBuLl3TaRga\nh15GRVEoEMcVrQIHEImgQgKTYEMwvWKQsMa4uDgxzMQLJt4GJ8qyTXeEjFoikiUqiqSlJIzCYHNH\nNLRI0wUI2DcoOiBNP/PHeQsPZVWdc/Z5d93e57NWrdqXdz/nPXWqzm/vd+99KiRhZmbl2Wm6O2Bm\nZtPDBcDMrFAuAGZmhXIBMDMrlAuAmVmhXADMzArlAmBmVigXADOzQrkAmJkVygXAzKxQ81o1iIjV\nwAnAY5IOHrPu/cDHgX0l/SoiArgEOB7YBpwh6dbUdjnwV2nTD0u6rNVj77PPPlq6dGkHT+eFnnrq\nKV784hdX3n625dWRWVpeHZkzPa+OzNLy6sjsJm/t2rW/krRvy4aSJv0C3ggcCtw9Zvn+wLXAemCf\ntOx44DtAAEcCN6flewH3p+8L0/TCVo992GGHqRvXX399V9vPtrw6MkvLqyNzpufVkVlaXh2Z3eQB\nP1aL91dJrYeAJN0AbBpn1aeB84DmT5M7Ebg89eEmYEFELAaOBa6TtEnSZuA64LiW1cnMzGpT6RxA\nRLwNeEjSHWNWLQE2NM0Pp2UTLTczs2kSauPjoCNiKfAtSQdHxO7A9cAxkrZGxANAnxrnAL4NXCzp\nxrTdGhpHCUcB8yV9OC3/ILBN0ifHeawVwAqA3t7ewwYHBys/uZGREXp6eipvP9vy6sgsLa+OzJme\nV0dmaXl1ZHaTNzAwsFZSX8uG7YwTAUtJ5wCAVwGPAQ+kr+3Ag8B/AD4PnNa03c+AxcBpwOeblr+g\n3URfPgcw/Zml5dWROdPz6sgsLa+OzBlxDmCcgnGXpEWSlkpaSmM451BJvwSuBt4VDUcCWyU9QuNk\n8TERsTAiFgLHpGVmZjZNWhaAiPgK8CPgZRExHBFnTdL8GhpX+KwDvgD8OYCkTcCHgFvS19+kZWZm\nNk1a3gcg6bQW65c2TQs4e4J2q4HVHfbPzMxq4juBzcwK1fIIwMxmmE33w5cvzZv5O+/Jm2ezgo8A\nzMwK5QJgZlYoFwAzs0K5AJiZFcoFwMysUC4AZmaFcgEwMyuUC4CZWaFcAMzMCuUCYGZWKBcAM7NC\nuQCYmRXKBcDMrFAuAGZmhXIBMDMrlAuAmVmhXADMzArlAmBmVigXADOzQrUsABGxOiIei4i7m5Z9\nPCJ+GhF3RsQ/R8SCpnUXRsS6iPhZRBzbtPy4tGxdRFyQ/6mYmVkn2jkC+CJw3Jhl1wEHS3o18G/A\nhQAR8QrgVOCVaZvPRcTOEbEz8FngLcArgNNSWzMzmyYtC4CkG4BNY5Z9T9L2NHsTsF+aPhEYlPSM\npF8A64Aj0tc6SfdL+jUwmNqamdk0yXEO4E+A76TpJcCGpnXDadlEy83MbJqEpNaNIpYC35J08Jjl\n/x3oA94hSRHxWeBHkv5PWr8KuIZGoTlW0rvT8tOBIySdM85jrQBWAPT29h42ODhY+cmNjIzQ09NT\nefvZlldHZml5dWRmz9uykZ4dW7PlAYzsumhmP+cZnldHZjd5AwMDayX1tWo3r1I6EBHLgROAo/Wb\nKjIM7N/UbD/g4TQ90fIXkLQSWAnQ19en/v7+ql1kaGiIbrafbXl1ZJaWV0dm9ryrVtP/9LXZ8gCG\n9nrPzH7OMzyvjsw6+jhWpSGgiDgOOB94m6RtTauuBk6NiPkRcSCwDPh/wC3Asog4MCJ2pXGi+Oru\num5mZt1oeQQQEV8B+oF9ImIYuIjGVT/zgesiAuAmSX8m6Z6IuAL4CbAdOFvScynnvcC1wM7Aakn3\n1PB8zMysTS0LgKTTxlm8apL2HwE+Ms7ya2icDzAzsxnAdwKbmRXKBcDMrFAuAGZmhXIBMDMrlAuA\nmVmhXADMzArlAmBmVigXADOzQrkAmJkVygXAzKxQLgBmZoVyATAzK5QLgJlZoVwAzMwK5QJgZlYo\nFwAzs0K5AJiZFcoFwMysUC4AZmaFcgEwMyuUC4CZWaFcAMzMCtWyAETE6oh4LCLublq2V0RcFxH3\npe8L0/KIiM9ExLqIuDMiDm3aZnlqf19ELK/n6ZiZWbvaOQL4InDcmGUXAGskLQPWpHmAtwDL0tcK\n4FJoFAzgIuB1wBHARaNFw8zMpkfLAiDpBmDTmMUnApel6cuAk5qWX66Gm4AFEbEYOBa4TtImSZuB\n6/jtomJmZlOo6jmAXkmPAKTvi9LyJcCGpnbDadlEy83MbJqEpNaNIpYC35J0cJrfImlB0/rNkhZG\nxLeBiyXdmJavAc4DjgLmS/pwWv5BYJukT47zWCtoDB/R29t72ODgYOUnNzIyQk9PT+XtZ1teHZml\n5dWRmT1vy0Z6dmzNlgcwsuuimf2cZ3heHZnd5A0MDKyV1Neq3bxK6fBoRCyW9Ega4nksLR8G9m9q\ntx/wcFreP2b50HjBklYCKwH6+vrU398/XrO2DA0N0c32sy2vjszS8urIzJ531Wr6n742Wx7A0F7v\nmdnPeYbn1ZFZRx/HqjoEdDUweiXPcuAbTcvfla4GOhLYmoaIrgWOiYiF6eTvMWmZmZlNk5ZHABHx\nFRp77/tExDCNq3k+BlwREWcBDwKnpObXAMcD64BtwJkAkjZFxIeAW1K7v5E09sSymZlNoZYFQNJp\nE6w6epy2As6eIGc1sLqj3pmZWW18J7CZWaFcAMzMCuUCYGZWKBcAM7NCuQCYmRXKBcDMrFAuAGZm\nhXIBMDMrlAuAmVmhXADMzArlAmBmVigXADOzQrkAmJkVygXAzKxQLgBmZoVyATAzK5QLgJlZoVwA\nzMwK5QJgZlYoFwAzs0K5AJiZFaqrAhAR/y0i7omIuyPiKxGxW0QcGBE3R8R9EfHViNg1tZ2f5tel\n9UtzPAEzM6umcgGIiCXAXwB9kg4GdgZOBf4W+LSkZcBm4Ky0yVnAZkkHAZ9O7czMbJp0OwQ0D3hR\nRMwDdgceAY4CrkzrLwNOStMnpnnS+qMjIrp8fDMzq6hyAZD0EPAJ4EEab/xbgbXAFknbU7NhYEma\nXgJsSNtuT+33rvr4ZmbWnZBUbcOIhcDXgT8EtgBfS/MXpWEeImJ/4BpJr4qIe4BjJQ2ndT8HjpC0\ncUzuCmAFQG9v72GDg4OV+gcwMjJCT09P5e1nW14dmaXl1ZGZPW/LRnp2bM2WBzCy66KZ/ZxneF4d\nmd3kDQwMrJXU16rdvErpDW8GfiHpcYCIuAr4z8CCiJiX9vL3Ax5O7YeB/YHhNGT0EmDT2FBJK4GV\nAH19ferv76/cwaGhIbrZfrbl1ZFZWl4dmdnzrlpN/9PXZssDGNrrPTP7Oc/wvDoy6+jjWN2cA3gQ\nODIidk9j+UcDPwGuB05ObZYD30jTV6d50vrvq+rhh5mZda2bcwA30ziZeytwV8paCZwPvC8i1tEY\n41+VNlkF7J2Wvw+4oIt+m5lZl7oZAkLSRcBFYxbfDxwxTtungVO6eTwzM8vHdwKbmRXKBcDMrFAu\nAGZmhXIBMDMrVFcngc1s6v37r5/j9g1bsmaun7+Ns754S8t2q844POvj2vTyEYCZWaFcAMzMCuUC\nYGZWKBcAM7NCuQCYmRXKBcDMrFAuAGZmhXIBMDMrlAuAmVmhXADMzArlAmBmVigXADOzQrkAmJkV\nygXAzKxQLgBmZoVyATAzK5QLgJlZoboqABGxICKujIifRsS9EfH6iNgrIq6LiPvS94WpbUTEZyJi\nXUTcGRGH5nkKZmZWRbdHAJcA35X0u8AhwL3ABcAaScuANWke4C3AsvS1Ari0y8c2M7MuVC4AEbEn\n8EZgFYCkX0vaApwIXJaaXQaclKZPBC5Xw03AgohYXLnnZmbWlW6OAF4KPA78Y0TcFhH/EBEvBnol\nPQKQvi9K7ZcAG5q2H07LzMxsGoSkahtG9AE3AW+QdHNEXAI8AZwjaUFTu82SFkbEt4GLJd2Ylq8B\nzpO0dkzuChpDRPT29h42ODhYqX8AIyMj9PT0VN5+tuXVkVlaXh2ZufOe2Pwrdnlmc7Y8gG279TLy\n3LyW7Q7Ye/e28mb6z3Cu/94MDAysldTXql3rV3xiw8CwpJvT/JU0xvsfjYjFkh5JQzyPNbXfv2n7\n/YCHx4ZKWgmsBOjr61N/f3/lDg4NDdHN9rMtr47M0vLqyMyd953BL7B4/ZXZ8gDuOOhcbnhy35bt\nVr3z8LbyZvrPsMTfm/FUHgKS9EtgQ0S8LC06GvgJcDWwPC1bDnwjTV8NvCtdDXQksHV0qMjMzKZe\nN0cAAOcAX4qIXYH7gTNpFJUrIuIs4EHglNT2GuB4YB2wLbU1M7Np0lUBkHQ7MN4409HjtBVwdjeP\nZ2Zm+fhOYDOzQrkAmJkVygXAzKxQLgBmZoVyATAzK5QLgJlZoVwAzMwK5QJgZlYoFwAzs0K5AJiZ\nFcoFwMysUC4AZmaFcgEwMyuUC4CZWaFcAMzMCuUCYGZWKBcAM7NCuQCYmRWq2/8JbGatbLofvnxp\nvryd3pwvy4rmIwAzs0K5AJiZFarrAhARO0fEbRHxrTR/YETcHBH3RcRXI2LXtHx+ml+X1i/t9rHN\nzKy6HEcA5wL3Ns3/LfBpScuAzcBZaflZwGZJBwGfTu3MzGyadFUAImI/4K3AP6T5AI4CrkxNLgNO\nStMnpnnS+qNTezMzmwbdHgH8L+A8YEea3xvYIml7mh8GlqTpJcAGgLR+a2pvZmbTICRV2zDiBOB4\nSX8eEf3A+4EzgR+lYR4iYn/gGkmvioh7gGMlDad1PweOkLRxTO4KYAVAb2/vYYODg9WeGTAyMkJP\nT0/l7WdbXh2ZpeXVkTmyZSM9O7Zmy3si9mSXZzZnywPYtlsvI8+1vir8gL13bytvpr/Os+L3pou8\ngYGBtZL6WrXr5j6ANwBvi4jjgd2APWkcESyIiHlpL38/4OHUfhjYHxiOiHnAS4BNY0MlrQRWAvT1\n9am/v79yB4eGhuhm+9mWV0dmaXl1ZA5dtZr+p6/Nlvednd7M4vVXtm7YgTsOOpcbnty3ZbtV7zy8\nrbyZ/jrPit+bGvo4VuUhIEkXStpP0lLgVOD7kv4IuB44OTVbDnwjTV+d5knrv6+qhx9mZta1Ou4E\nPh8YjIgPA7cBq9LyVcA/RcQ6Gnv+p9bw2GZT5qwv3tJWu5N3e47bN2zJ98AH5IuysmUpAJKGgKE0\nfT9wxDhtngZOyfF4ZmbWPd8JbGZWKBcAM7NC+dNAzYxFzz7EOY9e0rrhlxe0F/g77+muQzYlfARg\nZlYoFwAzs0K5AJiZFcrnAMysbe3ez7B+/ra275Nox+lLs0VZEx8BmJkVygXAzKxQLgBmZoVyATAz\nK5QLgJlZoVwAzMwK5QJgZlYoFwAzs0K5AJiZFcoFwMysUC4AZmaFcgEwMyuUC4CZWaFcAMzMCuUC\nYGZWqMoFICL2j4jrI+LeiLgnIs5Ny/eKiOsi4r70fWFaHhHxmYhYFxF3RsShuZ6EmZl1rpsjgO3A\nX0p6OXAkcHZEvAK4AFgjaRmwJs0DvAVYlr5WAJd28dhmZtalyv8RTNIjwCNp+smIuBdYApwI9Kdm\nlwFDwPlp+eWSBNwUEQsiYnHKqcX6jXn/K9Eb92gvb9UZh2d7TDOzumQ5BxARS4HXAjcDvaNv6un7\notRsCbChabPhtMzMzKZBNHbIuwiI6AF+AHxE0lURsUXSgqb1myUtjIhvAxdLujEtXwOcJ2ntmLwV\nNIaI6O3tPWxwcLBy3zZueYKR5/L92+Oenbe3lXfA3ru3lTcyMkJPT0+33ao1s7S8TjLXb9zWVt7C\nnbaxyzObu+3W856dvzBrXh2Z23brzfq3t/f8HXPm92Yq8gYGBtZK6mvVrqtXKCJ2Ab4OfEnSVWnx\no6NDOxGxGHgsLR8G9m/afD/g4bGZklYCKwH6+vrU399fuX+Xff0abnhy38rbj/XGPR5vK2/VO9sb\nAhoaGqKb5zcVmaXldZLZ7vDiybvdzuL1V3bZq9945ICTs+bVkXnHQedm/ds7fe+n5szvzXTljaeb\nq4ACWAXcK+lTTauuBpan6eXAN5qWvytdDXQksLXO8X8zM5tcN0cAbwBOB+6KiNvTsg8AHwOuiIiz\ngAeBU9K6a4DjgXXANuDMLh7bzMy61M1VQDcCMcHqo8dpL+Dsqo9n7fGVT2bWLt8JbGZWKBcAM7NC\nuQCYmRXKBcDMrFAuAGZmhcp3q56Z2RzU7lV17V4x167Tl2aLmpCPAMzMCuUCYGZWKA8BmdmMN103\nOM51PgIwMyuUC4CZWaFcAMzMCuUCYGZWKBcAM7NCuQCYmRXKBcDMrFAuAGZmhfKNYDWYrs8OaWRm\njTOzOcxHAGZmhXIBMDMrlIeAbFbL/Rkx4M+JsXJM+RFARBwXET+LiHURccFUP76ZmTVMaQGIiJ2B\nzwJvAV4BnBYRr5jKPpiZWcNUDwEdAayTdD9ARAwCJwI/meJ+WGbTdeWTr3oyq26qC8ASYEPT/DDw\nuinuQ2WLnn2Icx69JFveHXucmy1r1Gzoo5nNDCFp6h4s4hTgWEnvTvOnA0dIOqepzQpgRZp9GfCz\nLh5yH+BXXWw/2/LqyCwtr47MmZ5XR2ZpeXVkdpN3gKR9WzWa6iOAYWD/pvn9gIebG0haCazM8WAR\n8WNJfTmyZkNeHZml5dWROdPz6sgsLa+OzDr6ONZUXwV0C7AsIg6MiF2BU4Grp7gPZmbGFB8BSNoe\nEe8FrgV2BlZLumcq+2BmZg1TfiOYpGuAa6bo4bIMJc2ivDoyS8urI3Om59WRWVpeHZl19PEFpvQk\nsJmZzRz+LCAzs0K5AJiZFcoFwMysUP400BYiYh9JuW8YySYiXgv8J+AeSfd2mXXoJKufAR6U9GQH\necuAT6T+3QW8X9JD3fSxLrle54jYa5LVz0h6qsO8RcAHgINo/AwvlvREF10kIt4xyepngPu7/V2a\naSLiJNLPUNK1GfJm/Ovc1uPOlZPAEfG+SVY/A/wc+J6kHW3m/T6wGtgOPAf8gaQfdtG/bwIT/bBH\n+/dZSRsmaDNe5l8DfwyspfGRGhdL+kIXfbx+ktXzgP+Y+vh3beb9K3A5cAPwNuD1kiZ782mVl/U1\nTpm5X+df0HidY5zVoztcF0j6Upt536Xx+t4AnADsIemMqv1Lmf84yep5wMuBH0r6izbzXkfjipXR\nQn+WpMqf7xURd9H6b+ViSXe0mfc54JXAD4GjgW9K+lDV/qXMGf86t/W4c6gAXDTJ6nk0fgG2S/qD\nNvPupPFm8NP0C/53kt7URf8m23a0f6dJen0HmfcAh0vaFhF7A9+VdHjVPrbxePOB2yS19QmuEXG7\npNc0zd8qabKjjFZ5WV/jlJn1dW7j8fYFfjBdP8OU8Q5JV02yficae8qvbDPvx8CF/KbQv1vSsV30\n74BJVs8DDgb+h6TXtpl3N3CIpOciYnfgXyUdVrV/o32UtH6S9dP+OrdjzgwBSfqfrdqkP/Z2bZf0\n05R9c0R0+7mTZ7ao6Gsi4tUdZj4taRuApI3pD7eyiPiopA+k6f8i6brm9ZKeSZ/f1K7d0hDV6F7S\ni5rnJd3aYRc3Svr7yRp0+BpD5tc5It472seIeOXYGx0lPR4R53cWGQv5zc9w5+Z5SZsqdPOvgAkL\ngKQdEfHmDvJ2avpd+VpEXFihT80WS7ppkvU/bzFcOdavJT0HkHaWxttr79Q/AxP2YYa8zq0fdA4d\nAXxP0jFp+kJJF3eZNwx8qmnR+5rnJX3qtzaaPC97RY+ILTT2uqDxi/J7TfNIeluHec/3MdOe5hAT\nH8pL0lFV+5dLna9zpp/hA8AOxh9qkKSXVsjM+nOMiPuB9zct+kTz/GRHGxPkNf8Mf9TJUfEEeduA\ndaOzNIaq1qVpSep0x4uIuK3dI5A28x4g8+vcjjlzBAA0f/LdKUBXBQD4ArDHJPOd2n3M3vALVNgb\nhsb/Umj2iQoZtZHUP919aEPu17lZ13uakpZm6MdYvzvBkVLVN8QfAL8/wbyY5GhjAs0/t9063HY8\nL8+QMdaSiPjMRCvbPX/S5E2TDSnVZS4VgKyHMu0MKXVoCfBJJqjwQEd7w0mrYaVOLUonWqNp+nkV\n9oYnHVKq4NURMd6VEaNvXHtWyGw5rNShBRHxdhqXWO859oqbCnvDkw4pVfQLXviG3a1vdvq8Wtgp\nDX/s1DT9/N9NheGQVkNKVfw7jZO2uUw6pFSXuTQENDoc8ltDIVBpOCT3kFLWQ8aUmftQfrKTrB0X\nxRqGQ2bDz3CyK2wk6U86zMv6M0w5uYcvcv8MHyDjcEjuIaWxmTnU8bvdjrl0BNA8HJJjKCT3kFId\nsg4r1XDUUxxJZ9YYn+PkJcD/zZRTixqGvXIPKQH8OlPOqNxDSm2ZMwVA0g9yR2bOOy9zHmQeVoqI\nPwWGJN2XrpRYBbwTWA8sl3Rbh/3LOqQEfK3D9u3IOqwUEe+aZLUk/VNHvcs8pJTcMlk/JV3eYV7W\ncwrpMtAtkram+QHgJOABGvehdPrmm3tICeDUiHjJOH1cD/x9hT7mHlJqy1waArqeya84ObrDvNxD\nSln7lzJzH8rfDbxW0rMR8V+BvwSOAV4LXCTp9zrMyz2kNLZAraZRoB4AzqhyIr2Gn+H/Hm8xjTH3\nJZI62unKPaSUMnP38R7g+InWd3pyMyJuBt4u6eGIeA3wLzSOwF8NPKv0L2U7yHuA/FdS5e7jlFz3\nP9acOQLghZehjTqSxp73YxXycg8p5e5fHbZLejZNnwBcLmkj8C8R0dbdv81qGFI6F/himj6Nxh/b\ngTQK1CU0CvW00gv/v3UAfwScD9wEfKRCXvYhpdx9pHGdfc4rWF4kafRfxf4xjX8c9cl0n8vtnYbV\ndCVV1j6Sf0ipLXOmAEh6/vApGnfdfhCYD/yZpO9UyMs6pJS7f0nuYaUdEbEY2EzjlvnmN4MXdRpW\nw5BS1gKVZB9Wioh5wBk0jqBuBk6W9LOKWbmHlEZzs/WR/OcUmvfUj6Jxl/HoDWqdh+UfUsreR/IP\nKbVlzhQAgIg4lsYb69PARyRN9tk2rbLqGLLJ1r/kA5PcdVmlj38N/JjGv+u8evSSw1Sw7q/Qv7F7\n7IcAL6Wxx/4ZOt9jz1qgkscjYlmuYaWIOJvG814DHJdhz3i8j/Z4frgG6LgA1NDH3OcUvh8RVwCP\nAAuB7wOk177KG+EVwNuBrWm45ms0hmteA3wO6Gi4pqY+fnWCPh7SRR9bmkvnAG6hceXOx4EfjV1f\n4Q95vM8KeX7IRh1+5k7u/tXRx5Q5j8YHUW1uWvZiGr8rIx1mPf/5JhHxZeBmSZek+Y7HPCPiBODz\nNArUNyX9aVr+JuA8SW/tJC9tm/u8xw4aQ3qP88IdiMp3nTZlNw/X/ITGTkSnH32RvY81nFMI4A+B\nxcAVSp8gm654W6QOP80zIu4cfU4R8Qlgh6TzRodrqrwms6GPbT3uHCoAQ2T82IEx2c1DNh+tMmRT\nZ/9Sfo4+nqf0SZ8RcYqkrzWte/6mrg7ybgXeSmOPfT1wVNNRxb2SOr5DM2eBStvmLlKTfZBZxydE\nU+bY4ZqLuxiuqaWPTdlZilROEXGXpFel6VuBC0ffoJvfeKfTtPVRkr8m+AKOBW6kcYZ/YLr7U3cf\ngVvHmx5vvs28E4CHgF8CX2ha/ibg2xXyzmuaPmXMuo9Wfc409uJ2Ax4FXtm07t4Z8PqeDfwbcClw\nwHT3Z5J+zqMxTHEvjWG/l3WR9STwxDhfTwJPVMi7hMYw0CU07oLeJS1fDPx4rvaxna+5dASQe+81\n95BS1v7V1MfnL4kce3lk1cslMw8pTXhXbNXL6HIPK0XEk4x/pFf1voLsQ0o19LH5nMLHNA2faTOZ\n3MM1dZiuPs6lApD1zSH3kE1Nb14zuo81FOXsBSptm3VYKac6h2tyqfO8h9VrLl0FFBNMjzffkvJ/\nkmXW/kEtfTwkGnfFBo3P7h+9Qzaodgv9qcDo5ZkX8sJLLo+j8S/wOqEJpsebb0tTkdrcXKQkPRUR\nH63Qx6xmwht8Gw6c7g5MJvcRTx2mq49z6Z/CZ31ziIjzmqZPGbPuo53mUdObV9N0132UtLOkPSXt\nIWlemh6d36VKFyeYHm++HYdExBPpj+XVaXp0/lUV8qBRpEaNvaT2uIqZ2UTEk03Ps/nryRj/Iyym\nnKT1k33NgP7t0fS7vOeY3+tpf/OH6evjXCoAud8ccr8xFPfmReaiV0OBghqOzHKaDW9es6FI2fjm\nzBCQpJ0zR+YeUsrdv7H9mHFvXuQfUqpD9iOz0kjK9Q90bIrNmQJQg9nwxjCj+1hT0cttNhQps1rM\nmauAcouI54CnSG8MwLbRVcBuXQw5ZDMb+mhmM5cLgJlZoebSSWAzM+uAC4CZWaFcAMzMCuUCYGZW\nKBcAM7NC/X/a1s2vHbNIOwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a0d714f28>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(pd.Series(yTest).value_counts(), end='\\n\\n')\n",
    "print(pd.Series(model.predict(xTest)).value_counts())\n",
    "\n",
    "pd.Series(yTest).hist(xrot=90, alpha=0.7)\n",
    "pd.Series(model.predict(xTest)).hist(xrot=90, alpha=0.7)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In next 2 models ie **IE** and **NS** models we can see that class distribution is heavily biased, this accounts for high accuracy in our results for these specific models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-06T18:49:49.872975Z",
     "start_time": "2018-04-06T18:49:49.524188Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I    1680\n",
      "E     489\n",
      "dtype: int64\n",
      "\n",
      "I    2044\n",
      "E     125\n",
      "dtype: int64\n",
      "\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "          E       0.62      0.16      0.25       489\n",
      "          I       0.80      0.97      0.88      1680\n",
      "\n",
      "avg / total       0.76      0.79      0.74      2169\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAD8CAYAAACRkhiPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAEhVJREFUeJzt3W+M5dV93/H3pyR2XJzIUMwI725Y\nIq2jgtVuzASQLFtT0fBPUcAtruGBIbalsRFUsZQHZdMHWEYI08SxippsNC4rg2RDUAlhFW1ib1Bu\nUCQTA86UP8aUARMz7ApqY2zGrGh28+2D+5v4ZpmZnbl39l7Eeb+kq3vv957zO2ee/D7zO/c3c1JV\nSJLa9C8mPQFJ0uQYApLUMENAkhpmCEhSwwwBSWqYISBJDTMEJKlhhoAkNcwQkKSG/cykJ3Asp5xy\nSm3fvn2ovj/5yU848cQTN3dCkjQGo5y/Hnnkke9X1bvX0/ZNHwLbt2/n4YcfHqpvr9djZmZmcyck\nSWMwyvkryd+vt63LQZLUMENAkhpmCEhSwwwBSWqYISBJDTMEJKlhhoAkNcwQkKSGGQKS1LA3/V8M\nS9JEffWjkxn3PdeMZZhjXgkk2Zbkr5I8meSJJL/V1U9Osj/J093zSV09SW5NspDk0STvHzjW1V37\np5Ncffx+LEnSeqxnOegw8NtV9a+B84Brk5wJXA/cX1U7gPu79wAXAzu6xyywG/qhAdwAnAucA9yw\nHBySpMk4ZghU1cGq+lb3+lXgSWALcClwe9fsduCy7vWlwB3V9yDwriSnARcC+6vq5ar6IbAfuGhT\nfxpJ0oZs6DuBJNuBXwH+FpiqqoPQD4okp3bNtgDPD3Rb7Gqr1VcaZ5b+VQRTU1P0er2NTPOfLC0t\nDd1XkgD4uQsnMuy4zl/rDoEk7wTuAT5TVT9OsmrTFWq1Rv2Nxao5YA5genq6hv13qv4raUkj++ru\niQzbO/masZy/1nWLaJKfpR8AX6mqP+nKL3bLPHTPL3X1RWDbQPetwIE16pKkCVnP3UEBbgOerKrf\nH/hoL7B8h8/VwH0D9au6u4TOA37ULRt9DbggyUndF8IXdDVJ0oSsZznoA8DHgMeSzHe13wE+D9yd\n5JPA94CPdJ/tAy4BFoDXgI8DVNXLSW4EHurafa6qXt6Un0KSNJRjhkBV/Q0rr+cDnL9C+wKuXeVY\ne4A9G5mgJOn48d9GSFLDDAFJapghIEkNMwQkqWGGgCQ1zBCQpIYZApLUMENAkhpmCEhSwwwBSWqY\nISBJDTMEJKlhhoAkNcwQkKSGGQKS1DBDQJIatp7tJfckeSnJ4wO1P04y3z2eW95xLMn2JIcGPvuj\ngT5nJ3ksyUKSW7PGTvWSpPFYz/aSXwb+B3DHcqGqPrr8OskXgB8NtH+mqnaucJzdwCzwIP0tKC8C\n/nzjU5YkbZZjXglU1QPAinsBd7/N/yfgzrWOkeQ04Beq6hvd9pN3AJdtfLqSpM006ncCHwRerKqn\nB2pnJPm7JH+d5INdbQuwONBmsatJkiZoPctBa7mSf34VcBD4xar6QZKzgT9NchYrb1Rfqx00ySz9\npSOmpqbo9XpDTW5paWnovpIEwM9dOJFhx3X+GjoEkvwM8B+As5drVfU68Hr3+pEkzwDvpf+b/9aB\n7luBA6sdu6rmgDmA6enpmpmZGWqOvV6PYftKEgBf3T2RYXsnXzOW89coy0H/HvhOVf3TMk+Sdyc5\noXv9S8AO4NmqOgi8muS87nuEq4D7RhhbkrQJ1nOL6J3AN4BfTrKY5JPdR1fwxi+EPwQ8muR/A/8L\n+HRVLX+pfA3wP4EF4Bm8M0iSJu6Yy0FVdeUq9d9coXYPcM8q7R8G3rfB+UmSjiP/YliSGmYISFLD\nRr1FVJLe0uaff2UyA79nPMN4JSBJDTMEJKlhhoAkNcwQkKSGGQKS1DBDQJIaZghIUsMMAUlqmCEg\nSQ0zBCSpYYaAJDXMEJCkhhkCktSw9ewstifJS0keH6h9NskLSea7xyUDn+1KspDkqSQXDtQv6moL\nSa7f/B9FkrRR67kS+DJw0Qr1L1bVzu6xDyDJmfS3nTyr6/OHSU7o9h3+A+Bi4Ezgyq6tJGmC1rO9\n5ANJtq/zeJcCd1XV68B3kywA53SfLVTVswBJ7urafnvDM5YkbZpRvhO4Lsmj3XLRSV1tC/D8QJvF\nrrZaXZI0QcPuLLYbuBGo7vkLwCeArNC2WDlsarWDJ5kFZgGmpqbo9XpDTXJpaWnovpIEcOj0yycy\n7pExnb+GCoGqenH5dZIvAX/WvV0Etg003Qoc6F6vVl/p+HPAHMD09HTNzMwMM016vR7D9pUkgPlb\nbp7IuK+cu2ss56+hloOSnDbw9sPA8p1De4Erkrw9yRnADuCbwEPAjiRnJHkb/S+P9w4/bUnSZjjm\nlUCSO4EZ4JQki8ANwEySnfSXdJ4DPgVQVU8kuZv+F76HgWur6kh3nOuArwEnAHuq6olN/2kkSRuy\nnruDrlyhfNsa7W8Cblqhvg/Yt6HZSZKOK/9iWJIaZghIUsMMAUlqmCEgSQ0zBCSpYYaAJDXMEJCk\nhhkCktQwQ0CSGmYISFLDDAFJapghIEkNMwQkqWGGgCQ1zBCQpIYZApLUsGOGQJI9SV5K8vhA7XeT\nfCfJo0nuTfKurr49yaEk893jjwb6nJ3ksSQLSW5NstKm9JKkMVrPlcCXgYuOqu0H3ldV/wb4P8Cu\ngc+eqaqd3ePTA/XdwCz9fYd3rHBMSdKYHTMEquoB4OWjal+vqsPd2weBrWsdo9uY/heq6htVVcAd\nwGXDTVmStFk24zuBTwB/PvD+jCR/l+Svk3ywq20BFgfaLHY1SdIEHXOj+bUk+a/AYeArXekg8ItV\n9YMkZwN/muQsYKX1/1rjuLP0l46Ympqi1+sNNb+lpaWh+0oSwKHTL5/IuEfGdP4aOgSSXA38OnB+\nt8RDVb0OvN69fiTJM8B76f/mP7hktBU4sNqxq2oOmAOYnp6umZmZoebY6/UYtq8kAczfcvNExn3l\n3F1jOX8NtRyU5CLgvwC/UVWvDdTfneSE7vUv0f8C+NmqOgi8muS87q6gq4D7Rp69JGkkx7wSSHIn\nMAOckmQRuIH+3UBvB/Z3d3o+2N0J9CHgc0kOA0eAT1fV8pfK19C/0+gd9L9DGPweQZI0AccMgaq6\ncoXybau0vQe4Z5XPHgbet6HZSZKOK/9iWJIaZghIUsMMAUlqmCEgSQ0zBCSpYYaAJDXMEJCkhhkC\nktQwQ0CSGmYISFLDDAFJapghIEkNMwQkqWGGgCQ1zBCQpIYZApLUMENAkhq2rhBIsifJS0keH6id\nnGR/kqe755O6epLcmmQhyaNJ3j/Q5+qu/dPdRvWSpAla75XAl4GLjqpdD9xfVTuA+7v3ABfT32B+\nBzAL7IZ+aNDfn/hc4BzghuXgkCRNxrpCoKoeAF4+qnwpcHv3+nbgsoH6HdX3IPCuJKcBFwL7q+rl\nqvohsJ83BoskaYyOudH8Gqaq6iBAVR1McmpX3wI8P9BusautVn+DJLP0ryKYmpqi1+sNNcGlpaWh\n+0oSwKHTL5/IuEfGdP4aJQRWkxVqtUb9jcWqOWAOYHp6umZmZoaaSK/XY9i+kgQwf8vNExn3lXN3\njeX8NcrdQS92yzx0zy919UVg20C7rcCBNeqSpAkZJQT2Ast3+FwN3DdQv6q7S+g84EfdstHXgAuS\nnNR9IXxBV5MkTci6loOS3AnMAKckWaR/l8/ngbuTfBL4HvCRrvk+4BJgAXgN+DhAVb2c5Ebgoa7d\n56rq6C+bJUljtK4QqKorV/no/BXaFnDtKsfZA+xZ9+wkSceVfzEsSQ0zBCSpYYaAJDXMEJCkhhkC\nktQwQ0CSGmYISFLDDAFJapghIEkNMwQkqWGGgCQ1zBCQpIYZApLUMENAkhpmCEhSwwwBSWrY0CGQ\n5JeTzA88fpzkM0k+m+SFgfolA312JVlI8lSSCzfnR5AkDWtdO4utpKqeAnYCJDkBeAG4l/52kl+s\nqt8bbJ/kTOAK4CzgPcBfJnlvVR0Zdg6SpNFs1nLQ+cAzVfX3a7S5FLirql6vqu/S34P4nE0aX5I0\nhM0KgSuAOwfeX5fk0SR7kpzU1bYAzw+0WexqkqQJGXo5aFmStwG/AezqSruBG4Hqnr8AfALICt1r\nlWPOArMAU1NT9Hq9oea2tLQ0dF9JAjh0+uUTGffImM5fI4cAcDHwrap6EWD5GSDJl4A/694uAtsG\n+m0FDqx0wKqaA+YApqena2ZmZqiJ9Xo9hu0rSQDzt9w8kXFfOXfXWM5fm7EcdCUDS0FJThv47MPA\n493rvcAVSd6e5AxgB/DNTRhfkjSkka4EkvxL4NeATw2U/1uSnfSXep5b/qyqnkhyN/Bt4DBwrXcG\nSdJkjRQCVfUa8K+Oqn1sjfY3ATeNMqYkafP4F8OS1DBDQJIaZghIUsMMAUlqmCEgSQ0zBCSpYYaA\nJDXMEJCkhhkCktQwQ0CSGmYISFLDDAFJapghIEkNMwQkqWGGgCQ1zBCQpIaNHAJJnkvyWJL5JA93\ntZOT7E/ydPd8UldPkluTLCR5NMn7Rx1fkjS8zboS+HdVtbOqprv31wP3V9UO4P7uPfQ3pd/RPWaB\n3Zs0viRpCMdrOehS4Pbu9e3AZQP1O6rvQeBdR21ML0kao80IgQK+nuSRJLNdbaqqDgJ0z6d29S3A\n8wN9F7uaJGkCRtpovvOBqjqQ5FRgf5LvrNE2K9TqDY36YTILMDU1Ra/XG2piS0tLQ/eVJIBDp18+\nkXGPjOn8NXIIVNWB7vmlJPcC5wAvJjmtqg52yz0vdc0XgW0D3bcCB1Y45hwwBzA9PV0zMzNDze32\ne/Zx7/dPHKrvKG77zV8d+5iSjo/5W26eyLivnLuLYc99GzHSclCSE5P8/PJr4ALgcWAvcHXX7Grg\nvu71XuCq7i6h84AfLS8bSZLGb9QrgSng3iTLx/pqVf1FkoeAu5N8Evge8JGu/T7gEmABeA34+Ijj\nS5JGMFIIVNWzwL9dof4D4PwV6gVcO8qYkqTN418MS1LDDAFJapghIEkNMwQkqWGGgCQ1zBCQpIYZ\nApLUMENAkhpmCEhSwwwBSWqYISBJDTMEJKlhhoAkNcwQkKSGGQKS1DBDQJIaZghIUsOGDoEk25L8\nVZInkzyR5Le6+meTvJBkvntcMtBnV5KFJE8luXAzfgBJ0vBG2V7yMPDbVfWtbrP5R5Ls7z77YlX9\n3mDjJGcCVwBnAe8B/jLJe6vqyAhzkCSNYOgrgao6WFXf6l6/CjwJbFmjy6XAXVX1elV9l/5m8+cM\nO74kaXQjbTS/LMl24FeAvwU+AFyX5CrgYfpXCz+kHxAPDnRbZJXQSDILzAJMTU3R6/WGmtc7TzjM\nh37+/w7VdxTDzlfSm8+h0y+fyLhHlpbGci4ZOQSSvBO4B/hMVf04yW7gRqC65y8AnwCyQvda6ZhV\nNQfMAUxPT9fMzMxQc7v9nn088Oq7h+o7itv+46+OfUxJx8f8LTdPZNxXzt3FsOe+jRjp7qAkP0s/\nAL5SVX8CUFUvVtWRqvpH4Ev8dMlnEdg20H0rcGCU8SVJoxnl7qAAtwFPVtXvD9RPG2j2YeDx7vVe\n4Iokb09yBrAD+Oaw40uSRjfKctAHgI8BjyWZ72q/A1yZZCf9pZ7ngE8BVNUTSe4Gvk3/zqJrvTNI\nkiZr6BCoqr9h5XX+fWv0uQm4adgxJUmby78YlqSGGQKS1DBDQJIaZghIUsMMAUlqmCEgSQ0zBCSp\nYZvyD+TerE79hxf4zy/+9wmM/LUJjClJG+eVgCQ1zBCQpIYZApLUMENAkhpmCEhSwwwBSWqYISBJ\nDTMEJKlhYw+BJBcleSrJQpLrxz2+JOmnxhoCSU4A/gC4GDiT/laUZ45zDpKknxr3lcA5wEJVPVtV\n/w+4C7h0zHOQJHXGHQJbgOcH3i92NUnSBIz7H8ittDF9vaFRMgvMdm+Xkjw15HinAN8fsu/wrl/p\nx5Skjfj6KOev09fbcNwhsAhsG3i/FThwdKOqmgPmRh0sycNVNT3qcSRp3MZ1/hr3ctBDwI4kZyR5\nG3AFsHfMc5AkdcZ6JVBVh5NcR/8f7p8A7KmqJ8Y5B0nST419U5mq2gfsG9NwIy8pSdKEjOX8lao3\nfC8rSWqE/zZCkhr2ltxjOMkR4LGB0l1V9flJzUeSNirJUlW983iP85YMAeBQVe2c9CQk6c3O5SBJ\nathbNQTekWR+4PHRSU9Ikt6MXA6SpIa9Va8EJEnrYAhIUsPeqstB70gyP/D+L6rKXcwk6Sj+xbAk\nNczlIElqmCEgSQ0zBCSpYYaAJDXMEJCkhhkCktQwQ0CSGmYISFLD/j+q8+EuqXXOSwAAAABJRU5E\nrkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a0d29e1d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(pd.Series(yIETest).value_counts(), end='\\n\\n')\n",
    "print(pd.Series(ieModel.predict(xIETest)).value_counts(), end='\\n\\n')\n",
    "print(classification_report(y_pred=ieModel.predict(xIETest), y_true=yIETest))\n",
    "\n",
    "pd.Series(yIETest).hist(alpha=0.7)\n",
    "pd.Series(ieModel.predict(xIETest)).hist(alpha=0.7)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-06T18:49:52.796423Z",
     "start_time": "2018-04-06T18:49:52.523698Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "N    1859\n",
      "S     310\n",
      "dtype: int64\n",
      "\n",
      "N    2065\n",
      "S     104\n",
      "dtype: int64\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAD8CAYAAACRkhiPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAEyBJREFUeJzt3X+MXeV95/H3p7TxdpO2OCUZubaL\nqeREC1HXDSPCKtrsrOjyS20hFWnNrgJNkCapoNusstJCGwkUhFTtlkZit6WyN15ASnEQJMWKTB0X\n9SqtFBp+1MuPEIohOExswSaEkAlZCux3/7hnmlsztmfOHd+J/Lxf0tU993uf5zzP/ed85px77jyp\nKiRJbfqx1Z6AJGn1GAKS1DBDQJIaZghIUsMMAUlqmCEgSQ0zBCSpYYaAJDXMEJCkhv34ak/gWE45\n5ZTatGlTr77f//73efOb37yyE5KkCRjn+PXggw9+q6retpS2P/IhsGnTJh544IFefQeDATMzMys7\nIUmagHGOX0kOLLWtl4MkqWGGgCQ1zBCQpIYZApLUMENAkhpmCEhSwwwBSWqYISBJDTMEJKlhP/K/\nGB7LC0/Dn908+XH//WcnP6Yk9eCZgCQ1zBCQpIYdMwSSbEzyV0keT/JYkt/t6m9NsjfJk93z2q6e\nJDcl2Z/k4STvHtnX5V37J5Ncfvw+liRpKZZyJvAa8PGq+hfA2cCVSU4HrgburarNwL3da4ALgM3d\nYxa4GYahAVwLvAc4C7h2ITgkSavjmCFQVYeq6qFu+3vA48B64CLg1q7ZrcDF3fZFwG01dB9wcpJ1\nwHnA3qp6oaq+A+wFzl/RTyNJWpZl3R2UZBPwS8DfAlNVdQiGQZHk7V2z9cCzI93mutqR6ouNM8vw\nLIKpqSkGg8FypvmP5n/sZxj8s/N69R1Lz/lK0oL5+fnex77lWHIIJHkLcBfwsap6KckRmy5Sq6PU\n31is2gZsA5ienq6+CysMPreDmf+7p1ffscx4i6ik8UxqUawl3R2U5CcYBsBnqupzXfm57jIP3fPz\nXX0O2DjSfQNw8Ch1SdIqWcrdQQE+DTxeVX808tYuYOEOn8uBu0fql3V3CZ0NfLe7bLQHODfJ2u4L\n4XO7miRplSzlctB7gQ8CjyTZ19V+D/gD4I4kVwDfAD7QvbcbuBDYD7wMfAigql5Icj1wf9fuk1X1\nwop8CklSL8cMgar6Gxa/ng9wziLtC7jyCPvaAexYzgQlScePvxiWpIYZApLUMENAkhpmCEhSwwwB\nSWqYISBJDTMEJKlhhoAkNeyEXmP4B//wOvuefXHi426Z+IiS1I9nApLUMENAkhpmCEhSwwwBSWqY\nISBJDTMEJKlhhoAkNWwpy0vuSPJ8kkdHap9Nsq97PLOw4liSTUl+MPLen470OTPJI0n2J7kpR1mp\nXpI0GUv5sdgtwP8AblsoVNVvLmwnuRH47kj7p6pqsd9L3QzMAvcxXILyfOCe5U9ZkrRSjnkmUFVf\nAhZdC7j7a/43gNuPto8k64Cfrqovd8tP3gZcvPzpSpJW0rj/NuJfA89V1ZMjtdOS/B3wEvCJqvpr\nYD0wN9JmrqstKsksw7MGpqamGAwGvSb36pq1HDr1kl59x/Fiz/lK0oL5+fnex77lGDcELuWfngUc\nAn6+qr6d5Ezgz5OcweIL1deRdlpV24BtANPT0zUzM9Nrcvfs3M66A3f26juOLVv3THxMSSeWwWBA\n32PfcvQOgSQ/Dvw6cOZCrapeAV7pth9M8hTwDoZ/+W8Y6b4BONh3bEnSyhjnFtFfBr5WVf94mSfJ\n25Kc1G3/ArAZeLqqDgHfS3J29z3CZcDdY4wtSVoBS7lF9Hbgy8A7k8wluaJ7aytv/EL4fcDDSf43\ncCfw0apa+FL5t4H/CewHnsI7gyRp1R3zclBVXXqE+m8tUrsLuOsI7R8A3rXM+UmSjiN/MSxJDTME\nJKlhhoAkNcwQkKSGGQKS1DBDQJIaZghIUsMMAUlqmCEgSQ0zBCSpYYaAJDXMEJCkhhkCktQwQ0CS\nGmYISFLDDAFJathSVhbbkeT5JI+O1K5L8s0k+7rHhSPvXZNkf5Inkpw3Uj+/q+1PcvXKfxRJ0nIt\n5UzgFuD8Reqfqqot3WM3QJLTGS47eUbX50+SnNStO/zHwAXA6cClXVtJ0ipayvKSX0qyaYn7uwjY\nWVWvAF9Psh84q3tvf1U9DZBkZ9f2q8uesSRpxRwzBI7iqiSXAQ8AH6+q7wDrgftG2sx1NYBnD6u/\n50g7TjILzAJMTU0xGAx6TfDVNWs5dOolvfqO48We85WkBfPz872PfcvRNwRuBq4Hqnu+EfgwkEXa\nFotfdqoj7byqtgHbAKanp2tmZqbXJO/ZuZ11B+7s1XccW7bumfiYkk4sg8GAvse+5egVAlX13MJ2\nku3AF7qXc8DGkaYbgIPd9pHqkqRV0usW0STrRl6+H1i4c2gXsDXJmiSnAZuBrwD3A5uTnJbkTQy/\nPN7Vf9qSpJVwzDOBJLcDM8ApSeaAa4GZJFsYXtJ5BvgIQFU9luQOhl/4vgZcWVWvd/u5CtgDnATs\nqKrHVvzTSJKWZSl3B126SPnTR2l/A3DDIvXdwO5lzU6SdFz5i2FJapghIEkNMwQkqWGGgCQ1zBCQ\npIYZApLUMENAkhpmCEhSwwwBSWqYISBJDTMEJKlhhoAkNcwQkKSGGQKS1DBDQJIaZghIUsOOGQJJ\ndiR5PsmjI7X/luRrSR5O8vkkJ3f1TUl+kGRf9/jTkT5nJnkkyf4kNyVZbFF6SdIELeVM4Bbg/MNq\ne4F3VdUvAn8PXDPy3lNVtaV7fHSkfjMwy3Dd4c2L7FOSNGHHDIGq+hLwwmG1L1bVa93L+4ANR9tH\ntzD9T1fVl6uqgNuAi/tNWZK0Uo65xvASfBj47Mjr05L8HfAS8Imq+mtgPTA30mauqy0qySzDswam\npqYYDAa9JvbqmrUcOvWSXn3H8WLP+UrSgvn5+d7HvuUYKwSS/D7wGvCZrnQI+Pmq+naSM4E/T3IG\nsNj1/zrSfqtqG7ANYHp6umZmZnrN756d21l34M5efcexZeueiY8p6cQyGAzoe+xbjt4hkORy4FeA\nc7pLPFTVK8Ar3faDSZ4C3sHwL//RS0YbgIN9x5YkrYxet4gmOR/4L8CvVdXLI/W3JTmp2/4Fhl8A\nP11Vh4DvJTm7uyvoMuDusWcvSRrLMc8EktwOzACnJJkDrmV4N9AaYG93p+d93Z1A7wM+meQ14HXg\no1W18KXybzO80+gngXu6hyRpFR0zBKrq0kXKnz5C27uAu47w3gPAu5Y1O0nSceUvhiWpYYaAJDXM\nEJCkhhkCktQwQ0CSGmYISFLDDAFJapghIEkNMwQkqWGGgCQ1zBCQpIYZApLUMENAkhpmCEhSwwwB\nSWqYISBJDVtSCCTZkeT5JI+O1N6aZG+SJ7vntV09SW5Ksj/Jw0nePdLn8q79k90axZKkVbTUM4Fb\ngPMPq10N3FtVm4F7u9cAFzBcW3gzMAvcDMPQYLg05XuAs4BrF4JDkrQ6lhQCVfUl4IXDyhcBt3bb\ntwIXj9Rvq6H7gJOTrAPOA/ZW1QtV9R1gL28MFknSBI3zncBUVR0C6J7f3tXXA8+OtJvrakeqS5JW\nyTEXmu8hi9TqKPU37iCZZXgpiampKQaDQa+JvLpmLYdOvaRX33G82HO+krRgfn6+97FvOcYJgeeS\nrKuqQ93lnue7+hywcaTdBuBgV585rD5YbMdVtQ3YBjA9PV0zMzOLNTume3ZuZ92BO3v1HceWrXsm\nPqakE8tgMKDvsW85xrkctAtYuMPncuDukfpl3V1CZwPf7S4X7QHOTbK2+0L43K4mSVolSzoTSHI7\nw7/iT0kyx/Aunz8A7khyBfAN4ANd893AhcB+4GXgQwBV9UKS64H7u3afrKrDv2yWJE3QkkKgqi49\nwlvnLNK2gCuPsJ8dwI4lz06SdFz5i2FJapghIEkNMwQkqWGGgCQ1zBCQpIYZApLUMENAkhpmCEhS\nwwwBSWqYISBJDTMEJKlhhoAkNcwQkKSGGQKS1DBDQJIaZghIUsN6h0CSdybZN/J4KcnHklyX5Jsj\n9QtH+lyTZH+SJ5KctzIfQZLUV++F5qvqCWALQJKTgG8Cn2e4nOSnquoPR9snOR3YCpwB/Bzwl0ne\nUVWv952DJGk8K3U56Bzgqao6cJQ2FwE7q+qVqvo6wzWIz1qh8SVJPaxUCGwFbh95fVWSh5PsSLK2\nq60Hnh1pM9fVJEmrpPfloAVJ3gT8GnBNV7oZuB6o7vlG4MNAFuleR9jnLDALMDU1xWAw6DW3V9es\n5dCpl/TqO44Xe85XkhbMz8/3PvYtx9ghAFwAPFRVzwEsPAMk2Q58oXs5B2wc6bcBOLjYDqtqG7AN\nYHp6umZmZnpN7J6d21l34M5efcexZeueiY8p6cQyGAzoe+xbjpW4HHQpI5eCkqwbee/9wKPd9i5g\na5I1SU4DNgNfWYHxJUk9jXUmkOSfA/8O+MhI+b8m2cLwUs8zC+9V1WNJ7gC+CrwGXOmdQZK0usYK\ngap6GfjZw2ofPEr7G4AbxhlTkrRy/MWwJDXMEJCkhhkCktQwQ0CSGmYISFLDDAFJapghIEkNMwQk\nqWGGgCQ1zBCQpIYZApLUMENAkhpmCEhSwwwBSWqYISBJDTMEJKlhY4dAkmeSPJJkX5IHutpbk+xN\n8mT3vLarJ8lNSfYneTjJu8cdX5LU30qdCfzbqtpSVdPd66uBe6tqM3Bv9xqGi9Jv7h6zwM0rNL4k\nqYfjdTnoIuDWbvtW4OKR+m01dB9w8mEL00uSJmglQqCALyZ5MMlsV5uqqkMA3fPbu/p64NmRvnNd\nTZK0CsZaaL7z3qo6mOTtwN4kXztK2yxSqzc0GobJLMDU1BSDwaDXxF5ds5ZDp17Sq+84Xuw5X0la\nMD8/3/vYtxxjh0BVHeyen0/yeeAs4Lkk66rqUHe55/mu+RywcaT7BuDgIvvcBmwDmJ6erpmZmV5z\nu2fndtYduLNX33Fs2bpn4mNKOrEMBgP6HvuWY6zLQUnenOSnFraBc4FHgV3A5V2zy4G7u+1dwGXd\nXUJnA99duGwkSZq8cc8EpoDPJ1nY159V1V8kuR+4I8kVwDeAD3TtdwMXAvuBl4EPjTm+JGkMY4VA\nVT0N/MtF6t8GzlmkXsCV44wpSVo5/mJYkhpmCEhSwwwBSWqYISBJDTMEJKlhhoAkNcwQkKSGGQKS\n1DBDQJIaZghIUsMMAUlqmCEgSQ0zBCSpYYaAJDVsJZaXlKQT1hW33L8q435w02TG8UxAkhrWOwSS\nbEzyV0keT/JYkt/t6tcl+WaSfd3jwpE+1yTZn+SJJOetxAeQJPU3zuWg14CPV9VD3TrDDybZ2733\nqar6w9HGSU4HtgJnAD8H/GWSd1TV62PMQZI0ht5nAlV1qKoe6ra/BzwOrD9Kl4uAnVX1SlV9neE6\nw2f1HV+SNL4V+U4gySbgl4C/7UpXJXk4yY4ka7vaeuDZkW5zHD00JEnH2dh3ByV5C3AX8LGqeinJ\nzcD1QHXPNwIfBrJI9zrCPmeBWYCpqSkGg0Gvub26Zi2HTr2kV99xvNhzvpJ+9Lzvp15elXHn5/9f\n72PfcowVAkl+gmEAfKaqPgdQVc+NvL8d+EL3cg7YONJ9A3Bwsf1W1TZgG8D09HTNzMz0mt89O7ez\n7sCdvfqOY8vWPRMfU9LxsWq3iP7s9+l77FuOce4OCvBp4PGq+qOR+rqRZu8HHu22dwFbk6xJchqw\nGfhK3/ElSeMb50zgvcAHgUeS7OtqvwdcmmQLw0s9zwAfAaiqx5LcAXyV4Z1FV3pnkCStrt4hUFV/\nw+LX+Xcfpc8NwA19x5QkrSx/MSxJDTMEJKlhhoAkNcwQkKSGGQKS1DBDQJIaZghIUsMMAUlqmCEg\nSQ1zjWFJOorfee4TqzLui5uumcg4nglIUsMMAUlqmCEgSQ0zBCSpYYaAJDXMEJCkhk08BJKcn+SJ\nJPuTXD3p8SVJPzTREEhyEvDHwAXA6QyXojx9knOQJP3QpM8EzgL2V9XTVfUPwE7gognPQZLUmXQI\nrAeeHXk919UkSatg0v82YrGF6esNjZJZYLZ7OZ/kiZ7jnQJ8q2ff/q5e7GNK0nJ8cZzj16lLbTjp\nEJgDNo683gAcPLxRVW0Dto07WJIHqmp63P1I0qRN6vg16ctB9wObk5yW5E3AVmDXhOcgSepM9Eyg\nql5LchWwBzgJ2FFVj01yDpKkH5r4v5Kuqt3A7gkNN/YlJUlaJRM5fqXqDd/LSpIa4b+NkKSGnXAh\nkKSS3Djy+j8nuW4VpyRJy5Lk95M8luThJPuSvOd4jXXChQDwCvDrSU5Z7YlI0nIl+VfArwDvrqpf\nBH6Zf/oj2xV1IobAawy/UPlPqz0RSephHfCtqnoFoKq+VVVv+D3VSjkRQwCG/6TuPyT5mdWeiCQt\n0xeBjUn+PsmfJPk3x3OwEzIEquol4DbgP672XCRpOapqHjiT4b/O+T/AZ5P81vEa74S7RTTJfFW9\nJclbgYeA/8Xwc163ujOTpOVLcglweVX96vHY/wl5JgBQVS8AdwBXrPZcJGmpkrwzyeaR0hbgwPEa\nb+K/GJ6wG4GrVnsSkrQMbwH+e5KTGd7osp8f/lflFXfCXQ6SJC3dCXs5SJJ0bIaAJDXMEJCkhhkC\nktQwQ0CSGmYISFLDDAFJapghIEkN+//zdhAPZxxKMAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a0cc41a20>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(pd.Series(yNSTest).value_counts(), end='\\n\\n')\n",
    "print(pd.Series(nsModel.predict(xNSTest)).value_counts())\n",
    "\n",
    "pd.Series(yNSTest).hist(alpha=0.7)\n",
    "pd.Series(nsModel.predict(xNSTest)).hist(alpha=0.7)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-06T18:49:55.585675Z",
     "start_time": "2018-04-06T18:49:55.390701Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "F    1189\n",
      "T     980\n",
      "dtype: int64\n",
      "\n",
      "F    1294\n",
      "T     875\n",
      "dtype: int64\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAD2hJREFUeJzt3XFsndV5x/Hv02SElqpNgNXLkogw\nNdrGqi1DVmCrtF01Uwu0WpgEUuhUIhbJmkS7bkwayTaJaVNF0aqyVuqQTJM1ldqmLOtENIWlUdqr\nan/AgDaCUspiwULceEAVzOaSDpI9++OebLfBwc772vdGOd+PZPl9z/uc9xz/c35+z73XjsxEklSf\ntwx7ApKk4TAAJKlSBoAkVcoAkKRKGQCSVCkDQJIqZQBIUqUMAEmqlAEgSZVaOuwJvJnLL788165d\n27j/j370Iy655JKFm5AkDUib9evxxx//YWb+9Fx153UArF27lscee6xx/263S6fTWbgJSdKAtFm/\nIuLIfOrcApKkShkAklQpA0CSKmUASFKlDABJqpQBIEmVMgAkqVIGgCRVygCQpEqd158Ebu34s/Dl\n+wY/7oe/OvgxJekc+QQgSZUyACSpUgaAJFXKAJCkShkAklQpA0CSKmUASFKlDABJqpQBIEmVMgAk\nqVJzBkBE7IyIFyPiu31tfx0R34+IJyLiHyNied+17RExERHPRMQH+tqvK20TEbFt4X8USdK5mM8T\nwBeA685oOwC8JzN/Gfg3YDtARFwFbAZ+qfT524hYEhFLgM8B1wNXAbeUWknSkMwZAJn5LeD4GW1f\nz8yT5fRhYHU53gTszsz/zszngAlgQ/mayMxnM/M1YHeplSQNyUK8BvB7wEPleBVwtO/aZGk7W7sk\naUha/TnoiPgz4CTwpdNNs5QlswdNnuWeY8AYwMjICN1ut/H8Zt7yTroXf2DuwoXWYs6SBDAzM9Nq\n/ZuPxgEQEVuADwEbM/P0Yj4JrOkrWw0cK8dna/8JmTkOjAOMjo5mp9NpOkW6X9tJ58f7G/dvrOP/\nA5DUTrfbpc36Nx+NAiAirgPuBH4zM1/tu7QX+HJEfBr4WWAd8K/0ngzWRcSVwA/ovVD84TYTn48T\nr53i0NHpxR7mDdYPfERJOndzBkBEfAXoAJdHxCRwF713/SwDDkQEwMOZ+fuZ+VREPAB8j97W0O2Z\nearc56PAfmAJsDMzn1qEn0eSNE9zBkBm3jJL8443qf8E8IlZ2vcB+85pdpKkReMngSWpUgaAJFXK\nAJCkShkAklQpA0CSKmUASFKlDABJqpQBIEmVMgAkqVIGgCRVygCQpEoZAJJUKQNAkiplAEhSpQwA\nSaqUASBJlTIAJKlSBoAkVcoAkKRKGQCSVCkDQJIqZQBIUqUMAEmq1JwBEBE7I+LFiPhuX9ulEXEg\nIg6X7ytKe0TEZyNiIiKeiIir+/psKfWHI2LL4vw4kqT5ms8TwBeA685o2wYczMx1wMFyDnA9sK58\njQH3QS8wgLuAa4ANwF2nQ0OSNBxzBkBmfgs4fkbzJmBXOd4F3NjX/sXseRhYHhErgQ8ABzLzeGa+\nDBzgjaEiSRqgpq8BjGTmFED5/q7Svgo42lc3WdrO1i5JGpKlC3y/mKUt36T9jTeIGKO3fcTIyAjd\nbrfxZF5ftoKpK25q3L+p6RZzliSAmZmZVuvffDQNgBciYmVmTpUtnhdL+ySwpq9uNXCstHfOaO/O\nduPMHAfGAUZHR7PT6cxWNi8P7b6flUf2NO7f1PrN+wc+pqQLS7fbpc36Nx9Nt4D2AqffybMFeLCv\n/dbybqBrgVfKFtF+4P0RsaK8+Pv+0iZJGpI5nwAi4iv0fnu/PCIm6b2b55PAAxGxFXgeuLmU7wNu\nACaAV4HbADLzeET8FfBoqfvLzDzzhWVJOu9s/cKjcxctgo+sXfwx5gyAzLzlLJc2zlKbwO1nuc9O\nYOc5zU6StGj8JLAkVcoAkKRKGQCSVCkDQJIqZQBIUqUMAEmqlAEgSZUyACSpUgaAJFXKAJCkShkA\nklQpA0CSKmUASFKlDABJqpQBIEmVWuj/CSxJF5SPvfDnQxl3eu32RR/DJwBJqpQBIEmVMgAkqVIG\ngCRVygCQpEoZAJJUKQNAkirVKgAi4o8i4qmI+G5EfCUiLo6IKyPikYg4HBFfjYiLSu2ycj5Rrq9d\niB9AktRM4wCIiFXAHwCjmfkeYAmwGbgHuDcz1wEvA1tLl63Ay5n5buDeUidJGpK2W0BLgbdGxFLg\nbcAU8D5gT7m+C7ixHG8q55TrGyMiWo4vSWqocQBk5g+ATwHP01v4XwEeB6Yz82QpmwRWleNVwNHS\n92Spv6zp+JKkdhr/LaCIWEHvt/orgWng74HrZynN013e5Fr/fceAMYCRkRG63W7TKfL6shVMXXFT\n4/5NTbeYs6Tzy4khrCEAp2ZmWq1/89Hmj8H9FvBcZr4EEBFfA34dWB4RS8tv+auBY6V+ElgDTJYt\no3cCx8+8aWaOA+MAo6Oj2el0Gk/wod33s/LInrkLF9j6zfsHPqakxXHonruHMu70Ndtps/7NR5vX\nAJ4Hro2It5W9/I3A94BvAqcjcwvwYDneW84p17+RmW94ApAkDUab1wAeofdi7reBJ8u9xoE7gTsi\nYoLeHv+O0mUHcFlpvwPY1mLekqSWWv0/gMy8C7jrjOZngQ2z1P4YuLnNeJKkheMngSWpUgaAJFXK\nAJCkShkAklQpA0CSKmUASFKlDABJqpQBIEmVMgAkqVIGgCRVygCQpEoZAJJUKQNAkiplAEhSpQwA\nSaqUASBJlTIAJKlSBoAkVcoAkKRKGQCSVCkDQJIqZQBIUqUMAEmqVKsAiIjlEbEnIr4fEU9HxK9F\nxKURcSAiDpfvK0ptRMRnI2IiIp6IiKsX5keQJDXR9gngM8A/Z+YvAL8CPA1sAw5m5jrgYDkHuB5Y\nV77GgPtaji1JaqFxAETEO4DfAHYAZOZrmTkNbAJ2lbJdwI3leBPwxex5GFgeESsbz1yS1EqbJ4Cf\nA14C/i4ivhMRn4+IS4CRzJwCKN/fVepXAUf7+k+WNknSECxt2fdq4GOZ+UhEfIb/3+6ZTczSlm8o\nihijt0XEyMgI3W638QRfX7aCqStuaty/qekWc5Z0fjkxhDUE4NTMTKv1bz7aBMAkMJmZj5TzPfQC\n4IWIWJmZU2WL58W++jV9/VcDx868aWaOA+MAo6Oj2el0Gk/wod33s/LInsb9m1q/ef/Ax5S0OA7d\nc/dQxp2+Zjtt1r/5aLwFlJn/ARyNiJ8vTRuB7wF7gS2lbQvwYDneC9xa3g10LfDK6a0iSdLgtXkC\nAPgY8KWIuAh4FriNXqg8EBFbgeeBm0vtPuAGYAJ4tdRKkoakVQBk5iFgdJZLG2epTeD2NuNJkhaO\nnwSWpEoZAJJUKQNAkiplAEhSpQwASaqUASBJlTIAJKlSBoAkVcoAkKRKGQCSVCkDQJIqZQBIUqUM\nAEmqlAEgSZUyACSpUgaAJFXKAJCkShkAklQpA0CSKmUASFKlDABJqpQBIEmVMgAkqVKtAyAilkTE\ndyLin8r5lRHxSEQcjoivRsRFpX1ZOZ8o19e2HVuS1NxCPAF8HHi67/we4N7MXAe8DGwt7VuBlzPz\n3cC9pU6SNCStAiAiVgMfBD5fzgN4H7CnlOwCbizHm8o55frGUi9JGoK2TwB/A/wJ8D/l/DJgOjNP\nlvNJYFU5XgUcBSjXXyn1kqQhWNq0Y0R8CHgxMx+PiM7p5llKcx7X+u87BowBjIyM0O12m06R15et\nYOqKmxr3b2q6xZwlnV9ODGENATg1M9Nq/ZuPxgEAvBf47Yi4AbgYeAe9J4LlEbG0/Ja/GjhW6ieB\nNcBkRCwF3gkcP/OmmTkOjAOMjo5mp9NpPMGHdt/PyiN75i5cYOs37x/4mJIWx6F77h7KuNPXbKfN\n+jcfjbeAMnN7Zq7OzLXAZuAbmfm7wDeB05G5BXiwHO8t55Tr38jMNzwBSJIGYzE+B3AncEdETNDb\n499R2ncAl5X2O4BtizC2JGme2mwB/Z/M7ALdcvwssGGWmh8DNy/EeJKk9vwksCRVygCQpEoZAJJU\nKQNAkiplAEhSpQwASaqUASBJlTIAJKlSBoAkVcoAkKRKGQCSVCkDQJIqZQBIUqUMAEmqlAEgSZUy\nACSpUgaAJFXKAJCkShkAklQpA0CSKmUASFKlDABJqpQBIEmVahwAEbEmIr4ZEU9HxFMR8fHSfmlE\nHIiIw+X7itIeEfHZiJiIiCci4uqF+iEkSeeuzRPASeCPM/MXgWuB2yPiKmAbcDAz1wEHyznA9cC6\n8jUG3NdibElSS40DIDOnMvPb5fi/gKeBVcAmYFcp2wXcWI43AV/MnoeB5RGxsvHMJUmtLMhrABGx\nFvhV4BFgJDOnoBcSwLtK2SrgaF+3ydImSRqCpW1vEBFvB/4B+MPM/M+IOGvpLG05y/3G6G0RMTIy\nQrfbbTy315etYOqKmxr3b2q6xZwlnV9ODGENATg1M9Nq/ZuPVgEQET9Fb/H/UmZ+rTS/EBErM3Oq\nbPG8WNongTV93VcDx868Z2aOA+MAo6Oj2el0Gs/vod33s/LInsb9m1q/ef/Ax5S0OA7dc/dQxp2+\nZjtt1r/5aPMuoAB2AE9n5qf7Lu0FtpTjLcCDfe23lncDXQu8cnqrSJI0eG2eAN4LfAR4MiIOlbY/\nBT4JPBARW4HngZvLtX3ADcAE8CpwW4uxJUktNQ6AzPwXZt/XB9g4S30CtzcdT5K0sPwksCRVygCQ\npEoZAJJUKQNAkiplAEhSpQwASaqUASBJlTIAJKlSBoAkVcoAkKRKGQCSVCkDQJIqZQBIUqUMAEmq\nlAEgSZUyACSpUgaAJFXKAJCkShkAklQpA0CSKmUASFKlDABJqpQBIEmVGngARMR1EfFMRExExLZB\njy9J6hloAETEEuBzwPXAVcAtEXHVIOcgSeoZ9BPABmAiM5/NzNeA3cCmAc9BksTgA2AVcLTvfLK0\nSZIGbOmAx4tZ2vInCiLGgLFyOhMRz7QY73Lghy36N7Ntth9Tks7F19usX1fMp2jQATAJrOk7Xw0c\n6y/IzHFgfCEGi4jHMnN0Ie4lSYM0iPVr0FtAjwLrIuLKiLgI2AzsHfAcJEkM+AkgM09GxEeB/cAS\nYGdmPjXIOUiSega9BURm7gP2DWi4BdlKkqQhWPT1KzJz7ipJ0gXHPwUhSZUa+BbQIETEKeDJvqYb\nM/PfhzQdSZqXiLgMOFhOfwY4BbxUzjeUD9Au3HgX4hZQRMxk5tuHPQ9Jaioi/gKYycxPLdYYbgFJ\nUqUuyC0g4K0RcagcP5eZvzPU2UjSeehCDYATmbl+2JOQpPOZW0CSVCkDQJIqZQBIUqUuyLeBSpLm\n5hOAJFXKAJCkShkAklQpA0CSKmUASFKlDABJqpQBIEmVMgAkqVL/CwTIrkcSSSi+AAAAAElFTkSu\nQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a0d7dbbe0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(pd.Series(yFTTest).value_counts(), end='\\n\\n')\n",
    "print(pd.Series(ftModel.predict(xFTTest)).value_counts())\n",
    "\n",
    "pd.Series(yFTTest).hist(alpha=0.7)\n",
    "pd.Series(ftModel.predict(xFTTest)).hist(alpha=0.7)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-06T18:49:58.544826Z",
     "start_time": "2018-04-06T18:49:58.275451Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "P    1310\n",
      "J     859\n",
      "dtype: int64\n",
      "\n",
      "P    1601\n",
      "J     568\n",
      "dtype: int64\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAEmNJREFUeJzt3X+MXWed3/H3p3YTFtjdOJiMsrYV\ne4t324B2aTQkaVdFU9LND7rCkTaRHLXFpZZG0EBpabXE3UqRdoVC2qopqDTSsHbjSGyy2UA31so0\npIHbbKUmhFBvSMjSTIOIB3sx1HHKEC/B3m//uMfL4Iwz43tn7nXzvF/SaM75nuc5z3P/OZ85z713\nTqoKSVJ7/tK4JyBJGg8DQJIaZQBIUqMMAElqlAEgSY0yACSpUQaAJDXKAJCkRhkAktSoteOewKtZ\nv359bd68eeD+P/jBD3jDG96wchOSpBEZ5vr1xBNPfK+q3rxUu3M6ADZv3sxXvvKVgfv3ej2mpqZW\nbkKSNCLDXL+SfGs57VwCkqRGGQCS1CgDQJIaZQBIUqMMAElq1JIBkGRPkiNJnjqt/qEk30jydJJ/\nvaC+K8lsd+yaBfVru9pskltW9mVIks7Wcj4GehfwH4C7TxWS/G1gG/BLVfXDJBd19UuB7cBbgZ8D\n/muSX+i6fQr4VWAOeDzJvqr6+kq9EEnS2VkyAKrqkSSbTyt/APh4Vf2wa3Okq28D7u3q30wyC1ze\nHZutqucAktzbtTUAJGlMBn0P4BeAv5XksST/Lck7uvoG4OCCdnNd7Ux1SdKYDPpN4LXAOuBK4B3A\nfUl+HsgibYvFg2bRp9EnmQamASYmJuj1egNOEebn54fqL0kcfW4sw86fd9GqX78GDYA54HNVVcCX\nk/w5sL6rb1rQbiNwqNs+U/0nVNUMMAMwOTlZw/wrB/8VhKSh/e6dYxm2d+EHVv36NegS0B8A7wLo\n3uQ9D/gesA/YnuT8JFuArcCXgceBrUm2JDmP/hvF+4advCRpcEveASS5B5gC1ieZA24F9gB7uo+G\nvgzs6O4Gnk5yH/03d08AN1fVye48HwQeBNYAe6rq6VV4PZKkZVrOp4BuOsOhv3+G9h8DPrZIfT+w\n/6xmJ0laNX4TWJIaZQBIUqMMAElqlAEgSY0yACSpUQaAJDXKAJCkRhkAktQoA0CSGmUASFKjDABJ\napQBIEmNMgAkqVEGgCQ1ygCQpEYZAJLUKANAkhq1ZAAk2ZPkSPf4x9OP/YsklWR9t58kn0wym+TJ\nJJctaLsjybPdz46VfRmSpLO1nDuAu4BrTy8m2QT8KvD8gvJ19B8EvxWYBu7s2l5I/1nCVwCXA7cm\nWTfMxCVJw1kyAKrqEeDoIofuAH4DqAW1bcDd1fcocEGSi4FrgIeq6mhVvQA8xCKhIkkanYHeA0jy\nHuDbVfXHpx3aABxcsD/X1c5UlySNydqz7ZDk9cBvAlcvdniRWr1KfbHzT9NfPmJiYoJer3e2U/wL\n8/PzQ/WXJF53zViGHcX166wDAPgrwBbgj5MAbAS+muRy+n/Zb1rQdiNwqKtPnVbvLXbyqpoBZgAm\nJydrampqsWbL0uv1GKa/JPG7d45l2N6FH1j169dZLwFV1deq6qKq2lxVm+lf3C+rqj8F9gHv7T4N\ndCXwYlUdBh4Erk6yrnvz9+quJkkak+V8DPQe4H8Av5hkLsnOV2m+H3gOmAU+DfxjgKo6Cvw28Hj3\n81tdTZI0JksuAVXVTUsc37xgu4Cbz9BuD7DnLOcnSVolfhNYkhplAEhSowwASWqUASBJjTIAJKlR\nBoAkNWqQbwJLUjMOHDw2noF/bvWH8A5AkhplAEhSowwASWqUASBJjTIAJKlRBoAkNcoAkKRGGQCS\n1CgDQJIaZQBIUqOW80jIPUmOJHlqQe3fJPmTJE8m+c9JLlhwbFeS2STfSHLNgvq1XW02yS0r/1Ik\nSWdjOXcAdwHXnlZ7CHhbVf0S8L+AXQBJLgW2A2/t+vzHJGuSrAE+BVwHXArc1LWVJI3JkgFQVY8A\nR0+rfaGqTnS7jwIbu+1twL1V9cOq+ib9h8Nf3v3MVtVzVfUycG/XVpI0JivxHsA/Aj7fbW8ADi44\nNtfVzlSXJI3JUP8OOslvAieAz5wqLdKsWDxo6gznnAamASYmJuj1egPPb35+fqj+knT8khvGMu7J\nEVy/Bg6AJDuAXwOuqqpTF/M5YNOCZhuBQ932meo/oapmgBmAycnJmpqaGnSK9Ho9hukvSQduv20s\n4x67YteqX78GWgJKci3wUeA9VfXSgkP7gO1Jzk+yBdgKfBl4HNiaZEuS8+i/UbxvuKlLkoax5B1A\nknuAKWB9kjngVvqf+jkfeCgJwKNV9f6qejrJfcDX6S8N3VxVJ7vzfBB4EFgD7Kmqp1fh9UiSlmnJ\nAKiqmxYp736V9h8DPrZIfT+w/6xmJ0laNX4TWJIaZQBIUqMMAElqlAEgSY0yACSpUQaAJDXKAJCk\nRhkAktQoA0CSGmUASFKjDABJapQBIEmNMgAkqVEGgCQ1ygCQpEYZAJLUKANAkhq1ZAAk2ZPkSJKn\nFtQuTPJQkme73+u6epJ8MslskieTXLagz46u/bPdA+UlSWO0nDuAu4BrT6vdAjxcVVuBh7t9gOvo\nPwh+KzAN3An9wKD/LOErgMuBW0+FhiRpPJYMgKp6BDh6WnkbsLfb3gtcv6B+d/U9ClyQ5GLgGuCh\nqjpaVS8AD/HKUJEkjdCSD4U/g4mqOgxQVYeTXNTVNwAHF7Sb62pnqr9Ckmn6dw9MTEzQ6/UGnCLM\nz88P1V+Sjl9yw1jGPTmC69egAXAmWaRWr1J/ZbFqBpgBmJycrKmpqYEn0+v1GKa/JB24/baxjHvs\nil2rfv0a9FNA3+mWduh+H+nqc8CmBe02AodepS5JGpNBA2AfcOqTPDuABxbU39t9GuhK4MVuqehB\n4Ook67o3f6/uapKkMVlyCSjJPcAUsD7JHP1P83wcuC/JTuB54Mau+X7g3cAs8BLwPoCqOprkt4HH\nu3a/VVWnv7EsSRqhJQOgqm46w6GrFmlbwM1nOM8eYM9ZzU6StGr8JrAkNcoAkKRGGQCS1CgDQJIa\nZQBIUqMMAElqlAEgSY0yACSpUQaAJDXKAJCkRhkAktQoA0CSGmUASFKjVvqJYOeUb/2fl9h51+NL\nN1xhu//hO0Y+piSdLe8AJKlRBoAkNcoAkKRGDRUASf5ZkqeTPJXkniSvS7IlyWNJnk3ye0nO69qe\n3+3Pdsc3r8QLkCQNZuAASLIB+CfAZFW9DVgDbAduB+6oqq3AC8DOrstO4IWqegtwR9dOkjQmwy4B\nrQV+Ksla4PXAYeBdwP3d8b3A9d32tm6f7vhVSTLk+JKkAQ38MdCq+naSfws8DxwHvgA8ARyrqhNd\nszlgQ7e9ATjY9T2R5EXgTcD3Fp43yTQwDTAxMUGv1xt0irxxzQne+dPfHbj/oIaZs6Rzy/FLbhjL\nuCfn51f9WjJwACRZR/+v+i3AMeD3gesWaVqnurzKsR8XqmaAGYDJycmampoadIrs/ex+Hvn+mwfu\nP6jdv+73AKTXigO33zaWcY9dsYthrn/LMcwS0N8BvllV362qHwGfA/4mcEG3JASwETjUbc8BmwC6\n4z8LHB1ifEnSEIYJgOeBK5O8vlvLvwr4OvAl4NQ90w7ggW57X7dPd/yLVfWKOwBJ0mgMHABV9Rj9\nN3O/CnytO9cM8FHgI0lm6a/x7+667Abe1NU/AtwyxLwlSUMa6n8BVdWtwK2nlZ8DLl+k7Z8BNw4z\nniRp5fhNYElqlAEgSY0yACSpUQaAJDXKAJCkRhkAktQoA0CSGmUASFKjDABJapQBIEmNMgAkqVEG\ngCQ1ygCQpEYZAJLUKANAkhplAEhSowwASWrUUAGQ5IIk9yf5kyTPJPkbSS5M8lCSZ7vf67q2SfLJ\nJLNJnkxy2cq8BEnSIIa9A/gE8F+q6q8Cvww8Q/9Zvw9X1VbgYX787N/rgK3dzzRw55BjS5KGMHAA\nJPkZ4J10D32vqper6hiwDdjbNdsLXN9tbwPurr5HgQuSXDzwzCVJQxnmofA/D3wX+E9Jfhl4Avgw\nMFFVhwGq6nCSi7r2G4CDC/rPdbXDC0+aZJr+HQITExP0er2BJ/jGNSd4509/d+D+gxpmzpLOLccv\nuWEs456cn1/1a8kwAbAWuAz4UFU9luQT/Hi5ZzFZpFavKFTNADMAk5OTNTU1NfAE9352P498/80D\n9x/U7l9/x8jHlLQ6Dtx+21jGPXbFLoa5/i3HMO8BzAFzVfVYt38//UD4zqmlne73kQXtNy3ovxE4\nNMT4kqQhDBwAVfWnwMEkv9iVrgK+DuwDdnS1HcAD3fY+4L3dp4GuBF48tVQkSRq9YZaAAD4EfCbJ\necBzwPvoh8p9SXYCzwM3dm33A+8GZoGXurar6qIffZsPfecTqz3MIh4cw5iSdHaGCoCqOgBMLnLo\nqkXaFnDzMONJklaO3wSWpEYZAJLUKANAkhplAEhSowwASWqUASBJjTIAJKlRBoAkNcoAkKRGGQCS\n1CgDQJIaZQBIUqMMAElqlAEgSY0yACSpUQaAJDVq6ABIsibJ/0zyh93+liSPJXk2ye91Twsjyfnd\n/mx3fPOwY0uSBrcSdwAfBp5ZsH87cEdVbQVeAHZ29Z3AC1X1FuCOrp0kaUyGCoAkG4G/C/xOtx/g\nXcD9XZO9wPXd9rZun+74VV17SdIYDHsH8O+B3wD+vNt/E3Csqk50+3PAhm57A3AQoDv+YtdekjQG\nAz8UPsmvAUeq6okkU6fKizStZRxbeN5pYBpgYmKCXq836BT50fnrOHzJDQP3H9SxIeYs6dxyfAzX\nEICT8/NDXf+WY+AAAH4FeE+SdwOvA36G/h3BBUnWdn/lbwQOde3ngE3AXJK1wM8CR08/aVXNADMA\nk5OTNTU1NfAEP3/vp7n4W/cv3XCFvX37gyMfU9LqOHD7bWMZ99gVuxjm+rccAy8BVdWuqtpYVZuB\n7cAXq+rvAV8CTkXmDuCBbntft093/ItV9Yo7AEnSaKzG9wA+CnwkySz9Nf7dXX038Kau/hHgllUY\nW5K0TMMsAf2FquoBvW77OeDyRdr8GXDjSownSRqe3wSWpEYZAJLUKANAkhplAEhSowwASWqUASBJ\njTIAJKlRBoAkNcoAkKRGGQCS1CgDQJIaZQBIUqMMAElqlAEgSY0yACSpUQaAJDXKAJCkRg0cAEk2\nJflSkmeSPJ3kw139wiQPJXm2+72uqyfJJ5PMJnkyyWUr9SIkSWdvmDuAE8A/r6q/BlwJ3JzkUvrP\n+n24qrYCD/PjZ/9eB2ztfqaBO4cYW5I0pIEDoKoOV9VXu+3vA88AG4BtwN6u2V7g+m57G3B39T0K\nXJDk4oFnLkkayoq8B5BkM/DXgceAiao6DP2QAC7qmm0ADi7oNtfVJEljsHbYEyR5I/BZ4J9W1f9N\ncsami9RqkfNN018iYmJigl6vN/DcfnT+Og5fcsPA/Qd1bIg5Szq3HB/DNQTg5Pz8UNe/5RgqAJL8\nZfoX/89U1ee68neSXFxVh7slniNdfQ7YtKD7RuDQ6eesqhlgBmBycrKmpqYGnt/n7/00F3/r/oH7\nD+rt2x8c+ZiSVseB228by7jHrtjFMNe/5RjmU0ABdgPPVNW/W3BoH7Cj294BPLCg/t7u00BXAi+e\nWiqSJI3eMHcAvwL8A+BrSQ50tX8JfBy4L8lO4Hngxu7YfuDdwCzwEvC+IcaWJA1p4ACoqv/O4uv6\nAFct0r6AmwcdT5K0svwmsCQ1ygCQpEYZAJLUKANAkhplAEhSowwASWqUASBJjTIAJKlRBoAkNcoA\nkKRGGQCS1CgDQJIaZQBIUqMMAElqlAEgSY0yACSpUQaAJDVq5AGQ5Nok30gym+SWUY8vSeobaQAk\nWQN8CrgOuBS4Kcmlo5yDJKlv1HcAlwOzVfVcVb0M3AtsG/EcJEmMPgA2AAcX7M91NUnSiK0d8XhZ\npFY/0SCZBqa73fkk3xhivPXA94boP5hbFnuZknQ2vjDM9euS5TQadQDMAZsW7G8EDi1sUFUzwMxK\nDJbkK1U1uRLnkqRRGsX1a9RLQI8DW5NsSXIesB3YN+I5SJIY8R1AVZ1I8kHgQWANsKeqnh7lHCRJ\nfaNeAqKq9gP7RzTciiwlSdIYrPr1K1W1dCtJ0muO/wpCkhr1mg6AJPPjnoMknY0kJ5McSPJUkt9P\n8vrVGus1HQCS9P+h41X19qp6G/Ay8P7VGsgAkKRz1x8Bb1mtkxsAknQOSrKW/j/O/NpqjTHyj4FK\nkl7VTyU50G3/EbB7tQYyACTp3HK8qt4+ioFcApKkRr1mA6BbP/vhuOchSeeq12wAAG8F/ve4JyFJ\nZ6Oq3jiqsV6TAZDk/cA9wL8a91wk6Vzl/wKSpEa9Ju8AJElLMwAkqVEGgCQ1ygCQpEYZAJLUKANA\nkhr1/wBZQrsq2QzUGwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a0d04c390>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(pd.Series(yPJTest).value_counts(), end='\\n\\n')\n",
    "print(pd.Series(pjModel.predict(xPJTest)).value_counts())\n",
    "\n",
    "pd.Series(yPJTest).hist(alpha=0.7)\n",
    "pd.Series(pjModel.predict(xPJTest)).hist(alpha=0.7)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pickling our Models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-06T18:46:58.395812Z",
     "start_time": "2018-04-06T18:46:21.852Z"
    }
   },
   "outputs": [],
   "source": [
    "models = {\n",
    "    'models': [model, ieModel, nsModel, ftModel, pjModel],\n",
    "    'accuracies': accuracies,\n",
    "    'usable': [False, False, False, True, True]\n",
    "}\n",
    "joblib.dump(models, 'models.pkl')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Twitter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-06T18:49:17.192813Z",
     "start_time": "2018-04-06T18:49:17.178689Z"
    }
   },
   "outputs": [],
   "source": [
    "CONSUMER_KEY        = os.environ['TWITTER_CONSUMER_KEY']\n",
    "CONSUMER_SECRET     = os.environ['TWITTER_CONSUMER_SECRET']\n",
    "ACCESS_TOKEN        = os.environ['TWITTER_ACCESS_TOKEN']\n",
    "ACCESS_TOKEN_SECRET = os.environ['TWITTER_ACCESS_TOKEN_SECRET']\n",
    "\n",
    "AUTH = tweepy.OAuthHandler(CONSUMER_KEY, CONSUMER_SECRET)\n",
    "AUTH.set_access_token(ACCESS_TOKEN, ACCESS_TOKEN_SECRET)\n",
    "\n",
    "api = tweepy.API(AUTH)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Bootstrapping"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Fetching Tweets\n",
    "The username is the twitter handle of the target user without \"@\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-06T18:49:24.088438Z",
     "start_time": "2018-04-06T18:49:21.173684Z"
    }
   },
   "outputs": [],
   "source": [
    "#getting user tweets\n",
    "tweets = api.user_timeline('narendramodi', count=200)\n",
    "tweets = [tweet.text for tweet in tweets]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Procsssing Tweets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-06T18:49:26.777882Z",
     "start_time": "2018-04-06T18:49:26.767172Z"
    }
   },
   "outputs": [],
   "source": [
    "document = cv.transform([' '.join(tweets)])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-05T08:10:21.460236Z",
     "start_time": "2018-04-05T08:10:21.455000Z"
    }
   },
   "source": [
    "### Predicting MBTI Personality"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-04-06T18:49:29.172410Z",
     "start_time": "2018-04-06T18:49:29.154785Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['I']\n",
      "['N']\n",
      "['T']\n",
      "['J']\n"
     ]
    }
   ],
   "source": [
    "print(ieModel.predict(document))\n",
    "print(nsModel.predict(document))\n",
    "print(ftModel.predict(document))\n",
    "print(pjModel.predict(document))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  },
  "notify_time": "5",
  "toc": {
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": "block",
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "position": {
    "height": "433px",
    "left": "1070px",
    "right": "20px",
    "top": "120px",
    "width": "357px"
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}