Multi-Class Classification ADNI.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Alzheimer's Prediction (Multi-Class) \n",
    "* See [Alzheimer's Prediction](https://github.com/grantgasser/Alzheimers-Prediction) for a summary of this project"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Variable Descriptions\n",
    "* `directory.id`\tId assigned to imaging directories. Not from LONi data\n",
    "* `Subject (PTID)`\tParticipant ID\n",
    "* `RID`\tParticipant roster ID\n",
    "* `Image.Data.ID`\tMRI ID\n",
    "* `Modality`\tImage type\n",
    "* `Visit`\t1=screening scan\n",
    "* `Acq.Date`\tMRI date\n",
    "* `DX.bl`\tDiagnosis at baseline\n",
    "* `EXAMDATE`\tExamination Date\n",
    "* `AGE`\t    Age at baseline\n",
    "* `PTGENDER`\tSex\n",
    "* `PTEDUCAT`\tYears of Education\n",
    "* `PTETHCAT`\tEthnicity\n",
    "* `PTRACCAT`\tRace\n",
    "* `APOE4`\tAPOE4 genotype\n",
    "* `MMSE`\tMMSE score on cognitive test\n",
    "* `imputed_genotype`\tChallenge specific designation, TRUE=has imputed genotypes\n",
    "* `APOE Genotype`\tAPOE allele 1 and allele 2 genotypes\n",
    "* `Dx Codes for Submission`\tThe LMCI in the ADNI data is equivalent to MCI in test. This column just converts LMCI->MCI"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Note: This notebook is being refactored\n",
    "* More EDA\n",
    "* Proper train/test split \n",
    "* Use **AUC** as main metric for evaluation as well as **Accuracy**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import sklearn as sk\n",
    "import pandas as pd\n",
    "import os"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Read data\n",
    "dat = pd.read_csv('AD_Challenge_Training_Data_Clinical_Updated_7.22.2014/ADNI_Training_Q3_APOE_CollectionADNI1Complete 1Yr 1.5T_July22.2014.csv')\n",
    "\n",
    "#Check NA values, 1 NA for imputed_genotype\n",
    "# dat.isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Remove NA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dat = dat.dropna()\n",
    "dat.isnull().sum().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>directory.id</th>\n",
       "      <th>Subject</th>\n",
       "      <th>RID</th>\n",
       "      <th>Image.Data.ID</th>\n",
       "      <th>Modality</th>\n",
       "      <th>Visit</th>\n",
       "      <th>Acq.Date</th>\n",
       "      <th>DX.bl</th>\n",
       "      <th>EXAMDATE</th>\n",
       "      <th>AGE</th>\n",
       "      <th>PTGENDER</th>\n",
       "      <th>PTEDUCAT</th>\n",
       "      <th>PTETHCAT</th>\n",
       "      <th>PTRACCAT</th>\n",
       "      <th>APOE4</th>\n",
       "      <th>MMSE</th>\n",
       "      <th>imputed_genotype</th>\n",
       "      <th>APOE Genotype</th>\n",
       "      <th>Dx Codes for Submission</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>4702245ea294ce5d4e9b8a87027dfdf4</td>\n",
       "      <td>011_S_0003</td>\n",
       "      <td>3</td>\n",
       "      <td>32237</td>\n",
       "      <td>MRI</td>\n",
       "      <td>1</td>\n",
       "      <td>9/1/2005</td>\n",
       "      <td>AD</td>\n",
       "      <td>9/12/2005</td>\n",
       "      <td>81.3</td>\n",
       "      <td>Male</td>\n",
       "      <td>18</td>\n",
       "      <td>Not Hisp/Latino</td>\n",
       "      <td>White</td>\n",
       "      <td>1</td>\n",
       "      <td>20</td>\n",
       "      <td>True</td>\n",
       "      <td>3,4</td>\n",
       "      <td>AD</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2e89e352af743597b2368c412e0f6de2</td>\n",
       "      <td>022_S_0004</td>\n",
       "      <td>4</td>\n",
       "      <td>64631</td>\n",
       "      <td>MRI</td>\n",
       "      <td>1</td>\n",
       "      <td>9/22/2005</td>\n",
       "      <td>LMCI</td>\n",
       "      <td>11/8/2005</td>\n",
       "      <td>67.5</td>\n",
       "      <td>Male</td>\n",
       "      <td>10</td>\n",
       "      <td>Hisp/Latino</td>\n",
       "      <td>White</td>\n",
       "      <td>0</td>\n",
       "      <td>27</td>\n",
       "      <td>False</td>\n",
       "      <td>3,3</td>\n",
       "      <td>MCI</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>90419199306997753de8042f1fd55e38</td>\n",
       "      <td>011_S_0005</td>\n",
       "      <td>5</td>\n",
       "      <td>32246</td>\n",
       "      <td>MRI</td>\n",
       "      <td>1</td>\n",
       "      <td>9/2/2005</td>\n",
       "      <td>CN</td>\n",
       "      <td>9/7/2005</td>\n",
       "      <td>73.7</td>\n",
       "      <td>Male</td>\n",
       "      <td>16</td>\n",
       "      <td>Not Hisp/Latino</td>\n",
       "      <td>White</td>\n",
       "      <td>0</td>\n",
       "      <td>29</td>\n",
       "      <td>True</td>\n",
       "      <td>3,3</td>\n",
       "      <td>CN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                       directory.id     Subject  RID  Image.Data.ID Modality  \\\n",
       "0  4702245ea294ce5d4e9b8a87027dfdf4  011_S_0003    3          32237      MRI   \n",
       "1  2e89e352af743597b2368c412e0f6de2  022_S_0004    4          64631      MRI   \n",
       "2  90419199306997753de8042f1fd55e38  011_S_0005    5          32246      MRI   \n",
       "\n",
       "   Visit   Acq.Date DX.bl   EXAMDATE   AGE PTGENDER  PTEDUCAT  \\\n",
       "0      1   9/1/2005    AD  9/12/2005  81.3     Male        18   \n",
       "1      1  9/22/2005  LMCI  11/8/2005  67.5     Male        10   \n",
       "2      1   9/2/2005    CN   9/7/2005  73.7     Male        16   \n",
       "\n",
       "          PTETHCAT PTRACCAT  APOE4  MMSE imputed_genotype APOE Genotype  \\\n",
       "0  Not Hisp/Latino    White      1    20             True           3,4   \n",
       "1      Hisp/Latino    White      0    27            False           3,3   \n",
       "2  Not Hisp/Latino    White      0    29             True           3,3   \n",
       "\n",
       "  Dx Codes for Submission  \n",
       "0                      AD  \n",
       "1                     MCI  \n",
       "2                      CN  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Peek at data\n",
    "dat.head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Remove ID's and other features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Removing columns: ['directory.id', 'Subject', 'RID', 'Image.Data.ID', 'Modality', 'Visit', 'Acq.Date', 'DX.bl', 'EXAMDATE', 'Dx Codes for Submission']\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AGE</th>\n",
       "      <th>PTGENDER</th>\n",
       "      <th>PTEDUCAT</th>\n",
       "      <th>PTETHCAT</th>\n",
       "      <th>PTRACCAT</th>\n",
       "      <th>APOE4</th>\n",
       "      <th>MMSE</th>\n",
       "      <th>imputed_genotype</th>\n",
       "      <th>APOE Genotype</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>81.3</td>\n",
       "      <td>Male</td>\n",
       "      <td>18</td>\n",
       "      <td>Not Hisp/Latino</td>\n",
       "      <td>White</td>\n",
       "      <td>1</td>\n",
       "      <td>20</td>\n",
       "      <td>True</td>\n",
       "      <td>3,4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>67.5</td>\n",
       "      <td>Male</td>\n",
       "      <td>10</td>\n",
       "      <td>Hisp/Latino</td>\n",
       "      <td>White</td>\n",
       "      <td>0</td>\n",
       "      <td>27</td>\n",
       "      <td>False</td>\n",
       "      <td>3,3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>73.7</td>\n",
       "      <td>Male</td>\n",
       "      <td>16</td>\n",
       "      <td>Not Hisp/Latino</td>\n",
       "      <td>White</td>\n",
       "      <td>0</td>\n",
       "      <td>29</td>\n",
       "      <td>True</td>\n",
       "      <td>3,3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>80.4</td>\n",
       "      <td>Female</td>\n",
       "      <td>13</td>\n",
       "      <td>Not Hisp/Latino</td>\n",
       "      <td>White</td>\n",
       "      <td>0</td>\n",
       "      <td>25</td>\n",
       "      <td>True</td>\n",
       "      <td>3,3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>73.9</td>\n",
       "      <td>Female</td>\n",
       "      <td>12</td>\n",
       "      <td>Not Hisp/Latino</td>\n",
       "      <td>White</td>\n",
       "      <td>1</td>\n",
       "      <td>24</td>\n",
       "      <td>True</td>\n",
       "      <td>3,4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    AGE PTGENDER  PTEDUCAT         PTETHCAT PTRACCAT  APOE4  MMSE  \\\n",
       "0  81.3     Male        18  Not Hisp/Latino    White      1    20   \n",
       "1  67.5     Male        10      Hisp/Latino    White      0    27   \n",
       "2  73.7     Male        16  Not Hisp/Latino    White      0    29   \n",
       "3  80.4   Female        13  Not Hisp/Latino    White      0    25   \n",
       "4  73.9   Female        12  Not Hisp/Latino    White      1    24   \n",
       "\n",
       "  imputed_genotype APOE Genotype  \n",
       "0             True           3,4  \n",
       "1            False           3,3  \n",
       "2             True           3,3  \n",
       "3             True           3,3  \n",
       "4             True           3,4  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = dat\n",
    "Y = dat['DX.bl']\n",
    "del dat\n",
    "\n",
    "#Remove unnecessary columns (features), remove first 9 columns and 'Dx codes for submission'\n",
    "remove_columns = list(X.columns)[0:9]\n",
    "remove_columns.append('Dx Codes for Submission')\n",
    "print('Removing columns:', remove_columns)\n",
    "\n",
    "X = X.drop(remove_columns, axis=1)\n",
    "\n",
    "features = list(X.columns)\n",
    "X.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exploratory Data Analysis (EDA)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Categorical variable distributions:\n",
      "\n",
      "\n",
      "Distribution of imputed_genotype\n",
      "True     466\n",
      "False    161\n",
      "Name: imputed_genotype, dtype: int64\n",
      "\n",
      "Distribution of APOE Genotype\n",
      "3,3    275\n",
      "3,4    221\n",
      "4,4     71\n",
      "2,3     45\n",
      "2,4     13\n",
      "2,2      2\n",
      "Name: APOE Genotype, dtype: int64\n",
      "\n",
      "Distribution of PTGENDER\n",
      "Male      363\n",
      "Female    264\n",
      "Name: PTGENDER, dtype: int64\n",
      "\n",
      "Distribution of APOE4\n",
      "0    322\n",
      "1    234\n",
      "2     71\n",
      "Name: APOE4, dtype: int64\n",
      "\n",
      "Distribution of PTETHCAT\n",
      "Not Hisp/Latino    616\n",
      "Hisp/Latino          8\n",
      "Unknown              3\n",
      "Name: PTETHCAT, dtype: int64\n",
      "\n",
      "Distribution of PTRACCAT\n",
      "White    583\n",
      "Black     31\n",
      "Asian     13\n",
      "Name: PTRACCAT, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "numerical_vars = ['AGE', 'MMSE', 'PTEDUCAT']\n",
    "cat_vars = list(set(features) - set(numerical_vars))\n",
    "\n",
    "print('Categorical variable distributions:\\n')\n",
    "\n",
    "for var in cat_vars:\n",
    "    print('\\nDistribution of', var)\n",
    "    \n",
    "    print(X[var].value_counts())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Categorical Var Distribution Notes\n",
    "* Ethnicity and Race are separate vars\n",
    "* Distribution of Hispanic, Black, White, Asian not quite same as overall American ethnic/racial distribution\n",
    "* APOE Genotype with 3 in the first spot seems to be overwhelming majority `3,3` and `3,4`\n",
    "* More males in study than females"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Numerical Variable Distributions\n",
    "* All 3 distributions seem a bit right-skewed\n",
    "* **Age**: `mean = 75`, `min=55`, `max=89` - seems to be no obvious bad data\n",
    "* **MMSE**: `mean = 26.9`, `min=20`, `max=30` - seems to be no obvious bad data, NOTE: MMSE $\\in ~(0,30)$\n",
    "* **PTEDUCAT**: `mean = 15.6`, `min=6`, `max=20` - seems to be no obvious bad data. 13-14 yrs is probably high school/GED and 17-18 years is likely Bachelor's degree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Numerical Var Distributions:\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAT60lEQVR4nO3df7BkZX3n8fcnTEDBdWeAC8IMZsCwKFoLUiOLmlCUWCtIVkhKIiyJE8XMJkV2/ZEqGUwqmKzUguuubqIhNQvCbFYRlujC+mshLETdWsYdBBQYlRERRn7MRX5kwUQZ/e4ffa5pLn1n5t7untvz8H5VdXWf5zx9nu8cms89/fTp06kqJElt+bnFLkCSNHqGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3PScl+UKS1SPa1i8n+Vbf8r1JXj+KbXfbuzPJCaPanp4bDHftEkluSvJYkr0GrFuV5LPd+seT3JXkgiTLuvW/leQnSZ6cdTt4jrEqyVNdnx8kuSHJW/r7VNXJVbV+J+quJL+4vT5V9eWqOmJH29oZSS5P8oFZ2395Vd00iu3rucNw19glWQn8MlDAm2atew1wE/C/gZdW1VLgJGAbcFRf1/9TVS+YdXtgO8MeVVUvAI4ALgc+muT80fyLnlH/klFvUxoFw127wluBm+mF7OypkA8Cl1XVv6uqhwGq6r6qOn8UR6tV9UhV/SXwu8B5SfaDn72TeEf3+BeT/E2SJ5I8kuTKrv1L3WZu794FvCXJCUm2JDk3yUPAZTNts4Z+VfcO5LEklyV5XrfN30rylf6OM+8OkqwBzgLe2433P7r1P5vmSbJXko8keaC7fWTm3VBfbb+fZGuSB5O8bdh9qN2T4a5d4a3AJ7rbG5IcCJBkH+DVwF/tghquAZYAxw5Y92+B64BlwArgzwCq6vhu/VHdO4Uru+UXAfsCvwCsmWO8s4A3AC8B/gnwhzsqsKrW0dtHH+zG+xcDuv0BcBxwNL13NsfO2vaLgH8MLAfOBj42M72l5xbDXWOV5JfoheBVVXUL8B3gX3arl9F7DT7U1/+D3bz7U0n6Q+u4rn3m9p351FFVTwOP0Avl2Z7uajy4qv6+qr4yoE+/nwLnV9WPqurv5ujz0aq6v6oeBS4AzpxPvdtxFvAnVbW1qqaBPwZ+s2/90936p6vq88CT9Kam9BxjuGvcVgPXVdUj3fIn+YepmcfoBeVBM52r6r3dvPtn6B1pz7i5qpb23V4ynyKS/DwwBTw6YPV7gQBf7c5MefsONjddVX+/gz739z3+HjDww98FOLjb3lzb/kFVbetb/iHwghGNrd2IHwZpbJI8H/h1YI9ufhpgL2BpkqOq6vYkG4BfA24cczmn0vuQ9quzV1TVQ8BvdzX/EvDXSb5UVZvn2NbOXEr1kL7HLwZmPvx9Cth7ZkWSF81z2w/Qe5dx54BtSz/jkbvG6TTgJ8CR9OaIjwZeBnyZ3jw89I6a355kbZIDAJKsAA4dRQFJ9k1yFvAx4KKq+sGAPqd3Y0Lv3UR1dQM8DBy2gKHPSbIiyb7A+4CZ+frbgZcnObr7kPX9s563o/GuAP4wyVSS/YE/Av7rAupT4wx3jdNqemfC3FdVD83cgI8CZyVZ0s1vvw44Hvh2kseBL9I7PfLP+rb16gHnub9qO2PfnuRJYDPwDuDdVfVHc/R9FbCh638t8M6q+m637v3A+m6e/9fn8W//JL0Pae/pbh8AqKpvA38C/DVwNzB7fv9S4MhuvP8+YLsfADYCXwe+AXxtZttSv/hjHZLUHo/cJalBhrskNchwl6QGGe6S1KCJOM99//33r5UrVy52GZK0W7nlllseqaqpQesmItxXrlzJxo0bF7sMSdqtJPneXOuclpGkBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAZNxDdUpR1ZufZzizb2vReesijjLta/ebH+vRotj9wlqUE7DPckH0+yNckdfW3/Psk3k3w9yWeSLO1bd16SzUm+leQN4ypckjS3nTlyvxw4aVbb9cArquqfAt8GzgNIciRwBvDy7jl/nmSPkVUrSdopOwz3qvoS8Oistuuqalu3eDMw88vxpwKfqqofdT8wvBk4doT1SpJ2wijm3N8OfKF7vBy4v2/dlq7tWZKsSbIxycbp6ekRlCFJmjHU2TJJ/gDYBnxipmlAtxr03KpaB6wDWLVq1cA+0iRYzDN1pIVacLgnWQ38CnBiVc2E8xbgkL5uK4AHFl6eJGkhFjQtk+Qk4FzgTVX1w75V1wJnJNkryaHA4cBXhy9TkjQfOzxyT3IFcAKwf5ItwPn0zo7ZC7g+CcDNVfU7VXVnkquAu+hN15xTVT8ZV/GSpMF2GO5VdeaA5ku30/8C4IJhipIkDcdvqEpSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhq0w3BP8vEkW5Pc0de2b5Lrk9zd3S/r2pPkT5NsTvL1JMeMs3hJ0mA7c+R+OXDSrLa1wA1VdThwQ7cMcDJweHdbA1w8mjIlSfOxw3Cvqi8Bj85qPhVY3z1eD5zW1/5fqudmYGmSg0ZVrCRp5yx0zv3AqnoQoLs/oGtfDtzf129L1/YsSdYk2Zhk4/T09ALLkCQNMuoPVDOgrQZ1rKp1VbWqqlZNTU2NuAxJem5baLg/PDPd0t1v7dq3AIf09VsBPLDw8iRJC7HQcL8WWN09Xg1c09f+1u6smeOAJ2ambyRJu86SHXVIcgVwArB/ki3A+cCFwFVJzgbuA07vun8eeCOwGfgh8LYx1CxJ2oEdhntVnTnHqhMH9C3gnGGLkiQNx2+oSlKDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1KAdXs9d0nPLyrWfW7Sx773wlEUbuzUeuUtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoOGCvck705yZ5I7klyR5HlJDk2yIcndSa5MsueoipUk7ZwFh3uS5cC/AVZV1SuAPYAzgIuAD1fV4cBjwNmjKFSStPOGnZZZAjw/yRJgb+BB4HXA1d369cBpQ44hSZqnBYd7VX0f+BBwH71QfwK4BXi8qrZ13bYAywc9P8maJBuTbJyenl5oGZKkAYaZllkGnAocChwM7AOcPKBrDXp+Va2rqlVVtWpqamqhZUiSBhhmWub1wHerarqqngY+DbwGWNpN0wCsAB4YskZJ0jwNE+73Accl2TtJgBOBu4AbgTd3fVYD1wxXoiRpvoaZc99A74PTrwHf6La1DjgXeE+SzcB+wKUjqFOSNA9D/RJTVZ0PnD+r+R7g2GG2K0kajt9QlaQGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1aKgf65CkUVq59nOLMu69F56yKOOOk0fuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYNFe5Jlia5Osk3k2xK8uok+ya5Psnd3f2yURUrSdo5wx65/yfgi1X1UuAoYBOwFrihqg4HbuiWJUm70ILDPckLgeOBSwGq6sdV9ThwKrC+67YeOG3YIiVJ8zPMkfthwDRwWZJbk1ySZB/gwKp6EKC7P2DQk5OsSbIxycbp6ekhypAkzTZMuC8BjgEurqpXAk8xjymYqlpXVauqatXU1NQQZUiSZhsm3LcAW6pqQ7d8Nb2wfzjJQQDd/dbhSpQkzdeCw72qHgLuT3JE13QicBdwLbC6a1sNXDNUhZKkeRv2eu7/GvhEkj2Be4C30fuDcVWSs4H7gNOHHEOSNE9DhXtV3QasGrDqxGG2K0kajt9QlaQGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNWjocE+yR5Jbk3y2Wz40yYYkdye5Msmew5cpSZqPURy5vxPY1Ld8EfDhqjoceAw4ewRjSJLmYahwT7ICOAW4pFsO8Drg6q7LeuC0YcaQJM3fsEfuHwHeC/y0W94PeLyqtnXLW4Dlg56YZE2SjUk2Tk9PD1mGJKnfgsM9ya8AW6vqlv7mAV1r0POral1VraqqVVNTUwstQ5I0wJIhnvta4E1J3gg8D3ghvSP5pUmWdEfvK4AHhi9TkjQfCz5yr6rzqmpFVa0EzgD+V1WdBdwIvLnrthq4ZugqJUnzMo7z3M8F3pNkM705+EvHMIYkaTuGmZb5maq6Cbipe3wPcOwotitJWhi/oSpJDTLcJalBhrskNchwl6QGGe6S1CDDXZIaNJJTIbVrrVz7uUUb+94LT1m0sSXtPI/cJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGuRVITUvi3lFSmlcWrzSqkfuktQgw12SGmS4S1KDDHdJatCCwz3JIUluTLIpyZ1J3tm175vk+iR3d/fLRleuJGlnDHPkvg34/ap6GXAccE6SI4G1wA1VdThwQ7csSdqFFhzuVfVgVX2te/z/gE3AcuBUYH3XbT1w2rBFSpLmZyRz7klWAq8ENgAHVtWD0PsDABwwx3PWJNmYZOP09PQoypAkdYYO9yQvAP4KeFdV/e3OPq+q1lXVqqpaNTU1NWwZkqQ+Q4V7kp+nF+yfqKpPd80PJzmoW38QsHW4EiVJ8zXM2TIBLgU2VdV/7Ft1LbC6e7wauGbh5UmSFmKYa8u8FvhN4BtJbuva3gdcCFyV5GzgPuD04UqUJM3XgsO9qr4CZI7VJy50u5Kk4fkNVUlqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgf0N1CP6eqKRJ5ZG7JDXIcJekBu320zJOjUjSs3nkLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoPGFu5JTkryrSSbk6wd1ziSpGcbS7gn2QP4GHAycCRwZpIjxzGWJOnZxnXkfiywuaruqaofA58CTh3TWJKkWcb1M3vLgfv7lrcA/6y/Q5I1wJpu8ckk31rgWPsDjyzwuYvBesfLesfLekcsFz1jcb71/sJcK8YV7hnQVs9YqFoHrBt6oGRjVa0adju7ivWOl/WOl/WO1yjrHde0zBbgkL7lFcADYxpLkjTLuML9/wKHJzk0yZ7AGcC1YxpLkjTLWKZlqmpbkt8D/iewB/DxqrpzHGMxgqmdXcx6x8t6x8t6x2tk9aaqdtxLkrRb8RuqktQgw12SGrRbhXuSe5N8I8ltSTZ2be9P8v2u7bYkb1zsOmckWZrk6iTfTLIpyauT7Jvk+iR3d/fLFrvOGXPUO5H7N8kRfTXdluRvk7xrUvfvduqdyP0LkOTdSe5MckeSK5I8rztJYkO3f6/sTpiYGHPUfHmS7/bt46MXu84ZSd7Z1Xpnknd1bSN5De9Wc+5J7gVWVdUjfW3vB56sqg8tVl1zSbIe+HJVXdL9T7A38D7g0aq6sLvmzrKqOndRC+3MUe+7mND9O6O73MX36X1R7hwmdP/OmFXv25jA/ZtkOfAV4Miq+rskVwGfB94IfLqqPpXkL4Dbq+rixax1xnZqPgH4bFVdvZj1zZbkFfS+vX8s8GPgi8DvAr/NCF7Du9WR++4kyQuB44FLAarqx1X1OL3LMKzvuq0HTlucCp9pO/XuDk4EvlNV32NC9+8s/fVOsiXA85MsofeH/kHgdcBMSE7i/p1d8yR/v+ZlwM1V9cOq2gb8DfCrjOg1vLuFewHXJbmlu3zBjN9L8vUkH5+Ut+HAYcA0cFmSW5NckmQf4MCqehCguz9gMYvsM1e9MJn7t98ZwBXd40ndv/3664UJ3L9V9X3gQ8B99EL9CeAW4PEuiKD3ZcXli1Phsw2quaqu61Zf0O3jDyfZa9GKfKY7gOOT7Jdkb3rvig5hRK/h3S3cX1tVx9C72uQ5SY4HLgZeAhxN7z/of1jE+votAY4BLq6qVwJPAZN86eO56p3U/QtAN330JuC/LXYtO2NAvRO5f7s/MqcChwIHA/vQ+/9utomZ1x1Uc5LfAM4DXgq8CtgXmIhpuqraBFwEXE9vSuZ2YNt2nzQPu1W4V9UD3f1W4DPAsVX1cFX9pKp+CvxnevNXk2ALsKWqNnTLV9MLz4eTHATQ3W9dpPpmG1jvBO/fGScDX6uqh7vlSd2/M55R7wTv39cD362q6ap6Gvg08BpgaTflAZN3WZGBNVfVg9XzI+AyJmcfU1WXVtUxVXU88ChwNyN6De824Z5knyT/aOYx8M+BO2Z2QudX6b3VWXRV9RBwf5IjuqYTgbvoXYZhdde2GrhmEcp7lrnqndT92+dMnjnFMZH7t88z6p3g/XsfcFySvZOEf3j93gi8ueszaft3UM2b+oIy9OavJ2Ufk+SA7v7FwK/Re22M5DW825wtk+Qwekfr0JtC+GRVXZDkL+m9pS3gXuBfzcxXLbbulKtLgD2Be+idGfFzwFXAi+m9GE+vqkcXrcg+c9T7p0zu/t2b3qWlD6uqJ7q2/Zjc/Tuo3kl+/f4x8BZ6UwW3Au+gN8f+KXrTG7cCv9EdEU+EOWr+AjBF72q1twG/U1VPLlqRfZJ8GdgPeBp4T1XdMKrX8G4T7pKknbfbTMtIknae4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIa9P8Bf5YXbO/xVHAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "count    627.000000\n",
      "mean      75.161882\n",
      "std        6.689979\n",
      "min       55.100000\n",
      "25%       71.200000\n",
      "50%       75.400000\n",
      "75%       80.000000\n",
      "max       89.600000\n",
      "Name: AGE, dtype: float64\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAASL0lEQVR4nO3de7BdZX3G8e8jQWqLCpgEEcGjDjrQsaKNSL3UC1YBL+BUKmglWtu0Cq1YrUZt66VDi1ovpVo7OFLj3VhBUNCCVOtlChoUUcotYJAQJBFEUfAC/vrHXmfcnOxzy9k7m/Pm+5nZs9d617vX+r05J89e692Xk6pCktSWu427AEnS8BnuktQgw12SGmS4S1KDDHdJapDhLkkNMtylTpLHJ7l8iPv7bJKV3fILk3xliPt+fpJzhrU/tcdw19Ak2ZDkF0mWTmm/KEklmejW39+tP2tKv3d27S/s1u+e5G1JNib5SZLvJnnHlOPd1m2bvL1rmtrekOSXSW7pblckeVeSvSb7VNWXq+qhcxjnG5J8aLZ+VXVYVa2Zrd8cjjfR/bss6dv3h6vqqQvdt9pluGvYvgscM7mS5GHAPQb0uwJY2ddvCXAUcFVfn9cAK4CDgHsCTwK+OWU/z6yqXftux89Q28er6p7AHsCzgfsCF/YH/DCkx/9bGit/ATVsHwSO7VtfCXxgQL9PA49Nsnu3fihwMfD9vj6PAk6vqk3Vs6GqBu1rXqrql1V1CfBcYAvwCoAkT0yycbJfklcnua470788ySFJDgVeCzy3u1L4Vtf3i0lOTPJV4FbgQV3bn/YdOkn+NcmPklyW5JC+DRuSPKVvvf/q4Evd/c3dMX9v6jRPksck+Xq3768neUzfti8m+YckX+3Gcs7Uqyu1x3DXsJ0P3CvJ/kl2ohegg6YwfgacCRzdrR/L1k8C5wN/neSlSR6WJMMstKruAM4AHj91W5KHAscDj+rO9p8GbKiqzwH/SO8qYNeqenjfw14ArKJ3lXHNgEM+GrgaWAq8HjgtyR5zKPX3u/vdumP+75Ra9wDOAk4G7gO8HTgryX36uj0PeBGwHLg78Mo5HFeLmOGuUZg8e/8D4DLgumn6fQA4Nsm9gScAn5qy/Z+ANwPPB9YB102+QNnnU0lu7rv92Txr3URvmmaqO4BdgAOS7NxdNVw1oF+/91fVJVV1e1X9csD2zcA7uyuHjwOXA0+fZ72DPB24sqo+2B37o/T+3Z/Z1+c/quqKqroNWAscOITj6i7McNcofJDemeILGTwlA0BVfQVYBvwt8JkuePq331FV766qxwK7AScCpybZv6/bkVW1W9/tvfOsdW/gpgG1rQdOAN4AbE7ysST3m2Vf186y/bq68zf1XQPMts+5uB9bXylcQ29sk/qnu24Fdh3CcXUXZrhr6KrqGnovrB4OnDZL9w/Rm/OecS69qm6rqncDPwQOGEad3YuezwS+PM0xP1JVjwMeABS9qwi65YEPmeWQe0+ZWtqX3pUDwE+B3+zbdt957HdTV2O/fZn+ikk7AMNdo/Ji4MlV9dNZ+p1Mb/rmS1M3JDmhe5HzHkmWdFMy92Trd8zMS5Kdu7P/j9IL0bcP6PPQJE9Osgu91wduozdVA3ADMLEN74hZDvxVd/yjgP2Bs7ttFwFHd9tWAM/pe9wW4FfAg6bZ79nAQ5I8r/t3ei69J8DPzLM+NcRw10hU1VVVtW4O/W6qqvOmTFdMug14G70phR8AxwF/WFVX9/X59JT3uZ8+w+Gem+QnwM30Xsy9Efjdqto0oO8uwEndcb9PL5hf2237RHd/Y5JvzDbGPhcA+3X7PBF4TlXd2G37O+DB9K5M3gh8ZPJBVXVr1/+r3esKB/fvtNvHM+hdAd0IvAp4RlX9YB61qTHxj3VIUns8c5ekBhnuktQgw12SGmS4S1KDlszeZfSWLl1aExMT4y5DkhaVCy+88AdVtWzQtrtEuE9MTLBu3azvmpMk9Uky6DuMAKdlJKlJhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQXeJT6hK0jhNrD5rbMfecNIw/kb61jxzl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUoFnDPck+Sb6Q5NIklyR5Wde+R5Jzk1zZ3e/etSfJyUnWJ7k4ySNHPQhJ0p3N5cz9duAVVbU/cDBwXJIDgNXAeVW1H3Betw5wGLBfd1sFvGfoVUuSZjRruFfV9VX1jW75FuBSYG/gCGBN120NcGS3fATwgeo5H9gtyV5Dr1ySNK15zbknmQAeAVwA7FlV10PvCQBY3nXbG7i272Ebu7ap+1qVZF2SdVu2bJl/5ZKkac053JPsCnwSOKGqfjxT1wFttVVD1SlVtaKqVixbtmyuZUiS5mBO4Z5kZ3rB/uGqOq1rvmFyuqW739y1bwT26Xv4/YFNwylXkjQXc3m3TID3AZdW1dv7Np0JrOyWVwJn9LUf271r5mDgR5PTN5Kk7WPJHPo8FngB8O0kF3VtrwVOAtYmeTHwPeCobtvZwOHAeuBW4EVDrViSNKtZw72qvsLgeXSAQwb0L+C4BdYlSVoAP6EqSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUoFnDPcmpSTYn+U5f2xuSXJfkou52eN+21yRZn+TyJE8bVeGSpOnN5cz9/cChA9rfUVUHdrezAZIcABwN/Hb3mH9LstOwipUkzc2s4V5VXwJumuP+jgA+VlU/r6rvAuuBgxZQnyRpGyxkzv34JBd30za7d217A9f29dnYtW0lyaok65Ks27JlywLKkCRNta3h/h7gwcCBwPXA27r2DOhbg3ZQVadU1YqqWrFs2bJtLEOSNMg2hXtV3VBVd1TVr4D38uupl43APn1d7w9sWliJkqT52qZwT7JX3+qzgcl30pwJHJ1klyQPBPYDvrawEiVJ87Vktg5JPgo8EViaZCPweuCJSQ6kN+WyAfhzgKq6JMla4P+A24HjquqO0ZQuSZrOrOFeVccMaH7fDP1PBE5cSFGSpIXxE6qS1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNmjXck5yaZHOS7/S17ZHk3CRXdve7d+1JcnKS9UkuTvLIURYvSRpsLmfu7wcOndK2GjivqvYDzuvWAQ4D9utuq4D3DKdMSdJ8zBruVfUl4KYpzUcAa7rlNcCRfe0fqJ7zgd2S7DWsYiVJc7Otc+57VtX1AN398q59b+Davn4bu7atJFmVZF2SdVu2bNnGMiRJgwz7BdUMaKtBHavqlKpaUVUrli1bNuQyJGnHtq3hfsPkdEt3v7lr3wjs09fv/sCmbS9PkrQttjXczwRWdssrgTP62o/t3jVzMPCjyekbSdL2s2S2Dkk+CjwRWJpkI/B64CRgbZIXA98Djuq6nw0cDqwHbgVeNIKaJUmzmDXcq+qYaTYdMqBvAccttChJ0sL4CVVJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1KBZv/JXkraXidVnjbuEZnjmLkkNMtwlqUFOy0h3UeOaothw0tPHclwNl2fuktQgw12SGmS4S1KDDHdJapDhLkkN8t0y0iz8YI0WI8Nd0p34ZNYGp2UkqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1KAFfXFYkg3ALcAdwO1VtSLJHsDHgQlgA/BHVfXDhZUpSZqPYZy5P6mqDqyqFd36auC8qtoPOK9blyRtR6OYljkCWNMtrwGOHMExJEkzWGi4F3BOkguTrOra9qyq6wG6++ULPIYkaZ4W+sc6HltVm5IsB85NctlcH9g9GawC2HfffRdYhiSp34LO3KtqU3e/GTgdOAi4IcleAN395mkee0pVraiqFcuWLVtIGZKkKbb5zD3JbwF3q6pbuuWnAm8CzgRWAid192cMo1Dt2PzTb9L8LGRaZk/g9CST+/lIVX0uydeBtUleDHwPOGrhZUqS5mObw72qrgYePqD9RuCQhRQlSVoYP6EqSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNWghfyBbO6CJ1WeNuwRJc+CZuyQ1yHCXpAYZ7pLUIOfcF2Bc888bTnr6WI4rafHwzF2SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yLdCLkJ+BYCk2XjmLkkNWvRn7p7FStLWPHOXpAYZ7pLUIMNdkhpkuEtSgwx3SWrQyMI9yaFJLk+yPsnqUR1HkrS1kYR7kp2AdwOHAQcAxyQ5YBTHkiRtbVRn7gcB66vq6qr6BfAx4IgRHUuSNMWoPsS0N3Bt3/pG4NH9HZKsAlZ1qz9Jcvk2Hmsp8INtfOxi5Zh3DI55B5A3L2jMD5huw6jCPQPa6k4rVacApyz4QMm6qlqx0P0sJo55x+CYdwyjGvOopmU2Avv0rd8f2DSiY0mSphhVuH8d2C/JA5PcHTgaOHNEx5IkTTGSaZmquj3J8cB/ATsBp1bVJaM4FkOY2lmEHPOOwTHvGEYy5lTV7L0kSYuKn1CVpAYZ7pLUoEUV7kn2SfKFJJcmuSTJy7r2PZKcm+TK7n73cdc6LDOM+a1JLktycZLTk+w27lqHZbox921/ZZJKsnRcNQ7TTONN8pfd13hckuQt46xzmGb4vT4wyflJLkqyLslB4651WJL8RpKvJflWN+Y3du0PTHJBl18f796EsnBVtWhuwF7AI7vlewJX0Pt6g7cAq7v21cCbx13rdhjzU4ElXfubd4Qxd+v70Huh/hpg6bhrHfHP+EnA54Fdum3Lx13rdhjzOcBhXfvhwBfHXesQxxxg1255Z+AC4GBgLXB01/7vwEuGcbxFdeZeVddX1Te65VuAS+l9GvYIYE3XbQ1w5HgqHL7pxlxV51TV7V238+l9lqAJM/ycAd4BvIopH4pbzGYY70uAk6rq5922zeOrcrhmGHMB9+q63ZuGPh9TPT/pVnfubgU8GfjPrn1o+bWowr1fkgngEfSe/fasquuh90sDLB9fZaMzZcz9/gT47PauZ3voH3OSZwHXVdW3xlrUCE35GT8EeHx3yf4/SR41ztpGZcqYTwDemuRa4J+B14yvsuFLslOSi4DNwLnAVcDNfSdqG/n1icyCLMpwT7Ir8EnghKr68bjr2R6mG3OS1wG3Ax8eV22j0j9memN8HfD3Yy1qhAb8jJcAu9O7dP8bYG2SQV/tsWgNGPNLgJdX1T7Ay4H3jbO+YauqO6rqQHpX2gcB+w/qNoxjLbpwT7IzvV+GD1fVaV3zDUn26rbvRe9ZsRnTjJkkK4FnAM+vbsKuFQPG/GDggcC3kmyg95/jG0nuO74qh2ean/FG4LTucv5rwK/ofbFWE6YZ80pgcvkT9AKwOVV1M/BFek/cuyWZ/EDp0L6qZVGFe3fW8j7g0qp6e9+mM+n9UtDdn7G9axuV6cac5FDg1cCzqurWcdU3CoPGXFXfrqrlVTVRVRP0gu+RVfX9MZY6FDP8Xn+K3nwsSR4C3J1GvjFxhjFvAp7QLT8ZuHJ71zYqSZZNvqstyT2Ap9B7reELwHO6bkPLr0X1CdUkjwO+DHyb3lkMwGvpzdWtBfYFvgccVVU3jaXIIZthzCcDuwA3dm3nV9VfbP8Kh2+6MVfV2X19NgArqmrRh90MP+PPA6cCBwK/AF5ZVf89liKHbIYx/xj4F3pTUj8DXlpVF46lyCFL8jv0XjDdid6J9dqqelOSB9H7mxd7AN8E/njyRfQFHW8xhbskaW4W1bSMJGluDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUoP8H5DTxQaKN7twAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "count    627.000000\n",
      "mean      26.921850\n",
      "std        2.563934\n",
      "min       20.000000\n",
      "25%       25.000000\n",
      "50%       27.000000\n",
      "75%       29.000000\n",
      "max       30.000000\n",
      "Name: MMSE, dtype: float64\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAVs0lEQVR4nO3df7BkZX3n8fcnIGYREXQuBBEYIOCusjq6I+ImGCKaBXRFtlbDlDGDujuSlfgj2awoqUhShYW/l8SEFAYEIyIoohgxgWITWbOiDogIASPgAAPDcBVRECUOfPePPheay71zf3T39L1P3q+qrj7nOc855ztdtz9z+ulzTqeqkCS15RfGXYAkafgMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnu0hAkuT7JYUPa1muTXNo3X0l+eRjb7rZ3f5L9hrU9LU2Gux4nyYYkP+1CYHOSjyXZqQuw+7vHQ0l+1jf/riTHde33T3s8fdp270tyb5L/l+T4JL/Qt+9/SPLfptVzWJKNffNJ8pYk1yX5SZKNST6d5N9PW+/kLhgP7mt7V19dP5tW7/UzvBYru21M9dmc5G+SvKy/X1U9u6r+YY7XdWpb22+tX1WdW1W/sbU+8zXT61lVO1XVLcPYvpYuw12z+c9VtRPwfOAFwB92AbZT1/5/gROm5qvqPd16X+1rm3rcOW27Twb2AU4F3gGcucDaTgPeCrwFeCpwIPA54OVTHZIEeB1wD7B2qr2q3tP3bzh+Wr3P3so+d+nWeS5wGXBRkuMWWPec5gp+ab4Md21VVd0BfAk4aMjb/VFVXQz8JrA2yby2n+QA4M3Amqr6P1X1YFU90B3tntrX9VDg6fT+Ezg2yQ5DqvuuqjoNOBl479Snju5TyUu76YOTrE/y4+5I/0Pd6ld0z/d2nwJe1H3a+cckH05yD3By1/aVabs+KsktSb6f5P19+z05ySf6Xp9HPh0kOaV7HT7S7e8jXZ9HhnmSPCXJx5NMJrk1yR/2bfu4JF9J8oEkP0zyvSRHDuN11OgZ7tqqJHsBRwHfHMX2q+rrwEZ6ITQfhwMbu/W2Zi3wBeD8bv4Vi6twVp8FdgOeOcOy04DTqmpnYH/ggq79xd3zLt0nha928y8Ebum2d8os+zsGWE3vk9TRwBvmKrCqTuKxn7BOmKHbnwFPAfYDfg34beD1fctfCHwHWAG8Dziz+1SkJc5w12w+l+Re4CvAl4H3zNF/yiHdePrU4+Z5rHMnveGV+XgasGlrHZLsCLwa+GRV/Rz4DH1DM0MyNdQ0U90/B345yYqqur+qrpxrW1X1Z1W1pap+Okuf91bVPVV1G/C/gTWLrPsRSbaj98npnVV1X1VtAD5Ibzhryq1V9dGqegg4B9gD2H3QfWv0DHfN5lVVtUtV7VNV/2MroTPdld16U4/957HOnvTGxgG2AE+YtvwJ9AIT4Af0AmZrjum2c0k3fy5wZJKJedQyX3t2z/fMsOyN9L4HuDHJN5LM9anh9nnsr7/PrfSGnAa1Atih217/tvfsm79raqKqHugmdxrCvjVihrvGKskL6IXJ1BjzbcDKad325dEAuhx4RpLVW9nsWnoBdFuSu4BP0/sPYuCj3T7HAHfTG7J4jKr6blWtoTfM8l7gM0meBMx2C9b53Jp1r77pvXn0k8NPgB37lv3SArb9fXr/ae4zbdt3zKMeLXGGu8Yiyc7dEe2ngE9U1be7RecDr+++lEySA4G3d/2oqu8CfwGc150iuUOSX0xybJITk+xJb1z+FcCq7vFceiE78NBMkt2TnAC8m95wxsMz9PmtJBPdsnu75oeASeBheuPbC/UHSXbtvgN5K49+l3AN8OIkeyd5CvDOaettnm1/3VDLBcApSZ6cZB/g94BPzNRfy4vhrmF7UR5/nvsL+pZ/Icl99IYZTgI+RN8XeFX1d8CJwMeAH9EbWjkHOKNvG28BPgL8Ob3wvJnekfQX6I0XX1NVl3ZnttxVVXcBfwo8Z75n5czg3iQ/Ab5N7wvmV1fVWbP0PQK4Psn99L5cPbaqftYNa5wC/GP3fcQhC9j/54Gr6IX5F+lOH62qy+gF/bXd8r+Ztt5pwH/tznb50xm2+7v0jv5voffp6ZPAbP8uLSPxxzokqT0euUtSgwx3SWqQ4S5JDTLcJalBS+ImRStWrKiVK1eOuwxJWlauuuqq71fVjBfnLYlwX7lyJevXrx93GZK0rCS5dbZlDstIUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KD5rxCNclZ9H7V5u6qOqhrO59Hf/V9F+DeqlqVZCVwA4/+9NiVVXX8sIuWNForT/ziWPa74dSXj2W/LZrP7QfOpverNx+faqiq35yaTvJBer+YM+Xmqlo1rAIlSQs3Z7hX1RXdEfnjJAnwGuAlwy1LkjSIQcfcDwU2dz9aPGXfJN9M8uUkh862YpJ1SdYnWT85OTlgGZKkfoOG+xrgvL75TcDeVfU8er+i/skkO8+0YlWdUVWrq2r1xMSMd6yUJC3SosM9yfbAf6H3y+sAVNWDVfWDbvoqer9Kf+CgRUqSFmaQI/eXAjdW1caphiQTSbbrpvcDDgBuGaxESdJCzRnuSc4Dvgo8M8nGJG/sFh3LY4dkAF4MXJvkW8BngOOr6p5hFixJmtt8zpZZM0v7cTO0XQhcOHhZkqRBeIWqJDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaNGe4Jzkryd1JrutrOznJHUmu6R5H9S17Z5KbknwnyX8aVeGSpNnN58j9bOCIGdo/XFWrusclAEmeBRwLPLtb5y+SbDesYiVJ8zNnuFfVFcA989ze0cCnqurBqvoecBNw8AD1SZIWYZAx9xOSXNsN2+zate0J3N7XZ2PX9jhJ1iVZn2T95OTkAGVIkqZbbLifDuwPrAI2AR/s2jND35ppA1V1RlWtrqrVExMTiyxDkjSTRYV7VW2uqoeq6mHgozw69LIR2Kuv6zOAOwcrUZK0UIsK9yR79M0eA0ydSXMxcGySJybZFzgA+PpgJUqSFmr7uTokOQ84DFiRZCPwbuCwJKvoDblsAN4EUFXXJ7kA+CdgC/DmqnpoNKVLkmYzZ7hX1ZoZms/cSv9TgFMGKUqSNBivUJWkBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoPmDPckZyW5O8l1fW3vT3JjkmuTXJRkl659ZZKfJrmme/zlKIuXJM1sPkfuZwNHTGu7DDioqp4D/DPwzr5lN1fVqu5x/HDKlCQtxJzhXlVXAPdMa7u0qrZ0s1cCzxhBbZKkRRrGmPsbgC/1ze+b5JtJvpzk0NlWSrIuyfok6ycnJ4dQhiRpykDhnuQkYAtwbte0Cdi7qp4H/B7wySQ7z7RuVZ1RVauravXExMQgZUiSpll0uCdZC7wCeG1VFUBVPVhVP+imrwJuBg4cRqGSpPlbVLgnOQJ4B/DKqnqgr30iyXbd9H7AAcAtwyhUkjR/28/VIcl5wGHAiiQbgXfTOzvmicBlSQCu7M6MeTHwJ0m2AA8Bx1fVPTNuWJI0MnOGe1WtmaH5zFn6XghcOGhRkqTBeIWqJDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1aM4rVCVpW1l54hfHst8Np758LPsdJY/cJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDVoXrcfSHIW8Arg7qo6qGt7KnA+sBLYALymqn6YJMBpwFHAA8BxVXX18EuXtg0viddyNN8j97OBI6a1nQhcXlUHAJd38wBHAgd0j3XA6YOXKUlaiHmFe1VdAdwzrflo4Jxu+hzgVX3tH6+eK4FdkuwxjGIlSfMzyJj77lW1CaB73q1r3xO4va/fxq7tMZKsS7I+yfrJyckBypAkTTeKL1QzQ1s9rqHqjKpaXVWrJyYmRlCGJP3rNUi4b54abume7+7aNwJ79fV7BnDnAPuRJC3QIOF+MbC2m14LfL6v/bfTcwjwo6nhG0nStjHfUyHPAw4DViTZCLwbOBW4IMkbgduAV3fdL6F3GuRN9E6FfP2Qa5YkzWFe4V5Va2ZZdPgMfQt48yBFSZIG42+oSvpXb1wXqsHoLlbz9gOS1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkN8q6QWhbGedc+aTnyyF2SGmS4S1KDDHdJapDhLkkNMtwlqUGLPlsmyTOB8/ua9gP+CNgF+O/AZNf+rqq6ZNEVSpIWbNHhXlXfAVYBJNkOuAO4CHg98OGq+sBQKpQkLdiwhmUOB26uqluHtD1J0gCGFe7HAuf1zZ+Q5NokZyXZdUj7kCTN08DhnmQH4JXAp7um04H96Q3ZbAI+OMt665KsT7J+cnJypi6SpEUaxpH7kcDVVbUZoKo2V9VDVfUw8FHg4JlWqqozqmp1Va2emJgYQhmSpCnDCPc19A3JJNmjb9kxwHVD2IckaQEGunFYkh2BlwFv6mt+X5JVQAEbpi2TJG0DA4V7VT0APG1a2+sGqkiSNDCvUJWkBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMG+oFsgCQbgPuAh4AtVbU6yVOB84GVwAbgNVX1w0H3JUman2Eduf96Va2qqtXd/InA5VV1AHB5Ny9J2kZGNSxzNHBON30O8KoR7UeSNINhhHsBlya5Ksm6rm33qtoE0D3vNoT9SJLmaeAxd+BXqurOJLsBlyW5cT4rdf8RrAPYe++9h1CGJGnKwEfuVXVn93w3cBFwMLA5yR4A3fPdM6x3RlWtrqrVExMTg5YhSeozULgneVKSJ09NA78BXAdcDKztuq0FPj/IfiRJCzPosMzuwEVJprb1yar62yTfAC5I8kbgNuDVA+5HkrQAA4V7Vd0CPHeG9h8Ahw+ybUnS4nmFqiQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDFh3uSfZK8vdJbkhyfZK3du0nJ7kjyTXd46jhlStJmo/tB1h3C/D7VXV1kicDVyW5rFv24ar6wODlSZIWY9HhXlWbgE3d9H1JbgD2HFZhkqTFG8qYe5KVwPOAr3VNJyS5NslZSXadZZ11SdYnWT85OTmMMiRJnYHDPclOwIXA26rqx8DpwP7AKnpH9h+cab2qOqOqVlfV6omJiUHLkCT1GSjckzyBXrCfW1WfBaiqzVX1UFU9DHwUOHjwMiVJCzHI2TIBzgRuqKoP9bXv0dftGOC6xZcnSVqMQc6W+RXgdcC3k1zTtb0LWJNkFVDABuBNA1UoSVqwQc6W+QqQGRZdsvhyJEnD4BWqktSgQYZlJI3QyhO/OO4StIx55C5JDTLcJalBhrskNchwl6QGGe6S1CDPllmGxnkWxYZTXz62fUuaP4/cJalBhrskNchwl6QGGe6S1CC/UB2Al4dLWqoMdy2I/6FJy4PDMpLUIMNdkhrUxLCMQwWS9FgeuUtSgwx3SWqQ4S5JDRpZuCc5Isl3ktyU5MRR7UeS9HgjCfck2wF/DhwJPAtYk+RZo9iXJOnxRnXkfjBwU1XdUlX/AnwKOHpE+5IkTTOqUyH3BG7vm98IvLC/Q5J1wLpu9v4k3xlgfyuA7w+w/ra0nGqF5VWvtY7Ocqp3OdVK3jtQvfvMtmBU4Z4Z2uoxM1VnAGcMZWfJ+qpaPYxtjdpyqhWWV73WOjrLqd7lVCuMrt5RDctsBPbqm38GcOeI9iVJmmZU4f4N4IAk+ybZATgWuHhE+5IkTTOSYZmq2pLkBODvgO2As6rq+lHsqzOU4Z1tZDnVCsurXmsdneVU73KqFUZUb6pq7l6SpGXFK1QlqUGGuyQ1aFmHe5JdknwmyY1JbkjyonHXtDVJ3p7k+iTXJTkvyS+Ou6YpSc5KcneS6/ranprksiTf7Z53HWeN/Wap9/3d38K1SS5Ksss4a5wyU619y/5nkkqyYhy1zWS2epP8bndLkeuTvG9c9fWb5e9gVZIrk1yTZH2Sg8dZ45QkeyX5+y6rrk/y1q59JO+zZR3uwGnA31bVvwWeC9ww5npmlWRP4C3A6qo6iN4XzceOt6rHOBs4YlrbicDlVXUAcHk3v1SczePrvQw4qKqeA/wz8M5tXdQszubxtZJkL+BlwG3buqA5nM20epP8Or2rzJ9TVc8GPjCGumZyNo9/bd8H/HFVrQL+qJtfCrYAv19V/w44BHhzd1uWkbzPlm24J9kZeDFwJkBV/UtV3Tvequa0PfBvkmwP7MgSOve/qq4A7pnWfDRwTjd9DvCqbVrUVsxUb1VdWlVbutkr6V1fMXazvLYAHwb+F9Mu8Bu3Wer9HeDUqnqw63P3Ni9sBrPUWsDO3fRTWCLvs6raVFVXd9P30TsY3ZMRvc+WbbgD+wGTwMeSfDPJXyV50riLmk1V3UHvaOc2YBPwo6q6dLxVzWn3qtoEvT9MYLcx17MQbwC+NO4iZpPklcAdVfWtcdcyTwcChyb5WpIvJ3nBuAvaircB709yO7333FL5BPeIJCuB5wFfY0Tvs+Uc7tsDzwdOr6rnAT9haQ0bPEY3jnY0sC/wdOBJSX5rvFW1KclJ9D4CnzvuWmaSZEfgJHpDBsvF9sCu9IYT/gC4IMlMtxlZCn4HeHtV7QW8ne7T/VKRZCfgQuBtVfXjUe1nOYf7RmBjVX2tm/8MvbBfql4KfK+qJqvq58Bngf845prmsjnJHgDd85L4KL41SdYCrwBeW0v3Io796f0n/60kG+gNH12d5JfGWtXWbQQ+Wz1fBx6md4OupWgtvfcXwKfp3aV2SUjyBHrBfm5VTdU4kvfZsg33qroLuD3JM7umw4F/GmNJc7kNOCTJjt0Rz+Es4S+AOxfTe6PQPX9+jLXMKckRwDuAV1bVA+OuZzZV9e2q2q2qVlbVSnrB+fzub3qp+hzwEoAkBwI7sHTvvHgn8Gvd9EuA746xlkd07/szgRuq6kN9i0bzPquqZfsAVgHrgWvp/fHtOu6a5qj3j4EbgeuAvwaeOO6a+mo7j953AT+nFzZvBJ5G79v773bPTx13nXPUexO9W01f0z3+ctx1zlbrtOUbgBXjrnOO13YH4BPd3+7VwEvGXedWav1V4CrgW/TGtP/DuOvsav1Vel/2Xtv3N3rUqN5n3n5Akhq0bIdlJEmzM9wlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSg/4/zNwKv3rHwNQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "count    627.000000\n",
      "mean      15.612440\n",
      "std        2.978845\n",
      "min        6.000000\n",
      "25%       13.000000\n",
      "50%       16.000000\n",
      "75%       18.000000\n",
      "max       20.000000\n",
      "Name: PTEDUCAT, dtype: float64\n"
     ]
    }
   ],
   "source": [
    "from matplotlib import pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "print('Numerical Var Distributions:\\n')\n",
    "\n",
    "for var in numerical_vars:\n",
    "    plt.hist(X[var], bins=10)\n",
    "    plt.title(var + ' Distribution')\n",
    "    plt.show()\n",
    "    \n",
    "    # descriptive stats\n",
    "    print(X[var].describe())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Analyze target (y) distribution\n",
    "* High number (304) of limited impairment (LMCI). Maybe old age? Or the patients being evaluated are not random elderly patients but patients already known to have cognitive issues."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD4CAYAAAAXUaZHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAOyklEQVR4nO3df6xfdX3H8edLirgNM0AujLWNl2GNgonF3CEJ/6gsU9GkkMjWLtFq2OofuPkry6pZoiZjwcwfi27i6mDWxVmJyugGc2NVQ/hD8QIdUiqzQGevbehVlB/xx0Z974/vufql/d7eb++933vbD89H8s05530+53zfN9/01dPPPefbVBWSpLY8a7kbkCQtPsNdkhpkuEtSgwx3SWqQ4S5JDVqx3A0AnHnmmTU+Pr7cbUjSCeWuu+76flWNDdp3XIT7+Pg4k5OTy92GJJ1QkvzPbPuclpGkBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1aM5wT/KcJHcm+a8ku5J8oKufm+QbSb6T5PNJnt3VT+m293T7x0f7I0iSDjfME6o/A15VVU8mORm4I8m/Ae8CPlpV25J8ErgKuK5b/rCqXpBkPfBB4PdH1D/jm28Z1amf8fZe+7rlbkHSPM155V49T3abJ3evAl4FfKGrbwUu79bXddt0+y9NkkXrWJI0p6Hm3JOclGQncBC4DXgQ+FFVPdUNmQJWdusrgX0A3f7HgOcNOOemJJNJJqenpxf2U0iSnmaocK+qQ1W1FlgFXAS8eNCwbjnoKv2I/6i1qrZU1URVTYyNDfxSM0nSPB3T3TJV9SPga8DFwGlJZubsVwH7u/UpYDVAt//XgUcXo1lJ0nCGuVtmLMlp3fqvAL8D7Aa+CryhG7YRuLlb395t0+3/SlUdceUuSRqdYe6WOQfYmuQken8Z3FhV/5rkfmBbkr8A7gGu78ZfD/xjkj30rtjXj6BvSdJRzBnuVXUvcOGA+kP05t8Pr/8UuHJRupMkzYtPqEpSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSg+YM9ySrk3w1ye4ku5K8vau/P8n3kuzsXpf1HfOeJHuSPJDk1aP8ASRJR1oxxJingHdX1d1JngvcleS2bt9Hq+pD/YOTnA+sBy4AfhP4zyQvrKpDi9m4JGl2c165V9WBqrq7W38C2A2sPMoh64BtVfWzqnoY2ANctBjNSpKGc0xz7knGgQuBb3SltyW5N8kNSU7vaiuBfX2HTTHgL4Mkm5JMJpmcnp4+5sYlSbMbOtyTnAp8EXhHVT0OXAecB6wFDgAfnhk64PA6olC1paomqmpibGzsmBuXJM1uqHBPcjK9YP9sVX0JoKoeqapDVfVz4FP8cuplCljdd/gqYP/itSxJmsswd8sEuB7YXVUf6auf0zfsCuC+bn07sD7JKUnOBdYAdy5ey5KkuQxzt8wlwBuBbyXZ2dXeC2xIspbelMte4K0AVbUryY3A/fTutLnaO2UkaWnNGe5VdQeD59FvPcox1wDXLKAvSdIC+ISqJDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDVoznBPsjrJV5PsTrIrydu7+hlJbkvynW55eldPko8l2ZPk3iQvG/UPIUl6umGu3J8C3l1VLwYuBq5Ocj6wGdhRVWuAHd02wGuBNd1rE3DdonctSTqqOcO9qg5U1d3d+hPAbmAlsA7Y2g3bClzera8DPlM9XwdOS3LOoncuSZrVMc25JxkHLgS+AZxdVQeg9xcAcFY3bCWwr++wqa4mSVoiQ4d7klOBLwLvqKrHjzZ0QK0GnG9Tkskkk9PT08O2IUkawlDhnuRkesH+2ar6Uld+ZGa6pVse7OpTwOq+w1cB+w8/Z1VtqaqJqpoYGxubb/+SpAGGuVsmwPXA7qr6SN+u7cDGbn0jcHNf/U3dXTMXA4/NTN9IkpbGiiHGXAK8EfhWkp1d7b3AtcCNSa4Cvgtc2e27FbgM2AP8GHjLonYsSZrTnOFeVXcweB4d4NIB4wu4eoF9SZIWwCdUJalBw0zLSItqfPMty91Cs/Ze+7rlbkHHCa/cJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWrQnOGe5IYkB5Pc11d7f5LvJdnZvS7r2/eeJHuSPJDk1aNqXJI0u2Gu3D8NvGZA/aNVtbZ73QqQ5HxgPXBBd8wnkpy0WM1KkoYzZ7hX1e3Ao0Oebx2wrap+VlUPA3uAixbQnyRpHhYy5/62JPd20zand7WVwL6+MVNdTZK0hOYb7tcB5wFrgQPAh7t6BoytQSdIsinJZJLJ6enpebYhSRpkXuFeVY9U1aGq+jnwKX459TIFrO4bugrYP8s5tlTVRFVNjI2NzacNSdIs5hXuSc7p27wCmLmTZjuwPskpSc4F1gB3LqxFSdKxWjHXgCSfA14BnJlkCngf8Ioka+lNuewF3gpQVbuS3AjcDzwFXF1Vh0bTuiRpNnOGe1VtGFC+/ijjrwGuWUhTkqSF8QlVSWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1KA5b4WUpPHNtyx3C83ae+3rRnJer9wlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBs0Z7kluSHIwyX19tTOS3JbkO93y9K6eJB9LsifJvUleNsrmJUmDDXPl/mngNYfVNgM7qmoNsKPbBngtsKZ7bQKuW5w2JUnHYs5wr6rbgUcPK68DtnbrW4HL++qfqZ6vA6clOWexmpUkDWe+c+5nV9UBgG55VldfCezrGzfV1Y6QZFOSySST09PT82xDkjTIYv9CNQNqNWhgVW2pqomqmhgbG1vkNiTpmW2+4f7IzHRLtzzY1aeA1X3jVgH759+eJGk+5hvu24GN3fpG4Oa++pu6u2YuBh6bmb6RJC2dFXMNSPI54BXAmUmmgPcB1wI3JrkK+C5wZTf8VuAyYA/wY+AtI+hZkjSHOcO9qjbMsuvSAWMLuHqhTUmSFsYnVCWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQSsWcnCSvcATwCHgqaqaSHIG8HlgHNgL/F5V/XBhbUqSjsViXLm/sqrWVtVEt70Z2FFVa4Ad3bYkaQmNYlpmHbC1W98KXD6C95AkHcVCw72A/0hyV5JNXe3sqjoA0C3PGnRgkk1JJpNMTk9PL7ANSVK/Bc25A5dU1f4kZwG3Jfn2sAdW1RZgC8DExEQtsA9JUp8FXblX1f5ueRC4CbgIeCTJOQDd8uBCm5QkHZt5h3uSX0vy3Jl14HeB+4DtwMZu2Ebg5oU2KUk6NguZljkbuCnJzHn+qaq+nOSbwI1JrgK+C1y58DYlScdi3uFeVQ8BLx1Q/wFw6UKakiQtjE+oSlKDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDRhbuSV6T5IEke5JsHtX7SJKONJJwT3IS8LfAa4HzgQ1Jzh/Fe0mSjjSqK/eLgD1V9VBV/S+wDVg3oveSJB1mxYjOuxLY17c9Bby8f0CSTcCmbvPJJA+MqJfjzZnA95e7iWHkg8vdwXHDz+zEcsJ8XrDgz+z5s+0YVbhnQK2etlG1Bdgyovc/biWZrKqJ5e5Dw/MzO7H4efWMalpmCljdt70K2D+i95IkHWZU4f5NYE2Sc5M8G1gPbB/Re0mSDjOSaZmqeirJ24B/B04CbqiqXaN4rxPQM24qqgF+ZicWPy8gVTX3KEnSCcUnVCWpQYa7JDXIcF8ESZ4cUHt/kkrygr7aO7vaRLd9apK/S/Jgkl1Jbk/y8tnOqdFK8htJtnWfx/1Jbk3ywu4z++O+cX+T5M3L2Ko6Sa7oPp8XddvjSX6S5J4ku5PcmWTjcve5HAz30foWvTuFZrwBuL9v+++BR4E1VXUB8GZ6D2BoiSUJcBPwtao6r6rOB94LnA0cBN7e3fml48sG4A6e/ufswaq6sKpe3NXfmeQty9LdMjLcR+uf6b52IclvAY8B0932efSe2v3zqvo5QPd1DbcsU6/PdK8E/q+qPjlTqKqd9J60ngZ2AM/IK8DjVZJTgUuAq3h6uP9CVT0EvAv4kyVs7bhguI/W48C+JC+hd4Xx+b59FwA7q+rQsnSmw70EuOso+68F3t19KZ6OD5cDX66q/wYeTfKyWcbdDbxo6do6Phjuo7eN3lXF5fT+2a8TUFU9DNwJ/MFy96Jf2EDvzxfdcsMs4wZ9HUrzRvXdMvqlfwH+Cpisqsd7U7sA7AJemuRZM9MyWla76P1O5Gj+EvgCcPvo29HRJHke8CrgJUmK3sOSBXxiwPALgd1L2N5xwSv3EauqnwB/BlxzWP1BYBL4QPfLPJKsSeJXIy+PrwCnJPmjmUKS36bvW/eq6tv0fiH++qVvT4d5A/CZqnp+VY1X1WrgYXrfY/ULScaBDwEfX/IOl5lX7ovjV5NM9W1/pH9nVW1jsD8EPgzsSfJj4AfAn46mRR1NVVWSK4C/7v7nsJ8Ce4F3HDb0GuCeJW5PR9pA7/cg/b5I7w6n85LcAzwHeAL4eFX9wxL3t+z8+gFJapDTMpLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNej/AV5RiH0QzZn2AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.bar(Y.value_counts().index, Y.value_counts())\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pre-processing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Convert Categorical Variables to 1-Hot\n",
    "* Next time use `sklearn.preprocessing.OneHotEncoder`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Converting imputed_genotype to 1-hot encoding\n",
      "Converting APOE Genotype to 1-hot encoding\n",
      "Converting PTGENDER to 1-hot encoding\n",
      "Converting APOE4 to 1-hot encoding\n",
      "Converting PTETHCAT to 1-hot encoding\n",
      "Converting PTRACCAT to 1-hot encoding\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AGE</th>\n",
       "      <th>PTEDUCAT</th>\n",
       "      <th>MMSE</th>\n",
       "      <th>False</th>\n",
       "      <th>True</th>\n",
       "      <th>2,2</th>\n",
       "      <th>2,3</th>\n",
       "      <th>2,4</th>\n",
       "      <th>3,3</th>\n",
       "      <th>3,4</th>\n",
       "      <th>...</th>\n",
       "      <th>Male</th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>Hisp/Latino</th>\n",
       "      <th>Not Hisp/Latino</th>\n",
       "      <th>Unknown</th>\n",
       "      <th>Asian</th>\n",
       "      <th>Black</th>\n",
       "      <th>White</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>81.3</td>\n",
       "      <td>18</td>\n",
       "      <td>20</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>67.5</td>\n",
       "      <td>10</td>\n",
       "      <td>27</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>73.7</td>\n",
       "      <td>16</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>80.4</td>\n",
       "      <td>13</td>\n",
       "      <td>25</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>4 rows × 22 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    AGE  PTEDUCAT  MMSE  False  True  2,2  2,3  2,4  3,3  3,4  ...  Male  0  \\\n",
       "0  81.3        18    20      0     1    0    0    0    0    1  ...     1  0   \n",
       "1  67.5        10    27      1     0    0    0    0    1    0  ...     1  1   \n",
       "2  73.7        16    29      0     1    0    0    0    1    0  ...     1  1   \n",
       "3  80.4        13    25      0     1    0    0    0    1    0  ...     0  1   \n",
       "\n",
       "   1  2  Hisp/Latino  Not Hisp/Latino  Unknown  Asian  Black  White  \n",
       "0  1  0            0                1        0      0      0      1  \n",
       "1  0  0            1                0        0      0      0      1  \n",
       "2  0  0            0                1        0      0      0      1  \n",
       "3  0  0            0                1        0      0      0      1  \n",
       "\n",
       "[4 rows x 22 columns]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#for each categorical var, convert to 1-hot encoding\n",
    "for var in cat_vars:\n",
    "    print('Converting', var, 'to 1-hot encoding')\n",
    "    \n",
    "    #get 1-hot and replace original column with the >= 2 categories as columns\n",
    "    one_hot_df = pd.get_dummies(X[var])\n",
    "    X = pd.concat([X, one_hot_df], axis=1)\n",
    "    X = X.drop(var, axis=1)\n",
    "    \n",
    "X.head(4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Remember:\n",
    "* PTGENDER: (Male, Female)\n",
    "* PTETHCAT: (Hisp/Latino, Not Hisp/Latino, Unknown)\n",
    "* PTRACCAT: (White, Black, Asian)\n",
    "* APOE4: (0, 1, 2)\n",
    "* APOE4 Genotype: (22, 23, 24, 33, 34, 44)\n",
    "* imputed_genotype (True, False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Normalization code if necessary (for neural network)\n",
    "* Could also use from `sklearn.preprocessing` `StandardScaler` or `Normalizer` or `MinMaxScaler`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "def normalize(X):\n",
    "    \"\"\"\n",
    "    Normalize numerical variables for faster and better convergence\n",
    "    \n",
    "    Args:\n",
    "        X (pd.DataFrame): data\n",
    "        \n",
    "    Returns:\n",
    "        X (numpy array): same data but with normalized numerical features\n",
    "    \"\"\"\n",
    "\n",
    "    #Convert to numpy array\n",
    "    X = np.array(X)\n",
    "    sanity_check = 0\n",
    "\n",
    "    #Normalize numerical variables to speed up convergence\n",
    "    for i in range(3):\n",
    "        mean = np.mean(X[:, i])\n",
    "        sd = np.std(X[:, i])\n",
    "        print('\\nNormalizing', numerical_vars[i], 'with mean=', format(mean, '.2f'), 'and sd=', format(sd, '.2f'))\n",
    "\n",
    "        X[:, i] = (X[:, i] - mean) / sd\n",
    "        sanity_check += np.mean(X[:, i])\n",
    "\n",
    "    print('\\nSanity Check. Sum of all the means should be near 0:', sanity_check)\n",
    "\n",
    "    return X"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train/Test Split\n",
    "* `X_train` and `y_train` will further split into train/valid splits using K-fold CV\n",
    "* `X_test` and `y_test` will be held out"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_train: (501, 22) \ty_train: (501,)\n",
      "X_test: (126, 22) \ty_test: (126,)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=.2)\n",
    "\n",
    "print('X_train:', X_train.shape, '\\ty_train:', y_train.shape)\n",
    "print('X_test:', X_test.shape, '\\ty_test:', y_test.shape)\n",
    "num_test = X_test.shape[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Logistic Regression, Random Forest, KNN, MLP"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.neural_network import MLPClassifier\n",
    "\n",
    "from sklearn.model_selection import cross_val_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Validation Accuracy =  69.05%\n"
     ]
    }
   ],
   "source": [
    "log_clf = LogisticRegression(solver='lbfgs', penalty='l2', max_iter=1000000, multi_class='multinomial')\n",
    "print('Validation Accuracy = ', format(cross_val_score(log_clf, X_train, y_train, cv=5).mean(), '.2%'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Validation Accuracy =  64.88%\n"
     ]
    }
   ],
   "source": [
    "rf_clf = RandomForestClassifier(n_estimators=200)\n",
    "print('Validation Accuracy = ', format(cross_val_score(rf_clf, X_train, y_train, cv=5).mean(), '.2%'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Validation Accuracy =  68.06%\n"
     ]
    }
   ],
   "source": [
    "knn_clf = KNeighborsClassifier(n_neighbors=10)\n",
    "print('Validation Accuracy = ', format(cross_val_score(knn_clf, X_train, y_train, cv=5).mean(), '.2%'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Validation Accuracy =  53.06%\n"
     ]
    }
   ],
   "source": [
    "mlp_clf = MLPClassifier(hidden_layer_sizes=(15, 10), alpha=3, learning_rate='adaptive', max_iter=100000)\n",
    "print('Validation Accuracy = ', format(cross_val_score(mlp_clf, X_train, y_train, cv=5).mean(), '.2%'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tangent: Evaluate feature importances given by Random Forest"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Remember:\n",
    "* PTGENDER: (Male, Female)\n",
    "* PTETHCAT: (Hisp/Latino, Not Hisp/Latino, Unknown)\n",
    "* PTRACCAT: (White, Black, Asian)\n",
    "* APOE4: (0, 1, 2)\n",
    "* APOE4 Genotype: (22, 23, 24, 33, 34, 44)\n",
    "* imputed_genotype (True, False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RandomForestClassifier(bootstrap=True, class_weight=None, criterion='gini',\n",
       "            max_depth=None, max_features='auto', max_leaf_nodes=None,\n",
       "            min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "            min_samples_leaf=1, min_samples_split=2,\n",
       "            min_weight_fraction_leaf=0.0, n_estimators=200, n_jobs=None,\n",
       "            oob_score=False, random_state=None, verbose=0,\n",
       "            warm_start=False)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rf_clf.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "          Importance\n",
      "MMSE        0.362151\n",
      "AGE         0.297750\n",
      "PTEDUCAT    0.160082\n",
      "Male        0.019854\n",
      "Female      0.019552\n",
      "0           0.018643\n",
      "False       0.016984\n",
      "True        0.016215\n",
      "2,3         0.012140\n",
      "3,3         0.010497\n"
     ]
    }
   ],
   "source": [
    "# https://towardsdatascience.com/running-random-forests-inspect-the-feature-importances-with-this-code-2b00dd72b92e\n",
    "feature_importances = pd.DataFrame(rf_clf.feature_importances_, index=X_train.columns, \n",
    "                                   columns=['Importance']).sort_values('Importance', ascending=False)\n",
    "print(feature_importances[:10])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Conclusion: Feature Importance\n",
    "* Cognitive test score (MMSE), age (AGE), years of education (PTEDUCAT) are considered the three most important features by far\n",
    "* Interestingly, the genetic information (APOE4 and APOE4 Genotype) was not considered very important by the model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model Selection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Choosing Logistic Regression\n",
    "* `Validation Score = .69` - unable to take advantage of more complex models, likely due to small data set size"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Test Logistic Regression Model\n",
    "* `Test Score = .746` - take with a grain of salt since validation score was `.69`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test Accuracy = 74.60%\n"
     ]
    }
   ],
   "source": [
    "log_clf.fit(X_train, y_train)\n",
    "\n",
    "log_clf_preds = log_clf.predict(X_test)\n",
    "    \n",
    "log_clf_accuracy = (log_clf_preds == y_test)\n",
    "print('Test Accuracy =', format( np.sum(log_clf_accuracy)/num_test, '.2%'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Analyze misclassifications"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AGE</th>\n",
       "      <th>PTEDUCAT</th>\n",
       "      <th>MMSE</th>\n",
       "      <th>False</th>\n",
       "      <th>True</th>\n",
       "      <th>2,2</th>\n",
       "      <th>2,3</th>\n",
       "      <th>2,4</th>\n",
       "      <th>3,3</th>\n",
       "      <th>3,4</th>\n",
       "      <th>...</th>\n",
       "      <th>Male</th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>Hisp/Latino</th>\n",
       "      <th>Not Hisp/Latino</th>\n",
       "      <th>Unknown</th>\n",
       "      <th>Asian</th>\n",
       "      <th>Black</th>\n",
       "      <th>White</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>486</th>\n",
       "      <td>87.9</td>\n",
       "      <td>18</td>\n",
       "      <td>24</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>502</th>\n",
       "      <td>56.2</td>\n",
       "      <td>18</td>\n",
       "      <td>24</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>616</th>\n",
       "      <td>74.6</td>\n",
       "      <td>16</td>\n",
       "      <td>24</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>513</th>\n",
       "      <td>69.7</td>\n",
       "      <td>14</td>\n",
       "      <td>25</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>480</th>\n",
       "      <td>59.8</td>\n",
       "      <td>13</td>\n",
       "      <td>30</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 22 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      AGE  PTEDUCAT  MMSE  False  True  2,2  2,3  2,4  3,3  3,4  ...  Male  0  \\\n",
       "486  87.9        18    24      1     0    0    0    0    1    0  ...     1  1   \n",
       "502  56.2        18    24      1     0    0    0    0    1    0  ...     0  1   \n",
       "616  74.6        16    24      1     0    0    0    0    0    1  ...     0  0   \n",
       "513  69.7        14    25      0     1    0    0    0    0    0  ...     0  0   \n",
       "480  59.8        13    30      1     0    0    0    0    1    0  ...     0  1   \n",
       "\n",
       "     1  2  Hisp/Latino  Not Hisp/Latino  Unknown  Asian  Black  White  \n",
       "486  0  0            0                1        0      1      0      0  \n",
       "502  0  0            1                0        0      0      0      1  \n",
       "616  1  0            0                1        0      0      1      0  \n",
       "513  0  1            0                1        0      0      0      1  \n",
       "480  0  0            0                1        0      0      0      1  \n",
       "\n",
       "[5 rows x 22 columns]"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "misclassified_index = (log_clf_preds != y_test)\n",
    "\n",
    "X_misclassified = X_test[misclassified_index]\n",
    "y_misclassified = y_test[misclassified_index]\n",
    "preds_misclassified = log_clf_preds[misclassified_index]\n",
    "\n",
    "X_misclassified.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pred  Y\n",
      "-----------\n",
      "AD    LMCI\n",
      "AD    LMCI\n",
      "AD    LMCI\n",
      "AD    LMCI\n",
      "CN    LMCI\n",
      "LMCI  CN\n",
      "LMCI  CN\n",
      "AD    LMCI\n",
      "LMCI  CN\n",
      "LMCI  CN\n",
      "LMCI  AD\n",
      "LMCI  CN\n",
      "LMCI  CN\n",
      "AD    LMCI\n",
      "LMCI  AD\n",
      "LMCI  AD\n",
      "CN    LMCI\n",
      "AD    LMCI\n",
      "CN    LMCI\n",
      "AD    LMCI\n",
      "CN    LMCI\n",
      "LMCI  CN\n",
      "LMCI  AD\n",
      "CN    LMCI\n",
      "LMCI  CN\n",
      "CN    LMCI\n",
      "CN    LMCI\n",
      "AD    LMCI\n",
      "CN    LMCI\n",
      "CN    LMCI\n",
      "CN    LMCI\n",
      "CN    LMCI\n"
     ]
    }
   ],
   "source": [
    "print('Pred'.ljust(5), 'Y')\n",
    "print('-----------')\n",
    "\n",
    "over_estimate = under_estimate = total_miss = 0\n",
    "for pred, y in zip(preds_misclassified, y_misclassified):\n",
    "    print(pred.ljust(5), y)\n",
    "    \n",
    "    if (y == 'CN' and pred == 'AD') or (y == 'AD' and pred == 'CN'):\n",
    "        total_miss += 1\n",
    "    \n",
    "    if y == 'CN':\n",
    "        over_estimate += 1\n",
    "        \n",
    "    elif y == 'AD':\n",
    "        under_estimate += 1\n",
    "    \n",
    "    # y is 'LMCI'\n",
    "    else:\n",
    "        if pred == 'CN':\n",
    "            under_estimate += 1\n",
    "        elif pred == 'AD':\n",
    "            over_estimate += 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model over-estimated severity of cognitive problems: 17\n",
      "Model under-estimated severity of cognitive problems: 15\n",
      "Model was way off (total miss): 0\n"
     ]
    }
   ],
   "source": [
    "print('Model over-estimated severity of cognitive problems:', over_estimate)\n",
    "print('Model under-estimated severity of cognitive problems:', under_estimate)\n",
    "print('Model was way off (total miss):', total_miss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Conclusion\n",
    "* Fairly even misclassification, No total misses\n",
    "* Probably want to **lower** `under-estimated`. Would be willing to sacrifice accuracy to get a more conservative model.\n",
    "* e.g. **False Negative:** Predicting cognitively normal (CN) when a patient has Alzheimer's (AD) would be much worse than predicting AD when a patient is normal (CN)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Look at distribution of 3 most important vars: AGE, MMSE, PTEDUCAT in misclassified examples\n",
    "* means for entire dataset = `75, 26.9, 15.6`\n",
    "* distributions looks relatively similar to that of entire data set. No obvious discrepancy "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAEICAYAAAB25L6yAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAARzUlEQVR4nO3de5BkZX3G8e8jKwiIAWS8AK4LmmDQKpAaCIhSBoiCJGhSXiCgeMsmKUzUsgrRWIJGKsQyiYlapDYoUCogIkSDl6CJREkpZpeLchHlssJyHUTUxQsXf/mjz0Az9Oz0QPfMO+z3U9U13X3OnvPMWzPPnnn7dJ9UFZKkdj1usQNIkjbMopakxlnUktQ4i1qSGmdRS1LjLGpJapxFrSUvyZeTHDWibb0oydV9j9cmOXAU2+62d0WSF49qe9o4WNSatyQXJPlJks0GLJtMcl63/K4kVyY5Ick23fLXJ7k/yfoZt+1n2Vclubtb58dJ/ivJa/rXqaqDq+q0IXJXkmdvaJ2q+mZV7TLXtoaR5NQkH5ix/edW1QWj2L42Hha15iXJCuBFQAGHzlj2AuAC4H+B51TV1sBBwH3Abn2rfquqnjjjdvMGdrtbVT0R2AU4FfhokuNG8x09JP+yUW9TGgWLWvP1OuDb9Apz5nTDB4FTqurvquo2gKq6oaqOG8VRZFXdUVWfBP4SeFeSJ8MDR/hv7u4/O8n/JPlpkjuSfKZ7/hvdZi7rjs5fk+TFSdYleWeSW4FTpp+bses9u78MfpLklCRP6Lb5+iQX9q84fdSeZCVwBHBMt7//6JY/MJWSZLMkH05yc3f78PRfKX3Z3pHk9iS3JHnDox1DLU0WtebrdcCnu9tLkzwVIMmWwD7A5xYgw+eBZcBeA5b9LXA+sA2wI/ARgKrar1u+W3cE/5nu8dOAbYFnAitn2d8RwEuBZwG/A7xnroBVtYreGH2w298fDVjtb4C9gd3p/cWx14xtPw34LWAH4E3Ax6ankLRxsag1tCQvpFdoZ1XVGuBa4E+7xdvQ+3m6tW/9D3bz1Hcn6S+gvbvnp2/XzidHVd0L3EGvYGe6t8u4fVX9qqouHLBOv98Ax1XVr6vql7Os89GqurGq7gROAA6fT94NOAJ4f1XdXlVTwPuA1/Ytv7dbfm9VfQlYT2/6RxsZi1rzcRRwflXd0T0+nQenP35Cr/SePr1yVR3TzVOfS+8IeNq3q2rrvtuz5hMiyeOBCeDOAYuPAQJ8pzvD4o1zbG6qqn41xzo39t3/ETDwhc9HYPtue7Nt+8dVdV/f418ATxzRvrWE+OKJhpJkc+DVwCbdfC7AZsDWSXarqsuSXAT8CfD1Mcd5Ob0XKL8zc0FV3Qr8WZf5hcDXknyjqq6ZZVvDfHzkM/ruLwemX/i8G9hiekGSp81z2zfTO/q/YsC2pQd4RK1hvQK4H9iV3pzq7sDvAt+kN28NvaPZNyY5NslTAJLsCOw0igBJtk1yBPAx4O+r6scD1nlVt0/oHeVXlxvgNmDnR7Dro5PsmGRb4N3A9Pz2ZcBzk+zevcB4/Ix/N9f+zgDek2QiyXbAe4FPPYJ8eoyzqDWso+id0XFDVd06fQM+ChyRZFk3H7w/sB/wgyR3AV+hd8reR/q2tc+A86j33MC+L0uyHrgGeDPw9qp67yzr7glc1K3/BeCtVXV9t+x44LRuXvzV8/jeT6f3AuV13e0DAFX1A+D9wNeAHwIz58M/Duza7e/fB2z3A8Bq4LvA94CLp7ct9YsXDpCktnlELUmNs6glqXEWtSQ1zqKWpMaN5Tzq7bbbrlasWDGOTUvSY9KaNWvuqKqJQcvGUtQrVqxg9erV49i0JD0mJfnRbMuc+pCkxlnUktQ4i1qSGmdRS1LjLGpJapxFLUmNG6qok7y9+xD2y5OcMX3NOEnS+M1Z1El2AP4amKyq5wGbAIeNO5gkqWfYqY9lwOZJltG7ooVXoZCkBTLnOxOr6qYkHwJuAH5J75p5589cL8lKuqs4L1++fNQ59Riz4tgvLsp+1554yKLsd7G+X1i871mjM8zUxzb0rlG3E70Lb26Z5MiZ61XVqqqarKrJiYmBb1eXJD0Cw0x9HAhcX1VTVXUvcA7wgvHGkiRNG6aobwD2TrJFkgAHAFeNN5YkadqcRV1VFwFn07vw5ve6f7NqzLkkSZ2hPua0qo4DjhtzFknSAL4zUZIaZ1FLUuMsaklqnEUtSY2zqCWpcRa1JDXOopakxlnUktQ4i1qSGmdRS1LjLGpJapxFLUmNs6glqXEWtSQ1zqKWpMZZ1JLUuGEubrtLkkv7bj9L8raFCCdJGuIKL1V1NbA7QJJNgJuAc8ecS5LUme/UxwHAtVX1o3GEkSQ93HyL+jDgjHEEkSQNNnRRJ9kUOBT47CzLVyZZnWT11NTUqPJJ0kZvPkfUBwMXV9VtgxZW1aqqmqyqyYmJidGkkyTNq6gPx2kPSVpwQxV1ki2APwDOGW8cSdJMc56eB1BVvwCePOYskqQBfGeiJDXOopakxlnUktQ4i1qSGmdRS1LjLGpJapxFLUmNs6glqXEWtSQ1zqKWpMZZ1JLUOItakhpnUUtS4yxqSWqcRS1JjbOoJalxFrUkNW7YS3FtneTsJN9PclWSfcYdTJLUM9SluIB/Br5SVa9MsimwxRgzSZL6zFnUSZ4E7Ae8HqCq7gHuGW8sSdK0YaY+dgamgFOSXJLk5CRbzlwpycokq5OsnpqaGnlQSdpYDVPUy4A9gJOq6vnA3cCxM1eqqlVVNVlVkxMTEyOOKUkbr2GKeh2wrqou6h6fTa+4JUkLYM6irqpbgRuT7NI9dQBw5VhTSZIeMOxZH38FfLo74+M64A3jiyRJ6jdUUVfVpcDkmLNIkgbwnYmS1DiLWpIaZ1FLUuMsaklqnEUtSY2zqCWpcRa1JDXOopakxlnUktQ4i1qSGmdRS1LjLGpJapxFLUmNs6glqXEWtSQ1zqKWpMZZ1JLUuKGu8JJkLfBz4H7gvqryai+StECGvWYiwO9X1R1jSyJJGsipD0lq3LBFXcD5SdYkWTlohSQrk6xOsnpqamp0CSVpIzdsUe9bVXsABwNHJ9lv5gpVtaqqJqtqcmJiYqQhJWljNlRRV9XN3dfbgXOBvcYZSpL0oDmLOsmWSbaavg+8BLh83MEkST3DnPXxVODcJNPrn15VXxlrKknSA+Ys6qq6DthtAbJIkgbw9DxJapxFLUmNs6glqXEWtSQ1zqKWpMZZ1JLUOItakhpnUUtS4yxqSWqcRS1JjbOoJalxFrUkNc6ilqTGWdSS1DiLWpIaZ1FLUuMsaklq3NBFnWSTJJckOW+cgSRJDzWfI+q3AleNK4gkabChijrJjsAhwMnjjSNJmmmYq5ADfBg4BthqthWSrARWAixfvvzRJ5PGYMWxX1zsCAtusb7ntScesij7hcfe9zznEXWSPwRur6o1G1qvqlZV1WRVTU5MTIwsoCRt7IaZ+tgXODTJWuBMYP8knxprKknSA+Ys6qp6V1XtWFUrgMOA/66qI8eeTJIEeB61JDVv2BcTAaiqC4ALxpJEkjSQR9SS1DiLWpIaZ1FLUuMsaklqnEUtSY2zqCWpcRa1JDXOopakxlnUktQ4i1qSGmdRS1LjLGpJapxFLUmNs6glqXEWtSQ1zqKWpMZZ1JLUuGGuQv6EJN9JclmSK5K8byGCSZJ6hrkU16+B/atqfZLHAxcm+XJVfXvM2SRJDFHUVVXA+u7h47tbjTOUJOlBQ81RJ9kkyaXA7cBXq+qiAeusTLI6yeqpqalR55SkjdZQRV1V91fV7sCOwF5JnjdgnVVVNVlVkxMTE6POKUkbrXmd9VFVdwEXAAeNJY0k6WGGOetjIsnW3f3NgQOB7487mCSpZ5izPp4OnJZkE3rFflZVnTfeWJKkacOc9fFd4PkLkEWSNIDvTJSkxlnUktQ4i1qSGmdRS1LjLGpJapxFLUmNs6glqXEWtSQ1zqKWpMZZ1JLUOItakhpnUUtS4yxqSWqcRS1JjbOoJalxFrUkNc6ilqTGDXPNxGck+XqSq5JckeStCxFMktQzzDUT7wPeUVUXJ9kKWJPkq1V15ZizSZIY4oi6qm6pqou7+z8HrgJ2GHcwSVLPvOaok6ygd6HbiwYsW5lkdZLVU1NTo0knSRq+qJM8Efgc8Laq+tnM5VW1qqomq2pyYmJilBklaaM2VFEneTy9kv50VZ0z3kiSpH7DnPUR4OPAVVX1j+OPJEnqN8wR9b7Aa4H9k1za3V425lySpM6cp+dV1YVAFiCLJGkA35koSY2zqCWpcRa1JDXOopakxlnUktQ4i1qSGmdRS1LjLGpJapxFLUmNs6glqXEWtSQ1zqKWpMZZ1JLUOItakhpnUUtS4yxqSWqcRS1JjRvmmomfSHJ7kssXIpAk6aGGOaI+FThozDkkSbOYs6ir6hvAnQuQRZI0wJwXtx1WkpXASoDly5c/4u2sOPaLo4q0ZKw98ZBF2e/GONZaOP58jc7IXkysqlVVNVlVkxMTE6ParCRt9DzrQ5IaZ1FLUuOGOT3vDOBbwC5J1iV50/hjSZKmzfliYlUdvhBBJEmDOfUhSY2zqCWpcRa1JDXOopakxlnUktQ4i1qSGmdRS1LjLGpJapxFLUmNs6glqXEWtSQ1zqKWpMZZ1JLUOItakhpnUUtS4yxqSWqcRS1JjRuqqJMclOTqJNckOXbcoSRJDxrmmombAB8DDgZ2BQ5Psuu4g0mSeoY5ot4LuKaqrquqe4AzgZePN5YkadqcF7cFdgBu7Hu8Dvi9mSslWQms7B6uT3J1d3874I5HE3KRjT1//n6cW3f8F5n5F9eC5n+Uv8vPnG3BMEWdAc/Vw56oWgWsetg/TlZX1eQQ+2mS+ReX+ReX+dswzNTHOuAZfY93BG4eTxxJ0kzDFPX/Ab+dZKckmwKHAV8YbyxJ0rQ5pz6q6r4kbwH+E9gE+ERVXTGPfTxsOmSJMf/iMv/iMn8DUvWw6WZJUkN8Z6IkNc6ilqTGjbyok6xN8r0klyZZ3T13fJKbuucuTfKyUe93VJJsneTsJN9PclWSfZJsm+SrSX7Yfd1msXPOZpb8S2L8k+zSl/HSJD9L8ralMv4byL8kxh8gyduTXJHk8iRnJHlCdyLBRd34f6Y7qaBJs+Q/Ncn1feO/+2LnnK+Rz1EnWQtMVtUdfc8dD6yvqg+NdGdjkOQ04JtVdXL3A7kF8G7gzqo6sfusk22q6p2LGnQWs+R/G0tk/Kd1H11wE703Vx3NEhn/aTPyv4ElMP5JdgAuBHatql8mOQv4EvAy4JyqOjPJvwKXVdVJi5l1kA3kfzFwXlWdvZj5Hg2nPvokeRKwH/BxgKq6p6ruoveW+dO61U4DXrE4CTdsA/mXogOAa6vqRyyR8Z+hP/9SsgzYPMkyev/J3wLsD0yXXOvjPzP/Y+I9H+Mo6gLOT7Kme1v5tLck+W6ST7T6pyuwMzAFnJLkkiQnJ9kSeGpV3QLQfX3KYobcgNnyw9IY/36HAWd095fK+Pfrzw9LYPyr6ibgQ8AN9Ar6p8Aa4K6quq9bbR29j5VozqD8VXV+t/iEbvz/KclmixbyERpHUe9bVXvQ+7S9o5PsB5wEPAvYnd4A/sMY9jsKy4A9gJOq6vnA3cBS+ljX2fIvlfEHoJuyORT47GJneSQG5F8S49/9B/JyYCdge2BLer/HMzV5Tu+g/EmOBN4FPAfYE9gWaHrabJCRF3VV3dx9vR04F9irqm6rqvur6jfAv9H7RL4WrQPWVdVF3eOz6RXfbUmeDtB9vX2R8s1lYP4lNP7TDgYurqrbusdLZfynPST/Ehr/A4Hrq2qqqu4FzgFeAGzdTSVA2x8hMTB/Vd1SPb8GTqHd8Z/VSIs6yZZJtpq+D7wEuHz6l6zzx8Dlo9zvqFTVrcCNSXbpnjoAuJLeW+aP6p47Cvj8IsSb02z5l8r49zmch04bLInx7/OQ/Eto/G8A9k6yRZLw4M//14FXduu0PP6D8l/V95986M2vtzr+sxrpWR9JdqZ3FA29P8NPr6oTknyS3p99BawF/nx6zrE13ak7JwObAtfRe8X+ccBZwHJ6Pwyvqqo7Fy3kBsyS/19YOuO/Bb2P1d25qn7aPfdkls74D8q/lH7+3we8BrgPuAR4M7056TPpTRtcAhzZHZ02Z5b8XwYm6H0S6KXAX1TV+kUL+Qj4FnJJapyn50lS4yxqSWqcRS1JjbOoJalxFrUkNc6ilqTGWdSS1Lj/B6jEWyx5aYLdAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "count    32.000000\n",
      "mean     75.150000\n",
      "std       6.958077\n",
      "min      56.200000\n",
      "25%      70.875000\n",
      "50%      75.450000\n",
      "75%      78.500000\n",
      "max      87.900000\n",
      "Name: AGE, dtype: float64\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAEICAYAAAB25L6yAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAASyElEQVR4nO3df5RkZX3n8ffHGVAMGFSaaIBOa1Y5cuKKbktUokYwWRAlyQmukPgrP3ZyjCRi4rpgfojxmGV3ozHZeLJndkM0/kBZAyaKMSQbWX8cQWcQFTKggoMCKoMuQZQEwW/+uLelbGq6bvd0TT30vF/n1Om6dZ+69/tM9Xz6qefeqpuqQpLUrvvNugBJ0soMaklqnEEtSY0zqCWpcQa1JDXOoJakxhnU2pCSPDXJNeu4vb9J8qL+/ouTfGQdt/3zSS5er+1p4zGoNVaSnUnuTHLIssevSFJJFvrlN/fLJy9r98b+8Rf3y/sneX2SG5LcnuQLSf5w2f7u6Nct3f5kN7WdneTbSb7R3z6b5E+SPHypTVV9uKqOHNDPs5O8bVK7qjqxqt4yqd2A/S30/y6bR7b99qr6yT3dtjYug1or+QJw2tJCkscCB4xp91ngRSPtNgPPBa4daXMWsAgcAxwEPAP45LLtPKeqDhy5nb5Cbe+qqoOAhwA/AzwM2D4a1ushHf+faKb8BdRK3gq8cGT5RcBfjGn3XuDYJA/ul08APg18ZaTNE4ELq+qm6uysqnHbWpWq+nZVXQU8D9gF/CZAkh9PcsNSuyT/OcmN/Qj8miTHJzkBeBXwvH4E/6m+7SVJXpfko8C3gEf2j/3yyK6T5H8k+ackVyc5fmTFziTPHFkeHbV/qP95a7/PJy+fSknylCSf6Lf9iSRPGVl3SZLXJvlo35eLl7/r0cZjUGsllwIPSvKYJJvownDcNME/A38NnNovv5B7B/qlwG8k+dUkj02S9Sy0qu4G/gp46vJ1SY4ETgee2I/C/z2ws6o+APw+3ej8wKp63MjTXgBsoRv9Xz9mlz8KXAccArwauCDJQwaU+rT+58H9Pj+2rNaHABcBfww8FHgDcFGSh440+zngF4BDgf2BVwzYr+7DDGpNsjSq/gngauDG3bT7C+CFSb4feDrwnmXr/wvwX4GfB7YBNy4dnBvxniS3jtz+4yprvYluKmS5u4H7A0cl2a8fzV87pt2oN1fVVVV1V1V9e8z6m4E39iP6dwHXACetst5xTgI+V1Vv7fd9Ht2/+3NG2vx5VX22qu4AzgeOXof9qmEGtSZ5K90I7sWMn/YAoKo+AswBvw28rw+R0fV3V9WbqupY4GDgdcC5SR4z0uynq+rgkdv/WmWthwFfH1Pb54EzgLOBm5O8M8kPTtjWlyasv7G+9xvNrgcmbXOIH+TeI/jr6fq2ZHRK6VvAgeuwXzXMoNaKqup6uoOKzwIumND8bXRzxCvOPVfVHVX1JuD/A0etR539Ab/nAB/ezT7fUVU/BvwQUHSje/r7Y58yYZeHLZu+macb0QN8E3jgyLqHrWK7N/U1jppn9+9ktA8wqDXELwHHVdU3J7T7Y7opkg8tX5HkjP4A3wFJNvfTHgdx7zM/ViXJfv2o/Dy6QHzDmDZHJjkuyf3p5tPvoJsOAfgqsLCGMzsOBX693/9zgccA7+/XXQGc2q9bBE4Zed4u4DvAI3ez3fcDj07yc/2/0/Po/pi9b5X1aQMxqDVRVV1bVdsGtPt6Vf3fZVMCS+4AXk/3tv0W4KXAz1bVdSNt3rvsPOoLV9jd85LcDtxKdyDza8C/q6qbxrS9P3BOv9+v0IXsq/p1/6f/+bUkl0/q44jLgEf123wdcEpVfa1f9zvAD9O9Y3gN8I6lJ1XVt/r2H+3n4Z80utF+G8+me2fyNeCVwLOr6pZV1KYNJl44QJLa5ohakhpnUEtS4wxqSWqcQS1Jjds8ucnqHXLIIbWwsDCNTUvShrR9+/Zbqmpu3LqpBPXCwgLbtk08m0uS1Esy7jtlAKc+JKl5BrUkNc6glqTGGdSS1DiDWpIaZ1BLUuMGBXWSlye5KsmVSc5L8oBpFyZJ6kwM6iSHAb8OLFbVjwCbuOfaeJKkKRs69bEZOCDJZrorV4z7zl9J0hRM/GRiVd2Y5A+AL9J9+fvFVXXx8nZJttBdtZn5+fk1F7Rw5kVrfu6e2HnOelyXVJLW35CpjwcDPwU8gu7Cm9+X5PnL21XV1qparKrFubmxH1eXJK3BkKmPZwJfqKpdVfVtugucPmW6ZUmSlgwJ6i8CT0rywP6qy8cDO6ZbliRpycSgrqrLgHcDlwOf6Z+zdcp1SZJ6g77mtKpeDbx6yrVIksbwk4mS1DiDWpIaZ1BLUuMMaklqnEEtSY0zqCWpcQa1JDXOoJakxhnUktQ4g1qSGmdQS1LjDGpJapxBLUmNM6glqXEGtSQ1zqCWpMYNubjtkUmuGLndluSMvVGcJGnAFV6q6hrgaIAkm4AbgQunXJckqbfaqY/jgWur6vppFCNJurfVBvWpwHnTKESSNN6gi9sCJNkfOBk4azfrtwBbAObn59elOElai4UzL5rJfneec9JUtruaEfWJwOVV9dVxK6tqa1UtVtXi3Nzc+lQnSVpVUJ+G0x6StNcNCuokDwR+ArhguuVIkpYbNEddVd8CHjrlWiRJY/jJRElqnEEtSY0zqCWpcQa1JDXOoJakxhnUktQ4g1qSGmdQS1LjDGpJapxBLUmNM6glqXEGtSQ1zqCWpMYZ1JLUOINakhpnUEtS4wxqSWrc0EtxHZzk3UmuTrIjyZOnXZgkqTPoUlzAHwEfqKpTkuwPPHCKNUmSRkwM6iQPAp4GvBigqu4E7pxuWZKkJUNG1I8EdgF/nuRxwHbgZVX1zdFGSbYAWwDm5+fXu07pPm3hzItmtu+d55w0k/3Oss8bzZA56s3AE4A/rarHA98EzlzeqKq2VtViVS3Ozc2tc5mStO8aEtQ3ADdU1WX98rvpgluStBdMDOqq+grwpSRH9g8dD/zjVKuSJH3X0LM+fg14e3/Gx3XAL0yvJEnSqEFBXVVXAItTrkWSNIafTJSkxhnUktQ4g1qSGmdQS1LjDGpJapxBLUmNM6glqXEGtSQ1zqCWpMYZ1JLUOINakhpnUEtS4wxqSWqcQS1JjTOoJalxBrUkNc6glqTGDbrCS5KdwDeAu4G7qsqrvUjSXjL0mokAz6iqW6ZWiSRpLKc+JKlxQ4O6gIuTbE+yZVyDJFuSbEuybdeuXetXoSTt44YG9bFV9QTgROClSZ62vEFVba2qxapanJubW9ciJWlfNiioq+qm/ufNwIXAMdMsSpJ0j4lBneT7khy0dB/4SeDKaRcmSeoMOevjB4ALkyy1f0dVfWCqVUmSvmtiUFfVdcDj9kItkqQxPD1PkhpnUEtS4wxqSWqcQS1JjTOoJalxBrUkNc6glqTGGdSS1DiDWpIaZ1BLUuMMaklqnEEtSY0zqCWpcQa1JDXOoJakxhnUktQ4g1qSGjc4qJNsSvLJJO+bZkGSpO+1mhH1y4Ad0ypEkjTeoKBOcjhwEvC/p1uOJGm5oSPqNwKvBL6zuwZJtiTZlmTbrl271qU4SdKAoE7ybODmqtq+Uruq2lpVi1W1ODc3t24FStK+bsiI+ljg5CQ7gXcCxyV521SrkiR918SgrqqzqurwqloATgX+oaqeP/XKJEmA51FLUvM2r6ZxVV0CXDKVSiRJYzmilqTGGdSS1DiDWpIaZ1BLUuMMaklqnEEtSY0zqCWpcQa1JDXOoJakxhnUktQ4g1qSGmdQS1LjDGpJapxBLUmNM6glqXEGtSQ1zqCWpMYNuQr5A5J8PMmnklyV5DV7ozBJUmfIpbj+BTiuqm5Psh/wkSR/U1WXTrk2SRIDgrqqCri9X9yvv9U0i5Ik3WPQxW2TbAK2A/8GeFNVXTamzRZgC8D8/Px61qgpWTjzopnte+c5J81s39J9zaCDiVV1d1UdDRwOHJPkR8a02VpVi1W1ODc3t951StI+a1VnfVTVrcAlwAlTqUaSdC9DzvqYS3Jwf/8A4JnA1dMuTJLUGTJH/XDgLf089f2A86vqfdMtS5K0ZMhZH58GHr8XapEkjeEnEyWpcQa1JDXOoJakxhnUktQ4g1qSGmdQS1LjDGpJapxBLUmNM6glqXEGtSQ1zqCWpMYZ1JLUOINakhpnUEtS4wxqSWqcQS1JjTOoJalxQ66ZeESSDybZkeSqJC/bG4VJkjpDrpl4F/CbVXV5koOA7Un+rqr+ccq1SZIYMKKuqi9X1eX9/W8AO4DDpl2YJKmzqjnqJAt0F7q9bMy6LUm2Jdm2a9eu9alOkjQ8qJMcCPwlcEZV3bZ8fVVtrarFqlqcm5tbzxolaZ82KKiT7EcX0m+vqgumW5IkadSQsz4C/Bmwo6reMP2SJEmjhoyojwVeAByX5Ir+9qwp1yVJ6k08Pa+qPgJkL9QiSRrDTyZKUuMMaklqnEEtSY0zqCWpcQa1JDXOoJakxhnUktQ4g1qSGmdQS1LjDGpJapxBLUmNM6glqXEGtSQ1zqCWpMYZ1JLUOINakhpnUEtS44ZcM/HcJDcnuXJvFCRJ+l5DRtRvBk6Ych2SpN2YGNRV9SHg63uhFknSGBMvbjtUki3AFoD5+fn12uxes3DmRTPb985zTprZvvc1s3ydpbVat4OJVbW1qharanFubm69NitJ+zzP+pCkxhnUktS4IafnnQd8DDgyyQ1Jfmn6ZUmSlkw8mFhVp+2NQiRJ4zn1IUmNM6glqXEGtSQ1zqCWpMYZ1JLUOINakhpnUEtS4wxqSWqcQS1JjTOoJalxBrUkNc6glqTGGdSS1DiDWpIaZ1BLUuMMaklqnEEtSY0bFNRJTkhyTZLPJzlz2kVJku4x5JqJm4A3AScCRwGnJTlq2oVJkjpDRtTHAJ+vquuq6k7gncBPTbcsSdKSVNXKDZJTgBOq6pf75RcAP1pVpy9rtwXY0i8eCVyzxpoOAW5Z43Nbs1H6slH6AfalRRulH7Bnffmhqpobt2LiVciBjHnsXuleVVuBrass7N47S7ZV1eKebqcFG6UvG6UfYF9atFH6AdPry5CpjxuAI0aWDwduWu9CJEnjDQnqTwCPSvKIJPsDpwJ/Pd2yJElLJk59VNVdSU4H/hbYBJxbVVdNsaY9nj5pyEbpy0bpB9iXFm2UfsCU+jLxYKIkabb8ZKIkNc6glqTGzSyokxyR5INJdiS5KsnLlq1/RZJKcsisahxqd31JcnaSG5Nc0d+eNetaJ1npdUnya/1XCVyV5L/Nss5JVnhN3jXyeuxMcsWsa51khb4cneTSvi/bkhwz61onWaEvj0vysSSfSfLeJA+ada2TJHlAko8n+VTfl9f0jz8iyWVJPtf/vu2/xzurqpncgIcDT+jvHwR8FjiqXz6C7uDl9cAhs6pxT/sCnA28Ytb1rVNfngH8PXD/ft2hs651rb9fI21eD/zurGvdg9fkYuDE/vFnAZfMutY96MsngKf3j/8i8NpZ1zqgLwEO7O/vB1wGPAk4Hzi1f/x/Ai/Z033NbERdVV+uqsv7+98AdgCH9av/EHglYz5Y06IJfblPWaEvLwHOqap/6dfdPLsqJ5v0miQJ8B+A82ZT4XAr9KWApZHn93Mf+HzDCn05EvhQ3+zvgJ+dTYXDVef2fnG//lbAccC7+8ffAvz0nu6riTnqJAvA44HLkpwM3FhVn5ppUWs02pf+odOTfDrJuUkePLPC1mBZXx4NPLV/S/f/kjxxlrWtxpjXBOCpwFer6nOzqGmtlvXlDOC/J/kS8AfAWbOrbPWW9eVK4OR+1XP53g/ZNSvJpn767Ga6PzDXArdW1V19kxtYh0HbzIM6yYHAX9L90t0F/BbwuzMtao1G+1JVtwF/CvwwcDTwZbq32vcJY/qyGXgw3Vu7/wSc349KmzamH0tO4z4wmh41pi8vAV5eVUcALwf+bJb1rcaYvvwi8NIk2+mmRO6cZX1DVdXdVXU03Se2jwEeM67ZeuxolnM8+9HNRf9Gv/xYur9MO/vbXcAXgYfNej5qtX0Zs34BuHLWda61L8AHgB8fWb4WmJt1rWt5Tej+6HwVOHzWNe7ha/JP3PNZiAC3zbrOPXldRtY/Gvj4rOtcQ79eTTeIuQXY3D/2ZOBv93TbszzrI3QjgB1V9QaAqvpMVR1aVQtVtUD3tuEJVfWVWdU5xLi+9I8/fKTZz9C9vWva7voCvIdu7o0kjwb2p+FvPFuhHwDPBK6uqhv2fmWrt0JfbgKe3t8/Dmh+GmeF/yuH9j/vB/w23UG4piWZS3Jwf/8Aut+rHcAHgVP6Zi8C/mqP99Wn/l6X5MeADwOfAb7TP/yqqnr/SJudwGJVNRsIsPu+0L29Pprurc9O4Feq6suzqHGoFfry98C5dP25k+5sln+YSZEDrPT7leTNwKVV1XwYwIqvyW3AH9G9Q/hn4FeravtMihxohb48Cnhpv3wBcFbNKpwGSvJv6Q4WbqKbRj6/qn4vySPpvrf/IcAngedXfxB+zftq/N9CkvZ5Mz+YKElamUEtSY0zqCWpcQa1JDXOoJakxhnUktQ4g1qSGvevq/CLwkmULLgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "count    32.000000\n",
      "mean     27.312500\n",
      "std       2.334005\n",
      "min      24.000000\n",
      "25%      24.750000\n",
      "50%      28.000000\n",
      "75%      29.000000\n",
      "max      30.000000\n",
      "Name: MMSE, dtype: float64\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEICAYAAABGaK+TAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAASDElEQVR4nO3debBkZX3G8e8jI1E2QWdwGRnGBU3UKJpBcUMDklIgolVagVIDampKo4LEqKNYUf/AAjUoLjE1EYUExAVxjwrlGo2iwyIwjgZFlmEdQ1DABdBf/jjnQnPnbnO77/R94fup6uo+p0+f80zX7WfOeU8vqSokSe25x7gDSJLmxwKXpEZZ4JLUKAtckhplgUtSoyxwSWqUBS7NUZL1SZ45onW9KMmZA9OV5OGjWHe/vpuSPHRU69PiZIHfTSW5NMlv+xf6tUk+mmSHvqRu6i9/SPK7gek3Jzm8n3/TpMuDJq33xiQ3JPnvJK9Ico+BbX8zyd9NyvPMJBsHppPkiCQXJbk5ycYkn0ry55Me97a+/J44MO/NA7l+Nynv+imei5X9OiaWuTbJF5PsP7hcVT26qr45y/M6sa4lMy1XVadW1V/NtMxcTfV8VtUOVXXJKNavxcsCv3v766raAXgCsBfwlr6kdujn/xfw6onpqnpH/7jvDcybuFw1ab07ArsDxwJvBE7cwmwnAEcCRwD3BR4BfBY4cGKBJAFeAlwPHDYxv6reMfBveMWkvI+eYZs79495HHAW8Jkkh29h7lnNVu7SXFngoqquBL4MPGbE6/1VVX0e+BvgsCRzWn+SPYBXAYdW1der6vdV9Zt+r/XYgUWfDjyIrugPSbLtiHJfU1UnAG8Djps4euiPLp7V335iknVJft3vsR/fP/zb/fUN/d78k/ujlu8meU+S64G39fO+M2nTByS5JMkvk7xrYLtvS3LKwPNz+15+kmP65+ED/fY+0C9z+5BMkvsk+fckm5JcluQtA+s+PMl3krw7yf8l+UWS54ziedTCs8BFkt2AA4DzFmL9VfUDYCNd0czFfsDG/nEzOQz4AvCJfvqg+SWc1hnArsAjp7jvBOCEqtoJeBjwyX7+Pv31zv0e//f66ScBl/TrO2aa7T0fWEV3RHQw8LLZAlbV0dz5SOnVUyz2fuA+wEOBZwB/C7x04P4nAT8FlgLvBE7sj260yFngd2+fTXID8B3gW8A7Zll+wt79+PbE5edzeMxVdEMhc3E/4OqZFkiyHfBC4GNVdStwOgPDKCMyMSw0Ve5bgYcnWVpVN1XV92dbV1W9v6puq6rfTrPMcVV1fVVdDrwXOHSeuW+XZBu6I6A3VdWNVXUp8M90Q08TLquqf6uqPwAnAw8E7j/strXwLPC7t+dV1c5VtXtV/f0MxTLZ9/vHTVweNofHLKcbqwa4DbjnpPvvSVeKAP9LVyIzeX6/nv/sp08FnpNk2RyyzNXy/vr6Ke57Od24/E+S/DDJbHv/V8xhe4PLXEY3PDSspcC2/foG1718YPqaiRtV9Zv+5g4j2LYWmAWuBZdkL7rCmBjzvRxYOWmxh3BHyXwNeHCSVTOs9jC6krk8yTXAp+j+Exh6r3XA84Hr6IYX7qSqLq6qQ+mGRI4DTk+yPTDd13vO5Ws/dxu4vYI7jgBuBrYbuO8BW7DuX9L9x7j7pHVfOYc8WuQscC2YJDv1e6YfB06pqgv7uz4BvLQ/EZgkjwCO6pejqi4G/gU4rX974bZJ7pXkkCRrkiynGyc/CNizvzyOrkiHHkZJcv8krwbeSjf08McplnlxkmX9fTf0s/8AbAL+SDfevKVen2SX/pzEkdwxtn8+sE+SFUnuA7xp0uOunW57/bDIJ4FjkuyYZHfgH4BTplpebbHANR9PzubvA99r4P4vJLmRbkjgaOB4Bk6aVdVXgTXAR4Ff0Q2DnAysHVjHEcAHgA/SFeTP6faIv0A3fnt+VZ3Zv2Pkmqq6Bngf8Ni5vttlCjckuRm4kO6k7gur6iPTLPtsYH2Sm+hOaB5SVb/rhyCOAb7bnx/Yewu2/zngHLrC/hL9Wy+r6iy6Mr+gv/+Lkx53AvCC/l0k75tiva+h24u/hO4o6GPAdP8uNST+oIMktck9cElqlAUuSY2ywCWpURa4JDVqq36pztKlS2vlypVbc5OS1Lxzzjnnl1W12YfUtmqBr1y5knXr1m3NTUpS85JcNtV8h1AkqVEWuCQ1ygKXpEZZ4JLUKAtckhplgUtSo2Yt8CQfSXJdkosG5t03yVlJLu6vd1nYmJKkyeayB34S3VdnDloDfK2q9qD78v01I84lSZrFrAVeVd9m85+UOpju+5vpr5834lySpFnM95OY96+qqwGq6uoku063YJLVwGqAFStWzHNzku5KVq750li2e+mxB45luwtlwU9iVtXaqlpVVauWLRvl781K0t3bfAv82iQPBOivrxtdJEnSXMy3wD/PHT8eexjdb/lJkraiubyN8DTge8Ajk2xM8nLgWGD/JBcD+/fTkqStaNaTmFV16DR37TfiLJKkLeAnMSWpURa4JDXKApekRlngktQoC1ySGmWBS1KjLHBJapQFLkmNssAlqVEWuCQ1ygKXpEZZ4JLUKAtckhplgUtSoyxwSWqUBS5JjbLAJalRFrgkNcoCl6RGWeCS1CgLXJIaZYFLUqMscElqlAUuSY2ywCWpURa4JDXKApekRlngktQoC1ySGmWBS1KjLHBJapQFLkmNGqrAkxyVZH2Si5KcluReowomSZrZvAs8yXLgCGBVVT0G2AY4ZFTBJEkzG3YIZQlw7yRLgO2Aq4aPJEmaiyXzfWBVXZnk3cDlwG+BM6vqzMnLJVkNrAZYsWLFfDcnacRWrvnSuCNoSMMMoewCHAw8BHgQsH2SF09erqrWVtWqqlq1bNmy+SeVJN3JMEMozwJ+UVWbqupW4AzgKaOJJUmazTAFfjmwd5LtkgTYD9gwmliSpNnMu8Cr6mzgdOBc4MJ+XWtHlEuSNIt5n8QEqKq3Am8dURZJ0hbwk5iS1CgLXJIaZYFLUqMscElqlAUuSY2ywCWpURa4JDXKApekRlngktQoC1ySGmWBS1KjLHBJapQFLkmNssAlqVEWuCQ1ygKXpEZZ4JLUKAtckhplgUtSoyxwSWqUBS5JjbLAJalRFrgkNcoCl6RGWeCS1CgLXJIaZYFLUqMscElqlAUuSY2ywCWpURa4JDXKApekRg1V4El2TnJ6kp8k2ZDkyaMKJkma2ZIhH38C8JWqekGSbYHtRpBJkjQH8y7wJDsB+wCHA1TVLcAto4klSZrNMEMoDwU2AR9Ncl6SDyfZfkS5JEmzGKbAlwBPAD5UVY8HbgbWTF4oyeok65Ks27Rp0xCbkyQNGqbANwIbq+rsfvp0ukK/k6paW1WrqmrVsmXLhticJGnQvAu8qq4BrkjyyH7WfsCPR5JKkjSrYd+F8hrg1P4dKJcALx0+kiRpLoYq8Ko6H1g1oiySpC3gJzElqVEWuCQ1ygKXpEZZ4JLUKAtckhplgUtSoyxwSWqUBS5JjbLAJalRFrgkNcoCl6RGWeCS1CgLXJIaZYFLUqMscElqlAUuSY2ywCWpURa4JDXKApekRlngktQoC1ySGmWBS1KjLHBJapQFLkmNssAlqVEWuCQ1ygKXpEZZ4JLUKAtckhplgUtSoyxwSWqUBS5JjbLAJalRQxd4km2SnJfki6MIJEmam1HsgR8JbBjBeiRJW2CoAk/yYOBA4MOjiSNJmqslQz7+vcAbgB2nWyDJamA1wIoVK4bcnLQwVq750ti2femxB45t22rbvPfAkxwEXFdV58y0XFWtrapVVbVq2bJl892cJGmSYYZQngo8N8mlwMeBfZOcMpJUkqRZzbvAq+pNVfXgqloJHAJ8vapePLJkkqQZ+T5wSWrUsCcxAaiqbwLfHMW6JElz4x64JDXKApekRlngktQoC1ySGmWBS1KjLHBJapQFLkmNssAlqVEWuCQ1ygKXpEZZ4JLUKAtckhplgUtSoyxwSWqUBS5JjRrJ94FLozLOHxfWXd+4/r4W6oer3QOXpEZZ4JLUKAtckhplgUtSoyxwSWqUBS5JjbLAJalRFrgkNcoCl6RGWeCS1CgLXJIaZYFLUqMscElqlAUuSY2ywCWpURa4JDVq3gWeZLck30iyIcn6JEeOMpgkaWbD/CLPbcDrqurcJDsC5yQ5q6p+PKJskqQZzHsPvKqurqpz+9s3AhuA5aMKJkma2Uh+EzPJSuDxwNlT3LcaWA2wYsWKUWxOC8zfpZTaMPRJzCQ7AJ8GXltVv558f1WtrapVVbVq2bJlw25OktQbqsCT3JOuvE+tqjNGE0mSNBfDvAslwInAhqo6fnSRJElzMcwe+FOBlwD7Jjm/vxwwolySpFnM+yRmVX0HyAizSJK2gJ/ElKRGWeCS1CgLXJIaZYFLUqMscElqlAUuSY2ywCWpURa4JDXKApekRlngktQoC1ySGmWBS1KjLHBJapQFLkmNssAlqVEj+VHjrWFcP7R76bEHjmW74I8LS5qZe+CS1CgLXJIaZYFLUqMscElqlAUuSY2ywCWpURa4JDXKApekRlngktQoC1ySGmWBS1KjLHBJapQFLkmNssAlqVEWuCQ1ygKXpEZZ4JLUqKEKPMmzk/w0yc+SrBlVKEnS7OZd4Em2AT4IPAd4FHBokkeNKpgkaWbD7IE/EfhZVV1SVbcAHwcOHk0sSdJshvlR4+XAFQPTG4EnTV4oyWpgdT95U5KfznN7S4FfzvOx85bj5v3QseSdJ7MunFnzDvE3Nmp3ued2schxQ2fdfaqZwxR4pphXm82oWgusHWI73caSdVW1atj1bC0t5TXrwmkpb0tZoa28C5V1mCGUjcBuA9MPBq4aLo4kaa6GKfAfAnskeUiSbYFDgM+PJpYkaTbzHkKpqtuSvBr4KrAN8JGqWj+yZJsbehhmK2spr1kXTkt5W8oKbeVdkKyp2mzYWpLUAD+JKUmNssAlqVFNFHiSnZOcnuQnSTYkefK4M00nyVFJ1ie5KMlpSe417kyDknwkyXVJLhqYd98kZyW5uL/eZZwZJ0yT9V3938EFST6TZOdxZhw0Vd6B+/4xSSVZOo5sk02XNclr+q/HWJ/knePKN9k0fwt7Jvl+kvOTrEvyxHFmnJBktyTf6LtqfZIj+/kjf501UeDACcBXqupPgccBG8acZ0pJlgNHAKuq6jF0J3cPGW+qzZwEPHvSvDXA16pqD+Br/fRicBKbZz0LeExVPRb4H+BNWzvUDE5i87wk2Q3YH7h8aweawUlMyprkL+k+Tf3Yqno08O4x5JrOSWz+3L4TeHtV7Qn8Uz+9GNwGvK6q/gzYG3hV/zUjI3+dLfoCT7ITsA9wIkBV3VJVN4w31YyWAPdOsgTYjkX23viq+jZw/aTZBwMn97dPBp63VUNNY6qsVXVmVd3WT36f7vMHi8I0zy3Ae4A3MMUH3cZlmqyvBI6tqt/3y1y31YNNY5q8BezU374Pi+S1VlVXV9W5/e0b6XY4l7MAr7NFX+DAQ4FNwEeTnJfkw0m2H3eoqVTVlXR7LZcDVwO/qqozx5tqTu5fVVdD98cH7DrmPHP1MuDL4w4xkyTPBa6sqh+NO8scPAJ4epKzk3wryV7jDjSL1wLvSnIF3etuMR2NAZBkJfB44GwW4HXWQoEvAZ4AfKiqHg/czOI5xL+TfkzrYOAhwIOA7ZO8eLyp7pqSHE13qHrquLNMJ8l2wNF0h/ctWALsQnfY/3rgk0mm+sqMxeKVwFFVtRtwFP1R+mKRZAfg08Brq+rXC7GNFgp8I7Cxqs7up0+nK/TF6FnAL6pqU1XdCpwBPGXMmebi2iQPBOivF82h81SSHAYcBLyoFvcHGR5G95/5j5JcSjfcc26SB4w11fQ2AmdU5wfAH+m+MGqxOozuNQbwKbpvSF0UktyTrrxPraqJjCN/nS36Aq+qa4Arkjyyn7Uf8OMxRprJ5cDeSbbr91z2Y5GecJ3k83QvBvrrz40xy4ySPBt4I/DcqvrNuPPMpKourKpdq2plVa2kK8gn9H/Ti9FngX0BkjwC2JbF/W1/VwHP6G/vC1w8xiy361/7JwIbqur4gbtG/zqrqkV/AfYE1gEX0P2R7TLuTDNkfTvwE+Ai4D+APxl3pkn5TqMbn7+VrlBeDtyP7qz4xf31fcedc4asP6P7GuPz+8u/jjvnTHkn3X8psHTcOWd4brcFTun/ds8F9h13zlnyPg04B/gR3RjzX4w7Z5/1aXQnWC8Y+Ds9YCFeZ36UXpIateiHUCRJU7PAJalRFrgkNcoCl6RGWeCS1CgLXJIaZYFLUqP+H1n/atbaYAuFAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "count    32.000000\n",
      "mean     15.531250\n",
      "std       3.321333\n",
      "min       6.000000\n",
      "25%      13.000000\n",
      "50%      16.000000\n",
      "75%      18.000000\n",
      "max      20.000000\n",
      "Name: PTEDUCAT, dtype: float64\n"
     ]
    }
   ],
   "source": [
    "# DRY unless you're a data scientist\n",
    "for var in ['AGE', 'MMSE', 'PTEDUCAT']:\n",
    "    plt.hist(X_misclassified[var], bins=10)\n",
    "    plt.title(var + ' Distribution')\n",
    "    plt.show()\n",
    "    \n",
    "    # descriptive stats\n",
    "    print(X_misclassified[var].describe())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TODO\n",
    "* Evaluate probability predictions from log regr model\n",
    "* Go back and see if we can remove outliers\n",
    "* Switch to binary classificiation (so map CN, LMCI => Not AD) so we have classes (Not AD, AD) and use `AUC` as metric\n",
    "* Or lower log regr. threshold (e.g. .5 => .4) to minimize # of False Negatives"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}