P5-example.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# Решающие деревья"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "###  Загрузка данных\n",
    "\n",
    "Для улучшения качества классификации, устраним пропуски и прошкалируем данные с использованием тестового набора."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 164,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "train = pd.read_csv('titanic.train.csv')    \n",
    "test = pd.read_csv('titanic.test.csv')\n",
    "targets = train.Survived\n",
    "train.drop('Survived', 1, inplace=True)\n",
    "\n",
    "combined = train.append(test)\n",
    "combined.reset_index(inplace=True)\n",
    "combined.drop('index',inplace=True,axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 165,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Pclass                                               Name  \\\n",
       "0            1       3                            Braund, Mr. Owen Harris   \n",
       "1            2       1  Cumings, Mrs. John Bradley (Florence Briggs Th...   \n",
       "2            3       3                             Heikkinen, Miss. Laina   \n",
       "3            4       1       Futrelle, Mrs. Jacques Heath (Lily May Peel)   \n",
       "4            5       3                           Allen, Mr. William Henry   \n",
       "\n",
       "      Sex   Age  SibSp  Parch            Ticket     Fare Cabin Embarked  \n",
       "0    male  22.0      1      0         A/5 21171   7.2500   NaN        S  \n",
       "1  female  38.0      1      0          PC 17599  71.2833   C85        C  \n",
       "2  female  26.0      0      0  STON/O2. 3101282   7.9250   NaN        S  \n",
       "3  female  35.0      1      0            113803  53.1000  C123        S  \n",
       "4    male  35.0      0      0            373450   8.0500   NaN        S  "
      ]
     },
     "execution_count": 165,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "combined.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Извлечение имен пассажиров\n",
    "\n",
    "Рассматривая имена пассажиров, можно было бы задаться вопросом, как их обработать, чтобы извлечь легко интерпретируемую информацию.\n",
    "\n",
    "Посмотрим более внимательно на примеры:\n",
    "\n",
    "- Braund, <b> Mr.</b> Owen Harris\t\n",
    "- Heikkinen, <b>Miss.</b> Laina\n",
    "- Oliva y Ocana, <b>Dona.</b> Fermina\n",
    "- Peter, <b>Master.</b> Michael J\n",
    "\n",
    "Можно заметить, что у каждого имени есть приставка! Это может быть простая мисс или миссис, но иногда это может быть что-то более сложное, как Мастер, Сэр или Дона. В этом случае мы могли бы ввести дополнительную информацию о социальном статусе, просто проанализировав имя и извлекая заголовок."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 166,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "combined['Title'] = combined['Name'].map(lambda name:name.split(',')[1].split('.')[0].strip())\n",
    "\n",
    "titles = {\n",
    "    \"Capt\":       \"Officer\",\n",
    "    \"Col\":        \"Officer\",\n",
    "    \"Major\":      \"Officer\",\n",
    "    \"Jonkheer\":   \"Royalty\",\n",
    "    \"Don\":        \"Royalty\",\n",
    "    \"Sir\" :       \"Royalty\",\n",
    "    \"Dr\":         \"Officer\",\n",
    "    \"Rev\":        \"Officer\",\n",
    "    \"the Countess\":\"Royalty\",\n",
    "    \"Dona\":       \"Royalty\",\n",
    "    \"Mme\":        \"Mrs\",\n",
    "    \"Mlle\":       \"Miss\",\n",
    "    \"Ms\":         \"Mrs\",\n",
    "    \"Mr\" :        \"Mr\",\n",
    "    \"Mrs\" :       \"Mrs\",\n",
    "    \"Miss\" :      \"Miss\",\n",
    "    \"Master\" :    \"Master\",\n",
    "    \"Lady\" :      \"Royalty\"\n",
    "}\n",
    "\n",
    "combined['Title'] = combined.Title.map(titles)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Эта функция анализирует имена и извлекает заголовки. Затем он отображает заголовки в соответствующей категории.\n",
    "Мы выбрали:\n",
    "\n",
    "- Officer\n",
    "- Royalty \n",
    "- Mr\n",
    "- Mrs\n",
    "- Miss\n",
    "- Master"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 167,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>Title</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>Mr</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "      <td>Mrs</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>Miss</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "      <td>Mrs</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>Mr</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Pclass                                               Name  \\\n",
       "0            1       3                            Braund, Mr. Owen Harris   \n",
       "1            2       1  Cumings, Mrs. John Bradley (Florence Briggs Th...   \n",
       "2            3       3                             Heikkinen, Miss. Laina   \n",
       "3            4       1       Futrelle, Mrs. Jacques Heath (Lily May Peel)   \n",
       "4            5       3                           Allen, Mr. William Henry   \n",
       "\n",
       "      Sex   Age  SibSp  Parch            Ticket     Fare Cabin Embarked Title  \n",
       "0    male  22.0      1      0         A/5 21171   7.2500   NaN        S    Mr  \n",
       "1  female  38.0      1      0          PC 17599  71.2833   C85        C   Mrs  \n",
       "2  female  26.0      0      0  STON/O2. 3101282   7.9250   NaN        S  Miss  \n",
       "3  female  35.0      1      0            113803  53.1000  C123        S   Mrs  \n",
       "4    male  35.0      0      0            373450   8.0500   NaN        S    Mr  "
      ]
     },
     "execution_count": 167,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "combined.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Обработка возраста\n",
    "\n",
    "Изучая данные, можно увидеть, что в переменной Age отсутствует 177 значений. Это большое количество (~ 13% от набора данных). Просто заменить их средним или средним возрастом может быть не лучшим решением, так как возраст может отличаться по группам и категориям пассажиров.\n",
    "\n",
    "Чтобы понять, почему, давайте сгруппируем наш набор данных по полу, типу и классу пассажиров."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 168,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Sex</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Title</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"8\" valign=\"top\">female</th>\n",
       "      <th rowspan=\"4\" valign=\"top\">1</th>\n",
       "      <th>Miss</th>\n",
       "      <td>529.5</td>\n",
       "      <td>30.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>99.9625</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mrs</th>\n",
       "      <td>853.5</td>\n",
       "      <td>45.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>78.1125</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Officer</th>\n",
       "      <td>797.0</td>\n",
       "      <td>49.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>25.9292</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Royalty</th>\n",
       "      <td>760.0</td>\n",
       "      <td>39.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>86.5000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">2</th>\n",
       "      <th>Miss</th>\n",
       "      <td>606.5</td>\n",
       "      <td>20.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>20.2500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mrs</th>\n",
       "      <td>533.0</td>\n",
       "      <td>30.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>26.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">3</th>\n",
       "      <th>Miss</th>\n",
       "      <td>603.5</td>\n",
       "      <td>18.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.0500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mrs</th>\n",
       "      <td>668.5</td>\n",
       "      <td>31.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>15.5000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"9\" valign=\"top\">male</th>\n",
       "      <th rowspan=\"4\" valign=\"top\">1</th>\n",
       "      <th>Master</th>\n",
       "      <td>803.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>134.5000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mr</th>\n",
       "      <td>634.0</td>\n",
       "      <td>41.5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>47.1000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Officer</th>\n",
       "      <td>678.0</td>\n",
       "      <td>52.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>37.5500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Royalty</th>\n",
       "      <td>600.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>27.7208</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"3\" valign=\"top\">2</th>\n",
       "      <th>Master</th>\n",
       "      <td>550.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>26.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mr</th>\n",
       "      <td>723.5</td>\n",
       "      <td>30.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>13.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Officer</th>\n",
       "      <td>513.0</td>\n",
       "      <td>41.5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>13.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">3</th>\n",
       "      <th>Master</th>\n",
       "      <td>789.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>22.3583</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mr</th>\n",
       "      <td>640.5</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.8958</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                       PassengerId   Age  SibSp  Parch      Fare\n",
       "Sex    Pclass Title                                             \n",
       "female 1      Miss           529.5  30.0    0.0    0.0   99.9625\n",
       "              Mrs            853.5  45.0    1.0    0.0   78.1125\n",
       "              Officer        797.0  49.0    0.0    0.0   25.9292\n",
       "              Royalty        760.0  39.0    0.0    0.0   86.5000\n",
       "       2      Miss           606.5  20.0    0.0    0.0   20.2500\n",
       "              Mrs            533.0  30.0    1.0    0.0   26.0000\n",
       "       3      Miss           603.5  18.0    0.0    0.0    8.0500\n",
       "              Mrs            668.5  31.0    1.0    1.0   15.5000\n",
       "male   1      Master         803.0   6.0    1.0    2.0  134.5000\n",
       "              Mr             634.0  41.5    0.0    0.0   47.1000\n",
       "              Officer        678.0  52.0    0.0    0.0   37.5500\n",
       "              Royalty        600.0  40.0    0.0    0.0   27.7208\n",
       "       2      Master         550.0   2.0    1.0    1.0   26.0000\n",
       "              Mr             723.5  30.0    0.0    0.0   13.0000\n",
       "              Officer        513.0  41.5    0.0    0.0   13.0000\n",
       "       3      Master         789.0   6.0    3.0    1.0   22.3583\n",
       "              Mr             640.5  26.0    0.0    0.0    7.8958"
      ]
     },
     "execution_count": 168,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grouped = combined.groupby(['Sex','Pclass','Title'])\n",
    "grouped.median()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Посмотрите на столбец среднего возраста, как это значение может отличаться в зависимости от Sex, Pclass и Title?\n",
    "\n",
    "Например:\n",
    "\n",
    "- Если пассажир женского пола, из класса Pclass 1 - средний возраст 39 лет.\n",
    "- Если пассажир мужского пола, из класса Pclass 3, с титулом мистера - средний возраст 26 лет.\n",
    "\n",
    "Давайте создадим функцию, которая заполняет отсутствующий возраст."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 169,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "def fillAges(row):\n",
    "    if row['Sex']=='female' and row['Pclass'] == 1:\n",
    "        if row['Title'] == 'Miss':\n",
    "            return 30\n",
    "        elif row['Title'] == 'Mrs':\n",
    "            return 45\n",
    "        elif row['Title'] == 'Officer':\n",
    "            return 49\n",
    "        elif row['Title'] == 'Royalty':\n",
    "            return 39\n",
    "\n",
    "    elif row['Sex']=='female' and row['Pclass'] == 2:\n",
    "        if row['Title'] == 'Miss':\n",
    "            return 20\n",
    "        elif row['Title'] == 'Mrs':\n",
    "            return 30\n",
    "\n",
    "    elif row['Sex']=='female' and row['Pclass'] == 3:\n",
    "        if row['Title'] == 'Miss':\n",
    "            return 18\n",
    "        elif row['Title'] == 'Mrs':\n",
    "            return 31\n",
    "\n",
    "    elif row['Sex']=='male' and row['Pclass'] == 1:\n",
    "        if row['Title'] == 'Master':\n",
    "            return 6\n",
    "        elif row['Title'] == 'Mr':\n",
    "            return 41.5\n",
    "        elif row['Title'] == 'Officer':\n",
    "            return 52\n",
    "        elif row['Title'] == 'Royalty':\n",
    "            return 40\n",
    "\n",
    "    elif row['Sex']=='male' and row['Pclass'] == 2:\n",
    "        if row['Title'] == 'Master':\n",
    "            return 2\n",
    "        elif row['Title'] == 'Mr':\n",
    "            return 30\n",
    "        elif row['Title'] == 'Officer':\n",
    "            return 41.5\n",
    "\n",
    "    elif row['Sex']=='male' and row['Pclass'] == 3:\n",
    "        if row['Title'] == 'Master':\n",
    "            return 6\n",
    "        elif row['Title'] == 'Mr':\n",
    "            return 26\n",
    "\n",
    "combined.Age = combined.apply(lambda r : fillAges(r) if np.isnan(r['Age']) else r['Age'], axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 170,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 1309 entries, 0 to 1308\n",
      "Data columns (total 12 columns):\n",
      "PassengerId    1309 non-null int64\n",
      "Pclass         1309 non-null int64\n",
      "Name           1309 non-null object\n",
      "Sex            1309 non-null object\n",
      "Age            1309 non-null float64\n",
      "SibSp          1309 non-null int64\n",
      "Parch          1309 non-null int64\n",
      "Ticket         1309 non-null object\n",
      "Fare           1308 non-null float64\n",
      "Cabin          295 non-null object\n",
      "Embarked       1307 non-null object\n",
      "Title          1309 non-null object\n",
      "dtypes: float64(2), int64(4), object(6)\n",
      "memory usage: 122.8+ KB\n"
     ]
    }
   ],
   "source": [
    "combined.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Отлично. Недостающие возрасты были восстановлены.\n",
    "\n",
    "Тем не менее, мы замечаем отсутствие значения в Fare, два отсутствующих значения в Embarked и много недостающих значений в Cabin. Мы вернемся к этим переменным позже.\n",
    "\n",
    "Давайте теперь обработаем имена."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "combined.drop('Name',axis=1,inplace=True)\n",
    "\n",
    "titles_dummies = pd.get_dummies(combined['Title'],prefix='Title')\n",
    "combined = pd.concat([combined,titles_dummies],axis=1)\n",
    "\n",
    "combined.drop('Title',axis=1,inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 172,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>Title_Master</th>\n",
       "      <th>Title_Miss</th>\n",
       "      <th>Title_Mr</th>\n",
       "      <th>Title_Mrs</th>\n",
       "      <th>Title_Officer</th>\n",
       "      <th>Title_Royalty</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Pclass     Sex   Age  SibSp  Parch            Ticket     Fare  \\\n",
       "0            1       3    male  22.0      1      0         A/5 21171   7.2500   \n",
       "1            2       1  female  38.0      1      0          PC 17599  71.2833   \n",
       "2            3       3  female  26.0      0      0  STON/O2. 3101282   7.9250   \n",
       "3            4       1  female  35.0      1      0            113803  53.1000   \n",
       "4            5       3    male  35.0      0      0            373450   8.0500   \n",
       "\n",
       "  Cabin Embarked  Title_Master  Title_Miss  Title_Mr  Title_Mrs  \\\n",
       "0   NaN        S             0           0         1          0   \n",
       "1   C85        C             0           0         0          1   \n",
       "2   NaN        S             0           1         0          0   \n",
       "3  C123        S             0           0         0          1   \n",
       "4   NaN        S             0           0         1          0   \n",
       "\n",
       "   Title_Officer  Title_Royalty  \n",
       "0              0              0  \n",
       "1              0              0  \n",
       "2              0              0  \n",
       "3              0              0  \n",
       "4              0              0  "
      ]
     },
     "execution_count": 172,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "combined.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "В итоге :\n",
    "- больше нет имен.\n",
    "- появились новые двоичные переменные (Title_X).\n",
    "     - Например, если Title_Mr = 1, то соответствующий заголовок - Mr."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Обработка остальной части данных"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 173,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "combined.Fare.fillna(combined.Fare.mean(),inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 174,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "combined.Embarked.fillna('S',inplace=True)\n",
    "embarked_dummies = pd.get_dummies(combined['Embarked'],prefix='Embarked')\n",
    "combined = pd.concat([combined,embarked_dummies],axis=1)\n",
    "combined.drop('Embarked',axis=1,inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 175,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "combined.Cabin.fillna('U',inplace=True)\n",
    "combined['Cabin'] = combined['Cabin'].map(lambda c : c[0]) \n",
    "cabin_dummies = pd.get_dummies(combined['Cabin'],prefix='Cabin')\n",
    "combined = pd.concat([combined,cabin_dummies],axis=1)\n",
    "combined.drop('Cabin',axis=1,inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 176,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 1309 entries, 0 to 1308\n",
      "Data columns (total 26 columns):\n",
      "PassengerId      1309 non-null int64\n",
      "Pclass           1309 non-null int64\n",
      "Sex              1309 non-null object\n",
      "Age              1309 non-null float64\n",
      "SibSp            1309 non-null int64\n",
      "Parch            1309 non-null int64\n",
      "Ticket           1309 non-null object\n",
      "Fare             1309 non-null float64\n",
      "Title_Master     1309 non-null uint8\n",
      "Title_Miss       1309 non-null uint8\n",
      "Title_Mr         1309 non-null uint8\n",
      "Title_Mrs        1309 non-null uint8\n",
      "Title_Officer    1309 non-null uint8\n",
      "Title_Royalty    1309 non-null uint8\n",
      "Embarked_C       1309 non-null uint8\n",
      "Embarked_Q       1309 non-null uint8\n",
      "Embarked_S       1309 non-null uint8\n",
      "Cabin_A          1309 non-null uint8\n",
      "Cabin_B          1309 non-null uint8\n",
      "Cabin_C          1309 non-null uint8\n",
      "Cabin_D          1309 non-null uint8\n",
      "Cabin_E          1309 non-null uint8\n",
      "Cabin_F          1309 non-null uint8\n",
      "Cabin_G          1309 non-null uint8\n",
      "Cabin_T          1309 non-null uint8\n",
      "Cabin_U          1309 non-null uint8\n",
      "dtypes: float64(2), int64(4), object(2), uint8(18)\n",
      "memory usage: 104.9+ KB\n"
     ]
    }
   ],
   "source": [
    "combined.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Отлично, больше нет пропущенных значений."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 177,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Title_Master</th>\n",
       "      <th>Title_Miss</th>\n",
       "      <th>...</th>\n",
       "      <th>Embarked_S</th>\n",
       "      <th>Cabin_A</th>\n",
       "      <th>Cabin_B</th>\n",
       "      <th>Cabin_C</th>\n",
       "      <th>Cabin_D</th>\n",
       "      <th>Cabin_E</th>\n",
       "      <th>Cabin_F</th>\n",
       "      <th>Cabin_G</th>\n",
       "      <th>Cabin_T</th>\n",
       "      <th>Cabin_U</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 26 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Pclass     Sex   Age  SibSp  Parch            Ticket     Fare  \\\n",
       "0            1       3    male  22.0      1      0         A/5 21171   7.2500   \n",
       "1            2       1  female  38.0      1      0          PC 17599  71.2833   \n",
       "2            3       3  female  26.0      0      0  STON/O2. 3101282   7.9250   \n",
       "3            4       1  female  35.0      1      0            113803  53.1000   \n",
       "4            5       3    male  35.0      0      0            373450   8.0500   \n",
       "\n",
       "   Title_Master  Title_Miss   ...     Embarked_S  Cabin_A  Cabin_B  Cabin_C  \\\n",
       "0             0           0   ...              1        0        0        0   \n",
       "1             0           0   ...              0        0        0        1   \n",
       "2             0           1   ...              1        0        0        0   \n",
       "3             0           0   ...              1        0        0        1   \n",
       "4             0           0   ...              1        0        0        0   \n",
       "\n",
       "   Cabin_D  Cabin_E  Cabin_F  Cabin_G  Cabin_T  Cabin_U  \n",
       "0        0        0        0        0        0        1  \n",
       "1        0        0        0        0        0        0  \n",
       "2        0        0        0        0        0        1  \n",
       "3        0        0        0        0        0        0  \n",
       "4        0        0        0        0        0        1  \n",
       "\n",
       "[5 rows x 26 columns]"
      ]
     },
     "execution_count": 177,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "combined.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 178,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "combined['Sex'] = combined['Sex'].map({'male':1,'female':0})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 179,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "pclass_dummies = pd.get_dummies(combined['Pclass'],prefix=\"Pclass\")\n",
    "combined = pd.concat([combined,pclass_dummies],axis=1)    \n",
    "combined.drop('Pclass',axis=1,inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 180,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "def cleanTicket(ticket):\n",
    "    ticket = ticket.replace('.','')\n",
    "    ticket = ticket.replace('/','')\n",
    "    ticket = ticket.split()\n",
    "    ticket = map(lambda t : t.strip() , ticket)\n",
    "    ticket = list(filter(lambda t : not t.isdigit(), ticket))\n",
    "    if len(ticket) > 0:\n",
    "        return ticket[0]\n",
    "    else: \n",
    "        return 'XXX'\n",
    "\n",
    "\n",
    "combined['Ticket'] = combined['Ticket'].map(cleanTicket)\n",
    "tickets_dummies = pd.get_dummies(combined['Ticket'],prefix='Ticket')\n",
    "combined = pd.concat([combined, tickets_dummies],axis=1)\n",
    "combined.drop('Ticket',inplace=True,axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "* Эта функция выполняет предварительную обработку билетов путем извлечения префикса билета. Если извлечение префикса невозможно, возвращается XXX.\n",
    "* Затем она кодирует префиксы с использованием фиктивной кодировки."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Обработка семей"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Создадим новые переменные на основе размера семьи."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 181,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "combined['FamilySize'] = combined['Parch'] + combined['SibSp'] + 1\n",
    "combined['Singleton'] = combined['FamilySize'].map(lambda s : 1 if s == 1 else 0)\n",
    "combined['SmallFamily'] = combined['FamilySize'].map(lambda s : 1 if 2<=s<=4 else 0)\n",
    "combined['LargeFamily'] = combined['FamilySize'].map(lambda s : 1 if 5<=s else 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Эта функция создает 4 новые переменные:\n",
    "\n",
    "- FamilySize: общее количество родственников, включая пассажира (его / ее).\n",
    "- Sigleton: булевская переменная, описывающая семейства размером = 1\n",
    "- SmallFamily: логическая переменная, описывающая семейства из 2 <= size <= 4\n",
    "- LargeFamily: логическая переменная, описывающая семейства из 5 < size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 182,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Title_Master</th>\n",
       "      <th>Title_Miss</th>\n",
       "      <th>Title_Mr</th>\n",
       "      <th>Title_Mrs</th>\n",
       "      <th>...</th>\n",
       "      <th>Ticket_STONO2</th>\n",
       "      <th>Ticket_STONOQ</th>\n",
       "      <th>Ticket_SWPP</th>\n",
       "      <th>Ticket_WC</th>\n",
       "      <th>Ticket_WEP</th>\n",
       "      <th>Ticket_XXX</th>\n",
       "      <th>FamilySize</th>\n",
       "      <th>Singleton</th>\n",
       "      <th>SmallFamily</th>\n",
       "      <th>LargeFamily</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 68 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Sex   Age  SibSp  Parch     Fare  Title_Master  Title_Miss  \\\n",
       "0            1    1  22.0      1      0   7.2500             0           0   \n",
       "1            2    0  38.0      1      0  71.2833             0           0   \n",
       "2            3    0  26.0      0      0   7.9250             0           1   \n",
       "3            4    0  35.0      1      0  53.1000             0           0   \n",
       "4            5    1  35.0      0      0   8.0500             0           0   \n",
       "\n",
       "   Title_Mr  Title_Mrs     ...       Ticket_STONO2  Ticket_STONOQ  \\\n",
       "0         1          0     ...                   0              0   \n",
       "1         0          1     ...                   0              0   \n",
       "2         0          0     ...                   1              0   \n",
       "3         0          1     ...                   0              0   \n",
       "4         1          0     ...                   0              0   \n",
       "\n",
       "   Ticket_SWPP  Ticket_WC  Ticket_WEP  Ticket_XXX  FamilySize  Singleton  \\\n",
       "0            0          0           0           0           2          0   \n",
       "1            0          0           0           0           2          0   \n",
       "2            0          0           0           0           1          1   \n",
       "3            0          0           0           1           2          0   \n",
       "4            0          0           0           1           1          1   \n",
       "\n",
       "   SmallFamily  LargeFamily  \n",
       "0            1            0  \n",
       "1            1            0  \n",
       "2            0            0  \n",
       "3            1            0  \n",
       "4            0            0  \n",
       "\n",
       "[5 rows x 68 columns]"
      ]
     },
     "execution_count": 182,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "combined.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Как вы можете видеть, они имеют разные области значения. Давайте нормализуем их все, кроме PassengerId, в единичном интервале."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 184,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "features = list(combined.columns)\n",
    "features.remove('PassengerId')\n",
    "combined[features] = combined[features].apply(lambda x: x/x.max(), axis=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Возвращаясь к нашей проблеме, теперь мы должны:\n",
    "\n",
    "1. Разбить объединенный набор данных в набор для обучения и тестирования.\n",
    "2. Построить предсказания.\n",
    "3. Оцените модель."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Чтобы оценить нашу модель, мы будем использовать 5-fold cross validation с метрикой Accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 185,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "def compute_score(clf, X, y, scoring='accuracy'):\n",
    "    xval = cross_val_score(clf, X, y, cv = 5,scoring=scoring)\n",
    "    return np.mean(xval)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 208,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "targets = pd.read_csv('titanic.train.csv').Survived\n",
    "\n",
    "train = combined.ix[0:890]\n",
    "test = combined.ix[891:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Выбор признаков\n",
    "\n",
    "На данный момент у нас есть 68 признаков. Это довольно большое число.\n",
    "Предлагается выбрать из них лучшие.\n",
    "\n",
    "Отбор признаков имеет много преимуществ:\n",
    "\n",
    "- Это уменьшает избыточность среди данных\n",
    "- Это ускоряет процесс обучения\n",
    "- Это уменьшает overfitting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 209,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "classifier = ExtraTreesClassifier(n_estimators=200)\n",
    "classifier = clf.fit(train, targets)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 210,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "features = pd.DataFrame()\n",
    "\n",
    "features['feature'] = train.columns\n",
    "features['importance'] = clf.feature_importances_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 211,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>feature</th>\n",
       "      <th>importance</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Title_Mr</td>\n",
       "      <td>0.541909</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>64</th>\n",
       "      <td>FamilySize</td>\n",
       "      <td>0.140798</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Fare</td>\n",
       "      <td>0.115458</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Title_Officer</td>\n",
       "      <td>0.047081</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>PassengerId</td>\n",
       "      <td>0.042821</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>Pclass_3</td>\n",
       "      <td>0.037470</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Age</td>\n",
       "      <td>0.021493</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>Pclass_2</td>\n",
       "      <td>0.021451</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>60</th>\n",
       "      <td>Ticket_SWPP</td>\n",
       "      <td>0.013883</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>57</th>\n",
       "      <td>Ticket_STONO</td>\n",
       "      <td>0.010900</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          feature  importance\n",
       "8        Title_Mr    0.541909\n",
       "64     FamilySize    0.140798\n",
       "5            Fare    0.115458\n",
       "10  Title_Officer    0.047081\n",
       "0     PassengerId    0.042821\n",
       "26       Pclass_3    0.037470\n",
       "2             Age    0.021493\n",
       "25       Pclass_2    0.021451\n",
       "60    Ticket_SWPP    0.013883\n",
       "57   Ticket_STONO    0.010900"
      ]
     },
     "execution_count": 211,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "features.sort_values(['importance'],ascending=False)[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Как вы можете заметить, наиболее важными являются Title_Mr, Age, Fare и Sex.\n",
    "Существует также важная зависимость с Passenger_Id.\n",
    "\n",
    "Давайте теперь преобразуем наши данные."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 213,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "model = SelectFromModel(classifier, prefit=True)\n",
    "\n",
    "train_new = model.transform(train)\n",
    "test_new = model.transform(test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Настройка гиперпараметров"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Для нахождения наилучших параметров, будем использовать **GridSearch**.\n",
    "\n",
    "Преимущество прохода по сетке в том, что его легко имплементировать самому и легко распараллелить. Однако у него есть и серьезные недостатки:\n",
    "\n",
    "- Он перебирает много заведомо неудачных точек. Допустим, уже имеется набор каких-то конфигураций с результатами или какая-то другая информация. Человек может понять, какие конфигурации точно дадут неприемлимый результат, и догадается не проверять лишний раз эти регионы. Grid search так делать не умеет. \n",
    "\n",
    "- Если гиперпараметров много, то размер «ячейки» приходится делать слишком крупным, и можно упустить хороший оптимум. Таким образом, если включить в пространство поиска много лишних гиперпараметров, никак не влияющих на результат, то grid search будет работать намного хуже при том же числе итераций."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 218,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best score: 0.8092031425364759\n",
      "Best parameters: {'criterion': 'entropy', 'max_depth': 5}\n"
     ]
    }
   ],
   "source": [
    "ensemble = DecisionTreeClassifier(max_features='sqrt')\n",
    "\n",
    "parameter_grid = {\n",
    "    'max_depth' : [2, 3, 4, 5, 6, 7, 8, 9, 10],\n",
    "    'criterion': ['gini','entropy']\n",
    "}\n",
    "\n",
    "grid_search = GridSearchCV(\n",
    "    ensemble,\n",
    "    param_grid = parameter_grid,\n",
    "    cv = StratifiedKFold(n_splits=5).split(train, targets)\n",
    ")\n",
    "\n",
    "grid_search.fit(train_new, targets)\n",
    "\n",
    "print('Best score: {}'.format(grid_search.best_score_))\n",
    "print('Best parameters: {}'.format(grid_search.best_params_))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}