2.74 Machine Learning - scikit learn API.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are going to be using [scikit learn](http://scikit-learn.org/stable/index.html)\n",
    "\n",
    "<img src='files/resources/scikit-learn-logo-small.png' align='left'><h2>Machine Learning in Python</h2>\n",
    "\n",
    "<br>\n",
    "* Simple and efficient tools for data mining and data analysis\n",
    "* Accessible to everybody, and reusable in various contexts\n",
    "* Built on NumPy, SciPy, and matplotlib\n",
    "* Open source, commercially usable - BSD license\n",
    "\n",
    "## Loading Data\n",
    "\n",
    "We always start with a data set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn import datasets\n",
    "iris = datasets.load_iris()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The features are stored in `iris.data` a 2-d array of dimension (n_samples, n_features):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data = [[ 5.1  3.5  1.4  0.2]\n",
      " [ 4.9  3.   1.4  0.2]\n",
      " [ 4.7  3.2  1.3  0.2]\n",
      " [ 4.6  3.1  1.5  0.2]\n",
      " [ 5.   3.6  1.4  0.2]]\n",
      "\n",
      "Shape = (150, 4)\n"
     ]
    }
   ],
   "source": [
    "print('Data = ' + str(iris.data[0:5,]))\n",
    "print('\\nShape = ' + str(iris.data.shape))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The target is stored in `iris.target` and is usually an array of dimension (n_samples,):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Target = [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      " 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
      " 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2\n",
      " 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2\n",
      " 2 2]\n",
      "\n",
      "Shape = (150,)\n"
     ]
    }
   ],
   "source": [
    "print('Target = ' + str(iris.target))\n",
    "print('\\nShape = ' + str(iris.target.shape))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training / Test Data\n",
    "\n",
    "We want to fit a clasifier on a portion of this data (the training data) and retain a portion for model testing (the test data).  scikit learn API includes a method to help split the data randomly:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training data shape   = (90, 4)\n",
      "Testing data shape    = (60, 4)\n",
      "Training target shape = (90,)\n",
      "Testing target shape  = (60,)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/otter/anaconda2/lib/python2.7/site-packages/sklearn/cross_validation.py:44: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.\n",
      "  \"This module will be removed in 0.20.\", DeprecationWarning)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.cross_validation import train_test_split\n",
    "\n",
    "data_train, data_test, target_train, target_test = train_test_split(iris.data, iris.target, \n",
    "                                                                    train_size = 0.6, stratify=iris.target,\n",
    "                                                                    random_state = 72) \n",
    "\n",
    "print('Training data shape   = ' + str(data_train.shape))\n",
    "print('Testing data shape    = ' + str(data_test.shape))\n",
    "print('Training target shape = ' + str(target_train.shape))\n",
    "print('Testing target shape  = ' + str(target_test.shape))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Selecting a Model and Hyperparameters\n",
    "\n",
    "We want to classify the iris flowers in to 3 categories - so we need a multi category classifier - we will use a support vector classifer (`SVC`).  We will also set the model parameters - initially we will use a linear kernel. \n",
    "\n",
    "<img src='files/resources/ic_info_outline_black_24dp_2x.png' align='left'> Note that every machine learning algorithm will have different hyper-parameters.  \n",
    "The [scikit learn API reference](http://scikit-learn.org/stable/index.html) is a great place to learn about them. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn import svm\n",
    "\n",
    "clf = svm.SVC(kernel='linear')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fitting the Model\n",
    "\n",
    "At this point we have not fit the model - just defined it.  Now let's fit the model using the `fit()` method.  \n",
    "\n",
    "<img src='files/resources/ic_info_outline_black_24dp_2x.png' align='left'> Every estimator (model) in scikit learn has a `fit()` method - whether it is a regression, classification or clustering algorithm.  \n",
    "For other methods check the [scikit learn API reference](http://scikit-learn.org/stable/index.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,\n",
       "  decision_function_shape=None, degree=3, gamma='auto', kernel='linear',\n",
       "  max_iter=-1, probability=False, random_state=None, shrinking=True,\n",
       "  tol=0.001, verbose=False)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf.fit(X=data_train, y=target_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Predicting with the Model\n",
    "\n",
    "Now we can use the fitted model to predict unknown values.\n",
    "\n",
    "<img src='files/resources/ic_info_outline_black_24dp_2x.png' align='left'> Every estimator (model) in scikit learn has a `predict()` method that always requires and X argument to be supplied.  \n",
    "For other methods check the [scikit learn API reference](http://scikit-learn.org/stable/index.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2 1 1 1 1 2 1 0 1 0 0 0 0 0 0 1 2 1 2 2 1 1 1 0 0 0 2 2 1 0 2 2 2 0 1 2 0\n",
      " 2 1 0 1 1 1 0 2 0 2 2 2 0 2 2 0 2 0 1 1 0 1 1]\n"
     ]
    }
   ],
   "source": [
    "pred = clf.predict(X=data_test)\n",
    "\n",
    "print(pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Measuring Performance\n",
    "\n",
    "We can use metrics from sklearn to quantify performance:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy = 98.3% \n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "print(\"accuracy = {0:4.1f}% \".format(accuracy_score(pred, target_test)*100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## More information\n",
    "\n",
    "Start with the [scikit learn website](http://scikit-learn.org/stable/index.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn import datasets\n",
    "from sklearn.cross_validation import train_test_split\n",
    "\n",
    "iris = datasets.load_iris()\n",
    "\n",
    "data_train, data_test, \\\n",
    "target_train, target_test = train_test_split(iris.data, iris.target, \n",
    "                                                train_size = 0.6, stratify=iris.target,\n",
    "                                                random_state = 72) \n",
    "\n",
    "from sklearn import svm\n",
    "\n",
    "clf = svm.SVC(kernel='linear')\n",
    "\n",
    "clf.fit(X=data_train, y=target_train)\n",
    "\n",
    "pred = clf.predict(X=data_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After fitting a mode you can serialize it using `sklearn.externals` and the `joblib` class:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['filename.pkl']"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.externals import joblib\n",
    "joblib.dump(clf, 'filename.pkl') "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}