Jupyter/M03-BigData/M03D-DataAcquisition-I.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "FtcmYWJc-FBb"
   },
   "source": [
    "![Cloud-First](../image/CloudFirst.png) \n",
    "\n",
    "\n",
    "# SIT742: Modern Data Science\n",
    "**(Module: Big Data)**\n",
    "\n",
    "---\n",
    "- Materials in this module include resources collected from various open-source online repositories.\n",
    "- You are free to use, change and distribute this package.\n",
    "- If you found any issue/bug for this document, please submit an issue at [tulip-lab/sit742](https://github.com/tulip-lab/sit742/issues)\n",
    "\n",
    "\n",
    "Prepared by **SIT742 Teaching Team**\n",
    "\n",
    "---\n",
    "\n",
    "\n",
    "## Session 3D: Data Acquisition (1)\n",
    "\n",
    "In this week, we will learn how to use Python Packages to ETL the data and files."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Fb5TqlLE-FBo"
   },
   "source": [
    "## Content\n",
    "\n",
    "\n",
    "\n",
    "### Part 1 Numpy Module\n",
    "\n",
    "1.1 [Importing Numpy](#importnp)\n",
    "\n",
    "1.2 [Numpy arrays](#nparray)\n",
    "\n",
    "1.3 [Manipulating arrays](#maninp)\n",
    "\n",
    "1.4 [Array Operations](#arrayop)\n",
    "\n",
    "1.5 [np.random](#random)\n",
    "\n",
    "1.6 [Vectorizing Functions](#vecfunc)\n",
    "\n",
    "\n",
    "\n",
    "### Part 2 Data Loading\n",
    "\n",
    "2.1 [TXT](#txt)\n",
    "\n",
    "2.2 [CSV](#csv)\n",
    "\n",
    "2.3 [JSON](#json)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "fm6guAC1-FHe"
   },
   "source": [
    "---\n",
    "## <span style=\"color:#0b486b\">1. Numpy module</span>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "o1E02sGn-FHg"
   },
   "source": [
    "Python lists are very flexible for storing any sequence of Python objects. But usually flexibility comes at the price of performance and therefore Python lists are not ideal for numerical calculations where we are interested in performance. Here is where **NumPy** comes in. It adds support for large, multi-dimensional arrays and matrices, along with high-level mathematical functions to operate on these arrays to Python. \n",
    "\n",
    "Relying on `'BLAS'` and `'LAPACK'`, `'NumPy'` gives a functionality comparable with `'MATLAB'` to Python. NumPy facilitates advanced mathematical and other types of operations on large numbers of data. Typically, such operations are executed more efficiently and with less code than is possible using Python’s built-in sequences. It has become one of the fundamental packages used for numerical computations.\n",
    "\n",
    "In this tutorial we will review its basics, so to learn more about NumPy, visit [NumPy User Guide](http://docs.scipy.org/doc/numpy/user/index.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Hc_bsO3l-FHj"
   },
   "source": [
    "<a id = \"importnp\"></a>\n",
    "\n",
    "### <span style=\"color:#0b486b\">1.1 Importing Numpy</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "yWUvtzi8-FHm"
   },
   "source": [
    "As you have learnt in this session, first we have to import a package to be able to use it. NumPy is imported with:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "2-B8prOg-FHq"
   },
   "outputs": [],
   "source": [
    "import numpy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "a39r4M3p-FH1"
   },
   "source": [
    "Although it is the convention to import it like with an alias:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "SKj133sh-FH7"
   },
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "uAmsO-V8-FIR"
   },
   "source": [
    "<a id = \"nparray\"></a>\n",
    "\n",
    "### <span style=\"color:#0b486b\">1.2 Numpy arrays</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "tlnX1eG8-FIT"
   },
   "source": [
    "The core of NumPy is its arrays. You can create an array from a Python list or tuple using `'array'` function. They work similarly to lists apart from the fact that:\n",
    "\n",
    "* you can easily perform element-wise operation on them, and\n",
    "* unlike lists, they should be pre-allocated. This means that there is no equivalent to list append for arrays. The size of the arrays is known at the time it is defined.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ATvr-1ZS-FIW"
   },
   "source": [
    "#### <span style=\"color:#0b486b\">1.2.1 Create an array from a list</span>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "xC6z9Xyo-FIY"
   },
   "outputs": [],
   "source": [
    "x = [1, 7, 3, 4, 0, -5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "gdsnKRWa-FIi"
   },
   "outputs": [],
   "source": [
    "y = np.array(x)\n",
    "type(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "nnBy-g5b-FIq"
   },
   "source": [
    "#### <span style=\"color:#0b486b\">1.2.2 Create an array using a range</span>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "OBt00bhq-FIs"
   },
   "outputs": [],
   "source": [
    "range(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "e1KYDBLq-FI7"
   },
   "outputs": [],
   "source": [
    "print(np.array(range(5)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "LhL0M03i-FJD"
   },
   "outputs": [],
   "source": [
    "print(np.arange(2, 3, 0.2))  # Why is there no value 3.0 in the output?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "_SQTHmfL-FJL"
   },
   "outputs": [],
   "source": [
    "print(np.linspace(2, 3, 5))    # returns numbers spaced evenly on a linear scale, both endspoints are included\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "JDpURgHfM8-D"
   },
   "source": [
    "Just try to change the variable value 5 with 1, 2, 4  or 10? \\\n",
    "\n",
    "What pattern could you find ?\n",
    "\n",
    "Could you guess what is the function of **linspace**, if without the given comments?\n",
    "\n",
    "Then, you can try to use the same method to learn what is the function of **logspace**?\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Ol089FmC-FJS"
   },
   "outputs": [],
   "source": [
    "print(np.logspace(2, 3, 5))   # returns numbers spaced evenly on a log scale"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "hXqpHuub-FJZ"
   },
   "source": [
    "**Note:** If you need any help on how to use a function or what it does, you can use IPython help. Just add a question mark (?) at the end of the function and execute the cell:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "xFYTD-O--FJb"
   },
   "outputs": [],
   "source": [
    "np.logspace?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "PnwvGJX4-FJj"
   },
   "source": [
    "#### <span style=\"color:#0b486b\">1.2.3 Create a prefilled array</span>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "zSvnqTuR-FJn"
   },
   "outputs": [],
   "source": [
    "print(np.zeros(5))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "MuWwpVx2-FJ0"
   },
   "outputs": [],
   "source": [
    "print(\"The 1st sample is\",np.ones(5, dtype=int))   # you can specify the data type, default is float\n",
    "print(\"The 2nd sample is \",np.ones(5, dtype=float)) \n",
    "print(\"The 3rd sample is \",np.ones((5,5), dtype=int)) \n",
    "print(\"The 4th sample is \",np.ones((5,5,5), dtype=int)) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.ones?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "g7FgDgFa-FJ7"
   },
   "source": [
    "#### <span style=\"color:#0b486b\">1.2.4 `'mgrid'`</span>\n",
    "similar to meshgrid in MATLAB:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "U5OdZS-L-FKB"
   },
   "outputs": [],
   "source": [
    "x, y = np.mgrid[0:5, 0:3]\n",
    "\n",
    "print(x)\n",
    "print(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "vI0XK1HEPpRb"
   },
   "outputs": [],
   "source": [
    "np.mgrid?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "BK24IR2b-FKJ"
   },
   "source": [
    "#### <span style=\"color:#0b486b\">1.2.5 Array attributes</span>\n",
    "\n",
    "NumPy arrays have multiple attributes and methods. The cell below shows a few of them. You can press tab after typing the dot operator `'(.)'` to use IPython auto-complete and see the rest of them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "MirmVfAm-FKN"
   },
   "outputs": [],
   "source": [
    "y = np.array([3, 0, -4, 6, 12, 2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "lPHoezhC-FKW"
   },
   "outputs": [],
   "source": [
    "print(\"number of dimensions:\\t\", y.ndim)        \n",
    "print(\"dimension of the array:\", y.shape)       \n",
    "print(\"numerical data type:\\t\", y.dtype)\n",
    "print(\"maximum of the array:\\t\", y.max())       \n",
    "print(\"index of the array max:\", y.argmax())    \n",
    "print(\"mean of the array:\\t\", y.mean())      "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "CrbiDDhr-FKk"
   },
   "source": [
    "#### <span style=\"color:#0b486b\">1.2.6 Multi-dimensional arrays</span>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "qDBhj18y-FKn"
   },
   "source": [
    "You can define arrays with 2 (or higher) dimensions in numpy:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ltKaGI6K-FKo"
   },
   "source": [
    "##### from lists"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "d8M2wkbR-FKr"
   },
   "outputs": [],
   "source": [
    "x = [[1, 2, 10, 20], [3, 4, 30, 40]]\n",
    "y = np.array(x)\n",
    "print(y)\n",
    "print()\n",
    "print(y.ndim, y.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "XmXmLGk1-FKz"
   },
   "source": [
    "##### pre-filled "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "qiG5NGcB-FK2"
   },
   "outputs": [],
   "source": [
    "x = np.ones((3, 5), dtype='int')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "qSRQFimN-FK8",
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "print(x)\n",
    "print()\n",
    "print(x.ndim, x.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "foS348qb-FLB"
   },
   "source": [
    "##### `'diag()'`\n",
    "diagonal matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "M-ttVrqJ-FLN"
   },
   "outputs": [],
   "source": [
    "np.diag([1, 2, 3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "3fRD43wu-FLY"
   },
   "source": [
    "<a id = \"maninp\"></a>\n",
    "\n",
    "### <span style=\"color:#0b486b\">1.3 Manipulating arrays</span>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "cgGZKWuW-FLa"
   },
   "source": [
    "#### <span style=\"color:#0b486b\">1.3.1 Indexing</span>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "reg5bJIY-FLc"
   },
   "source": [
    "Similar to lists, you can index elements in an array using `'[]'` and indices:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "47U6VVq--FLf"
   },
   "source": [
    "If `'x'` is a 1-dimensional array, `'x[i]'` will index `'ith'` element of `'x'`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "WI3XEpO0-FLi"
   },
   "outputs": [],
   "source": [
    "x = np.array([2, 8, -2, 4, 3])\n",
    "print(x[3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "kOhiutVu-FLp"
   },
   "source": [
    "If 'x' is a 2-dimensional arrray:\n",
    "\n",
    "* '`x[i, j]'` or `'x[i][j]'` will index the element in `'ith'` row and `'jth'` column\n",
    "* '`x[i, :]'` will index the `'ith'` row \n",
    "* `'x[:, j]'` will index `'jth'` column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "5dZ8bMdA-FLr"
   },
   "outputs": [],
   "source": [
    "x = np.array([[7, 6, 8, 6, 4],\n",
    "              [4, 7, -2, 0, 9]])\n",
    "              \n",
    "print(x[1, 3])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "RYHDHduJ-FLw"
   },
   "outputs": [],
   "source": [
    "print(x[1, :])      # or x[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "WUyelaZV-FL1"
   },
   "outputs": [],
   "source": [
    "print(x[:, 3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "YMP2cU6y-FMB"
   },
   "source": [
    "Arrays can also be indexed with other arrays:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "QgEan722-FMC"
   },
   "outputs": [],
   "source": [
    "x = np.array([2, 8, -2, 4, 3, 9, 0])\n",
    "\n",
    "idx1 = [1, 3, 4]        # list\n",
    "idx2 = np.array(idx1)   # array\n",
    "\n",
    "print(x[idx1], x[idx2])\n",
    "x[idx2] = 0\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "0KBUG3Eg-FMI"
   },
   "source": [
    "You can also index masks. The index mask should be a NumPy arrays of data type Bool. Then the element of the array is selected only if the index mask at the position of the element is True."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "X1I8PH1Q-FMK"
   },
   "outputs": [],
   "source": [
    "x = np.array([2, 8, -2, 4, 3, 9, 0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "7ge96tsf-FMQ"
   },
   "outputs": [],
   "source": [
    "mask = np.array([False, True, True, False, False, True, False])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "PEoatrjz-FMU"
   },
   "outputs": [],
   "source": [
    "x[mask]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "rHObUufk-FMb"
   },
   "source": [
    "Combining index masks with comparison operators enables you to conditionally select elements of the array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "ylAy1AHp-FMd"
   },
   "outputs": [],
   "source": [
    "x = np.array([2, 8, -2, 4, 3, 9, 0])\n",
    "mask = (x>=2) * (x<9)\n",
    "x[mask]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "HA-J3zZr-FMq"
   },
   "source": [
    "#### <span style=\"color:#0b486b\">1.3.2 Slicing</span>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "blX9Xzbl-FMs"
   },
   "source": [
    "Similar to Python lists, arrays can also be sliced:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "QJ550Wak-FMu"
   },
   "outputs": [],
   "source": [
    "x = np.array([2, 8, -2, 4, 3, 9, 0])\n",
    "\n",
    "print(x[3:])    # slicing\n",
    "print(x[3:7:2])  # slicing with a specified step"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "w8V9DnNs-FMz"
   },
   "outputs": [],
   "source": [
    "x = np.array([[7, 6, 8, 6, 4, 3],\n",
    "              [4, 7, 0, 5, 9, 5],\n",
    "              [7, 3, 6, 3, 5, 1]])\n",
    "              \n",
    "\n",
    "print(x[1, 1:4])\n",
    "print()\n",
    "print(x[:2, 1::2])    # rows zero up to 2, cols 1 up to end with a step=2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "hWTmUPd6-FM5"
   },
   "source": [
    "#### <span style=\"color:#0b486b\">1.3.3 Iteration over items</span>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Eu9gjTKo-FM7"
   },
   "source": [
    "Since most of NumPy functions are capable of operating on arrays, in many cases iteration over items of an arrays can be (and should be) avoided. Otherwise it is pretty much similar to iterating over values of a list:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "eayPM8on-FNA"
   },
   "outputs": [],
   "source": [
    "a = np.arange(0, 50, 7)\n",
    "print(a)\n",
    "for item in a:\n",
    "    print(item,) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "uEEZs2Pd-FNG"
   },
   "source": [
    "Of course you could iterate over items using their indices too:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "W35khYsr-FNJ"
   },
   "outputs": [],
   "source": [
    "a = np.arange(0, 50, 7)\n",
    "for i in range(a.shape[0]):\n",
    "    print(a[i],)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "E9efgG2f-FNQ"
   },
   "source": [
    "There are also many functions for manipulating arrays. The most used ones are:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Jq2e_fTx-FNS"
   },
   "source": [
    "#### <span style=\"color:#0b486b\">1.3.4 `copy()`</span>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Yqcj120w-FNU"
   },
   "source": [
    "**Remember** that assignment operator is not an equivalent for copying arrays. In fact Python does not pass the values. It passess the references."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Wkx8l22X-FNa"
   },
   "outputs": [],
   "source": [
    "x = [1, 2, 3]\n",
    "y = x\n",
    "print(x, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "69hYgW58-FNf"
   },
   "outputs": [],
   "source": [
    "y[0] = 0       # now we alter an element of y\n",
    "print(x, y)     # note that x has changed as well"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Oof00tVc-FNq"
   },
   "source": [
    "Same is true for numpy arrays. That's why if you need a copy of an array, you should use `'copy()'` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "ir5585gc-FNs"
   },
   "outputs": [],
   "source": [
    "x = np.array([1, 2, 3])\n",
    "y = x\n",
    "\n",
    "y[0] = 0       # now we alter an element of y\n",
    "print(x, y)     # note that x has changed as well"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Uo-iZzvC-FN3"
   },
   "outputs": [],
   "source": [
    "x = np.array([1, 2, 3])\n",
    "y = x.copy()  # or np.copy(x)\n",
    "y[0] = 0\n",
    "\n",
    "print(x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "tSrPOa8t-FN_"
   },
   "source": [
    "#### <span style=\"color:#0b486b\">1.3.5 `reshape()`</span>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "8tnEtiPp-FOC"
   },
   "outputs": [],
   "source": [
    "x1 = np.arange(6)\n",
    "x2 = x1.reshape((2, 3))    # or np.reshape(x1, (2, 3))\n",
    "\n",
    "print(x1)\n",
    "print()\n",
    "print(x2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "7_aHU31V-FOL"
   },
   "source": [
    "#### <span style=\"color:#0b486b\">1.3.6 `astype()`</span>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "76fjepzM-FON"
   },
   "source": [
    "Used for type casting:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "k1GoBiQV-FOO"
   },
   "outputs": [],
   "source": [
    "x1 = np.arange(5)\n",
    "x2 = x1.astype(float)\n",
    "\n",
    "print(type(x1), x1)\n",
    "print(type(x2), x2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "k6bOoR14-FOU"
   },
   "source": [
    "#### <span style=\"color:#0b486b\">1.3.7 `T` Transpose</span> "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "oe1PUocl-FOc"
   },
   "source": [
    "transpose method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "DXLlgFaT-FOe"
   },
   "outputs": [],
   "source": [
    "x1 = np.random.randint(5, size=(2, 4))\n",
    "x2 = x1.T\n",
    "\n",
    "print(x1)\n",
    "print()\n",
    "print(x2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "axdyp7nJ-FOi"
   },
   "source": [
    "<a id = \"arrayop\"></a>\n",
    "\n",
    "### <span style=\"color:#0b486b\">1.4 Array operations</span>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Pdnp48Ov-FOp"
   },
   "source": [
    "#### <span style=\"color:#0b486b\">1.4.1 Arithmetic operators</span>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "tbF09PSC-FOr"
   },
   "source": [
    "Arrays can be added, subtracted, multiplied and divided using +, -, \\* and, /. Operations done by these operators are **element wise**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Rrdn8VQE-FOt"
   },
   "outputs": [],
   "source": [
    "x1 = np.array([[2, 3, 5, 7], \n",
    "               [2, 4, 6, 8]], dtype=float)\n",
    "x2 = np.array([[6, 5, 4, 3], \n",
    "               [9, 7, 5, 3]], dtype=float)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "NsdLlPTm-FO1"
   },
   "outputs": [],
   "source": [
    "print(x1)\n",
    "print()\n",
    "print(x2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "6NWyxuog-FO7"
   },
   "outputs": [],
   "source": [
    "print(x1 + x2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "xb4lLmlt-FPG"
   },
   "outputs": [],
   "source": [
    "print(x1 - x2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "7NM1P_u6-FPM"
   },
   "outputs": [],
   "source": [
    "print(x1 * x2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "pBirpVSg-FPR"
   },
   "outputs": [],
   "source": [
    "print(x1 / x2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "BWYmg3gc-FPZ"
   },
   "outputs": [],
   "source": [
    "print(3 + x1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "bHFn1Mxt-FPd"
   },
   "outputs": [],
   "source": [
    "print(3 * x1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Piegik6m-FPl"
   },
   "outputs": [],
   "source": [
    "print(3 / x1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "2xqoJwUZ-FPr"
   },
   "source": [
    "#### <span style=\"color:#0b486b\">1.4.2 Boolean operators</span>\n",
    "\n",
    "Much like arithmetic operators discussed above, boolean (comparison) operators perform element-wise on arrays."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "aM5g731W-FPs"
   },
   "outputs": [],
   "source": [
    "x1 = np.array([2, 3, 5, 7])\n",
    "x2 = np.array([2, 4, 6, 7])\n",
    "y = x1<x2\n",
    "\n",
    "print( y, y.dtype)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "cnIFJb0j-FPw"
   },
   "source": [
    "use methods `'.any()'` and `'.all()'` to return a single boolean value indicating whether any or all values in the array are True respectively. This value in turn can be used as a condition for an `'if'` statement."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "7NGdUs29-FPy"
   },
   "outputs": [],
   "source": [
    "print (y.all())\n",
    "print (y.any())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "xy2gjc2T-FP3"
   },
   "source": [
    "NumPy has many other functions that you can read about them in [NumPy User Guide](http://docs.scipy.org/doc/numpy/user/). Specially read about:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "0lc_o_C5-FP4"
   },
   "source": [
    "* `np.unique`, returns unique elements of an array\n",
    "* `np.flatten`, flattens a multi-dimensional array\n",
    "* `np.mean`, `np.std`, `np.median`\n",
    "* `np.min`, `np.max`, `np.argmin`, `np.argmax`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "xTcRRTbn-FP5"
   },
   "source": [
    "<a id = \"random\"></a>\n",
    "\n",
    "### <span style=\"color:#0b486b\">1.5 np.random</span>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "MPPLTWcv-FP7"
   },
   "source": [
    "NumPy has a module called `random` to generate arrays of random numbers. There are different ways to generate a random number:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "OlyjQz8z-FP8"
   },
   "outputs": [],
   "source": [
    "print( np.random.rand())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "IxnFBbRv-FQD"
   },
   "outputs": [],
   "source": [
    "# 2x5 random array drawn from standard normal distribution\n",
    "print( np.random.random([2, 5]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "YZ_OPkSs-FQI"
   },
   "outputs": [],
   "source": [
    "# 2x5 random array drawn from standard normal distribution\n",
    "print (np.random.rand(2, 5))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "9GIPOTP_-FQM"
   },
   "outputs": [],
   "source": [
    "# 2x5 random array drawn from a uniform distribution on {0, 1, 2, ..., 9}\n",
    "print (np.random.randint(10, size=[2, 5])) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "C9UFe6xH-FQQ"
   },
   "source": [
    "##### <span style=\"color:#0b486b\">1.5.1 Random seed</span>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "yqKfnar6-FQR"
   },
   "source": [
    "Random numbers generated by computers are not really random. They are called pseudo-random. Thus we can set the random generator to generate the same set of random numbers every time. This is useful while testing the code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "d55atcWW-FQS"
   },
   "outputs": [],
   "source": [
    "for i in range(5):\n",
    "    print (np.random.random(),)    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "IEhNqhH2-FQX"
   },
   "outputs": [],
   "source": [
    "for i in range(5):\n",
    "    np.random.seed(100)\n",
    "    print (np.random.random(),)    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "AZWRunty-FQd"
   },
   "source": [
    "<a id = \"vecfunc\"></a>\n",
    "\n",
    "### <span style=\"color:#0b486b\">1.6 Vectorizing functions</span>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "cO7BR00T-FQf"
   },
   "source": [
    "As mentioned earlier in operators, to get a good performance you should avoid looping over elements in an array and use vectorized algorithms. Many methods and functions of NumPy already support vectors, so keep this in mind while writing your own code.\n",
    "\n",
    "But for now, suppose you have written a step function which does not work with arrays, as the cell below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "TZLJoQZq-FQh"
   },
   "outputs": [],
   "source": [
    "def step_func(x):\n",
    "    \"\"\"\n",
    "    scalar implementation of step function\n",
    "    \"\"\"\n",
    "    \n",
    "    if x>=0:\n",
    "        return 1\n",
    "    else:\n",
    "        return 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "4PRIQauM-FQm"
   },
   "source": [
    "Obviously it fails when dealing with an array, because it expects a scalar as its input. Execute the cell below and see that it raises an error:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "pqcpMBr7-FQn"
   },
   "outputs": [],
   "source": [
    "# since step_func expects a scalar and recieves an array instead, \n",
    "# it raises an error\n",
    "\n",
    "step_func(np.array([2, 7, -4, -9, 0, 4]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "iSkI0M-G-FQr"
   },
   "source": [
    "You can use the function `'np.vectorize()'` to obtain a vectorized version of `'step_func'` that can handle vector data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "VWZEN82_-FQt"
   },
   "outputs": [],
   "source": [
    "step_func_vectorized = np.vectorize(step_func)\n",
    "step_func_vectorized(np.array([2, 7, -4, -9, 0, 4]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "bnn0lc9P-FQy"
   },
   "source": [
    "Although `'vectorize()'` can automatically derive a vectorized version of a scalar function, but it is always better to keep this in mind and write functions vector-compatible, from the beginning. For example we could write the step function as it is shown in the cell below, so it can handle scalar and vector data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "7qdNrJ3R-FQ1"
   },
   "outputs": [],
   "source": [
    "def step_func2(x):\n",
    "    \"\"\"\n",
    "    vector and scalar implementation of step function\n",
    "    \"\"\"\n",
    "    \n",
    "    return 1 * (x>=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "pB2wcxK0-FQ5"
   },
   "outputs": [],
   "source": [
    "step_func2(np.array([2, 7, -4, -9, 0, 4]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "05sezgN2-FQ9"
   },
   "source": [
    "---\n",
    "## <span style=\"color:#0b486b\">2. File I/O</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Z6iEGtaO-FQ-"
   },
   "source": [
    "For Online platforms such as Google Colab, or IBM Cloud, it is important for you to get familiar with the provided data storage or cloud data storage function. Alternatively, you might want to directly access the file, and load into your Notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "DopvIhrG-FRA"
   },
   "outputs": [],
   "source": [
    "!pip install wget"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "QXFY49LC-FRE"
   },
   "source": [
    "Then you can download the file into GPFS file system."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "ab6v2GxS-FRF"
   },
   "outputs": [],
   "source": [
    "import wget\n",
    "\n",
    "link_to_data = 'https://github.com/tulip-lab/sit742/raw/master/Jupyter/data/csv_data1.csv'\n",
    "DataSet = wget.download(link_to_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "uluOknMC-FRJ"
   },
   "source": [
    "<a id = \"txt\"></a>\n",
    "\n",
    "### <span style=\"color:#0b486b\">2.1 TXT</span>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "o1ZysbwZ-FRK"
   },
   "source": [
    "TXT file format is the most simplistic way to store data. \n",
    "\n",
    "Load a TXT file with `'np.loadtxt()'`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Q2zCykcT-FRL"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "# This code is for local PC\n",
    "# x = np.loadtxt(\"data/txt_data1.txt\")\n",
    "\n",
    "# The following code for IBM Cloud\n",
    "link_to_data = 'https://github.com/tulip-lab/sit742/raw/master/Jupyter/data/txt_data1.txt'\n",
    "DataSet = wget.download(link_to_data)\n",
    "\n",
    "x = np.loadtxt(\"txt_data1.txt\")\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "iZZfyuL1-FRP"
   },
   "source": [
    "Save a TXT file with `'np.savetxt()'`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "-RkfeDje-FRR"
   },
   "outputs": [],
   "source": [
    "y = np.random.randint(10, size=5)\n",
    "np.savetxt(\"txt_data2.txt\", y)\n",
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "OT2pfvIA-FRW"
   },
   "source": [
    "<a id = \"csv\"></a>\n",
    "\n",
    "### <span style=\"color:#0b486b\">2.2 CSV</span>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "XN8xU7cR-FRX"
   },
   "source": [
    "Comma Separated Values format and its variations, are one the most used file format to store data.\n",
    "\n",
    "You can use `'np.genfromtxt()'` to read a CSV file:\n",
    "\n",
    "**NOTE:** The best way to read CSV and XLS files is using **pandas** package that will be introduced later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "gHHWnj8r-FRZ"
   },
   "outputs": [],
   "source": [
    "import wget\n",
    "\n",
    "link_to_data = 'https://github.com/tulip-lab/sit742/raw/master/Jupyter/data/csv_data1.csv'\n",
    "DataSet = wget.download(link_to_data)\n",
    "\n",
    "print(DataSet)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "PlB0GcDT-FRd"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "\n",
    "x = np.genfromtxt(\"csv_data1.csv\", delimiter=\",\")\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "eMR_478x-FRk"
   },
   "source": [
    "Use `'np.savetxt()'` to save a 2d-array in a CSV file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "BvkGTJF_-FRl"
   },
   "outputs": [],
   "source": [
    "x = np.random.randint(10, size=(6,4))\n",
    "np.savetxt(\"csv_data2.csv\", x, delimiter=',')\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "jWORjUcv-FRp"
   },
   "source": [
    "<a id = \"json\"></a>\n",
    "\n",
    "### <span style=\"color:#0b486b\">2.3 JSON</span>\n",
    "\n",
    "\n",
    "JSON is the most used file format when dealing with web services. \n",
    "\n",
    "To read a JSON file, use `'json'` package and `'load()'` function, or `'loads()'` if the data is serialized. It reads the data and parses it into a dictionary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "yABGeta3-FRq"
   },
   "outputs": [],
   "source": [
    "link_to_data = 'https://github.com/tulip-lab/sit742/raw/master/Jupyter/data/json_data1.json'\n",
    "DataSet = wget.download(link_to_data)\n",
    "\n",
    "print(DataSet)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "7FAA_6uT-FRu"
   },
   "outputs": [],
   "source": [
    "import json\n",
    "with open(\"json_data1.json\", 'rb') as fp:\n",
    "    fcontent = fp.read()\n",
    "# data = json.loads(fcontent)\n",
    "data = json.loads(fcontent.decode('utf-8'))\n",
    "data.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "DWTeTfFq-FRz"
   },
   "outputs": [],
   "source": [
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "hc-aODdh-FR2"
   },
   "outputs": [],
   "source": [
    "data['phoneNumbers']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "dOuOtK50-FR5"
   },
   "source": [
    "You can also write a python dictionary into a JSON file. To do this use `'dump()'` or `'dumps()'` functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "GuglShkt-FR6"
   },
   "outputs": [],
   "source": [
    "data = [{'Name': 'Zara', 'Age': 7, 'Class': 'First'}, \n",
    "        {'Name': 'Lily', 'Age': 9, 'Class': 'Third'}];\n",
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Ql8yHsKw-FR9"
   },
   "outputs": [],
   "source": [
    "with open(\"json_data_now.json\", 'w') as fp:\n",
    "    json.dump(data, fp)"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "M03D-DataAcquisition-I.ipynb",
   "provenance": [],
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}