01-tensor_tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "PIuEiTJpzSZC"
   },
   "source": [
    "# What is PyTorch?\n",
    "\n",
    "It’s a Python based scientific computing package targeted at two sets of\n",
    "audiences:\n",
    "-  A replacement for NumPy to use the power of GPUs\n",
    "-  a deep learning research platform that provides maximum flexibility\n",
    "   and speed\n",
    "\n",
    "## Getting Started\n",
    "\n",
    "### Tensors\n",
    "\n",
    "Tensors are similar to NumPy’s ndarrays, with the addition being that\n",
    "Tensors can also be used on a GPU to accelerate computing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "iEW7SkqcuivT"
   },
   "outputs": [],
   "source": [
    "from __future__ import print_function\n",
    "import torch\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ldSkq2qmz5kx"
   },
   "source": [
    "Construct a 5x3 matrix, uninitialized:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "xUvCfSG0z6Za"
   },
   "outputs": [],
   "source": [
    "x = torch.empty(5, 3)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "eg32wRh2z_Kg"
   },
   "source": [
    "Construct a randomly initialized matrix:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "I_vzhrblz8wm"
   },
   "outputs": [],
   "source": [
    "x = torch.rand(5, 3)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "wtSN6OfX0GZE"
   },
   "source": [
    "  Construct a matrix filled zeros and of dtype `int64`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "4-QlwAkf0B7J"
   },
   "outputs": [],
   "source": [
    "x = torch.zeros(5, 3, dtype=torch.int64)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "fgZMFkVa0GHw"
   },
   "source": [
    "Construct a tensor directly from data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "MJvNb2lR0Qi1"
   },
   "outputs": [],
   "source": [
    "x = torch.tensor([5.5, 3])\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "TIBFW9pa0UF0"
   },
   "source": [
    "or create a tensor based on an existing tensor. These methods will reuse properties of the input tensor, e.g. dtype, unless new values are provided by user"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "0LkwcTUs0ZIj"
   },
   "outputs": [],
   "source": [
    "x = x.new_ones(5, 3, dtype=torch.double)      # new_* methods take in sizes\n",
    "print(x)\n",
    "\n",
    "x = torch.randn_like(x, dtype=torch.float)    # override dtype!\n",
    "print(x)                                      # result has the same size\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "9r9EKpUD0fWp"
   },
   "source": [
    "Get its size:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "acdRMK7g0cLL"
   },
   "outputs": [],
   "source": [
    "print(x.size())\n",
    "print(x.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "pKnafQWZ0vc1"
   },
   "source": [
    "#### note\n",
    "     torch.Size is in fact a tuple, so it supports all tuple operations.\n",
    "\n",
    "### Operations\n",
    "\n",
    "Operations work as expected"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "PWiVkDzz0fvQ"
   },
   "outputs": [],
   "source": [
    "y = torch.rand(5, 3)\n",
    "print(x + y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "OcixlkFY1mSK"
   },
   "source": [
    "Addition: in-place"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "b0ATzKnP1osR"
   },
   "outputs": [],
   "source": [
    "y += x\n",
    "print(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "lXZbk73m1mqv"
   },
   "source": [
    "#### note\n",
    "    Any operation that mutates a tensor in-place is post-fixed with an _.\n",
    "    For example: x.copy_(y), x.t_(), will change x.\n",
    "\n",
    "You can use standard NumPy-like indexing with all bells and whistles!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "IK1MI7yu1p1T"
   },
   "outputs": [],
   "source": [
    "print(x[:, 1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "bH-85hbn1mx8"
   },
   "source": [
    "Resizing: If you want to resize/reshape tensor, you can use ``torch.view``:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = torch.randn(4, 4)\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Numpy-style reshaping "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x.reshape(16)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "osFNs8dm1qYY"
   },
   "outputs": [],
   "source": [
    "x.reshape(-1, 8)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "88GoIW_Z1m6y"
   },
   "source": [
    "If you have a one element tensor, use ``.item()`` to get the value as a Python number"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "tOx3JDK21q7k"
   },
   "outputs": [],
   "source": [
    "x = torch.randn(1)\n",
    "print(x)\n",
    "print(x.item())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "5N2cNKBY1nAm"
   },
   "source": [
    "**Read later:**\n",
    "\n",
    "\n",
    "100+ Tensor operations, including transposing, indexing, slicing,\n",
    "mathematical operations, linear algebra, random numbers, etc.,\n",
    "are described\n",
    "[here](http://pytorch.org/docs/torch).\n",
    "\n",
    "## NumPy Bridge\n",
    "\n",
    "Converting a Torch Tensor to a NumPy array and vice versa is a breeze.\n",
    "\n",
    "The Torch `Tensor` and NumPy `array` will share their underlying memory\n",
    "locations, and changing one will change the other.\n",
    "\n",
    "### Converting a Torch Tensor to a NumPy Array"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "sN5zoFkS1rT1"
   },
   "outputs": [],
   "source": [
    "a = torch.ones(5)\n",
    "print(a)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "Sx-qD8zV4Okq"
   },
   "outputs": [],
   "source": [
    "b = a.numpy()\n",
    "print(b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "HY_4kJjP1nGS"
   },
   "source": [
    "See how the numpy array changed in value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "avoIX_pV1rjs"
   },
   "outputs": [],
   "source": [
    "a += 1\n",
    "print(a)\n",
    "print(b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Ko_pjkCr1nKO"
   },
   "source": [
    "### Converting NumPy Array to Torch Tensor\n",
    "\n",
    "See how changing the np array changed the Torch Tensor automatically"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "2uv4BMX41r2b"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "a = np.ones(5)\n",
    "b = torch.from_numpy(a)\n",
    "np.add(a, 1, out=a)\n",
    "print(a)\n",
    "print(b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercices:**\n",
    "\n",
    "- L2-normalize the rows of a matrix\n",
    "\n",
    "- generate a random matrix then set the diagonal of a matrix to 0\n",
    "\n",
    "- generate a random matrix and make a matrix with 3x4 tiles of that matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "TJseasKM1nMm"
   },
   "source": [
    "All the Tensors on the CPU except a CharTensor support converting to\n",
    "NumPy and back.\n",
    "\n",
    "## CUDA Tensors\n",
    "\n",
    "Tensors can be moved onto any device using the ``.to`` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "aHFzRRwJ1sX5"
   },
   "outputs": [],
   "source": [
    "# let us run this cell only if CUDA is available\n",
    "# We will use ``torch.device`` objects to move tensors in and out of GPU\n",
    "if torch.cuda.is_available():\n",
    "    device = torch.device(\"cuda\")          # a CUDA device object\n",
    "    y = torch.ones_like(x, device=device)  # directly create a tensor on GPU\n",
    "    x = x.to(device)                       # or just use strings ``.to(\"cuda\")``\n",
    "    z = x + y\n",
    "    print(z)\n",
    "    print(z.to(\"cpu\", torch.double))       # ``.to`` can also change dtype together!"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "default_view": {},
   "name": "tensor_tutorial.ipynb",
   "provenance": [],
   "version": "0.3.2",
   "views": {}
  },
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.15"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}