week5.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Eigenvalues and eigenvectors\n",
    "\n",
    "Eigen means \"characteristic\", which can be contextualized in our study of linear algebra as the characteristic of vectors.\n",
    "\n",
    "Think of eigenvectors as the vectors that remains in the same span even after doing a transformation\n",
    "\n",
    "If you apply a scaling to a square (let's use a scale of 2 in the y axis) the X and Y axis vectors remains in the same \"line\" (same span) and so those are eigeinvectors (as in they are the characteristic of this transformtion). The vector of the angle of the square changes it's span when you apply this scaling, so he isn't a eigenvectors.\n",
    "\n",
    "We applied a scale of 2 in the y axis, so the vector of the axis now have twice its original size, this means that the this vector eigenvalue is 2. The x axis vectors didn't change directions and didn't change size, so its eigenvalue is 1 (it keeps the exact same vector.\n",
    "\n",
    "Some special cases of eigenvectors are, for instance, a uniform scalling, where all vectores are in fact, eigeinvectors.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Calculating Eigenvectors\n",
    "\n",
    "$Ax = \\lambda x$\n",
    "\n",
    "x is a eigenvectors, so when we multiply a transformation matrix (A) to the eigenvectors, we get the same eigenvectors scaled by a scalar (for a scalar of 1 the eigenvectors remains exactly the same) $\\lambda$.\n",
    "\n",
    "We want to solve this equation, and A is a nxn square matrix, so x must be a n dimensional vector.\n",
    "\n",
    "To solve, we first need to transform the equation to the following form:\n",
    "\n",
    "> $Ax = \\lambda x$ \n",
    "\n",
    "> $Ax - \\lambda x = 0$\n",
    "\n",
    "> $x(A-\\lambda) = 0$\n",
    "\n",
    "> $x(A-I\\lambda) = 0$\n",
    "\n",
    "(Note that we apply the identity matrix I to do the minus operation between a matrix and a scalar, this is just a math formality but it can be useful for writing code)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For this operation to be equal to 0: x or $(A-I\\lambda)$. We are not interested in the case where x is 0, since it means our egenvectors is null. We want to solve  $(A-I\\lambda)$ = 0\n",
    "\n",
    "We can identitify if the result of this operation is 0 if it's determinant is 0, so:\n",
    "\n",
    "det$(A-I\\lambda)$ = 0\n",
    "\n",
    "example for a 2x2 general case:\n",
    "\n",
    "det($\\begin{bmatrix} a & b \\\\ c & d \\end{bmatrix} - \\begin{bmatrix} \\lambda & 0 \\\\ 0 & \\lambda \\end{bmatrix} $) = 0\n",
    "\n",
    "given us the equation: $\\lambda^2 -(a+d)\\lambda + (ad-bc) = 0$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Practice exercises\n",
    "\n",
    "# 1 -\n",
    "\n",
    "Given a matrix $\\begin{bmatrix} a & b \\\\ c & d \\end{bmatrix}$ recall that one can calculate its eigenvalues by solving the characteristic polynomial $\\lambda^2 -(a+d)\\lambda + (ad-bc) = 0$. In this quiz, you will practice calculating and solving the characteristic polynomial to find the eigenvalues of simple matrices.\n",
    "For the matrix $A = \\begin{bmatrix} 1 & 0 \\\\ 0 & 2 \\end{bmatrix}$ what is the characteristic polynomial, and the solutions to the characteristic polynomial?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> $\\lambda^2 -(1+2)\\lambda + (2-0) = 0$\n",
    "\n",
    "> $\\lambda^2 - 3\\lambda + 2 = 0$\n",
    "\n",
    "> $\\lambda = 1$ or $\\lambda = 2$ (You can use bhaskara)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2 -\n",
    "\n",
    "For A = $\\begin{bmatrix} 1 & 0 \\\\ 0 & 2 \\end{bmatrix}$ as immediately above, select all eigenvectors of this matrix.\n",
    "\n",
    "Let's use our ol' formula: $x(A-I\\lambda) = 0$\n",
    "\n",
    "We have A, and we have our two $\\lambda$ values from exercise 1. we can subtract the matrix A when $\\lambda$ is 1, then multiply by x (our 2-dimensional vectors: x1 and x2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> $\\begin{bmatrix} 1-1 & 0 \\\\ 0 & 2-1 \\end{bmatrix}$ =  $\\begin{bmatrix} 0 & 0 \\\\ 0 & 1 \\end{bmatrix}$ x = 0\n",
    "\n",
    "> now multiply this resultant matrix with the vector x:\n",
    "\n",
    "> $\\begin{bmatrix} 0 & 0 \\\\ 0 & 1 \\end{bmatrix}$ * $\\begin{bmatrix} x_1 \\\\ x_2 \\end{bmatrix}$ = 0\n",
    "\n",
    "> $\\begin{bmatrix} 0 \\\\ x_2 \\end{bmatrix}$ = 0\n",
    "\n",
    "> for this equation to hold, x_2 must be 0 ( [0,0] = 0 ). But we didn't get information in $x_1$, this means that any vector in the x-axis can be a eigenvector for this solution: \n",
    "\n",
    "> $\\begin{bmatrix} t \\\\ 0 \\end{bmatrix}$\n",
    "\n",
    "Now we repeat the same steps for when $\\lambda$ is equal 2:\n",
    "\n",
    "> $\\begin{bmatrix} 1-2 & 0 \\\\ 0 & 2-2 \\end{bmatrix}$ =  $\\begin{bmatrix} -1 & 0 \\\\ 0 & 0 \\end{bmatrix}$ x = 0\n",
    "\n",
    "> now multiply this resultant matrix with the vector x:\n",
    "\n",
    "> $\\begin{bmatrix} -1 & 0 \\\\ 0 & 0 \\end{bmatrix}$ * $\\begin{bmatrix} x_1 \\\\ x_2 \\end{bmatrix}$ = 0\n",
    "\n",
    "> $\\begin{bmatrix} -x_1 \\\\ 0 \\end{bmatrix}$ = 0\n",
    "\n",
    "> for this equation to hold, x_1 must be 0 ( [0,0] = 0 ). But we didn't get information in $x_2$, this means that any vector in the y-axis can be a eigenvector for this solution:\n",
    "\n",
    "> $\\begin{bmatrix} 0 \\\\ t \\end{bmatrix}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Tyding all up, the solutions for this exercise is any case where x_1 is a number and x_2 is 0 **or** when x_1 is 0 and x_2 is any number.\n",
    "So the eigenvectors in the exercise are: \n",
    "\n",
    "$\\begin{bmatrix} 1 \\\\ 0 \\end{bmatrix}$\n",
    "\n",
    "$\\begin{bmatrix} 0 \\\\ 3 \\end{bmatrix}$\n",
    "\n",
    "$\\begin{bmatrix} 0 \\\\ 2 \\end{bmatrix}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Changing to a eigenbasis\n",
    "\n",
    "When we have to do multiplications of the same transformation matrix (given a matrix T: T * T * T and so on), if the matrix is diagonal (only elements in the principal diagonal) we can elevate T to the potency of multiplications, i.e: T * T * T = $T^3$ and elevate all components of the principal diagonal to the potency to get the result:\n",
    "\n",
    "$T^3 = \\begin{bmatrix} 1^3 & 0 \\\\ 0 & 2^3 \\end{bmatrix} = \\begin{bmatrix} 1 & 0 \\\\ 0 & 8 \\end{bmatrix}$\n",
    "\n",
    "but if the matrix is not diagonal, we need to change to a basis where our matrix becomes diagonal: A eigenbasis. Then we do the potency to this eigenbasis matrix, and change basis again to our original basis. \n",
    "\n",
    "\n",
    "\n",
    "To achieve this, first we need to find our eigenvectors and put then as columns in a new matrix, let's call it C. This works because each column of our transformation matrix T, simple represents the new location of the transformed unit vectors. \n",
    "\n",
    "So when we multiply T * C we are just scaling the matrix (remember that eigenvectos only scale the matrix?). This means now we can represent this operation as a diagonal matrix (call it D), this diagonal matrix contains the corresponding eigens values of the matrix T.\n",
    "\n",
    "This gives us the final formula that let's us find $T^n$ for when our T isnt a diagonal matrix:\n",
    "\n",
    "> $T^n = CD^nC^{-1}$\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> $\\begin{bmatrix} 1-1 & 1 \\\\ 0 & 2-1 \\end{bmatrix}$ =  $\\begin{bmatrix} 0 & 1 \\\\ 0 & 1 \\end{bmatrix}$ x = 0\n",
    "\n",
    "> $\\begin{bmatrix} 0 & 1 \\\\ 0 & 1 \\end{bmatrix}$ $\\begin{bmatrix} x1 \\\\ x2 \\end{bmatrix}$= 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> $\\begin{bmatrix} x2 \\\\ x2 \\end{bmatrix}$ = 0 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> $\\begin{bmatrix} 1-2 & 1 \\\\ 0 & 2-2 \\end{bmatrix}$ =  $\\begin{bmatrix} -1 & 1 \\\\ 0 & 0 \\end{bmatrix}$ x = 0\n",
    "\n",
    "> $\\begin{bmatrix} -1 & 1 \\\\ 0 & 0 \\end{bmatrix}$ $\\begin{bmatrix} x1 \\\\ x2 \\end{bmatrix}$= 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Practice exercises - diagonalisation and applications\n",
    "\n",
    "# 1 - \n",
    "Given the matrix $T = \\begin{bmatrix} 6 & -1 \\\\ 2 & 3\\end{bmatrix}$\n",
    "and change of basis matrix $C = \\begin{bmatrix} 1 & 1 \\\\ 1 & 2\\end{bmatrix}$ (whose columns are eigenvectors of T), calculate the diagonal matrix $D = C^{-1}TC$.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$C^{-1} = \\begin{bmatrix} 2 & -1 \\\\ -1 & 1 \\end{bmatrix}$\n",
    "\n",
    "> $\\begin{bmatrix} 2 & -1 \\\\ -1 & 1 \\end{bmatrix} \\times \\begin{bmatrix} 6 & -1 \\\\ 2 & 3\\end{bmatrix}$ = \n",
    "\n",
    "> = $\\begin{bmatrix} 10 & -5 \\\\ -4 & 4 \\end{bmatrix} \\times \\begin{bmatrix} 1 & 1 \\\\ 1 & 2\\end{bmatrix}$ =\n",
    "\n",
    "> = $\\begin{bmatrix} 5 & 0 \\\\ 0 & 4 \\end{bmatrix}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5 - \n",
    "\n",
    "Given that $T = \\begin{bmatrix} 6 & -1 \\\\ 2 & 3\\end{bmatrix} = \\begin{bmatrix} 1 & 1 \\\\ 1 & 2\\end{bmatrix} \\begin{bmatrix} 5 & 0 \\\\ 0 & 4\\end{bmatrix} \\begin{bmatrix} 2 & -1 \\\\ -1 & 1\\end{bmatrix}$, calculate $T^3$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Remember our eigen formula for matrix potencies:\n",
    "\n",
    "$T^n = CD^nC^{-1}$\n",
    "\n",
    "The exercise gives us the full multiplication, we only need to add potency to the D matrix (in this case, its elevated to 3):\n",
    "\n",
    "$T^3 =\\begin{bmatrix} 1 & 1 \\\\ 1 & 2\\end{bmatrix} \\begin{bmatrix} 5 & 0 \\\\ 0 & 4\\end{bmatrix}^3 \\begin{bmatrix} 2 & -1 \\\\ -1 & 1\\end{bmatrix}$ =\n",
    "\n",
    "> = $\\begin{bmatrix} 1 & 1 \\\\ 1 & 2\\end{bmatrix} \\begin{bmatrix} 125 & 0 \\\\ 0 & 64 \\end{bmatrix} \\begin{bmatrix} 2 & -1 \\\\ -1 & 1\\end{bmatrix}$ =\n",
    "\n",
    "> = $\\begin{bmatrix} 186 & -61 \\\\ 122 & 3\\end{bmatrix}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}