HW4_LDE.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CTtrdqGpCpIZ"
      },
      "source": [
        "# Homework 4 - Low Dimensional Embedding\n",
        "This notebook provides some skeleton code to get you started on the homework. Add in your own code and markdown cells to answer the homework questions. If you want to submit the notebook as a PDF, make sure your code and markdowns are clear and concise to make grading easy for the TAs.\n",
        "This notebook can be opened in Colab Open In Colab\n",
        "\n",
        "Before starting, select \"Runtime->Factory reset runtime\" to start with your directories and environment in the base state.\n",
        "\n",
        "If you want to save changes to the notebook, select \"File->Save a copy in Drive\" from the top menu in Colab. This will save the notebook in your Google Drive.\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zr4vM1Sd4RQd"
      },
      "source": [
        "# Clones, imports, and **GPU**"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Use a GPU\n",
        "\n",
        "You will find this homework much easier if you use a GPU.  Make sure your runtime is set to a GPU.  "
      ],
      "metadata": {
        "id": "mk6wms--l4qB"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "5O8Y-XweyEbA"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xxAnuxWhC8Dx"
      },
      "source": [
        "## Clone GitHub Repository\n",
        " Make sure to change directory to `\"social_media_analytics\"`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "nu5VjT4uC81x"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Install UMAP"
      ],
      "metadata": {
        "id": "yDv1L0AmlfoO"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "BkB_THcSlhhN"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bhvV8ImiCpIa"
      },
      "source": [
        "## Import Packages"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "y3C8G8FfCpIa"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H92DV5YyCpIc"
      },
      "source": [
        "# Problem 1. Load Image Data (17 points)\n",
        "\n",
        "Our dataset for this assignment will be images from the Instagram accounts of Cristiano Ronaldo and Lionel Messi, the two greatest football players of this generation.\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DG9ifhLjCpIc"
      },
      "source": [
        "#### 1. (6 points) Load Messi data\n",
        "\n",
        "Load the image filenames in the folder `\"data/image_compressed_leomessi\"` into a list.  Add the list to a dataframe called `df_messi` as a column `image_path`.  Also add a column to the dataframe called `screen_name` that equals `\"leomessi\"`.    How many image filenames in this dataframe?  Print out your answer.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2bxI5u3vCpId"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### 1. (6 points) Load Ronaldo data\n",
        "\n",
        "Load the image filenames in the folder `\"data/image_compressed_cristiano\"` into a list.  Add the list to a dataframe called `df_ronaldo` as a column `image_path`. Also add a column to the dataframe called `screen_name` that equals `\"cristiano\"`.   How many image filenames in this dataframe?  Print out your answer."
      ],
      "metadata": {
        "id": "-gblh5zxzLcD"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "0HR4cZ_xzjl3"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### 3. (3 points) Concatenate Dataframes\n",
        "\n",
        "Concatenate `df_messi` and `df_ronaldo` into one big dataframe along `axis=0` and call it `df`.  The `pd.concat` function may be useful here (ask the AI).  Print out the head of the dataframe."
      ],
      "metadata": {
        "id": "tUlc3zut0YO0"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "H7sf6Z6V1Syb"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### 4. (2 points) Histogram of Screen Names\n",
        "\n",
        "Make a `countplot` of the `screen_name` column in `df`.  Make sure your plot looks nice."
      ],
      "metadata": {
        "id": "RYLX-Ph41zqj"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "nsPMB0b117oj"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Problem 2.  Image Emeddings (41 points)\n",
        "\n",
        "Now we will calculate all sorts of embeddings of the images."
      ],
      "metadata": {
        "id": "0MbjPNZn2ET0"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Atsr_qBkCpIe"
      },
      "source": [
        "#### 1. (14 points) Calculate image embeddings.\n",
        "\n",
        "Calculate the embedding of all of the images in `df` using the Resnet50 model (you will need to convert the `image_path` column into a list, ask the AI).    Make sure you use the GPU provided in Colab to make things faster.  Also make sure to standardize the embeddings as was done in class. Save the embeddings to an array called `resnet_embedding`.  Print the shape of this embedding array.  \n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "UQgB12S5CpIf"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "blOMzPjMCpIh"
      },
      "source": [
        "#### 2. (5 points)  PCA Embedding\n",
        "\n",
        "Compute the two dimensional PCA embedding of `resnet_embedding`.  Save the two PCA components as columns in `df` called `\"pca_x\"` and `\"pca_y\"`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "xRaXWN9GCpIi"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x4SG1Fa5CpIi"
      },
      "source": [
        "#### 4. (5 points) t-SNE Embedding\n",
        "\n",
        "  Compute the two dimensional t-SNE embedding of `resnet_embedding`.  Save the two t-SNE components as columns in `df` called `\"tsne_x\"` and `\"tsne_y\"`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2-cuG6gLCpIi"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eUMTofBfCpIj"
      },
      "source": [
        "#### 5. (5 points) UMAP Embedding\n",
        "\n",
        "  Compute the two dimensional UMAP embedding of `resnet_embedding`.  Set `n_neighbors=10`, `min_dist=0.1`, and `metric='euclidean`.  Save the two UMAP components as columns in `df` called `\"umap_x\"` and `\"umap_y\"`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Kz_mlxfjCpIj"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Advice: Save Dataframe\n",
        "\n",
        "At this point, we suggest you save `df` to a csv file.  This way you can do the rest of the homework without having to recompute the image embeddings with ResNet50."
      ],
      "metadata": {
        "id": "YWSGaGfJ6HmG"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RMaWTUjZCpIj"
      },
      "source": [
        "#### 6. (7 points)  Scatter Plot of Embeddings\n",
        "\n",
        "Make a scatter plot of each embedding (just dots, no images in the plot) and make the `hue` equal to `screen_name`.  Include a title with the embedding name for each plot.    Which embedding do you find most appealing and why?  Print out your answer or write it in a text cell."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GtkdJUWdCpIj",
        "scrolled": true
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Which embedding do you like best?**\n",
        "\n",
        "**Write your answer in this cell**"
      ],
      "metadata": {
        "id": "-b35qNPh6Xpw"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### (5 points) Scatter Plot of UMAP Embeddings with Images\n",
        "\n",
        "Make a scatterplot of the UMAP embeddings, but this time put a scaled down version of each image at the embedding locations, as was done in class."
      ],
      "metadata": {
        "id": "vLV2ryiUpTlZ"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "f4tcCKgSphc-"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VnNytKv5CpIk"
      },
      "source": [
        "# Problem 3:  Clustering Images (42 points)\n",
        "\n",
        "Now we will analyze the images by clustering their embeddings."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AOeI4vbZCpIk"
      },
      "source": [
        "#### 1. (6 points)  2 Clusters\n",
        "\n",
        "Apply K-means clustering to the UMAP embedding with `k=2` clusters, `n_init='auto'` and `random_state = 0`.  Add these labels to `df` in a column called `kmeans_label_2`.  Make a `countplot` of the `screen_name` distribution for each cluster.  Are the two clusters seperating the two football players Instagram content?"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "S95M7zB7CpIk"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### 2. (10 points) Optimal K-Means Clusters\n",
        "\n",
        "Calculate the silhouette score of the K-means clusters on the UMAP embedding for `k = 1,2,..., 10`.  Plot the silhouette score vs. `k`.  Which value of `k` has the highest score?  "
      ],
      "metadata": {
        "id": "G1Xd_Tog8Yxm"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "VieFR-UF8wIa"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "w49WyEzZCpIm"
      },
      "source": [
        "#### 4. (6 points) K-means Clusters with a \"Good\" k\n",
        "\n",
        "Calculate the K-means clusters of the UMAP embedding using four clusters.  This may not be optimal, but sometimes the silhouette score is not perfect.  Add a column to the dataframe with the tweets called `\"kmeans_label\"` that contains the K-means cluster labels.  Make a `countplot` of the `screen_name` for each cluster.  \n",
        "\n",
        "**Optional**: If you want to make it look nicer, use a 2 x 2 subplot for the four count plots.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "uMw7uBbgCpIm"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QqU5egTFgRvA"
      },
      "source": [
        "#### 6. (6 points) Scatterplot of Clusters\n",
        "\n",
        "Make two `scatterplot` of the UMAP embeddings.  One has `hue` as the `screen_name` and one has `hue` as `kmeans_label`.  Use the `subplot` function so the plots are side by side."
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "vAvzB9ZWXfe1"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### 4. (13 points) Sampling Images in Clusters\n",
        "\n",
        "Plot 16 samples from each cluster in a 4 x 4 grid, and add the `kmeans_label` to the title for each figure.\n",
        "\n",
        "**Advice** A `figsize` of `(9,9)` per cluster will look nice."
      ],
      "metadata": {
        "id": "CJUlUoO7_QXs"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "e3GsTX1GXggk"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### 5. (1 point) Insights\n",
        "\n",
        "Based on your analysis of the prevalence of each `screen_name` in each `kmeans_label` cluster, along with your samples of the images in each cluster, provide a summary of what types of images are in the clusters and who posts that type of image more frequently.  \n",
        "\n",
        "**Optional:** For fun, you can try to infer why each football player posts that type of image."
      ],
      "metadata": {
        "id": "Pj3oiexEB1ix"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Write your answer in this cell**"
      ],
      "metadata": {
        "id": "B_HmoRNcXj4m"
      }
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "provenance": [],
      "gpuType": "T4",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.2"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}