workshop-3_Process_Image.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Workshop Notebook 3: Process a Single Image with Pillow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Mandatory Disclosures\n",
    "\n",
    "1. This is a whirlwind introduction, not exhaustive instruction\n",
    "1. All images are by courtesy of the University Archives at Texas State University: http://www.univarchives.txstate.edu\n",
    "1. img_qc_workshop is licensed under the GNU General Public License v3.0, https://github.com/photosbyjeremy/img_qc_workshop/blob/master/LICENSE\n",
    "1. *Any and all code provided is done so without any warranty or expectation of support by Jeremy Moore, Todd Peters, or Texas State University*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "code_folding": [
     0
    ],
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# import necessary modules\n",
    "from pathlib import Path\n",
    "from PIL import Image\n",
    "import matplotlib.pyplot as plt\n",
    "import img_qc.img_qc as img_qc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "code_folding": [
     0
    ]
   },
   "outputs": [],
   "source": [
    "# matplotlib options\n",
    "\n",
    "# magic that lets us plot directly in the notebook\n",
    "%matplotlib inline\n",
    "\n",
    "# parameters for matplotlib to increase our default figure size -- NOTE: figure sizes are in INCHES\n",
    "plt.rcParams[\"figure.figsize\"] = (20, 12)  # set as needed for your screen and eyes (width, height)\n",
    "\n",
    "# on a high-dpi monitor this will increase the quality of plots on-screen\n",
    "%config InlineBackend.figure_format = 'retina'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "code_folding": [
     0
    ],
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# set current_directory with Path function cwd() [Current Working Directory]\n",
    "current_directory = Path.cwd()\n",
    "\n",
    "# path to access our image\n",
    "image_path = current_directory.joinpath('data/workshop-3/AS-36-T4-E9-1943-c2_0002.tif')\n",
    "\n",
    "print(f'current_directory: {current_directory}')\n",
    "print(f'image_path: {image_path}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "code_folding": [
     0
    ]
   },
   "outputs": [],
   "source": [
    "# open image with Pillow\n",
    "image = Image.open(image_path)\n",
    "\n",
    "# display image with MatPlotLib\n",
    "plt.imshow(image)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Rotate Image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "code_folding": [
     0
    ]
   },
   "outputs": [],
   "source": [
    "# rotate image 23 degrees clockwise\n",
    "rotated_image = image.rotate(-23)  # negative angle is Clockwise\n",
    "\n",
    "# show rotated image\n",
    "plt.imshow(rotated_image)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Interpolation\n",
    "\n",
    "Rotating in anything other than 90 degree increments will result in the interpolation of pixel data -- the computer has to make up new tones. There are different algorithms available for interpolation while rotating and there's a trade-off between performance (how intensive the calculations are) and quality.\n",
    "\n",
    "`shift+tab` keyboard shortcut will show options\n",
    "\n",
    "Performance < ----- > Quality\n",
    "- Nearest Neighbor <> Bilinear <> Bicubic\n",
    "\n",
    "Always TEST and VERIFY which algorithm is right for your use case.\n",
    "\n",
    "http://pillow.readthedocs.io/en/5.1.x/handbook/concepts.html#filters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "code_folding": [
     0
    ]
   },
   "outputs": [],
   "source": [
    "# rotate with higher quality interpolation and expand size to not crop\n",
    "rotated_image = image.rotate(-23, resample=Image.BICUBIC, expand=True)  # negative angle is Clockwise\n",
    "\n",
    "# show rotated image\n",
    "plt.imshow(rotated_image)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Resize with Image.resize()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "code_folding": [
     0
    ]
   },
   "outputs": [],
   "source": [
    "# resize image\n",
    "image_resized = image.resize((500, 500))  # (width, height)\n",
    "\n",
    "# show resized image\n",
    "plt.imshow(image_resized)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Resize with image.thumbnail() WARNING!!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "code_folding": [
     0
    ]
   },
   "outputs": [],
   "source": [
    "# create a copy of our image as Image.thumbnail() MODIFIES THE IMAGE IN-PLACE\n",
    "thumbnail = image.copy()\n",
    "\n",
    "# resize the image with thumbnail\n",
    "thumbnail.thumbnail((500,500))\n",
    "\n",
    "# show the resized image\n",
    "plt.imshow(thumbnail)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Resize with img_qc.get_image_resized_pillow()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "code_folding": [
     0
    ]
   },
   "outputs": [],
   "source": [
    "# resize image\n",
    "image_resized = img_qc.get_image_resized_pillow(image, width=500)  # (width, height)\n",
    "\n",
    "# show resized image\n",
    "plt.imshow(image_resized)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Jupyter Magic: %timeit\n",
    "Compare the speed of 3 different interpolation settings for rotate on image_resized"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "code_folding": [
     0
    ]
   },
   "outputs": [],
   "source": [
    "# create resample dictionary with names and Pillow resize methods\n",
    "resample_dictionary = {'Nearest Neighbor': Image.NEAREST, 'Bilinear 2x2': Image.BILINEAR, 'Bicubic 4x4': Image.BICUBIC}\n",
    "\n",
    "for name in resample_dictionary:\n",
    "    \n",
    "    print(name)\n",
    "    \n",
    "    # get resample_method by accessing the resample_dictionary with the key `name`\n",
    "    resample_method = resample_dictionary[name]\n",
    "    \n",
    "    # call magic %timeit to time this line in our loop\n",
    "    %timeit rotated_image = image_resized.rotate(-23, resample=resample_method, expand=True)  # negative angle is Clockwise"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Crop\n",
    "\n",
    "As previously mentioned, digital images are graphs of pixels channels and intensities in a 2D plane.\n",
    "\n",
    "The graph's origin (0, 0) is the starting point of the image and the x-value increases with each pixel of width.\n",
    "\n",
    "Even though we're technically graphing into negative y-values, the y-value increases with each pixel of height and extends BELOW the image.\n",
    "\n",
    "We will use this coordinate system to crop our image.\n",
    "\n",
    "http://pillow.readthedocs.io/en/5.1.x/handbook/concepts.html#coordinate-system"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "code_folding": [
     0
    ]
   },
   "outputs": [],
   "source": [
    "# Pillow needs a box with upper-left (x, y) values and lower-right (x, y) values to crop an image\n",
    "image_cropped = image.crop(box=(0, 0, 500, 500))  # start in upper-left and go right 500 pixels, down 500 pixels\n",
    "\n",
    "plt.imshow(image_cropped)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "code_folding": [
     0
    ]
   },
   "outputs": [],
   "source": [
    "# show image\n",
    "plt.imshow(image)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "code_folding": [
     0
    ]
   },
   "outputs": [],
   "source": [
    "# crop image around page and color bar\n",
    "image_cropped = image.crop(box=(2450, 800, 6450, 6000))\n",
    "\n",
    "# print width & height\n",
    "print(f'width: {image_cropped.size[0]}')  # (width, height)\n",
    "print(f' height: {image_cropped.size[1]}')\n",
    "\n",
    "# show image\n",
    "plt.imshow(image_cropped)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Channels\n",
    "\n",
    "Our RGB image has 3 color channels that we can access using Pillow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "code_folding": [
     0
    ]
   },
   "outputs": [],
   "source": [
    "# split into separate channels\n",
    "red_channel, green_channel, blue_channel = image_cropped.split()\n",
    "\n",
    "figure, (red, green, blue) = plt.subplots(ncols=3, figsize=(18, 8)) # figsize is (width, height) in inches\n",
    "\n",
    "red.imshow(red_channel)\n",
    "red.set_title(\"Red Channel\")\n",
    "\n",
    "green.imshow(green_channel)\n",
    "green.set_title(\"Green Channel\")\n",
    "\n",
    "blue.imshow(blue_channel)\n",
    "blue.set_title(\"Blue Channel\")\n",
    "\n",
    "# some MatPlotLib code that draws subplots close together while padding axes so they don't overlap\n",
    "plt.tight_layout()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "code_folding": [
     0
    ]
   },
   "outputs": [],
   "source": [
    "# Let's crop our cropped image down to the color bar to better see our different color channels\n",
    "color_bar = image_cropped.crop(box=(0, 4600, 4000, 5200))\n",
    "\n",
    "plt.imshow(color_bar)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "code_folding": [
     0
    ]
   },
   "outputs": [],
   "source": [
    "# split into separate channels\n",
    "red_channel, green_channel, blue_channel = color_bar.split()\n",
    "\n",
    "figure, (color, red, green, blue) = plt.subplots(nrows=4, figsize=(20, 15))  # figsize is (width, height) in inches\n",
    "\n",
    "color.imshow(color_bar)\n",
    "# MatPlotLib code to remove the tick marks on the respective x- and y-axis\n",
    "color.set_xticks([]), color.set_yticks([])\n",
    "color.set_title(\"RGB Image\")\n",
    "\n",
    "red.imshow(red_channel)\n",
    "red.set_xticks([]), red.set_yticks([])\n",
    "red.set_title(\"Red Channel\")\n",
    "\n",
    "green.imshow(green_channel)\n",
    "green.set_xticks([]), green.set_yticks([])\n",
    "green.set_title(\"Green Channel\")\n",
    "\n",
    "blue.imshow(blue_channel)\n",
    "blue.set_xticks([]), blue.set_yticks([])\n",
    "blue.set_title(\"Blue Channel\")\n",
    "\n",
    "#plt.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Convert to Grayscale\n",
    "\n",
    "http://pillow.readthedocs.io/en/5.1.x/handbook/tutorial.html?highlight=convert#color-transforms\n",
    "\n",
    "When converting from color to grayscale a choice is made on how much to weigh the intensity of each color band on each pixel. By default, Pillow uses the ITU-R 601-2 luma transform: \n",
    "\n",
    "`L(umninance) = Red * 299/1000 + Green * 587/1000 + Blue * 114/1000`\n",
    "\n",
    "http://pillow.readthedocs.io/en/5.1.x/reference/Image.html#PIL.Image.Image.convert"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "code_folding": [
     0
    ]
   },
   "outputs": [],
   "source": [
    "# convert image to grayscale\n",
    "image_grayscale = color_bar.convert(mode='L')\n",
    "\n",
    "# parameters for matplotlib to increase our default figure size -- NOTE: figure sizes are in INCHES\n",
    "plt.rcParams[\"figure.figsize\"] = (20,12)  # set as needed for your screen and eyes (width, height)\n",
    "\n",
    "# show grayscale image\n",
    "plt.imshow(image_grayscale)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Convert to Bitonal with Image.convert()\n",
    "\n",
    "When converting from color (`RGB`) or grayscale (`L`) to Bitonal (`1`) using Image.convert the default is to use the Floyd-Steinberg dither, which we DON'T want if we're converting images for OCR\n",
    "\n",
    "We can alternatively set the dither to Image.NONE, but this just converts non-zero to white according to the documentation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "code_folding": [
     0
    ]
   },
   "outputs": [],
   "source": [
    "# convert image to bitonal with Floyd-Steinberg dithering\n",
    "image_bitonal_floyd_steinberg = image_grayscale.convert(mode='1')\n",
    "\n",
    "# convert image to bitonal with no dithering\n",
    "image_bitonal_no_dithering = image_grayscale.convert(mode='1', dither=Image.NONE)\n",
    "\n",
    "plt.figure()\n",
    "# show bitonal image with Floyd-Steinberg dithering\n",
    "plt.imshow(image_grayscale)\n",
    "\n",
    "plt.figure()\n",
    "# show bitonal image with Floyd-Steinberg dithering\n",
    "plt.imshow(image_bitonal_floyd_steinberg)\n",
    "\n",
    "plt.figure()\n",
    "# show bitonal image with no dithering\n",
    "plt.imshow(image_bitonal_no_dithering)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The documentation is wrong\n",
    "\n",
    "If you look at our third image with `dither=Image.NONE`, it actually looks like it used a 50% threshold. Values up to 50% of the intensity range round down to 0 (black) and those above 50% round up to 255 (white)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Convert to Bitonal with Image.point()\n",
    "\n",
    "We can convert our image to bitonal and choose a threshold using the Image.point() function.\n",
    "\n",
    "Image.point() does something to every pixel in the image. We can set a threshold value and use `lambda` with point to process each pixel in the image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "code_folding": [
     0
    ]
   },
   "outputs": [],
   "source": [
    "# set threshold value on 0-255 scale\n",
    "threshold = 85\n",
    "\n",
    "# threshold image where every pixel with value \n",
    "image_bitonal_85 = image_grayscale.point(lambda pixel: pixel > threshold and 255)\n",
    "\n",
    "plt.figure()\n",
    "# show image\n",
    "plt.imshow(image_bitonal_85)\n",
    "\n",
    "# set threshold value on 0-255 scale\n",
    "threshold = 120\n",
    "\n",
    "# threshold image where every pixel with value \n",
    "image_bitonal_120 = image_grayscale.point(lambda pixel: pixel > threshold and 255)\n",
    "\n",
    "plt.figure()\n",
    "# show image\n",
    "plt.imshow(image_bitonal_120)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Process Page into Bitonal"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "code_folding": [
     0
    ]
   },
   "outputs": [],
   "source": [
    "# crop scan down to the page\n",
    "image_page_crop = image_cropped.crop(box=(300, 125, 3800, 4600))\n",
    "\n",
    "# convert to grayscale\n",
    "image_page_gray = image_page_crop.convert(mode='L')\n",
    "\n",
    "# set threshold value\n",
    "threshold = 185\n",
    "\n",
    "# convert to bitonal with Image.point() method\n",
    "image_page_bitonal = image_page_gray.point(lambda pixel: pixel > threshold and 255)\n",
    "\n",
    "# show image\n",
    "plt.imshow(image_page_bitonal)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}