corpus_analysis_tutorial.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.2"
    },
    "colab": {
      "name": "corpus_analysis_tutorial.ipynb",
      "provenance": [],
      "collapsed_sections": [
        "J1ql-Zu01tnJ",
        "HcO6DooZfLQz",
        "A59wVV6BfLQ9"
      ],
      "toc_visible": true,
      "include_colab_link": true
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/tuomaseerola/music21/blob/master/corpus_analysis_tutorial.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Vv5kdOJzfLPe"
      },
      "source": [
        "# Music21 Corpus Analysis\n",
        "\n",
        "**Music and Science**\n",
        "\n",
        "February 2022, Tuomas Eerola, Durham University, UK.\n",
        "\n",
        "This is a companion to the first `music21` demo. The Colab ([Google Colaboratory](http://colab.research.google.com/) idea is from Myke Cuthbert, which allows any computer to run `music21` analyses without fuss in a browser window. These demos have been built around the examples provided in `music21` [Documentation](http://web.mit.edu/music21/doc/index.html).\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "l4JOrFGjfLPg"
      },
      "source": [
        "# 1 Build music21 environment in Colab\n",
        "\n",
        "First we build a virtual machine that will be able run _music21_ in our browser.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KH6lywqEfLPh"
      },
      "source": [
        "## 1.1 Install Music21 and other elements needed to run the environment\n",
        "\n",
        "Press play and wait for all commands to be executed - this intiial command might take some time as it needs to build the music21 environment."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "64Pcp5eHfLPj"
      },
      "source": [
        "!pip install --upgrade music21\n",
        "!add-apt-repository ppa:mscore-ubuntu/mscore-stable -y\n",
        "!apt-get update\n",
        "!apt-get install musescore\n",
        "!apt-get install xvfb\n",
        "!sh -e /etc/init.d/x11-common start\n",
        "import os\n",
        "os.putenv('DISPLAY', ':99.0')\n",
        "!start-stop-daemon --start --pidfile /var/run/xvfb.pid --make-pidfile --background --exec /usr/bin/Xvfb -- :99 -screen 0 1024x768x24 -ac +extension GLX +render -noreset\n",
        "from music21 import *\n",
        "us = environment.UserSettings()\n",
        "us['musescoreDirectPNGPath'] = '/usr/bin/mscore'\n",
        "us['directoryScratch'] = '/tmp'"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r7Gc0kakfLPq"
      },
      "source": [
        "## 1.2 Let's check out that our environment works\n",
        "\n",
        "We will do this by trying to import one of the scores available in the music21 corpus. A list of available scores can be found at http://web.mit.edu/music21/doc/about/referenceCorpus.html \n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hmd3vjuofLP7"
      },
      "source": [
        "from music21 import *\n",
        "opus133 = corpus.parse('beethoven/opus133.mxl') # We \"parse\" one specific work from the corpus which in this case will be a work by Beethoven\n",
        "opus133.measures(1, 4).show() # This command lets us specify the number of bars (measures) we want to display, in this case we want to see bars 1 to 4"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Gu9zgODwfLQk"
      },
      "source": [
        "# 2 Corpus analysis\n",
        "In the previous seminar we got to know `music21` and extracted some interesting musical properties from single pieces of music. Now it is time to apply the analyses to a corpus, a collection of pieces. We will also follow a simillar structure to our last seminar, where we will be trying out different functions to analyse a corpus, and carrying out some learning tasks."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JjpFiqxIfLQl"
      },
      "source": [
        "## 2.1 Metadata of a work\n",
        "Let's work with the built-in corpus of *Music21*. The system has a neat architecture for searching, combining, and loading all pieces that have certain metadata. Also, it is typical that the pieces themselves contain different types of metadata."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "z4ZbF8o8fLQl"
      },
      "source": [
        "# What other information besides the score do we have about a piece of music? Let's try this out first with one work.\n",
        "opus3no1 = corpus.parse('corelli/opus3no1/1grave') # Let's import one movement of Corelli's Opus 3 work\n",
        "print(opus3no1.metadata.all()) # This function will allow us to see the work's metadata \n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "partStream = opus3no1.parts.stream() # Show the list of instruments\n",
        "for p in partStream:\n",
        "    print(p.id)"
      ],
      "metadata": {
        "id": "M_LeS5SvBDYO"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O02wm5tOyeoR"
      },
      "source": [
        "---\n",
        "#### Learning Task 1\n",
        "---\n",
        "Let's now look at the first four bars of the work. Do you remember which command you need to use to do this? NB: The object's name is `opus3no1`.\n"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "opus...       # Fill in the code to show the first four bars of the work we just loaded from Music21's built-in corpus."
      ],
      "metadata": {
        "id": "MQcIIarYBJMW"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 2.2 Corpus and metadata\n",
        "\n",
        "Let's pull up the metadata of a collection of works. First, we will search for all works by a particular composer (works that are available in the `Music21` library, see [Reference Corpus in Music21](https://web.mit.edu/music21/doc/about/referenceCorpus.html))."
      ],
      "metadata": {
        "id": "k6tzyVMVCaQb"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZtRce2tgykZX"
      },
      "source": [
        "# Let's select a corpus of all works composed by Giovanni Palestrina\n",
        "corpusPalestrina = corpus.search(composer='palestrina')\n",
        "print(corpusPalestrina) # let's see how many works there are"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "k_KCERr-yzSw"
      },
      "source": [
        "# From this corpus, let us find works that are titled 'Kyrie'\n",
        "corpusKyrie= corpusPalestrina.search(title='Kyrie') # we do this by filtering with the title function within our previously loaded object corpusPalestrina\n",
        "print(corpusKyrie)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OILxRxKoy3Zu"
      },
      "source": [
        "# What about even more specific compositions that have the word \"Papae\" in it, in Palestrina's Kyrie corpus?\n",
        "corpusPapae = corpusKyrie.search(parentTitle='Papae')\n",
        "print(corpusPapae)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Qa_7wqCkfLQn"
      },
      "source": [
        "#Let's display the first seven bars of the above entry\n",
        "ex1 =corpus.parse(corpusPapae[0]) #let's give a name to our object (ex1) and pull up the work that has the name Papae in its title\n",
        "ex1.measures(1,7).show()\n",
        "\n",
        "#Do you recognise this famous work by Palestrina?"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J1ql-Zu01tnJ"
      },
      "source": [
        "---\n",
        "#### Learning Task 2\n",
        "---\n",
        "Let's try to find other famous masses from the corpus just using the titles.\n",
        "\n",
        "First, let's see how many masses named Sine Nomine we can find in the corpus. Continue the code below to search within our corpusKyrie object for workks with 'Sine Nomine' as their parent title."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "ex3 =  corpusKyrie.search(ParentTitle='???') #fill in the code in the brackets for parentTitle of works to match 'Sine Nomine'\n",
        "print(ex3)"
      ],
      "metadata": {
        "id": "q6wcGR1ASxeb"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "How many entries have Sine Nomine as their parent title? "
      ],
      "metadata": {
        "id": "_-mAdiVIUCpw"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's do the same by now searching for works titled 'Missa Brevis'"
      ],
      "metadata": {
        "id": "OYmPUthVVJS1"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "ex4= corpusKyrie.search(parentTitle ='Missa Brevis')\n",
        "print(ex4)"
      ],
      "metadata": {
        "id": "I75-b9qKMFDR"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's create a new object called ex5 to display the first 7 bars of the Missa Brevis."
      ],
      "metadata": {
        "id": "ZZ2Sm7YuWZVn"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "ex5 =corpus.parse(ex4[0]) \n",
        "ex5.measures(1,7).show()"
      ],
      "metadata": {
        "id": "w_pzXj83VIB5"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can also display the full titles of particular works by Palestrina. For example, let's find out all the titles of Palestrina's Kyries."
      ],
      "metadata": {
        "id": "ybIxYTnGXT63"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XWSfRjiafLQp"
      },
      "source": [
        "for work in corpusKyrie:\n",
        "    score = corpus.parse(work)\n",
        "    print(score.metadata.parentTitle)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EawtVr5bfLQs"
      },
      "source": [
        "# 3 Corpus analysis of keys\n",
        "\n",
        "Were the Bach chorales written in specific keys? Perhaps the keys with only one or two sharps and flats were regularly utilised since they are easier to perform and notate? Are there more chorales in major mode than in minor? Let's find out.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 3.1 Key Distribution"
      ],
      "metadata": {
        "id": "SPXzEfTmbB6g"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "First, let's look at the key distribution across chorales."
      ],
      "metadata": {
        "id": "VCobWtdAbHfO"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6UPDdw9yfLQs"
      },
      "source": [
        "# We first select Bach 4-part works in music21\n",
        "chorales = corpus.search(composer='J.S. Bach',numberOfParts=4)\n",
        "print(chorales) # We print the number of chorales available"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lSUXcuKafLQu"
      },
      "source": [
        "# Next we analyse these 320 entries and extract the key from each of them. This will take some time.\n",
        "from music21 import*\n",
        "import matplotlib.pyplot as plt # Load some extra plotting libraries\n",
        "\n",
        "chorales = corpus.search(composer='J.S. Bach',numberOfParts=4)\n",
        "\n",
        "# For each chorale, it will analyse the key signature of the work and sort them out in major and minor\n",
        "# A print function is being used so we can see the progress of the command as it's running\n",
        "\n",
        "dict = {}\n",
        "dict2 = {}\n",
        "counter=1; maxlen = len(chorales) \n",
        "#maxlen = 10 # if want to limit this to a few chorales, uncomment this line and adjust the number \n",
        "for chorale in chorales[0:maxlen]:\n",
        "   print('Analysing', counter+1,'/',maxlen, chorale.metadata.title,'...')\n",
        "   score = corpus.parse(chorale)\n",
        "   key = score.analyze('key').tonicPitchNameWithCase\n",
        "   key2 = score.analyze('key').mode\n",
        "   dict[key] = dict[key] + 1 if key in dict.keys() else 1\n",
        "   dict2[key2] = dict2[key2] + 1 if key2 in dict2.keys() else 1\n",
        "   counter +=1\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8lC_x5MVfLQx"
      },
      "source": [
        "# Plot the results\n",
        "ind = [i for i in range(len(dict))]\n",
        "fig, ax = plt.subplots()\n",
        "ax.bar(ind, dict.values())\n",
        "ax.set_title('Frequency of Each Key')\n",
        "ax.set_ylabel('Frequency')\n",
        "plt.xticks(ind, dict.keys(), rotation='horizontal',size=12)\n",
        "plt.show()\n",
        "print(dict2) # print the frequency of major and minor keys"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HcO6DooZfLQz"
      },
      "source": [
        "---\n",
        "#### Learning Task 3\n",
        "---\n",
        "How do you interpret the key data of chorales? Which three key signatures were most frequent in Bach's chorales? \n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "K6Zcjt-ofLQ1"
      },
      "source": [
        "## 3.2 Clarity of Key\n",
        "\n",
        "It's good to know that within Bach chorales, there are also some modal chorales and sometimes the definition of the key is ambiguous. In the example above, we analysed the most likely key in each chorale. There are some chorales where the key is ambiguous, which can be explored by obtaining the **tonalCertainty** measure, which underlies the key analysis."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kD5WOCX-fLQ1"
      },
      "source": [
        "chorales = corpus.search(composer='J.S. Bach',numberOfParts=4) #let's load the Bach chorales again\n",
        "print(chorales)\n",
        "\n",
        "#Let's create a loop function that will analyse the chorales in relation to the key signatures and identify the key to which they have the highest correlation\n",
        "#Do you remember what the correlation coefficient tells us?\n",
        "\n",
        "c=[]\n",
        "title=[]\n",
        "counter=0; maxlen = len(chorales)\n",
        "#maxlen = 10 # if want to limit this to a few chorales, uncomment this line and adjust the number \n",
        "for chorale in chorales[0:maxlen]:\n",
        "   print('Analysing', counter+1,'/',maxlen, chorale.metadata.title,'...')\n",
        "   score = corpus.parse(chorale)\n",
        "   tc = score.analyze('key')\n",
        "   c.append(tc.correlationCoefficient) # get the correlation to the highest key\n",
        "   title.append(score.metadata.title)\n",
        "   counter +=1\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Am5ls1PbfLQ3"
      },
      "source": [
        "# Let's see what the correlations look like\n",
        "fig, ax = plt.subplots()\n",
        "ax.plot(c, 'bo:',markersize=5,markerfacecolor='r')\n",
        "fig.set_size_inches(20, 10)\n",
        "ax.set_ylabel('Correlation coefficient',size=15)\n",
        "ax.set_xlabel('Nro in corpus',size=15)\n",
        "plt.show()\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "S9IEL9cyfLQ5"
      },
      "source": [
        "# There are a few chorales where the correlations are lower than the other chorales, say under 0.85. \n",
        "# Let's look at those chorales\n",
        "\n",
        "ambiguous = [ n for n,i in enumerate(c) if i < 0.85 ] # get indices of tonally ambiguous chorales.\n",
        "print('Ambiguous keys can be found in:', ambiguous)\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# let's look at one of these\n",
        "score = corpus.parse(chorales[ambiguous[3]])\n",
        "tc = score.analyze('key')\n",
        "print(score.metadata.title,':',tc.correlationCoefficient)"
      ],
      "metadata": {
        "id": "jeBrfw2xclhr"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "score.show()\n",
        "\n",
        "# This is G minor Dorian"
      ],
      "metadata": {
        "id": "MzcAeTdZdPes"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "---\n",
        "#### Learning Task 4\n",
        "---\n",
        "Let's now have a look at the works with the highest correlation coefficient. How many of them have correlation coefficient above 0.98? "
      ],
      "metadata": {
        "id": "BY-Km3_sdtDC"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Let's look at the chorales with a coefficient higher than 0.98\n",
        "\n",
        "totalClarity = [ n for n,i in enumerate(c) if i > 0.98 ] # edit the code to print the number of the works which have a coefficient higher than 0.98\n",
        "print('Clarity in key can be found in:', totalClarity)"
      ],
      "metadata": {
        "id": "dmC_SAzhdbLV"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RUklZH5EfLQ6"
      },
      "source": [
        "# 4 Corpus analysis of vocal range\n",
        "\n",
        "Are the basses expected to sing over a larger range than the tenors?\n",
        "\n",
        "Has the vocal range been the same for SATB works over the centuries? Of course we do not always know what pitch was the score originally mapped onto but at least the vocal ranges should be comparable between soprano, alto, tenor and bass voices.\n",
        "\n",
        "Let's explore the vocal ranges."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "r1DJhXESfLQ7"
      },
      "source": [
        "import statistics\n",
        "# Start with Bach chorales\n",
        "chorales = corpus.search(composer='J.S. Bach',numberOfParts=4)\n",
        "\n",
        "soprano_range = []\n",
        "alto_range = []\n",
        "tenor_range = []\n",
        "bass_range = []\n",
        "for chorale in chorales:                                          # Loop across chorales\n",
        "    s = corpus.parse(chorale)\n",
        "    for el in s.recurse().parts:                                  # Loop across the parts\n",
        "        if 'Soprano' in el.partName:\n",
        "            soprano_range.append(el.analyze('range').semitones)   # Calculate ranges for different voices in semitones\n",
        "        if 'Alto' in el.partName:\n",
        "            alto_range.append(el.analyze('range').semitones)\n",
        "        if 'Tenor' in el.partName:\n",
        "            tenor_range.append(el.analyze('range').semitones)\n",
        "        if 'Bass' in el.partName:\n",
        "            bass_range.append(el.analyze('range').semitones)\n",
        "# Summarise the results\n",
        "print('Soprano', round(statistics.mean(soprano_range),2))\n",
        "print('Alto', round(statistics.mean(alto_range),2))\n",
        "print('Tenor', round(statistics.mean(tenor_range),2))\n",
        "print('Bass', round(statistics.mean(bass_range),2))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Which part had the largest range?"
      ],
      "metadata": {
        "id": "jmYCQRx-fPTs"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "A59wVV6BfLQ9"
      },
      "source": [
        "---\n",
        "#### Learning Task 5\n",
        "---\n",
        "\n",
        "Are the ranges of the voices specific to the Bach chorales or would similar results be evident in another, polyphonic vocal corpus? Let's try this using Monteverdi's vocal corpus. \n",
        "\n",
        "You could try Monteverdi (replace 'J.S. Bach' with 'Monteverdi', change the 'NumberOfParts' to 6)."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import statistics\n",
        "# Let's get Monteverdi's vocal works\n",
        "monteverdi = corpus.search(composer='Monteverdi',numberOfParts=6) #change composer to Monteverdi and number of parts to 6\n",
        "\n",
        "canto_range = []\n",
        "alto_range = []\n",
        "tenor_range = []\n",
        "basso_range = []\n",
        "for madrigal in monteverdi:                                          # Loop across chorales\n",
        "    s = corpus.parse(madrigal)\n",
        "    for el in s.recurse().parts:                                  # Loop across the parts\n",
        "        #print(el.offset, el, el.analyze('range').semitones)\n",
        "        #print(el.partName)\n",
        "        if 'Canto' in el.partName:\n",
        "            soprano_range.append(el.analyze('range').semitones)   # Calculate ranges for different voices in semitones\n",
        "        if 'Alto' in el.partName:\n",
        "            alto_range.append(el.analyze('range').semitones)\n",
        "        if 'Tenor' in el.partName:\n",
        "            tenor_range.append(el.analyze('range').semitones)\n",
        "        if 'Basso' in el.partName:\n",
        "            bass_range.append(el.analyze('range').semitones)\n",
        "# Summarise the results\n",
        "print('Canto', round(statistics.mean(soprano_range),2))\n",
        "print('Alto', round(statistics.mean(alto_range),2))\n",
        "print('Tenor', round(statistics.mean(tenor_range),2))\n",
        "print('Basso', round(statistics.mean(bass_range),2))"
      ],
      "metadata": {
        "id": "4JQ0tsSIfjx9"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eps7XyN4fLQ9"
      },
      "source": [
        "# 5 Corpus search\n",
        "\n",
        "Sometimes the useful approach is not to summarise the entire collection of music in terms of a specific feature but to search for a musical excerpt. Let's search the corpus for a theme that we have in mind. First we will select a suitable corpus of music and then search for a theme with or without the rhythms."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": true,
        "id": "Pk7bzAY6fLQ-"
      },
      "source": [
        "from music21 import *\n",
        "## Select all Bach Chorales\n",
        "chorales = corpus.search(composer='J.S. Bach')\n",
        "chorales2 = corpus.search('bwv364') # Let's get the Dorian chorale again\n",
        "bwv364=corpus.parse(chorales2[0]) # Put it into a specific variable for convenience\n",
        "bwv364.measures(0, 9).show() # Display the notation"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rkZHwcXgfLRB"
      },
      "source": [
        "## 5.1 Theme search (pitch only)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6xsqsydSfLRB"
      },
      "source": [
        "# define a theme to search. Let's first try a simple theme (G F E) without considering the rhythm.\n",
        "searchList = [note.Note('G'), note.Note('F'), note.Note('E')] # define a search pattern G-F-E\n",
        "s = bwv364.recurse().notes                 # prepares the piece for the search\n",
        "p = search.noteNameSearch(s, searchList) # executes the search\n",
        "\n",
        "# show were the exact matches were\n",
        "for notePosition in p:\n",
        "    startingNote = s[notePosition]\n",
        "    startingMeasure = startingNote.measureNumber\n",
        "    startingBeat = startingNote.beat\n",
        "    startingPart = startingNote.getContextByClass('Part')\n",
        "    print(startingNote, startingMeasure, startingBeat, startingPart)\n",
        "# This report below shows in which voice, in which bar and which beat, does the theme occur: "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "bwv364.measures(8, 9).show()"
      ],
      "metadata": {
        "id": "aw4rmrUBhLBe"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lgD9pUDhfLRD"
      },
      "source": [
        "## 5.2 Theme search (pitch and rhythm)\n",
        "The search above was simple and unrealistic. Let's search for a real theme with note durations.\n",
        "Let's find the theme from \"Vom Himmel hoch, da komm ich her\" (\"From Heaven Above to Earth I Come\"), which was supposedly composed by Luther in 1539. The theme has been used in Bach's Christmas oratorio, but which chorale does it come from? Here we want to preserve the rhythm as well."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "doRwKRDmfLRD"
      },
      "source": [
        "# Define the theme \"From Heaven Above to Earth I Come\")\n",
        "theme = converter.parse(\"tinynotation: 4/4 g4_From f#_hea- e_ven f#_above d_to e_earth f#_I g_come\")\n",
        "theme.show()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4NlrFFkRfLRG"
      },
      "source": [
        "# Here we want to preserve the approximate rhythm, but I will make all notes equally long (crotchets).\n",
        "searchStream2 = stream.Stream([key.KeySignature(1),\n",
        "                               note.Note('G4', type='quarter'),\n",
        "                               note.Note('F#4', type='quarter'),\n",
        "                               note.Note('E4', type='quarter'),\n",
        "                               note.Note('F#4', type='quarter'),\n",
        "                               note.Note('D4', type='quarter'),\n",
        "                               note.Note('E4', type='quarter'),\n",
        "                               note.Note('F#4', type='quarter'),\n",
        "                               note.Note('G4', type='quarter')])\n",
        "\n",
        "target1=[]\n",
        "target2=[]\n",
        "import time\n",
        "t = time.time()\n",
        "for i in range(100): # loop through the first 100 chorales ...\n",
        "    tmp = chorales[i].parse()\n",
        "    s = tmp.recurse().notes\n",
        "    for unused in range(12): # loop through different transpositions (up to 12 semitones)\n",
        "        s2=searchStream2.transpose(unused)\n",
        "        entryPoints = search.noteNameRhythmicSearch(s, s2.notes)\n",
        "        len1=len(entryPoints)\n",
        "        target1.append(len1)\n",
        "    len2=sum(target1)\n",
        "    target2.append(len2)\n",
        "    #print(i,target2[i])\n",
        "    target1=[]\n",
        "elapsed = time.time() - t\n",
        "print('Done! This search took',round(elapsed,1),'seconds')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xSXjlvCMfLRJ"
      },
      "source": [
        "# Display results\n",
        "hits=[i for i, x in enumerate(target2) if x]\n",
        "print(\"These works contain the theme:\",hits)\n",
        "\n",
        "catalog = stream.Opus()\n",
        "\n",
        "for i in range(0,len(hits)):\n",
        "    tmp=chorales[hits[i]].parse()\n",
        "    incipit = tmp.measures(0,3)\n",
        "    catalog.insert(0, incipit.implode())\n",
        "catalog.show() # Display the works that contain the theme"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hEYWfyTmfLRM"
      },
      "source": [
        "## 5.3 Search for the same theme in other collections"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IXDpXzF1fLRM"
      },
      "source": [
        "# Let's see if a similar note pattern has been used in other works such as Palestrina's works.\n",
        "\n",
        "palestrina = corpus.search('palestrina')\n",
        "print(palestrina)\n",
        "\n",
        "# let's allow some rhythmic variations and remove the note durations from the search\n",
        "searchStream3 = stream.Stream([key.KeySignature(1),\n",
        "                               note.Note('G4'),\n",
        "                               note.Note('F#4'),\n",
        "                               note.Note('E4'),\n",
        "                               note.Note('F#4'),\n",
        "                               note.Note('D4'),\n",
        "                               note.Note('E4'),\n",
        "                               note.Note('F#4'),\n",
        "                               note.Note('G4')])\n",
        "target1=[]\n",
        "target2=[]\n",
        "import time\n",
        "t = time.time()\n",
        "for i in range(50):\n",
        "    tmp = palestrina[i].parse()\n",
        "    s = tmp.recurse().notes\n",
        "    for unused in range(12): \n",
        "        s2=searchStream3.transpose(unused)\n",
        "        entryPoints = search.noteNameSearch(s, s2.notes)\n",
        "        len1=len(entryPoints)\n",
        "        target1.append(len1)\n",
        "    len2=sum(target1)\n",
        "    target2.append(len2)\n",
        "    print(i,target2[i])\n",
        "    target1=[]\n",
        "print('Done! This search took',round(elapsed,1),'seconds')\n",
        "    \n",
        "hits=[i for i, x in enumerate(target2) if x]\n",
        "print(\"These works contain the theme:\",hits)\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "coJUaEELfLRQ"
      },
      "source": [
        "# display one example\n",
        "print(hits)\n",
        "tmp=palestrina[hits[6]].parse()        # We are looking at the fifth example in the list above\n",
        "s = tmp.recurse().notes                 # prepares the piece for the search\n",
        "p = search.noteNameSearch(s, searchStream3) # executes the search\n",
        "target1=[]\n",
        "target2=[]\n",
        "for unused in range(12): \n",
        "    s2=searchStream3.transpose(unused)\n",
        "    entryPoints = search.noteNameSearch(s, s2.notes)\n",
        "    len1=len(entryPoints)\n",
        "    target1.append(len1)\n",
        "tr=[i for i, x in enumerate(target1) if x]\n",
        "\n",
        "# show were the exact matches were\n",
        "s2=searchStream3.transpose(tr[0])\n",
        "p = search.noteNameSearch(s, s2.notes) # executes the search\n",
        "for notePosition in p:\n",
        "    startingNote = s[notePosition]\n",
        "    startingMeasure = startingNote.measureNumber\n",
        "    startingBeat = startingNote.beat\n",
        "    startingPart = startingNote.getContextByClass('Part')\n",
        "    print(startingNote, startingMeasure, startingBeat, startingPart)\n",
        "tmp.measures(1, 6).show()\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "---\n",
        "#### Learning Task 6\n",
        "---\n",
        "What kind of matches with the search theme did you find? This is not the best way of doing the search as there are more efficient function in `music21`."
      ],
      "metadata": {
        "id": "MecLkB1mmdxH"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "There are more efficient ways of searching for melodic similarity in `music21`. See [User’s Guide, Chapter 43: Searching in and Among Scores](https://web.mit.edu/music21/doc/usersGuide/usersGuide_43_searching1.html) and [music21.search.segment](https://web.mit.edu/music21/doc/moduleReference/moduleSearchSegment.html) for more clever examples. \n"
      ],
      "metadata": {
        "id": "TVUXSu4qeNt2"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JSakGXCffLRU"
      },
      "source": [
        "# 6 References\n",
        "\n",
        "* Cuthbert, M. S., & Ariza, C. (2010). music21: A Toolkit for Computer-Aided Musicology and Symbolic Music Data. In J. Stephen Downie and Remco C. Veltkamp (Eds.). 11th International Society for Music Information Retrieval Conference (ISMIR 2010), August 9-13, 2010, Utrecht, Netherlands. pp. 637-642. [link](http://ismir2010.ismir.net/proceedings/ismir2010-108.pdf)\n",
        "\n",
        "* Cuthbert, M. S., Ariza, C., & Friedland, L. (2011). Feature Extraction and Machine Learning on Symbolic Music using the music21 Toolkit. In 11th International Society for Music Information Retrieval Conference (ISMIR 2011) (pp. 387--392). [link](http://ismir2011.ismir.net/papers/PS3-6.pdf)\n"
      ]
    }
  ]
}