music21_demo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Music21 Tutorial\n",
    "*Music and Science, January 2019*\n",
    "\n",
    "Tuomas Eerola\n",
    "\n",
    "## 1 Introduction\n",
    "\n",
    "*Music21* is a Python “toolkit for analyzing, searching, and transforming music in symbolic (score-based) forms” (Cuthbert 2010). It is a very powerful musicological tool which has been used for various analytical tasks (Cuthbert 2011) and rhythm extraction in polyphonic music (Levé el al. 2011). *Music21* reads files in the humdrum and MusicXML formats, which makes large corpora of music such as CCARH’s Kern Scores collection (https://kern.ccarh.org/)  available. Together with *IPython* (Pérez; Granger 2007), an advanced Python shell, *Music21* is an appealing environment to explore, analyze and process music interactively.\n",
    "\n",
    "*Music21* has an excellent web-site, which contains instructions from installation to complex musical analyses: http://web.mit.edu/music21/\n",
    "\n",
    "This tutorial is my adaptation of the easy and basic demonstrations available in *Music21 User's Guide (http://web.mit.edu/music21/doc/usersGuide/index.html), although I have customised all examples to be concise and relevant for this Module.\n",
    "\n",
    "### 1.2 Basics\n",
    "\n",
    "Music21 runs in *python* (preferably in Python 3), which is versatile and open programming environment that is freely available for all operating systems. Python is considered as one of the easiest languages to learn and is often taught as a first programming language. You don’t need to be a seasoned programmer; just a little bit of Python and you will be able to get started and explore music in new ways with *music21*.\n",
    "\n",
    "In our facilities, we have set the latest Python and *Music21* in the machines. Therefore it should be really easy to start using *Music21*. In this demo, you have already launched music21 using a so-called \"jupyter notebook\" that gives you this dynamic notebook that you are seeing right now, where you can see separate `code blocks`, run them easily, and you can also alter the commands and see the results added to the web page. *music21* assumes that musescore or some other (finale, lilypond) music21 compatible notation software is installed in your machine.\n",
    "\n",
    "You will not be an expert in *music21* after single session of course, but at least these demos will give you ideas how music can be analysed with the help of computer tools such as *music21*.\n",
    "\n",
    "*Terminologies*. *Music21* has been created and maintained by a team at Harvard University, led by Michael Cuthbert. For this reason, the note durations (\"half and quarter notes\" in US) and bars (\"measures\" in US) and many other technical terms may seem unfamiliar to British readers, but I'm sure you are able to cope with this.\n",
    "\n",
    "### 1.3 Learning Tasks\n",
    "\n",
    "<p style=\"border:3px; border-style:solid; border-color:#335EFF; padding: 1em;\">\n",
    "<span style=\"color:blue\">In this document we will use bright blue color sections as prompts for <B>LEARNING TASKS</B> where you are asked to modify the commands to make the program to do something else in order to learn how it works. </span></P>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2 Load music\n",
    "\n",
    "You can load any of the thousands of scores that come with *music21*. A list of available scores is available at http://web.mit.edu/music21/doc/about/referenceCorpus.html and it contains a lot of Bach, some Beethoven, Chopin, Handel, Haydn, Mozart, Palestrina, Monteverdi, Josquin des Prez, Schubert, Schumann (both Clara and Robert) and thousands of folk songs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1 Load a score from build-in corpus in Music21"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from music21 import * # Every Music21 session will have to start with this definition\n",
    "opus133 = corpus.parse('beethoven/opus133.mxl') # we \"parse\" one specific work from the corpus\n",
    "opus133.measures(1, 4).show() # Show first 4 bars"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "opus133violin = opus133.getElementById('1st Violin') # just select the 1st violin part \n",
    "opus133violin.measures(1,8).show() # let's look at first 8 bars "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 Load a score from online corpus\n",
    "You can also take any score from online collections such as **kernscores** or **musedata**. For instance, *Kernscores* has over 100,000 scores available, see http://kern.ccarh.org, and musescore has thousands of works, see http://www.musedata.org, and there is a new openscore initiative which holds vasts amounts of music, see https://musescore.com/openscore"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Here's an example of how to load J.S. Bach Suite No. 1 in G major from internet:\n",
    "environment.set('autoDownload', 'allow') # We just allow the software to access internet\n",
    "no1 = converter.parse('http://kern.ccarh.org/cgi-bin/ksdata?l=cc/bach/cello&file=bwv1007-01.krn&f=xml')\n",
    "no1.metadata.movementName=\"J.S. Bach Suite No. 1 in G major, BWV 1007\" # Add title\n",
    "no1.measures(1, 2).show() # show first 2 bars"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 Create simple score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# You can create simple scores using a shorthand called tinynotation:\n",
    "ex1 = converter.parse(\"tinynotation: 3/4 c4 e8 g a-16 g f# g c'2 r4\")\n",
    "ex1.show()\n",
    "ex2 = converter.parse('tinyNotation: 4/4 c4 c## cn c- c-- c trip{c8 d e}')\n",
    "ex2.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"border:3px; border-style:solid; border-color:#335EFF; padding: 1em;\">\n",
    "<span style=\"color:blue\"><B>LEARNING TASK</B>: Can you create the theme from \"From Heaven Above to Earth I Come\" using tinynotation?</span></p>\n",
    "\n",
    "![](https://hymnary.org/flexscore/CWLH1993/38/LargePrint.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ex1 = converter.parse(\"tinynotation:  c'2\") # I'll give you the starting note\n",
    "ex1.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Summary\n",
    "* You were able to run *Music21* within the dynamic workbook (this document).\n",
    "* You were also able to load some music from the internal corpus that comes with *Music21*.\n",
    "* You noticed how you can *select* **parts** or **measures** of music.\n",
    "* You could display the using **show** command.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3 Visualise music\n",
    "Let's look at some ways of summarising the musical content."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from music21 import *\n",
    "schoenberg = corpus.parse('schoenberg/opus19', 2)\n",
    "schoenberg.measures(1, 6).show() # Show first 4 bars"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.1 Visualise raw note events"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's look at the pianoroll representation of the 2nd mov.\n",
    "schoenberg.measures(1, 10).plot() # Piano roll"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3 Summary of note and duration frequencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# What about the distributions of pitches (pitch-class distribution)? \n",
    "schoenberg.plot('histogram', 'pitchClass') # Easy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# What about durations and pitch-classes? Are they organised in a particular fashion?\n",
    "p = graph.plot.ScatterWeightedPitchClassQuarterLength(schoenberg) # A bit more arcane but understandable.\n",
    "p.run()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3 Summary over time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# How are the pitch-classes organised across time?\n",
    "schoenberg.plot('scatter', 'measure', 'pitchClass') # This is a variant of the command above."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.4 Visualise dynamics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Visualise the dynamics\n",
    "beethoven = corpus.parse('beethoven/opus133.mxl')\n",
    "beethoven.measures(1,62).plot('dolan', fillByMeasure=True, segmentByTarget=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"border:3px; border-style:solid; border-color:#335EFF; padding: 1em;\">\n",
    "<span style=\"color:blue\"><B>LEARNING TASK</B>: Can you compare the pitch distributions of the Schoenberg and Beethoven examples? Note that you can visualise all pitches simply by using the keyword pitch: </span>\n",
    "\n",
    "    schoenberg.plot('histogram', 'pitch')\n",
    "\n",
    "</p>\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# write you command here...\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Summary\n",
    "* You were able to display the raw events of music (pianoroll).\n",
    "* Examples of simple visual summaries were introduced (pitch-class distribution, durations, etc.).\n",
    "* There are plenty of other visualisations available, but let's move on to analysis."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4 Analyse music\n",
    "\n",
    "So far we have not been able to much more than an advanced music sequencer might have been able to do to visualise and select parts of music. Let's turn our attention to the analytical qualities of *Music21*.\n",
    "\n",
    "### 4.1 Qualities of chords\n",
    "\n",
    "*Music21* can infer a lot of information from stacks of pitches, that is, chords. They are such an important concept for music theory and analysis, so it is only natural that the software has a lot of optional to deal with chords. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# Let's put some pitches together to form three chords:\n",
    "chord1 = chord.Chord([\"C4\",\"G4\",\"E5\"]) # pitches\n",
    "chord1.duration.type = 'half'          # duration\n",
    "kc = key.Key('C')                      # key (optional but useful later)\n",
    "chord1B = chord1.closedPosition()      # A variant in closed position\n",
    "\n",
    "chord2 = chord.Chord(\"C F A\")\n",
    "chord2.duration.type = 'half'\n",
    "\n",
    "chord3 = chord.Chord(\"D F G B\")\n",
    "chord3.duration.type = 'half'\n",
    "chord1.show()                          # Just show one of these chords"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Music21 can tell you quite useful things about any of the chords. For instance:\n",
    "chord3.show()\n",
    "print(chord3.commonName)\n",
    "print(chord3.quality)\n",
    "print(chord3.forteClass)\n",
    "print(chord3.orderedPitchClasses)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We can analyse the chords in a key context and put the roman names into the score\n",
    "chord1.lyric = roman.romanNumeralFromChord(chord1, kc).figure\n",
    "chord2.lyric= roman.romanNumeralFromChord(chord2, kc).figure\n",
    "chord3.lyric= roman.romanNumeralFromChord(chord3, kc).figure\n",
    "chord1B.lyric= roman.romanNumeralFromChord(chord1B, kc).figure\n",
    "\n",
    "# Let's append the chords into a stream and look at the score with roman numerals.\n",
    "stream1 = stream.Stream()\n",
    "stream1.append(chord1)\n",
    "stream1.append(chord2)\n",
    "stream1.append(chord3)\n",
    "stream1.append(chord1B)\n",
    "stream1.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<span style=\"color:blue\"><B>LEARNING TASK</B>: Explore some more innovative chords by changing the pitches of the chords 1, 2 and 3. For instance, these chords were used by Elliott Carter frequently: C D- E F# and C D- E- G, or you might try jazz chords and neapolitan chord.</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.2 Chord analysis\n",
    "We can use the chord analysis capacity of *Music21* to carry out harmonic analysis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# You are all experts in analysis of Bach chorales, so let's try to analyse them.\n",
    "b = corpus.parse('bwv66.6') # We load a Bach Chorale BWV 66\n",
    "b.measures(0, 2).show()\n",
    "\n",
    "# Slice the chords for each beat with chordify\n",
    "bChords = b.chordify()\n",
    "bChords.metadata.movementName = 'Chord reduction'         # Put a label to the score\n",
    "bChords.measures(0, 2).show()\n",
    "\n",
    "#b.insert(0, bChords)\n",
    "\n",
    "for c in bChords.recurse().getElementsByClass('Chord'):   # This is to simplify the chords\n",
    "    c.closedPosition(forceOctave=4, inPlace=True)\n",
    "\n",
    "for c in bChords.recurse().getElementsByClass('Chord'):   # This is where the analysis happens\n",
    "    rn = roman.romanNumeralFromChord(c, key.Key('A'))\n",
    "    c.addLyric(str(rn.figure))\n",
    "\n",
    "bChords.metadata.movementName = 'Analysis with Roman Numerals'\n",
    "bChords.measures(0,2).show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"border:3px; border-style:solid; border-color:#335EFF; padding: 1em;\">\n",
    "<span style=\"color:blue\"><B>LEARNING TASK</B>: Is this a plausible harmonic analysis of the chorale beginning? What happens in the last beat of the last bar? What is the III6 chord actually? If in doubt, expand the selection of bars to show how the chorale continues. Optionally, you could also check what another chorale analysis would look like. The music21 filenames of all chorales can be found at <A HREF=\"http://web.mit.edu/music21/doc/about/referenceCorpus.html\">http://web.mit.edu/music21/doc/about/referenceCorpus.html</A>. Don't forget to put the correct key into the analysis if you select another chorale. There is also an automatic way of estimating the key, which will be covered later.</span></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.3 Analysis of twelve-tone rows (optional)\n",
    "There are also various build-in musical representations for famous historical (Webern, Berg) tone rows. These can be also analysed in terms of Fortean set-theoretical constructs. We are not going into these theories in this Module, but this is more of a demonstration of the versatility of *Music21*. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load a twelve-tone row from Alban Berg's Violin Concerto  \n",
    "aRow = serial.getHistoricalRowByName('RowBergViolinConcerto')\n",
    "bStream = stream.Stream()\n",
    "# Display them as prime form pitch classes and Forte's set classes\n",
    "for i in range(0, 12, 3):\n",
    "    c = chord.Chord(aRow[i:i + 3])\n",
    "    c.addLyric(c.primeFormString)\n",
    "    c.addLyric(c.forteClass)\n",
    "    bStream.append(c)\n",
    "bStream.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Summary\n",
    "* We learnt that *Music21* can infer chord qualities from any stacked collection of pitches.\n",
    "* If the key is known, any music can be reduced into slices of chords that can be turned into harmonic analysis.\n",
    "* *Music21* has a lot of set-theoretical and post-tonal analysis options."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.4 Metrical analysis\n",
    "Moving on to the temporal aspects of music, *Music21* has build-in metrical analysis module, which does Lerdahl-Jackendoff style metrical analysis. As you remember from your first year Analysis Module, this indicates how important are the different beats in the metrical hierarchy suggested by the time-signature."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Visualise metrical structures\n",
    "from music21.analysis import metrical\n",
    "\n",
    "# load a Bach Chorale from the music21 corpus of supplied pieces\n",
    "bwv30_6 = corpus.parse('bach/bwv30.6.xml')\n",
    "# get just the bass part\n",
    "bass = bwv30_6.getElementById('Bass')\n",
    "# get measures 1 through 10\n",
    "excerpt = bass.measures(1,10)\n",
    "# apply a Lerdahl/Jackendoff-style metrical analysis to the piece.\n",
    "metrical.labelBeatDepth(excerpt)\n",
    "# display the results\n",
    "excerpt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Summary\n",
    "* We explored simple harmonic analysis using chord and reduction functions in *Music21*.\n",
    "* An example of metrical analysis was given. \n",
    "* In *Music21* you can combine analytical options, which might be used to carry out a reduction of music based on metrical hierarchy just to give an example."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5 Corpus analysis\n",
    "Up to this point we have extracted some interesting musical properties from single pieces of music. Now it is time to apply the analysis to a corpus, a collection of pieces."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.1 Corpus and metadata\n",
    "Let's work with the build-in corpus of *Music21*. The system has neat architecture for searching, combining and loading all pieces with certain metadata. Also, it is typical that the pieces themselves contain different types of metadata."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# What other information besides the score do we have about a piece of music?\n",
    "opus3no1 = corpus.parse('corelli/opus3no1/1grave') # Get one Corelli Sonata\n",
    "print(opus3no1.metadata.all()) # Show metadata\n",
    "\n",
    "partStream = opus3no1.parts.stream() # Show the list of instruments\n",
    "for p in partStream:\n",
    "    print(p.id)\n",
    "\n",
    "opus3no1.measures(1,4).show()        # Plot first 4 bars"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# OK, let's select a corpus of all works composed by Giovanni Palestrina\n",
    "corpus1 = corpus.search(composer='palestrina')\n",
    "print(corpus1)\n",
    "\n",
    "# Let's select works within the corpus titled 'Kyrie'\n",
    "corpus2 = corpus1.search(title='Kyrie')\n",
    "print(corpus2)\n",
    "\n",
    "# What about even more specific, compositions that have the word \"Papae\" in it in Palestrina's Kyrie corpus?\n",
    "corpus3 = corpus2.search(parentTitle='Papae')\n",
    "print(corpus3)\n",
    "\n",
    "s=corpus.parse(corpus3[0]) # is this the famous Missa Papae Marcelli?\n",
    "s.measures(1,7).show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# You could display all full titles of Palestrina's Kyries:\n",
    "for work in corpus2:\n",
    "    score = corpus.parse(work)\n",
    "    print(score.metadata.parentTitle)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.2 Corpus analysis of keys\n",
    "\n",
    "Were Bach chorales written in specific keys? Perhaps the keys with only one or two sharps and flats were regularly utilised since they are easier to perform and notate? Are there more chorales in major mode than in minor? Let's look at the key distribution across chorales."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "chorales = corpus.search(composer='J.S. Bach',numberOfParts=4)\n",
    "print(chorales)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from music21 import*\n",
    "import matplotlib.pyplot as plt # Load some extra plotting libraries\n",
    "\n",
    "chorales = corpus.search(composer='J.S. Bach',numberOfParts=4)\n",
    "\n",
    "dict = {}\n",
    "dict2 = {}\n",
    "counter=1; maxlen = len(chorales)\n",
    "for chorale in chorales:\n",
    "   print('Analysing', counter,'/',maxlen, chorale.metadata.title,'...')\n",
    "   score = corpus.parse(chorale)\n",
    "   key = score.analyze('key').tonicPitchNameWithCase\n",
    "   key2 = score.analyze('key').mode\n",
    "   dict[key] = dict[key] + 1 if key in dict.keys() else 1\n",
    "   dict2[key2] = dict2[key2] + 1 if key2 in dict2.keys() else 1\n",
    "   counter +=1\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plot the results\n",
    "ind = [i for i in range(len(dict))]\n",
    "fig, ax = plt.subplots()\n",
    "ax.bar(ind, dict.values())\n",
    "ax.set_title('Frequency of Each Key')\n",
    "ax.set_ylabel('Frequency')\n",
    "plt.xticks(ind, dict.keys(), rotation='horizontal',size=12)\n",
    "plt.show()\n",
    "\n",
    "print(dict2) # print the frequency of major and minor keys\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"border:3px; border-style:solid; border-color:#335EFF; padding: 1em;\">\n",
    "<span style=\"color:blue\"><B>LEARNING TASK</B>: How do you interpret the key data of chorales? Note that within Bach chorales, there are also some modal chorales and sometimes the definition of the key is ambiguous.</span>\n",
    "</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Clarity of key (optional)\n",
    "In the example above, we analysed the most likely in each chorael. There are some chorales where they key is ambiguous, which can be explored by obtaining the tonalCertainty measure, which underlies the key analysis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "chorales = corpus.search(composer='J.S. Bach',numberOfParts=4)\n",
    "print(chorales)\n",
    "\n",
    "c=[]\n",
    "title=[]\n",
    "counter=0; maxlen = len(chorales)\n",
    "for chorale in chorales:\n",
    "   print('Analysing', counter,'/',maxlen, chorale.metadata.title,'...')\n",
    "   score = corpus.parse(chorale)\n",
    "   tc = score.analyze('key')\n",
    "   c.append(tc.correlationCoefficient) # get the correlation to the highest key\n",
    "   title.append(score.metadata.title)\n",
    "   counter +=1\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's see what the correlations look like\n",
    "\n",
    "fig, ax = plt.subplots()\n",
    "ax.plot(c, 'bo:',markersize=5,markerfacecolor='r')\n",
    "fig.set_size_inches(20, 10)\n",
    "ax.set_ylabel('Correlation coefficient',size=15)\n",
    "ax.set_xlabel('Nro in corpus',size=15)\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# There are few chorales where the correlations are lower than the other chorales, say under 0.85. \n",
    "# Let's look at those chorales\n",
    "\n",
    "ambiguous = [ n for n,i in enumerate(c) if i < 0.85 ] # get indices of tonally ambiguous chorales.\n",
    "print('Ambiguous keys can be found in:',ambiguous)\n",
    "\n",
    "# let's look at one of these\n",
    "score = corpus.parse(chorales[ambiguous[3]])\n",
    "tc = score.analyze('key')\n",
    "print(score.metadata.title,':',tc.correlationCoefficient)\n",
    "score.show()\n",
    "\n",
    "# This is G minor Dorian"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.2 Corpus analysis of vocal range\n",
    "\n",
    "Are the basses expected to sing over a larger range than tenors? Has the vocal range tended to be the same for SATB works over the centuries? Of course we do not always know what pitch was the score originally mapped onto but at least the vocal ranges should be comparable between in soprano, alto, tenor and bass voices.\n",
    "\n",
    "Let's explore the vocal ranges."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import statistics\n",
    "# Start with Bach chorales\n",
    "chorales = corpus.search(composer='J.S. Bach',numberOfParts=4)\n",
    "\n",
    "soprano_range = []\n",
    "alto_range = []\n",
    "tenor_range = []\n",
    "bass_range = []\n",
    "for chorale in chorales:                                          # Loop across chorales\n",
    "    s = corpus.parse(chorale)\n",
    "    for el in s.recurse().parts:                                  # Loop across the parts\n",
    "        #print(el.offset, el, el.analyze('range').semitones)\n",
    "        #print(el.partName)\n",
    "        if 'Soprano' in el.partName:\n",
    "            soprano_range.append(el.analyze('range').semitones)   # Calculate range if the part is soprano\n",
    "        if 'Alto' in el.partName:\n",
    "            alto_range.append(el.analyze('range').semitones)\n",
    "        if 'Tenor' in el.partName:\n",
    "            tenor_range.append(el.analyze('range').semitones)\n",
    "        if 'Bass' in el.partName:\n",
    "            bass_range.append(el.analyze('range').semitones)\n",
    "# Summarise the results\n",
    "print('Soprano', round(statistics.mean(soprano_range),2))\n",
    "print('Alto', round(statistics.mean(alto_range),2))\n",
    "print('Tenor', round(statistics.mean(tenor_range),2))\n",
    "print('Bass', round(statistics.mean(bass_range),2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"border:3px; border-style:solid; border-color:#335EFF; padding: 1em;\">\n",
    "<span style=\"color:blue\"><B>LEARNING TASK</B>: Which part had the largest range and why? Is this related to the Bach chorales or would similar results be evident in another, polyphonic vocal corpus? You could try Monteverdi (replace 'J.S. Bach' with 'Monteverdi', change the 'NumberOfParts' to 6, and also replace 'Soprano' with 'Canto' and 'Bass' with 'Basso'.\n",
    "</span>\n",
    "</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6 Corpus search\n",
    "\n",
    "Sometimes the useful approach is not to summarise the entire collection of music in terms of a specific feature but to search for a musical excerpt. Let's search the corpus for a theme that we have in mind. First we will select a suitable corpus of music and then search for a theme with or without the rhythms."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from music21 import *\n",
    "## Select all Bach Chorales\n",
    "#chorales = corpus.search('bach', fileExtensions='xml')\n",
    "chorales = corpus.search(composer='J.S. Bach')\n",
    "#print(chorales)            # shows how many pieces there are in the corpus\n",
    "chorales2 = corpus.search('bwv364') # Let's get the Dorian chorale again\n",
    "bwv364=corpus.parse(chorales2[0]) # is this the famous Missa Papae Marcelli?\n",
    "bwv364.measures(0, 4).show() # display the notation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.1 Theme search (pitch only)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# define a theme to search. Let's first try a simple theme (G F E) without considering the rhythm.\n",
    "searchList = [note.Note('G'), note.Note('F'), note.Note('E')] # define a search pattern G-F-E\n",
    "s = bwv364.recurse().notes                 # prepares the piece for the search\n",
    "p = search.noteNameSearch(s, searchList) # executes the search\n",
    "\n",
    "# show were the exact matches were\n",
    "for notePosition in p:\n",
    "    startingNote = s[notePosition]\n",
    "    startingMeasure = startingNote.measureNumber\n",
    "    startingBeat = startingNote.beat\n",
    "    startingPart = startingNote.getContextByClass('Part')\n",
    "    print(startingNote, startingMeasure, startingBeat, startingPart)\n",
    "# This report below shows in which voice, in which bar and which beat, does the theme occur: "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.2 Theme search (pitch and rhythm)\n",
    "The search above was simple and unrealistic. Let's search for a real theme with note durations\n",
    "Let's find the theme from \"Vom Himmel hoch, da komm ich her\" (\"From Heaven Above to Earth I Come\"), which was supposedly composed by Luther in 1539. The theme has been used in Bach's Christmas oratorio, but which chorale does it come from? Here we want to preserve the rhythm as well."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define the theme \"From Heaven Above to Earth I Come\")\n",
    "theme = converter.parse(\"tinynotation: 4/4 g4_From f#_hea- e_ven f#_above d_to e_earth f#_I g_come\")\n",
    "theme.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Here we want to preserve the approximate rhythm, but I will make all notes equally long (crotchets).\n",
    "searchStream2 = stream.Stream([key.KeySignature(1),\n",
    "                               note.Note('G4', type='quarter'),\n",
    "                               note.Note('F#4', type='quarter'),\n",
    "                               note.Note('E4', type='quarter'),\n",
    "                               note.Note('F#4', type='quarter'),\n",
    "                               note.Note('D4', type='quarter'),\n",
    "                               note.Note('E4', type='quarter'),\n",
    "                               note.Note('F#4', type='quarter'),\n",
    "                               note.Note('G4', type='quarter')])\n",
    "\n",
    "target1=[]\n",
    "target2=[]\n",
    "import time\n",
    "t = time.time()\n",
    "for i in range(100): # loop through the first 100 chorales ...\n",
    "    tmp = chorales[i].parse()\n",
    "    s = tmp.recurse().notes\n",
    "    for unused in range(12): # loop through different transpositions (up to 12 semitones)\n",
    "        s2=searchStream2.transpose(unused)\n",
    "        entryPoints = search.noteNameRhythmicSearch(s, s2.notes)\n",
    "        len1=len(entryPoints)\n",
    "        target1.append(len1)\n",
    "    len2=sum(target1)\n",
    "    target2.append(len2)\n",
    "    #print(i,target2[i])\n",
    "    target1=[]\n",
    "elapsed = time.time() - t\n",
    "print('Done! This search took',round(elapsed,1),'seconds')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Display results\n",
    "hits=[i for i, x in enumerate(target2) if x]\n",
    "print(\"These works contain the theme:\",hits)\n",
    "\n",
    "catalog = stream.Opus()\n",
    "\n",
    "for i in range(0,len(hits)):\n",
    "    tmp=chorales[hits[i]].parse()\n",
    "    incipit = tmp.measures(0,3)\n",
    "    catalog.insert(0, incipit.implode())\n",
    "catalog.show() # Display the works that contain the theme"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.3 Search for the same theme in other collections"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Has the theme from \"Vom Himmel hoch, da komm ich her\" used earlier?\n",
    "\n",
    "palestrina = corpus.search('palestrina')\n",
    "print(palestrina)\n",
    "\n",
    "# let's allow some rhythmic variations and remove the note durations from the search\n",
    "searchStream3 = stream.Stream([key.KeySignature(1),\n",
    "                               note.Note('G4'),\n",
    "                               note.Note('F#4'),\n",
    "                               note.Note('E4'),\n",
    "                               note.Note('F#4'),\n",
    "                               note.Note('D4'),\n",
    "                               note.Note('E4'),\n",
    "                               note.Note('F#4'),\n",
    "                               note.Note('G4')])\n",
    "target1=[]\n",
    "target2=[]\n",
    "import time\n",
    "t = time.time()\n",
    "for i in range(100):\n",
    "    tmp = palestrina[i].parse()\n",
    "    s = tmp.recurse().notes\n",
    "    for unused in range(12): # unison to seventh\n",
    "        s2=searchStream3.transpose(unused)\n",
    "        entryPoints = search.noteNameSearch(s, s2.notes)\n",
    "        len1=len(entryPoints)\n",
    "        target1.append(len1)\n",
    "    len2=sum(target1)\n",
    "    target2.append(len2)\n",
    "#    print(i,target2[i])\n",
    "    target1=[]\n",
    "print('Done! This search took',round(elapsed,1),'seconds')\n",
    "    \n",
    "hits=[i for i, x in enumerate(target2) if x]\n",
    "print(\"These works contain the theme:\",hits)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# display one example\n",
    "print(hits)\n",
    "tmp=palestrina[hits[3]].parse()        # We are looking at the fourth example in the list above\n",
    "s = tmp.recurse().notes                 # prepares the piece for the search\n",
    "p = search.noteNameSearch(s, searchStream3) # executes the search\n",
    "target1=[]\n",
    "target2=[]\n",
    "for unused in range(12): # unison to seventh\n",
    "    s2=searchStream3.transpose(unused)\n",
    "    entryPoints = search.noteNameSearch(s, s2.notes)\n",
    "    len1=len(entryPoints)\n",
    "    target1.append(len1)\n",
    "tr=[i for i, x in enumerate(target1) if x]\n",
    "\n",
    "# show were the exact matches were\n",
    "s2=searchStream3.transpose(tr[0])\n",
    "p = search.noteNameSearch(s, s2.notes) # executes the search\n",
    "for notePosition in p:\n",
    "    startingNote = s[notePosition]\n",
    "    startingMeasure = startingNote.measureNumber\n",
    "    startingBeat = startingNote.beat\n",
    "    startingPart = startingNote.getContextByClass('Part')\n",
    "    print(startingNote, startingMeasure, startingBeat, startingPart)\n",
    "tmp.measures(10, 12).show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The soprano part from Palestrina's Agnus shows a rhythmic variation of the same patterns as our search theme. Is this a coincidence or an actual precursor of the theme? Well, we could calculate how commonly this pattern has been created in the past or how likely you would get the same pattern by scrambling music randomly, but these are advanced topics.   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## References\n",
    "\n",
    "* Cuthbert, M. S., & Ariza, C. (2010). music21: A Toolkit for Computer-Aided Musicology and Symbolic Music Data. In J. Stephen Downie and Remco C. Veltkamp (Eds.). 11th International Society for Music Information Retrieval Conference (ISMIR 2010), August 9-13, 2010, Utrecht, Netherlands. pp. 637-642. [link](http://ismir2010.ismir.net/proceedings/ismir2010-108.pdf)\n",
    "\n",
    "* Cuthbert, M. S., Ariza, C., & Friedland, L. (2011). Feature Extraction and Machine Learning on Symbolic Music using the music21 Toolkit. In 11th International Society for Music Information Retrieval Conference (ISMIR 2011) (pp. 387--392). [link](http://ismir2011.ismir.net/papers/PS3-6.pdf)\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}