Music information retrieval based on tonal harmony

Abstract

With the emergence of large scale digitalisation of music, content-based methods to maintain, structure, and provide access to digital music repositories have become increasingly important. This doctoral dissertation covers a wide range of methods that aim to aid in the organisation of music information. From both a practical as well as cognitive point of view, it is logical to structure musical content by defining similarity relations between documents. Consequently, the notion of music similarity has become a fundamental concept within the area music information retrieval (MIR) research. In this dissertation we study a particular type of music similarity: the similarity of musical harmony. Because both musically trained and untrained listeners have extensive knowledge about music, it is rather unlikely that all information needed for sound similarity judgement can be found in the musical information source alone. Therefore, to be able to place chord sequences in the context of Western tonal harmony, we investigate two approaches towards automatic harmony analysis. Although the first generative grammar-based solution yields good results on a small dataset, it exposed some practical challenges that prevented it to be extended to process larger datasets. Hence, the second harmonic analysis solution exploits state-of-the-art functional programming techniques, like type-level computation and error-correcting parsers, to meet these challenges. This model, named HarmTrace, is fast, flexible, and returns analyses that are in accordance with harmony theory. We evaluate these harmonic annotations, which explain the role of a chord in its tonal context, both qualitatively as well as quantitatively, and show how they can aid in harmonic similarity estimation and automatic chord transcription. We investigate three novel approaches to harmonic similarity: a geometric, a local alignment, and a common embeddable subtree based approach. The geometric approach, named TPSD, uses a music theoretically motivated step functions to assess the similarity of two chord sequences; the common embeddable subtree approach estimates harmonic similarity by matching hierarchical harmonic analysis annotations; and the local alignment solution uses context-aware substitution functions to align sequences of chords. For each of these harmonic similarity solutions, the adjustable parameters are discussed and evaluated. For the evaluation a large new chord sequence corpus is assembled consisting of 5028 different chord sequences, some of which describe the same song. The results show that an alignment approach that uses the HarmTrace harmony model performs best in retrieving these similar chord sequences. All proposed similarity measures rely on the availability of sequences of symbolic chord labels. To extend the application domain, we demonstrate how automatic chord transcription from musical audio can be improved by exploiting our model of tonal harmony