ICMAI offers the possibility to assign grades

DH Regensburg

Manuel Burghardt

Lecture at the DH colloquium of the Berlin-Brandenburg Academy of Sciences

The use of digital resources and computer-aided methods in the humanities is commonly summarized under the buzzword Digital Humanities. Against the background of existing methods in the field of corpus and computational linguistics, there has so far been a strong focus on text-based sciences (see, for example, the concept of distant reading). In the digital humanities, however, other humanities disciplines are increasingly moving into the focus of computer-aided modeling and analysis attempts, for example in the areas of art history (image), film studies (film) and musicology (music). Accordingly, the lecture is located at the intersection of digital humanities and musicology. With the help of a concrete project for the digitization of a large collection of handwritten song sheets, basic possibilities and methods for the indexing, representation and analysis of notated music are to be demonstrated. Although parallels to methods and procedures from the text-based humanities become evident in many places, it also becomes clear that “musical text” is more than just words, and thus some peculiarities and challenges arise in digitization.

On the subject of indexing, existing tools from the field of optical music recognition (OMR) will be presented in the course of the lecture, which can only be used to a limited extent for handwritten music (Burghardt et al., 2017). As an alternative, a crowdsourcing approach for transcribing the melodies is presented (Meier et al., 2015; Burghardt & Spanner, 2017). Furthermore, the lecture shows different levels of the representation of melodies, e.g. as a precisely notated melody, as a sequence of intervals or as an abstract melody contour (Parsons code). In addition, existing storage formats for the representation of music information (e.g. MusicXML and MEI) are presented and discussed. In the last part of the lecture, fundamental possibilities of the computer-aided analysis of digitized music data are shown and fundamental concepts of music information retrieval, especially melodic similarity, are introduced (Burghardt et al., 2015; Burghardt et al., 2016; Burghardt & Lamm, 2017 ). In addition, it should be shown which new questions can be dealt with by computer-aided approaches in musicology.

literature

  • Burghardt, M., & Lamm, L. (2017). Development of a music information retrieval tool for the melodic similarity analysis of German-language folk songs. In M. Eibl & M. Gaedke (Eds.), INFORMATIK 2017, Lecture Notes in Informatics (LNI), Society for Computer Science - Workshop “Music meets Computer Science” (pp. 15-27). Bonn: Springer.
  • Burghardt, M., & Spanner, S. (2017). Allegro: User-centered Design of a Tool for the Crowdsourced Transcription of Handwritten Music Scores. In Proceedings of the DATeCH (Digital Access to Textual Cultural Heritage) conference. ACM.
  • Burghardt, M., Spanner, S., Schmidt, T., Fuchs, F., Buchhop, K., Nickl, M., & Wolff, C. (2017). Digital indexing of a collection of folk songs from German-speaking countries. In Book of Abstracts, DHd 2017.
  • Burghardt, M., Lamm, L., Lechler, D., Schneider, M., & Semmelmann, T. (2016). Tool-based Identification of Melodic Patterns in MusicXML Documents. In Book of Abstracts of the International Digital Humanities Conference (DH).
  • Burghardt, M., Lamm, L., Lechler, D., Schneider, M., & Semmelman, T. (2015). MusicXML Analyzer - An analysis tool for the computer-aided identification of melody patterns. In proceedings of the 9th Hildesheim Evaluation and Retrieval Workshop (HiER 2015) (pp. 29–42).
  • Meier, F., Bazo, A., Burghardt, M., & Wolff, C. (2015). A Crowdsourced Encoding Approach for Handwritten Sheet Music. In J. Roland, Perry; Kepper (Ed.), Music Encoding Conference Proceedings 2013 and 2014 (pp. 127-130).

"More than Words" - from text to music

Franco Moretti's concept of "distant reading" is a central metaphor in digital humanities that has meanwhile expanded to include many other areas, for example.

Approaches to "Distant hearing“Computer-aided analysis of music data are still relatively rare in the digital humanities community, but they are under the keyword in the IT sector Music information retrieval (MIR) established for many years. A central organization and conference of the same name is the ISMIR (International Society for Music Information Retrieval).

A fundamental distinction is made in the MIR between approaches in the area of Signal processing (Audio) and the Symbol processing (Grades).

The lecture focuses on symbolic music, i.e. sheet music. Specifically, challenges and particularities ("notes are more than words") in the computer-aided indexing, modeling and analysis based on the Regensburg song sheet collection are shown.

Case study: Regensburg song sheet collection

This project is a cooperation with the University Library of Regensburg, which has a - in terms of scope and coverage - unique song sheet collection of German-language folk songs.

The approx. 140,000 song sheets contain handwritten, monophonic melodies, mostly song texts typed with a typewriter, as well as various metadata such as archive location, year and song sheet number.

Further information to Song sheet collection:

  • Krüger, G. (2013). The “Regensburg Folk Music Portal” of the Regensburg University Library. Holdings - Problems - Perspectives. Interim report from a development project. In E. R. Mohrmann (Ed.), Audio archives - digitize, develop and evaluate audio documents (p. 119-131). Münster et al .: Waxmann Verlag.

(I) machine readable Development / digitalization

Overview to Optical Music Recognition (OMR) Tools: http://homes.soic.indiana.edu/donbyrd/OMRSystemsTable.html

Current OMR tools:

Existing collection with scans:

Existing collections of transcribed music:

Unfortunately, OMR works very poorly with handwritten music sheets:

OMR for handwritten scores as a major unresolved problem (Müller, 2007)

This assessment was also confirmed by the evaluation of three existing OMR tools for the Regensburger Liedblattsammlung. Evaluation design based on Bellini, Bruno & Nesi (2007).Average Detection rates: Photoscore (36%), CapellaScan (8%) and SharpEye (4%).

More information about the Evaluation study:

  • Burghardt, M., Spanner, S., Schmidt, T., Fuchs, F., Buchhop, K., Nickl, M., and Wolff, C. (2017). Digital indexing of a collection of folk songs from German-speaking countries. In Book of Abstracts, DHd 2017.

Since OMR is out of the question because of the poor results, we will Crowdsourcing as alternative Development strategy chosen because transcription According to Oomen & Aroyo (2011), it is one of the typical application areas of crowdsourcing:

  • Contextualization
  • Complementing collections
  • Classification
  • Co-curation
  • Crowdfunding
  • C.orrection and transcription

Existing Transcription tools, but none of them are suitable for a remote crowdsourcing approach.

Allegro

Development of our own crowdsourcing tool called Allegro, which meets the following requirements:

  • web-based (HTML / JavaScript) and can be used by several transcriptors in parallel
  • easy to handle (developed with the help of a systematic UCD approach), so intuitive that it is also possible for music laypersons to create sketches

More information about Allegro in:

  • Burghardt, M., & Spanner, S. (2017). Allegro: User-centered Design of a Tool for the Crowdsourced Transcription of Handwritten Music Scores. In Proceedings of the DATeCH (Digital Access to Textual Cultural Heritage) conference. ACM.

OMR literature

  • Bainbridge, D. and Bell, T. (2001). The challenge of optical music recognition. In Computers and the Humanities, 35, p. 95-121.
  • Bellini, P., Bruno, I., and Nesi, P. (2007). Assessing optical music recognition tools. In Computer Music Journal, 31 (1), 68-93.
  • Grachten, M., Arcos, J. L., and de Mántaras, R. L. (2002). A comparison of different approaches to melodic similarity. In Proceedings of the 2nd International Conference in Music and Artificial Intelligence (ICMAI).
  • Homenda, W. and Luckner, M. (2006). Automatic Knowledge Acquisition: Recognizing Music Notation with Methods of Centroids and Classifications Trees. In Proceedings of the IEEE International Joint Conference on Neural Network, p. 6414-6420.
  • Raphael, C. and Wang, J. (2011). New Approaches to Optical Music Recognition. In Proceedings of the 12th International Society for Music Information Retrieval Conference (ISMIR), p. 305-310.
  • Rebelo, A., Capela, G., and Cardoso, J.S. (2010). Optical recognition of music symbols. In International Journal on Document Analysis and Recognition, 13, 19–31.
  • Müller, M. (2007). Information Retrieval for Music and Motion. Berlin: Springer.

Crowdsourcing literature

  • Causer, T. and Wallace, V. (2012). Building A Volunteer Community: Results and Findings from Transcribe Bentham. In Digital Humanities Quarterly, 6 (2).
  • Dunn, S. and Hedges, M. (2013). Crowd-sourcing as a Component of Humanities Research Infrastructures. In International Journal of Humanities and Arts Computing, 7 (1-2), 147-169.
  • Fornés, A., Lladós, J., Mas, J., Pujades, J. M. and Cabré, A. (2014). A Bimodal Crowdsourcing Platform for Demographic Historical Manuscripts. In Proceedings of the First International Conference on Digital Access to Textual Cultural Heritage, p. 103-108.
  • Holley, R. (2010). Crowdsourcing: How and why should libraries do it? D-Lib Magazine, 16 (3-4).
  • Howe, J. (2006). The rise of crowdsourcing. Wired 14 (6). Retrieved from http://archive.wired.com/wired/archive/14.06/crowds.html
  • Ipeirotis, P. G. and Gabrilovich, E. (2014). Quizz: Targeted Crowdsourcing with a Billion (Potential) Users. In Proceedings of the 23rd International Conference on World Wide Web, pp 143-154.
  • Lee, T. Y., Dugan, C., Geyer, W., Ratchford, T., Rasmussen, J., Shami, N. S. and Lupushor, S. (2013). Experiments on motivational feedback for crowdsourced workers. In Proceedings of the 7th International Conference on Weblogs and Social Media (ICWSM), p. 341-350.
  • Morschheuser, B., Hamari, J. and Koivisto, J. (2016). Gamification in crowdsourcing: A review. In Proceedings of the 49th Annual Hawaii International Conference on System Sciences, p. 4375-4384.
  • Mühlberger, G., Zelger, J. and Sagmeister, D. (2014). User-Driven Correction of OCR Errors: Combining Crowdsourcing and Information Retrieval Technology. In Proceedings of the First International Conference on
  • Oomen, J. and Aroyo, L. (2011). Crowdsourcing in the Cultural Heritage Domain: Opportunities and Challenges. In Proceedings of the 5th International Conference on Communities and Technologies, p. 138-149.

(II) modeling and formal representation

Encoding formats

(III) Computerized analysis

Music Information Retrieval Definition (Downie, 2004)

Music Information Retrieval (MIR) is a multidisciplinary research endeavor that strives to develop innovative content-based searching schemes, novel interfaces, and evolving networked delivery mechanisms in an effort to make the world's vast store of music accessible to all.

Overview of existing MIR systems in the Web:

In addition to the search for specific melodies, there are also more abstract levels of the melody search, e.g. the search for Interval sequences or after Melody contours in the Parsons code.

A first MIR prototype was implemented for the Regensburg song sheet collection, which uses the Mongeau-Sankhoff algorithm as a melodic similarity measure. This is an edit distance-based method for determining the similarity of two melody sequences. In order to avoid distortions in the editing distance, an Ngram approach is also implemented, i.e. the melody query is searched for in certain partial sequences (ngrams) of the song sheets (see Burghardt & Lamm, 2017). A Melodic Similarity search demo in the Regensburger Liedblattsammlung is available at:

Additional Information:

  • Burghardt, M., & Lamm, L. (2017). Development of a music information retrieval tool for the melodic similarity analysis of German-language folk songs. In M. Eibl & M. Gaedke (Eds.), INFORMATIK 2017, Lecture Notes in Informatics (LNI), Society for Computer Science - Workshop "Music meets Computer Science" (pp. 15-27). Bonn: Springer.

Literature MIR

  • Casey, M., Veltkamp, ​​R., Goto, M., Leman, M., Rhodes, C., & Slaney, M. (2008). Content-based music information retrieval: Current directions and future challenges. Proceedings of the IEEE, 96 (4), 668-696.
  • Downie, J.S. (2004). The Scientific Evaluation of Music Information Retrieval Systems: Foundations and Future. In Computer Music Journal 28 (2), 12-23.
  • Selfridge-Field, E. (1998). Conceptual and representational issues in melodic comparison. Computing in Musicology, 11, 3-64.

Literature Melodic Similarity

  • Berenzweig, A., Logan, B., Ellis, D. P. W., & Whitman, B. (2004). A Large-Scale Evaluation of Acoustic and Subjective Music-Similarity Measures. Computer Music Journal, 28, 63-76. http://doi.org/10.1162/014892604323112257
  • Cahill, M., Cahill, M., Music, C., & Music, C. (2005). Melodic similarity algorithms - using similarity ratings for development and early evaluation. Star, 450-453.
  • Hofmann-Engl, L. (2001). Towards a cognitive model of melodic similarity. Ismir, 44 (0), 143-151.
  • Grachten, M., Arcos, J. L., & Mántaras, R. L. De. (2004). Melodic Similarity: Looking for a Good Abstraction Level. Proceedings of the 5th International Society for Music Information Retrieval.
  • Grachten, M., Arcos, J. L., and de Mántaras, R. L. (2002). A comparison of different approaches to melodic similarity. In Proceedings of the 2nd International Conference in Music and Artificial Intelligence (ICMAI).
  • Miura, T., & Shioya, I. (2003). Similarity among melodies for music information retrieval. In Proceedings of the twelfth international conference on Information and knowledge management - CIKM ’03 (p. 61).
  • Mongeau, M. and Sankoff, D. (1990). Comparison of Musical Sequences. In Computers and the Humanities, 24, 161-175.
  • Müllensiefen, D., & Frieler, K. (2004). Optimizing Measures Of Melodic Similarity For The Exploration Of A Large Folk Song Database. 5th International Conference on Music Information Retrieval ISMIR 2004, 274-280.
  • Müllensiefen, D., & Frieler, K. (2004). Melodic Similarity: Approaches and Applications. In Proceedings of the 8th International Conference on Music Perception & Cognition (pp. 283-289).
  • Orio, N., & Rodá, A. (2009). A Measure of Melodic Similarity Based on a Graph Representation of the Music Structure. In Proceedings of the 10th International Society for Music Information Retrieval Conference (ISMIR 2009) (pp. 543-548).
  • Typke, R., Wiering, F., & Veltkamp, ​​R. C. (2005). A survey of music information retrieval systems. Transition, 153-160.
  • Typke, R. (2007). Music Retrieval based on Melodic Similarity. Ph.D Thesis, (April 1973).

I like it:

LikeLoading ...
written by: dhregensburgPosted under Crowdsourcing, Melodic Similarity, Music Information Retrieval, Musicology, Resources, Tools, Lecture

Digital Humanities at the University of Regensburg

Contact:

Wan-Hua Her
University of Regensburg
Chair for media informatics
D-93040 Regensburg
[email protected]

Aenne Knierim
University of Regensburg
Chair for media informatics
D-93040 Regensburg
[email protected]

Current Posts

Tagcloud