Evaluating Music Information Retrieval Systems
Musical Information Retrieval is a fairly new area of research. In fact, this year will be only the second year in which a community-wide algorithm and systems evaluation contest is held. The Music Information Retrieval Evaluation eXchange (MIREX) will take place during the 6th ISMIR Conference in London, UK, September 11-15, 2005. The goal of this contest is to compare state-of-the-art algorithms and systems relevant for Music Information Retrieval.
Nine evaluation tasks are planned:
- Audio Artist Identification
- Audio Drum Detection
- Audio Genre Classification
- Audio Melody Extraction
- Audio Onset Detection
- Audio Tempo Extraction
- Audio and Symbolic Key Finding
- Symbolic Genre Classification
- Symbolic Melodic Similarity
I find the Artist and the Genre classification tasks to be particularly interesting.
This year, the MIREX will be using the new M2k (Music to Knowledge) toolset as the framework for the evaluation. The deadlines for MIREX are:
- Participation Statement: June 12, 2005
- Algorithms and 1-page Abstracts: June 26, 2005
- Extended Abstracts: September 2, 2005