Duke Listens!: Visit my main blog at MusicMachinery.com

MIREX 2006 has officially opened its doors for submissions. 

MIREX 2006 is the second  running of  the Music Information Retrieval Evaluation eXchange, an annual evaluation of MIR systems.  In some sense, MIREX is the TREC of the MIR world.    MIREX provides a set of standard evaluations for MIR researchers to benchmark their systems, with the hope that with these evaluations we will see improvement from year to year in these systems, just as standard evaluations have helped improved the state of the art in information retrieval  and speech recognition.

This year MIREX will be conducting the following evaluation tasks:

• Audio Beat Tracking
• Audio Melody Extraction
• Audio Music Similarity and Retrieval
• Audio Cover Song
• Audio Onset Detection
• Audio Tempo Extraction
• QBSH: Query-by-Singing/Humming
• Score Following
• Symbolic Melodic Similarity

Near and dear to my heart is the Audio Music Similarity and Retrieval task. This is the first attempt by MIREX to evaluate similarity systems.  Music similarity is a key component for content-based music recommenders.  However, music similarity is a subjective concept so it is hard to evaluate.  Previously MIREX  relied on objective evaluations for the various tasks, but for this music similarity task, MIREX will break with tradition and will base the evaluation primarily on human judgements.  Of course, setting up a human evaluation that will scale to large numbers of  submissions is quite difficult, but the folks at the IMIRSEL are working hard to make sure that it be a successful evaluation.  The data that this evaluation generates will be useful for use by MIR researchers for years to come.

Results for this years MIREX will be published during ISMIR 2006.