ISMIR 2006 - MIREX Panel
On the last day of ISMIR, Stephen Downie and the MIREX crew had a panel
and discussion session about this year's MIR evaluations.
Stephen highlighted the fact that these he likes to call these evaluations 'tasks' instead of 'evaluations'. He wants to avoid treating these as win/lose contests - but instead to treat them as a way to test how well an algorithm works when compared to others. Treating the evaluations as tasks instead of contests may encourage those that are reluctant to participate to join in. Still, it is difficult, especially for commercial ventures to participate unless they can guaranteed a good result.
Some of the MIREX crew (Cameron, Andreas, Kris and Mert) joined Stephen in commenting on how the evaluations were run.
There are all sorts of results on the MIREX results page.
Rainer Typke has been augmenting some of the result pages with some interesting graphs.
And finally, check out the MIREX audio similarity tee-shirt, ably modelled by conference organizer Holger Hoos.
Stephen highlighted the fact that these he likes to call these evaluations 'tasks' instead of 'evaluations'. He wants to avoid treating these as win/lose contests - but instead to treat them as a way to test how well an algorithm works when compared to others. Treating the evaluations as tasks instead of contests may encourage those that are reluctant to participate to join in. Still, it is difficult, especially for commercial ventures to participate unless they can guaranteed a good result.
Some of the MIREX crew (Cameron, Andreas, Kris and Mert) joined Stephen in commenting on how the evaluations were run.
There are all sorts of results on the MIREX results page.
Rainer Typke has been augmenting some of the result pages with some interesting graphs.
And finally, check out the MIREX audio similarity tee-shirt, ably modelled by conference organizer Holger Hoos.