Duke Listens!: Visit my main blog at MusicMachinery.com

Here's a first look at the new Genius feature of iTunes 8. First of all, there have been a few reports that the Genius feature will be 'Pandora-like.' This is not true - the iTunes recommendations seem to be rather run-of-the-mill collaborative filtering recommendations based upon the wisdom of the crowds - as compared to the core technology behind Pandora which uses a detailed content-analysis of the music (by a trained staff of musicians) to generate recommendations.

So how are the Genius recommendations and playlists? Well, there are some funny bits to start off with. For instance, if try to get recommendations based upon a Beatles song, the 'Genius' can't help you at all.

Instead, it just falls back on what's popular. It doesn't even try to match the Genre (britpop? 60s psychedelia) or artist (John Lennon, Wings?). Pink and Rihanna are right out. This glaring gap is not just confined to the Beatles, but it seems to occur for other artists like Bob Seger, and AC/DC that have shunned the iTunes store .

Similarly, for long tail artists, the Genius didn't have too much to say. For example, the Genius recognized artist DJ Markitos and was ready to sell me another DJ Markitos album, but couldn't offer me any other recommendations:

When I avoid the long tail, or the iTunes-averse artists, I can coax the Genius to give me some real recommendations. The first thing I notice is that the recommendations seem to be artist-based and not album or track-based. For instance if I select as a seed track Emerson Lake and Palmer's 'Food for your Soul' (which is a big band jazz-fusion track), I get recommendations for Yes, Jethro Tull, The Moody Blues, Procol Harem, and Renaissance. When I pick ELPs - Two Part Invention (Bach on a xylophone), I get the same recommendations for tracks by Yes, The Moody Blues, Procol Harem, Renaissace etc. And surprise, surprise, if I pick a progressive rock track by ELP I get recommendations for the same set of artists.

The same is true for the playlists generated by the Genius - the Genius just picks tracks from similar artists regardless of how well the track is representative for the artist. For instance, on the album 'Return to the center of the earth' by Rick Wakeman the tracks alternate between music and narration. If I use one of the narration tracks as a seed, the Genius gives me run of the mill tracks by progressive rock artists. No track-similarity is used to generate the playlists. This is not Pandora-like behavior.

So, the Genius isn't great with the long tail, and can't give you good recommendations based upon a track or an album. So, how does it do with regular old recommendations? Let's look at a few:

Seed song:: Dazed and Confused by Led Zeppelin.
Recommendations:: Spoon, RHCP, Pure Parie League, Chuck Berry, System of a Down, Madonna, Linkin Park (the Chuck Berry song was the Christmas ditty'Run Rudolph Run')
Comments:I think we are seeing some extreme popularity bias here. The artists just don't match.
My Score: D

Seed song:: My Name is Jonas by Weezer
Recommendations:: Audioslave, Death Cab For Cutie, blink-182, Tenacious D, Bright Eyes, Weezer, The Shins
Comments:No clunkers this time, but nothing new - no novelty or serendipity. I could get a much better list from All Music guide
My Score: C

Seed song:: Fanfare for the Common Man by Emerson Lake and Palmer
Recommendations:: Yes, Focus, Genesis, The Moody Blues, Procol Harum
Comments:All recommendations from the progressive rock bands of the 70s. Again, no clunkers, but this list is not going to help people find new music - no Gentle Giant, no Porcupine Tree or Dream Theater. This is classic rock radio
My Score: C

Seed song:: Fidelity by Regina Spektor
Recommendations:: Ingrid Michaelson, Augustana, Lily Allen, Hellgoodbye, Mat Kearney, KT Tunstall, Corinne Baily Rae, Blue October, OK Go, Death Cab for Cutie, Gwen Stefani, Dashboard Confessional, Jason Mraz, John Mayer
Comments:Regina Spektor is a quirky indie singer-songwriter. One of these recommendations seem good to me: Ingrid Michaelson (hadn't heard of her, and she has that indie vibe), but the rest are just bad, (the Gwen Stefani recommendation is downright laughable). One good recommendation in ten - not too good
My Score: D

Seed song:: Money by Pink Floyd
Recommendations:: Eric Clapton, Jimi Hendrix, The Doors, Cream, ZZ Top, Pink Floyd, Deep Purple, Alice Cooper, Blue Oyster Cult, The Rolling Stones, Steppenwolf
Comments:Another set of artists from the greatest hits of the 60s and 70s album. Yawn
My Score: D

I like to rate recommendations on 4 factors:

• Relevance - how well do the recommended items match the taste - the genius does OK here, with a few glaring errors like the Gwen Stefani recommendation for Regina Spektor.
• Novelty - how well does the recommender find items that I haven't heard of - the Genus doesn't fair well at all - when it did recommend items that I hadn't heard of, they were almost always irrelevant.
• Reach - how far does the recommender get into the long tail. The Genius couldn't give recommendations for the long tail items that I tried, nor did it give recommendations that were far into the tail (and I couldn't steer it to hipster items).
• Transparency - how well does the recommender explain the recommendations. A recommender like Pandora does a great job explaining why an item was recommended, however, the Genius doesn't even try to explain its recommendations.

The iTunes Genius is just a run-of-the-mill collaborative filtering recommender - the recommendations are nothing special - there's no advanced content analysis like you get from MusicIP. There's no deep analysis of content and context like The Echo Nest. There's no social community or tags like at Last.fm. There's no transparency in the recommendations like at Pandora. - the Genius just gives rather pedestrian recommendations and playlists.

Apple and iTunes are at a choke point in the music ecosystem - Apple has an opportunity to help people explore for and discovery new music. It is good to see Apple recognize this and at least start to provide recommendation tools. I hope they will start to look at all of the interesting work being done by researchers in the Recommenders and Music Information Retrieval community and try to make their Genius a little smarter.