Duke Listens!: Visit my main blog at MusicMachinery.com

Last month, idiomag launched its new developer API that gives 3rd parties access to all sorts of music related data. In particular, the API gives you access to artist data such as news articles, reviews and interviews, videos, images and MP3s. They provide 'similar artist'-style recommendations as well as personalized recommendations based on an individual's listening habits (they'll base the recommendations on your listening data that they hoover from sites like Pandora, Last.fm or iMeem). They also will give you recommendations based upon your APML (attention profile markup language). (I like how they use our TasteBroker as source of APML for their sample documentation).

The API is designed rather well. They support 3 types of output: 'xml', 'json' and 'rss' (and for some methods, 'apml' 'xspf' and 'foaf') - so parsing the output will be pretty straightforward for anyone who's used any kind of web services. The do have a few rather funky conventions like the parameter name artists[]' to indicate multiple artists like so:

http://www.idiomag.com/api/recommendation/articles/xml?key=secret&artists[]=muse&artists[]=radiohead

but thats no big deal. All of their API methods require an application key. One beef I have is with the documentation - they don't have any clickable examples in the docs - so trying out the API is rather laborious - first you have to copy the url prefix, add your secret key, add the method arguments and finally the parameters. Update 2 The Idiomag folks are fixing this ... some of the methods now have clickable link/examples. Woohoo! They should follow the lead of APIs like Last.fm or the EchoNest and create a demo key that they can use in their docs so that all services can have a clickable example. (The gold standard for web api documentation is the Flickr services - their 'flickr Api Explorer' lets you easily try out all aspects of the API from your web browser). There was one glitch in the API - any attempt to format output in APML triggers an error Fatal error: Class 'xmlWriter' not found in /var/www/html/app/views/helpers/apml.php on line 11 - looks like something is misconfigured in the web server. Update: the Idiomag guys have fixed this already

The API provides 5 types of services: Articles, Artist, User, Tag and Recommendation.

• articles - returns recent and featured articles from the Idiomag catalog
• artist - returns all sorts of info about artists including related artists, tags, articles, playlists, videos images. The tag coverage seems quite sparse and uneven. The Beatles have only 3 tags (Classic rock, rock and pop) while Aphex Twin has 6 (electronic, ambient, IDM, experimental, electronica, techno).
• user - returns info tailored to a specific user - surprisingly, APML is not one of the supported output types for user/info. Update - APML is supported, it was just an error in the documentation. Fixed already.
• tag - returns information about a tag- apparently these are not traditional social tags - they seem to come from a fixed vocabulary of tags (the link that is supposed to give me the list of all tags currently available is broken however, so I can't see how big the list is). The results seem a bit unusual. For instance, the first 4 artists returned for the tag 'rock' are 'Forward, Russia, '10 Years', '12 Stones', '3 Doors down'. So perhaps we are getting an alphabetical list (but I don't know how 'Forward Russia' got to the front of the list). I would have expected a list ordered by tag frequency which should yield something more like: the beatles, radiohead, coldplay, oasis.
• recommendation - gives artist recommendations, based either on an APML file, an artist list, or my listening behavior at one of 10 or so social music sites.

Lets take a look at the recommendations. I tried the API with a single artist as a seed to start things off:

seed: the beatles

• Sting and Edin Karamazov
• William Shatner
• The Kinks
• Elvis
• The Rolling Stones
• Led Zeppelin
• Paul McCartney
• The Mamas and the Papas
• The Beach Boys
• George Harrison

This is funny list - Sting and Edin Karamazov have been playing 16th-century Elizebethan music, Shater is, well, Shatner - so for the top two recommendations, these certain win on the novelty scale (but perhaps fail on the relevant scale), also notable in its absence is John Lennon in the top 20 (he does appear paired with Yoko, but not by himself)

seed: emerson, lake and palmer

• Rick Wakeman
• Gentle Giant
• Atomic Rooster
• Moody Blues
• Van der Graaf Generator
• Asia
• Aphrodite's Child
• Kansas
• Caravan
• Camel

That's a pretty reasonable list.

seed:deerhoof

• Blonde Redhead
• Asobi Seksu
• Odiorne
• Quickspace
• The Fiery Frnaces
• Menomena

Again, a reasonable list.

seed:miles davis

• John Coltrane
• Miles Davis and Gil Evans
• Chet Baker
• Bill Carrothers
• Charles Mingus
• Stan Getz

A totally fine list

One of the neat things about Idiomag is that they can suck up my user profile from Last.fm and give me recommendations based on my listening habits. However, the recommendations seemed to be a bit skewed to the novel side of the spectrum. Of the 20 artists recommended, I recognized only 2 (one I know I don't like, the other is already in regular rotation) - it would be helpful if Idiomag could offer some explanation of why an artist was recommended (such as 'we are recommending wolf eyes because their noise rock is similar in style to deerhoof')

The catalog of artists seems to be pretty deep - but I did notice some omissions. No Yo Yo Ma, no Maynard Ferguson, no Keith Fullerton Whitman.

One shortcoming of the API is that there doesn't seem to be any way to resolve ambiguous artists, so if I want recommendations for Prodigy (the rap artist) and not Prodigy (the guitarist) or The Prodigy (the UK electronic group), I am out of luck. There's no way to search the catalog for near spellings - so if you try to get information for 'beatles' instead of 'the beatles' you will get no results. Idiomag should adopt the model that last.fm and the Echonest use - to provide an artist search api that returns a list of scored matches with an associated musicbrainz id, and allow the id to be used in any subsequent method calls.

The API status feedback is also a bit thin - there's no way to tell from the API results if there's a problem. For instance, I tried to get recommendations based on seed artists Metallica and Yo Yo Ma (hoping to get a recommendation for Apocalypctica) - but the resulting recommendations were just the standard set of heavy metal bands - no sign of the cello at all. It took a while before I figured out that Yo Yo Ma wasn't factoring into the recommendations because Idiomag doesn't know about him.

One last issue, I can't seem to find any terms-of-use for the API that tells me how often I can make calls to the API, and whether I can use the data for commercial purposes - perhaps the TOS is hidden somewhere. But before I would consider using the API, I would really like to know what the rules of the road are.

Update 3 - idiomag has now published their terms-of-use It looks very similar (to my non-lawyerly eyes) to the Last.fm terms - free non-commercial use, commercial use is OK, but if you start making insane amounts of money, idiomag may start charging you for the service

Despite a few quibbles, I like the Idiomag API - it puts lots of interesting data in the hands of the developer, much of it is data that would be hard to come by without paying lots of money. Given the large number of music-related startups (here's 200 that started just last year alone) finding a way to help all of these companies provide context for their users about music seems like a good idea.

Update - the Idiomag folk sure are responsive. I posted this to my blog, went to lunch, came back, killed some zombies, and then checked my comments here to find that they've already fixed a number of the issues that I pointed out. Looks like they really care about what they are doing. I like that.

I've been enjoying all of the year-end 'best of' lists. My favorite so far has been the Hype Machine Music Blog Zeitgeist. Especially since it is so easy to listen to them:

I've also been enjoying the Pitchfork 500 - which is a list of the greatest songs since 1977 (the year I graduated from High School). This list is a bit harder to listen to than the Hype Machine lists - but I've been building up a Spotify playlist - the pitchfork500 on spotify: (I'm only up to 1979), but if you are a spotify listener feel free to listen. The song list is posted on from closer to near but you really will want to read the full book, each song has a 2 or 3 paragraph description of why the song was picked, why it was influential, where it fits in the evolving genre hierarchy.

I see on Josh Slack's blog that the Ardor3D team have made the first source code release for Ardor3D - the clean slate rewrite of JMonkeyEngine.

Here's a nifty interview of RJ of last.fm by Tom Corelis. Tom asked all the questions that I would have asked. Good job Tom. Here's what RJ had to say about Sun Hardware:

A few years ago we were buying from a local supplier here, and over the years it became more important to us to get really power efficient equipment, because at the data centers in London and the UK power is a real premium; it was hard to get enough power. So we started looking around for machines that were more tailored to low-powered stuff. So we have a mix of different suppliers but right now we’re buying quite a lot from Sun. We just got some new low-power blades that we’ve put in to do web serving, and our main database – with which we use PostgreSQL – is also on Sun hardware, for example. So yeah, we’ve been getting some good stuff from them. Sun seems to make a good range of servers that are quite conscious on the power requirements, and are quite good about giving you the spec about how much power they’re going to draw.

And Java

We actually do a lot of our storage and processing in Hadoop, which is a framework based on a paper that Google released on the same subject. So, that’s actually a distributed computing framework written in Java.

Girl Talk at Work

Working in bed is not such a bad thing

Girl Talk At Play

ffwd is a site with a goal of doing for video what Last.fm and iLike do for music. ffwd uses collaborative filtering to create video channels that are customized for a user.

This week I received an email from ffwd where they excitedly claimed to have some new recommendations for me, including this set of channels that I should check out:

Two of these seem to be rather general: Recommendations for 'business' and 'tv show' would be like the local convenience store recommending that I might want to buy some milk and bread. However, it is the third recommendation, the 'panties' channel that caught my eye, so to speak. I didn't recall doing anything on ffwd that would trigger an 'adult' recommendation, so this particular recommendation was a puzzler. I checked out my history on ffwd to see what they know about me to see if I could figure out why they gave me this particular recommendation. It turns out, ffwd knows only a little bit about me - I rated a few shows last year:

Its just typical nerd TV - nothing that should trigger a recommendation for a tour of the nether regions. More than likely, ffwd's collaborative filtering algorithm is just not dealing with sparse data very well, causing it to make freakomendations. Unfortunately for ffwd, these aren't just bad recommendations, they are likely to be offensive to some.

Sure enough, a few days later, I had this email in my inbox:

Subject: apology from the ceo

Dear plamere,

On behalf of the ffwd team, I apologize if our recent email recommendations didn't make sense, or worse, offended you. We are testing a new channel recommendation system and may have put it into service prematurely.

Most subscribers found the suggestions useful, but if you didn't, please be patient as we work out the kinks and simply respond honestly to the recommendations. By ignoring the mistakes, and clicking on the relevant suggestions, you will improve the system for you and all other viewers.

Again I apologize for any offensive recommendations and appreciate your patience as we improve our system.

Happy New Year!

Patrick
CEO, ffwd

And so it seems, I was not the only one of ffwd's users who received such an recommendation. As corporate apologies go, this is not a bad one, but I don't like this bit "Most subscribers found the suggestions useful, but if you didn't, please be patient...". This seems to imply that if I took offense, it is really my fault, not ffwd's, since most of the ffwd subscribers didn't seem to be offended. If you are going to apologize, just do it, don't waffle or weasel about it.

Over the holiday break, I like to do a bit of recreational programming ... this year I shall be working on the Music Explaura. This is a rich internet application that will allow you to explore your own music collection, get recommendations for new music and generally just have fun while discovering new music. Of course it will hook up to Project Aura to get all the recommendations. I have enough of the pieces put together that I was able to do a little demo showing some of the interface. I made a video and posted it to Youtube:

(and by the way, his is my first Youtube video)

And here are some screenshots:

Here's an interesting logic test to see how well you can relate music to abstract shapes and sensors.

Here are my scores:

Here's another Freakomendation from Amazon. Since you purchased The Starter Kit for the IBM iSeries and AS/400 - Amazon recommends Reducing stress-related behaviors in Persons with Dementia. I've never worked with AS/400 so perhaps this really is a valid recommendation.

Via reddit.

Nielsen SoundScan is reporting data about artists who have released songs on its "Guitar Hero" platform. Guitar Hero track sales rise by an average of 3X after the release on Guitar Hero. Some examples: More data in the AP Release. Via PC World, Game On

An early Christmas present from the Recording Industry of America. Music Industry to Abandon Mass Suits

Today on the Spotify blog, Andreas announced that Spotify now supports scrobbling to Last.fm. Once you set it up,all of your Spotify plays will be automatically scrobbled.

Setting this up just takes a couple of seconds in the Spotify preferences:

Andreas also points out this nifty Greasemonkey script that lets you play the Last.fm site in Spotify. Here's an example, the greasemonky script has added the green music notes to my 'Friends listening now' section. When I click on the green note, spotify opens, ready to play the artist, album or track. Totally cool.

Jeremy points to this nifty write-up on Anita's Music Box. If you are interested in music discovery visualizations, be sure to read Anita's thesis.

Photo stream from Last.fm's HackDay. Looks like great fun, - wish I could have been there.

The guys at the BBC's radiolabs are having way too much fun. There's no way can they call this 'work'.