Duke Listens!: Visit my main blog at MusicMachinery.com

The folks who rendered the artistic visualization of artist similarity have just released a new set of images that shows the flickr social network. Very nice.

First seen on The arts journal.

Brian Whitman and Tristan Jehan ... two MIT media-lab alums that have done interesting work using automatic music analysis to inform and control music synthesis, have teamed up to form a company called The Echo Nest. They are currently in stealth mode, but Brian's home page calls it the "largest eigenvalue problem in Davis Square.", which is saying quite a bit, given the size of eigens typically seen roaming the square (especially at night). - Thanks to Graham

Let's continue our look at music recommendation services ...

last.fm is a personalized streaming radio and music recommendation system. Last.fm builds up a listening profile for each user and based upon their listening habits can make recommendations and generate a 'radio' station (streaming audio) customized to the tastes of each listener. What makes last.fm unique among all of the music recommenders out there is that it uses an audioplayer plug-in to track what the user is playing, instead of relying on user ratings or uploaded playlists. When you enroll in last.fm, you can download and install the audioscrobbler plugin. Whenever you play a song, the plugin 'phones-home' with the details about what song you played, when you played it, and who you are. This means that last.fm can track not just what songs you have, or what artists you collect, but what music you are actually listening to. I think this is an extremely important advantage that last.fm has over other recommenders. Most people have songs in their music collection that they never listen to (for instance, my 10-year-old daughter's DisneyMania CDs are part of my music collection), a recommender that bases recommendations on what is in your collection is going to use those songs that you never listen to and give you bad recommendations. With the last.fm model, only the songs that you listen to factor into the recommendations. The more you play a song, the more weight that song is given. It is like having an automatic rating system for your music.

There are some other advantages to the last.fm model. Since it relies on an instrumented player to automatically send info back to the server, last.fm has been able to amass a very large database of music profiles. For any kind of recommendation system, the more data the better. Last.fm gives very good music recommendations (the best I've seen) with very good coverage (it is extremely rare to encounter a band that last.fm doesn't know about). Just to give you an idea of their coverage, here are some stats. In the last.fm database:

• 1,240,919 people have listened to the Beatles
• 698,795 people have listened to Weezer
• 18,324 people have listened to Dave Brubeck

One really cool thing about last.fm is that since they are watching what thousands of people are playing, they have extremely accurate and up to date music charts. One can watch a new song or CD rapidly climb the charts. No need to wait a week for the new Billboard to arrive.

There are some downsides to the last.fm model. For one thing, having to download and install a music player plugin is a technical barrier to a very large class of listeners. The non-technical users are definitely under-representated in the last.fm database. Looking at the top artist charts we find bands like Coldplay, Radiohead, Weezer and the Red Hot Chili Peppers, definitely a geek bias here, (not that that is necessarily a bad thing though...). And of course there's the whole privacy issue. Some folks just don't like the idea that their music player is sending information about their music listening habits to some remote server.

Last.fm seems to be doing a lot of things the right way. They are friendly to developers, they publish their plugin protocol, they expose their derived data via a web services interface. They even release dumps of their database under a creative commons license for use by researchers.

It will be interesting to see if last.fm can compete even as the mega-music sites like Yahoo and iTunes move into this space. I think they have a lot to offer.

This weekend we took another visit to Greenfield NH to watch the Yankee Farmer toss pumpkins a thousand feet with his Trebuchet:

The Yankee Siege holds the world record trebuchet pumpkin toss of 1394.29 feet. It's great fun, plus we picked up some naturally delicious apples while we were there.

Up to 11 is a music recommender service that generates recommendations based upon the analysis of playlists and collections on P2P networks. Using their 'proprietary algorithms' they can infer artist similarity based upon the frequency of co-occurrence of artists. They use this artist similarity to generate recommendations for you. For instance, if you go to up to 11 and enter 'led zeppelin' you'll receive recommendations for: Jimi Hendrix, Pink Floyd, The Animals, Deep Purple, the Rolling Stones and so on. Up to 11 will point you at the Wikipedia entry for the band and will point you at the usual places to buy music (amazon, iTunes).

The main drawback with Up to 11 right now seems to be the size of their usage database. In the Up to 11 database, Led Zeppelin is found in about 8500 music collections while at last.fm (another music recommender) Led Zeppelin is found in the collections of over half a million people. The more data, the better. It may not matter too much for the big bands like 'Led Zeppelin' but for the smaller bands it makes all the difference in the world. For instance, I asked for recommendations based upon the 70's prog rock band 'the nice' (keith emerson's gig before ELP). The top 5 recommendations from Up to 11 are:

• Rammstein
• Dusty Springfield
• Enya
• Enigma
• George Winston.

Any recommendation that includes both Rammstein and Enya is certainly suspect. now compare this to the results from last.fm:

• Keith Emerson
• Emerson, Lake & Palmer
• Strawbs
• Curved Air
• United States Marine Band

Well, that 'united states marine band' recommendation is certainly suspect, but the others make pretty good sense.

Music recommenders like last.fm and Up to 11 seem to be all the rage right now. It will be interesting to see what happens in this space over the next year or so.

MusicLens is a search system and interface for selecting and building music playlists. Instead of the normal browse through the a music collection by scrolling through artist, genre, album and song menus, when you use MusicLens you adjust sliders associated with the music properties. If you want quiet music, slide the appropriate slider to the 'silent' end, or if you want loud, driving music, slide it to the 'ear-busting' end. MusicLens give you sliders associated with speed, instrumentation, emotion, year of release. If you don't care about a particular property you can deselect a slider and that property won't be used in selecting music. MusicLens is certainly a novel way of accessing a music collection. It relieves one of having to browse through the words that surround the music. Instead, you can interact with your music collection using musical terms. I can say: give me a playlist of fast, happy songs with female vocalists, and I'll get a list that may span genres and artists, but fits that criteria.

I'm not so sure though, that the slider interface is appropriate for all of these properties. Some properties such as emotion are hard to represent on a linear scale. Does 'happy' come before or after 'content' and 'joyful'. Is 'angry' closer to 'sad' or 'frantic'. Still, I do like the idea that you can use musical terms to interact with your music collection instead of just the words.

More info in this paper.

One of the most beautiful renderings of the music space is shown in The World of Music by researchers at Standford, MIT and Yahoo!. This visualizations shows 10,000 artists and how they are related to each other. The artist relation data is mined from user ratings of artists in the Yahoo! Music service. They use a technique called semidefinite programming (which is sometimes called Semidefinite embedding) to layout and cluster the data. Semidefinite embedding is a method for mapping high dimensional data into a lower dimensional euclidean vector space.

There's a brief paper on how this visualization was constructed here: The World of Music: SDP layout of high dimensional data

Learn more about: Semidefinite programming

MusicMagic Mixer, by Predixis, is a personal music organizer and player that along with the usual features that you associate with such a program will perform a content-based analysis of your music collection to derive music similarity information. It will then use this information to help you build playslists (they call them Power Mixes) .

Unfortunately MusicMagic Mixer fails to run for me on my machine. I get an error about not being able to load the 'NativeEngine' library. Bummer, as I'd like to try it. Anyway, MusicMagic Mixer is one of the first personal music organizers that will use content analysis to help generate playlists. I think we shall be seeing a whole lot more systems like this soon, however, with more and more people buying music from places like iTunes (with its DRM), it will get harder and harder for the content analysis to be done on the client, since the DRM will not allow the analysis to get at the bits. Instead, the analysis will need to be done on the server at the music service before the DRM is applied. Update: The folks at Predixis indicate that they are working on an update that should address my problem

Pandora (formerly 'Savage Beast') is a service that will automatically fill your playlist with songs that it thinks you will like based upon an initial seed artist and any ongoing feed back you provide ('I like this' , 'Never play this again'). It is similar to Last.FM in that it attempts to build a personal radio station based upon your listening habits, however there are some rather fundamental differences too. First, you pay for Pandora. It costs $36 per year, which is half the price of a subscription to somthing like Yahoo Music Unlimited. Second, Pandora recommends music not just based upon collaborative filtering of music usage (like Last.FM), it also takes into account the music content. Interestingly, this content analysis does not seem to be done by computational means, but instead real people listened to music and 'analyzed the musical qualities of each song one attribute at a time.'

Its an interesting idea, but I'm not sure of the business model. For a few dollars more, I can get access to millions of songs that I can play in any order I want with a subscription service like Yahoo Music Unlimited. With Pandora, I can only listen to music that Pandora thinks I might like. This may be OK for discovering music, but finding new music is just one part of the listening cycle. When I find a song or artist that I like, I tend to spend a lot of time listening to the same songs over and over until I really know them. I can't do that with Pandora. (I can't even listen to the same song twice in a row).

Probably one of the most unusual ways of interacting and exploring a music collection is offered by Stephen Baumann a researcher on AI, media and music. Stephen points me to MP3 Konzert Archiv. Mp3 Konzert Archive was an exhibition built in an old markethall. They transformed the main hall into a 'meadow of music' by covering the floor with real grass and planting 'music flowers' that played songs from different playlists. This exhibition was, in effect, a huge analog interface to a large music collection. Visitors interacted with the collection by listening to the various flowers.

An interesting and novel idea for sure.

Continuing the theme of visualizing large music collections ...

The Databionic MusicMiner is an open-source music browser that uses data mining techniques to create 'MusicMaps' that visualize the similarity of songs and artists. The maps allow you to explore your music collection and create playlists directly from these maps.

Here's the browser interfacce:

Here's a MusicMap:

This system was developed by the Databionics Research Group at the University of Marburg, Germany. This group has released a number of open source tools that perform data mining tasks such as clustering, visualization and classification with Emergent Self-Organizing Maps. There's a paper giving an overview of their toolkit here: ESOM-Maps: Tools for clustering, visualization, and classification with Emergent SOM

One interesting problem in dealing with very large music collections is how to visualize such a collection. The nested menus of genre/artist/album/song method that most players such as the iPod use fail miserably as the size of the music collection gets very large. One method for visualizing large collections of music that is emerging in the music information retrieval research community is to use Self-Organizing-Maps (SOM) to layout and display songs. A SOM attemps to represent a high dimensional space through the use of self-organizing neural networks.

MIR researcher Elias Pamplak pioneered the use SOMS in his "Islands of Music" project. This project explores how to help the user explore vast amounts of music in an efficient way. Islands of music are generated automatically based on psychoacoustics models and self-organizing maps.

Images from 'Islands of Music' by Elias Pampalk

Self-organizing-maps are an appealing way to visualize music because they are easy to understand. People have experience with geographic maps and can tranfer the skill to SOMs. On the other hand, SOMs are computationally expensive to generate.

There's an excellent tutorial on SOMs written by Tom Germano at this page: Self Organizing Maps. He includes a Java applet demonstration of SOMs.

Some highlights from ISMIR day 4:

• Masataka Goto Masataka, in his enthusiastic way demonstrated his Musicream system. This system is intended to help people explore very large music collections. The interface is quite novel. It has the Wow factor. Check out the video at Masataka's page: Musicream. Someone should probably mention to Masataka that 'musicream' may not be the best name for the American and UK audiences
• MusicMiner - The data bionics research group presented MusicMiner - a browser for music based on data mining techniques. You can create MusicMaps to visualize the similarity of songs and artists.

  

• Jean-Julien Aucouturier Demonstrated the Ringomatic A real-time musical agent that generates a drum-track by concantenating audio segments automatically extracted from pre-existing music files. It's Ringo Starr in a box. Jean-Juliean played his squeezbox while the Ringomatic machine accompanied him. Great Fun.

Well ... again I'm pressed for time. I'm hoping to be able to take a deeper look at interesting things over the next few weeks so keep reading.

Lots of good talks today, some great posters, an excellent panel session and a reception at the British library. An excellent day at ISMIR! Some interesting links encountered:

• Variations 2 The Indiana University Digital Music Libarary.
• NIME New Interfaces for Musical Expression
• The Music Animation Machine
• A Survey of Music Information Retrieval Systems
• Laboratory for the recognition and organization of speech and audio
• Sampling History - A rather cool animation showing when music sampling started to become common.
• BBCut2 a SuperCollider extensions library that suports bbcut2 supports beat tracking of audio streams and the scheduling follows induced clocks. There is an event analysis component which can construct a database of tagged events in an audio stream on-the-fly or as an NRT process.
• The Continuator The Continuator project involves a real time interaction with a system that learns musical styles. With the Continuator, users can play music as they wish, and the system will automatically engage in a dialogue by producing musical phrases designed as continuations of user input.
• Popular music browser
• Plunderphonics Sampling, mash-ups, cut-ups and more.
• Folksonomy Social tagging.
• The Illustrious Company Music soundscapes or exhibitions
• The British Library
• The Semantic Grid

The Panel Session

The panel session discussion for the day was all about things that can be done or will be done with MIR. Lots of forward thinking stuff, lots of new ideas. THe highlight of the day.

British Museum

The evening reception at the British Library offered wine and a tour of the BL Treasures. Very Nice.

I'm going to have to keep this update short, since it is time to head into ISMIR day 3. I'm giving a poster session today on our Artist Identification submission to MIREX. If you are reading this and attending ISMIR, drop by and say 'hi'.

I wanted to mention four highlights of ISMIR day 3:

Stephen Robertson Microsoft researcher and Professor of information science at City University of London - gave an excellent keynote on the 'ascendency of words". One of my take-away was: "When looking for data, it is better to go for richness than to eliminate noise"

The Mirex Panel - a discussion of lessons learned from this years MIREX and some early plans for next years MIREX

Meinard Muller of uni-bonn.de gave one of the best talks of the week. He described a method for audio matching using chroma-based features. There's a paper describing some similar work here.

Thomas Dolby (yes, that Thomas Dolby) gave a talk about sonification, using sound to explore data, very interesting.

Sorry so short today, hopefully more later ...