If you go to one of the many social music sites out there and get
'similar artists' recommendations for Jimi Hendrix. You are
likely to get a list such as the one you get from Last.fm:
There's
no arguing that this is a good list - but it is also a rather diverse
list. Eric Clapton's blues guitar is quite different from the
psychedelic acid rock of the Doors. I'd really like to know a bit more
about the recommendations - in particular I'd like to know why a
particular artist was recommended. This can help me gain trust in the
recommender as well as help direct me to artists based on criteria
that are most relevant to me. Unfortunately, most
recommenders are based strictly on consumption habits, so the only
recommendation explanation available is the Amazonian "People who
listened to Jimi Hendrix also listened to Eric Clapton and the
Doors" - which is not too helpful for me.
We want to make
recommendations transparent - so that you can ask 'why did you recommend
this' and get a useful answer beyond the typical 'people who bought X
also bought Y'. In order to generate transparent
recommendations you need to have some understanding of the content -
there has to be some way of knowing that Hendix tracks typically contain
distorted guitar, for example. Companies like Pandora, with their
music genome project have spent years analyzing to hundreds of
thousands of tracks, assigning 400+ attributes to each track. This lets
Pandora give those great explanation that makes them so popular with
their users:
Pandora
can generate these excellent explanations because they've taken the
time (and spent lots of money) to listen to and extensively label all of
their music. Most companies won't have the time, money or
patience to do this - and even Pandora that is committed to this
approach, can't keep up with the volume of new music that is generated
every year. Luckily, there are other sources of content
description that we can use to generate recommendations.
One such
source are social tags. Social tags have been all of the rage in
the web 2.0 world. Sites like del.icio.us, Flickr and Last.fm
demonstrate how the such tags can be extremely useful for searching and
organizing content, especially non-text content. Social tags can
also be used to give us good transparent recommendations.
Here are the most frequent social tags applied to Jimi Hendrix at Last.fm:
Now
tags like 'rock' and 'blues' occur rather frequently for many artists,
so are less descriptive than tags like 'guitar god' which occur
infrequently across artists. So we can take this into account and
generate a list of the most distinctive tags for Jimi Hendrix.
These are tags that occur rarely across the entire set of artists, but
occur frequently for Hendrix. (This is the classic TF-IDF term weighting technique). Here are the distinctive tags for Hendrix:
The
less descriptive tags such as 'rock' have fallen off the list while
extremely targeted tags like 'jimi hendix' and 'acid rock' have risen to
the top. This list is a much better description of Hendrix than
the 'Frequent Tags' list and can serve as the basis for our transparent
recommendations.
With these distinctive tags, we can generate recommendations based upon the cosine distance
of these distinctive tags to the distinctive tag sets of other
artists. This type of recommendation, based upon the similarity of
distinctive tags, gives us surprisingly good results. My
colleague, and resident neologist, Steve,
coined the term 'tagomendations' for recommendations based on the
social tags. Here are the tagomendations for Jimi Hendrix (ordered
by artist popularity):
Interestingly
(and surprisingly) our tagomendations compared favorably to
recommendations generated with the more traditional collaborative
filtering techniques when we evaluated them in a survey. And of
course, with these tagomendations based upoon distinctive tags we can
now explain why a recommendation was made, based upon how the
distinctive tags for two artists overlap.
For instance, for the Clapton recommedation, we can look at how the distinctive tags for Clapton and Hendrix overlap:
Clearly
if you like Hendrix because of his blues guitar playing, you might want
to give Clapton a listen. Compare this to the overlapping tags of The
Door and Hendrix:
This is quite a different vibe - with the focus on psychedelia of the 60s.
These
tag clouds showing how the distinctive tags for recommended artists
overlap with the seed artist give me the opportunity to explore the
recommendations based on my taste. If I like Hendrix because I am a
fan of face-melting guitar, I will quickly find artists, like Joe
Satriani, Gary Moore, or Steve Vai - but if the reason I like Hendrix is
because of the 60s, psychedelic vibe, I may find Jefferson Airplane or
Steppenwolf more to my taste.
That's it in a nutshell - how we are
using social tags to generate transparent, explainable
recommendations. And by the way, to do the heavy lifting for
our Tagomendations we are using the text search engine called
Minion, developed by the Advanced Search Technology
group here at Sun Labs. Minion is a high quality, highly
configurable search engine that is perfect for doing these types of
experiments. Look for Minion, It's coming to an open source
repository near you very soon.
Likewise, we hope to release our
web-based music explorer that can generate transparent tagomendations
soon. Here's a little bit of what it looks like:
Music
sites like Last.fm and Pandora that offer you a customized radio based
upon your listening tastes have been extremely popular over the last few
years. These sites are like listening to a radio station where
the DJ knows exactly what kind of music you like. These customized
radio stations are great for general listening and as a way to discover
new music. But just like traditional radio, if you in the
mood for a particular song by a particular artist, you are out of luck -
Internet radio providers have to follow a strict set of rules to make
sure that listeners aren't too happy with their Internet
listening. You can't play an artist or track on demand and
the number of 'skips' are limited. If you really want to listen to
Deep Purple's "Smoke on the Water" you are out of luck.
