Determining synonyms from tags
One issue that is encountered when working with social tags is
synonymy, that is, taggers have lots of ways to say the same thing.
For instance, looking at the tags that have been applied to an artist
like Jay-Z we see tags such as "Hip-hop", "hiphop", "hip hop" and
"rap". Now it it pretty clear that "Hip-hop" and "hiphop" probably
mean the same thing. This diversity can be good sometimes - for
instance, if we know that "hip-hop" and "hip hop" usually mean the
same thing then if someone is searching for "hip-hop" we can include
results that match "hip-hop" and "hip hop". Of course, we could
figure out that "hip-hop" and "hip hop" are closely related just by
using text methods - but things are not usually so easy. What we'd
like to do is develop a method that can look at the set of tags and
determine which tags are synonyms, and which tags are closely related
but are probably not synonyms (such as "hip hop" and "rap").
For a first experiment, I'd like to see if I can automatically find synonyms for the tag "female vocalists". To do this, I need to establish some ground truth. By hand, I've gone through the 5,000 most frequently applied artist tags, looking for tags that may be related to "female vocalists". I found 59 of them shown here (along with the tag rank and tag frequency).
12 89277 female vocalists 80 15874 female 113 8281 Female fronted metal 128 7150 female vocalist 227 2841 riot grrrl 236 2716 female vocals 278 2180 Female Voices 365 1424 female artists 480 955 female vocal 569 698 female singers 571 691 female fronted 619 619 Girl Groups 633 600 Girl Rock 722 501 girls 739 475 diva 786 433 riot grrl 880 374 female singer-songwriter 885 370 women 1023 301 girl group 1064 289 chick rock 1067 287 girl power 1119 269 female singer-songwriters 1202 245 female singer 1224 239 female voice 1544 179 female-vocalists 1587 173 female rock 1625 167 female dance vocals 1650 163 girl music 1727 154 french female 1757 151 chick music 2113 120 Girl 2130 118 Female-fronted Metal 2246 110 chicks 2252 109 woman singer 2681 89 female fronted rock 2757 86 Female Artist 2803 84 girl bands 2893 81 girlie rock 2895 81 female fronted band 3077 75 japanese female vocalists 3082 75 divas 3135 73 girly 3136 73 girl band 3194 71 girl pop 3201 71 eleCtro grrls 3263 69 solo female 3405 65 with female singers 3426 65 front girl band 3609 62 60s girls 3676 60 grrl 3788 58 females 3884 56 woman 3957 55 female vocal trance 4044 54 Female country 4188 51 grrls 4370 49 Female solo artists 4778 44 Favourite Females 4828 43 girl punk 4923 42 girls aloudNow I want to place the tags into 3 separate buckets - In bucket 1, I'll put tags that I think are synonyms for "female vocalists". In bucket 2, I'll put tags that are related but not synonyms, and in bucket 3, I'll place tags that are not related to "female vocalists" at all.
Bucket #1 - Synonyms for "female vocalists"
These are female oriented tags (singular or plural), that don't imply any type of genre.12 89277 female vocalists 80 15874 female 128 7150 female vocalist 236 2716 female vocals 278 2180 Female Voices 365 1424 female artists 480 955 female vocal 569 698 female singers 571 691 female fronted 619 619 Girl Groups 722 501 girls 739 475 diva 885 370 women 1023 301 girl group 1202 245 female singer 1224 239 female voice 1544 179 female-vocalists 1650 163 girl music 1757 151 chick music 2113 120 Girl 2246 110 chicks 2252 109 woman singer 2757 86 Female Artist 2803 84 girl bands 2895 81 female fronted band 3082 75 divas 3135 73 girly 3136 73 girl band 3263 69 solo female 3405 65 with female singers 3426 65 front girl band 3788 58 females 3884 56 woman 4370 49 Female solo artists
Bucket #2 - Related but not synonyms to "Female Vocalists"
female-oriented tags that imply genre or include another type of qualifier such as 'favorite'.113 8281 Female fronted metal 227 2841 riot grrrl 633 600 Girl Rock 786 433 riot grrl 880 374 female singer-songwriter 1064 289 chick rock 1067 287 girl power 1119 269 female singer-songwriters 1587 173 female rock 1625 167 female dance vocals 1727 154 french female 2130 118 Female-fronted Metal 2681 89 female fronted rock 2893 81 girlie rock 3077 75 japanese female vocalists 3194 71 girl pop 3201 71 eleCtro grrls 3609 62 60s girls 3676 60 grrl 3957 55 female vocal trance 4044 54 Female country 4188 51 grrls 4778 44 Favourite Females 4828 43 girl punk 4923 42 girls aloud
Bucket #3 - Not related to "Female Vocalists"
(all of the rest of the 5,000 tags).Dividing the female-oriented tags like this is not so clear cut, but we have to start somewhere ... I'm open to any other suggestions as to how to divide this space up. Once we have some ground-truth (imperfect as it may be), we can develop an evaluation criteria that will let us determine how well our synonym detector works.
The next step is to figure out how we can evaluate our synonym predictor. That will be the next post.
I'd argue that there is a very important difference between "girly" and "female vocalist". But I guess it really depends on what you plan to do with those lists. Doing a query expansion if a system does not return enough results sounds reasonable. Using it to compute similarity could be tricky.
Posted by elias on June 26, 2008 at 05:18 PM EDT #
Hi paul, just came across your blog this is very interesting - I'm going to read your white paper when I have a minute. I think you might be interested in the new beta of Jaikoz Audio Tagger ( http://www.jthink.net/jaikoz/jsp/beta/start.jsp )that supports reading Musicbrainz Folksonomy tags as genres AND submitting them genres back to Musicbrainz Folksonomy. Jaikoz is written in Java so possibly we could help each other with the issue of accurate genre tagging.
Posted by paul taylor on July 01, 2008 at 06:15 AM EDT #
Would somebody (Paul, Elias?) know of any ground truth data out there for this kind of experiments? That is, a bunch of music-related tags divided into synonyms and not-synonyms.
Posted by fabien on July 11, 2008 at 04:42 AM EDT #