Testing Accuracy
To test how accurately the function classifies songs we compared our function’s classification to descriptions of the song’s themes and messages provided by Genius. Although Genius annotations are made by the public, they are reviewed by editors on most popular songs and contributions made by the artists themselves are given priority, so we think this will be a reliable metric for our function’s accuracy (Genius, 2016)
To understand how our function reached its conclusion we printed the song’s lyrics twice, colouring each word according to the NLTK Sentiment analysis score with green as positive and red as negative, then according to the NRC emotion analysis with a different colour for each emotion.
​
​
​
​
​
​
​
​
​
​
​
First, for normal test data, we looked at songs which we thought should have an obvious classification. DIAMONDS by Rhianna was given a highly positive score and joy as the top emotion which fits with the Genius annotations saying that the song urges a “person not to succumb to the pressures of the world.” (Genius, 2012)
As is shown by our first printing of the lyrics, the NLTK score was so high because the large number of positive words increased the positive component of the score but also the only negative word present is preceded by ‘never’ a word which the NLTK function uses as a cue to negate the score of the following word.
I FALL APART by Post Malone was given a highly negative score and sadness as the top emotion in alignment with the artist’s own comment from an interview where he said he “dove into his really emotional side.” (Genius, 2016)
Our second printing of the lyrics shows that although a range of emotions are present in the song, sadness was the top emotion mainly thanks to the repetition of the word ‘fall’.
​
Whilst analysing our data, we found some classifications that initially seemed erroneous but upon further inspection their assessment seems correct. Here are two examples of songs where the NLTK score and top emotion initially seemed incompatible:
1) MY EX'S BEST FRIEND by Machine Gun Kelly X blackbear
‘My Ex’s Best Friend’ is described on Genius as a song about “hooking up with his ex’s best friend.” (Genius, 2020)
This theme is charged with a range of conflicting emotions, and this is reflected by our functions classification as we can see joy (yellow) and sadness (blue) are prevalent. The top emotion, joy, seems to align more with the events in the song and the significantly negative score seems to align more with the artist’s thoughts on those events.
2) CLOSE TO ME by Ellie Goulding
‘Close to Me’ is described on Genius as a song about the artists’ “connection” with “two kindred spirits… who fuel their fire” and “how wild they are with them.” (Genius, 2018)
The top emotion, fear, fits with the relationship struggles described in the song while the positive score reflects the artists’ hopeful outlook on the matter.
