DanielJohnBenton / SentimentAnalysis

:performing_arts: Score sentences in a source text by POSITIVE or NEGATIVE sentiment; find best and worst sentences; plot sentiment on graphs in various ways

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

β›³ Aims

  • Score sentences in a source text by POSITIVE or NEGATIVE sentiment
  • Find best (most POSITIVE) and worst (most NEGATIVE) sentences from the source text
  • Plot sentence sentiment over time through the source text
  • Plot sentence scores sorted from best to worst on a graph for distribution

πŸ”­ Method

Scoring sentences

Sentences were split using common punctuation delimiters and double spaces. A sentence is scored as the total of the scores of its words, e.g.:

We were happy(+3 πŸ‘) with the excellent(+3 πŸ‘) service, but the boring(-3 πŸ‘Ž) music made us uncomfortable(-2 πŸ‘Ž).

Sentence score: +1 πŸ‘

Scoring words

I used a file called AFINN (paper) which contains a list of words with a POSITIVE or NEGATIVE score.

Sentiment words have different scores to distinguish intensity - for example:

πŸ‘‰ Word Score
πŸ‘ outstanding +5
πŸ‘ fantastic +4
πŸ‘ delight +3
πŸ‘ advantage +2
πŸ‘ adequate +1
πŸ‘Ž alas -1
πŸ‘Ž upset -2
πŸ‘Ž violence -3
πŸ‘Ž catastrophic -4
πŸ‘Ž (expletives and racism) -5

It was made for analysing informal modern text (e.g. Tweets) and contains slang, but I have mostly used it for old, public-domain texts and found it to be very reliable.

Unlisted words are scored 0.

πŸ“š Analysis case studies

I mostly ran the analysis on public domain books from [Project Gutenberg] (https://www.gutenberg.org/). I also tried it out on movie scripts and a couple news articles. I am still adding to this section. :^)


πŸ™ Case study: King James Bible

Most positive sentence

πŸ‘‰ Sentence Score
πŸ‘ 12:43 Also that day they offered great sacrifices, and rejoiced: for
God had made them rejoice with great joy: the wives also and the
children rejoiced: so that the joy of Jerusalem was heard even afar
off.
+25

Most negative sentence

πŸ‘‰ Sentence Score
πŸ‘Ž 5:21 Ye have heard that it was said by them of old time, Thou shalt
not kill; and whosoever shall kill shall be in danger of the judgment:
5:22 But I say unto you, That whosoever is angry with his brother
without a cause shall be in danger of the judgment: and whosoever
shall say to his brother, Raca, shall be in danger of the council: but
whosoever shall say, Thou fool, shall be in danger of hell fire.
-25

πŸ“Š Graphs: Sentiment throughout the King James Bible

A graph showing sentence sentiment throughout the King James Bible

Graphing sentiment over the sentences of the text does not seem to reveal anything interesting. It is just not showing the right level of information.

So I tried splitting it out by book, and I think this reveals some more interesting features.

A graph showing sentence sentiment throughout the King James Bible The bars show the mean average sentiment score of sentences with sentiment score β‰  0 (sentences with score = 0 are ignored/discarded)

πŸ“Š Chart: Sentiment score distribution

A chart showing sentence sentiment score distribution in the King James Bible


🏰 Case study: The Complete Works of Shakespeare

Most positive sentences

πŸ‘‰ Sentence Score Source
πŸ‘ None here, he hopes,
In all this noble bevy, has brought with her
One care abroad; he would have all as merry
As, first, good company, good wine, good welcome,
Can make good people.
+23 Henry VIII
πŸ‘ O wonderful, wonderful, most wonderful wonderful, and yet
again wonderful, and after that, out of all whooping!
+20 As You Like It
πŸ‘ Tranio, since for the great desire I had
To see fair Padua, nursery of arts,
I am arriv'd for fruitful Lombardy,
The pleasant garden of great Italy,
And by my father's love and leave am arm'd
With his good will and thy good company,
My trusty servant well approv'd in all,
Here let us breathe, and haply institute
A course of learning and ingenious studies.
+20 The Taming of the Shrew

Most negative sentences

πŸ‘‰ Sentence Score Source
πŸ‘Ž Fear and be slain-no worse can come to fight;
And fight and die is death destroying death,
Where fearing dying pays death servile breath.
-21 Richard II
πŸ‘Ž Come down and welcome me to this world's light;
Confer with me of murder and of death;
There's not a hollow cave or lurking-place,
No vast obscurity or misty vale,
Where bloody murder or detested rape
Can couch for fear but I will find them out;
And in their ears tell them my dreadful name-
Revenge, which makes the foul offender quake.
-21 Titus Andronicus
πŸ‘Ž I'll put't in proof,
And when I have stol'n upon these sons-in-law,
Then kill, kill, kill, kill, kill, kill!
-20 King Lear

πŸ“Š Chart: Sentiment score distribution

A chart showing sentence sentiment score distribution in Shakespeare's complete works


πŸŽ„ Case study: A Christmas Carol by Charles Dickens

Most positive sentences

πŸ‘‰ Sentence Score
πŸ‘ He became as good a friend, as good a
master, and as good a man, as the good old city knew, or
any other good old city, town, or borough, in the good old
world.
+18
πŸ‘ And perhaps it was the pleasure the good Spirit had in
showing off this power of his, or else it was his own kind,
generous, hearty nature, and his sympathy with all poor
men, that led him straight to Scrooge's clerk's; for there he
went, and took Scrooge with him, holding to his robe; and
on the threshold of the door the Spirit smiled, and stopped
to bless Bob Cratchit's dwelling with the sprinkling of his
torch.
+16
πŸ‘ Wonderful party, wonderful
games, wonderful unanimity, won-der-ful happiness!
+15

Most negative sentences

πŸ‘‰ Sentence Score
πŸ‘Ž Not a latent echo in the house, not a squeak and scuffle
from the mice behind the panelling, not a drip from the
half-thawed water-spout in the dull yard behind, not a sigh among
the leafless boughs of one despondent poplar, not the idle
swinging of an empty store-house door, no, not a clicking in
the fire, but fell upon the heart of Scrooge with a softening
influence, and gave a freer passage to his tears.
-13
πŸ‘Ž At one of these a lonely
boy was reading near a feeble fire; and Scrooge sat down
upon a form, and wept to see his poor forgotten self as he
used to be.
-9
πŸ‘Ž He recoiled in terror, for the scene had changed, and now
he almost touched a bed: a bare, uncurtained bed: on which,
beneath a ragged sheet, there lay a something covered up,
which, though it was dumb, announced itself in awful
language.
-9

πŸ“Š Chart: Sentiment score distribution

A chart showing sentence sentiment score distribution in A Christmas Carol


πŸ”ͺ Case study: War and Peace by Leo Tolstoy

Most positive sentences

πŸ‘‰ Sentence Score
πŸ‘ Every young man who came to the houseβ€”seeing those
impressionable, smiling young faces (smiling probably at their own
happiness), feeling the eager bustle around him, and hearing the fitful
bursts of song and music and the inconsequent but friendly prattle of
young girls ready for anything and full of hopeβ€”experienced the same
feeling; sharing with the young folk of the RostΓ³vs’ household a
readiness to fall in love and an expectation of happiness.
+19
πŸ‘ Only it seems to me that Christian
love, love of one’s neighbor, love of one’s enemy, is worthier,
sweeter, and better than the feelings which the beautiful eyes of a
young man can inspire in a romantic and loving young girl like yourself.
+18

Most negative sentences

πŸ‘‰ Sentence Score
πŸ‘Ž Before partisan warfare had been officially recognized by the
government, thousands of enemy stragglers, marauders, and foragers had
been destroyed by the Cossacks and the peasants, who killed them off
as instinctively as dogs worry a stray mad dog to death.
-18
πŸ‘Ž It was not Napoleon alone who had experienced that nightmare feeling
of the mighty arm being stricken powerless, but all the generals and
soldiers of his army whether they had taken part in the battle or not,
after all their experience of previous battlesβ€”when after one tenth of
such efforts the enemy had fledβ€”experienced a similar feeling of terror
before an enemy who, after losing HALF his men, stood as threateningly
at the end as at the beginning of the battle.
-16

πŸ“Š Chart: Sentiment score distribution

A chart showing sentence sentiment score distribution in War and Peace


βœ‚οΈ Technologies used

  • Word/sentiment list: AFINN (paper)
  • Language: Javascript
  • Environment: Node.js
  • Graphs: Microsoft Excel
  • Libraries: None

πŸ† Acknowledgements

About

:performing_arts: Score sentences in a source text by POSITIVE or NEGATIVE sentiment; find best and worst sentences; plot sentiment on graphs in various ways

License:Other


Languages

Language:JavaScript 100.0%