ritikamotwani / Deception-Detection

Detecting Deception using Verbal Cues | Dataset Used: Real life trial data collected during a series of experiments at Michigan (http://web.eecs.umich.edu/~zmohamed/PDFs/Trial.ICMI.pdf) and Deceptive Opinion Spam Corpus v1.4(https://myleott.com/op-spam.html)

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Deception-Detection

Classifiers added to detect deception through transcripts.
Dataset Used:

  1. Real life trial data collected during a series of experiments at Michigan (http://web.eecs.umich.edu/~zmohamed/PDFs/Trial.ICMI.pdf) (Folder - dataset)
  2. Deceptive Opinion Spam Corpus v1.4 (https://myleott.com/op-spam.html)

Installation

Run the following command to install all python packages that'll be used in the project:

pip install -r requirements.txt

Approaches

Opinion Spam Dataset:

Folder: OpinionSpam

  1. Parameters: NGram Approach Classifiers: SVM, NB, Random Forest
   To execute:
      python3 Ngrams_And_Classifiers.py 

SVM
Accuracy: 91.25%
F1Score: 90.27%
NB
Accuracy: 85.6%
F1Score: 83.6%
Random Forest
Accuracy: 68.7%
F1Score: 62.1%

  1. Parameters: LIWC Classifiers: SVM, NB, Random Forest
   To execute:
      python3 LIWC_And_Classifiers.py 

NB
Accuracy: 65.4%
F1Score: 71%
SVM
Accuracy: 79.1%
F1Score: 79.8%
Random Forest
Accuracy: 67.9%
F1Score: 59.6%


  1. Parameters: NGrams, LIWC Classifiers: SVM
   To execute:
      python3 SVM_Ngrams_LIWC.py

Accuracy: 84.4%
F1Score: 83.2%

  1. Recurrent Neural Networks
   To execute:
      python3 RNN

Accuracy: 71%

Real Life Dataset

Folder: RealLife

  1. Classifiers: SVM, NB
   To execute:
      python3  Classifiers.py

SVM
Accuracy: 73.7%
F1Score: 78.3%
NB
Accuracy: 68.4%
F1Score: 66.7%

  1. Parameters: NGram Approach Classifiers: SVM, NB, Random Forest
   To execute:
      python3 Ngrams_And_Classifiers.py 

SVM
Accuracy: 76.9%
F1Score: 80%
NB
Accuracy: 69.2%
F1Score: 71.4%
Random Forest
Accuracy: 46.1%
F1Score: 53.3%

  1. Parameters: LIWC Classifiers: SVM, NB, Random Forest
   To execute:
      python3 LIWC_And_Classifiers.py 

NB
Accuracy: 63.1%
F1Score: 70.99%
SVM
Accuracy: 57.8%
F1Score: 60%
Random Forest
Accuracy: 52.6%
F1Score: 40%


  1. RNN
   To execute:
      python3 RNN.py

Accuracy: 57.9%

About

Detecting Deception using Verbal Cues | Dataset Used: Real life trial data collected during a series of experiments at Michigan (http://web.eecs.umich.edu/~zmohamed/PDFs/Trial.ICMI.pdf) and Deceptive Opinion Spam Corpus v1.4(https://myleott.com/op-spam.html)


Languages

Language:Python 100.0%