Python program for understanding the relationship between gaze and entrainment at different linguistic levels.
We used the Gaze Aversion corpus to study and extract lexical, syntactic, semantic, and acoustic-prosodic features in an HRI corpus.
The dataset is available upon request by mailing it to the original creators of the dataset (https://www.frontiersin.org/journals/robotics-and-ai/articles/10.3389/frobt.2023.1127626/full)
The input folder provides examples of sample files needed for processing.
textgrid (Install textgrid from https://github.com/kylebgorman/textgrid)
ffmpeg (Download from https://www.ffmpeg.org/download.html)
transformers (pip install -U flash-attn --no-build-isolation)
sentence-transformers (pip install sentence-transformers)
tensorflow (pip install tensorflow)
PRAAT toolkit (Download from https://www.fon.hum.uva.nl/praat/download_win.html)
TRILL vectors model (Download from https://tfhub.dev/google/nonsemantic-speech-benchmark/trill/3)
Stanfor CoreNLP (https://github.com/stanfordnlp/CoreNLP) (Download from https://drive.google.com/file/d/1iQlFl9laJ1bK6qziqLqKfqcT_MRdNN62/view?usp=sharing)
Stanza (pip install stanza)
A Jupyter Notebook file is uploaded. It presents a step-by-step procedure for extracting features and measuring entrainment.
J. Kejriwal, C. Mishra, T. Offrede, G. Skantze and Š. Beňuš, "Does a Robot’s Gaze Behavior Affect Entrainment in HRI?," (2024). Submitted to Computing and Informatics (Paper accepted).