Unifying Data Perspectivism and Personalization: An Application to Social Norms
Codebase for EMNLP22 work Unifying Data Perspectivism and Personalization: An Application to Social Norms and NLP+CSS work Understanding Interpersonal Conflict Types and their Impact on Perception Classification
Dataset
You can download our dataset here.
Running
- Create user attribution embeddings
python attribution_training.py --split_type='verdicts' --path_to_data='data/'
-
Extract user embeddings
- Extract average embeddings:
python users_sbert_embeddings.py --path_to_data='data/' --dirname='data/path_to_history_comments/' --output_dir='data/embeddings/'
- Extract user attribution embeddings:
python users_attribution_embeddings.py --checkpoint_dir='verdicts_linear_layers_2022-06-16_12:12:57:866975.pt' --embedding_type='prediction'
- Extract average embeddings:
-
Find the scripts to run models here
Citation
If our code or models aided your research, please cite our paper and if you used the models or data for conflict types please cite our NLP+CSS paper:
@inproceedings{plepi-etal-2022-unifying,
title = "Unifying Data Perspectivism and Personalization: An Application to Social Norms",
author = "Plepi, Joan and
Neuendorf, B{\'e}la and
Flek, Lucie and
Welch, Charles",
booktitle = "Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing",
month = dec,
year = "2022",
address = "Abu Dhabi, United Arab Emirates",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2022.emnlp-main.500",
pages = "7391--7402",
abstract = "Instead of using a single ground truth for language processing tasks, several recent studies have examined how to represent and predict the labels of the set of annotators. However, often little or no information about annotators is known, or the set of annotators is small. In this work, we examine a corpus of social media posts about conflict from a set of 13k annotators and 210k judgements of social norms. We provide a novel experimental setup that applies personalization methods to the modeling of annotators and compare their effectiveness for predicting the perception of social norms. We further provide an analysis of performance across subsets of social situations that vary by the closeness of the relationship between parties in conflict, and assess where personalization helps the most.",
}
@inproceedings{welch-etal-2022-understanding,
title = "Understanding Interpersonal Conflict Types and their Impact on Perception Classification",
author = "Welch, Charles and
Plepi, Joan and
Neuendorf, B{\'e}la and
Flek, Lucie",
booktitle = "Proceedings of the Fifth Workshop on Natural Language Processing and Computational Social Science (NLP+CSS)",
month = nov,
year = "2022",
address = "Abu Dhabi, UAE",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2022.nlpcss-1.10",
pages = "79--88",
abstract = "Studies on interpersonal conflict have a long history and contain many suggestions for conflict typology. We use this as the basis of a novel annotation scheme and release a new dataset of situations and conflict aspect annotations. We then build a classifier to predict whether someone will perceive the actions of one individual as right or wrong in a given situation. Our analyses include conflict aspects, but also generated clusters, which are human validated, and show differences in conflict content based on the relationship of participants to the author. Our findings have important implications for understanding conflict and social norms.",
}