Kevin Roitero's repositories
crowdsourcingTruthfulness
repository for the papers: "Crowdsourcing Truthfulness: The Impact of Judgment Scale and Assessor Bias" and "Can The Crowd Identify Misinformation Objectively? The Effects of Judgment Scale and Assessor's Background"
NewBestSub
mirror of https://github.com/Miccighel/NewBestSub
agreement-phi
mirror of https://github.com/AlessandroChecco/agreement-phi
CrowdsourcingRelevanceScales
repository for the paper "On the Effect of Relevance Scales in Crowdsourcing Relevance Assessments for Information Retrieval Evaluation."
Extended-HITS
Implementation of an extended version of the HITS algorithm proposed by Kleinberg which allows to include negative values for links weights.
how-many-crowd-workers
repository for the TOIS paper "How Many Crowd Workers Do I Need? On Statistical Power when Crowdsourcing Relevance Judgments"
human-and-machine-confidence-in-truthfulness
repository for the JDIQ paper "Combining Human and Machine Confidence in Truthfulness Assessment"
kevinroitero.github.io
website
LeToE-Code
code for the paper "Effectiveness Evaluation Without Human RelevanceJudgments: A Systematic Analysis of Existing Methodsand of their Combinations"
PairwiseAgreement
Pairwise Agreement Metric with tie management
preferences-on-a-budget
repository for the paper "Preferences on a Budget: Prioritizing Document Pairs when Crowdsourcing Relevance Judgments" accepted at TheWebConf 2022
Reproducibility_SI_EvalNoJudg
Repo for the paper: "Reproduce. Generalize. Extend. On Information Retrieval Evaluation without Relevance Judgments." by Kevin Roitero, Marco Passon, Giuseppe Serra, and Stefano Mizzaro.
set-size-estimation
python implementation of the tools from http://sakailab.com/download/