Meta-Science and Evaluation (20-00-1086-se Meta-Science)
Content
This is a seminar series held each summer semester at TU Darmstadt discussing the analysis of scientific text using natural language processing (NLP) techniques, forecasting methods (e.g., for citation counts) as well as the proper evaluation of NLP/ML systems.
Based on the current revolution in deep learning, one of the questions we ask in the seminar is whether such revolutions can be predicted in advance. In addition, the seminar has a focus on the correct evaluation of models in NLP and Machine Learning since this is the basis for correct assessment of progress in science.
Particular issues discussed:
- Analysis of scientific literature with NLP methods
- Trend and citation count prediction
- Evaluation methods and metrics
- poor research practices: biases in research, poor study design, abuse of statistics, false claim of superiority of one method over another
- citation cartels & cliques
- ethics in research, including self-citations, and forms of cheating