Classifier-based Cranfield search
What is this?
This is a very simple ranking implementation using classifier for the cranfield dataset.
What should be the goal for this project?
Update the 1) create_model file, 2) make sure you implement evaluate_model andload_models functions as required such that you can increase the nDCG score.
What is the cranfield dataset?
Cranfield is a small curated dataset that is very extensively used in the information retrieval experiments. In the dataset, there are 226 queries (search terms), 1400 documents, and 1837 (evaluations). The dataset is supposed to be complete in the sense that the documents that should be returned for each known are known. This makes the evaluation easier. Click here more details
What is nDCG score?
nDCG is a very common metric used in search evaluations. Higher nDCG score (close to 1.0 ) describes a search system that gives all the relevant results with most relevant ones on the top.
What is the current nDCG score?
Final ncdg for all queries is 0.197136293042
What are the current classification metrics?
label precision recall f1-score support
1 0.24 0.17 0.20 122
2 0.25 0.23 0.24 120
3 0.39 0.50 0.44 246
4 0.17 0.13 0.15 119
avg / total 0.29 0.31 0.30 607
What I should not do?
Please do not modify search.py or readers.py or eval.py