BM25
A Python implementation of the BM25 ranking function.
Implementation
There are 4 main modules of the program: parser, query processor, ranking function, and data structures. The parser module parses the query file and the corpus file to produce a list and a dictionary, respectively. The query processor takes each query in the query list and scores the documents based on the terms. The ranking function is an implementation of the BM25 ranking function; it uses the natural logarithm in its calculations. Finally, the data structures module contains an inverted index and a document length table. The inverted index use a dictionary to map each word to a dictionary; this secondary dictionary maps each document id to the word frequency in the outer dictionary. The document length table contains the length of each document, and also has a function to calculate the average document length of the collection.
How To Run
To run, simply run $ python main.py in the src folder.
更新说明
对原版本稍作修改： 1.原版本未考虑一个单词在查询（query）中出现的频次 2.将文集（corpus）格式改为，一行即为一篇文档，文档号为行号（从0开始）
使用说明
查询在queries文档中，文档在corpus文档中，最终生成一个结果文档result，格式为查询号、排名、文档号和得分