JASSjr

JASSjr, the minimalistic BM25 search engine for indexing and searching the TREC WSJ collection.

Released under the 2-clause BSD licence.

Please fork our repo. Please report any bugs.

Why?

JASSjr is the little-brother to all other search engines, especially JASSv2 and ATIRE. The purpose of this code base is to demonstrate how easy it is to write a search engine that will perform BM25 on a TREC collection.

This particular code was originally written as a model answer to the University of Otago COSC431 Information Retrieval assignment requiring the students to write a search engine that can index the TREC WSJ collection, search in less than a second, and can rank.

As an example ranking function this code implements the ATIRE version of BM25 with k1= 0.9 and b=0.4

Gotchas

If the first word in a query is a number it is assumed to be a TREC query number
There are many variants of BM25, JASSjr uses the ATIRE BM25 function which ignores the k3 query component. It is assmed that each term is unique and occurs only once (and so the k3 clause is set to 1.0).

Usage

To build simply use

make

To index use

JASSjr_index <filename>

where <filename> is the name of the TREC encoded file. The example file test_documents.xml shows the required file format. Documents can be split over multiple lines, but whitespace is needed around <DOC> and <DOCNO> tags.

To search use

JASSjr_search

Queries a sequences of words. If the first token is a number it is assumed to the a TREC query number and is used in the output (and not searched for).

JASSjr will produce (on stdout) a trec_eval compatible results list.

Java

The Java version is build in the same way, but run with

java JASSjr_index <filename>

and

java JASSjr_search

Evaluation

Indexing the TREC WSJ collection of 173,252 documents takes less than 20 seconds on my Mac (3.2 GHz Intel Core i5).
Searching and generating a trec_eval compatible output for TREC queries 51-100 (top k=1000) takes 1 second on my Mac.
trec_eval reports:

runid                 	all	JASSjr
num_q                 	all	50
num_ret               	all	46725
num_rel               	all	6228
num_rel_ret           	all	3509
map                   	all	0.2080
gm_map                	all	0.0932
Rprec                 	all	0.2563
bpref                 	all	0.2880
recip_rank            	all	0.5974
iprec_at_recall_0.00  	all	0.6456
iprec_at_recall_0.10  	all	0.4286
iprec_at_recall_0.20  	all	0.3451
iprec_at_recall_0.30  	all	0.3005
iprec_at_recall_0.40  	all	0.2399
iprec_at_recall_0.50  	all	0.1864
iprec_at_recall_0.60  	all	0.1561
iprec_at_recall_0.70  	all	0.1002
iprec_at_recall_0.80  	all	0.0665
iprec_at_recall_0.90  	all	0.0421
iprec_at_recall_1.00  	all	0.0089
P_5                   	all	0.4320
P_10                  	all	0.4040
P_15                  	all	0.3813
P_20                  	all	0.3660
P_30                  	all	0.3407
P_100                 	all	0.2484
P_200                 	all	0.1846
P_500                 	all	0.1125
P_1000                	all	0.0702

So JASSjr is not as fast as JASSv2, and not quite as good at ranking as JASSv2, but that isn't the point. JASSjr is a minimalistic code base demonstrating how to write a search engine from scratch. It performs competatively well.

Manifest

Filename	Purpose
README.md	This file
JASSjr_index.cpp	C/C++ source code to indexer
JASSjr_search.cpp	C/C++ source code to search engine
JASSjr_index.java	Java source code to indexer
JASSjr_search.java	Java source code to search engine
GNUmakefile	GNU make makefile for macOS / Linux
makefile	NMAKE makefile for Windows
test_documents.xml	Example of how documents should be layed out for indexing
51-100.titles.txt	TREC topics 51-100 titles as queries
51-100.qrels.txt	TREC topics 51-100 human judgments

amallia / JASSjr