To generate a single-dataset-style mturk evaluation package for NELL geolocations,
1) Place a copy of Locator-out.tsv in this folder (find it in latlongWorkingDir)
Locator-out.tsv has the following fields:
1 category:concept
2 maxVarianceInMiles
3 latVariance
4 lonVariance
5 lat
6 lon
7 count
8 list of matched names
2) Run make.sh with the following arguments:
1 dataset - give a name to your dataset, to distinguish these files from others
2 Locator-out.tsv - as above
3 minCategoryCount - only consider concepts from categories in Locator-out which have at least this many concepts listed
4 maxVarInMiles - only consider geolocations in Locator-out which have this maxVar or less
5 sampleSize - select this many random concepts from each category in Locator-out (should be equal or less than minCategoryCount)
3) Add a symlink to www/ from somewhere publicly browsable on the web. Make sure you have PHP turned on.
4) Generate gold data:
a. Move www/save.safe.php to www/save.php
b. Visit www/gold.html and do a few concepts in each category
c. Move www/save.php to www/save.safe.php
5) Upload hitinput.txt to Mechanical Turk
6) Double check that the google links work
7) Run Mechanical Turk
8) Follow the directions in gal/README