nl2sql
Learning a Neural Semantic Parser with User Feedback
Setup
Install torch libraries
luarocks install hdf5
Install MySQL libraries
sudo apt-get install libmysqlclient-dev
sudo apt-get install python-mysqldb
Install required python packages
pip install python-gflags MySQL-python futures pattern commentjson numpy h5py
Download word2vec embeddings and replace word2vec_sample.txt. Its is a tab separated list of the words followed by 300 dim embeddings
download from https://drive.google.com/file/d/0B7XkCwpI5KDYNlNUTTlSS21pQmM/edit?usp=sharing
convert the bin file to tsv: see https://gist.github.com/dav009/10a742de43246210f3ba
Note: This script outputs a comma separated list. Change that to tabs.
Unzip ppdb file
cd data/ppdb
gunzip ppdb-l-combined.gz
Setup databases for computing results
Databases are provided as a mysql dump file
mysql -u root -ptmppassword geo < data/geo/geo_mysql_dump.db
mysql -u root -ptmppassword atis < data/atis/atis_mysql_dump.db
Export DB environment variables
export DBHOST="my.database.host"
export DBUSER="me"
export DBPASSWD="secret"
GeoQuery
Train geo
cd src
./train_geo.sh
Predictions for each epoch on the development set are now in run_geo/preds/
Results on the dev set are summarized in run_geo.summary
Test geo. Pick a trained model based on the run_geo.summary file.
$ ./train_geo.sh test run_geo_1001/s2s-model_epoch50.00_1.42.t7
loading run_geo_1001/s2s-model_epoch50.00_1.42.t7...
280.0
Correct=231 , Total=280.0 , Percent=82.5
ATIS
Train ATIS
cd src
./train_atis.sh
Predictions for each epoch on the development set are now in run_atis/preds/
Results on the dev set are summarized in run_atis.summary
Test atis. Pick a trained model based on the run_atis.summary file.
$ ./train_atis.sh test run_atis/s2s-model_epoch60.00_1.13.t7
loading run_atis/s2s-model_epoch60.00_1.13.t7...
448.0
Correct=355 , Total=448.0 , Percent=79.2410714286
SCHOLAR
Download the dataset from:
https://drive.google.com/file/d/0Bw5kFkY8RRXYRXdYYlhfdXRlTVk/view?usp=sharing
Load it
mysql -u root -ptmppassword scholar < data/scholar/s2_mysql_dump.db
Train SCHOLAR
cd src
./train_scholar.sh
Predictions for each epoch on the development set are now in run_scholar/preds/
Results on the dev set are summarized in run_scholar.summary
Test scholar. Pick a trained model based on the run_scholar.summary file.
$ ./train_scholar.sh test run_scholar/s2s-model_epoch50.00_1.60.t7
loading run_scholar/s2s-model_epoch50.00_1.60.t7...
218.0
Correct=145 , Total=218.0 , Percent=66.5137614679
Run Simulated Interactive Experiments
python geo_interactive_batch.py --ppscale 3 --init_grammar_file ../data/grammar_empty.sql --data_dir ../data/geo/tmp_no_tem --models_dir ../data/geo/tmp_no_tem/models --db_host my.db.host --db_user me --db_pass my.secret.password
python atis_interactive_batch.py --ppscale 3 --init_grammar_file ../data/grammar_empty_atis.sql --data_dir ../data/atis/tmp_no_tem --models_dir ../data/atis/tmp_no_tem/models --host my.db.host --user me --passwd my.secret.password
Useful SQL commands
Make MySQL ignore spaces after function names
SET sql_mode='IGNORE_SPACE';
Check number of threads connection to the database server
SHOW STATUS WHERE `variable_name` = 'Threads_connected';