BiDAF (bi-att-flow)

TensorFlow implementation of Bidirectional Attention Flow for Machine Comprehension with SQuAD Task.

Requirements

Python 3.6
TensorFlow 1.4
hb-config (Singleton Config)
nltk (tokenizer)
tqdm (progress bar)

Project Structure

init Project by hb-base

.
├── config                  # Config files (.yml, .json) using with hb-config
├── data                    # dataset path
├── notebooks               # Prototyping with numpy or tf.interactivesession
├── BiDAF                   # bi-att-flow architecture graphs (from input to logits)
    ├── __init__.py             # Graph logic
    ├── attention.py            # Attention (Query2Context, Context2Query)
    ├── embedding.py            # Word Embedding, Character Embedding
    └── layer.py                # RNN and other modules
├── data_loader.py          # raw_date -> precossed_data -> generate_batch (using Dataset)
├── hook.py                 # training or test hook feature (eg. print_variables)
├── main.py                 # define experiment_fn
└── model.py                # define EstimatorSpec

Reference : hb-config, Dataset, experiments_fn, EstimatorSpec

Todo

Implements BiDAF with SQuAD task.

Config

Can control all Experimental environment.

example: check-tiny.yml

data:
  base_path: 'data/'

model:
  batch_size: 4

train:
  learning_rate: 0.0001
  optimizer: 'Adam'  ('Adagrad', 'Adam', 'Ftrl', 'Momentum', 'RMSProp', 'SGD')
  
  train_steps: 15000
  model_dir: 'logs/check_tiny'
  
  save_checkpoints_steps: 1000
  check_hook_n_iter: 100
  min_eval_frequency: 100
  
  print_verbose: True
  debug: False

debug mode : using tfdbg

Usage

Install requirements.

pip install -r requirements.txt

Then, pre-process raw data.

python data_loader.py --config check-tiny

Finally, start train and evaluate model

python main.py --config check-tiny --mode train_and_evaluate

Experiments modes

✅ : Working
◽ : Not tested yet.

◽ evaluate : Evaluate on the evaluation data.
◽ extend_train_hooks : Extends the hooks for training.
◽ reset_export_strategies : Resets the export strategies with the new_export_strategies.
◽ run_std_server : Starts a TensorFlow server and joins the serving thread.
◽ test : Tests training, evaluating and exporting the estimator for a single step.
◽ train : Fit the estimator using the training data.
◽ train_and_evaluate : Interleaves training and evaluation.

Tensorboar

tensorboard --logdir logs

Reference

hb-research/notes - Bidirectional Attention Flow for Machine Comprehension
Paper - Bidirectional Attention Flow for Machine Comprehension (2016. 11) by Minjoon Seo

Author

Dongjun Lee (humanbrain.djlee@gmail.com)

DongjunLee / bi-att-flow-tensorflow