Model Evaluation

[Keyvan Sasani]

Thanks for sharing this interesting framework. I am trying to run the codes to understand it better. In the (wikisql) evaluation function, based on what I understood, you call ParseResult.eval(pred, gold, sql_gold) to calculate the accuracy of the model. Inside the eval function, when you want to execute the pred_query, you change the conditions with the gold ones using recover_cond_to_gloss() function. Why do you do that? Is the resulted query still representing the predicted_query?

[Li Dong]

The predicted text spans in the conditions are converted to strings in order to execute the sql query over the table. It doesn't change the predictions.

In the prediction, the tuple "span" in https://github.com/donglixp/coarse2fine/blob/master/wikisql/table/ParseResult.py#L70 represents the predicted indexes. We append the question words to the list "tk_list" and concatenate them together. For example, the input question (gold['question']) is "w_0 w_1 ... w_5". If the predicted tuple "span" is "(2, 3)", the function recover_cond_to_gloss() recovers the condition value as "w_2 w_3".