malllabiisc / RESIDE

EMNLP 2018: RESIDE: Improving Distantly-Supervised Neural Relation Extraction using Side Information

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

cannot reproduce the result in the paper

zhoudayang opened this issue · comments

hello, I run the code, and I cannot reproduce the result announced in paper, I use the default parameter as your code, what's wrong with this issue?
2018-11-20 1 58 00

the photo showed that the performance of reside is worse than pcnn

Seems like the model has not converged, you need to train it for more epochs with a lower learning rate. After 3-4 epochs with Adam you can train it further with SGD with learning rate .001 or lower. I think then it should be fine. You can do that easily by executing:
python -restore -name <full_name_of_run> -opt sgd -lr 0.001

thanks, I will try it again asap.

Hi @zhoudayang.
I looked into the problem, there was a slight error in copying hyperparameters. You can pull the code and start afresh training and let it get finished on its own.

python reside.py -name new_train_run -data data/riedel_processed.pkl

After that, you can restore and train the model again with SGD for 2-3 epochs

python reside.py -name new_train_run_xx_xx_xxxx_xx:xx:xx -restore -opt sgd -lr 0.001 -l2 0.0 -epoch 3

Then run python plot_pr.py -name new_train_run_20_11_2018_13:05:18 to get the plot.