Giters

xyjigsaw / CENET

Temporal Knowledge Graph Reasoning with Historical Contrastive Learning

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

the experiment performance

XYHmelons opened this issue · comments

commented

hi! it's a wonderful job in this mission, but i want to ask some question about your code performance. as your instructions, i use the hyper parameter noticed in the md, but i cannot get the same performance as your paper says. the all test hits@ is a bit lower than the paper performance, like 0.8%. could you give me your seed or some example of hyperparameter setting, or maybe it's some performance fluctuation? it really bothers me. thank your project! i like your idea!

commented

Thanks for your concerns. It would be nice of you if you can provide us with the parameter settings and your results of different datasets to achieve better reproducibility.

You can try seed 987 on dataset YAGO

seed = 987
np.random.seed(seed)
torch.manual_seed(seed)

and you can get better performance than the results reported in our paper.

commented

i have tried several dataset. in yago and wiki,the performance is pretty good, but in icews18, it seems that cenet cannot achieve enough performance. my result is 58% in all test samples. i have used 0.2 alpha 2 lambda, and your seed. it confused me for a week. if you have some tips on that, i will appreciate it. i will try my best to do more experiments on that. thanks a lot!