princeton-nlp / SimCSE

[EMNLP 2021] SimCSE: Simple Contrastive Learning of Sentence Embeddings https://arxiv.org/abs/2104.08821

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

small result gap between reproduction and huggingface model

kimwongyuda opened this issue · comments

Hi, first of all, thank you for your great work.

I trained SimCSE myself, evaluated the model, and got 75.99 avg score of STS tasks.
It is slightly lower compared to 76.25 in the paper and from huggingface model.

I think it is because of seed.
However, I used default seed 42.

Is it possible that different results occur despite of same seed? And is it possible that results based on same seed can be different between various hardwares?
Or could you tell me your opinion about this situation?

Thank you!!

Stale issue message