How to train your model to better fit SentEval

Question

How to train your model to better fit SentEval

ZBWpro opened this issue a year ago · comments

Hi~

SentEval requires users to implement a function called "batcher(params, batch)".

When it comes to STS tasks, the parameter "batch" of the batcher function only contains one sentence of a certain sentence pair.

This may cause conflicts, as your model typically requires the entire sentence pair as input.

If you convert sentences of a pair to embeddings one by one, it means you will need to call forward twice. This behavior is poorly supported by DDP.

github-actions · Answer 1 · Wed May 24 2023 17:20:48 GMT+0800 (China Standard Time)

Stale issue message

Tianyu Gao · Answer 2 · Fri May 26 2023 08:13:31 GMT+0800 (China Standard Time)

Hi,

Sorry about the late replyl. I'm not sure I totally follow. SentEval is only used for eval, thus it shouldn't affect DDP.

Victor · Answer 3 · Fri May 26 2023 10:31:19 GMT+0800 (China Standard Time)

Thanks for your reply, I found that I could solve this problem by introducing an extra encoding function into the class and calling it several times in forward.