The first turn is not in the context
xba0 opened this issue · comments
in function MI_score_turn
lpx = - get_avg_nll(text + sep + m)
response = text.split(sep)[-1]
context = sep.join(text.split(sep)[1:-1]) # hree
the text starts with "<|endoftext|> " (without leading space)
the split function will return ["<|endoftext|> turn0...", "turn1", ...]
so the first turn will not be in the context
Thanks for pointing this out! I have updated the results after fixing this undesired behavior. By the way, may I know your name? So that I can acknowledge you in the paper.