Question about the indexing of the hidden state
saunair opened this issue · comments
In the forward pass and the evaluation function, you'll seem to be indexing from the first token and then adding the 255 falses.
https://github.com/dvlab-research/LISA/blob/main/model/LISA.py#L187
What's the logic behind this? Shouldn't the output-hidden states and input-ids have a 1-1 mapping?