ValueError: The input provided to the model are wrong. The number of image tokens is 0 while the number of image given to the model is 1. This prevents correct indexing and breaks batch generation.

[zhangchunjie1999]

This problem occurred when I was running a demo of the 34b model.
How can I fix this ？

[zhangchunjie1999]

[Kaiyue Sun]

same problem

[zhangchunjie1999]

I found a solution.
In tasks/eval/model_util.py,
remove
# try: # processor = PllavaProcessor.from_pretrained(repo_id) # except Exception as e: # processor = PllavaProcessor.from_pretrained('llava-hf/llava-1.5-7b-hf')
and add:
processor = PllavaProcessor.from_pretrained(repo_id)
then:
Install the required packages

[PavanMV]

Its image_token_index problem, looks like new LlamaTokenizer image_token_index = 64000 where as pllava config image_token_index = 64002, change it to 64000.