Llava-v1.5 llamafile seems to be leaking outputs?
jjshoots opened this issue · comments
Jet commented
Using the following command:
llava-v1.5-7b-q4-main.llamafile --image test_images/apanese_style_buildings.jpg --grammar 'root ::= [a-z]+ (" " [a-z]+)+' -n 10 -p '### User: The image has... ### Assistant:' --silent-prompt --no-display-prompt 2>/dev/null
I get the following output:
1 -> ''
835 -> ' ###'
4911 -> ' User'
29901 -> ':'
450 -> ' The'
1967 -> ' image'
756 -> ' has'
856 -> '...'
835 -> ' ###'
4007 -> ' Ass'
22137 -> 'istant'
29901 -> ':'
13 -> '
'
22933 -> 'ASS'
9047 -> 'IST'
13566 -> 'ANT'
29901 -> ':'
eunaciousness towards a building
I don't think the first many lines of text are supposed to be in there?
Justine Tunney commented
Sorry about that! Something got lost in translation with the last sync. I'll upstream my modifications soon for better synchronicity.
Justine Tunney commented
Fixed in llamafile 0.7.2 release. Weights uploaded https://huggingface.co/jartine/llava-v1.5-7B-GGUF