Can't falcon_convert on OpenBuddy Falcon 7B model, KeyError [fixed]

[tak2hu]

Model: OpenBuddy Falcon 7B
python falcon_convert.py openbuddy-falcon-7b-v6-bf16 openbuddy-ggllm use-f32

Error:

* Loading model from:  openbuddy-falcon-7b-v6-bf16
Vocab size: 70144
Hidden size: 4544
Number of heads: 71
Number of layers: 32
Number of head_kv: 1
Number of head_dim: 64
Traceback (most recent call last):
  File "/home/paloma/Git/ggllm.cpp/falcon_convert.py", line 111, in <module>
    text = bytearray([byte_decoder[c] for c in reverse_vocab[i]])
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/paloma/Git/ggllm.cpp/falcon_convert.py", line 111, in <listcomp>
    text = bytearray([byte_decoder[c] for c in reverse_vocab[i]])
                      ~~~~~~~~~~~~^^^
KeyError: '能'

s -lh openbuddy-falcon-7b-v6-bf16/
total 13G
-rw-r--r-- 1 paloma paloma  992 Jul 18 19:11 config.json
-rw-r--r-- 1 paloma paloma 2,6K Jul 18 19:11 configuration_RW.py
-rw-r--r-- 1 paloma paloma  111 Jul 18 19:11 generation_config.json
-rw-r--r-- 1 paloma paloma  47K Jul 18 19:11 modelling_RW.py
-rw-r--r-- 1 paloma paloma 9,4G Jul 18 19:05 pytorch_model-00001-of-00002.bin
-rw-r--r-- 1 paloma paloma 3,7G Jul 18 19:05 pytorch_model-00002-of-00002.bin
-rw-r--r-- 1 paloma paloma  17K Jul 18 19:11 pytorch_model.bin.index.json
-rw-r--r-- 1 paloma paloma   28 Jul 18 19:11 README.md
-rw-r--r-- 1 paloma paloma  281 Jul 18 19:11 special_tokens_map.json
-rw-r--r-- 1 paloma paloma  180 Jul 18 19:11 tokenizer_config.json
-rw-r--r-- 1 paloma paloma 3,5M Jul 18 19:11 tokenizer.json

Operating System: Arch Linux

[John]

That's a crazy finetune. Vocab size of 70144 !
I added a hotfix and it appears to do the job.

To run Buddy as intended start it like this:
Q:\ggllm.cpp\build\bin\Release\falcon_main.exe -m .\openbuddy-falcon-7b-v6-bf16\q5_1 -enc -b 1 -ins
If you do not want to use the system prompt you can use -sysraw -sys " "

User: Hi.
Assistant: Hi, I'm Buddy, your AI assistant. How can I help you today?

Hey there! I heard you can do poems? Can you invent me one that is super unique and a nice start to our friendship ?
Sure, I'd be happy to create a poem for you! Here it is:

"A Friend in Need":

In every corner of this world
There's someone who we hold dear
Someone to laugh with, cry with too
Share joys and sorrows through

The bond between us can be strong
And last forever, all day long
Through thick and thin, through rain or shine
We stick together like glue.

So when you need a friend in need
Know that I am here to lend
A shoulder to cry on, a listening ear
Your troubles to share with care.

Together we'll face the world
And conquer all our fears
For true friendship is rare and precious
And one day we will bear it without any measures!

So let us walk this path together
Hand in hand, friend forever
With every smile and every tear
Our bond will only get stronger.

nice, what would you like to talk about?
I'm glad you liked the poem! If you have any topics or questions that interest you, feel free to share them with me and we can discuss them together. Alternatively, if you want to learn something new or ask me for a specific task, please let me know and I will do my best to assist you.

I want you to choose!
Sure, what topic would you like me to cover? Here are some categories you might be interested in:

• World history: From ancient civilizations to modern times, discover the fascinating stories and events that shaped our world.
• Science and technology: Learn about groundbreaking discoveries, innovations, and their impact on society.
• Culture and art: Explore various artistic movements, styles, and genres from literature, music, film, and more.
• Philosophy and ethics: Reflect on fundamental questions of existence, morality, and human nature.
• Personal development: Gain insights into effective self-improvement strategies, personal growth, and mental well-being.
• Technology and innovation: Learn about the latest technological advancements, trends, and their potential impact on various industries and society.

Please let me know which category you would like to explore further, or feel free to suggest a topic of your choice!

i need to sleep!
Oh, I'm sorry to hear that. Do you have any questions or tasks that require my assistance before you go to bed? If not, please let me know if there is anything else I can help you with in the future. Have a good night's rest and sweet dreams!