Loading SeaLLM-hybrid-7b

[Rina Buoy]

Hello there,

Thanks for sharing your remarkable works of SeaLLMs - Large Language Models for Southeast Asia.

My name is Rina and I'm from Cambodia. I have been playing around with your 7b chat model. I have managed to load it using the Llama 2 code base.

However, when I tried to load the SeaLLM-hybrid-7b, the following error was raised.

[Xuan-Phi Nguyen]

Hi, thanks for your interest. That's because the hybrid files are missing the metadata['format'] = 'pt' attributes. I will update it soon. In the mean time, you have to download the safetensors files and overwrite them with the solution here

import safetensors
from safetensors.torch import save_file

tensors = dict()
with safetensors.safe_open(safetensors_path, framework="pt") as f:
    for key in f.keys():
        tensors[key] = f.get_tensor(key)

save_file(tensors, safetensors_path, metadata={'format': 'pt'})

[Rina Buoy]

Thanks very much ! The proposed fix worked.