Giters

vivo-ai-lab / BlueLM

BlueLM(蓝心大模型): Open large language models developed by vivo AI Lab

Home Page:https://developers.vivo.com/product/ai/bluelm

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

GGUF version

eramax opened this issue · comments

commented

I'm trying to create gguf version of this model using llama.cpp but I got an error

!python llama.cpp/convert.py /content/BlueLM-7B-Chat-32K \
  --outfile  /content/BlueLM-7B-Chat-32K.gguf \
  --outtype q8_0

Loading model file /content/BlueLM-7B-Chat-32K/pytorch_model-00001-of-00008.bin
Loading model file /content/BlueLM-7B-Chat-32K/pytorch_model-00001-of-00008.bin
Loading model file /content/BlueLM-7B-Chat-32K/pytorch_model-00002-of-00008.bin
Loading model file /content/BlueLM-7B-Chat-32K/pytorch_model-00003-of-00008.bin
Loading model file /content/BlueLM-7B-Chat-32K/pytorch_model-00004-of-00008.bin
Loading model file /content/BlueLM-7B-Chat-32K/pytorch_model-00005-of-00008.bin
Loading model file /content/BlueLM-7B-Chat-32K/pytorch_model-00006-of-00008.bin
Loading model file /content/BlueLM-7B-Chat-32K/pytorch_model-00007-of-00008.bin
Loading model file /content/BlueLM-7B-Chat-32K/pytorch_model-00008-of-00008.bin
Traceback (most recent call last):
  File "/content/llama.cpp/convert.py", line 1228, in <module>
    main()
  File "/content/llama.cpp/convert.py", line 1172, in main
    params = Params.load(model_plus)
  File "/content/llama.cpp/convert.py", line 287, in load
    params = Params.loadHFTransformerJson(model_plus.model, hf_config_path)
  File "/content/llama.cpp/convert.py", line 226, in loadHFTransformerJson
    raise NotImplementedError(f'Unknown rope scaling type: {typ}')
NotImplementedError: Unknown rope scaling type: ntkmixed