convert.py cannot find params.json

[Peter Greis]

Attempting to convert ChatMusician, which is based on llama 2 7b to mlx format.

python convert.py --torch-path  /Users/petergreis/Dropbox/Leeds/Project/chatmusician_model_tokenizer

[INFO] Loading
Traceback (most recent call last):
  File "/Users/petergreis/Library/CloudStorage/Dropbox/Leeds/Project/other_platforms/mlx/mlx-examples/llms/llama/convert.py", line 219, in <module>
    weights, params = globals()[args.model_name](torch_path, dtype=args.dtype)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/petergreis/Library/CloudStorage/Dropbox/Leeds/Project/other_platforms/mlx/mlx-examples/llms/llama/convert.py", line 61, in llama
    with open(model_path / "params.json", "r") as f:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/Users/petergreis/Dropbox/Leeds/Project/chatmusician_model_tokenizer/params.json'

And indeed there is no such file in my model. Can this be created or bypassed? How can I solve this?

(testml) petergreis@MacBook-Pro-M1-Max-2021 llama % ls -al /Users/petergreis/Dropbox/Leeds/Project/chatmusician_model_tokenizer
total 78970816
drwxr-xr-x@ 17 petergreis  staff         544 Apr 22 16:40 .
drwxr-xr-x@ 20 petergreis  staff         640 Apr 22 18:14 ..
-rw-r--r--@  1 petergreis  staff          21 Apr  2 14:27 added_tokens.json
-rw-r--r--@  1 petergreis  staff         624 Apr  2 14:27 config.json
-rw-r--r--@  1 petergreis  staff         132 Apr  2 14:27 generation_config.json
-rw-r--r--@  1 petergreis  staff  4938985248 Mar 30 22:13 model-00001-of-00003.safetensors
-rw-r--r--@  1 petergreis  staff  4947390768 Mar 30 22:13 model-00002-of-00003.safetensors
-rw-r--r--@  1 petergreis  staff  3590488736 Mar 30 22:13 model-00003-of-00003.safetensors
-rw-r--r--@  1 petergreis  staff       23950 Mar 30 22:13 model.safetensors.index.json
-rw-r--r--@  1 petergreis  staff  9877982386 Apr  2 14:27 pytorch_model-00001-of-00003.bin
-rw-r--r--@  1 petergreis  staff  9894793766 Apr  2 14:27 pytorch_model-00002-of-00003.bin
-rw-r--r--@  1 petergreis  staff  7180985861 Apr  2 14:27 pytorch_model-00003-of-00003.bin
-rw-r--r--@  1 petergreis  staff       23950 Apr  2 14:27 pytorch_model.bin.index.json
-rw-r--r--@  1 petergreis  staff         435 Apr  2 14:27 special_tokens_map.json
-rw-r--r--@  1 petergreis  staff     1842945 Apr  2 14:27 tokenizer.json
-rw-r--r--@  1 petergreis  staff      499723 Apr  2 14:27 tokenizer.model
-rw-r--r--@  1 petergreis  staff         773 Apr  2 14:27 tokenizer_config.json

[Awni Hannun]

Can you use MLX LM instead? It should work with the model directory you are showing. Also you might not need to convert it unless you want to quantize it. If the arch is the same as Llama 2 7B then it should load out of the box with e.g. mlx_lm.generate.

Install MLX LM:

pip install -U mlx-lm

Conversion command:

mlx_lm.convert --hf-path /Users/petergreis/Dropbox/Leeds/Project/chatmusician_model_tokenizer -q

Docs: https://github.com/ml-explore/mlx-examples/blob/main/llms/README.md