Training on TinyDolphin model throws errors in MLX training script
dadmobile opened this issue · comments
Model: mlx-community/TinyDolphin-2.8-1.1b-4bit-mlx
Dataset: samsum
Plugin: mlx-lora-trainer
`Loading pretrained model
Fetching 7 files: 0%| | 0/7 [00:00<?, ?it/s]
Fetching 7 files: 100%|██████████| 7/7 [00:00<00:00, 75475.91it/s]
Traceback (most recent call last):
File "/Users/timk/.transformerlab/workspace/plugins/mlx_lora_trainer/mlx-examples/lora/lora.py", line 321, in
model, tokenizer, _ = lora_utils.load(args.model)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/timk/.transformerlab/workspace/plugins/mlx_lora_trainer/mlx-examples/lora/utils.py", line 171, in load
model.load_weights(list(weights.items()))
File "/Users/timk/miniconda3/envs/transformerlab/lib/python3.11/site-packages/mlx/nn/layers/base.py", line 167, in load_weights
raise ValueError(f"Missing parameters: {missing}.")
ValueError: Missing parameters: lm_head.biases lm_head.scales.
Finished training.
AI made approachable — Today at 4:07 AM
I don't think the error with running models created by mlx_lora_trainer is fixed. Still got this today:
Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/Users/timk/.transformerlab/workspace/models/TinyLlama-1.1B-Chat-v1.0_test/TinyLlama-1.1B-Chat-v1.0_test`
Possibly need to file bug with MLX.