yangjianxin1 / Firefly

Firefly: 大模型训练工具,支持训练Qwen2、Yi1.5、Phi-3、Llama3、Gemma、MiniCPM、Yi、Deepseek、Orion、Xverse、Mixtral-8x7B、Zephyr、Mistral、Baichuan2、Llma2、Llama、Qwen、Baichuan、ChatGLM2、InternLM、Ziya2、Vicuna、Bloom等大模型

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

微调gemma,报错KeyError: 'target_mask'

CyrusLiu158 opened this issue · comments

无论是使用官方的jsonl,还是我自己的。都是报错:
0%| | 0/27 [00:00<?, ?it/s]Traceback (most recent call last):
File "/home/seatune/桌面/workspace/Firefly/train.py", line 400, in
main()
File "/home/seatune/桌面/workspace/Firefly/train.py", line 388, in main
train_result = trainer.train()
File "/home/seatune/anaconda3/lib/python3.10/site-packages/transformers/trainer.py", line 1624, in train
return inner_training_loop(
File "/home/seatune/anaconda3/lib/python3.10/site-packages/transformers/trainer.py", line 1928, in _inner_training_loop
for step, inputs in enumerate(epoch_iterator):
File "/home/seatune/anaconda3/lib/python3.10/site-packages/accelerate/data_loader.py", line 452, in iter
current_batch = next(dataloader_iter)
File "/home/seatune/anaconda3/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 634, in next
data = self._next_data()
File "/home/seatune/anaconda3/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 678, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "/home/seatune/anaconda3/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 54, in fetch
return self.collate_fn(data)
File "/home/seatune/anaconda3/lib/python3.10/site-packages/transformers/trainer_utils.py", line 773, in call
return self.data_collator(features)
File "/home/seatune/桌面/workspace/Firefly/component/collator.py", line 24, in call
target_mask = x['target_mask']
KeyError: 'target_mask'
0%| | 0/27 [00:00<?, ?it/s]
Training completed.

我的配置文件:
{
"output_dir": "./training_outputs",
"model_name_or_path": "google/gemma-1.1-7b-it",
"train_file": "./jsonl/data_inspiration_0408.jsonl",
"template_name":"gemma",
"num_train_epochs": 3,
"per_device_train_batch_size": 3,
"save_steps": 10000,
"logging_dir": "./logs",
"learning_rate": 5e-6,
"max_seq_length": 1024,
"logging_steps": 100,
"save_total_limit": 2,
"seed": 42,
"lora_rank":32,
"lora_alpha":16,
"lora_dropout":0.1,
"fp16": true,
"bf16": false
}