Cpm bee 预训练出现 Failed to load config
YingLaiLin opened this issue · comments
单卡运行或者使用 pretrain_com_bee.sh 脚本运行，都会出现如下错误
Traceback (most recent call last):
File "/ms_test2/miniconda3/envs/py375/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
self.run()
File "/ms_test2/miniconda3/envs/py375/lib/python3.7/multiprocessing/process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "/ms_test1/lin/CPM-Live/cpm-live/cpm_live/training_tasks/bee/pretrain.py", line 776, in _mixed_dataset_process
config = cfg_mgr.get_config()
File "/ms_test1/lin/CPM-Live/cpm-live/cpm_live/training_tasks/bee/pretrain.py", line 736, in get_config
raise RuntimeError("Failed to load config")
RuntimeError: Failed to load config
使用这个推理数据制作的数据集 https://github.com/OpenBMB/CPM-Bee/blob/main/tutorials/basic_task_finetune/bee_data/train.jsonl， 出现如下报错：
ile "/home/miniconda3/envs/ci/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
self.run()
File "/home/miniconda3/envs/ci/lib/python3.7/multiprocessing/process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "/ms_test1/lyl/CPM-Live/cpm-live/cpm_live/training_tasks/bee/pretrain.py", line 930, in _mixed_dataset_process
batch = packer.add_data(config[ds_id])
File "/ms_test1/lyl/CPM-Live/cpm-live/cpm_live/training_tasks/bee/pretrain.py", line 636, in add_data
) = self.build_instance(config)
File "/ms_test1/lyl/CPM-Live/cpm-live/cpm_live/training_tasks/bee/pretrain.py", line 437, in build_instance
inp = ds.read()
File "/ms_test1/lyl/CPM-Live/cpm-live/cpm_live/dataset/distributed_dataset.py", line 555, in read
next_block_id = self._get_next_block()
File "/ms_test1/lyl/CPM-Live/cpm-live/cpm_live/dataset/distributed_dataset.py", line 395, in _get_next_block
raise RuntimeError("Empty dataset {}".format(self._path))
RuntimeError: Empty dataset /ms_test1/lyl/CPM-Live/wikitext-2-cpm/train/
请使用finetune_cpm_bee.sh。
请使用
finetune_cpm_bee.sh。
这里的意思是 finetune_cpm_bee.sh 只要不启用“--use-delta”就相当于pretrain_cpm_bee.sh全量参数预训练？
是的，不使用--use-delta即为全参数训练。
好的，非常感谢
是的，不使用
--use-delta即为全参数训练。
想问下，增量预训练的话，怎么处理成自监督掩码的形式呢，这部分预处理的代码有没有呀