seastar105 / pflow-encodec

Thank you for sharing this project!
The inference sounds great on the multilingual_base_bs100x4.ckpt
I am able to start training from a new dataset but I'm wondering if there is a way to fine-tune the pretrained model that has been released.

I did try to convert the model with this script
but i get the following error

  File "venv/lib/python3.10/site-packages/lightning/pytorch/trainer/connectors/checkpoint_connector.py", line 364, in restore_optimizers_and_schedulers
    raise KeyError(
KeyError: 'Trying to restore optimizer state but checkpoint contains only the model. This is probably due to `ModelCheckpoint.save_weights_only` being set to `True`.'

released checkpoint is weight-only, so you can initialize model weight by using this field

pflow-encodec/configs/experiment/multilingual_base.yaml

Line 30 in ddf74ce

    
           net_ckpt_path: /home/seastar105/Work/pflow-encodec/checkpoints/multilingual_base.ckpt

set this field to released checkpoint. it would work. also check mean, std carefully.

thanks, that worked!

how to finetune from pretrained model