Plans for Encodec fine-tuning

Question

Plans for Encodec fine-tuning

0xprincess opened this issue a year ago · comments

Hi! Do you have any plans for writing encodec fine-tuning code? The current checkpoint seems to perform really poorly on most sorts of electronic music so it's definitely a barrier for achieving high-quality generations.

rkfg · Answer 1 · Sat Jun 17 2023 03:02:49 GMT+0800 (China Standard Time)

Training LoRAs would be great! That way you can expand the Encodec's capabilities to infinity by automatically loading the appropriate LoRA for the genre.

chavez · Answer 2 · Sat Jun 17 2023 03:47:44 GMT+0800 (China Standard Time)

Hi! Do you have any plans for writing encodec fine-tuning code? The current checkpoint seems to perform really poorly on most sorts of electronic music so it's definitely a barrier for achieving high-quality generations.
@0xprincess

Yes

chavez · Answer 3 · Sat Jun 17 2023 03:48:34 GMT+0800 (China Standard Time)

Training LoRAs would be great! That way you can expand the Encodec's capabilities to infinity by automatically loading the appropriate LoRA for the genre.

will try to add LoRAs (LM, not encodec yet) for the next release

0xprincess · Answer 4 · Sat Jun 17 2023 04:59:28 GMT+0800 (China Standard Time)

@chavinlo btw, I noticed that the current release of musicgen uses mono version of encodec, but the original encodec release has 48khz stereo model. If we're talking about fine-tuning, do you think that swapping the current 32khz musicgen's encodec for 48khz model is possible? I personally think that having stereo audio would be great.