AppleHolic / multiband_melgan

An unofficial implementation of https://arxiv.org/abs/2005.05106

Multi-Band MelGAN

It's an naive implementation of Multi-band MelGAN: Faster Waveform Generation for High-Quality Text-to-Speech.

Goals

• Comparable Quality with other vocoders
• Mobile Inference Example

TODOs

• Make inference code & example.
• Enhance vocoder quality.

Prerequisite

• install pytorch_sound
  • More detail about installation is on repository.

git clone -b v0.0.4 https://github.com/appleholic/pytorch_sound
cd pytorch_sound
pip install -e .

• Preprocess vctk
  • After run it, you can find 'meta' directory in "OUT DIR"
  • Download Link

python pytorch_sound/scripts/preprocess.py vctk [VCTK DIR] [OUT DIR] [[Sample Rate: default 22.05k]]

• Install multiband melgan

pip install -e .

Environment

• Machine
  • pytorch 1.5.0
  • rtx titan 1 GPU / ryzen 3900x / 64GB
• Dataset
  • VCTK

Train

python multiband_melgan/train_mb.py run [META DIR] [SAVE DIR] [SAVE PREFIX] [[other arguments...]]

Example

import torch
import librosa
from multiband_melgan.inferencer import Inferencer

# make inferencer
inf = Inferencer()

# load sample audio
sample_path = ''
wav, sr = librosa.load(sample_path, sr=22050)
wav_tensor = torch.FloatTensor(wav).unsqueeze(0).cuda()

# convert to mel
mel = inf.encode(wav_tensor)  # (N, Cm, Tm)

# convert back to wav
pred_wav = inf.decode(mel, is_denoise=True)  # (N, Tw)

Reference

• descriptinc/melgan-neurips
• kan-bayashi/ParallelWaveGAN

Others

• Model

  • multiband generator (22.05k) :
    • checkpoint file size : 6.6M
    • numb. parameters : 1710008

• Audio Parameters

• Sample Rate : 22.05k
• Window Length & fft : 1024
• Hop length : 256
• Mel dim : 80
• Mel min/max : 80 / 7600
• Crop Size (in training) : 8192 samples

Author

Ilji Choi @appleholic