This repository is the official PyTorch implementation of our AAAI-2022 paper, in which we propose DiffSinger (for Singing-Voice-Synthesis) and DiffSpeech (for Text-to-Speech).
| DiffSinger/DiffSpeech at training | DiffSinger/DiffSpeech at inference |
|---|---|
 |
 |
🎉 🎉 🎉 Updates:
- Jul.27, 2022: Update documents for SVS. Add easy inference; Add Interactive SVS running on HuggingFace🤗 SVS.
- Mar.2, 2022: MIDI-B-version.
- Mar.1, 2022: NeuralSVB, for singing voice beautifying, has been released.
- Feb.13, 2022: NATSpeech, the improved code framework, which contains the implementations of DiffSpeech and our NeurIPS-2021 work PortaSpeech has been released.
- Jan.29, 2022: support MIDI-A-version SVS.
- Jan.13, 2022: support SVS, release PopCS dataset.
- Dec.19, 2021: support TTS. HuggingFace🤗 TTS
🚀 News:
- Feb.24, 2022: Our new work, NeuralSVB was accepted by ACL-2022
. Demo Page.
- Dec.01, 2021: DiffSinger was accepted by AAAI-2022.
- Sep.29, 2021: Our recent work
PortaSpeech: Portable and High-Quality Generative Text-to-Speechwas accepted by NeurIPS-2021 - May.06, 2021: We submitted DiffSinger to Arxiv
conda create -n your_env_name python=3.8
source activate your_env_name
pip install -r requirements_2080.txt (GPU 2080Ti, CUDA 10.2)
or pip install -r requirements_3090.txt (GPU 3090, CUDA 11.4)tensorboard --logdir_spec exp_name |
Old audio samples can be found in our demo page. Audio samples generated by this repository are listed here:
Speech samples (test set of LJSpeech) can be found in resources/demos_1213.
Singing samples (test set of PopCS) can be found in resources/demos_0112.
@article{liu2021diffsinger,
title={Diffsinger: Singing voice synthesis via shallow diffusion mechanism},
author={Liu, Jinglin and Li, Chengxi and Ren, Yi and Chen, Feiyang and Liu, Peng and Zhao, Zhou},
journal={arXiv preprint arXiv:2105.02446},
volume={2},
year={2021}}
Our codes are based on the following repos:
Also thanks Keon Lee for fast implementation of our work.