singhranjodh / pretrain_speech_model

Speech Model Pre-training for End-to-End Spoken Language Understanding

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Speech Model Pre-training for End-to-End Spoken Language Understanding

This repo contains the code for the paper "Speech Model Pre-training for End-to-End Spoken Language Understanding". The paper describes the Fluent Speech Commands corpus (released by Fluent.ai), which is a free speech command dataset for spoken language understanding (click here to download it). The work also explores useful pre-training strategies for end-to-end spoken language understanding.

If you have any questions about this code or have problems getting it to work, please send me an email at <the email address listed for Loren in the paper>.

Dependencies

PyTorch, numpy, soundfile, pandas, tqdm, textgrid.py

Usage

First, change the asr_path and/or slu_path in the config file (like experiments/no_unfreezing.cfg, or whichever experiment you want to run) to point to where the LibriSpeech data and/or Fluent Speech Commands data are stored on your computer.

SLU training: To train the model on Fluent Speech Commands, run the following command:

python main.py --train --config_path=<path to .cfg>

ASR pre-training: Note: the experiment folders in this repo already have a pre-trained LibriSpeech model that you can use. LibriSpeech is pretty big (>100 GB uncompressed), so don't do this part unless you want to re-run the pre-training part with different hyperparameters. If you want to do this, you will first need to download our LibriSpeech alignments here, put them in a folder called "text", and put the LibriSpeech audio in a folder called "audio". To pre-train the model on LibriSpeech, run the following command:

python main.py --pretrain --config_path=<path to .cfg>

Citation

If you find this repo or our Fluent Speech Commands dataset useful, please cite our paper:

  • Loren Lugosch, Mirco Ravanelli, Patrick Ignoto, Vikrant Singh Tomar, and Yoshua Bengio, "Speech Model Pre-training for End-to-End Spoken Language Understanding", arXiv:1904.03670, 2019.

About

Speech Model Pre-training for End-to-End Spoken Language Understanding

License:Apache License 2.0


Languages

Language:Python 100.0%