- (05/1/2021) Trained Depthwise Seprable Convolution (DSC) based vanilla Contextnet over Librispeech Dataset http://arxiv.org/abs/2005.03191
- (05/6/2021) Implemented Low rank Convolution (LRC) in ContextNet with both spectrogram and raw audio input.
- (05/16/2021) Generated Loss Landscapes for the trained models, see demo_loss
- (05/20/2020) Trained a LRC based deepnet with wave input Low rank decomposition model
- (06/7/2020) Generated Integrated Gradients for trained models Keras Integrated Gradients documentation
- (20/7/2020) Added demo COLAB notebooks and animation for visualization
- What's New?
- Table of Contents
- Publications
- Installation
- Training & Testing Steps
- Visualisation Loss Landscapes and Gradient Maps
- References & Credits
- Contact
-
ContextNet (Reference)
-
Raw Waveform Based CNN Through Low-Rank Spectro-Temporal Decoupling (Reference)
git clone https://github.com/Cross-Caps/STFADE.git
cd STFADE
python setup.py build
python setup.py install
- Define config YAML file, see the
config.yml
files in the contextnent folder for reference (you can copy and modify values such as parameters, paths, etc.. to match your local machine configuration) - Download your corpus (a.k.a datasets) and run
download_links.sh
scripts folder to download files For more detail, see datasets. Note: Make sure your data contain only characters in your language, for example, english hasa
toz
and'
. Do not usecache
if your dataset size is not fit in the RAM. - run
create_transcripts_from_data.sh
from scripts folder to generate .tsv files(the format in which the input is given is .tsv). Librispeech has been used in this work. - For training, see
train.py
files in the contextnet folder to see the options - For testing, see
test.py
files in the contextnet folder to see the options.
-
Loss Landscapes
cd contextnet/contextnet_visualisation/loss_landscape_visualisation python generate_lists.py python plot_loss.py python video_create.py
For Loss Landscape, go to loss video
-
Gradient Maps
cd contextnet/contextnet_visualisation/gradient_visualisation python integrated_grad_vis.py python plot_gradients.py python video_create.py
Gradient Visualisation: Demo Notebook
For Gradient Maps, go to gradients videos
Vaibhav Singh (vaibhav.singh@nyu.edu)
Dr. Vinayak Abrol (abrol@iiitd.ac.in)