- Lecture and seminar materials for each week are in ./week* folders, see README.md for materials and instructions
- Any technical issues, ideas, bugs in course materials, contribution ideas - add an issue
- The current version of the course is conducted in autumn 2022 at the CS Faculty of HSE
-
week01 Introduction to Course
- Lecture: Introduction to Course
- Seminar: Intro in
pytorch
-
week02 Introduction to Digital Signal Processing
- Lecture: Signals, Fourier Transform, spectrograms, MelScale, MFCC
- Seminar: DSP in practice, spectrogram creation, training a model for audio MNIST
-
week03 Speech Recognition I
- Lecture: Metrics, datasets, Connectionist Temporal Classification (CTC), Listen Attend and Spell (LAS), Beam Search
- Seminar: Audio Augmentations, Beam Search
-
week04 Speech Recognition II
- Lecture: RNN-T, language model fusion, Byte-Pair Encoding (BPE)
- Seminar: Forced Alignment
-
week05 Speaker verification and identification
- Lecture: Metric Learning: Cosine, Contrastive, Triplet Losses. Angular Softmax. ArcFace
- Seminar: Generalized End2End Loss for Speaker Verification
-
week06 Key-word spottind
- Lecture: (DNN, CNN, RNN+Attention) based KWS, SVDF, Orthogonality Regularization and other Tricks
- Seminar: CNN+Attention+RNN KWS model
-
week07 Text to Speech (TTS)
- Lecture: Tacotron, DeepVoice, GST, FastSpeech, AdaSpeech, Attention Tricks
- Seminar: FastSpeech I
-
week08 Neural Vocoders
- Lecture: WaveNet, Parallel WaveGAN
- Seminar: WaveNet
-
week09 Advanced TTS and Vocoders
- Lecture: Introduction into generative models. ParallelWaveNet, WaveGlow, WaveFlow, MelGAN, HiFiGAN
-
week10 Voice Conversion
- Lecture: Disentanglement & Direct based methods,
- Seminar: Homework Q&A
-
week11 Self-supervision in Audio and Speech
-
week12 Invited talks
- ASR Training speech recognition model
- KWS Implementation of KWS model
- TTS Implementation of TTS model
- NV Implementation of Neural Vocoder Model
- Lecture recordings on YouTube (in russian): YouTube
Course materials and teaching performed by