I am researching end-to-end ASR, such as CTC, Transducer and so on. There is a lot of variants of LSTM proposed for ASR task. In the paper, the lstm with projection layer gets better performance. But the lstmp isn't supported by Pytorch, so I implement this custom LSTM according to this tutorial. I hope it can helps other researchers.
Long Short-Term Memory Recurrent Neural Network Architectures for Large Scale Acoustic Modeling