audtorch
Deep learning with PyTorch and audio.
Have a look at the installation and usage instructions as a starting point.
If you are interested in PyTorch and audio you should also check out the efforts to integrate more audio directly into PyTorch: