nupic.audio
Auditory experiments using cortical learning algorithms (CLA) and hierarchical temporal memory (HTM).
Repositories of interest
- Numenta's nupic.critic Audio streaming
- NuMozart Digital (MIDI) streaming and composition
- HTMforGenreClassification Genre classification
- Hackathon scripts and data for Musenta
Note: These repositories currently are all deemed to be work-in-progress.
Potential areas of investigation
- Genre and style classification
- Musical prediction and composition
- Acoustic correlation using canonical correlation analysis (CCA)
- Transient analysis (harmonic tracking)
- Motion derivative encoding (similar to optical flow)
- Echo location and spatial positioning (e.g. Anterior Ventral Cochlea Nucleus)
- Stream segmentation and seperation (includes selective attention)
- Cortical pathways and projections, 'What' and 'Where' pathways (belts?)
- Auditory nerve spike firing (e.g. IHC to CN GBC integrators)
- Dendritic micro-circuits and synaptic placement (temporal smoothing)
- Spike-timing dependent plasticity
- Acetylcholine inhibition enhancing discharge frequency but decreasing synaptic adaption
- Acoustic related cell, and dendrite, membrane properties (cascading conductances, shunting)
An alternative for the encoding of audio signals is the modelling of spike firing of auditory-nerve fibers. A collection of models can be found in the EarLab @ Boston University (http://earlab.bu.edu/ See Modelling -> Downloadable Models).
Online books and references
-
https://ccrma.stanford.edu/~jos/dft/
Mathematics of the Discrete Fourier Transform (DFT) with audio appliccations
By Julius O. Smith III, Center for Computer Research in Music and Acoustics (CCRMA) -
http://www.dspguide.com/
The Scientist and Engineer's Guide to Digital Signal Processing
By Steven W. Smith, Ph.D. -
http://www.eecs.qmul.ac.uk/~simond/pub/2012/PlumbleyDixon12-ima-tutorial-slides.pdf
Tutorial: Music Signal Processing
By Mark Plumbley and Simon Dixon, Centre for Digital Music (Queen Mary University of London)