Deep Neural Network for Real-Time EEG Decoding of Musical Rhythm Imagery: Towards a Brain-Computer Interface Application
深度神经网络实时解码脑电图的音乐节奏图像:面向脑-机接口应用
Iran R Roman (Music PhD), Jay McClelland (Psychology), Chris Chafe (Music)
Iran R Roman(音乐博士)，Jay McClelland(心理学)，Chris Chafe(音乐)
Data-driven sound creation is a central feature of computer music. An EEG-based drum machine will explore the potential for musical rhythm imagery classification to operate a musical device. Interestingly, neural activities during perception and imagery of temporal structures of music also overlap closely with those during motor imagery. Studies have explored the motor imagery task for Brain-Computer Interface (BCI) applications using noninvasive EEG. Convolutional Neural Networks (CNNs) can extract features from EEG during motor imagery to control a BCI apparatus in real time. Recently, research has shown that EEG is modulated during music imagery reflecting the speed of musical tempo. Furthermore, CNN-based classification is also successful for EEG during musical rhythm imagery. Thus, our idea is to create an EEG-BCI-based drum machine linked to musical rhythm imagery. We will train CNNs with open-source existing data during musical rhythm imagery tasks to recognize musical features and then incorporate the results into a prototype BCI system. This opens the door to the future BCIs with imagery for other musical features such as pitch, timbre, and harmony.
数据驱动的声音创作是计算机音乐的一个核心特征。 一个基于脑电图的机器将探索音乐节奏图像分类的潜力来操作音乐设备。 有趣的是，在感知和想象音乐的时间结构时的神经活动也与运动想象时的神经活动紧密重叠。 研究已经探索了脑机接口(BCI)的运动图像任务使用无创脑电图。 卷积神经网络(CNNs)可以在运动成像过程中从脑电图中提取特征，对脑机接口设备进行实时控制。 最近，研究表明，脑电图在音乐图像中被调节，反映音乐节奏的速度。 此外，基于cnn的脑电分类在音乐节奏图像中也取得了成功。 因此，我们的想法是创建一个基于eeg - bci的鼓机连接到音乐节奏图像。 在音乐节奏图像任务中，我们将使用开源的现有数据对cnn进行训练，以识别音乐特征，然后将结果整合到原型机接口系统中。 这就为BCIs的未来打开了一扇门，BCIs将为其他音乐特性(如音高、音色和和声)提供图像。