Epileptic seizure detection in raw EEG signals using Vision Nystromformer
Introduction
Epilepsy is a chronic brain disease characterized by persistent susceptibility to cause recurrent seizures. Electroencephalography (EEG) is a neuroimaging technique measuring the electrophysiological activity of the cerebral cortex. EEG has been commonly used to diagnose and treat patients with epilepsy.
Main Contributions of this study
Using Transformer model directly with raw EEG signals without any removal of noise and artifacts.
Using Transformer with very few parameters and as small size as possible.
Optimizing Square time and space complexity into Linear Time and Space complexity of Attention mechanism in Transformer using Nystrom Attention mechanism.
Why Transformer instead of CNN and RNN?
First, due to high temporal resolution, EEG signals are usually extremely long sequences. The sequence models, e.g., RNNs and LSTMs, process the EEG signals sequentially, namely, they train the data at each time step one by one, which largely increases the training time for convergence. In addition, although some deep learning frameworks can capture temporal dependencies, such as RNN-based models for long-term dependencies and CNN-based models for neighboring interactions, they can only achieve limited performance when the sequences are extremely long.
Methodology
Model
Datasets
Three datasets are used: CHB-MIT, Bonn and IIT-Delhi EEG datasets.
Training
Number of Trainable parameters for CHB-MIT, Bonn and IIT-Delhi datasets are 162894, 7250, 2058 only respectively which is very less for a transformer model.
Before training, for reliable results, I performed 6-fold cross validation in which one fold is taken as test dataset and remaining folds are taken as train dataset. I further took 25% of train dataset as validation dataset and remaining 75% is taken as train dataset.
I then trained the model for 75 epochs using Adam optimizer, batch size of 32 and cross entropy loss function.
Hyperparameters for each dataset is as follows:
Dataset
sequence length
Embedding dimension
learning rate
signal input size
patch size
depth
# attention heads
embedding dimension scale
Feedforward multiplier
Number of landmarks
CHB-MIT
168
64
0.005
(21,256)
(1,32)
3
4
2
4
32
Bonn
32
16
0.005
(1,256)
(1,8)
3
4
2
2
8
IIT-Delhi
32
8
0.001
(1,128)
(1,4)
3
4
2
2
8
Results:
Four metrics are considered: Accuracy, Sensitivity, Specificity and harmonic mean of sensitivity and specificity.
Accuracy, Sensitivity and Specificity in % on Test Dataset for CHB-MIT are 96.78, 97.46, 96.16 respectively, for Bonn are 98.36, 97.73, 98.66 on average respectively, and for IIT-Delhi are 96.77, 96.02, 97.98 respectively.
