Giters

NeuroJS / eeg-stream-data-model

An attempt to standardize a data model for EEG streaming

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Standardize EEG Data Model for Streaming

alexcastillo opened this issue · comments

commented

Hi everyone,

I've created this repo with the purpose of discussing a data model standard for streaming EEG.
Please review the README.md file and feel free to submit PRs.

cc @aj-ptw @marionleborgne @urish @teonbrooks @andrewheusser

Best,

Alex

commented

There is a lot of redundancy with this model. The only things that should be in a sample are dynamic variables i.e.:

{
  "buffer": [ // List of samples
    {
      "timestamp": Date, 
      "data": [ // List of channels
         float, 
         float, 
         float, 
         float
      ]
    }
  ]
}

Having to send these over the wire means we need to be lean as possible. I find the best results when i minimize the amount of redundant data between sample.

I think a header or start byte define what the metrics are:

{
  "metric": String // E.g. EEG, ACCL, etc (optional)
  "source": String // E.g. Muse, OpenBCI, etc (optional)
  "sampleRate": Number // E.g. 250. always in hz (optional)
  "mock": Boolean // E.g. false (optional),
  "topology": [ 'Pz', 'Oz', 'Cz', 'C1']
}

especially with hierarchal topics we can inject and split on these exact metrics.

commented
commented

Basically what AJ says. Except the streaminfo (what he calls the headers) should respect the LSL convention (really, it's the XDF format they're using)

commented

Out of curiosity, why not using LSL ? There is no js implementation but I might be easier to make one rather than creating a new standard from scratch.

commented

Thanks everyone for the feedback!

Sounds like we want to go leaner and use LSL.
I'm not familiar with LSL but definitely don't want to reinvent the wheel.

@alexandrebarachant can you point me to an LSL implementation in other languages? Maybe a Python implementation.

Maybe we can port it to JavaScript to make the communication between server/client more efficient.

Best,

Alex

commented

That's the one. I'm using it extensively.

Now, the typical usecase is streaming on a local network, so that is why I'm asking the question.

commented
commented

I went ahead and created a small node program to stream Muse 2016 data to LSL:

https://github.com/urish/muse-lsl

Basically, it leverages muse-js over noble (BLE bindings for Node.js) using bleat (emulates Web Bluetooth for muse-js), and then uses ffi to interface with liblsl.

So now I got the data streaming to LSL (I can see it in their Lab Recorder program).

Do you have any good pointers for LSL compatible software that I can use to visualize the EEG data, etc.?

commented

This is awesome, Uri!

Could we separate the lsl part into its own lib that can accept an EEG Observable?
That way we could use it for the Muse, OpenBCI, etc.

And of course, let me know how I can help.

commented

How do I stream to LSL from OpenBCI shield? Arduino C/C++ compatible.

commented

thank Alex!

Basically, the LSL wrapper sits in https://github.com/urish/muse-lsl/blob/master/lsl.js, so it can be quite easily separated. I don't have much experience with LSL though - I first heard about it today :)

commented
commented

Thank @alexandrebarachant , going to try to feed my muse-lsl into the viewer in yours!

What's the best way to communicate with you? Got some questions regarding how to analyze muse data, seems like you have some practical experience with this

commented

@urish you can contact me by email. I'm also hanging around in the NeurotechX slack

commented

cool, I just joined it! Ping you there

commented

hey everyone, I'm just now catching up. I'm in Cape Town, which is GMT+2. I'm flying tomorrow evening and landing Tuesday afternoon. I will also poke my head into the slack. So far, I like what I am seeing :)

commented

Quick update: node js implementation of muse-lsl can now stream values successfully, tested with lsl-viewer. Thanks for @alexandrebarachant a few bugs were fixed :-)

commented

Hi everyone,

This is the data structure of the cloudbrain pub/sub lib. It extends LSL beyond the lab network but is still using the LSL data structure for ease of use in other applications.

LSL refresher

In the LSL model, you have 3 main objects:

  1. StreamInfo to describe the structure of the samples. (The stream info can be used for XDF file headers)
  2. StreamInlet to consume samples.
  3. StreamOutlet to publish samples.

I the following section, I explain the data model that we have been using in Cloudbrain to extend LSL beyond the lab network. It's basically the same as LSL, except that we serialize objects to JSON so that we can have a uniform data contract between distributed applications.

Data model

You have two main objects that are being passed:

  • Samples: they contain time-stamped channel values. Used by StreamOutlet/ StreamInlet to send/receive data.
  • StreamInfo: Describes the sample with attributes like sampling_rate, channel_count
    , etc..

These 2 objects (Samples and StreamInfo) are being serialized to JSON strings to be passed between different applications.

Samples

StreamInlets and StreamOutlets send and receive samples. A sample has the following data format: {"data": [<float>, ..., <float>], "timestamp": <float> }

You can send and receive chunks of samples. In this case:

{
 "chunk": [
    {"data": [<float>, ..., <float>],  "timestamp": <float> },
    ...
    {"data": [<float>, ..., <float>],  "timestamp": <float> }
  ]
}

StreamInfo

In LSL, StreamInfo can be serialized to XML. In the cloudbrain lib that extends LSL, I added the ability to serialize StreamInfo to JSON since this is how other types of data are being serialized and shared between distributed apps.

{
  "info": {
    "name": <string>,
    "type": <string>,
    "channel_count": <int>,
    "nominal_srate": <float>,
    "channel_format": <string>,
    "source_id": <string>,
    "version": <string>,
    "created_at": <timestamp>,
    "uid": <string>,
    "session_id": <string>,
    "hostname": <string>,
    "v4address": <string>,
    "v4data_port": <int>,
    "v4service_port": <int>,
    "v6address": <string>,
    "v6data_port": <string>,
    "v6service_port": <int>,
    "desc": {
      "channels": {
        "channel": [
          {
            "label": <string>,
            "unit": <string>,
            "type": <string>
          },
           ...
          {
            "label": <string>,
            "unit": <string>,
            "type": <string>
          }
        ]
      },
      "cap": {
        "name": <string>,
        "size": <string>
        "labelscheme": <string>
      },
      "manufacturer": <string>
    }
  }
}

Examples

  • Buffer of 5 samples. Each sample has 2 channels:
{ 
  "chunk": [
    { "data":[ 7.056745022195285, 4.754953953750924], "timestamp": 1497479774194733.0},
    { "data":[ 7.056745022195285, 4.754953953750924], "timestamp": 1497479774194733.0},
    { "data":[ 7.056745022195285, 4.754953953750924], "timestamp": 1497479774194733.0},
    { "data":[ 7.056745022195285, 4.754953953750924], "timestamp": 1497479774194733.0},
    { "data":[ 7.056745022195285, 4.754953953750924], "timestamp": 1497479774194733.0}
  ]
}
  • StreamInfo
{
  "info": {
    "name": "muse",
    "type": "EEG",
    "channel_count": 8,
    "nominal_srate": 0,
    "channel_format": "float32",
    "source_id": "some_id",
    "version": "1.1",
    "created_at": "0",
    "uid": null,
    "session_id": null,
    "hostname": null,
    "v4address": null,
    "v4data_port": 0,
    "v4service_port": 0,
    "v6address": null,
    "v6data_port": 0,
    "v6service_port": 0,
    "desc": {
      "channels": {
        "channel": [
          {
            "label": "FP1",
            "unit": "millivolts",
            "type": "undefined"
          },
          {
            "label": "FP1",
            "unit": "millivolts",
            "type": "undefined"
          },
          {
            "label": "FP1",
            "unit": "millivolts",
            "type": "undefined"
          },
          {
            "label": "FP1",
            "unit": "millivolts",
            "type": "undefined"
          },
          {
            "label": "FP1",
            "unit": "millivolts",
            "type": "undefined"
          },
          {
            "label": "FP1",
            "unit": "millivolts",
            "type": "undefined"
          },
          {
            "label": "FP1",
            "unit": "millivolts",
            "type": "undefined"
          },
          {
            "label": "FP1",
            "unit": "millivolts",
            "type": "undefined"
          }
        ]
      },
      "cap": {
        "name": "undefined",
        "size": "undefined",
        "labelscheme": "undefined"
      },
      "manufacturer": "Interaxon"
    }
  }
}
commented

@marionleborgne
This is perfect, exactly what I was looking for. I'll create some charts that will take this data format as an input.

@urish What do you think? We could update our work to use this data format.

Thanks, everyone.

commented

@alexcastillo glad you like it. @aj-ptw is also using this data contract when pushing samples to cloudbrain from his wifi shield.

commented

@alexcastillo looking forward to the charts! 👍

commented

Thank you so much @marionleborgne !

I just updated muse-js to emit the sample data in this format, and muse-lsl + angular-muse to consume it.

@alexcastillo - also looking forward to the charts :)

commented

How am I supposed to stream from Arduino to LSL? Any ideas here?

commented

@aj-ptw : your best chance is to ask LSL people directly. there is a C implementation, but i have no idea if you can compile it in a Arduino library.

commented

@aj-ptw another option, since we are in the process of adding cloudbrain connectivity to the wifi shield, is to publish data over MQTT to cloudbrain and then just start the cloudbrain module that publishes data to LSL.

commented

Haha, first thing that comes up when googling this question :p sccn/labstreaminglayer#52 @aj-ptw beats me to it

commented

@alexcastillo @alexandrebarachant @urish @teonbrooks

Question from @aj-ptw and I: anyone has an objections to sending voltage values as nano volts. The channels format will be int.

See OpenBCI/OpenBCI_WIFI#13 (comment) for background.

commented
commented
commented
commented

@urish i just looked at your muse-lsl repo. It's awesome! Thanks 👍

commented

@urish it occurred to me that you could actually abstract / pull out the LSL part out of the muse-lsl repo and this would be a generic LSL lib for JS - doesn't have to be tied to muse. For example, OpenEXP (@teonbrooks ) can use it publish event markers to LSL. His app is in JS.

You just created the missing JS LSL lib 😄

commented
commented

This is great @urish. Thanks!