An application of Machine Learning on brainwave donations in order to remotely pilot a drone in real-time through brainwave reading.
Explore the docs »
Get help in the Wiki »
View Demo
·
Report Bug
·
Request Feature
Table of Contents
This is an open source project from the Cloud Computing Club of Saint Cloud State University. Brainwaves are read using an OpenBCI headset. These brainwaves are sent to a server and randomly renamed and dated for privacy. After shuffling, the data is loaded into Spark where it is processed and a piloting prediction is made. A client-side request reads the most recent prediction and instructs the drone for its next action.
Brainwave collection is considered human subject donation, and this project is compliant with the policies of the IRB .
There several components to the software. The server directory handles operations needed for storing data and generating ML models. The brainwave-prediction-app directory is operations to complete a live reading, receive a prediction, and then pilot the drone.
OpenBCI Headset with Cyton 16 Channel board
Clone the repo (use sparse checkout if only wanting to run the client app for live demos)
git clone https://github.com/3C-SCSU/Avatar.git-
On the VPS, install and launch the server.py file. It can be run from the command line with the command:
python server.py
-
Using the OpenBCI software, test the connections on the headset for the person wearing it. Adjust as necessary. Try and make all connections “Not Railed”
-
Locally, launch the GUI application using either an IDE play button or from terminal
python GUI.py
- Note the relative pathing to the images folder and launch with both a terminal and local files appropriately pathed
- If you receive a module error it can be installed using pip i.e.
pip3 install <module name>
-
In the GUI select Brainwave Reading
-
Future implementations - select between Manual Control or AutoPilot
-
Connect the computer to the Drone’s wifi and press the Connect button in the GUI to connect to the drone.
-
Give the person wearing the headset a 5 second countdown and instructions on which drone action to think about.
-
Press Read my Mind to automatically initiate a 10 second reading. This triggers the brainflow1.py file and sends the data to the server.py endpoint. The results of the prediction will automatically be displayed in the output table of the GUI. (The endpoint IP may need to be manually configured until environment variables are set up to handle this operation)
-
If the prediction is correct, press the Execute button to have the drone perform the action. If the prediction is wrong, override the drone action by typing an alternative in the text input box and press Not what I was thinking…
-
At any time, press “Keep Drone Alive” to attempt to maintain the drone’s hovering while it waits for a prediction command. (Note: This isn’t working well and will need to be improved)
This section will be updated further as the project progresses.
- Read Brainwaves using OpenBCI software
- Configure workstation Chromebooks for compatibility with BCI dongle
- Encrypted File Transfer from workstation to VPS
- Automated file shuffling
- Containerize VPS applications
- Implement K8s
- Connect Spark with GCP bucket
- Configure Spark with Delta Lake
- Create Application to Control Drone
- GUI
- Drone Control
- Headset Data transferring
- Connect with ML Predictions
See the open issues for a full list of proposed features (and known issues).
If you have been informed that your submission for contribution was selected and approved by the Cloud Computing Club, follow the below steps to add it to the project and receive the bounty.
- Create your Feature Branch (
git checkout -b feature/AmazingFeature) - Name the feature branch using format
feature-bountyex.file_shuffler-bounty - Commit your Changes (
git commit -m 'Add some AmazingFeature') - Push to the Branch (
git push origin feature/AmazingFeature) - Open a Pull Request and select a review from the list of Collaborators.
- Update the Wiki with an explanation of your added feature.
Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.
If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star! Thanks again!
- Fork the Project
- Create your Feature Branch (
git checkout -b feature/AmazingFeature) - Commit your Changes (
git commit -m 'Add some AmazingFeature') - Push to the Branch (
git push origin feature/AmazingFeature) - Open a Pull Request
Pull requests will be rieviewed and merged:
Members by promotion and merit voted in the weekly's #3C meeting:
-
Sporadic contributors: any developer who decides to start contributing to the project's code base.
-
Committers: contributors promoted above the so-called sporadic contributors. To become a committer, the person must get 20 patches approved. When becoming a committer, then the committer will review and support code submitted from sporadic contributors
-
Reviewers: are generally either the founding developers of the project - like initial administrator developers; or contributors who become committers, who then, after 50 patch contributions to the project's code base, are promoted to the status of reviewers. Reviewers will decide and review any conflict in code contributions to the project.
Distributed under the MIT License. See LICENSE for more information.
Cloud Computing Club 3c.scsu@gmail.com
Discord: https://discord.gg/mxbQ7HpPjq
Project Link: https://github.com/3C-SCSU/Avatar