OrcaHello: A real-time AI-assisted killer whale notification system 🎱 🐋
Orcasound has set up a hydrophone network in the Puget Sound. Killer whales sometimes swim by these hydrophones and vocalize in the form of calls.
We trained a deep learning model to find these calls in hydrophone audio. Each overlapping 2 second sub-segment is classified as whale call/ no-whale call. Shown below is a 1-minute segment of hydrophone audio visualized as a spectrogram with whale calls detected by the model.
When whale-activity is detected by the model, it sends an email to our Moderators who are killer whale experts.
Once they receive this email, they can visit the Moderator Portal shown below to confirm or reject model detections.
Most importantly, they tag detections with information if the whale call was emitted by a Southern Resident Killer Whale (SRKW) - an endangered ecotype found in the Puget Sound. If a SRKW is confirmed, an email is sent to subscribers.
This repository contains the implementations for the following components that make up OrcaHello.
- ModeratorFrontEnd - Frontend code for the Moderator Portal.
- NotificationSystem - Code to trigger email notifications.
- InferenceSystem - Code to perform inference with the trained model.
- ModelTraining - Data preparation and model training.
- ModelEvaluation - Benchmarking trained models on test sets.
System overview
The diagram below describes the flow of data through OrcaHello and the technologies used.
Contributing
You can contribute by
- Creating an issue to capture problems with the Moderator Portal and documentation here.
- Create a pull request to fix an issue or add documentation.
To contribute a pull request for a specific subsystem, please read the corresponding contributing guidelines and READMEs.
- ModeratorFrontEnd | README | Contributing Guidelines
- NotificationSystem | README | Contributing Guidelines
- InferenceSystem | README | Contributing Guidelines
- ModelTraining | README | Contributing Guidelines
General Resources
Project Page - contains information about the system and a brief history of the project.
Related Projects
- aifororcas-podcast - A tool to crowdsource labeling of whale calls in Orcasound's hydrophone data.
- aifororcas-orcaml - Original baseline machine learning model and data preparation code.