This Unity application demonstrates the sonosthesia MIDI packages, which are part of a wider set of packages aimed at facilitating the creation of immersive, interactive audio visual art. These modules provide an abstraction layer for MIDI messages, with a human readable API for MIDI input and output. The core message types and abstract input and output APIs are provided by
While concrete implementations are provided by
Note that to add those packages to your Unity project you will need to add the following scoped registeries to your Packages/package.json file (Keijiro only necessary for com.sonosthesia.rtmidi). Note this can
"scopedRegistries": [
{
"name": "Neuecc",
"url": "https://package.openupm.com",
"scopes": [
"com.neuecc.unirx",
"com.cysharp.unitask"
]
},
{
"name": "Keijiro",
"url": "https://registry.npmjs.com",
"scopes": [
"jp.keijiro"
]
},
{
"name": "Sonosthesia",
"url": "https://registry.npmjs.com",
"scopes": [
"com.sonosthesia"
]
}
]
UI test and monitoring tools which plug into the abstraction layer API are contained in the Assets folder of the application itself.
Platform support is limited to Windows, macOS and Linux due to the dependency on jp.keijiro.rtmidi
Listen to MIDI input port messages on the local machine using the RtMIDIInputStream component from com.sonosthesia.rtmidi.
Send messages to MIDI output port on the local machine using the RtMIDIOutputStream from com.sonosthesia.rtmidi.
Supports all platforms. Generate MIDI messages from midi file tracks using the Unity timeline. Uses the TimelineMIDIOutput Implemented in com.sonosthesia.timelinemidi.
Supports all platforms. Connect to a running sonosthesia-daw-connector to send and receive MIDI messages from a remote machine (including mobile and VR platforms). Follow setup and config instructions and ensure that the installed major/minor version matches the com.sonosthesia.pack package version in your unity project. You can ensure a specific major/minor version of the sonosthesia-daw-connector using @ without specifying patch version (the latest patch release will be installed)
npm install -g sonosthesia-daw-connector@1.3
MIDI input uses the PackRawMIDIInputStream
MIDI output uses the PackRawMIDIOutputStream
Channels provide a higher level of abstraction to MDI notes and MPE notes by combining different types of messages to create UniRx data streams for each note. They are implemented in the com.sonosthesia.midi package.
MIDI note on, note off and polyphonic aftertouch messages can be combined to create a sonosthesia channel with each note represented as a separate stream of MIDINote with a variable Pressure field.
MIDI note on, note off, control change (74), channel aftertouch and pitch bend messages can be combined to create a sonosthesia channel with each MPE note represented as a stream of MPENote with variable Slide, Pressure and Bend fields.
MIDI Song Position Pointer, Clock, Start, Stop and Continue messages can be used to synchronize Unity with a DAW. Bar, Beat and Sixteenth info is infered based on provided time signature.
Note the MIDI Sync functionality must be enabled in your DAW for the relevant MIDI output. In Ableton Live the song MIDI clock type must be used. If unsure of the required settings of your particular DAW you can use applications like MIDIMonitor to check that it is generating the required sync messages.
Provide example transport tracking with two different MIDI backends.
- Song Position Pointer messages are not sent by Ableton Live when looping for some reason.
- com.sonosthesia.rtmidi version misses a Song Position Pointer message when clicking on the scrub area in Ableton Live. This problem does not occur with com.sonosthesia.pack.
- Time signature must be provided as it is not provided by MIDI sync messages.
- Time signature changes are not supported.