WebAM

WebAM is "Web Audio and MIDI" in TypeScript/JavaScript; currently the emphasis is on General MIDI interactive playback using SoundFont (.sf2) files (and for now it is not intended to play General MIDI files!)

Currently the timing is provided via WAAClock, and the SoundFont processing is based on sf2synth.js (with several fixes). The API tries to follow that in MIDI.js.

The basic technologies for WebAM are described in Web MIDI API and Web Audio API.

NOTE: This software is still under major developments (alpha version) with no release number yet!

Files

The source code of WebAM is webam.ts. To incorporate it in an HTML file, a JavaScript file has been created with

tsc webam.ts -t es5

with tsc v2.0.3 and taking out the "exports" line

exports.WebAudioMidi = WebAudioMidi;

and data.handleId

this.frmReq = (window.requestAnimationFrame(function () {
    _this.scheduleFrame();
})).data.handleId;

from the resulting webam.js. An HTML file (index.html) has been provided as an example. WebAM was tested so far using only the Chrome browser (primarily version 54.0.2840.71 m (64-bit) on Windows PC and other versions on Android devices).

How to Use

In your HTML:

<script src="WAAClock-latest.js"></script>
<script src="webam.js"></script>

In your script:

let wam = new WebAudioMidi(callback, 'sf/Chaos_V20.sf2');
function callback() {
  wam.runTests(); // an example
  ...
}

You have to provide a General MIDI SoundFont file, and in this example, the file is Chaos_V20.sf2 (which can be obtained from Some GM SoundFonts at SynthFont website) which is located in the directory sf.

Instead of using the delay argument in the functions (see the API for Sound Generation methods), you can also use WAAClock directly for the timings; for example:

let clock = new WAAClock(new AudioContext);
clock.start();
clock.setTimeout(function() {wam.noteOn (0, 60);}, 1);
clock.setTimeout(function() {wam.noteOff(0, 60);}, 2);

The latest was WAAClock-latest.js as of Oct 12, 2016 (release 0.5).

Examples

NOTE: Because requestAnimationFrame is used for the MusicEngine, the WebAM browser tab has to be in focus for it to play normally. (It cannot be in the background.)

Similar Projects

Installation in Windows

In Windows you can use WebAM using the Web MIDI API part (instead of Web Audio API) by installing CoolSoft VirtualMIDISynth; some explanations are provided at Enabling Sound in Windows at DrawMusic website. Hopefully, the Windows built-in Microsoft GS Wavetable SW Synth will be re-enabled by Google at some time in the future as discussed in "Web MIDI Does Not Work on 43.0.2357.130".

Without any MIDI devices connected, WebAM will show a notification in the beginning, such as "No real MIDI output ports. (Sounds will be generated via SoundFont.)".

SoundFont Files

For WebAM to work on any device under Chrome browser, you have to provide a General MIDI SoundFont file. Here, we have used Chaos_V20.sf2 (11.9 MB). In Enabling Sound in Windows, they use TimGM6mb.sf2 (5.9 MB) which is one of the smallest General MIDI SoundFont files. sf2synth.js, the current basis for the WebAM's Web Audio API part, uses A320U.sf2 (9.5 MB).

Chaos_V20.sf2 was selected because it is relatively small (some General MIDI SoundFont file sizes are about 1 TB!) and in my opinion, it has the best percussion sounds relative to its size. You may experiment with other General MIDI SoundFont files by searching them in the Internet.

The SoundFont file format is described in "SoundFont Technical Specification, Version 2.04, February 3, 2006". A white paper by Dave Rossum is available as "The SoundFont 2.0 File Format".

API

Creation

let wam = new WebAudioMidi(callbackFunction, soundfontURL, optional WebamOptions options);

dictionary WebamOptions {
  optional boolean sysex;
  optional boolean software;
  optional boolean conolog;
  optional boolean engine;
}

callbackFunction is a function that WebAM will call(back) once it is ready.

soundfontURL is the URL for the SoundFont file.

In the optional WebamOptions, "sysex" and "software" correspond to those in Section 4.2 of Web MIDI API W3C Editor's Draft 09 June 2016 (http://webaudio.github.io/web-midi-api/), whereas "conolog" specifies whether WebAM prints to the console log. If "engine" is true then WebAM also creates a MusicEngine (see next section). All these by default are false.

Example:

let wam = new WebAudioMidi(callback, 'sf/Chaos_V20.sf2', {software: true, conolog: true});

Member Accesses

// Read-only
mAccess   : any         // MIDIAccess object in Web MIDI API
failureMsg: string      // if mAccess is null, provides the failure message
audioCtx  : any         // AudioContext object in Web Audio API
wClock    : any         // built-in WAAClock

musicEngn : MusicEngine // music engine

// Read and write
conolog       : boolean // whether WebAM prints to the console log
sconolog      : boolean // whether WebAM prints to the console log for sending (MIDI) data; by default it is false

dfltOutIdx    : number  // default output port index      (-1); negative means the last one
dfltNoteOnVel : number  // default MIDI note on velocity  (80)
dfltNoteOffVel: number  // default MIDI note off velocity (64)
dfltChnl      : number  // default MIDI channel           (0)

List Methods

listMIDIAccessProperties(optional string excludedType): string // list MIDI Access properties
listOutPorts            (optional string excludedType): string // list output ports
listInPorts             (optional string excludedType): string // list input ports

All the methods above return a string. The properties that are of type excludedType will not be listed.

Example:

console.log(wam.listMIDIAccessProperties('function'));

Utility Methods

noteToKey(number): string // 60 -> 'C4'
keyToNote(string): number // 'bb7' -> 106

In noteToKey(), it is assumed that the input is within the 0 to 127 range. In keyToNote(), it is assumed that the first character is A to G (either upper or lower case), possibly followed by some '#' (which represents sharp) or 'b' (which represents flat), and ended by an integer (which can be negative).

Sound Generation Methods

noteOn  (channel, note,    velocity = wam.dfltNoteOnVel, delay = 0, outIndex = wam.dfltOutIdx) : void
noteOff (channel, note,    delay = 0, outIndex = wam.dfltOutIdx, velocity = wam.dfltNoteOffVel): void

chordOn (channel, notes[], velocity = wam.dfltNoteOnVel, delay = 0, outIndex = wam.dfltOutIdx) : void
chordOff(channel, notes[], delay = 0, outIndex = wam.dfltOutIdx, velocity = wam.dfltNoteOffVel): void

programChange(channel, program, delay = 0, outIndex = wam.dfltOutIdx): void

In all the methods above, the delay is in milliseconds and all the arguments are numbers (or arrays of numbers).

For any general MIDI message, you can also use

send(message, delay = 0, outIndex = wam.dfltOutIdx): void

sendAt(message, at = wam.audioCtx.currentTime, outIndex: number = wam.dfltOutIdx): void

While the delay (relative time) is in milliseconds, the at (absolute time) is in seconds! Therefore, for example, the noteOn(c, n, v, d, o) method is identical to send([0x90 + c, n, v], d, o) and sendAt([0x90 + c, n, v], wam.audioCtx.currentTime + d/1000, o).

Music Engine

If you want tighter timings (or if you don't want to manage timings by yourself), the WebamOptions also includes engine, which if is set to true, then WebAM also creates a MusicEngine:

let wam = new WebAudioMidi(callback, 'sf/Chaos_V20.sf2', {engine: true});
let me = wam.musicEngn;

Or, we can combine them into a single line:

let me = (new WebAudioMidi(callback, 'sf/Chaos_V20.sf2', {engine: true})).musicEngn;

The event scheduling in the music engine is based on the article "A Tale of Two Clocks - Scheduling Web Audio with Precision" by Chris Wilson.

However, if MusicEngine is included, and you want to use WebAM Sound Generation methods, you will have to start the clock manually first:

wam.wClock.start();

before you call those methods.

Because requestAnimationFrame() is used in MusicEngine to provide the timeouts, one drawback is that the WebAM browser tab has to be in focus for it to play normally (i.e., it cannot be in the background).

API

Member Accesses

// Read-only
webAM: WebAudioMidi // WebAM object

// Read and write
conolog: boolean // whether MusicEngine prints to the console log

quanPerQuarterNote: number // quantizationn per quarter note (24)
lookaheadTime     : number // lookahead time (sec)

tempo      : number // in bpm (beats per minute); must be greater than zero; can be fraction
numerator  : number // numerator in time signature (how many beats in each measure)
denominator: number // denominator in time signature (note value for each beat)

The "Tune" Structure

To play something with MusicEngine, you have to create a "tune", and load( ) it. A tune is an object which consists of arbitrary number of tracks, as described below:

load(TuneInterface tune): void

dictionary TuneInterface {
  optional number tempo;       // in bpm (by default 120)
  optional number numerator;   // numerator in time signature (by default 4)
  optional number denominator; // denominator in time signature (by default 4)

  TrackInterface tracks[];     // array of TrackInterface
}

dictionary TrackInterface {
  optional number  program;     // General MIDI program number (0 - 127)
  optional number  channel;     // MIDI channel (by default wam.dfltChnl)
  optional boolean pitchChange; // TBD: for transpose and octave (by default true)
  optional number  outIndex;    // output port index (by default wam.dfltOutIdx)
  optional number  repeat;      // how many times this track will be repeated if shorter than other tracks
                                // if less than 1, it implies it will be repeated for the whole tune length

  NoteInterface    notes[];     // array of NoteInterface
}

array NodeInterface [
  number note,
  optional number duration,
  optional number velocity
]

A NoteInterface is an array which must have at least one member (note) and at most three members (with duration and velocity):

The note represents a MIDI note (0 - 127). If it is negative, it means it is a rest (or silence).
The duration represents the duration of the note in note value, such as 1/4, 1/8, or 1/2. If it is omitted, by default it is equal to 1/4 (quarter note).
The velocity represents a MIDI velocity (0 - 127). If it is omitted, by default it is equal to wam.dfltNoteOnVel. And a velocity of 0 also represents a rest.

Example:

me.load({tempo: 120, tracks:
         [{channel: 0, notes: [[60], [64], [65], [67]], program: 53},
          {channel: 9, notes: [[36], [-1]], repeat: -1},
          {channel: 9, notes: [[-1], [38]], repeat: -1},
          {channel: 9, notes: [[42, 1/8]],  repeat: -1},
          {channel: 2, notes: [[36, 1/8], [48, 1/8]], repeat: -1, program: 32},
         ]});

Transport Methods

start(times = 1): void // from Stop state to Play state
pause()         : void // from Play state to Pause state
resume()        : void // from Pause state to Play state
stop()          : void // go to Stop state

In the start() method, the times argument specifies how many times the tune will be played.

Future Developments

It seems that music in Java/Clojure has progressed much more significantly; therefore the string input notation for the Music Engine tune may try to follow that in JFugue, semitone, Alda, and/or Overtone.
It has been the original objective of this effort is to make the sound outputs to be identical as those of CoolSoft VirtualMIDISynth. sf2synth.js is a great first attempt, but if you compare them in Windows, the sound outputs of them are different (VirtualMIDISynth sounds better). Hopefully, Web Audio API contains the complete API that makes it possible to tweak the JavaScript codes so that the two sound identical. (The drawbacks of VirtualMIDISynth are first, it has to be installed manually by the user, and second, it works only in Windows, whereas WebAM will work on any Chrome browser without any further installations.)

Other References

Wikipedia provides comparison of free software for audio.
YouTube provides some background in computer music in Programming Music with Overtone - Sam Aaron.
Wikipedia also provides musical notation, List of musical symbols and ABC notation.

midixman / webam

WebAM

Files

How to Use

Examples

Similar Projects

Installation in Windows

SoundFont Files

API

Creation

Member Accesses

List Methods

Utility Methods

Sound Generation Methods

Music Engine

API

Member Accesses

The "Tune" Structure

Transport Methods

Future Developments

Other References

License

About

Languages