Ability to use walt-compiler in the browser?

[Jeffrey Warren]

Apologies if this is written somewhere, but I couldn't quite figure it out -- can the walt compiler be run in a browser environment, perhaps webpacked itself? I didn't see that https://ballercat.github.io/walt/ dynamically converted walt code to JS or WASM.

Anyways I'm curious about this because we have a project where we dynamically generate WebGL shaders within a narrow set of guidelines: https://github.com/publiclab/infragram/blob/main/src/processors/webgl.js#L145

In another related project, we are looking to take basic math expressions, and in a similar vein, dynamically WASM programs around them which can be run on pixel buffers at high speed:

publiclab/image-sequencer#400

These are mostly going to be arithmetic work in the general format of accepting 4 color values for a pixel, and returning them with some really basic transformations:

https://github.com/publiclab/image-sequencer/blob/1311119787e6b7e7100cec80da59253adb0eba8b/src/modules/Brightness/Module.js#L22-L29

A number of our simple image manipulation modules rely on a common per-pixel processing module, and if we could build in an automated Walt-based WASM compiler, we could dramatically accelerate these at runtime. Alternatively we could try to wire it to precompile this on a per-module basis, but it'd be awesome to try to integrate this in a more universal way in our stack.

Thank you so much, this is a very cool project, congrats!

[Arthur Buldauskas]

Hey, thanks!

Yes, the compiler can run in both node and the browser. The explorer pages do the compilation to WebAssembly on the fly.

[Max Graey]

Hmm, I think you could dramatically speedup without wasm. Such of code is quite inefficient:

function changePixel(r, g, b, a){
  var val = (options.brightness)/100.0
  r = val*r<255?val*r:255
  g = val*g<255?val*g:255
  b = val*b<255?val*b:255
  return [r , g, b, a]
}

1. For every pixel you do unnecessary division (options.brightness)/100.0. This can easily precalculate.
2. r = val*r<255?val*r:255 could simplify to r = 255 * Math.min(val, 1)
3. For every pixel you generate Array object [r, g, b, a] which is dramatically slow

You could much better:

function changePixel(buffer, value) { // buffer is inout `Uint8ClampedArray`, `value` in range [0.0, 1.0]
  buffer[0] *= value; // inplace r *= value
  buffer[1] *= value; // inplace g *= value
  buffer[2] *= value; // inplace b *= value
}

or immutable way:

function changePixel(outBuffer, inBuffer, value) { // where outBuffer/inBuffer is `Uint8ClampedArray`, `value` in range [0.0, 1.0]
  outBuffer[0] = inBuffer[0] * value;
  outBuffer[1] = inBuffer[1] * value;
  outBuffer[2] = inBuffer[2] * value;
}

Or better - provide method for change whole buffer and to by pixel which allow user iterate through buffer by himself.

[Jeffrey Warren]

Thanks for the excellent input. Yes, there's a lot of room for optimization in that particular module, and I'll copy back your suggestions to get them worked in! Awesome.

But more broadly it'll be awesome to see a path towards walt based modules. We aspire to do real-time video processing with these, at some point. 👍 🎥

Thanks, everyone!

[Job van der Zwan]

BTW, if you want another example: I just tried embedding the Walt in Observable (using @jtiscione's wave example), and it was really easy. Took me less than half an hour :)

https://beta.observablehq.com/@jobleonard/compiling-walt-in-the-observable-experiment