leops / rasen

Generate SPIR-V bytecode from an operation graph

Rasen

What is Rasen ?

Rasen is an umbrella project for a bunch of SPIR-V and Rust related experiments. The core idea of Rasen is to provide a fast and safe compiler for SPIR-V modules, however the core compiler only provide a backend to generate a shader program from an intermediate representation, specifically a "dataflow graph". In traditional, text-based languages such a graph is a usually a metadata generated by the compiler, but some game engine (eg. Unreal) provide an editor to directly edit such a graph, which proved to be a very useful way of quickly prototyping materials.

In practice, Rasen as a whole is designed with two (seemingly unrelated) use case in mind:

1. To be used as an online shader compiler embedded in a game engine (or any other application doing 3D rendering). Rasen aims to be fast enough to recompile a shader within a frame's budget, and the resulting SPIR-V bytecode can then be retargeted to GLSL, HLSL or MSL using SPIRV-Cross to accomodate the current platform.
2. To provide a fallback environment for compute shaders in Rust. Rasen uses a special DSL crate to parse Rust code "from the inside" and generate a dataflow graph for execution on the GPU, but on constrained platforms where this capability is unavailable the Rust code is still valid and can be run on the CPU instead (and ultimately should be just as fast as raw rust code). As a sidenote, this should also help making shaders more testable and debuggable as they can be now be run on the CPU too.

Core

The rasen crate contains the core graph compiler itself. It provides graph building utilities (the Graph struct), various types (rasen::types::*) and operations (rasen::node::*) definitions, and SPIR-V compilation utilities (the ModuleBuilder struct).

The API is intentionally low-level, as the use case for the core compiler is to act as a backend for a graph-based material editor in a game engine. It's perfectly possible to use this crate as-is by creating a Graph struct and building the module node-by-node, however this method tends to be quite verbose for "static" shaders:

extern crate rasen;

use rasen::prelude::*;

fn main() {
    let mut graph = Graph::new();

    // A vec3 input at location 0
    let normal = graph.add_node(Node::Input(0, TypeName::VEC3, VariableName::Named(String::from("a_normal"))));

    // Some ambient light constants
    let min_light = graph.add_node(Node::Constant(TypedValue::Float(0.1)));
    let max_light = graph.add_node(Node::Constant(TypedValue::Float(1.0)));
    let light_dir = graph.add_node(Node::Constant(TypedValue::Vec3(0.3, -0.5, 0.2)));

    // The Material color (also a constant)
    let mat_color = graph.add_node(Node::Constant(TypedValue::Vec4(0.25, 0.625, 1.0, 1.0)));

    // Some usual function calls
    let normalize = graph.add_node(Node::Normalize);
    let dot = graph.add_node(Node::Dot);
    let clamp = graph.add_node(Node::Clamp);
    let multiply = graph.add_node(Node::Multiply);

    // And a vec4 output at location 0
    let color = graph.add_node(Node::Output(0, TypeName::VEC4, VariableName::Named(String::from("o_color"))));

    // Normalize the normal
    graph.add_edge(normal, normalize, 0);

    // Compute the dot product of the surface normal and the light direction
    graph.add_edge(normalize, dot, 0);
    graph.add_edge(light_dir, dot, 1);

    // Restrict the result into the ambient light range
    graph.add_edge(dot, clamp, 0);
    graph.add_edge(min_light, clamp, 1);
    graph.add_edge(max_light, clamp, 2);

    // Multiply the light intensity by the surface color
    graph.add_edge(clamp, multiply, 0);
    graph.add_edge(mat_color, multiply, 1);

    // Write the result to the output
    graph.add_edge(multiply, color, 0);

    let bytecode = build_program(&graph, ShaderType::Fragment).unwrap();
    // bytecode is now a Vec<u8> you can pass to Vulkan to create the shader module
}

DSL

To reduce the amount of boilerplate, the rasen_dsl crate provides a bunch of utility function to write shaders as perfectly valid Rust code:

extern crate rasen;
extern crate rasen_dsl;

use rasen_dsl::prelude::*;

fn main() {
    let shader = Module::new();

    let normal: Value<Vec3> = normalize(shader.input(0, "a_normal"));
    let light = vec3(0.3, -0.5, 0.2);
    let color = vec4(0.25, 0.625, 1.0, 1.0);

    let res = clamp(dot(normal, light), 0.1f32, 1.0f32) * color;
    shader.output(0, "o_color", res);

    let bytecode = shader.build(ShaderType::Fragment).unwrap();
    // bytecode is now a Vec<u8> you can pass to Vulkan to create the shader module
}

This crate is even more experimental than the Rasen compiler itself, it already provides all the features exposed by the compiler but they might not be completely spec compliant (for instance the typings constraint on the various GLSL functions may be more, or less strict than required by the OpenGL specification).

Ultimately, the goal for the DSL crate (beside being a statically-checked equivalent of the graph builder) is to expose an API to test the execution of a shader on the CPU, with all the debugging tools that such an environment provides. The library currently provides all the conversion primitives to turn your scalar / vectors / matrices into Value<_> types to test your program, however most GLSL operations are left unimplemented.

Plugin

Finally, the rasen_plugin crate is a compiler plugin exposing a few utility macro and attributes to make writing shaders in Rust event easier:

use rasen_dsl::prelude::*;

#[rasen(module)]
pub fn basic_vert(a_pos: Value<Vec3>, projection: Value<Mat4>, view: Value<Mat4>, model: Value<Mat4>) -> Value<Vec4> {
   let mvp = projection * view * model;
   mvp * vec4!(a_pos, 1.0f32)
}