Implementation

This implementation is still a work in progress (it currently passes 99 out of 246 tests), but implementation details include:

Overall structure similar to clox, with a bytecode VM.
Uses existing code from Rust's ecosystem rather than reimplementing things like hashtables or garbage collection.
- Uses LALRPOP to parse an AST. This should avoids the rather large number of jumps that clox uses to implement for loops, once I get around to implementing them.
The bytecode format is serializable. The compiler and VM should are callable separately.
- The serialized bytecode format starts with a 0xc0 byte, which does not occur in valid UTF-8, so the interpreter can run both source code and bytecode without having to be passed any additional command-line options.
- To make the implementation of the remaining Lox features easier, the bytecode format is not yet stable across versions.

These implementation choices are mostly motivated by the fact that I'm using this as practice, so to speak, for another project I'm planning.

Things I learned

Working around the dangling else problem required a fair amount of code duplication in the parser. For my own language, it would be easier to avoid the problem altogether by always requiring braces like Rust and Swift do.
Due to the test suite's rather strict requirements regarding error output, I had to implement a custom lexer. This adds line information to runtime errors but makes some of the code much more verbose. When implementing my own language, I see two options:
- Use LALRPOP's built-in lexer (customized for line comments), omit line information from runtime errors and instead design the language to catch more errors at compile time (e.g. via strong static typing). The tree before this change can be used as a reference.
- Use my own lexer that produces a token type annotated with span metadata. This allows the lexer to consume a Read and the compiler to include line information in proper compile errors (i.e. after parsing). Since LALRPOP expects the token type to be an enum, this would make for awkward, repetitive code. Instead, look for a parser generator with better support for these things or hand-write the parser.