libsyntax2.0

libsyntax2.0 is an experimental parser of the Rust language, intended for the use in IDEs. RFC.

Quick Start

$ cargo test
$ cargo parse < crates/libsyntax2/src/lib.rs

Trying It Out

This installs experimental VS Code plugin

$ cargo install-code

It's better to remove existing Rust plugins to avoid interference. Warning: plugin is not intended for general use, has a lot of rough edges and missing features (notably, no code completion). That said, while originally libsyntax2 was developed in IntelliJ, @matklad now uses this plugin (and thus, libsytax2) to develop libsyntax2, and it doesn't hurt too much :-)

Features:

syntax highlighting (LSP does not have API for it, so impl is hacky and sometimes fall-backs to the horrible built-in highlighting)
commands (ctrl+shift+p or keybindings)
- Show Rust Syntax Tree (use it to verify that plugin works)
- Rust Extend Selection (works with multiple cursors)
- Rust Matching Brace (knows the difference between < and <)
- Rust Parent Module
- Rust Join Lines (deals with trailing commas)
Go to symbol in file
Go to symbol in workspace
- #Foo searches for Foo type in the current workspace
- #foo# searches for foo function in the current workspace
- #Foo* searches for Foo type among dependencies, excluding stdlib
- Sorry for a weired UI, neither LSP, not VSCode have any sane API for filtering! :)
code actions:
- Flip , in comma separated lists
- Add #[derive] to struct/enum
- Add impl block to struct/enum
- Run tests at caret
Go to definition ("correct" for mod foo; decls, index-based for functions).

Code Walk-Through

`crates/libsyntax2`

yellow, red/green syntax tree, heavily inspired by this
grammar, the actual parser
parser_api/parser_impl bridges the tree-agnostic parser from grammar with yellow trees
grammar.ron RON description of the grammar, which is used to generate syntax_kinds and ast modules.
algo: generic tree algorithms, including walk for O(1) stack space tree traversal (this is cool) and visit for type-driven visiting the nodes (this is double plus cool, if you understand how Visitor works, you understand libsyntax2).

`crates/libeditor`

Most of IDE features leave here, unlike libanalysis, libeditor is single-file and is basically a bunch of pure functions.

`crates/libanalysis`

A stateful library for analyzing many Rust files as they change. WorldState is a mutable entity (clojure's atom) which holds current state, incorporates changes and handles out Worlds --- immutable consistent snapshots of WorldState, which actually power analysis.

`crates/server`

An LSP implementation which uses libanalysis for managing state and libeditor for actually doing useful stuff.

`crates/cli`

A CLI interface to libsyntax

`crate/tools`

Code-gen tasks, used to develop libsyntax2:

cargo gen-kinds -- generate ast and syntax_kinds
cargo gen-tests -- collect inline tests from grammar
cargo install-code -- build and install VS Code extension and server

`code`

VS Code plugin

Performance

Non-incremental, but seems pretty fast:

$ cargo build --release --package cli
$ wc -l ~/projects/rust/src/libsyntax/parse/parser.rs
7546 /home/matklad/projects/rust/src/libsyntax/parse/parser.rs
$ ./target/release/cli parse < ~/projects/rust/src/libsyntax/parse/parser.rs --no-dump  > /dev/null
parsing: 21.067065ms

Getting in touch

@matklad can be found at Rust discord, in #ides-and-editors.

License

libsyntax2 is primarily distributed under the terms of both the MIT license and the Apache License (Version 2.0).

See LICENSE-APACHE and LICENSE-MIT for details.

darksv / libsyntax2