jhvst / scie

基于正则表达式的通用语言转换器。An interpreter for grammar files as defined by TextMate with Rust

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool



Scie is a research about how to build simple code identify engine for different languages.

goal: build a better code figure engine for code refactoring.

  • scie-cli. cli part of Scie.
  • scie-bingen. generate languages bin-data.
  • scie-detector. detector for different frameworks & languages.
  • scie-grammar. A library that helps tokenize text using Text Mate grammars.
  • scie-infra. common infrastructure support, like fs
  • scie-onig. Wrapper Rust Oniguruma FFI api based on rust-onig
  • scie-model. common model of VSCode models & Miao Model.
  • scie-scanner. Rewrite of VS Code Oniguruma


Scie CLI

scie-cli 0.1
Phodal HUANG<h@phodal.com>

    scie-cli [FLAGS] [OPTIONS]

    -h, --help       Prints help information
    -v, --verbose
    -V, --version    Prints version information

    -c, --config <config>    [default: default.conf]
    -p, --path <path>        [default: .]


  1. git clone
git clone https://github.com/phodal/scie/
  1. run
cargo run scie

install just

curl --proto '=https' --tlsv1.2 -sSf https://just.systems/install.sh | bash -s -- --to DEST

run tests

just tests


major issues:

  • performance
    • rule in Grammar seed in rule_container.rs
    • UTF 8 to UTF 16 in UtfString
    • normal issue
  • unstable
    • Random test failure on OnigScanner.
    • GC issues on OnigScanner.
      • GC issue seems resolved with Jemalloc.
      • Signal 6 (SIGABRT) = SIGABRT is commonly used by libc and other libraries to abort the program in case of critical errors. For example, glibc sends an SIGABRT in case of a detected double-free or other heap corruptions.
      • maybe UTF8 encoding issue


  • replace with fancy-regex for pure Rust impl
  • process todo
    • back references
    • multiple languages
  • rewrite VSCode-textmate with Rust
    • language for testing
    • support others language
  • benchmark
    • fast than VSCode version
  • multiple languages one project support
  • analyser
    • line counts
    • keywords map




scie-grammar based on vscode-textmate with MIT LICENSE see in scie-grammar/src/scanner/LICENSE

onigvs based on rust-onig

Phodal's Idea

@ 2020 A Phodal Huang's Idea. This code is distributed under the MPL license. See LICENSE in this directory.

ezoic increase your site revenue


基于正则表达式的通用语言转换器。An interpreter for grammar files as defined by TextMate with Rust

License:MIT License


Language:JavaScript 81.4%Language:TypeScript 12.4%Language:Rust 5.1%Language:Makefile 0.5%Language:C 0.2%Language:SCSS 0.1%Language:Groovy 0.1%Language:CSS 0.0%Language:C++ 0.0%Language:Python 0.0%Language:Perl 0.0%Language:HTML 0.0%Language:Ruby 0.0%Language:C# 0.0%Language:PowerShell 0.0%Language:Objective-C 0.0%Language:Objective-C++ 0.0%Language:PHP 0.0%Language:Clojure 0.0%Language:Less 0.0%Language:Visual Basic .NET 0.0%Language:Java 0.0%Language:Go 0.0%Language:F# 0.0%Language:Shell 0.0%Language:CoffeeScript 0.0%Language:Batchfile 0.0%Language:R 0.0%Language:Swift 0.0%Language:Lua 0.0%Language:Hack 0.0%