maciejsmolinski / antlr-grammar-based-syntax-highlighter

🎨 A *proof of concept* minimal code editor with automatic syntax highlighting powered by ANTLR grammars via simple mapping from ANTLR tokens to color codes

Antlr Syntax Highlighter

A work in progress simple syntax highlighter and code editor powered by ANTLR grammars.

Given an antlr-generated lexer and a mapping from lexer-specific token ids to general token types, it is going to highlight the code without the need to install extra libraries.

Sample Usage

First, we import some helpers from this project:

import {
  CodeEditor,
  makeTokenizeFn,
  TOKEN_TYPES,
} from 'antlr-grammar-based-syntax-highlighter';

Next, we import our lexer generated by ANTLR from the grammar file and we pick the highlighting style for each of the tokens.

import { MyLanguageLexer } from 'antlr-generated-my-language-lexer-and-parser';

const TOKEN_MAPPING = new Map([
  [MyLanguageLexer.Integer, TOKEN_TYPES.NUMBER],
  [MyLanguageLexer.Identifier, TOKEN_TYPES.IDENTIFIER],
  [MyLanguageLexer.Return, TOKEN_TYPES.KEYWORD],
  [MyLanguageLexer.Break, TOKEN_TYPES.KEYWORD],
  [MyLanguageLexer.Continue, TOKEN_TYPES.KEYWORD],
  [MyLanguageLexer.LeftBrace, TOKEN_TYPES.BRACE],
  [MyLanguageLexer.RightBrace, TOKEN_TYPES.BRACE],
]);

Then, we generate a tokenize function that our code editor will use for code highlighting:

const tokenize = makeTokenizeFn(MyLanguageLexer, TOKEN_MAPPING);

We're almost ready. Let's glue everything together with a custom component:

const MyLanguageEditor = ({ code }) => {
  return <CodeEditor code={code} tokenize={tokenize} />;
};

export default MyLanguageEditor;

And finally, let's highlight some code with that component:

import React from 'react';
import ReactDOM from 'react-dom';
import MyLanguageEditor from './MyLanguageEditor';

const code = 'return { id: 175 }';

ReactDOM.render(
  <MyLanguageEditor code={code} />,
  document.getElementById('root')
);

Token Types

const TOKEN_TYPES = {
  NONE: 'none',
  RAW: 'raw',
  IDENTIFIER: 'identifier',
  NUMBER: 'number',
  KEYWORD: 'keyword',
  BRACE: 'brace',
};

Inspiration

• Creating an Editable Textarea That Supports Syntax-Highlighted Code

Local Setup

Clone the repository and execute the following commands in the project's dir from the terminal:

$ npm install --force
$ npm start # starts a server at localhost:3000

Updating Grammar and generating JavaScript parser/lexer

An example grammar definition is located under src/example/antlr/Lang.g4. To generate Parser/Lexer/Listener javascript files, please run make js in the root dir of the project.

ANTLR installation is required.