This is a learning project for me to learn a bit about lexing, parsing, and evaluating code. It's focused on the GitHub Actions context and expression syntax.
I was inspired to work specifically on this while I was digging in and learning about the nektos/act project, and specifically seeing this issue in there about how the syntax is not quite valid javascript. nektos/act uses the Otto JS virtual machine to evaluate these expressions, so was failing (until an awesome workaround was submitted at least).
Under the hood I didn't want to just fall back on a JS VM, and also didn't want to go right to direct byte-by-byte lexing and parsing, so I built on top of the Participle package as it is both well used (lots of stars, etc), pure Go, and also had a lot of useful examples of different languages.
Nothing is done in a vacuum, especially stuff that I think of a "deep computer science-y". I read a bunch of blog posts and perused a many existing projects in order to put this together. Ideas came from all over:
- Rob Pike's presentation about
Lexical Scanning in Goon Youtube is a great dive into the space - The awesome work done by @cschleiden on his project GitHub Actions Parser that uses the JS library Chevrotain (which has a very well written tutorial by the way!)
- Another co-worker @fatih's blog post on go's lexer/scanner packages is a great read, plus looking at the resulting HCL parsing library that he build and then handed over to Hashicorp
- Go's own
text/scannergo/scanner, andgo/tokenpackages - The PEG project, as well as reading up on the ANTLR project got me thinking about language grammars
- I loved the sentence
Lex is not code that you live in. It is code you write once and then use for a long time. Ok if the code is not clean.from this article about using GoYACC - Various other blog posts as well
- Oh, and Regex101 for always being super helpful!