This project is discontinued
Token.cs (TokenCs)
A simple library to abstract the gritty process of lexical parsing.
Installation
This package is available on nuget.
Samples
For extended samples, see /samples/.
Quirks and Miscellaneous Features
- Symbols are always parsed as single units. That is, a token containing more than one symbol will never be generated by the tokenizer or lexer. The same goes for whitespace (CRLF is an exception).
- LexerTokenType.AlphaNumeric will always contain alphabetical characters. It may or may not contain numeric characters, and will never be purely numeric.