Lexer match but not consume?
yonderblue opened this issue · comments
Is it possible to have the lexer match something like OR but also matches OR^ or OR or any kind of punctation but still only consume OR so the lexer can handle the next char with the next token?
I would have used regex lookahead but that isn't available I believe.
No, that should be done by the parser.
To clarify, the lexer should be as straightforward as possible, with no ambiguity between the tokens.
@alecthomas thanks for the response. Moved to discussion at #262.