Include token type in unexpected token error message
andremarianiello opened this issue · comments
Sometimes I get an unexpected parsing error like
unexpected token \"group\"
and I get confused because group should be a valid next token in my grammar. However, it turns out that group is a Keyword token in my lexer, not an Identifier token like I wanted. Upon realizing that the token was of the wrong type, the error makes sense now. However, realizing that the token type was the problem can take a while, especially if the lexer is complicated or contains overlapping rules, because the error message doesn't give any indication of the types of the tokens involved (found OR expected). I think that including the token types in the error message would help people facing this issue solve it much more quickly.
I think it would be good to add the token type if you'd like to send a PR.
Maybe something like unexpected token "group" of type keyword
I had a bit of a dig into this and unfortunately it would not be straightforward to fix this. The human-readable token names are contained in the lexer.Definition, but these errors come from locations that don't have access to that type. I don't have any good ideas for this at the moment.