LLM generates text that the grammar does not allow
mattiasarro opened this issue · comments
When running the example code, the following exception gets thrown:
UnexpectedCharacters: No terminal matches '"' in the current parser context, at line 1 col 7
The model had generated output {"a": " and the parser did not parse this particular output. Since {"a": " is a valid start of a JSON object, the error is a bit confusing. But playing around with different grammar, prompts and models a bit shows that the model can (and often does) generate text the parser can not parse, which results in lark.lexer.UnexpectedCharacters exception.
This is actually a bug in rellm, created a PR there.