Chevrotain / chevrotain

Parser Building Toolkit for JavaScript

Home Page:https://chevrotain.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Cannot recovery by insert token just before EOF

hackwaly opened this issue · comments

Because EOF never appears in follows

  canRecoverWithSingleTokenInsertion(
    this: MixedInParser,
    expectedTokType: TokenType,
    follows: TokenType[],
  ): boolean {
    if (!this.canTokenTypeBeInsertedInRecovery(expectedTokType)) {
      return false;
    }

    // must know the possible following tokens to perform single token insertion
    if (isEmpty(follows)) {
      return false;
    }

    const mismatchedTok = this.LA(1);
    const isMisMatchedTokInFollows =
      find(follows, (possibleFollowsTokType: TokenType) => {
        return this.tokenMatcher(mismatchedTok, possibleFollowsTokType);
      }) !== undefined;

    return isMisMatchedTokInFollows;
  }

Hmm, I don't think EOF can be automatically added to all follows set.
Not all parsing rules may terminate with EOF, normally it is only the root one.

Can you try explicitly adding $.CONSUME(EOF) at the end of your root rule?
Does it solve your issue?

    $.RULE("json", () => {
      $.OR([
        { ALT: () => $.SUBRULE($.object) },
        { ALT: () => $.SUBRULE($.array) },
      ]);

      // explicitly consume End Of File.
      $.CONSUME(EOF);
    });

closing, re-open if this workaround does not work.