Parser raise read error on empty source

[Davide Lettieri]

I think there might be an issue with the parser generator or at least a missing detail in the documentation. It is not clear how to handle an empty input, the following code shows the issue using a simple lexer and parser definition.

#lang racket

(require parser-tools/lex
         parser-tools/yacc)

(define-tokens tokens (ANY))
(define-empty-tokens empty-tokens (EOF))

(define sample-lexer
  (lexer
   [(eof) (token-EOF)]
   [any-string (token-ANY lexeme)]))

(define sample-parser
  (parser
   [start expr]
   [end EOF]
   [error void]
   [tokens tokens empty-tokens]
   [grammar
    [expr [(ANY) $1]]]))

(define (parse s)
  (define in (open-input-string s))
  (sample-parser (lambda () (sample-lexer in))))

(parse "abc")
(parse "")

The output

"abc"
parser: Cannot continue after error
  context...:
   /snap/racket/current/usr/share/racket/collects/syntax/readerr.rkt:15:2: -raise-read-error
   /snap/racket/current/usr/share/racket/pkgs/parser-tools-lib/parser-tools/yacc.rkt:356:16: parsing-loop
   body of "/home/davide/repos/sample-racket-language/sample.rkt"

The issue is more complex when the parser has multiple end tokens for example

#lang racket

(require parser-tools/lex
         parser-tools/yacc)

(define-tokens tokens (ANY))
(define-empty-tokens empty-tokens (EOF NEWLINE))

(define sample-lexer
  (lexer
   [(eof) (token-EOF)]
   [";" (token-NEWLINE)]
   [(repetition 1 +inf.0 (char-complement ";")) (token-ANY lexeme)]))

(define sample-parser
  (parser
   [start expr]
   [end EOF NEWLINE]
   [error void]
   [tokens tokens empty-tokens]
   [grammar
    [expr [(ANY) $1]]]))


(define in (open-input-string "abc;def"))
(sample-parser (lambda () (sample-lexer in)))
(sample-parser (lambda () (sample-lexer in)))
(sample-parser (lambda () (sample-lexer in)))

The output is

"abc"
"def"
parser: Cannot continue after error
  context...:
   /snap/racket/current/usr/share/racket/collects/syntax/readerr.rkt:15:2: -raise-read-error
   /snap/racket/current/usr/share/racket/pkgs/parser-tools-lib/parser-tools/yacc.rkt:356:16: parsing-loop
   body of "/home/davide/repos/sample-racket-language/sample-2.rkt"

What I would like to achieve is the same behavior of read for example

(define in (open-input-string "1 2"))
(read in)
(read in)
(read in)

Output is

1
2
#<eof>

Is this possible with the parser generator? Am I missing something?