Issue #59 ("Comment handling interferes with simple grammars") is back
fizbin opened this issue · comments
Daniel Martin commented
It appears that this bug was re-introduced in the 5.7 series. Values of None
for comments_re
and eol_comments_re
are no longer respected, even if used explicitly. This is a change in behavior from version 5.6.1.
Using the reproduction from that bug:
% python
Python 3.10.12 (main, Jul 28 2023, 18:34:01) [Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import tatsu
>>> tatsu.compile("start = '#' $;").parse("#", trace=True)
↙start ~1:1
#
≢'#'
≢start ~1:1
#
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/private/tmp/tatsutest/lib/python3.10/site-packages/tatsu/grammars.py", line 1075, in parse
return ctx.parse(text, start, config=config)
File "/private/tmp/tatsutest/lib/python3.10/site-packages/tatsu/contexts.py", line 240, in parse
raise self._furthest_exception
File "/private/tmp/tatsutest/lib/python3.10/site-packages/tatsu/contexts.py", line 233, in parse
result = rule()
File "/private/tmp/tatsutest/lib/python3.10/site-packages/tatsu/grammars.py", line 865, in parse
result = self._parse_rhs(ctx, self.exp)
File "/private/tmp/tatsutest/lib/python3.10/site-packages/tatsu/grammars.py", line 877, in _parse_rhs
result = ctx._call(ruleinfo)
File "/private/tmp/tatsutest/lib/python3.10/site-packages/tatsu/contexts.py", line 602, in _call
result = self._recursive_call(ruleinfo)
File "/private/tmp/tatsutest/lib/python3.10/site-packages/tatsu/contexts.py", line 630, in _recursive_call
return self._invoke_rule(ruleinfo, self.memokey)
File "/private/tmp/tatsutest/lib/python3.10/site-packages/tatsu/contexts.py", line 679, in _invoke_rule
ruleinfo.impl(self)
File "/private/tmp/tatsutest/lib/python3.10/site-packages/tatsu/grammars.py", line 443, in parse
ctx.last_node = [s.parse(ctx) for s in self.sequence]
File "/private/tmp/tatsutest/lib/python3.10/site-packages/tatsu/grammars.py", line 443, in <listcomp>
ctx.last_node = [s.parse(ctx) for s in self.sequence]
File "/private/tmp/tatsutest/lib/python3.10/site-packages/tatsu/grammars.py", line 326, in parse
return ctx._token(self.token)
File "/private/tmp/tatsutest/lib/python3.10/site-packages/tatsu/contexts.py", line 717, in _token
self._error(token, exclass=FailedToken)
File "/private/tmp/tatsutest/lib/python3.10/site-packages/tatsu/contexts.py", line 539, in _error
raise self._make_exception(item, exclass=exclass)
tatsu.exceptions.FailedToken: (1:1) expecting '#' :
#
^
start
>>> tatsu.compile("start = '#' $;").parse("#", trace=True, eol_comments_re=None)
↙start ~1:1
#
≢'#'
≢start ~1:1
#
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/private/tmp/tatsutest/lib/python3.10/site-packages/tatsu/grammars.py", line 1075, in parse
return ctx.parse(text, start, config=config)
File "/private/tmp/tatsutest/lib/python3.10/site-packages/tatsu/contexts.py", line 240, in parse
raise self._furthest_exception
File "/private/tmp/tatsutest/lib/python3.10/site-packages/tatsu/contexts.py", line 233, in parse
result = rule()
File "/private/tmp/tatsutest/lib/python3.10/site-packages/tatsu/grammars.py", line 865, in parse
result = self._parse_rhs(ctx, self.exp)
File "/private/tmp/tatsutest/lib/python3.10/site-packages/tatsu/grammars.py", line 877, in _parse_rhs
result = ctx._call(ruleinfo)
File "/private/tmp/tatsutest/lib/python3.10/site-packages/tatsu/contexts.py", line 602, in _call
result = self._recursive_call(ruleinfo)
File "/private/tmp/tatsutest/lib/python3.10/site-packages/tatsu/contexts.py", line 630, in _recursive_call
return self._invoke_rule(ruleinfo, self.memokey)
File "/private/tmp/tatsutest/lib/python3.10/site-packages/tatsu/contexts.py", line 679, in _invoke_rule
ruleinfo.impl(self)
File "/private/tmp/tatsutest/lib/python3.10/site-packages/tatsu/grammars.py", line 443, in parse
ctx.last_node = [s.parse(ctx) for s in self.sequence]
File "/private/tmp/tatsutest/lib/python3.10/site-packages/tatsu/grammars.py", line 443, in <listcomp>
ctx.last_node = [s.parse(ctx) for s in self.sequence]
File "/private/tmp/tatsutest/lib/python3.10/site-packages/tatsu/grammars.py", line 326, in parse
return ctx._token(self.token)
File "/private/tmp/tatsutest/lib/python3.10/site-packages/tatsu/contexts.py", line 717, in _token
self._error(token, exclass=FailedToken)
File "/private/tmp/tatsutest/lib/python3.10/site-packages/tatsu/contexts.py", line 539, in _error
raise self._make_exception(item, exclass=exclass)
tatsu.exceptions.FailedToken: (1:1) expecting '#' :
#
^
start
Juancarlo Añez commented
@fizbin, Have you checked against the version in the master
branch? I ask because I'm sure I applied a fix to this issue not too long ago.