Giters

ziprandom / cltk

A crystal port of the Ruby Language Toolkit

Home Page:https://ziprandom.github.io/cltk

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

CLTK::Scanner dislikes some alternations

hinrik opened this issue · comments

commented

Some patterns give this error:

cast from DFA::AST::LiteralNode to DFA::AST::AlternationNode failed, at /home/hinrik/src/fjo/lib/crystal-dfa/src/crystal-dfa/parser.cr:111:15:111 (TypeCastError)
0x555b67cd0843: consumeLastOrAlternative at /home/hinrik/src/fjo/lib/crystal-dfa/src/crystal-dfa/parser.cr 112:45
0x555b67cceb4c: parse at /home/hinrik/src/fjo/lib/crystal-dfa/src/crystal-dfa/parser.cr 41:13
0x555b67cce5b2: parse at /home/hinrik/src/fjo/lib/crystal-dfa/src/crystal-dfa/parser.cr 12:13
0x555b67cce57f: parse at /home/hinrik/src/fjo/lib/crystal-dfa/src/crystal-dfa/parser.cr 11:5
0x555b67cce52d: initialize at /home/hinrik/src/fjo/lib/crystal-dfa/src/crystal-dfa/regexp.cr 15:11
0x555b67cce501: new at /home/hinrik/src/fjo/lib/crystal-dfa/src/crystal-dfa/regexp.cr 12:5
0x555b67c5963e: __crystal_main at /home/hinrik/src/fjo/bin/test.cr 11:5
0x555b67c693d9: main at /opt/crystal/src/main.cr 12:15
0x7fb5f50612b1: __libc_start_main at ??
0x555b67c5277a: _start at ??
0x0: ??? at ??

Here's a minimal test case:

module CLTK
  alias TokenValue = (String|Int32)?
end

require "cltk"

module Fjo
  class Lexer < CLTK::Scanner
    extend CLTK::Scanner::LexerCompatibility
    rule(/(?:a)|b/) { { :FOO } } # works
    rule(/b|(?:a)/) { { :BAR } } # doesn't work
  end
end
commented

This seems like a parsing error probably due to the regex parser being poorly implemented. opening an issue on crystal-dfa