Giters

cjheath / treetop

A Ruby-based parsing DSL based on parsing expression grammars.

Home Page:https://cjheath.github.io/treetop

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

AST squashing issue in treetop

justrajdeep opened this issue · comments

commented

Hi

I have the following grammar

grammar Sexp

  rule bodies
    blankLine* interesting:(body+) blankLine* <Bodies>
  end

  rule body
    commentPortString (ifdef_blocks / interface)+ (blankLine / end_of_file) <Body>
  end

  rule interface
    space? (intf / intfWithSize) space?  newLine <Interface>
  end

  rule commentPortString
    space? '//' space portString space?  <CommentPortString>
  end

  rule portString
    'Port' space? '.' newLine <PortString>
  end

  rule expression
    space? '(' body ')' space? <Expression>
  end

  rule intf
    (input / output) space wire:wireName space? ';' <Intf>
  end

  rule intfWithSize
    (input / output) space? width:ifWidth space? wire:wireName space? ';' <IntfWithSize>
  end

  rule input
    'input' <InputDir>
  end

  rule output
    'output' <OutputDir>
  end

  rule ifdef_blocks
    ifdef_line (interface / ifdef_block)* endif_line <IfdefBlocks>
  end

  rule ifdef_block
    ifdef_line interface* endif_line <IfdefBlocks>
  end

  rule ifdef_line
    space? (ifdef / ifndef) space+  allCaps space? newLine <IfdefLine>
  end

  rule endif_line
    space? (endif) space? newLine <EndifLine>
  end

  rule ifdef
    '`ifdef' <Ifdef>
  end

  rule ifndef
    '`ifndef' <Ifndef>
  end

  rule endif
    '`endif' <Endif>
  end

  rule ifWidth
    '[' space? msb:digits space? ':' space? lsb:digits ']' <IfWidth>
  end

  rule digits
    [0-9]+ <Digits>
  end

  rule integer
    ('+' / '-')? [0-9]+ <IntegerLiteral>
  end

  rule float
    ('+' / '-')? [0-9]+ (('.' [0-9]+) / ('e' [0-9]+)) <FloatLiteral>
  end

  rule string
    '"' ('\"' / !'"' .)* '"' <StringLiteral>
  end

  rule identifier
    [a-zA-Z\=\*] [a-zA-Z0-9_\=\*]* <Identifier>
  end

  rule allCaps
    [A-Z] [A-Z0-9_]*
  end

  rule wireName
    [a-zA-Z] [a-zA-Z0-9_]* <WireName>
  end

  rule non_space
    !space .
  end

  rule space
    [^\S\n]+
  end

  rule non_space
    !space .
  end

  rule blankLine
    space* newLine
  end

  rule not_newLine
    !newLine .
  end

  rule newLine
    [\n\r]
  end

  rule end_of_file
    !.
  end

end

and the following text

// Port.
input         CLK;

// Port.
input         REFCLK;

// Port.
input [ 41:0] mem_power_ctrl;
output data;

But treetop ends up doing a AST squashing and all the nodes come into the body. AST squashing is mentioned here

I could not find any doc, pointers as to how to solve it. Can someone point me in the right direction.

Thanks in Advance

commented
commented

Hi @cjheath

Thanks for the detailed reply.

I have checked in the small example in
https://github.com/justrajdeep/treetop_ruby_issue

As you would see in my node_extensions.rb all the nodes except the Body raise an exception in the method to_array.

But none of them gets triggered. As per your comments

Any methods that might be in a block on “x” are added in a module. “x” methods can still call the same-named method on “a” or “b” using super; this is normal Ruby. In the above example the “x” method called “foo” overrides but also calls the “foo” method on “a” or “b”.

and even that was my understanding and I expected the exceptions to get triggered.
Am I missing something?

Regards
Rajdeep

commented

I'm travelling now, and short of time. Can't you make an example with 2 rules, instead of 27?
Or post to the proper support channel treetop-dev@googlegroups.com, since you've shown no evidence of a bug, yet. Someone else might have time to help you sooner than I do.

commented

Your top rule is "bodies". Bodies#to_array returns self. What makes you think that this should trigger an exception?

commented

Hi @cjheath
In your example if i do

 rule x (a / b) { def foo; “foo”; end } end

if i call tree.foo will that call the foo method in rules a / b ?

commented

hi @cjheath
I have trimmed the example. I am not expecting to_array of Bodies to raise exception but I am expecting PortString and CommentPortString to raise the exception when to_array gets triggered.

commented

You'r test code calls tree.to_array. The top tree node is a Bodies. Bodies#to_array does not return an exception. What part of this is difficult? Why won't you listen? Why won't you take your support question to the support forum? There is no bug here. You even went to StackOverflow - if I had my credential on my travel laptop I would downvote you there for not listening. I try to be helpful, as many will attest, but this is the end of this conversation.

commented

I finally understood what you meant. Sorry for wasting your time. My sincere apologies.

commented

Ok, mistakes happen. I just get riled when people think that the Internet is their debugger.

commented

Also, if you look at https://rubygems.org/gems/treetop or https://github.com/cjheath/treetop you'll see that the very first section says where to get support.
Please retract or adjust your StackOverflow post. Treetop does not do any such thing as "tree squashing". That was a misunderstanding by the blogger. A single node is just the node. You only get a sequence parent node when there's more than one item in the sequence. That's all, no magic.

commented

Finally, the first Google hit for "treetop documentation" yields the original page: http://cjheath.github.io/treetop/syntactic_recognition.html

commented

hi @cjheath
Updated the heading in Stack Overflow post.
I should have read through your description in github repo.
BTW the the google result might be location dependent. If i search ruby treetop, the first result is https://github.com/nathansobo/treetop, followed by http://thingsaaronmade.com/blog/a-quick-intro-to-writing-a-parser-using-treetop.html.

Anyways, now I know where to look for in case I am stuck. :)