How to trigger functions in subnodes in Ruby Treetop tree. (was:How to prevent ruby Treetop doing AST squashing)

Question

How to trigger functions in subnodes in Ruby Treetop tree. (was:How to prevent ruby Treetop doing AST squashing)

93 views Asked by justrajdeep At 01 August 2018 at 18:04

I am been using treetop for a while. I wrote rules following

http://thingsaaronmade.com/blog/a-quick-intro-to-writing-a-parser-using-treetop.html

I can parse my whole input string but i none of the other to_array function gets triggered other than the initial one.

Then I found https://whitequark.org/blog/2011/09/08/treetop-typical-errors/ which talk about AST squashing and I figured out that my rule is doing the same.

The first rule I have is

  rule bodies
    blankLine* interesting:(body+) blankLine* <Bodies>
  end

and everything is getting gobbled up by body.

Can someone suggest me what can I do to fix this?

Edit Adding code snippet:

grammar Sexp

  rule bodies
    blankLine* interesting:(body+) blankLine* <Bodies>
  end

  rule body
    commentPortString (ifdef_blocks / interface)+ (blankLine / end_of_file) <Body>
  end

  rule interface
    space? (intf / intfWithSize) space?  newLine <Interface>
  end

  rule commentPortString
    space? '//' space portString space?  <CommentPortString>
  end

  rule portString
    'Port' space? '.' newLine <PortString>
  end

  rule expression
    space? '(' body ')' space? <Expression>
  end

  rule intf
    (input / output) space wire:wireName space? ';' <Intf>
  end

  rule intfWithSize
    (input / output) space? width:ifWidth space? wire:wireName space? ';' <IntfWithSize>
  end

  rule input
    'input' <InputDir>
  end

  rule output
    'output' <OutputDir>
  end

  rule ifdef_blocks
    ifdef_line (interface / ifdef_block)* endif_line <IfdefBlocks>
  end

  rule ifdef_block
    ifdef_line interface* endif_line <IfdefBlocks>
  end

  rule ifdef_line
    space? (ifdef / ifndef) space+  allCaps space? newLine <IfdefLine>
  end

  rule endif_line
    space? (endif) space? newLine <EndifLine>
  end

  rule ifdef
    '`ifdef' <Ifdef>
  end

  rule ifndef
    '`ifndef' <Ifndef>
  end

  rule endif
    '`endif' <Endif>
  end

  rule ifWidth
    '[' space? msb:digits space? ':' space? lsb:digits ']' <IfWidth>
  end

  rule digits
    [0-9]+ <Digits>
  end

  rule integer
    ('+' / '-')? [0-9]+ <IntegerLiteral>
  end

  rule float
    ('+' / '-')? [0-9]+ (('.' [0-9]+) / ('e' [0-9]+)) <FloatLiteral>
  end

  rule string
    '"' ('\"' / !'"' .)* '"' <StringLiteral>
  end

  rule identifier
    [a-zA-Z\=\*] [a-zA-Z0-9_\=\*]* <Identifier>
  end

  rule allCaps
    [A-Z] [A-Z0-9_]*
  end

  rule wireName
    [a-zA-Z] [a-zA-Z0-9_]* <WireName>
  end

  rule non_space
    !space .
  end

  rule space
    [^\S\n]+
  end

  rule non_space
    !space .
  end

  rule blankLine
    space* newLine
  end

  rule not_newLine
    !newLine .
  end

  rule newLine
    [\n\r]
  end

  rule end_of_file
    !.
  end

end

Test string

// Port.
input         CLK;

// Port.
input         REFCLK;

// Port.
input [ 41:0] mem_power_ctrl;
output data;

EDIT: Adding more details

The test code is checked into: https://github.com/justrajdeep/treetop_ruby_issue.

As you would see in my node_extensions.rb all the nodes except the Bodies raise an exception in the method to_array. But none of the exceptions trigger.

Original Q&A

There are 2 answers

Josh Voigts On 02 August 2018 at 18:09

Try breaking (body+) into a new rule like this:

rule bodies
   blankLine* interesting:interesting blankLine* <Bodies>
end

rule interesting
   body+ <Interesting>
end

Otherwise, it would be helpful to see the SyntaxNode classes.

**sepp2k** · Accepted Answer · 2018-08-02 21:16:05

You call to_array on tree, which is a Bodies. That is the only thing you ever call to_array on, so no other to_array method will be called.

If you want to_array to be called on child nodes of the Bodies node, Bodies#to_array needs to call to_array on those child nodes. So if you want to call it on the Body nodes you labelled interesting, you should iterate over interesting and call .to_array on each element.

TechQA.

How to trigger functions in subnodes in Ruby Treetop tree. (was:How to prevent ruby Treetop doing AST squashing)

There are 2 answers

Related Questions in RUBY

Related Questions in PARSING

Related Questions in TREETOP

Popular Questions

Popular Tags

Trending Questions