Sweet.js: Error: ASSERT: enforest assumes there are tokens to work with

Question

I was creating my macros and there is a bug than pops up (from my point of view) randomly:

ASSERT: enforest assumes there are tokens to work with

I'm not sure about what does this mean but I did my best to encapsulate the problem and I end with this code:

macro foo {
  rule { $bar:expr ? } => { $bar }
  rule { $bar:expr } => { $bar }
}

If I invoke foo macro between parentheses and add the ? at the end this error is thrown:

(foo test ?)
// produces ASSERT: enforest assumes there are tokens to work with

But if I remove the parentheses or the ? it just works

foo test ?
// produces: test
(foo test)
// produces: test

You can see it failing here:

http://sweetjs.org/browser/editor.html#macro%20foo%20%7B%0A%20%20rule%20%7B%20$bar:expr%20?%20%7D%20=%3E%20%7B%20$bar%20%7D%0A%20%20rule%20%7B%20$bar:expr%20%7D%20=%3E%20%7B%20$bar%20%7D%0A%7D%0A%0A(foo%20test%20?)%0A

So please, this is driving me crazy I'm facing this error again and again. It should be related to the :expr operator, but I can't replace it with ... because it's so greedy and it selects everything to the last ? it found.

I can see the line than throws this error is here. But I don't know the context.

Any help?

Answer 1

Thanks to @natefaubion I have a answer, it's a bug since it's interprets ? as a ternary operator and fails since it can't find the rest of the expression.

Also to my specific case a custom pattern class (instead of :expr) would be a better solution. The idea was proposed here: https://github.com/mozilla/sweet.js/issues/203.

For now a dirty workaround would be to use a case to parse the pattern manually: http://bit.ly/1aPmwyX

As @natefaubion said

its not really a good answer! It works (kind of), but 1) there's a bug with how ternary operators are enforested 2) what you are really wanting is extensible pattern classes.