Dynamically create parboiled2 rules

Question

Can I generate rules dynamically in parboiled2 parser? The use case is that I have a bunch of rules already defined, but want to add more and not compile every time I add a rule.

Answer 1

If you want to generate rules at runtime (like you did in Parboiled1) it's impossible. Parboiled 2 is using marco exressions, So you can not generate rules in the runtime. All stuff happens during the compilation phase.

If you have a number of defined rules, and want to compose them in arbitrary order, even with some of them missing. It's possible. I did it.

There are two known options, how you can achieve this:

The first option (I didn't try it yet) is called DynamicRuleDispatch you can find it in the documentation and look at it's test here.

The second option is to peform dispatch manually (As I did).

1. You should create a bunch of rules that can be combined dynamically. Those rules must have Rule0 type. They should not affect the value stack. Add side effect to those rules, after the rule evaluation. Side effect operation must save the captured data somewhere inside your parser's sate. You should use capture + ~ + run bundle to achieve this, look at the sample below:

   def RuleWithSideEffect = rule { capture(EmailAddress) ~ run { address: String => saveItSomewhere(address) } ~ EOI

2. You need to create some kind of mutable state inside your parser. A save place where you can save your parsing results. It can be hashmap. This Hashmap should store all possible rules and their values. You can't use values stack because you won't have determined number of matched rules. Be careful when maintaining mutable state. It must be cleaned after each call.

3. You need a format for rule alignment to align the rules. For example:

   class Parser(input: ParserInput, format: String) extends Parser....

Then you need to parse the format string and get the format tokens. Use pattern matching to dispatch a list of corresponding format tokens to the Rule0 fields dispatchRule method below. And then map the list of string tokens to rules.

When you have a list of rules you need to perform the following operation:

// concatenates two rules. Must be inside rule block
def concatRules(f: Rule0, s: Rule0): Rule0 = rule {
  f ~ s
}

val rootRule = 
    stringTokens map dispatchRule reduce concatRules

def RootRule: Rule0 = rule { mainRule ~ EOI }

def dispatchRule(token: String): Rule0 = match token {
   case "DATE" => DateParser.DateRule
   ....
}

You can parse generater access log by having logformat in your parser constructor. So, you need kind of data model to aligh rules.