Parsing dhcpd.conf with textX

Question

I'm using https://github.com/igordejanovic/textX to parse dhcpd.conf file (no, https://pypi.org/project/iscconf/ does not work for me, it crashes on my dhcpd.conf file), specifically extract hosts with fixed addresses.

Records are like:

    host example1 {
    option host-name "example1";
    ddns-hostname "example1";
    fixed-address 192.168.1.181;
    }

    host example2 {
    hardware ethernet aa:bb:ff:20:fa:13;
    fixed-address 192.168.1.191;
    option host-name "example2";
    ddns-hostname "example2";
    }

Code:

def get_hosts(s):
    grammar = """
    config: hosts*=host ;

    host: 'host' hostname=ID '{'
        (
            ('hardware ethernet' hardware_ethernet=/[0-9a-fA-F:]+/';')?

            'fixed-address' fixed_address=/([0-9]{1,3}\.){3}[0-9]{1,3}/';'

            ('option host-name' option_host_name=STRING';')?

            ('ddns-hostname' ddns_hostname=STRING';')?
        )#
    '}'
    ;
    """
    mm = metamodel_from_str(grammar)
    model = mm.model_from_str(s)
    for host in model.hosts:
        print host.hostname, host.fixed_address

Now, I cannot parse entire dhcpd.conf with this grammar (obviously, I get syntax errors since there are so many other elements in the file that grammar does not account for); on the other hand, I do not want to construct complete grammar for this file, as I need only extraction of specific type of host records.

I can certainly extract just the host records using regular expressions and parse them separately, but I wonder if there is some way to make textX extract only host records out of the file and ignore the rest of the content?

Answer 1

textX author here. I'm not a frequent visitor of SO :). You can play around with regex matches and regex lookaheads to consume unwanted content. Here is a complete example that correctly handles intermediate text even if there is keyword host. Rule config first consume a char if there is not a word host ahead, and this repeats due to zero or more operator. When we get a word host we try do match host rule one or more times and collect all host objects, if the rule didn't succeed at least one (notice the usage of +=) we consume word host and repeat the process. This can probably be done better (more performant) but you get the idea. When doing this kind of stuff it's good to know that textX by default consume whitespaces but you can turn this off either globaly or per-rule using noskipws (see the docs).

from textx import metamodel_from_str


def test_get_hosts():
    grammar = r"""
    config: ( /(?!host)./ | hosts+=host | 'host' )* ;

    host: 'host' hostname=ID '{'
        (
            ('hardware ethernet' hardware_ethernet=/[0-9a-fA-F:]+/';')?
            'fixed-address' fixed_address=/([0-9]{1,3}\.){3}[0-9]{1,3}/';'
            ('option host-name' option_host_name=STRING';')?
            ('ddns-hostname' ddns_hostname=STRING';')?
        )#
    '}'
    ;
    """
    conf_file = r"""
    host example1 {
    option host-name "example1";
    ddns-hostname "example1";
    fixed-address 192.168.1.181;
    }

    some arbitrary content in between
    with word host but that fails to match host config.

    host example2 {
    hardware ethernet aa:bb:ff:20:fa:13;
    fixed-address 192.168.1.191;
    option host-name "example2";
    ddns-hostname "example2";
    }
    """
    mm = metamodel_from_str(grammar)
    model = mm.model_from_str(conf_file)
    assert len(model.hosts) == 2
    for host in model.hosts:
        print(host.hostname, host.fixed_address)


if __name__ == "__main__":
    test_get_hosts()

Edit: Here are two more ideas for config rule: A simple one:

config: ( hosts+=host | /./ )* ;

And (probably) a more performant that consume as much as it can using regex engine before trying host:

config: ( /(?s:.*?(?=host))/ hosts*=host | 'host' )*
        /(?s).*/;