Rsyslog lognormalizer date field parse failure

Question

I am trying to use lognorm/lognormalizer to test my .rb file to use with rsyslog mmnormalize module. My log file looks like this:

2017-08-19T17:00:12.52Z,john,26,engineer
2017-08-19T17:00:12.59Z,susan,28,doctor

My rb file is as follows:

version=2
rule=:%date:date-rfc3164%,%name:word%,%age:number%,%job:word%

When running lognormalizer:

head -2 /home/debian/olas/test.log | /usr/lib/x86_64-linux-gnu/lognorm/lognormalizer -r /home/debian/olas/rule.rb -e json

I get:

{ "originalmsg": "2017-08-19T17:00:12.52Z,john,26,engineer", "unparsed-data": "2017-08-19T17:00:12.52Z,john,26,engineer" }
{ "originalmsg": "2017-08-19T17:00:13.56Z,susan,28,doctor", "unparsed-data": "2017-08-19T17:00:13.56Z,susan,28,doctor" }

This means the rb script is not correct. Does anyone know what am i doing wrong? The date field I guess is not correctly configured, should I insert any other module? I cant find anything on the web. Thank you

Answer 1

You can use this rule:

version=2

rule=:%date:char-to{"extradata":","}%,%name:char-to{"extradata":","}%,%age:number{"format":"number"}%,%job:rest%

which produces the following output using Lognormalizer (pretty printed):

{
    "job": "engineer",
    "age": 26,
    "name": "john",
    "date": "2017-08-19T17:00:12.52Z"
},
{
    "job": "doctor",
    "age": 28,
    "name": "susan",
    "date": "2017-08-19T17:00:12.59Z"
}

Test command:

lognormalizer -P -H -r my.rule < mylog.log