Fluentd NewRelic output "time must be a Fluent::EventTime (or Integer): Float"

691 views Asked by At

I'm encountering an issue that I don't understand, extract of config of td-agent:

<source>
  @type gelf
  tag gelf-nr
  <parse>
    @type json
  </parse>
  port 12201
</source>

<match gelf-nr**>
  @type copy
  <store>
    @type stdout
  </store>
  <store>
    @type newrelic
     api_key XXXXXXXXXXXXXXXXXXXXXXXXXX
  </store>
</match>

the output plugin newrelic is always complaining gelf failed to emit error="time must be a Fluent::EventTime (or Integer): Float" error_class="ArgumentError

I tried to edit the field timestamp of the json to nanosecond, but it's not this one which cause the issue. Extract of the message:

2020-09-22 10:00:18.135967969 +0200 gelf-nr: {"version":"1.0","host":"plop-01.stag-21","message":"TEST log plop","level":3,"facility":"plop.log","misc":{"reason":"test erreur","status_code":400,"quote_ids":[1234,5678]},"timestamp":"1600761476000000000","subject"...

I'm suspecting that the issue is the initial 2020-09-22 10:00:18.135967969 +0200. What do you think ? Could you help?

2

There are 2 answers

8
Max Lobur On BEST ANSWER

Get rid of nanoseconds, apparently bug in NR output plugin (outdated implementation) :

<source>
...
</source>

<filter **>
  @type record_modifier
  <record>
    timestamp "${timestamp.to_datetime.iso8601(6)}"
  </record>
</filter>

<match **>
...
</match>
1
Mickael Brangeon On

I got it working thanks to Max:

I used:

<filter **>
@type record_modifier
  <record>
  timestamp "${timestamp.to_datetime.iso8601(6)}"
  </record>
</filter>

And removed time parsing (time_Format, keep_time_key). But not time_key as the field in jsons was not time but timestamp. The issue seems to be in NR not in parser