How to replace Telegraf's default timestamp?

Question

I use telegraf to send some data from a database to InfluxDB in regular intervals which works fine apart from one issue:

I need to replace telegraf's auto-generated timestamp (which is the current time at the moment of telegraf reading the data to transmit) with a field from the data.

(To answer the "why?" question: So the data I get in InfluxDB as a result actually matches the time of the event I want to record).

I would have thought there's some standard configuration parameter or an easy to find processor plugin, that let's me replace the default timestamp with the content of a field, but I didn't find any.

It does not seem to me to be a very exotic request and Telegraf's "Metric" does have a "SetTime" function, so I hope someone already solved that and can answer that.

Answer 1

Due to the fact that

• no one had an idea so far,
• that feature isn't available in all plugins coming with telegraf, and
• I have to solve this pretty urgently,

I wrote a telegraf processor plugin that does exactly what I needed.

(I will likely offer to contribute this to telegraf in the future, when I have a bit more time to breathe than right now.)

Answer 2

There exists a method to do this both for JSON format input and CSV input.

Here's a link to the JSON format description and how to set timestamps based on a value that's in the payload (to the format you can specify).

Answer 3

You can use a starlark processor to accomplish this, at least if you are able to put the correct timestamp into a tag first.

Here an example of how I use a starlark processor to replace the timestamp of the measurement with the content of a tag that I already populated with the correct timestamp in the input plugin:

[[processors.starlark]]
  source = '''
  def apply(metric):

      if "nanoseconds" in metric.tags:
        metric.time = int(metric.tags["nanoseconds"])
        return metric

  '''

Answer 4

use

[[processors.date]]
    tag_key = "name_of_column of your timestamp"
    date_format = {time format applicable in Go language}