Telegraf inputs.tail with zimbra.log

Question

I have some questions, how I can set telegraf.conf file for collect logs from the "zimbra.conf" file? Now I tried to use this config text, but it does not work :((( I want to send this logs to grafana

One of the lines "zimbra.conf" for example:

Oct 1 10:20:46 webmail postfix/smtp[7677]: BD5BAE9999: [email protected], relay=mo94.cloud.mail.com[92.97.907.14]:25, delay=0.73, delays=0.09/0.01/0.58/0.19, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 4C25fk2pjFz32N5)

And I do not understand exactly how works the "grok_patterns ="

[[inputs.tail]]
  files = ["/var/log/zimbra.log"]
  from_beginning = false
  grok_patterns = ['%{SYSLOGTIMESTAMP:timestamp} %{SYSLOGHOST} %{DATA:program}(?:\[%{POSINT}\])?: %{GREEDYDATA:message}']
  name_override = "zimbra_access_log"
  grok_custom_pattern_files = []
  grok_custom_patterns = '''
  TS_UNIX %{MONTH}%{SPACE}%{MONTHDAY}%{SPACE}%{HOUR}:%{MINUTE}:%{SECOND}
  TS_CUSTOM %{MONTH}%{SPACE}%{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND}
  '''
  grok_timezone = "Local"
  data_format = "grok"

Answer 1

I have copied your example line into a log file called Prueba.txt wich contains the following lines:

Oct 3 00:52:32 webmail postfix/smtp[7677]: BD5BAE9999: [email protected], relay=mo94.cloud.mail.com[92.97.907.14]:25, delay=0.73, delays=0.09/0.01/0.58/0.19, dsn=2.0.0, status=sent (250 2.0$
Oct 13 06:25:01 webmail systemd-logind[949]: New session 229478 of user zimbra.
Oct 13 06:25:02 webmail zmconfigd[27437]: Shutting down. Received signal 15
Oct 13 06:25:02 webmail systemd-logind[949]: Removed session c296.
Oct 13 06:25:03 webmail sshd[28005]: Failed password for invalid user julianne from 120.131.2.210 port 10570 ssh2

I have been able to parse the data with this configuration of the tail.input plugin:

[[inputs.tail]]
  files = ["Prueba.txt"]
  from_beginning = true
  data_format = "grok"
  grok_patterns = ['%{TIMESTAMP_ZIMBRA} %{GREEDYDATA:source} %{DATA:program}(?:\[%{POSINT}\])?: %{GREEDYDATA:message}']


  grok_custom_patterns = '''
    TIMESTAMP_ZIMBRA (\w{3} \d{1,2} \d{2}:\d{2}:\d{2})
  '''

  name_override = "log_frames"

You need to match the input string with regular expressions. For that there are some predefined patters such as GREEDYDATA = .* that you can use to match your input (another example will be NUMBER = (?:%{BASE10NUM}) BASE16NUM (?<![0-9A-Fa-f])(?:[+-]?(?:0x)?(?:[0-9A-Fa-f]+))) . You can also define your own patterns in grok_custom_patterns. Take a look at this website with some patters: https://streamsets.com/documentation/datacollector/latest/help/datacollector/UserGuide/Apx-GrokPatterns/GrokPatterns_title.html

In this case I defined a TIMESTAMP_ZIMBRA pattern for matching Oct 3 00:52:32 and Oct 03 00:52:33 alike inputs.

Here is the collected metric by Prometheus:

# HELP log_frames_delay Telegraf collected metric
# TYPE log_frames_delay untyped
log_frames_delay{delays="0.09/0.01/0.58/0.19",dsn="2.0.0",host="localhost.localdomain",message="BD5BAE9999:",path="Prueba.txt",program="postfix/smtp",relay="mo94.cloud.mail.com[92.97.907.14]:25",source="webmail",status="sent (250 2.0.0 Ok: queued as 4C25fk2pjFz32N5)",to="[email protected]"} 0.73

P.D.: Ensure that telegraf has access to the log files.