Can you use environment variables in config file for fluentd

Question

I was wondering how to use env vars in the Fluentd config, I tried:

<match **>
type elasticsearch
logstash_format true
logstash_prefix $ENV_VAR
host ***
port ***
include_tag_key true
tag_key _key
</match>

but it doesn't work, any idea?

Answer 1

EDIT:

Here is a much better solution:

If you pass "--use-v1-config" option to Fluentd, this is possible with the "#{ENV['env_var_name']" like this:

<match foobar.**> # ENV["FOO"] is foobar
  type elasticsearch
  logstash_prefix "#{ENV['FOO']}"
  logstash_format true
  include_tag_key true
  tag_key _key
  host ****
  port ****
</match>

Old, kludgey answer is here.

1. Install fluent-plugin-record-reformer and fluent-plugin-forest
2. Update your config as follows.

<match hello.world>
  type record_reformer
  tag ${ENV["FOO"]}.${tag_prefix[-1]} # adding the env variable as a tag prefix
</match>

<match foobar.**> # ENV["FOO"] is foobar
  type forest
  subtype elasticsearch
  <template>
    logstash_prefix ${tag_parts[0]}
    logstash_format true
    include_tag_key true
    tag_key _key
    host ****
    port ****
  </template>
</match>

In particular, do NOT use <match **> there. That would catch all events and will lead to behaviors that are hard to debug.