fluentd is not able to connect aws elasticsearch from eks

Question

I have installed a aws-es-proxy helm chart. I am using https://github.com/kokuwaio/helm-charts/tree/main/charts/fluentd-elasticsearch repo for created fluentd. 2020-10-07 04:34:38 +0000 [error]: unexpected error error_class=NoMethodError error="undefined method []' for nil:NilClass" 2020-10-07 04:34:38 +0000 [error]: /usr/local/bundle/gems/fluent-plugin-elasticsearch-4.1.0/lib/fluent/plugin/out_elasticsearch.rb:464:in detect_es_major_version' 2020-10-07 04:34:38 +0000 [error]: /usr/local/bundle/gems/fluent-plugin-elasticsearch-4.1.0/lib/fluent/plugin/out_elasticsearch.rb:455:in block in handle_last_seen_es_major_version' 2020-10-07 04:34:38 +0000 [error]: /usr/local/bundle/gems/fluent-plugin-elasticsearch-4.1.0/lib/fluent/plugin/elasticsearch_index_template.rb:35:in retry_operate' 2020-10-07 04:34:38 +0000 [error]: /usr/local/bundle/gems/fluent-plugin-elasticsearch-4.1.0/lib/fluent/plugin/out_elasticsearch.rb:454:in handle_last_seen_es_major_version' 2020-10-07 04:34:38 +0000 [error]: /usr/local/bundle/gems/fluent-plugin-elasticsearch-4.1.0/lib/fluent/plugin/out_elasticsearch.rb:312:in configure' 2020-10-07 04:34:38 +0000 [error]: /usr/local/bundle/gems/fluentd-1.11.1/lib/fluent/plugin.rb:173:in configure' 2020-10-07 04:34:38 +0000 [error]: /usr/local/bundle/gems/fluentd-1.11.1/lib/fluent/agent.rb:132:in add_match' 2020-10-07 04:34:38 +0000 [error]: /usr/local/bundle/gems/fluentd-1.11.1/lib/fluent/agent.rb:74:in block in configure' 2020-10-07 04:34:38 +0000 [error]: /usr/local/bundle/gems/fluentd-1.11.1/lib/fluent/agent.rb:64:in each' 2020-10-07 04:34:38 +0000 [error]: /usr/local/bundle/gems/fluentd-1.11.1/lib/fluent/agent.rb:64:in configure' 2020-10-07 04:34:38 +0000 [error]: /usr/local/bundle/gems/fluentd-1.11.1/lib/fluent/label.rb:31:in configure' 2020-10-07 04:34:38 +0000 [error]: /usr/local/bundle/gems/fluentd-1.11.1/lib/fluent/root_agent.rb:143:in block in configure' 2020-10-07 04:34:38 +0000 [error]: /usr/local/bundle/gems/fluentd-1.11.1/lib/fluent/root_agent.rb:143:in each' 2020-10-07 04:34:38 +0000 [error]: /usr/local/bundle/gems/fluentd-1.11.1/lib/fluent/root_agent.rb:143:in configure' 2020-10-07 04:34:38 +0000 [error]: /usr/local/bundle/gems/fluentd-1.11.1/lib/fluent/engine.rb:105:in configure' 2020-10-07 04:34:38 +0000 [error]: /usr/local/bundle/gems/fluentd-1.11.1/lib/fluent/engine.rb:80:in run_configure' 2020-10-07 04:34:38 +0000 [error]: /usr/local/bundle/gems/fluentd-1.11.1/lib/fluent/supervisor.rb:602:in block in run_worker' 2020-10-07 04:34:38 +0000 [error]: /usr/local/bundle/gems/fluentd-1.11.1/lib/fluent/supervisor.rb:840:in main_process' 2020-10-07 04:34:38 +0000 [error]: /usr/local/bundle/gems/fluentd-1.11.1/lib/fluent/supervisor.rb:594:in run_worker' 2020-10-07 04:34:38 +0000 [error]: /usr/local/bundle/gems/fluentd-1.11.1/lib/fluent/command/fluentd.rb:361:in <top (required)>' 2020-10-07 04:34:38 +0000 [error]: /usr/local/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in require' 2020-10-07 04:34:38 +0000 [error]: /usr/local/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in require' 2020-10-07 04:34:38 +0000 [error]: /usr/local/bundle/gems/fluentd-1.11.1/bin/fluentd:8:in <top (required)>' 2020-10-07 04:34:38 +0000 [error]: /usr/local/bundle/bin/fluentd:23:in load' 2020-10-07 04:34:38 +0000 [error]: /usr/local/bundle/bin/fluentd:23:in 'enter code here

Answer 1

I advise you to do this workshop in particular this step: https://www.eksworkshop.com/intermediate/230_logging/ (EKS-logging fluetbit + elasticsearch + kibana)

I just tried this and found no problems.