TechQA.

How to specify a self signed SSL CERT for td-agent/fluentd?

5k views Asked by At

I've continued to get the following error:

  2015-06-16 05:21:17 +0000 [warn]: temporarily failed to flush the buffer. next_retry=2015-06-16 05:21:17 +0000 error_class="Faraday::SSLError" error="Unable to verify certificate, please set `Excon.defaults[:ssl_ca_path] = path_to_certs`, `ENV['SSL_CERT_DIR'] = path_to_certs`, `Excon.defaults[:ssl_ca_file] = path_to_file`, `ENV['SSL_CERT_FILE'] = path_to_file`, `Excon.defaults[:ssl_verify_callback] = callback` (see OpenSSL::SSL::SSLContext#verify_callback), or `Excon.defaults[:ssl_verify_peer] = false` (less secure)." plugin_id="object:127283c"
  2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/ssl_socket.rb:129:in `connect'
  2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/ssl_socket.rb:129:in `initialize'
  2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/connection.rb:387:in `new'
  2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/connection.rb:387:in `socket'
  2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/connection.rb:106:in `request_call'
  2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/middlewares/mock.rb:47:in `request_call'
  2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/middlewares/instrumentor.rb:22:in `request_call'
  2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/middlewares/base.rb:15:in `request_call'
  2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/middlewares/base.rb:15:in `request_call'
  2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/middlewares/base.rb:15:in `request_call'
  2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/connection.rb:233:in `request'
  2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.1/lib/faraday/adapter/excon.rb:54:in `call'
  2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.1/lib/faraday/rack_builder.rb:139:in `build_response'
  2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.1/lib/faraday/connection.rb:377:in `run_request'
  2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/transport/http/faraday.rb:21:in `block in perform_request'
  2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/transport/base.rb:190:in `call'
  2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/transport/base.rb:190:in `perform_request'
  2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/transport/http/faraday.rb:20:in `perform_request'
  2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/client.rb:119:in `perform_request'
  2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-api-1.0.12/lib/elasticsearch/api/actions/ping.rb:20:in `block in ping'
  2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-api-1.0.12/lib/elasticsearch/api/utils.rb:189:in `__rescue_from_not_found'
  2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-api-1.0.12/lib/elasticsearch/api/actions/ping.rb:19:in `ping'
  2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-elasticsearch-0.9.0/lib/fluent/plugin/out_elasticsearch.rb:64:in `client'
  2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-elasticsearch-0.9.0/lib/fluent/plugin/out_elasticsearch.rb:171:in `rescue in send'
  2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-elasticsearch-0.9.0/lib/fluent/plugin/out_elasticsearch.rb:169:in `send'
  2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-elasticsearch-0.9.0/lib/fluent/plugin/out_elasticsearch.rb:163:in `write'
  2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/buffer.rb:325:in `write_chunk'
  2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/buffer.rb:304:in `pop'
  2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/output.rb:321:in `try_flush'
  2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/output.rb:140:in `run'
2015-06-16 05:21:18 +0000 [warn]: temporarily failed to flush the buffer. next_retry=2015-06-16 05:21:19 +0000 error_class="Faraday::SSLError" error="Unable to verify certificate, please set `Excon.defaults[:ssl_ca_path] = path_to_certs`, `ENV['SSL_CERT_DIR'] = path_to_certs`, `Excon.defaults[:ssl_ca_file] = path_to_file`, `ENV['SSL_CERT_FILE'] = path_to_file`, `Excon.defaults[:ssl_verify_callback] = callback` (see OpenSSL::SSL::SSLContext#verify_callback), or `Excon.defaults[:ssl_verify_peer] = false` (less secure)." plugin_id="object:127283c"
  2015-06-16 05:21:18 +0000 [warn]: suppressed same stacktrace
2015-06-16 05:21:20 +0000 [warn]: temporarily failed to flush the buffer. next_retry=2015-06-16 05:21:24 +0000 error_class="Faraday::SSLError" error="Unable to verify certificate, please set `Excon.defaults[:ssl_ca_path] = path_to_certs`, `ENV['SSL_CERT_DIR'] = path_to_certs`, `Excon.defaults[:ssl_ca_file] = path_to_file`, `ENV['SSL_CERT_FILE'] = path_to_file`, `Excon.defaults[:ssl_verify_callback] = callback` (see OpenSSL::SSL::SSLContext#verify_callback), or `Excon.defaults[:ssl_verify_peer] = false` (less secure)." plugin_id="object:127283c"
  2015-06-16 05:21:20 +0000 [warn]: suppressed same stacktrace
^C2015-06-16 05:21:22 +0000 [info]: shutting down fluentd
2015-06-16 05:21:23 +0000 [warn]: before_shutdown failed error="Unable to verify certificate, please set `Excon.defaults[:ssl_ca_path] = path_to_certs`, `ENV['SSL_CERT_DIR'] = path_to_certs`, `Excon.defaults[:ssl_ca_file] = path_to_file`, `ENV['SSL_CERT_FILE'] = path_to_file`, `Excon.defaults[:ssl_verify_callback] = callback` (see OpenSSL::SSL::SSLContext#verify_callback), or `Excon.defaults[:ssl_verify_peer] = false` (less secure)."
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/ssl_socket.rb:129:in `connect'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/ssl_socket.rb:129:in `initialize'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/connection.rb:387:in `new'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/connection.rb:387:in `socket'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/connection.rb:106:in `request_call'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/middlewares/mock.rb:47:in `request_call'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/middlewares/instrumentor.rb:22:in `request_call'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/middlewares/base.rb:15:in `request_call'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/middlewares/base.rb:15:in `request_call'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/middlewares/base.rb:15:in `request_call'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/connection.rb:233:in `request'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.1/lib/faraday/adapter/excon.rb:54:in `call'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.1/lib/faraday/rack_builder.rb:139:in `build_response'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.1/lib/faraday/connection.rb:377:in `run_request'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/transport/http/faraday.rb:21:in `block in perform_request'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/transport/base.rb:190:in `call'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/transport/base.rb:190:in `perform_request'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/transport/http/faraday.rb:20:in `perform_request'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/client.rb:119:in `perform_request'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-api-1.0.12/lib/elasticsearch/api/actions/ping.rb:20:in `block in ping'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-api-1.0.12/lib/elasticsearch/api/utils.rb:189:in `__rescue_from_not_found'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-api-1.0.12/lib/elasticsearch/api/actions/ping.rb:19:in `ping'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-elasticsearch-0.9.0/lib/fluent/plugin/out_elasticsearch.rb:64:in `client'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-elasticsearch-0.9.0/lib/fluent/plugin/out_elasticsearch.rb:171:in `rescue in send'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-elasticsearch-0.9.0/lib/fluent/plugin/out_elasticsearch.rb:169:in `send'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-elasticsearch-0.9.0/lib/fluent/plugin/out_elasticsearch.rb:163:in `write'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/buffer.rb:325:in `write_chunk'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/buffer.rb:304:in `pop'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/buf_memory.rb:93:in `block in before_shutdown'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/2.1.0/monitor.rb:211:in `mon_synchronize'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/buf_memory.rb:89:in `before_shutdown'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/output.rb:396:in `before_shutdown'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/output.rb:160:in `block in run'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/output.rb:159:in `synchronize'
  2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/output.rb:159:in `run'
2015-06-16 05:21:23 +0000 [info]: process finished code=0

I've tried setting the following environment variables:

  • setting SSL_CERT_FILE=/path/to/my/cert.crt
  • setting SSL_CERT_DIR=/path/to/my
  • setting SSL_CERT_DIR=/etc/ssl/certs
  • setting SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt

I've tried creating /home/td-agent/.bash_profile and updating the /home/td-agent/.bashrc files to export similar environment variables. When I run the td-agent service, I still get this error.

I've tried running the td-agent service at the command line with the previously mentioned env vars present.

I have run dpkg-reconfigure ca-certificates with my self signed cert the appropriate place. /usr/share/ca-certificates/<my cert dir>/my-self-signed-cert.crt

I've tried running the td-agent without either the SSL_CERT_DIR or SSL_CERT_FILE env vars set.

NB. I was able to curl https://my.elasticsearch.myltd and get a 200 response so I know the os (Ubuntu) has picked up my cert.

As for versions, I'm running ubuntu 14.04 LTS and when I start the td-agent, the following is printed to the log file /var/log/td-agent/td-agent.log:

2015-06-16 05:07:05 +0000 [info]: reading config file path="/etc/td-agent/td-agent.conf"
2015-06-16 05:07:05 +0000 [info]: starting fluentd-0.12.7
2015-06-16 05:07:06 +0000 [info]: gem 'fluent-mixin-config-placeholders' version '0.3.0'
2015-06-16 05:07:06 +0000 [info]: gem 'fluent-mixin-plaintextformatter' version '0.2.6'
2015-06-16 05:07:06 +0000 [info]: gem 'fluent-plugin-elasticsearch' version '0.9.0'
2015-06-16 05:07:06 +0000 [info]: gem 'fluent-plugin-mongo' version '0.7.8'
2015-06-16 05:07:06 +0000 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '1.4.1'
2015-06-16 05:07:06 +0000 [info]: gem 'fluent-plugin-s3' version '0.5.9'
2015-06-16 05:07:06 +0000 [info]: gem 'fluent-plugin-s3' version '0.5.7'
2015-06-16 05:07:06 +0000 [info]: gem 'fluent-plugin-scribe' version '0.10.14'
2015-06-16 05:07:06 +0000 [info]: gem 'fluent-plugin-td' version '0.10.26'
2015-06-16 05:07:06 +0000 [info]: gem 'fluent-plugin-td-monitoring' version '0.2.0'
2015-06-16 05:07:06 +0000 [info]: gem 'fluent-plugin-webhdfs' version '0.4.1'
2015-06-16 05:07:06 +0000 [info]: gem 'fluentd' version '0.12.7'
ruby ubuntu ssl-certificate self-signed fluentd
Original Q&A
1

There are 1 answers

1
Paul On

Found an answer here: https://groups.google.com/forum/#!topic/fluentd/z-1vIsQ4kHU

Turns out, to set the environment variables for the td-agent service, you need to export the variables in the file /etc/default/td-agent.

For example:

# This file is sourced by /bin/sh from /etc/init.d/td-agent
# Options to pass to td-agent
DAEMON_ARGS=""
export SSL_CERT_DIR=/etc/ssl/certs
export MYOTHERVAR=jazzjazzjazz

Related Questions in RUBY

Related Questions in UBUNTU

Related Questions in SSL-CERTIFICATE

Related Questions in SELF-SIGNED

Related Questions in FLUENTD

Popular Questions

Popular Tags

javascript python java c# php android html jquery c++ css ios sql mysql r reactjs node.js arrays c asp.net json python-3.x ruby-on-rails .net sql-server swift django angular objective-c pandas excel

Trending Questions