Logstash not writing to Elasticsearch with Shield

Question

I have been trying to make logstash write to elasticseach with shield without success.

My setup was working nromally before installing the shield plugin to elasticsearch. I've followed this guide from elastic.co and created a new user for the logstash user role using:

esusers useradd logstashadmin -r logstash

I've also updated the logstash output configuration and added the protocol, user, and password as suggested in the guide.

After restarting both logstash and elasticsearch, I am still not receiving anything on elasticsearch coming from logstash. Did I miss anything?

Here is my setup:

$ esusers roles logstashadmin                         
logstashadmin  : logstash

$ cat shield/roles.yml
...
# The required role for logstash users
logstash:
  cluster: indices:admin/template/get, indices:admin/template/put
  indices:
    'logstash-*': indices:data/write/bulk, indices:data/write/delete, indices:data/write/update, indices:data/read/search, indices:data/read/scroll, create_index
...


$ cat logstash/output.conf

output {
  elasticsearch {
    protocol => "http"
    cluster => "logstash"
    user => "logstashadmin"
    password => "logstashadmin123"
  }
}

Note: I've also installed the transport plugin in logstash and tried it instead of protocol => "http" with the same negative results.

Let me know if you need more info. Thank you

Edit 1:

Elastic search logs:

[2015-06-12 05:59:16,952][INFO ][node                     ] [Silver Sable] stopping ...
[2015-06-12 05:59:17,087][INFO ][shield.license           ] [Silver Sable] DISABLING LICENSE FOR [shield]
[2015-06-12 05:59:17,088][INFO ][node                     ] [Silver Sable] stopped
[2015-06-12 05:59:17,088][INFO ][node                     ] [Silver Sable] closing ...
[2015-06-12 05:59:17,104][INFO ][node                     ] [Silver Sable] closed
[2015-06-12 05:59:20,479][INFO ][node                     ] [Lionheart] version[1.4.5], pid[28662], build[2aaf797/2015-04-27T08:06:06Z]
[2015-06-12 05:59:20,480][INFO ][node                     ] [Lionheart] initializing ...
[2015-06-12 05:59:20,586][INFO ][plugins                  ] [Lionheart] loaded [license, shield], sites []
[2015-06-12 05:59:21,301][INFO ][transport                ] [Lionheart] Using [org.elasticsearch.shield.transport.ShieldServerTransportService] as transport service, overridden by [shield]
[2015-06-12 05:59:21,301][INFO ][transport                ] [Lionheart] Using [org.elasticsearch.shield.transport.netty.ShieldNettyTransport] as transport, overridden by [shield]
[2015-06-12 05:59:21,301][INFO ][http                     ] [Lionheart] Using [org.elasticsearch.shield.transport.netty.ShieldNettyHttpServerTransport] as http transport, overridden by [shield]
[2015-06-12 05:59:27,166][INFO ][node                     ] [Lionheart] initialized
[2015-06-12 05:59:27,166][INFO ][node                     ] [Lionheart] starting ...
[2015-06-12 05:59:28,148][INFO ][shield.transport         ] [Lionheart] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/10.1.0.25:9300]}
[2015-06-12 05:59:28,209][INFO ][discovery                ] [Lionheart] logstash/uuDCpM6hTKKvLNd2oFGYpA
[2015-06-12 05:59:32,032][INFO ][cluster.service          ] [Lionheart] new_master [Lionheart][uuDCpM6hTKKvLNd2oFGYpA][0ba2a1c6e1de][inet[/10.1.0.25:9300]], reason: zen-disco-join (elected_as_master)
[2015-06-12 05:59:32,119][INFO ][http                     ] [Lionheart] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/10.1.0.25:9200]}
[2015-06-12 05:59:32,119][INFO ][node                     ] [Lionheart] started
[2015-06-12 05:59:33,007][INFO ][shield.license           ] [Lionheart] enabling license for [shield]
[2015-06-12 05:59:33,013][INFO ][license.plugin.core      ] [Lionheart] license for [shield] - valid
[2015-06-12 05:59:33,028][ERROR][shield.license           ] [Lionheart] 
#
# Shield license will expire on [Saturday, July 11, 2015]. Cluster health, cluster stats and indices stats operations are
# blocked on Shield license expiration. All data operations (read and write) continue to work. If you
# have a new license, please update it. Otherwise, please reach out to your support contact.
#
[2015-06-12 05:59:33,162][INFO ][gateway                  ] [Lionheart] recovered [2] indices into cluster_state

Logstash logs: (this part gets duplicated many times)

NotImplementedError: block device detection unsupported or native support failed to load
       blockdev? at org/jruby/RubyFileTest.java:67
         device? at /opt/logstash/vendor/bundle/jruby/1.9/gems/filewatch-0.6.2/lib/filewatch/helper.rb:67
  _sincedb_write at /opt/logstash/vendor/bundle/jruby/1.9/gems/filewatch-0.6.2/lib/filewatch/tail.rb:230
   sincedb_write at /opt/logstash/vendor/bundle/jruby/1.9/gems/filewatch-0.6.2/lib/filewatch/tail.rb:203
        teardown at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-file-0.1.10/lib/logstash/inputs/file.rb:151
     inputworker at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.0-java/lib/logstash/pipeline.rb:203
     synchronize at org/jruby/ext/thread/Mutex.java:149
     inputworker at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.0-java/lib/logstash/pipeline.rb:203
     start_input at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.0-java/lib/logstash/pipeline.rb:171

Answer 1

I think this is a non shield related issue. Check this issue: https://github.com/elastic/logstash/issues/3127

Just like the post mentions, executing the following did the trick for me:

ln -s /lib/x86_64-linux-gnu/libcrypt.so.1 /usr/lib/x86_64-linux-gnu/libcrypt.so