I would like to setup my .NETCore microservice running via Dotnet/Tye project with local ELK docker.
I followed next guidelines: https://github.com/dotnet/tye/blob/main/docs/recipes/logging_elastic.md and set this to my tye.yaml:
extensions:
- name: elastic
logPath: ./.logs
And running this project with
tye run --watch --logs elastic=http://localhost:9200
For some reason I'm not getting any indexes in Kibana configuration.
Update 1
Attached Elasticsearch logs. Also I would like to say that since I'm using Mac M1 I needed to compile sebp/elk image for arm64 locally (according official docs).
chown: changing ownership of '/var/lib/elasticsearch/indices/Ii0QK5l8QLi_yzEVLBZyyg/_state': Permission denied
chown: changing ownership of '/var/lib/elasticsearch/indices/Ii0QK5l8QLi_yzEVLBZyyg/0/_state/retention-leases-0.st': Permission denied
chown: changing ownership of '/var/lib/elasticsearch/indices/Ii0QK5l8QLi_yzEVLBZyyg/0/_state/state-4.st': Permission denied
chown: changing ownership of '/var/lib/elasticsearch/indices/Ii0QK5l8QLi_yzEVLBZyyg/0/_state': Permission denied
chown: changing ownership of '/var/lib/elasticsearch/indices/Ii0QK5l8QLi_yzEVLBZyyg/0/translog/translog.ckp': Permission denied
chown: changing ownership of '/var/lib/elasticsearch/indices/Ii0QK5l8QLi_yzEVLBZyyg/0/translog/translog-6.tlog': Permission denied
chown: changing ownership of '/var/lib/elasticsearch/indices/Ii0QK5l8QLi_yzEVLBZyyg/0/translog': Permission denied
chown: changing ownership of '/var/lib/elasticsearch/indices/Ii0QK5l8QLi_yzEVLBZyyg/0/index/segments_2': Permission denied
chown: changing ownership of '/var/lib/elasticsearch/indices/Ii0QK5l8QLi_yzEVLBZyyg/0/index/write.lock': Permission denied
chown: changing ownership of '/var/lib/elasticsearch/indices/Ii0QK5l8QLi_yzEVLBZyyg/0/index': Permission denied
chown: changing ownership of '/var/lib/elasticsearch/indices/Ii0QK5l8QLi_yzEVLBZyyg/0': Permission denied
chown: changing ownership of '/var/lib/elasticsearch/indices/Ii0QK5l8QLi_yzEVLBZyyg': Permission denied
chown: changing ownership of '/var/lib/elasticsearch/indices/BpgSZOOjQEGvmc8CikJ7JQ/_state': Permission denied
chown: changing ownership of '/var/lib/elasticsearch/indices/BpgSZOOjQEGvmc8CikJ7JQ/0/_state': Permission denied
chown: changing ownership of '/var/lib/elasticsearch/indices/BpgSZOOjQEGvmc8CikJ7JQ/0/translog': Permission denied
chown: changing ownership of '/var/lib/elasticsearch/indices/BpgSZOOjQEGvmc8CikJ7JQ/0/index': Permission denied
chown: changing ownership of '/var/lib/elasticsearch/indices/BpgSZOOjQEGvmc8CikJ7JQ/0': Permission denied
chown: changing ownership of '/var/lib/elasticsearch/indices/BpgSZOOjQEGvmc8CikJ7JQ': Permission denied
chown: changing ownership of '/var/lib/elasticsearch/indices': Permission denied
chown: changing ownership of '/var/lib/elasticsearch/nodes': Permission denied
chown: changing ownership of '/var/lib/elasticsearch/snapshot_cache/segments_6': Permission denied
chown: changing ownership of '/var/lib/elasticsearch/snapshot_cache/write.lock': Permission denied
chown: changing ownership of '/var/lib/elasticsearch/snapshot_cache': Permission denied
chown: changing ownership of '/var/lib/elasticsearch': Permission denied
* Starting Elasticsearch Server
...done.
waiting for Elasticsearch to be up (1/30)
waiting for Elasticsearch to be up (2/30)
waiting for Elasticsearch to be up (3/30)
waiting for Elasticsearch to be up (4/30)
waiting for Elasticsearch to be up (5/30)
waiting for Elasticsearch to be up (6/30)
waiting for Elasticsearch to be up (7/30)
waiting for Elasticsearch to be up (8/30)
waiting for Elasticsearch to be up (9/30)
waiting for Elasticsearch to be up (10/30)
Waiting for Elasticsearch cluster to respond (1/30)
logstash started.
* Starting Kibana5
...done.
==> /var/log/elasticsearch/elasticsearch.log <==
[2022-08-13T20:02:23,121][INFO ][o.e.b.BootstrapChecks ] [elk] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2022-08-13T20:02:23,122][INFO ][o.e.c.c.Coordinator ] [elk] cluster UUID [BCwoAnKSQYqAe1XVWAkKQg]
[2022-08-13T20:02:23,243][INFO ][o.e.c.s.MasterService ] [elk] elected-as-master ([1] nodes joined)[{elk}{xL-uMsG2RD2KxDzGf8SGhw}{5WXUD0v3Txe1UtMW0ws07A}{192.168.208.2}{192.168.208.2:9300}{cdfhilmrstw} completing election, _BECOME_MASTER_TASK_, _FINISH_ELECTION_], term: 7, version: 169, delta: master node changed {previous [], current [{elk}{xL-uMsG2RD2KxDzGf8SGhw}{5WXUD0v3Txe1UtMW0ws07A}{192.168.208.2}{192.168.208.2:9300}{cdfhilmrstw}]}
[2022-08-13T20:02:23,352][INFO ][o.e.c.s.ClusterApplierService] [elk] master node changed {previous [], current [{elk}{xL-uMsG2RD2KxDzGf8SGhw}{5WXUD0v3Txe1UtMW0ws07A}{192.168.208.2}{192.168.208.2:9300}{cdfhilmrstw}]}, term: 7, version: 169, reason: Publication{term=7, version=169}
[2022-08-13T20:02:23,385][INFO ][o.e.h.AbstractHttpServerTransport] [elk] publish_address {192.168.208.2:9200}, bound_addresses {0.0.0.0:9200}
[2022-08-13T20:02:23,385][INFO ][o.e.n.Node ] [elk] started
[2022-08-13T20:02:23,566][WARN ][o.e.x.s.i.SetSecurityUserProcessor] [elk] Creating processor [set_security_user] (tag [null]) on field [_security] but authentication is not currently enabled on this cluster - this processor is likely to fail at runtime if it is used
[2022-08-13T20:02:23,683][INFO ][o.e.l.LicenseService ] [elk] license [2cc9ac6a-c421-4021-aa2d-daa12f2e2d0a] mode [basic] - valid
[2022-08-13T20:02:23,685][INFO ][o.e.g.GatewayService ] [elk] recovered [8] indices into cluster_state
[2022-08-13T20:02:25,510][INFO ][o.e.c.r.a.AllocationService] [elk] current.health="GREEN" message="Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[.geoip_databases][0]]])." previous.health="RED" reason="shards started [[.geoip_databases][0]]"
==> /var/log/logstash/logstash-plain.log <==
==> /var/log/kibana/kibana5.log <==
==> /var/log/elasticsearch/elasticsearch.log <==
[2022-08-13T20:02:25,940][INFO ][o.e.i.g.DatabaseNodeService] [elk] successfully loaded geoip database file [GeoLite2-Country.mmdb]
[2022-08-13T20:02:26,012][INFO ][o.e.i.g.DatabaseNodeService] [elk] successfully loaded geoip database file [GeoLite2-ASN.mmdb]
[2022-08-13T20:02:26,934][INFO ][o.e.i.g.DatabaseNodeService] [elk] successfully loaded geoip database file [GeoLite2-City.mmdb]
==> /var/log/logstash/logstash-plain.log <==
[2022-08-13T20:02:39,552][INFO ][logstash.runner ] Log4j configuration path used is: /opt/logstash/config/log4j2.properties
[2022-08-13T20:02:39,561][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"8.1.0", "jruby.version"=>"jruby 9.2.20.1 (2.5.8) 2021-11-30 2a2962fbd1 OpenJDK 64-Bit Server VM 11.0.13+8 on 11.0.13+8 +indy +jit [linux-aarch64]"}
[2022-08-13T20:02:39,562][INFO ][logstash.runner ] JVM bootstrap flags: [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -Djruby.regexp.interruptible=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED, -Djava.io.tmpdir=/opt/logstash]
[2022-08-13T20:02:39,577][INFO ][logstash.settings ] Creating directory {:setting=>"path.queue", :path=>"/opt/logstash/data/queue"}
[2022-08-13T20:02:39,584][INFO ][logstash.settings ] Creating directory {:setting=>"path.dead_letter_queue", :path=>"/opt/logstash/data/dead_letter_queue"}
[2022-08-13T20:02:39,813][INFO ][logstash.agent ] No persistent UUID file found. Generating new UUID {:uuid=>"7710d596-7725-4b7c-b1c6-371f4360a636", :path=>"/opt/logstash/data/uuid"}
==> /var/log/elasticsearch/elasticsearch.log <==
[2022-08-13T20:02:41,002][INFO ][o.e.t.LoggingTaskListener] [elk] 190 finished with response BulkByScrollResponse[took=331.9ms,timed_out=false,sliceId=null,updated=19,created=0,deleted=0,batches=1,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[2022-08-13T20:02:41,034][INFO ][o.e.t.LoggingTaskListener] [elk] 184 finished with response BulkByScrollResponse[took=462.3ms,timed_out=false,sliceId=null,updated=11,created=0,deleted=0,batches=1,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
==> /var/log/logstash/logstash-plain.log <==
[2022-08-13T20:02:41,833][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[2022-08-13T20:02:43,307][INFO ][org.reflections.Reflections] Reflections took 181 ms to scan 1 urls, producing 120 keys and 417 values
[2022-08-13T20:02:43,821][INFO ][logstash.javapipeline ] Pipeline `main` is configured with `pipeline.ecs_compatibility: v8` setting. All plugins in this pipeline will default to `ecs_compatibility => v8` unless explicitly configured otherwise.
[2022-08-13T20:02:43,855][INFO ][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost"]}
[2022-08-13T20:02:44,105][INFO ][logstash.outputs.elasticsearch][main] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}}
[2022-08-13T20:02:44,281][WARN ][logstash.outputs.elasticsearch][main] Restored connection to ES instance {:url=>"http://localhost:9200/"}
[2022-08-13T20:02:44,292][INFO ][logstash.outputs.elasticsearch][main] Elasticsearch version determined (8.1.0) {:es_version=>8}
[2022-08-13T20:02:44,294][WARN ][logstash.outputs.elasticsearch][main] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>8}
[2022-08-13T20:02:44,373][INFO ][logstash.outputs.elasticsearch][main] Config is not compliant with data streams. `data_stream => auto` resolved to `false`
[2022-08-13T20:02:44,381][INFO ][logstash.outputs.elasticsearch][main] Config is not compliant with data streams. `data_stream => auto` resolved to `false`
[2022-08-13T20:02:44,382][WARN ][logstash.outputs.elasticsearch][main] Elasticsearch Output configured with `ecs_compatibility => v8`, which resolved to an UNRELEASED preview of version 8.0.0 of the Elastic Common Schema. Once ECS v8 and an updated release of this plugin are publicly available, you will need to update this plugin to resolve this warning.
[2022-08-13T20:02:44,437][WARN ][logstash.filters.grok ][main] ECS v8 support is a preview of the unreleased ECS v8, and uses the v1 patterns. When Version 8 of the Elastic Common Schema becomes available, this plugin will need to be updated
[2022-08-13T20:02:44,530][WARN ][logstash.filters.grok ][main] ECS v8 support is a preview of the unreleased ECS v8, and uses the v1 patterns. When Version 8 of the Elastic Common Schema becomes available, this plugin will need to be updated
[2022-08-13T20:02:44,594][INFO ][logstash.javapipeline ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, "pipeline.sources"=>["/etc/logstash/conf.d/02-beats-input.conf", "/etc/logstash/conf.d/10-syslog.conf", "/etc/logstash/conf.d/11-nginx.conf", "/etc/logstash/conf.d/30-output.conf"], :thread=>"#<Thread:0x2926b0a run>"}
[2022-08-13T20:02:45,171][INFO ][logstash.javapipeline ][main] Pipeline Java execution initialization time {"seconds"=>0.57}
[2022-08-13T20:02:45,184][INFO ][logstash.inputs.beats ][main] Starting input listener {:address=>"0.0.0.0:5044"}
[2022-08-13T20:02:45,213][INFO ][logstash.javapipeline ][main] Pipeline started {"pipeline.id"=>"main"}
[2022-08-13T20:02:45,313][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2022-08-13T20:02:45,315][INFO ][org.logstash.beats.Server][main][829cd21b7fbde9c57f6074e54675a6dd14081ec403bdd5ea935fd37106249341] Starting server on port: 5044
==> /var/log/elasticsearch/elasticsearch.log <==
[2022-08-13T20:02:52,829][INFO ][o.e.c.m.MetadataMappingService] [elk] [.kibana_8.1.0_001/67wIFep3T4qg_Cn32rVbgg] update_mapping [_doc]
[2022-08-13T20:02:54,059][INFO ][o.e.c.m.MetadataMappingService] [elk] [.kibana_8.1.0_001/67wIFep3T4qg_Cn32rVbgg] update_mapping [_doc]
[2022-08-13T20:03:54,361][WARN ][o.e.x.s.i.SetSecurityUserProcessor] [elk] Creating processor [set_security_user] (tag [null]) on field [_security] but authentication is not currently enabled on this cluster - this processor is likely to fail at runtime if it is used
[2022-08-13T20:03:54,380][INFO ][o.e.c.m.MetadataMappingService] [elk] [.kibana_8.1.0_001/67wIFep3T4qg_Cn32rVbgg] update_mapping [_doc]
[2022-08-13T20:05:09,475][INFO ][o.e.c.m.MetadataMappingService] [elk] [.kibana_8.1.0_001/67wIFep3T4qg_Cn32rVbgg] update_mapping [_doc]