Logs appear out of order when burst logging is done

553 views Asked by At

When the logs appear to have the same timestamps, some of it appears out of order. Reading through some of the topics, I couldn't find a suitable solution. But I believe there must be something done about it since this issue was reported as early as Logstash 1.

Basically, I am using logstash to listen on a certain port for incoming tcp using the json lines codec [using the logstash-logback-encoder here]. For now, the filter is empty and I'm outputting the data to elasticsearch and stdout (codec rubydebug).

The console log in my IDE:

2017-Aug-30 15:15:30.191 [main] INFO  com.sbsatter.logbackLogstash.App - Testing LOG Order;
Expected Order: 1 => 10 
2017-Aug-30 15:15:30.193 [main] INFO  com.sbsatter.logbackLogstash.App - 1 
2017-Aug-30 15:15:30.194 [main] INFO  com.sbsatter.logbackLogstash.App - 2 
...
2017-Aug-30 15:15:30.195 [main] INFO  com.sbsatter.logbackLogstash.App - 9 
2017-Aug-30 15:15:30.195 [main] INFO  com.sbsatter.logbackLogstash.App - 10 

The terminal which launched logstash reads accordingly:

{
    "@timestamp" => 2017-08-30T09:15:30.197Z,
          "port" => 47820,
      "@version" => "1",
          "host" => "127.0.0.1",
          "time" => "2017-08-30 15:15:30.191+0600",
       "message" => "Testing LOG Order;\nExpected Order: 1 => 10"
}
{
    "@timestamp" => 2017-08-30T09:15:30.198Z,
          "port" => 47820,
      "@version" => "1",
          "host" => "127.0.0.1",
          "time" => "2017-08-30 15:15:30.193+0600",
       "message" => "1"
}
{
    "@timestamp" => 2017-08-30T09:15:30.198Z,
          "port" => 47820,
      "@version" => "1",
          "host" => "127.0.0.1",
          "time" => "2017-08-30 15:15:30.194+0600",
       "message" => "2"
}
.....
{
    "@timestamp" => 2017-08-30T09:15:30.216Z,
          "port" => 47820,
      "@version" => "1",
          "host" => "127.0.0.1",
          "time" => "2017-08-30 15:15:30.195+0600",
       "message" => "9"
}
{
    "@timestamp" => 2017-08-30T09:15:30.224Z,
          "port" => 47820,
      "@version" => "1",
          "host" => "127.0.0.1",
          "time" => "2017-08-30 15:15:30.195+0600",
       "message" => "10"
}

BUT, kibana shows the following :frowning: : result in kibana

While the difference is not great in here, however, when used with real-time logs, changed order makes the log appear nonsense. How can I fix this issue?

Note, I have asked this question in the elasticsearch forum as well. I've dived through the documentation to find anything close to this but to no avail.

0

There are 0 answers