When the logs appear to have the same timestamps, some of it appears out of order. Reading through some of the topics, I couldn't find a suitable solution. But I believe there must be something done about it since this issue was reported as early as Logstash 1.
Basically, I am using logstash to listen on a certain port for incoming tcp using the json lines codec [using the logstash-logback-encoder here]. For now, the filter is empty and I'm outputting the data to elasticsearch and stdout (codec rubydebug).
The console log in my IDE:
2017-Aug-30 15:15:30.191 [main] INFO com.sbsatter.logbackLogstash.App - Testing LOG Order;
Expected Order: 1 => 10
2017-Aug-30 15:15:30.193 [main] INFO com.sbsatter.logbackLogstash.App - 1
2017-Aug-30 15:15:30.194 [main] INFO com.sbsatter.logbackLogstash.App - 2
...
2017-Aug-30 15:15:30.195 [main] INFO com.sbsatter.logbackLogstash.App - 9
2017-Aug-30 15:15:30.195 [main] INFO com.sbsatter.logbackLogstash.App - 10
The terminal which launched logstash reads accordingly:
{
"@timestamp" => 2017-08-30T09:15:30.197Z,
"port" => 47820,
"@version" => "1",
"host" => "127.0.0.1",
"time" => "2017-08-30 15:15:30.191+0600",
"message" => "Testing LOG Order;\nExpected Order: 1 => 10"
}
{
"@timestamp" => 2017-08-30T09:15:30.198Z,
"port" => 47820,
"@version" => "1",
"host" => "127.0.0.1",
"time" => "2017-08-30 15:15:30.193+0600",
"message" => "1"
}
{
"@timestamp" => 2017-08-30T09:15:30.198Z,
"port" => 47820,
"@version" => "1",
"host" => "127.0.0.1",
"time" => "2017-08-30 15:15:30.194+0600",
"message" => "2"
}
.....
{
"@timestamp" => 2017-08-30T09:15:30.216Z,
"port" => 47820,
"@version" => "1",
"host" => "127.0.0.1",
"time" => "2017-08-30 15:15:30.195+0600",
"message" => "9"
}
{
"@timestamp" => 2017-08-30T09:15:30.224Z,
"port" => 47820,
"@version" => "1",
"host" => "127.0.0.1",
"time" => "2017-08-30 15:15:30.195+0600",
"message" => "10"
}
BUT, kibana shows the following :frowning: :
While the difference is not great in here, however, when used with real-time logs, changed order makes the log appear nonsense. How can I fix this issue?
Note, I have asked this question in the elasticsearch forum as well. I've dived through the documentation to find anything close to this but to no avail.