I am working on passing
nginx access logs to
aws kinesis to
AWS S3 via kinesis firehose. nginx logs will be pushed to
AWS Glacier during
I am at the initial steps where I need need to pass nginx access logs to fluentd via syslog/rsyslog.
Nginx version used is 1.14.
My OS is :
NAME="Amazon Linux" VERSION="2" ID="amzn" ID_LIKE="centos rhel fedora" VERSION_ID="2" PRETTY_NAME="Amazon Linux 2" ANSI_COLOR="0;33" CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2"
There are many ways to push nginx logs to rsyslog.
Way 1 : Using module imfile , and updating rsyslog.conf as shown in https://petermolnar.net/log-nginx-to-rsyslog/
Way 2 : Updating nginx.conf to write directly in syslog. I am worried this will slow nginx down during high traffix. https://docs.nginx.com/nginx/admin-guide/monitoring/logging/#logging-to-syslog
Way 3: Skip rsyslog/syslog and directly pass data from nginx to fluentd via module https://github.com/fluent/nginx-fluentd-module . Here i am worried that this module is reliable or no and also if nginx should not become less responsive due to this module being used.
What is the best way to achieve the goal , the goal is for realtime data flow from nginx access logs to fluentd , without blocking nginx to serve requests.