Kubernetes: How to collect stdout/stderr logs using sidecar container

Question

There is a container which print logs to stdout/stderr, and I have no access to host machines, so can not use node log collector to collect and send them to a central logging system(ElasticSearch here), is there a way to use a sidecar container to do such thing?

Answer 1

You can use fluentbit as a sidecar to send logs to Elasticsearch. There is no process that you need to run in the host for this. Below is an example yaml.

apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: fluentbit-logging-sidecar
  spec:
    selector:
      matchLabels:
        app: fluentbit-logging-sidecar
    replicas: 1
    template:
      metadata:
        labels:
          app: fluentbit-logging-sidecar
        volumes:
          - name: shared-data
            emptyDir: {}
          - name: config-volume
            configMap:
              name: fb-agent-config
        containers:
          - name: sample-logging
            image: <image>
            volumeMounts:
            - name: shared-data
              mountPath: /app/logs
          - name: fb-sidecar
            image: fluent/fluent-bit  
            volumeMounts:
            - name: shared-data
              mountPath: /app/logs
            - name: config-volume
              mountPath: /fluent-bit/etc/fluent-bit.conf
              subPath: fluent-bit.conf

You need to configure your application to write log in filesystem in the path /app/logs. Because the app container and fluentbit container share the path using volumeMounts fluentbit sidecar will be able to read logs from that path and stream it to Elasticsearch. You need to configure details of elastic search in fluent-bit.conf file.