How to print Spring Boot log in both text and json format using logback?

Question

For logging we are using Logback. Currently we are using Splunk for viewing the Spring Boot log. As per the project requirement we need to move towards Kibana. For that first we should have log in JSON format so that Kibana can easily process it. Currently we are not replacing Splunk with Kibana but when integration successful with Kibana then gradually we can replace Splunk in all the projects.

Hence as per the requirement we need existing log i.e. in text log but for R&D purpose we need JSON log as well.

Answer 1

This is very simple. You just need to add logstash-logback-encoder dependency in pom.xml file and write some configuration in your logback.xml file. Please follow the steps below to fulfill your requirement.

Step1: Add logstash-logback-encoder dependency in pom.xml.

<dependency>
    <groupId>net.logstash.logback</groupId>
    <artifactId>logstash-logback-encoder</artifactId>
    <version>6.4</version>
</dependency>

Step2: Create logback.xml file inside src/main/resources/ and copy-paste the below content.

<?xml version="1.0" encoding="UTF-8"?>
<configuration debug="false">
  <include resource="org/springframework/boot/logging/logback/defaults.xml" />
  <include resource="org/springframework/boot/logging/logback/console-appender.xml" />
  <property name="LOG_PATH" value="logs" />
  <property name="CONSOLE_LOG_PATTERN" value="[%-5level] %d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %X{id} %c{1} - %msg%n" />
  <property name="FILE_LOG_PATTERN" value="[%-5level] %d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %X{id} %c{1} - %msg%n" />

  <appender name="CONSOLE_APPENDER" class="ch.qos.logback.core.ConsoleAppender">
    <encoder>
      <pattern>${CONSOLE_LOG_PATTERN}</pattern>
      <charset>utf8</charset>
    </encoder>
  </appender>

  <appender name="ROLLING_TEXT_FILE_APPENDER" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>${LOG_PATH}/application-text.log</file>
    <encoder>
      <Pattern>${FILE_LOG_PATTERN}</Pattern>
    </encoder>
    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
      <fileNamePattern>${LOG_PATH}/application-text.%d{yyyy-MM-dd}.%i.gz
      </fileNamePattern>
      <timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
        <maxFileSize>100MB</maxFileSize>
      </timeBasedFileNamingAndTriggeringPolicy>
      <maxHistory>10</maxHistory>
    </rollingPolicy>
  </appender>
  
  <appender name="ROLLING_JSON_FILE_APPENDER" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>${LOG_PATH}/application-json.log</file>
    <encoder class="net.logstash.logback.encoder.LogstashEncoder" />
    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
      <fileNamePattern>${LOG_PATH}/application-json.%d{yyyy-MM-dd}.%i.gz
      </fileNamePattern>
      <timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
        <maxFileSize>100MB</maxFileSize>
      </timeBasedFileNamingAndTriggeringPolicy>
      <maxHistory>10</maxHistory>
    </rollingPolicy>
  </appender>

  <root level="INFO">
    <appender-ref ref="CONSOLE_APPENDER" />
    <appender-ref ref="ROLLING_TEXT_FILE_APPENDER" />
    <appender-ref ref="ROLLING_JSON_FILE_APPENDER" />
  </root>
</configuration>

Step3: Now you can use log statement anywhere in the class. Let's say you are using in the main class of Spring Boot.

import org.slf4j.MDC;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import com.altafjava.constant.AppConstant;
import lombok.extern.slf4j.Slf4j;

@Slf4j
@SpringBootApplication
public class JsonLogApplication {

    public static void main(String[] args) {
        SpringApplication.run(JsonLogApplication.class, args);
        MDC.put("id", AppConstant.PROJECT_NAME_PREFIX + "_" + AppConstant.MARKET_CODE + "_" + AppConstant.PARAM_TYPE);
        log.info("---------- Spring Application started successfully ----------");
        log.debug("This is a debug message.");
        log.info("This is an info message.");
        log.warn("This is a warn message.");
        log.error("This is an error message.");
        log.info("---------- Spring Application ended successfully ----------");
    }
}

This will create two log files inside the logs directory.

1. application-text.log will look like this

[INFO ] 2020-09-11 22:36:13.627 [main] DEMO_IND_PARAM c.a.JsonLogApplication - ---------- Spring Application started successfully ----------
[INFO ] 2020-09-11 22:36:13.630 [main] DEMO_IND_PARAM c.a.JsonLogApplication - This is an info message.
[WARN ] 2020-09-11 22:36:13.630 [main] DEMO_IND_PARAM c.a.JsonLogApplication - This is a warn message.
[ERROR] 2020-09-11 22:36:13.631 [main] DEMO_IND_PARAM c.a.JsonLogApplication - This is an error message.
[INFO ] 2020-09-11 22:36:13.631 [main] DEMO_IND_PARAM c.a.JsonLogApplication - ---------- Spring Application ended successfully ----------

2. application-json.log will look like this

{"@timestamp":"2020-09-11T22:36:13.627+05:30","@version":"1","message":"---------- Spring Application started successfully ----------","logger_name":"com.altafjava.JsonLogApplication","thread_name":"main","level":"INFO","level_value":20000,"id":"DEMO_IND_PARAM"}
{"@timestamp":"2020-09-11T22:36:13.630+05:30","@version":"1","message":"This is an info message.","logger_name":"com.altafjava.JsonLogApplication","thread_name":"main","level":"INFO","level_value":20000,"id":"DEMO_IND_PARAM"}
{"@timestamp":"2020-09-11T22:36:13.630+05:30","@version":"1","message":"This is a warn message.","logger_name":"com.altafjava.JsonLogApplication","thread_name":"main","level":"WARN","level_value":30000,"id":"DEMO_IND_PARAM"}
{"@timestamp":"2020-09-11T22:36:13.631+05:30","@version":"1","message":"This is an error message.","logger_name":"com.altafjava.JsonLogApplication","thread_name":"main","level":"ERROR","level_value":40000,"id":"DEMO_IND_PARAM"}
{"@timestamp":"2020-09-11T22:36:13.631+05:30","@version":"1","message":"---------- Spring Application ended successfully ----------","logger_name":"com.altafjava.JsonLogApplication","thread_name":"main","level":"INFO","level_value":20000,"id":"DEMO_IND_PARAM"}

Note: Here @Slf4j is a Lombok annotation. This means without creating LoggerFactory you can directly use log methods. Fo that you need to install Lombok in your IDE & add a dependency in pom.xml

If you need a demo project, please visit https://github.com/altafjava/json-log

Answer 2

You can configure in logback.xml as like below snippet for both json and text format.

    <appender name="json" class="ch.qos.logback.core.ConsoleAppender">
    <layout class="ch.qos.logback.contrib.json.classic.JsonLayout">
        <jsonFormatter
            class="ch.qos.logback.contrib.jackson.JacksonJsonFormatter">
            <prettyPrint>true</prettyPrint>
        </jsonFormatter>
        <timestampFormat>yyyy-MM-dd' 'HH:mm:ss.SSS</timestampFormat>
    </layout>
</appender>




<appender name="FILE" class="ch.qos.logback.core.FileAppender">
        <file>logfile.log</file>
        <append>true</append>
        <encoder>
            <pattern>%-4relative [%thread] %-5level %logger{35} - %msg %n</pattern>
        </encoder>
    </appender>

just change the class here iam printing one to an consoleAppender and another to a file you can point both to a consoleappender as well.