Log to AWS CloudWatch Logs in Nodejs log middleware (ex: morgan, pino)

Question

morgan, pino is node.js logs middleware. I want to export the log to the AWS cloudWatchLogs.

For example

morgan for express:

var express = require('express');
var loggerM= require('morgan');
var app = express();
app.use(logger('dev'));

With this setting, you can see the log on the terminal. But it can only be displayed on the terminal, I don’t know how to get the log string, I have to get the log string to transfer it to the cloud.

I currently only find the stream method, which can only be directly saved to the local file:

const appLogStream = fs.createWriteStream(path.join(__dirname, 'app.log'), { flags: 'a' })
app.use(morgan('combined', { stream: appLogStream}));

AWS clouldWatchLogs putLogEvents api:

var params = {
  logEvents: [ /* required */
    {
      message: 'STRING_VALUE', /* required */
      timestamp: 'NUMBER_VALUE' /* required */
    },
    /* more items */
  ],
  logGroupName: 'STRING_VALUE', /* required */
  logStreamName: 'STRING_VALUE', /* required */
  sequenceToken: 'STRING_VALUE'
};
cloudwatchlogs.putLogEvents(params, function(err, data) {
  if (err) console.log(err, err.stack); // an error occurred
  else     console.log(data);           // successful response
});

Parameter message can be filled in log information.

Answer 1

Since morgan gives us the option to attach a log stream to the log its generating. We can use this for our purpose to write logs to cloudwatch logs. But how do we get a stream connected to aws cloudwatch?

Well we can do it like below:

//Create a logger using winston
var winston = require('winston'),
    CloudWatchTransport = require('winston-aws-cloudwatch');
const auth = require('./config/auth');

//add console transport to logger ---> logs to machine console
const logger = new winston.createLogger({
  transports: [
    new (winston.transports.Console)({
      timestamp: true,
      colorize: true,
    })
  ]
});

//below config creates a new log stream (snippet) in the same log gp specified at every new version of deployment
var config = {
  logGroupName: 'server-log-gp', //your aws cloudwatch log gp name
  logStreamName: `app.${new Date().getTime()}`, //log stream name
  createLogGroup: false,
  createLogStream: true, //creates new stream at deployment
  awsConfig: {
    accessKeyId: auth.aws.accessKeyId,
    secretAccessKey: auth.aws.secretAccessKey,
    region: auth.aws.region
  },
  formatLog: function (item) {
    return item.level + ': ' + item.message + ' ' + JSON.stringify(item.meta)
  }
}

//add cloudwatch logs transport to logger only in production mode
//may add in dev mode as well if you want
if (process.env.NODE_ENV === 'production') logger.add(new CloudWatchTransport(config));

logger.level = process.env.LOG_LEVEL || "silly";

logger.stream = {
  write: function(message, encoding) {
    logger.info(message);
  }
};

Now you need to add morgan as a middleware and attach the above logger.stream to it.

app.use(
  require('morgan')(
    'dev',
    { stream: logger.stream }
  )
);

See this for reference :

Node.js - logging / Use morgan and winston