NodeJS script works locally but not in the handler of lambda

Question

I have a NodeJS Lambda function that reads from a Kinesis stream, does some processing and writes it to another Kinesis stream. The writing part is causing an error on lambda. We use the aws-sdk npm module to write the data. When we call putRecord no callback is fired and lambda times out. This writing code works well when run locally as a node app on a laptop.

Local config: - The credentials of a programmatic user are with Kinesis Full Access policy are stored in the credentials file and the putRecord runs successfully returning the shardId, sequence number, etc.

Lambda function: - The code for putRecord does not return and it hangs. No errors are thrown either. Lambda has a role with the Kinesis Full Access policy.

Code:

var AWS = require('aws-sdk');
    var kinesis = new AWS.Kinesis({
        region: 'us-east-1',
    });
    var randomNumber = Math.floor(Math.random() * 100000);
            var data = 'data-' + randomNumber;
            var partitionKey = 'pk-' + randomNumber;
            var recordParams = {
                Data: data,
                PartitionKey: partitionKey,
                StreamName: streamName
            };
            kinesis.putRecord(recordParams, function(err, data) {
                console.log(data);

                if (err) {
                    console.error(err);
                }
            });

Any idea what could be causing the issue. VPC or security group related maybe? Thoughts and suggestions appereciated. Thanks.

Answer 1

When you add a function to a VPC it only has access to resources inside that VPC. Any attempt to access resources outside the VPC will hang and eventually timeout. Since Kinesis doesn't exist inside your VPC, you can't access it.

The fix is to either run the Lambda function outside the VPC, or add a NAT Gateway to your VPC.

Answer 2

If you have uploaded the exact Node.js script code above to Lambda, it will definitely not work.

Lamda requires you to export a handler function that it will call.

So, your script should be written like this if you want it to be a Lambda function...

'use strict';

var AWS = require('aws-sdk');
var kinesis = new AWS.Kinesis({
  region: 'us-east-1',
});


exports.handler = function (event, context, callback) {
  var randomNumber = Math.floor(Math.random() * 100000);
  var data = 'data-' + randomNumber;
  var partitionKey = 'pk-' + randomNumber;
  var recordParams = {
    Data: data,
    PartitionKey: partitionKey,
    StreamName: streamName,
  };

  kinesis.putRecord(recordParams, function (err, data) {
    callback(null, data);

    if (err) {
      callback(err);
    }
  });
};

Take note that instead of using console.log or console.error, you should call callback instead.