Trouble using python request module to make API post call within AWS lambda

Question

After connecting to the requisite AWS resources at the beginning of my lambda execution function, I have a lambda_handler function that looks like the following:

def lambda_handler(event, context, dst):

    bucket = event['Records'][0]['s3']['bucket']['name']
    key = urllib.unquote_plus(event['Records'][0]['s3']['object']['key'].encode('utf8'))
    print('Bucket: %s' % bucket)
    print('Object key: %s' % key)

    crm_file_name = key.split('/')[-1]
    crm_query = make_crm_db_query(crm_file_name)
    cur = conn.cursor()
    status = cur.execute(crm_query)

    if status == 1:
        details = cur.fetchone()
        opportunity_id = details[0]

        tmp = dst.get_key('%s/%s' % (opportunity_id, crm_file_name))
        print('starting API request...')
        s = requests.Session()
        r = s.post('http://link/to/endpoint/',\
            files={'pdf': tmp}, data={'opportunity_id': opportunity_id})
        print(r)
        print(r.content)
    else:
        print('not the right file type')

In my development environment this returns the following, indicating that the post was successful:

starting API request...
<Response [201]>
{"opportunity_id":253,"pdf":"https://s3.storage.asset.com:443/253/253___PDF.pdf?Signature=[CONFIDENTIAL STUFF HERE ;)]"}

In AWS Cloud Watch logs, however the process hang when attempting to execute the post request. Here is a log sample:

starting API request...
END RequestId: beedb0c4-ce07-11e6-a715-53b3bd8edccc
REPORT RequestId: beedb0c4-ce07-11e6-a715-53b3bd8edccc  Duration: 30002.89 ms   Billed Duration: 30000 ms Memory Size: 128 MB   Max Memory Used: 22 MB  
2016-12-29T20:46:24.356Z beedb0c4-ce07-11e6-a715-53b3bd8edccc Task timed out after 30.00 seconds 

The S3 bucket, API endpoint, and RDS all belong to the same VPC. The process works in dev but hangs in production. Any pointers on how to debug this?

I checked this post that says that connections to external internet resources require an NAT gateway, but our API endpoint is running on an EC2 instance within the same VPC. Does AWS think that we are still trying to establish an external connection because we are working with API calls? How do I debug this?

Answer 1

I encountered same timeout problem, the reason is below.

AWS document:

When you add VPC configuration to a Lambda function, it can only access resources in that VPC. If a Lambda function needs to access both VPC resources and the public Internet, the VPC needs to have a Network Address Translation (NAT) instance inside the VPC.

Mark B's comment is right.

I advice you can follow this blog to build NAT.