How to reliability pass a JSON object from API Gateway Into SQS Queue

Question

There does not seem to be much documentation on this, and most of the other questions don't appear to be as complete.

I would like to reliably transform a json object that hits my API Gateway endpoint into application/x-www-form-urlencoded so that I may pass it into an SQS queue.

So far I have tried every available option I could find online:

This method above produces the following body after transformation:

MessageBody=%7B%22event_type%22%3A%22click%22%2C%22click_time%22%3A%222023-10-17T00%3A55%3A04Z%22%2C%22user_agent%22%3A%22Mozilla%2F5.0%20(iPhone%3B%20CPU%20iPhone%20OS%2016_3_1%20like%20Mac%20OS%20X)%20AppleWebKit%2F605.1.15%20(KHTML%2C%20like%20Gecko)%20Version%2F16.3%20Mobile%2F15E148%20Safari%2F604.1%22%7D&Action=SendMessage

I also tried:

Which produced a result similar to the one above.

In both cases, the same error was thrown within api gateway: Received response. Status: 404, Integration latency

And a few lines further down:

Endpoint response body before transformations: <UnknownOperationException/>

I even tried to do some fancy mapping template stuff, that just didn't work (maybe you can spot the error(s) in this:

#set($inputJson = $input.body)

#set($urlEncodedParams = "")
#foreach($key in $inputJson.keySet())
  #set($value = $inputJson.get($key))
  #if($foreach.hasNext)
    #set($urlEncodedParams = "$urlEncodedParams${esc.url($key)}=${esc.url($value)}&")
  #else
    #set($urlEncodedParams = "$urlEncodedParams${esc.url($key)}=${esc.url($value)}")
  #end
#end

MessageBody=$urlEncodedParams&Action=SendMessage

Note: I currently have no content-type headers set, it is open as that endpoint will receive both urlencoded and json data, relying on the correct mapping template to pick it up.

Please contribute what you can, I feel like this could help many people over time if solved reliably.

Ethan

TLDR: I want to transform JSON data hitting my api gateway into a x-www-form-encoded so I can forward it to my SQS Queue.

Before (passed in):

{
  "event_type": "click",
  "click_time": "2023-10-16T22:39:59Z",
  "user_agent": "Mozilla/5.0 (iPhone; CPU iPhone OS 16_3_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.3 Mobile/15E148 Safari/604.1"
}

After (passed to SQS):

MessageBody=event_type%3Dclick%26click_time%3D2023-10-16T22%3A39%3A59Z%26user_agent%3DMozilla%2F5.0%20%28iPhone%3B%20CPU%20iPhone%20OS%2016_3_1%20like%20Mac%20OS%20X%29%20AppleWebKit%2F605.1.15%20%28KHTML%2C%20like%20Gecko%29%20Version%2F16.3%20Mobile%2F15E148%20Safari%2F604.1&Action=SendMessage

Update 2:

When I switch to a native integration like shown below, I get this error: {"Error":{"Code":"AWS.SimpleQueueService.NonExistentQueue","Message":"The specified queue does not exist for this wsdl version...

Checking cloudwatch, I see:

Sending request to https://sqs.us-east-1.amazonaws.com/?Action=SendMessage

Obviously, that's missing the path to the actual Queue, even though I include it in the request body after transformations:

{
    "MessageBody": "{\"event_type\":\"click\"}",
    "QueueUrl": "https://sqs.us-east-1.amazonaws.com/..."
}

Previously, I set action type to Use Path Override where I was able to put the Queue url directly like so:

{ACCOUNTID}/{QUEUE-PATH}

This worked, but not with the mapping templates below.

Answer 1

You don't need to use application/x-www-form-urlencoded to call SQS, JSON works fine.

1. Create a native AWS Service integration:
   1. SQS for Service
   2. Path override for Action (leave the path empty)
   3. Add two headers to the integration request (the single quotes matter):
      1. X-Amz-Target: 'AmazonSQS.SendMessage'
      2. Content-Type: 'application/x-amz-json-1.0'
2. Use this template for application/json:

   {
     "MessageBody": "$util.escapeJavaScript($input.json('$'))",
     "QueueUrl": "[queue-url]"
   }
   

This also serves as poor man's validation: if your input is not valid JSON, the endpoint will fail.

If you have proper validation in place, or if you don't care if the JSON is well-formed, you can simplify it a little bit:

{
  "MessageBody": "$util.escapeJavaScript($input.body)",
  "QueueUrl": "[queue-url]"
}