I have a VPC, and inside the VPC I have two private subnets, I am using AWS DocumentDB which is also inside the VPC and is attached to the two private subnets mentioned above. I enabled Change Streams on DocumentDB, And created a dummy Lambda, with Event Source Mapping (ESM) trigger attached to it. The lambda logs the event received, which is the same event that caused the invocation of the Lambda.
I am using two VPC Interface Endpoints, one is for the Secrets Manager service, as we need that to talk to resources outside VPC, and since Secrets Manager is AWS Managed service, it expects a public endpoint. So I am okay with that.
But I cannot explain the need to have a second endpoint for Lambda Service, which helps invocation of the dummy Lambda when the trigger of DocumentDB change streams is met.
My security group rule allows all traffic for the inbound rule from the same VPC and the outbound rule allows all traffic to anywhere on the internet - 0.0.0.0/0
I tried removing the endpoint, and my other Lambdas for CRUD operations which are in a private subnet - they do not have any issue with the same, they are working as they should.
But the changes being done in DocDB aren't triggering the dummy Lambda which has an ESM trigger attached to it, it says VPC, STS, and Secrets Manager cannot talk to each other.
If I use VPC Endpoint for Lambda, I can log the events that caused the trigger for dummy Lambda. The usage of both endpoints is also recommended by AWS docs, whose link I will share:
AWS Tutorial: Using AWS Lambda with Amazon DocumentDB Streams
AWS Tutorial Lambda Event Source Mapping with Document DB - You should check the video at 37:55, this is the exact point where the instructor uses the same, but he has his Lambda in public subnet, so he cannot connect to Doc DB without using VPC Endpoint.
But since I have all my resources within the same VPC same subnet and the same security group attached to it, which allows all inbound and outbound rules, with no restrictions, then also I need the VPC endpoint for invoking the dummy Lambda, which has ESM trigger attached to change in DocumentDB Change Streams.