I'm implementing a data lake architecture on AWS, storing raw data in the bronze layer and transformed data in the silver layer. During the storage of data in the silver layer, I would like to append additional columns to hold metadata details such as "created by" and "last modified by." For AWS Glue jobs, I want to retrieve details about the user who triggered the job. I'm aware of CloudTrail's Lookup Events API, but I'm looking for an alternative approach to retrieve this information from the server-side without using a client library.
I utilized AWS Glue for ETL jobs and AWS CloudTrail for user actions tracking. I attempted to integrate CloudTrail to fetch details like "created by" and "last modified by" during data storage in the silver layer. My goal was to extract user information from CloudTrail based on Glue job events.