I have requirement to load data from one Redshift cluster to another, each in different region.

For same first creating table in target schema and then on source running UNLOAD command, which will put files in an S3 bucket.

To load data from S3 to target Redshift schema, I will use python code in AWS Lambda. The AWS Lambda function will trigger on the S3 path where all files will come when the unload command is executed.

After ingestion I need to run a transformation query that will put data in fact and dimension tables. How can I know the status for target table that a particular table loaded successfully?

The Lambda python code will triggered for each file, and will copy data to the target table. If I know the status I will issue the transformation query.

Is there any feature in AWS Lambda that will help me to know the status of the COPY command?

0 Answers