DynamoDB point-in-time-recovery files to parquet

204 views Asked by At

DynamoDB point in time recovery export option under "Export and streams" seems to be dumping the file in json.gz file format when selected with "DynamoDB JSON" under advanced settings. When I am trying to convert that file (json.gz) to parquet using glue ETL studio. However when we choose input file type as JSON in Glue ETL studio job, it is failing. What is the easiest way to dump DynamoDB data incrementally into parquet format in S3 while taking care of out of memory issues (Lambda/Glue ETL)?

0

There are 0 answers