AWS Glue Dev Endpoint - Cache Virtual Env

142 views Asked by At

I'm using the AWS Glue Dev Endpoint. It takes a while for the python virtualenv stuff to be uploaded. Is there a way that I can cache these files to tighten the development loop?

Any help is greatly appreciated.

1

There are 1 answers

0
Black Dynamite On BEST ANSWER

I think the Dev Endpoint method of development has been superceded by using a Docker environment.

https://aws.amazon.com/blogs/big-data/developing-aws-glue-etl-jobs-locally-using-a-container/