Is there any session available across AWS Glue Jobs?

Question

In my AWS Glue Workflow (defined using CloudFormation) I have a few jobs (pythonshell) that have a repeated logic to create some parameters like the same db connection string etc. Also there is a repeated logic via Boto3 like this:

import boto3
sm_client = boto3.client(service_name='secretsmanager', region_name=region_name)
db_password = sm_client.get_secret_value(SecretId="MY_DB_PASSWORD")["SecretString"]

to fetch the relative password of the DB.

As said this sort of logic is repeated across multiple jobs. Is there any sort session or you have any idea about storing this sort of info across multiple jobs for the same workflow?

Answer 1

You can share information across Glue Jobs using workflow properties as defined at this https://docs.aws.amazon.com/glue/latest/dg/workflow-run-properties-code.html

But for sensitive information I would rather use SecretManager or Glue database connection.