Implemented botocore.credentials's RefreshableCredentials logic but the script is still failing after an hour

179 views Asked by At

My script is connecting to a remote account, finding the k8 pods and then downloading the required files from each one of them. Each file is taking about 1 second to get downloaded and there are more than 10,000 files. The assume_role gives me credentials to connect to the remote account but they expire after 3600 seconds as AWS has a hard limit on it.

After researching a little bit on it, I found out botocore.credentials's RefreshableCredentials and followed this https://dev.to/li_chastina/auto-refresh-aws-tokens-using-iam-role-and-boto3-2cjf and How to correctly refresh aws credentials with Python to implement the logic to my code.

I just tested the code after troubleshooting the errors and now after the 3600th hit, it shows an error

An error occurred (ExpiredToken) when calling the AssumeRole operation: The security token included in the request is expired
Unable to connect to the server: getting credentials: exec: executable /usr/bin/aws failed with exit code 255

Please guide me on how to get it working. I really need help. Here's my code:

from functools import partial
from botocore.credentials import RefreshableCredentials
from botocore.session import get_session
import boto3
import botocore

def get_target_account_session(target_account_id: str, region: str):
    sts = client("sts")
    try:
        response = sts.assume_role(
            RoleArn=f"arn:aws:iam::{target_account_id}:role/<role_name>",
            RoleSessionName="<session_name>"
        )
        if "Credentials" in response:
            creds = response["Credentials"]
            return {
                "access_key" : creds["AccessKeyId"],
                "secret_key" : creds["SecretAccessKey"],
                "token" : creds["SessionToken"],
                'expiry_time': creds["Expiration"].isoformat(),
                "region_name" : region
            }
    except ClientError as err:
        exit(err)
    return None

def get_aws_autorefresh_session(target_account_id, region):
    session_credentials = RefreshableCredentials.create_from_metadata(
        metadata = get_target_account_session(target_account_id, region),
        refresh_using = partial(get_target_account_session, target_account_id, region),
        method = "sts-assume-role"
    )

    session = get_session()
    session._credentials = session_credentials
    autorefresh_session = boto3.Session(botocore_session=session)

    return session_credentials, autorefresh_session

creds, ses = get_aws_autorefresh_session(value[0], region)
eks = ses.client("eks", region_name=region)
env_var={"AWS_ACCESS_KEY_ID":creds.access_key, "AWS_SECRET_ACCESS_KEY":creds.secret_key, "AWS_SESSION_TOKEN":creds.token, "KUBECONFIG":"/root/.kube/<kube_config>"}
clusters = list_clusters_boto3(eks)
grab_pod_information(value[0], clusters, region, env_var, value[1])

What am I doing wrong here ? Thank you!

0

There are 0 answers