Invoke a aws lambda function using MWAA DAG and schedule it for 15 minutes

140 views Asked by At

Invoke a aws lambda function using MWAA DAG and schedule it for 15 minutes

Tried and followed below solution but it didnt work

Python/Airflow AttributeError: 'AwsLambdaHook' object has no attribute 'update_relative'

Error is "DAG Import Error"

Broken DAG: [/usr/local/airflow/dags/invoke_lambda_dag.py] Traceback (most recent call last): File "", line 241, in _call_with_frames_removed File "/usr/local/airflow/dags/invoke_lambda_dag.py", line 6, in from airflow.providers.amazon.aws.operators.aws_lambda import AwsLambdaInvokeFunctionOperator ModuleNotFoundError: No module named 'airflow.providers.amazon.aws.operators.aws_lambda'

Referred below documentation as well

https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/_api/airflow/providers/amazon/aws/operators/lambda_function/index.html#airflow.providers.amazon.aws.operators.lambda_function.LambdaInvokeFunctionOperator

Below is the code:

import json, boto3 from airflow import DAG, utils from datetime import datetime, timedelta from airflow.operators.python import PythonOperator, BranchPythonOperator from airflow.providers.amazon.aws.operators.lambda_function import ( LambdaCreateFunctionOperator, LambdaInvokeFunctionOperator, )

DAG_ID = "InvokeLambda"

with DAG( DAG_ID, schedule="*/15 * * * *", start_date=datetime(2023, 11, 24), tags=["example"], catchup=False, ) as dag:

lambda_function_name = "lambda_name"

invoke_lambda_function = LambdaInvokeFunctionOperator(
    task_id="invoke_lambda_function",
    function_name=lambda_function_name,
    payload=None,
)

invoke_lambda_function

It is working but the problem here is , the aws lambda is triggering twice with a gap of 1 min in 15min schedule.

enter image description here

0

There are 0 answers