I have deployed Airflow in Docker with the following containers:
new-airflow-docker-airflow-scheduler-1 new-airflow-docker-airflow-worker-1 new-airflow-docker-airflow-triggerer-1 new-airflow-docker-airflow-webserver-1 new-airflow-docker-postgres-1 new-airflow-docker-redis-1
I have one worker.
My Airflow Configs are the following
worker_concurrency 16
sync_parallelism 0
parallelism 32
max_active_tasks_per_dag 16
dags_are_paused_at_creation true
max_active_runs_per_dag 16`
DAG configs are
schedule_interval='@hourly',
start_date=datetime(2021,1,1,0, 0, 0),
catchup=True
But When I trigger a DAG. Only 1 task is running. I was expecting to run a lot of tasks because start_date is previous date and catchup is true.