Airflow get stuck and never finish on running PapermillOperator

Question

I'm trying to run a DAG that have a task that is running a jupyter notebook. Every task is doing well until it get's to Papermill operator. the progress bar stay on 0 and the task never finish. when looking inside the pod there is an:

AirflowException : "Celery command failed on host <airflow-worker id>"
i'm using airflow 2.3.3 and the pod have papermill(2.4.0) and ipykernel(6.15.2) installed
Arflow is running on openshift

Answer 1

The solution was to give more ressources (cpu and memory) to the pod.

inside your dag:

from kubernetes.client import models as k8s

task_ressource = k8s.V1ResourceRequirements(limits={"cpu": "800m", "memory": "5Gi"}, request={"cpu": "200m", "memory": "2Gi"})
task = KubernetesPodOperator('task_id'='task_id',image='your image',resources=task_ressource)