How to view task durations over time when Airflow web UI Task Duration screen for DAG non-responsive?

42 views Asked by At

Airflow v1.10.7 running in LocalExecutor mode.

Trying to find a way to determine what tasks in a DAG have recently jumped up in duration time; the Task Duration web UI for the DAG is being slow and unresponsive. Is there another way to do this? (Any way to debug the slowness of the web UI?)

Trying to view the Task Duration screen for a DAG with many tasks (mostly Apache Sqoop jobs), but web browser keeps hanging when attempting this (happens regardless of web browser type and even when trying locally on the machine running the airflow-webserver).

If I wait very long, I can see the graph load (showing way more than the default 'last 25 number of runs' for some reason) and can see that around the past month a few tasks made a step-wise jump in duration and have since stayed at that elevated level for each subsequent run. Now I am trying to figure out what those tasks are, but the fact that the web UI is so laggy when looking at this DAG is making the Task Duration page impossible to use.

Any other common best-practices way to debug this situation? (Any suggestions for debugging the web UI so I can use the Task Duration page again?)

0

There are 0 answers