Airflow picking up queued task very slow

Question

I have an airflow in the docker on AWS t2-medium, tasks are getting queued, but it seems it sometimes waits quite a while before being picked up.

It can be as long as a few minutes, and could also get stuck (not being picked up after hours).

I have the following in my config:

job_heartbeat_sec = 5
scheduler_heartbeat_sec = 5

Any suggestions?

Thanks!