No response from gunicorn master within 120 seconds

Question

Need assistance with launching my airflow webserver, the process takes longer than usual and I keep getting the same error.

[2023-11-02T01:46:30.297+0100] {webserver_command.py:225} ERROR - No response from gunicorn master within 120 seconds 
[2023-11-02T01:46:30.298+0100] {webserver_command.py:226} ERROR - Shutting down webserver

What I've done: Reinstalled Ubuntu cd project directory created a virtual environment. Set export AIRFLOW_HOME=pwd to my current directory Uninstalled and re-installed apache-airflow in the environment, migrate db and created admin-user changed both to 0

• worker_refresh_batch_size = 0
• worker_refresh_interval = 0 changed port (e.g to 8081)

Here is what my current directory looks like:

I suspect the issues might be in the configured $AIRFLOW_HOME as it impacts the .cfg file, but i'm really not familiar with the installation process. and would appreciate your assistance.

dags_folder = /mnt/c/Users/buasc/PycharmProjects/airflow/dags
sql_alchemy_conn = sqlite:////mnt/c/Users/buasc/PycharmProjects/airflow/airflow.db

Answer 1

Apparently, this issue was from the set-up to installing airflow, solving the problem meant uninstalling and restarting the process from scratch.

Step 1: Enable linux properties on windows (windows 11)

• Go to Start. Search for "Turn Windows features on or off."
• Check the option
• • Windows Subsystem for Linux
• • Virtual Machine Platform

Step2: Visit Microsoft Store and download Ubuntu

• Install Ubuntu
• Launch cmd as adminstrator and run wsl --install to enable the windows subsytem
• • If the returns an error run the following:
• • wsl --shutdown
• • wsl --list this returns the installed linux distro
• • wslconfig /unregister <distro>
• • wsl --install
• A prompt returns to set a username and password, don't forget your credentials

Step 3: Set up c directory.

• Update everything: sudo apt update && sudo apt upgrade
• On the (Ubuntu) terminal cd ~ to your root directory Now we will configure the file system to direct to the c without mounting it (without using /mnt/ prefix):
• sudo nano /etc/wsl.conf to modify it by pasting the following, save and exit with ctrl+s ctrl+x:

[automount]

root = /
options = "metadata"

• Run nano ~/.bashrc, insert the line below, save and exit with ctrl+s ctrl+x

export AIRFLOW_HOME=/c/Users/<PARENT_DIR>/<PROJECT_DIR>

• close your terminal and restart
• Test that everything works:
• • cd ~
• • cd \
• • ls: Now you should see the c directory
• • echo $AIRFLOW_HOME: you should see the directory you set

Step 4: Install Packages.

• sudo apt get virtualenv
• cd $AIRFLOW_HOME
• virtualenv <env-name>: creates a virtual environment with the env-name
• source <env-name>/bin/activate: to activate the environment.

Install airflow:

• pip install apache-airflow
• airflow db migrate

airflow users create \
   --username admin \
   --firstname Peter \
   --lastname Parker \
   --role Admin \
   --email [email protected]

then a prompt asks for you to set a login password

• now, run airflow webserver --port 8080 to launch the webserver

The issue from the original attempt was having to mount with the /mnt/ prefix and not wanting to use docker. Very thankful for the following tutorials:

https://towardsdatascience.com/run-apache-airflow-on-windows-10-without-docker-3c5754bb98b4

https://www.freecodecamp.org/news/install-apache-airflow-on-windows-without-docker/