environment variables not available to supervisor-run celery workers

Question

I have been having trouble reading environment variables in some Celery Tasks when celery is run via supervisor

in /etc/supervisord.conf ...

...
[program:celery]
...
command = /home/myuser/mydevelopment/git/ers_data_app/env/bin/celery worker -A webapp.celery --loglevel=info
stdout_logfile=/home/webdev/logs/celeryworker.log
stderr_logfile=/home/myuser/logs/celeryworkererr.log
environment=FLASK_CONFIG="testing"

in myapp/myfile.py ...

...
print 'the system config is', os.getenv('FLASK_CONFIG')
dburi = app_config.config[os.getenv('FLASK_CONFIG')].DATABASE_URI

in celeryworker.log ...

the system config is None

in celeryworkererr.log ...

...
File "/home/webdev/mydevelopment/git/ers_data_app/webapp/dbmodels/database.py", line 16, in <module>
    dburi = app_config.config[os.getenv('FLASK_CONFIG')].DATABASE_URI
KeyError: None

Since supervisor doesn't start a shell, the following is unnecessary, but its here for completeness..

in /etc/profile.d/my_app.sh

export FLASK_CONFIG='testing'

Answer 1

In your .conf file under the supervisord block, you can add all the environment key=value pairs as such

[supervisord]
environment=CELERY_BROKER_URL="amqp://guest:[email protected]:5672//",FLASK_CONFIG="TESTING"

[program:celeryd]
command=celery worker -A celery --loglevel=info -P gevent -c 1000

If you dont want to hardcode the variables but want to pull it in from the os environment, step 1 on your bash

Export env var

>> sudo export CELERY_BROKER_URL="amqp://guest:[email protected]:5672//"

Reload Bash

>> . ~/.bashrc

Check if env vars are set properly

>> env

Now modify the conf file to read - Note: prepend your env variables with ENV_

[supervisord]
environment=CELERY_BROKER_URL="%(ENV_CELERY_BROKER_URL)s",FLASK_CONFIG="%(ENV_FLASK_CONFIG)s"

[program:celeryd]
command=celery worker -A celery --loglevel=info -P gevent -c 1000