How to make Django 3, channels and uvicorn work together

Question

I have been trying to switch from daphne to uvicorn for production with a project using django 3 and channels. I have been encountering errors while loading the classical asgi file for channels. Either I can't use it due to synchronous call to django.setup or get_application. I tried to tweak this file with the sync_to_async call without success. Has anyone managed to make it work ?

Original asgi.py

Code

import os
import django
from channels.routing import get_default_application


os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings")

django.setup()
application = get_default_application()

StackTrace

Traceback (most recent call last):                                                                                                                                              
File "/usr/local/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap                                                                                           
  self.run()                                                                                                                                                                  
File "/usr/local/lib/python3.8/multiprocessing/process.py", line 108, in run                                                                                                  
  self._target(*self._args, **self._kwargs)                                                                                                                                   
File "/var/www/.cache/pypoetry/virtualenvs/project-4ffvdAoS-py3.8/lib/python3.8/site-packages/uvicorn/subprocess.py", line 61, in subprocess_started                           
  target(sockets=sockets)                                                                                                                                                     
File "/var/www/.cache/pypoetry/virtualenvs/project-4ffvdAoS-py3.8/lib/python3.8/site-packages/uvicorn/main.py", line 407, in run                                               
  loop.run_until_complete(self.serve(sockets=sockets))                                                                                                                        
File "/usr/local/lib/python3.8/asyncio/base_events.py", line 612, in run_until_complete                                                                                       
  return future.result()                                                                                                                                                      
File "/var/www/.cache/pypoetry/virtualenvs/project-4ffvdAoS-py3.8/lib/python3.8/site-packages/uvicorn/main.py", line 414, in serve                                             
  config.load()                                                                                                                                                               
File "/var/www/.cache/pypoetry/virtualenvs/project-4ffvdAoS-py3.8/lib/python3.8/site-packages/uvicorn/config.py", line 300, in load                                            
  self.loaded_app = import_from_string(self.app)                                                                                                                              
File "/var/www/.cache/pypoetry/virtualenvs/project-4ffvdAoS-py3.8/lib/python3.8/site-packages/uvicorn/importer.py", line 20, in import_from_string                             
  module = importlib.import_module(module_str)                                                                                                                                
File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module                                                                                             
  return _bootstrap._gcd_import(name[level:], package, level)                                                                                                                 
File "<frozen importlib._bootstrap>", line 1014, in _gcd_import                                                                                                               
File "<frozen importlib._bootstrap>", line 991, in _find_and_load                                                                                                             
File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked                                                                                                    
File "<frozen importlib._bootstrap>", line 671, in _load_unlocked                                                                                                             
File "<frozen importlib._bootstrap_external>", line 783, in exec_module                                                                                                       
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed                                                                                                  
File "./config/asgi.py", line 16, in <module>                                                                                                                                 
  django.setup()                                                                                                                                                              
File "/var/www/.cache/pypoetry/virtualenvs/project-4ffvdAoS-py3.8/lib/python3.8/site-packages/django/__init__.py", line 24, in setup                                           
  apps.populate(settings.INSTALLED_APPS)                                                                                                                                      
File "/var/www/.cache/pypoetry/virtualenvs/project-4ffvdAoS-py3.8/lib/python3.8/site-packages/django/apps/registry.py", line 122, in populate                                  
  app_config.ready()                                                                                                                                                          
File "/var/www/.cache/pypoetry/virtualenvs/project-4ffvdAoS-py3.8/lib/python3.8/site-packages/django_prometheus/apps.py", line 23, in ready                                    
  ExportMigrations()                                                                                                                                                          
File "/var/www/.cache/pypoetry/virtualenvs/project-4ffvdAoS-py3.8/lib/python3.8/site-packages/django_prometheus/migrations.py", line 52, in ExportMigrations                   
  executor = MigrationExecutor(connections[alias])                                                                                                                            
File "/var/www/.cache/pypoetry/virtualenvs/project-4ffvdAoS-py3.8/lib/python3.8/site-packages/django/db/migrations/executor.py", line 18, in __init__                          
  self.loader = MigrationLoader(self.connection)                                                                                                                              
File "/var/www/.cache/pypoetry/virtualenvs/project-4ffvdAoS-py3.8/lib/python3.8/site-packages/django/db/migrations/loader.py", line 49, in __init__                            
  self.build_graph()                                                                                                                                                          
File "/var/www/.cache/pypoetry/virtualenvs/project-4ffvdAoS-py3.8/lib/python3.8/site-packages/django/db/migrations/loader.py", line 212, in build_graph                        
  self.applied_migrations = recorder.applied_migrations()                                                                                                                     
File "/var/www/.cache/pypoetry/virtualenvs/project-4ffvdAoS-py3.8/lib/python3.8/site-packages/django/db/migrations/recorder.py", line 76, in applied_migrations                
  if self.has_table():                                                                                                                                                        
File "/var/www/.cache/pypoetry/virtualenvs/project-4ffvdAoS-py3.8/lib/python3.8/site-packages/django/db/migrations/recorder.py", line 56, in has_table                         
  return self.Migration._meta.db_table in self.connection.introspection.table_names(self.connection.cursor())                                                                 
File "/var/www/.cache/pypoetry/virtualenvs/project-4ffvdAoS-py3.8/lib/python3.8/site-packages/django/utils/asyncio.py", line 24, in inner
  raise SynchronousOnlyOperation(message)
django.core.exceptions.SynchronousOnlyOperation: You cannot call this from an async context - use a thread or sync_to_async.

Modified asgi.py

Code

import os
import django
from asgiref.sync import sync_to_async
from channels.routing import get_default_application


os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings")
sync_to_async(django.setup, thread_sensitive=True)
application = sync_to_async(get_default_application, thread_sensitive=True)

Stacktrace

Traceback (most recent call last):
File "/var/www/.cache/pypoetry/virtualenvs/project-4ffvdAoS-py3.8/lib/python3.8/site-packages/uvicorn/lifespan/on.py", line 55, in main
  await app(scope, self.receive, self.send)
File "/var/www/.cache/pypoetry/virtualenvs/project-4ffvdAoS-py3.8/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 45, in __call__
  return await self.app(scope, receive, send)
File "/var/www/.cache/pypoetry/virtualenvs/project-4ffvdAoS-py3.8/lib/python3.8/site-packages/asgiref/sync.py", line 296, in __call__
  ret = await asyncio.wait_for(future, timeout=None)
File "/usr/local/lib/python3.8/asyncio/tasks.py", line 455, in wait_for
  return await fut
File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
  result = self.fn(*self.args, **self.kwargs)
File "/var/www/.cache/pypoetry/virtualenvs/project-4ffvdAoS-py3.8/lib/python3.8/site-packages/asgiref/sync.py", line 334, in thread_handler
  return func(*args, **kwargs)
TypeError: get_default_application() takes 0 positional arguments but 3 were given
ERROR:    Application startup failed. Exiting.

Thanks for reading me guys

Answer 1

1. The callable you pass to uvicorn is going to be run in an event loop as per this function
2. Because of 1. django.setup() needs to be converted to a co-routine
3. get_default_application() should not be run asynchronously, it's going to auto-magically return the router application. That's what uvicorn should be pointed to.
4. django.setup() must return before the main event loop is started

Given those four points, your code should be as follow:

import os
import django
from asgiref.sync import sync_to_async
from channels.routing import get_default_application


os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings")
await sync_to_async(django.setup, thread_sensitive=True)()
application = get_default_application()

But, as of writing this, you cannot use await as a top-level instruction AND if you remove it, there's no guarantee the setup would be done before the module is resolved (it won't). For this reason you can't use uvicorn's command line if you have synchronous code to run prior to launching the application.

You can however launch it programmatically:

# server.py

import uvicorn
import os
import django

from channels.routing import get_default_application


os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings")
django.setup()

app = get_default_application()

    
if __name__ == "__main__":
    uvicorn.run("server:app") # Pass additional command line options as kwargs

Answer 2

I did a slightly different (and in my opinion easier to use) approach to this problem, that is executing sync initialization of Django stuff before launching the uvicorn server.

Just create a python file uvicorn-django with exec permissions with this content:

#!/usr/bin/env python

# Patch uvicorn to work properly with django setup

import uvicorn
import importlib


# monkeypatch uvicorn run function to import app before launching server
orig_run = uvicorn.run


def run(*args, **kwargs):
    # import application entrypoint before running uvicorn
    importlib.import_module(args[0])

    orig_run(*args, **kwargs)


uvicorn.run = run

if __name__ == "__main__":
    # start uvicorn programmatically after initializing django application
    #  - django needs to be initialized in non async context (which is not the case if application is started from uvicorn directly)
    #  - uvicorn.main is the cli (click) implementation that would be executed when starting uvicorn from shell, therefore parsing all the passed arguments as if uvicorn was started directly
    uvicorn.main()

This way you can use this python script as if you'd start uvicorn CLI itself, including all the parameters uvicorn offers:

$ uvicorn-django --host 0.0.0.0 --port 8080 YOUR_MODULE:YOUR_APPLICATION

---

Another solution would be to use gunicorn with the uvicorn worker which doesn't have the async issues either:

$ gunicorn -k uvicorn.workers.UvicornWorker YOUR_MODULE:YOUR_APPLICATION