Django circular import error - how to debug it?

Question

I tried to follow this tutorial https://blog.nicolasmesa.co/posts/2018/10/saas-like-isolation-in-django-rest-framework/

using: Django 3.1 Python 3.6

Everything I did including 'The User Messages App' paragraph worked perfectly right before the 'Refactoring the views' passage

and then I got the error when running manage.py runserver

django.core.exceptions.ImproperlyConfigured: The included URLconf 'saas_django.urls' does not appear to have any pa
tterns in it. If you see valid patterns in the file then the issue is probably caused by a circular import

I tried different types of import but I can't figure out where the cirular import happens

My steps to find a bug:

the saas_django/urls.py refers to user_messages app:

path('api/v1/user-messages/', include('user_messages.urls')),

user_messages/urls.py:

from django.urls import path
from . import views


urlpatterns = [


    path('', views.UserMessageList.as_view(),
        name=views.UserMessageList.name),

    path('<uuid:pk>', views.UserMessageDetail.as_view(),
         name=views.UserMessageDetail.name),

]

It seems like something is wrong with the imported user_messages/views.py

from rest_framework import permissions
from rest_framework import generics

from . import serializers
from .models import UserMessage


class UserMessageList(generics.ListCreateAPIView):
    name = 'usermessage-list'
    permission_classes = (
        permissions.IsAuthenticated,
    )
    serializer_class = serializers.UserMessageSerializer
    queryset = UserMessage.objects.all()

    def perform_create(self, serializer):
        user = self.request.user
        company_id = user.company_id
        # Added from_user
        serializer.save(company_id=company_id, from_user=user)

    def get_queryset(self):
        # Changed this to use the UserMessageManager's method
        return UserMessage.objects.get_for_user(self.request.user)


class UserMessageDetail(generics.RetrieveAPIView):
    name = 'usermessage-detail'
    permission_classes = (
       permissions.IsAuthenticated,
    )
    serializer_class = serializers.UserMessageSerializer

    def get_queryset(self):
        # Changed this to use the UserMessageManager's method
        return UserMessage.objects.get_for_user(self.request.user)

I gues the cause of the error is something about these 2 lines:

from . import serializers
from .models import UserMessage

the user_messages/serializers.py also has the import of UserMessages

from rest_framework import permissions
from rest_framework import generics

from . import serializers
from .models import UserMessage


class UserMessageList(generics.ListCreateAPIView):
    name = 'usermessage-list'
    permission_classes = (
        permissions.IsAuthenticated,
    )
    serializer_class = serializers.UserMessageSerializer
    queryset = UserMessage.objects.all()

    def perform_create(self, serializer):
        user = self.request.user
        company_id = user.company_id
        # Added from_user
        serializer.save(company_id=company_id, from_user=user)

    def get_queryset(self):
        # Changed this to use the UserMessageManager's method
        return UserMessage.objects.get_for_user(self.request.user)


class UserMessageDetail(generics.RetrieveAPIView):
    name = 'usermessage-detail'
    permission_classes = (
       permissions.IsAuthenticated,
    )
    serializer_class = serializers.UserMessageSerializer

    def get_queryset(self):
        # Changed this to use the UserMessageManager's method
        return UserMessage.objects.get_for_user(self.request.user)

I tried to rewrite import as:

from .models import UserMessage as UM

but it didn't work.

Project structure is:

│   db.sqlite3
│   manage.py
│   sqlite3.exe
│
├───accounts
│      admin.py
│      apps.py
│      models.py
│      serializers.py
│      tests.py
│      urls.py
│      views.py
│      __init__.py
│
├───core
│      models.py
│      serializers.py
│      views.py
│      __init__.py
│
├───saas_django
│      asgi.py
│      settings.py
│      urls.py
│      wsgi.py
│      __init__.py
│
└───user_messages
    │   admin.py
    │   apps.py
    │   models.py
    │   serializers.py
    │   tests.py
    │   urls.py
    │   views.py
    │   __init__.py
    │
    ├───migrations

My full code is here: https://github.com/SergSm/test-django-saas/

The question is how to properly debug this type of error?

Answer 1

First of all there are few issues in the project itself. There are unresolved references to serializers without which no one will be able to help out. For example:

from core.serializers import CompanySafeSerializerMixin

This is referenced in a few other serializers. Also, other than this, there is one major issue in the way you have defined the models for:

class User(AbstractUser):

    id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False)
    company = models.ForeignKey(Company,
                                related_name='%(class)s',
                                on_delete=models.CASCADE,
                                editable=False,
                                default=get_default_company)

The default method looks like this:

def get_default_company():
    """get or create the non existent company for new users"""

    return Company.objects.get_or_create(is_default_company=True,
                                         name=DEFAULT_COMPANY)[0].pk

Not sure in what sequence you made these models but since you have not pushed your migrations onto the VCS, this will cause the issue.

Remember, always push your migrations on the VCS. I cannot highlight this enough. It can lead to serious issues as and when the project grows.

Else, on new setup, Django will never be able to identify correct dependency between models.

Coming to the other problem, since you have used default as method that gets ref from another model, Django will not allow me to make the migrations in the first place since the model itself that its referring to is not yet made.

  File "/home/dhwanil/work/packages/test-django-saas/venv/lib64/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: relation "companies" does not exist
LINE 1: ...."address", "companies"."is_default_company" FROM "companies...
                                                             ^

This would have been solved if lets say you had 0001_initial one with the company model and the 0002_users where you refer the Company model. Also, you might want to look at the concept of fixtures that will help you populate a model on every fresh installation.

Coming to the main question you asked, if its indeed a circular, the best way is to closely inspect the traceback, it would show where the circular dep lies. Since I am unable to setup the project until the above mentioned issues are solved, I will not be able to give you a more directed answer.

Summary:

1. Always push the migrations that are created after python manage.py makemigrations on the VCS, migrations are like commits of the state of your DB, you cannot expect to work with latest state without knowing the steps it was built from.
2. Avoid using methods to get default from another model, the better way is always use fixtures to first populate the DB. Again, this will not be possible without splitting your migrations smartly.

return Company.objects.get_or_create(is_default_company=True,
                                         name=DEFAULT_COMPANY)[0].pk

3. Alternative to use fixtures would be "Data Migrations", I always recommend to add a custom RunPython code in your migrations to assign defaults. This takes off the overhead of worrying if the data is there in the DB or not, because it will be populated the moment you run migrate.
4. Most circular imports can be identified by the traceback, inspect it closely.

That's all for now, if you fix the import issues, let me know. I'll be happy to take a look.

Cheers!