Mypy complaining about Name "Optional" is not defined without the use of Optional

Question

I've recently started using mypy, and have run into some weird problems that i cannot for the life of me seem to figure out.

I'm using mypy 0.950, django-stubs 1.11.0, django 4.0.5 and python 3.10.2.

Running mypy through the command line returns this:

project/suppliers/models.py:6: error: Name "Optional" is not defined
project/suppliers/models.py:6: note: Did you forget to import it from "typing"? (Suggestion: "from typing import Optional")
project/users/models.py:6: error: Name "Optional" is not defined
project/users/models.py:6: note: Did you forget to import it from "typing"? (Suggestion: "from typing import Optional")
project/products/models.py:6: error: Name "Optional" is not defined
project/products/models.py:6: note: Did you forget to import it from "typing"? (Suggestion: "from typing import Optional")(Suggestion: "from typing import Optional")

However, line 6 in project/suppliers/models.py is completely empty:

from django.contrib.sites.managers import CurrentSiteManager
from django.contrib.sites.models import Site
from django.db import models
from django.utils.text import slugify
from django.utils.translation import gettext_lazy as _

from django_countries.fields import CountryField

from project.core.models import BaseImageModel, BaseModel
from project.suppliers.managers import SupplierQuerySet

_SupplierManager = models.Manager.from_queryset(SupplierQuerySet)


class Supplier(BaseModel, BaseImageModel):
    ...

Line 6 in project/users/models.py is a django import from django.contrib.contenttypes.models import ContentType:

import random
from typing import Any

from django.conf import settings
from django.contrib.auth.models import AbstractBaseUser, PermissionsMixin
from django.contrib.contenttypes.models import ContentType
from django.contrib.sites.managers import CurrentSiteManager
from django.contrib.sites.models import Site
from django.core import signing
from django.core.mail import send_mail
from django.db import models
from django.forms import ValidationError
from django.http import HttpRequest
from django.template.loader import render_to_string
from django.utils import timezone
from django.utils.encoding import force_bytes, force_str
from django.utils.http import (
    urlsafe_base64_decode as uid_decoder,
    urlsafe_base64_encode as uid_encoder,
)
from django.utils.translation import gettext_lazy as _

import phonenumbers

from project.users.enums import AvatarColors
from project.users.managers import UserQuerySet
from project.users.schemas.records import (
    UserAuditLogsRecord,
    UserNotesRecord,
    UserProfileRecord,
)

_UserManager = models.Manager.from_queryset(UserQuerySet)


class User(AbstractBaseUser, PermissionsMixin):

And line 6 in project/products/models.py is yet another django import from django.utils.text import slugify:

from decimal import Decimal

from django.contrib.sites.managers import CurrentSiteManager
from django.contrib.sites.models import Site
from django.db import models
from django.utils.text import slugify
from django.utils.translation import gettext_lazy as _

from imagekit.models import ImageSpecField
from imagekit.processors import ResizeToFill
from mptt.models import TreeManyToManyField
...

My mypy config is as follows:

[tool.mypy]
plugins = ["mypy_django_plugin.main", "pydantic.mypy"]
follow_imports = "normal"

ignore_missing_imports = true

disallow_untyped_calls = true
disallow_untyped_defs = true
disallow_incomplete_defs = true

warn_unused_configs = true
warn_redundant_casts = true
warn_unused_ignores = true
warn_return_any = true
warn_unreachable = true

no_implicit_optional = true
no_implicit_reexport = true
check_untyped_defs = true
strict_equality = true

[tool.django-stubs]
django_settings_module = "project.settings"

[tool.pydantic-mypy]
init_forbid_extra = true
init_typed = true
warn_required_dynamic_aliases = true
warn_untyped_fields = true

# Admin files uses some patterns that are not easily typed
[[tool.mypy.overrides]]
module = "project.*.admin"
ignore_errors = true

[[tool.mypy.overrides]]
module = "project.*.tests.*"
ignore_errors = true

[[tool.mypy.overrides]]
module = "project.*.migrations.*"
ignore_errors = "true"

[[tool.mypy.overrides]]
module = "project.*.management.*"
disallow_untyped_defs = false

I've tried googling around, but can not seem to find anyone else that has experienced this. Anything obvious that I've missed, or does it look like a bug? Seems to me at least that this would affect quite a lot if it was something wrong with mypy/django stubs.

Answer 1

Fast 1st-party fix

You can add from typing import Optional to files that use CurrentSiteManager. It will resolve this problem (yes, # noqa: F401 is your friend).

One-time fix for package

As a quick workaround, you can modify django-stubs/contrib/sites/managers.pyi to have the following content:

from typing import Optional, TypeVar
from django.db import models

_T = TypeVar('_T', bound=models.Model)

class CurrentSiteManager(models.Manager[_T]):
    def __init__(self, field_name: Optional[str] = ...) -> None: ...

Problem root

It is not a real solution and is a bug in mypy_django_plugin. I'm looking for a solution now. The problem is that helpers.copy_method_to_another_class (and consequently transformers.models.AddManagers.create_new_model_parametrized_manager) is using context of model class definition. I don't understand now how to get and pass another context properly without symmetrical issue, and merging is definitely not an option. I'll update this on success and raise a PR to maintainer. It would be great if you file an issue to django-stubs (and attach link to this question) so that I have less to explain (or somebody else could help if I fail).

Better fix #1

here we can add original_module_name=base_manager_info.module_name call argument. It will allow resolving everything inherited from first MRO parent. However, it seems like this should fail for longer inheritance chains. My bad, we're actually iterating here over methods of last ancestor only, so this seems to be the final solution.

(I'm sorry for using this as a changelog, but once started...)

This bug was fixed in this my PR.