Django: Module import error with shell_plus --notebook

Question

In a Django project:

python manage.py shell_plus --ipython

is loading as expected with model imports, e.g., working just fine.

When trying to

python manage.py shell_plus --notebook

Jupyter will start. But when trying to create a new django shell plus notebook I will get the following import error:

[I 12:43:00.851 NotebookApp] Kernel started: 68eac163-4064-41ac-824d-5c839562814c, name: django_extensions
[IPKernelApp] WARNING | Error in loading extension: django_extensions.management.notebook_extension
Check your config files in /home/pumpkin/.ipython/profile_default
Traceback (most recent call last):
  File "/home/pumpkin/miniconda3/envs/django22/lib/python3.7/site-packages/IPython/core/shellapp.py", line 261, in init_extensions
    self.shell.extension_manager.load_extension(ext)
  File "/home/pumpkin/miniconda3/envs/django22/lib/python3.7/site-packages/IPython/core/extensions.py", line 87, in load_extension
    if self._call_load_ipython_extension(mod):
  File "/home/pumpkin/miniconda3/envs/django22/lib/python3.7/site-packages/IPython/core/extensions.py", line 134, in _call_load_ipython_extension
    mod.load_ipython_extension(self.shell)
  File "/home/pumpkin/miniconda3/envs/django22/lib/python3.7/site-packages/django_extensions/management/notebook_extension.py", line 10, in load_ipython_extension
    style=no_style(),
  File "/home/pumpkin/miniconda3/envs/django22/lib/python3.7/site-packages/django_extensions/management/shells.py", line 152, in import_objects
    setup()
  File "/home/pumpkin/miniconda3/envs/django22/lib/python3.7/site-packages/django/__init__.py", line 19, in setup
    configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
  File "/home/pumpkin/miniconda3/envs/django22/lib/python3.7/site-packages/django/conf/__init__.py", line 79, in __getattr__
    self._setup(name)
  File "/home/pumpkin/miniconda3/envs/django22/lib/python3.7/site-packages/django/conf/__init__.py", line 66, in _setup
    self._wrapped = Settings(settings_module)
  File "/home/pumpkin/miniconda3/envs/django22/lib/python3.7/site-packages/django/conf/__init__.py", line 157, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/home/pumpkin/miniconda3/envs/django22/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/pumpkin/django_projects/book_management/book_management/__init__.py", line 3, in <module>
    from .celery import app as celery_app
  File "/home/pumpkin/django_projects/book_management/book_management/celery.py", line 9, in <module>
    from celery import Celery
ModuleNotFoundError: No module named 'celery'

I might be missing something and I am happy to give more details and help reproduce the issue.

Versions:

• django-extensions 3.1.1
• Django 3.1.6
• jupyter 1.0.0
• jupyter-client 6.1.12
• jupyter-console 6.4.0
• jupyter-core 4.7.1
• jupyterlab-pygments 0.1.2
• jupyterlab-widgets 1.0.0

EDIT:

I could generalize the problem a bit. When executing: python manage.py shell_plus --notebook and printing sys.path we get

sys.path=
['/home/pumpkin/django_projects/book_management',
 '/home/pumpkin/miniconda3/envs/booking_j2/lib/python39.zip',
 '/home/pumpkin/miniconda3/envs/booking_j2/lib/python3.9',
 '/home/pumpkin/miniconda3/envs/booking_j2/lib/python3.9/lib-dynload',
 '/home/pumpkin/miniconda3/envs/booking_j2/lib/python3.9/site-packages']

which is expected (it is the conda env that is currently activated)

But when trying to start a new Django shell-plus notebook, we can see that the trace relies on another conda env:

[I 14:21:31.694 NotebookApp] Creating new notebook in 
[I 14:21:32.373 NotebookApp] Kernel started: 8d4013f7-ae9e-4bf4-80a3-ca9483e72f0b, name: django_extensions
[IPKernelApp] WARNING | Error in loading extension: django_extensions.management.notebook_extension
Check your config files in /home/pumpkin/.ipython/profile_default
Traceback (most recent call last):
  File "/home/pumpkin/miniconda3/envs/django22/lib/python3.7/site-packages/IPython/core/shellapp.py", line 261, in init_extensions
    self.shell.extension_manager.load_extension(ext)

In this conda env the required modules are not installed. Hence the error.

Now my question would be, why does ipython(?) does not run in the proper conda env ?

Note that when running python manage.py shell_plus --ipython the sys.path is as expected. Also when starting a classical python 3 notebook, imports are also happening as expected.

Answer 1

You should change kernel to Django shell Plus or something like it which in your Django app environment.

For more info check this post: https://medium.com/ayuth/how-to-use-django-in-jupyter-notebook-561ea2401852