Different JupyterLab working directories for different environments

Question

I have several projects I want to be able to work on in a best practice way. For this reason I created different conda environments. For my first project I have a dedicated directory, which I defined as Jupyter working directory, like described here. Now after starting Anaconda and JupyterLab I can start quickly since I'm starting from the right directory and also all my last tabs are opened from the JupyterLab workspace. So everything seems to be well so far.

But now I want to start a second project in a second conda environment. I did not manage to define a working directory and JupyterLab workspace for this specific conda environment. The solution out of the hyperlink above, seems to be a global setting, which is applied for all conda environment. And also the workspace, I mean all the tabs which are opened out of the last session, seems to be something global. I don't want to mix up code from different projects in the same working directory.

I tried out so far something with ipykernel and also this approach with nb_conda_kernels. But since I'm quiet new to Anaconda/Jupyter and even Python, I did not managed it to work with these approaches like it is intended by me. And I start wondering, if I'm missing some details or if the best practice for working on different projects in JupyterLab is completely different from what I did so far or what I want to do.

So how can I define a different working directory and ideally a different Jupyter workspace for a different conda environment? Or how does the best practice look like, when working on different projects in Anaconda/JupyterLab?

Answer 1

Jupyterlab can be installed as pip package in any virtual environment. Because of this one may have several instances of Jupyterlab in different environments and think that they should be totally independent. This was my misunderstanding initially and this was assumed in the first version of this answer.

However, I have come to the conclusion that Jupyterlab is more like an IDE than a package. Similarly to PyCharm for example, one should have a single global installation of Jupyterlab which is shared between environments. And like PyCharm where one can change Python interpreter to the virtual environment, in Jupyterlab we can change kernels.

So the first thing to ensure is that a correct kernel is used for a virtual environment. The kernel can be changed in the menu and a notebook remembers the last kernel, so this should not be a problem. To add a new kernel one should follow IPython documentation: activate the environment, make sure ipykernel is installed and issue a command

python -m ipykernel install --user --name <env_name> --display-name "<name in Jupyterlab UI>"

Additionally, Jupyterlab may be started with the command line parameter that changes the default kernel, so that the environment kernel is the first in the launcher menu:

jupyter lab --MultiKernelManager.default_kernel_name=<env_name>

Regarding the working directory, one may start Jupyterlab from the desired location which is not very convenient. Generating jupyter_notebook_config.py and setting the working directory there is not an option, unfortunately, because this config is global. The way to go is a command-line parameter as Steven Silvester suggests in this github issue:

the working directory can be set when launching as jupyter lab --NotebookApp.notebook_dir=<directory_name>, or using Jupyter config.

As for the workspace one is allowed to change the default location using a system environment variable. Here is the documentation:

By default, the location is ~/.jupyter/lab/workspaces/, where ~ is the user’s home directory. This folder is not in the JupyterLab application directory, because these files are typically shared across Python environments. The location can be modified using the JUPYTERLAB_WORKSPACES_DIR environment variable.

There is a similar environment variable for the settings directory:

The user settings directory contains the user-level settings for Jupyter extensions. By default, the location is ~/.jupyter/lab/user-settings/, where ~ is the user’s home directory. This folder is not in the JupyterLab application directory, because these settings are typically shared across Python environments. The location can be modified using the JUPYTERLAB_SETTINGS_DIR environment variable.

The sharing of settings is a reasonable assumption, not so for workspaces if one wants a seamless separate environment configuration.

So finally, one would want to have something like this in their environment:

alias jl="jupyter lab --NotebookApp.notebook_dir=<working_directory> --MultiKernelManager.default_kernel_name=<env_name>"
export JUPYTERLAB_WORKSPACES_DIR=<workspaces directory>

For this to be a solution to a problem it has to be automated so that the above commands are executed when an environment is started and only this environment has to be affected. I use a DIY unpublished environments management solution that consists of several shell scripts and does the automation for me.