Use layered requirements with project.optional-dependencies in pyproject.toml

Question

I'd like to lock my dependencies tracked in pyproject.toml using pip-compile. How can I use the layered approach for development-only dependencies (quoting https://github.com/jazzband/pip-tools#workflow-for-layered-requirements):

If you have different environments that you need to install different but compatible packages for, then you can create layered requirements files and use one layer to constrain the other.

For example, if you have a Django project where you want the newest 2.1 release in production and when developing you want to use the Django debug toolbar, then you can create two *.in files, one for each layer:

# requirements.in
django<2.2

At the top of the development requirements dev-requirements.in you use -c requirements.txt to constrain the dev requirements to packages already selected for production in requirements.txt.

# dev-requirements.in
-c requirements.txt
django-debug-toolbar<2.2

…but using dependencies tracked in pyproject.toml instead of *.in files? E.g.:

[project]
name = "hello"
version = "0.1.0"
dependencies = ["django<2.2"]

[project.optional-dependencies]
dev = ["django-debug-toolbar<2.2"]

I've tried adding "-c requirements.txt" to the optional dependencies, but pip complains that this is not PEP 508 compliant:

  ValueError: invalid pyproject.toml config: `project.optional-dependencies.dev[0]`.
  configuration error: `project.optional-dependencies.dev[0]` must be pep508

Is this supported or do I have to revert back to tracking my non-locked dependencies in requirements.in files?

Answer 1

I think I would try something like this:

pip-compile --extra=dev --output-file=dev-requirements.txt pyproject.toml
pip-compile --constraint=dev-requirements.txt --output-file=requirements.txt pyproject.toml