github action error missing dependency even if it already installed from requirements.txt

Question

I am running a github actions and it throws an error about missing matplotlib package even if it was already installed as a dependency from requirements.txt file.

name: install and run test package

on: [push]

jobs:
  build:

    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        os: [ubuntu-latest, macos-latest, windows-latest] 
        python-version: ["3.11"]
        
    steps:
      - uses: actions/checkout@v3
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v4
        with:
          python-version: ${{ matrix.python-version }}
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt
          # pip install -e .
      - name: Test with pytest
        run: |
          pytest

Output:

Collecting matplotlib (from -r requirements.txt (line 3))
  Downloading matplotlib-3.8.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.8 kB)
Collecting lightkurve==2.4.1 (from -r requirements.txt (line 4))
  Downloading lightkurve-2.4.1-py3-none-any.whl.metadata (6.0 kB)
Collecting numpy==1.23 (from -r requirements.txt (line 5))
  Downloading numpy-1.23.0.tar.gz (10.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.7/10.7 MB 87.8 MB/s eta 0:00:00
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting transitleastsquares==1.0.31 (from -r requirements.txt (line 6))
  Downloading transitleastsquares-1.0.31-py3-none-any.whl.metadata (5.4 kB)
Collecting wotan==1.10 (from -r requirements.txt (line 7))
  Downloading wotan-1.10-py3-none-any.whl.metadata (11 kB)
Collecting aesthetic==0.6 (from -r requirements.txt (line 8))
  Downloading aesthetic-0.6.tar.gz (474 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 474.2/474.2 kB 52.7 MB/s eta 0:00:00
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'error'
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [20 lines of output]
      Traceback (most recent call last):
        File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-gjzfg86r/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-gjzfg86r/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-gjzfg86r/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 487, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-gjzfg86r/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 13, in <module>
      ModuleNotFoundError: No module named 'matplotlib'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

Here's the requirements.txt file:

pre-commit==3.4.0
matplotlib
lightkurve==2.4.1
numpy==1.23
transitleastsquares==1.0.31
wotan==1.10
aesthetic==0.6
flammkuchen==1.0.3
reproject==0.12.0
astroplan==0.9

Answer 1

It appears that aesthetic==0.6 has the following near the top of its setup.py.

import matplotlib

(aesthetic/setup.py)

That means that matplotlib needs to be installed prior to building aesthetic.

The code raising the error you've shown is happening at the point where pip install is downloading + building but not yet installing packages. Even though you've listed matplotlib in the requirements, at the moment where pip tries to build aesthetic, matplotlib isn't actually installed or importable yet.

To make this work without any changes to aesthetic, you'll have to ensure that matplotlib is already installed in the environment first.

For example, like this:

- name: Install dependencies
  run: |
    python -m pip install --upgrade pip
    pip install matplotlib
    pip install -r requirements.txt