scipy 1.7.x from source build error with python 3.7.x on macbook M1 Sonoma

Question

I need to install SciPy version 1.7.0 -1.7.3 with python 3.7.0 - 3.7.17 on macbook M1 Sonoma 14.0. To work with python version 3.7.17 on M1, I use pyenv.

I have gcc, gfortran, openblas, pkg-config installed via brew. The versions are listed below.

Cython, pybind11, pythran, numpy are also installed. The versions are listed below.

Initially there was a Cython error, but I found out by experience that the version should be Cython<3. And now I'm getting a lower-level compilation error. Maybe I need to use a different version of the Mac OS X.sdk? Or change the versions of dependent packages in a special way?

$OPENBLAS=$(brew --prefix openblas) CFLAGS="-falign-functions=8 ${CFLAGS}" pip3 install --no-binary :all: --no-use-pep517 scipy==1.7.3

[...]
In file included from scipy/spatial/ckdtree.cxx:1007:
      /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/complex:1401:53: error: expected unqualified-id
          if (__x.real() == 0 && (__x.imag() == 0 || std::isnan(__x.imag())))
                                                          ^
      scipy/_lib/_c99compat.h:124:30: note: expanded from macro 'isnan'
                  #define isnan(x) ((x) != (x))
                                   ^
      1 warning and 1 error generated.
      error: Command "clang++ -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -falign-functions=8 -I/opt/homebrew/opt/imagemagick@6/include -I/Users/dwdsf/.pyenv/versions/3.7.17/include/python3.7m -I/Users/dwdsf/workflow/alfa/fs_backend_api/venv/lib/python3.7/site-packages/numpy/core/include -Iscipy/_lib -Iscipy/_build_utils/src -Iscipy/spatial/ckdtree/src -I/Users/dwdsf/workflow/alfa/fs_backend_api/venv/lib/python3.7/site-packages/numpy/core/include -Ibuild/src.macosx-11.6-arm64-3.7/numpy/distutils/include -I/Users/dwdsf/workflow/alfa/fs_backend_api/venv/include -I/Users/dwdsf/.pyenv/versions/3.7.17/include/python3.7m -c scipy/spatial/ckdtree.cxx -o build/temp.macosx-11.6-arm64-cpython-37/scipy/spatial/ckdtree.o -MMD -MF build/temp.macosx-11.6-arm64-cpython-37/scipy/spatial/ckdtree.o.d -std=c++14 -mmacosx-version-min=10.9" failed with exit status 1
      
      ########### EXT COMPILER OPTIMIZATION ###########
      Platform      :
        Architecture: aarch64
        Compiler    : clang
      
      CPU baseline  :
        Requested   : 'min'
        Enabled     : NEON NEON_FP16 NEON_VFPV4 ASIMD
        Flags       : none
        Extra checks: none
      
      CPU dispatch  :
        Requested   : 'max -xop -fma4'
        Enabled     : ASIMDHP ASIMDDP
        Generated   : none
      CCompilerOpt.cache_flush[809] : write cache to path -> /private/var/folders/tk/m40lxrrx49vf4s05nf9wt5_c0000gn/T/pip-install-r_a3ufrb/scipy_ffa7d0ec687a4ad8bb86f38250ba2117/build/temp.macosx-11.6-arm64-cpython-37/ccompiler_opt_cache_ext.py
      
      ########### CLIB COMPILER OPTIMIZATION ###########
      Platform      :
        Architecture: aarch64
        Compiler    : clang
      
      CPU baseline  :
        Requested   : 'min'
        Enabled     : NEON NEON_FP16 NEON_VFPV4 ASIMD
        Flags       : none
        Extra checks: none
      
      CPU dispatch  :
        Requested   : 'max -xop -fma4'
        Enabled     : ASIMDHP ASIMDDP
        Generated   : none
      CCompilerOpt.cache_flush[809] : write cache to path -> /private/var/folders/tk/m40lxrrx49vf4s05nf9wt5_c0000gn/T/pip-install-r_a3ufrb/scipy_ffa7d0ec687a4ad8bb86f38250ba2117/build/temp.macosx-11.6-arm64-cpython-37/ccompiler_opt_cache_clib.py
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

My versions:

gcc/13.2.0
openblas/0.3.24
pkg-config/0.29.2_3

Cython/0.29.36
pybind11/2.11.1
pythran/0.14.0
numpy/1.21.6

xcode-select version 2399.
Apple clang version 15.0.0 (clang-1500.0.40.1)
Target: arm64-apple-darwin23.0.0

Answer 1

I'm experience exact the same thing as you were. Need to get scipy with python3.7 down into arm64 platform. Same Cython problem if version is over 3. Same clang compile error and everything.

I believe the problem lies in the define statement:

#define isnan(x) ((x) != (x))

This line is meant to shortcut the function isnan(x) to use ((x) != (x)) instead while compiling.

I'm guessing that in a later version of clang, it just doesn't allow the syntax like (xxxx) anymore for some reason. So I guess we should either downgrade clang to a certain version, or add some parameters to bypass that compile check or something. Unluckily I'm not any sort of good at c/c++, but will try what I could later.

Will update here if I figure out anything related.

By the way would you please share any of your progress now?

---

Update:

Just now compiled smoothly with clang 13(llvm@13 via brew). Exact same install command as yours. Just make sure you use a lower version of clang for compiling.

I haven't tried with clang 12 but I'm pretty sure it should be good as well. After all, apple's macOS SDK gotta be clang 12 for macOSX 12 .

(Have to mask my company's pypi source)

---

For anyone who's not so sure about how to install or temporarily use a separate clang for compiling, here's how.

brew install llvm@13

After installation, you should notice from brew that there're some configuration you need to setup. Basically you need to set up PATH to make sure the installed executable binary files are first selected.

export PATH="/opt/homebrew/opt/llvm/bin:$PATH"

I also set LDFLAGS and CPPFLAGS as instructed

export LDFLAGS="-L/opt/homebrew/opt/llvm/lib/c++ -Wl,-rpath,/opt/homebrew/opt/llvm/lib/c++"
export CPPFLAGS="-I/opt/homebrew/opt/llvm/include"

(not very sure if CPPFLAGS is needed, but did it anyway)

Then install pip

OPENBLAS=$(brew --prefix openblas) pip3 install --no-use-pep517 scipy==1.7.3

(LDFLAGS="${LDFLAGS}" is not needed if it is already your env variable.)