TechQA.

Failed to install llama-cpp-python with Metal on M2 Ultra

837 views Asked by At

I followed the instruction on https://llama-cpp-python.readthedocs.io/en/latest/install/macos/.

My macOS version is Sonoma 14.4, and xcode-select is already installed (version: 15.3.0.0.1.1708646388).

I created a conda environment with
"conda version : 24.1.2"
"python version : 3.10.13.final.0"
"platform : osx-arm64".

And I run the following command to install llama-cpp-python

CMAKE_ARGS="-DLLAMA_METAL=on" pip install -U llama-cpp-python --no-cache-dir

, but it ends up with the following error message.

Collecting llama-cpp-python
  Downloading llama_cpp_python-0.2.57.tar.gz (36.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 36.9/36.9 MB 34.2 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: typing-extensions>=4.5.0 in /Users/mjchoi/miniforge3/envs/llama/lib/python3.10/site-packages (from llama-cpp-python) (4.8.0)
Requirement already satisfied: numpy>=1.20.0 in /Users/mjchoi/miniforge3/envs/llama/lib/python3.10/site-packages (from llama-cpp-python) (1.26.1)
Collecting diskcache>=5.6.1 (from llama-cpp-python)
  Downloading diskcache-5.6.3-py3-none-any.whl.metadata (20 kB)
Requirement already satisfied: jinja2>=2.11.3 in /Users/mjchoi/miniforge3/envs/llama/lib/python3.10/site-packages (from llama-cpp-python) (3.1.2)
Requirement already satisfied: MarkupSafe>=2.0 in /Users/mjchoi/miniforge3/envs/llama/lib/python3.10/site-packages (from jinja2>=2.11.3->llama-cpp-python) (2.1.3)
Downloading diskcache-5.6.3-py3-none-any.whl (45 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 45.5/45.5 kB 405.9 MB/s eta 0:00:00
Building wheels for collected packages: llama-cpp-python
  Building wheel for llama-cpp-python (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for llama-cpp-python (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [66 lines of output]
      *** scikit-build-core 0.8.2 using CMake 3.27.7 (wheel)
      *** Configuring CMake...
      2024-03-19 17:44:25,052 - scikit_build_core - WARNING - libdir/ldlibrary: /Users/mjchoi/miniforge3/envs/llama/lib/libpython3.10.a is not a real file!
      2024-03-19 17:44:25,052 - scikit_build_core - WARNING - Can't find a Python library, got libdir=/Users/mjchoi/miniforge3/envs/llama/lib, ldlibrary=libpython3.10.a, multiarch=darwin, masd=None
      loading initial cache file /var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/tmp6kq2va35/build/CMakeInit.txt
      -- The C compiler identification is AppleClang 15.0.0.15000309
      -- The CXX compiler identification is AppleClang 15.0.0.15000309
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- Found Git: /usr/bin/git (found version "2.39.3 (Apple Git-146)")
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
      -- Found Threads: TRUE
      -- Accelerate framework found
      -- Metal framework found
      -- Warning: ccache not found - consider installing it for faster compilation or disable this warning with LLAMA_CCACHE=OFF
      -- CMAKE_SYSTEM_PROCESSOR: arm64
      -- ARM detected
      -- Performing Test COMPILER_SUPPORTS_FP16_FORMAT_I3E
      -- Performing Test COMPILER_SUPPORTS_FP16_FORMAT_I3E - Failed
      CMake Warning (dev) at vendor/llama.cpp/CMakeLists.txt:1218 (install):
        Target llama has RESOURCE files but no RESOURCE DESTINATION.
      This warning is for project developers.  Use -Wno-dev to suppress it.
      
      CMake Warning (dev) at CMakeLists.txt:21 (install):
        Target llama has PUBLIC_HEADER files but no PUBLIC_HEADER DESTINATION.
      This warning is for project developers.  Use -Wno-dev to suppress it.
      
      CMake Warning (dev) at CMakeLists.txt:30 (install):
        Target llama has PUBLIC_HEADER files but no PUBLIC_HEADER DESTINATION.
      This warning is for project developers.  Use -Wno-dev to suppress it.
      
      -- Configuring done (0.4s)
      -- Generating done (0.0s)
      -- Build files have been written to: /var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/tmp6kq2va35/build
      *** Building project with Ninja...
      Change Dir: '/var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/tmp6kq2va35/build'
      
      Run Build Command(s): /opt/homebrew/bin/ninja -v
      [1/25] cd /var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/tmp6kq2va35/build/vendor/llama.cpp && xcrun -sdk macosx metal -O3 -c /var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/tmp6kq2va35/build/bin/ggml-metal.metal -o /var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/tmp6kq2va35/build/bin/ggml-metal.air && xcrun -sdk macosx metallib /var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/tmp6kq2va35/build/bin/ggml-metal.air -o /var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/tmp6kq2va35/build/bin/default.metallib && rm -f /var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/tmp6kq2va35/build/bin/ggml-metal.air && rm -f /var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/tmp6kq2va35/build/bin/ggml-common.h && rm -f /var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/tmp6kq2va35/build/bin/ggml-metal.metal
      FAILED: bin/default.metallib /var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/tmp6kq2va35/build/bin/default.metallib
      cd /var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/tmp6kq2va35/build/vendor/llama.cpp && xcrun -sdk macosx metal -O3 -c /var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/tmp6kq2va35/build/bin/ggml-metal.metal -o /var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/tmp6kq2va35/build/bin/ggml-metal.air && xcrun -sdk macosx metallib /var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/tmp6kq2va35/build/bin/ggml-metal.air -o /var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/tmp6kq2va35/build/bin/default.metallib && rm -f /var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/tmp6kq2va35/build/bin/ggml-metal.air && rm -f /var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/tmp6kq2va35/build/bin/ggml-common.h && rm -f /var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/tmp6kq2va35/build/bin/ggml-metal.metal
      xcrun: error: unable to find utility "metal", not a developer tool or in PATH
      [2/25] cd /private/var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/pip-install-u_i4wi0c/llama-cpp-python_6f999557aa7a4fc790f0ac043d8dc610/vendor/llama.cpp && /opt/homebrew/Cellar/cmake/3.27.7/bin/cmake -DMSVC= -DCMAKE_C_COMPILER_VERSION=15.0.0.15000309 -DCMAKE_C_COMPILER_ID=AppleClang -DCMAKE_VS_PLATFORM_NAME= -DCMAKE_C_COMPILER=/Library/Developer/CommandLineTools/usr/bin/cc -P /private/var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/pip-install-u_i4wi0c/llama-cpp-python_6f999557aa7a4fc790f0ac043d8dc610/vendor/llama.cpp/common/../scripts/gen-build-info-cpp.cmake
      -- Found Git: /usr/bin/git (found version "2.39.3 (Apple Git-146)")
      [3/25] /Library/Developer/CommandLineTools/usr/bin/cc -DACCELERATE_LAPACK_ILP64 -DACCELERATE_NEW_LAPACK -DGGML_SCHED_MAX_COPIES=4 -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_DARWIN_C_SOURCE -D_XOPEN_SOURCE=600 -I/private/var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/pip-install-u_i4wi0c/llama-cpp-python_6f999557aa7a4fc790f0ac043d8dc610/vendor/llama.cpp/. -F/Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/System/Library/Frameworks -O3 -DNDEBUG -std=gnu11 -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk -fPIC -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wunreachable-code-break -Wunreachable-code-return -Wdouble-promotion -MD -MT vendor/llama.cpp/CMakeFiles/ggml.dir/ggml-alloc.c.o -MF vendor/llama.cpp/CMakeFiles/ggml.dir/ggml-alloc.c.o.d -o vendor/llama.cpp/CMakeFiles/ggml.dir/ggml-alloc.c.o -c /private/var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/pip-install-u_i4wi0c/llama-cpp-python_6f999557aa7a4fc790f0ac043d8dc610/vendor/llama.cpp/ggml-alloc.c
      [4/25] /Library/Developer/CommandLineTools/usr/bin/cc -DACCELERATE_LAPACK_ILP64 -DACCELERATE_NEW_LAPACK -DGGML_SCHED_MAX_COPIES=4 -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_DARWIN_C_SOURCE -D_XOPEN_SOURCE=600 -I/private/var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/pip-install-u_i4wi0c/llama-cpp-python_6f999557aa7a4fc790f0ac043d8dc610/vendor/llama.cpp/. -F/Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/System/Library/Frameworks -O3 -DNDEBUG -std=gnu11 -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk -fPIC -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wunreachable-code-break -Wunreachable-code-return -Wdouble-promotion -MD -MT vendor/llama.cpp/CMakeFiles/ggml.dir/ggml-backend.c.o -MF vendor/llama.cpp/CMakeFiles/ggml.dir/ggml-backend.c.o.d -o vendor/llama.cpp/CMakeFiles/ggml.dir/ggml-backend.c.o -c /private/var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/pip-install-u_i4wi0c/llama-cpp-python_6f999557aa7a4fc790f0ac043d8dc610/vendor/llama.cpp/ggml-backend.c
      [5/25] /Library/Developer/CommandLineTools/usr/bin/c++ -DGGML_USE_METAL -DLLAMA_BUILD -DLLAMA_SHARED -I/private/var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/pip-install-u_i4wi0c/llama-cpp-python_6f999557aa7a4fc790f0ac043d8dc610/vendor/llama.cpp/examples/llava/. -I/private/var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/pip-install-u_i4wi0c/llama-cpp-python_6f999557aa7a4fc790f0ac043d8dc610/vendor/llama.cpp/examples/llava/../.. -I/private/var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/pip-install-u_i4wi0c/llama-cpp-python_6f999557aa7a4fc790f0ac043d8dc610/vendor/llama.cpp/examples/llava/../../common -I/private/var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/pip-install-u_i4wi0c/llama-cpp-python_6f999557aa7a4fc790f0ac043d8dc610/vendor/llama.cpp/. -O3 -DNDEBUG -std=gnu++11 -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk -fPIC -Wno-cast-qual -MD -MT vendor/llama.cpp/examples/llava/CMakeFiles/llava.dir/llava.cpp.o -MF vendor/llama.cpp/examples/llava/CMakeFiles/llava.dir/llava.cpp.o.d -o vendor/llama.cpp/examples/llava/CMakeFiles/llava.dir/llava.cpp.o -c /private/var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/pip-install-u_i4wi0c/llama-cpp-python_6f999557aa7a4fc790f0ac043d8dc610/vendor/llama.cpp/examples/llava/llava.cpp
      [6/25] /Library/Developer/CommandLineTools/usr/bin/cc -DACCELERATE_LAPACK_ILP64 -DACCELERATE_NEW_LAPACK -DGGML_SCHED_MAX_COPIES=4 -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_DARWIN_C_SOURCE -D_XOPEN_SOURCE=600 -I/private/var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/pip-install-u_i4wi0c/llama-cpp-python_6f999557aa7a4fc790f0ac043d8dc610/vendor/llama.cpp/. -F/Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/System/Library/Frameworks -O3 -DNDEBUG -std=gnu11 -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk -fPIC -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wunreachable-code-break -Wunreachable-code-return -Wdouble-promotion -MD -MT vendor/llama.cpp/CMakeFiles/ggml.dir/ggml-metal.m.o -MF vendor/llama.cpp/CMakeFiles/ggml.dir/ggml-metal.m.o.d -o vendor/llama.cpp/CMakeFiles/ggml.dir/ggml-metal.m.o -c /private/var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/pip-install-u_i4wi0c/llama-cpp-python_6f999557aa7a4fc790f0ac043d8dc610/vendor/llama.cpp/ggml-metal.m
      [7/25] /Library/Developer/CommandLineTools/usr/bin/cc -DACCELERATE_LAPACK_ILP64 -DACCELERATE_NEW_LAPACK -DGGML_SCHED_MAX_COPIES=4 -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_DARWIN_C_SOURCE -D_XOPEN_SOURCE=600 -I/private/var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/pip-install-u_i4wi0c/llama-cpp-python_6f999557aa7a4fc790f0ac043d8dc610/vendor/llama.cpp/. -F/Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/System/Library/Frameworks -O3 -DNDEBUG -std=gnu11 -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk -fPIC -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wunreachable-code-break -Wunreachable-code-return -Wdouble-promotion -MD -MT vendor/llama.cpp/CMakeFiles/ggml.dir/ggml-quants.c.o -MF vendor/llama.cpp/CMakeFiles/ggml.dir/ggml-quants.c.o.d -o vendor/llama.cpp/CMakeFiles/ggml.dir/ggml-quants.c.o -c /private/var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/pip-install-u_i4wi0c/llama-cpp-python_6f999557aa7a4fc790f0ac043d8dc610/vendor/llama.cpp/ggml-quants.c
      [8/25] /Library/Developer/CommandLineTools/usr/bin/c++ -DACCELERATE_LAPACK_ILP64 -DACCELERATE_NEW_LAPACK -DGGML_SCHED_MAX_COPIES=4 -DGGML_USE_ACCELERATE -DGGML_USE_METAL -DLLAMA_BUILD -DLLAMA_SHARED -D_DARWIN_C_SOURCE -D_XOPEN_SOURCE=600 -Dllama_EXPORTS -I/private/var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/pip-install-u_i4wi0c/llama-cpp-python_6f999557aa7a4fc790f0ac043d8dc610/vendor/llama.cpp/. -F/Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/System/Library/Frameworks -O3 -DNDEBUG -std=gnu++11 -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk -fPIC -Wmissing-declarations -Wmissing-noreturn -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi -MD -MT vendor/llama.cpp/CMakeFiles/llama.dir/unicode.cpp.o -MF vendor/llama.cpp/CMakeFiles/llama.dir/unicode.cpp.o.d -o vendor/llama.cpp/CMakeFiles/llama.dir/unicode.cpp.o -c /private/var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/pip-install-u_i4wi0c/llama-cpp-python_6f999557aa7a4fc790f0ac043d8dc610/vendor/llama.cpp/unicode.cpp
      [9/25] /Library/Developer/CommandLineTools/usr/bin/c++ -DGGML_USE_METAL -DLLAMA_BUILD -DLLAMA_SHARED -I/private/var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/pip-install-u_i4wi0c/llama-cpp-python_6f999557aa7a4fc790f0ac043d8dc610/vendor/llama.cpp/examples/llava/. -I/private/var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/pip-install-u_i4wi0c/llama-cpp-python_6f999557aa7a4fc790f0ac043d8dc610/vendor/llama.cpp/examples/llava/../.. -I/private/var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/pip-install-u_i4wi0c/llama-cpp-python_6f999557aa7a4fc790f0ac043d8dc610/vendor/llama.cpp/examples/llava/../../common -I/private/var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/pip-install-u_i4wi0c/llama-cpp-python_6f999557aa7a4fc790f0ac043d8dc610/vendor/llama.cpp/. -O3 -DNDEBUG -std=gnu++11 -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk -fPIC -Wno-cast-qual -MD -MT vendor/llama.cpp/examples/llava/CMakeFiles/llava.dir/clip.cpp.o -MF vendor/llama.cpp/examples/llava/CMakeFiles/llava.dir/clip.cpp.o.d -o vendor/llama.cpp/examples/llava/CMakeFiles/llava.dir/clip.cpp.o -c /private/var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/pip-install-u_i4wi0c/llama-cpp-python_6f999557aa7a4fc790f0ac043d8dc610/vendor/llama.cpp/examples/llava/clip.cpp
      [10/25] /Library/Developer/CommandLineTools/usr/bin/cc -DACCELERATE_LAPACK_ILP64 -DACCELERATE_NEW_LAPACK -DGGML_SCHED_MAX_COPIES=4 -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_DARWIN_C_SOURCE -D_XOPEN_SOURCE=600 -I/private/var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/pip-install-u_i4wi0c/llama-cpp-python_6f999557aa7a4fc790f0ac043d8dc610/vendor/llama.cpp/. -F/Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/System/Library/Frameworks -O3 -DNDEBUG -std=gnu11 -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk -fPIC -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wunreachable-code-break -Wunreachable-code-return -Wdouble-promotion -MD -MT vendor/llama.cpp/CMakeFiles/ggml.dir/ggml.c.o -MF vendor/llama.cpp/CMakeFiles/ggml.dir/ggml.c.o.d -o vendor/llama.cpp/CMakeFiles/ggml.dir/ggml.c.o -c /private/var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/pip-install-u_i4wi0c/llama-cpp-python_6f999557aa7a4fc790f0ac043d8dc610/vendor/llama.cpp/ggml.c
      [11/25] /Library/Developer/CommandLineTools/usr/bin/c++ -DACCELERATE_LAPACK_ILP64 -DACCELERATE_NEW_LAPACK -DGGML_SCHED_MAX_COPIES=4 -DGGML_USE_ACCELERATE -DGGML_USE_METAL -DLLAMA_BUILD -DLLAMA_SHARED -D_DARWIN_C_SOURCE -D_XOPEN_SOURCE=600 -Dllama_EXPORTS -I/private/var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/pip-install-u_i4wi0c/llama-cpp-python_6f999557aa7a4fc790f0ac043d8dc610/vendor/llama.cpp/. -F/Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/System/Library/Frameworks -O3 -DNDEBUG -std=gnu++11 -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk -fPIC -Wmissing-declarations -Wmissing-noreturn -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi -MD -MT vendor/llama.cpp/CMakeFiles/llama.dir/llama.cpp.o -MF vendor/llama.cpp/CMakeFiles/llama.dir/llama.cpp.o.d -o vendor/llama.cpp/CMakeFiles/llama.dir/llama.cpp.o -c /private/var/folders/kl/zn5jq0yn7fbb5rr67rmyfsrr0000gn/T/pip-install-u_i4wi0c/llama-cpp-python_6f999557aa7a4fc790f0ac043d8dc610/vendor/llama.cpp/llama.cpp
      ninja: build stopped: subcommand failed.
      
      
      *** CMake build failed
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for llama-cpp-python
Failed to build llama-cpp-python
ERROR: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based projects

I tried the following commands instead

export CMAKE_ARGS="-DLLAMA_METAL=on" 
export FORCE_CMAKE=1 
pip install llama-cpp-python --no-cache-dir

, but I just got the same error message.

I also tried an older python version as in the link (3.9.16) but it did not work either.

Can someone please help me on this? Thanks much in advance.

macos metal llama-cpp-python
Original Q&A
2

There are 2 answers

0
ooyeon On

Firstly, I'd like to thank Spo1ler's finding.

I only installed xcode-select not Xcode.app and its path was

/Library/Developer/CommandLineTools

Based on https://github.com/gfx-rs/gfx/issues/2309#issuecomment-506130902, I installed Xcode.app from Apple App Store. Then, I can see that the path of xcode-select is changed automatically to

/Applications/Xcode.app/Contents/Developer

Then, I had to run Xcode.app first to agree to the Xcode license agreements. After that, the following commands work.

export CMAKE_ARGS="-DLLAMA_METAL=on" 
export FORCE_CMAKE=1 
pip install llama-cpp-python --no-cache-dir

But, I got another error message when I call LlamaCpp module as follows.

ggml_metal_init: default.metallib not found, loading from source
ggml_metal_init: GGML_METAL_PATH_RESOURCES = nil
ggml_metal_init: loading '/Users/.../miniforge3/envs/llama/lib/python3.10/site-packages/llama_cpp/ggml-metal.metal'    
ggml_metal_init: error: Error Domain=MTLLibraryErrorDomain Code=3 "program_source:3:10: fatal error: 'ggml-common.h' file not found

This is resolved after I uninstall llama-cpp-python by

pip uninstall llama-cpp-python -y

and install again with the following command with additional option

CMAKE_ARGS="-DLLAMA_METAL_EMBED_LIBRARY=ON -DLLAMA_METAL=on" pip install llama-cpp-python --no-cache-dir

Then, I can confirm that LlamaCpp works well.

0
Chetan Narsude On

The following worked for me.

  1. Install Xcode.app from the appstore
  2. sudo xcode-select -r

Related Questions in MACOS

Related Questions in METAL

Related Questions in LLAMA-CPP-PYTHON

Popular Questions

Popular Tags

javascript python java c# php android html jquery c++ css ios sql mysql r reactjs node.js arrays c asp.net json

Trending Questions