Why trying to build this python package returns "Backend 'poetry.core.masonry.api' is not available"

Question

Unfortunately the Python package DocArray is not available in the Nix package collection. Therefore I try to build it myself.

This website helped me to use pkgs.python3Packages.buildPythonPackage and I got the sha256 hash in this website.

{
  description = "Python environment with ollama";

  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
  inputs.flake-utils.url = "github:numtide/flake-utils";

  outputs = { self, nixpkgs, flake-utils }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = nixpkgs.legacyPackages.${system};
        python = pkgs.python311;
        ollama = pkgs.ollama;
        docarray = pkgs.python3Packages.buildPythonPackage rec {
          pname = "docarray";
          version = "0.40.0";
          format = "pyproject"; # Specify the package format
          src = pkgs.fetchPypi {
            inherit pname version;
            sha256 = "c3f3b88b7b6128c10308dddbd21650c9845e270da850cf6718cb1d3d867d5986"; # TODO
      };
          #code proposed by phind (something like chatgpt) but it didn't help. Therefore it is commented
          # Add Poetry to the build inputs
          #buildInputs = [ pkgs.poetry ];
          # Add any runtime dependencies here
          #propagatedBuildInputs = [];
          # Add any Python package metadata here
          #meta = with pkgs.lib; {
          #  homepage = "https://github.com/docarray/docarray";
          #  description = "A Python package for managing document arrays";
          #  license = licenses.mit;
          #};
        };

        Py = python.withPackages (ps: with ps; [
            langchain-community
            langchainplus-sdk
            langchain
            tiktoken
            docarray
        ]);
      in {
        devShells.default = pkgs.mkShell {
          buildInputs = [
            pkgs.curl
            ollama
            Py
          ];
        };
      });
}

After running nix develop I got this error message:

error: builder for '/nix/store/0dq2bsq9zkgmp845l2ncqkk2i9p1pqlm-python3.11-docarray-0.40.0.drv' failed with exit code 1;
       last 10 log lines:
       >   File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
       >   File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
       >   File "<frozen importlib._bootstrap>", line 1126, in _find_and_load_unlocked
       >   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
       >   File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
       >   File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
       >   File "<frozen importlib._bootstrap>", line 1140, in _find_and_load_unlocked
       > ModuleNotFoundError: No module named 'poetry'
       >
       > ERROR Backend 'poetry.core.masonry.api' is not available.
       For full logs, run 'nix-store -l /nix/store/0dq2bsq9zkgmp845l2ncqkk2i9p1pqlm-python3.11-docarray-0.40.0.drv'. error: 1 dependencies of derivation '/nix/store/rqg7i9w02fmbxqf2c4i3gz16icsx7q68-python3-3.11.8-env.drv' failed to build error: 1 dependencies of derivation '/nix/store/qxx83jf2n4l8zchdwagg6wsk4pz439v7-nix-shell-env.drv' failed to build

Answer 1

You need to add poetry-core to the build inputs (nativeBuildInputs) to install Poetry in the build environment. Then you must specify the package dependencies in propagatedBuildInputs:

{
  description = "Python environment with ollama";

  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
  inputs.flake-utils.url = "github:numtide/flake-utils";

  outputs = {
    self,
    nixpkgs,
    flake-utils,
  }:
    flake-utils.lib.eachDefaultSystem (system: let
      pkgs = nixpkgs.legacyPackages.${system};
      docarray = with pkgs.python3Packages;
        buildPythonPackage rec {
          pname = "docarray";
          version = "0.40.0";
          format = "pyproject";

          src = pkgs.fetchPypi {
            inherit pname version;
            sha256 = "c3f3b88b7b6128c10308dddbd21650c9845e270da850cf6718cb1d3d867d5986";
          };

          nativeBuildInputs = [
            poetry-core
          ];

          propagatedBuildInputs = [
            numpy
            orjson
            pydantic
            rich
            types-requests
            typing-inspect
          ];
        };

      pythonEnv = pkgs:
        pkgs.python3.withPackages (ps:
          with ps; [
            langchain-community
            langchainplus-sdk
            langchain
            tiktoken
            docarray
          ]);
    in {
      devShells.default = pkgs.mkShell {
        buildInputs = with pkgs; [
          curl
          ollama

          (pythonEnv pkgs)
        ];
      };
    });
}

An alternative would be to use poetry2nix to avoid specifying the dependencies manually. However, since this package has only a few dependencies (and they are already packaged in nixpkgs), it seems like overkill to me.