Installing python packages with extras using Bazel pip_parse

1k views Asked by At

I use Python Rules for Bazel to build my python projects. I use pip_parse to install pip packages, as described in the guide, but that doesn't seem to be working for packages with extras.

For example, I have the following dependency in my requirements.txt:

ray[air,data,default]==2.1.0

WORKSPACE

pip_parse(
    name = "pip",
    python_interpreter_target = interpreter,
    requirements_lock = "//python:requirements.txt",
)
load("@pip//:requirements.bzl", "install_deps")
install_deps()

BUILD.bazel

load("@pip//:requirements.bzl", "requirement")

py_library(
    name = "lib",
    srcs = glob(["*.py"]),
    deps = [
        requirement("ray"),
    ],
)

py_binary(
    name = "app",
    srcs = ["app.py"],
    deps = [":lib"],
)

When I run py_binary with bazel run :app I see the following error:

File "/private/var/tmp/_bazel_andrii/.../app.py", line 2, in <module>
  import ray
File "/private/var/tmp/_bazel_andrii/...pip_ray/site-packages/ray/__init__.py", line 171, in <module>
  from ray import data  # noqa: E402,F401

ImportError: cannot import name 'data' from partially initialized module 'ray' (most likely due to a circular import)

If I change BUILD.bazel to use requirement as requirement("ray[data]") I see different error:

invalid label '@pip_ray[data]//:pkg' in element 1 of attribute 'deps' in 'py_library' rule: invalid repository name '@pip_ray[data]': workspace names may contain only A-Z, a-z, 0-9, '-', '_' and '.'

How do I install and use pip packages with extras?

0

There are 0 answers