RUN pip install -r requirements.txt not working inside a Dockerfile

Question

I am trying to build a new docker image by adding some python packages to my base docker image. The sudo docker build -t myimage1:cuda10.2 . runs without any issues but I cannot import any of the packages in the new image. I am using sudo docker run --gpus all -it myimage1:cuda10.2 to run the image. Can somebody help me understand what am I missing here?

Dockerfile

FROM nvcr.io/nvidia/rapidsai/rapidsai:cuda10.2-runtime-ubuntu18.04

WORKDIR /rapids/notebooks/

COPY requirements.txt ./

RUN pip install --upgrade pip && \
    pip install --no-cache-dir -r requirements.txt

requirements.txt

ujson

This is the trace for docker build

Sending build context to Docker daemon  4.096kB
Step 1/5 : FROM nvcr.io/nvidia/rapidsai/rapidsai:cuda10.2-runtime-ubuntu18.04
 ---> 98901cabda0a
Step 2/5 : WORKDIR /rapids/notebooks/
 ---> Using cache
 ---> 162a9bb732c7
Step 3/5 : COPY requirements.txt ./
 ---> 7bb48a384987
Step 4/5 : RUN pip install --upgrade pip &&     pip install --no-cache-dir -r requirements.txt
 ---> Running in 5f3ca4ed3f93
Collecting pip
  Downloading pip-20.2.4-py2.py3-none-any.whl (1.5 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 20.0.2
    Uninstalling pip-20.0.2:
      Successfully uninstalled pip-20.0.2
Successfully installed pip-20.2.4
Collecting ujson
  Downloading ujson-4.0.1-cp38-cp38-manylinux1_x86_64.whl (181 kB)
Installing collected packages: ujson
Successfully installed ujson-4.0.1
Removing intermediate container 5f3ca4ed3f93
 ---> 58e94a2c4c98
Step 5/5 : COPY . .
 ---> d5897f4da4ff
Successfully built d5897f4da4ff
Successfully tagged myimage1:cuda10.2

If I run the image and do pip install ujson This is what I get

Collecting ujson
  Downloading ujson-4.0.1-cp37-cp37m-manylinux1_x86_64.whl (179 kB)
     |████████████████████████████████| 179 kB 947 kB/s
Installing collected packages: ujson
Successfully installed ujson-4.0.1

The only difference is that the ujson package is cp37 while that installed by Dockerfile is cp38? Can somebody explain why the packages installed are for different Python versions?

Answer 1

In the rapids containers there's a virtual environment named rapids where all of the packages are installed and is activated in the default entrypoint for the container. You should change your RUN command to:

RUN source activate rapids && \
    pip install --upgrade pip && \
    pip install --no-cache-dir -r requirements.txt