Building Singularity recipe from Nipype docker image CommandNotFound

Question

I have the following Singularity container recipe:

#!/bin/bash

Bootstrap: docker
From: nipype/nipype:latest

%labels
  Version v1.0

%post
  # Install nano
  apt-get update
  apt-get install nano

  # Set up Python environment
  CONDA_ENV=/opt/conda/bin
  export PATH=$CONDA_ENV:$PATH
  chmod -R 777 $CONDA_ENV

  # Activate conda environment
  conda activate neuro
  conda install seaborn
  pip install pybids

and I build the container with Singularity as follows:

sudo singularity build swish.simg Singularity.swish

The install of dependencies and majority of build goes OK until I hit the error that source not found. To reiterate the issue and what I've tried:

• I'm building a Nipype image from the recipe. Within %post, I'd like to install two additional packages (seaborn and pybids) into the "neuro" conda environment.
• However, when I try to activate the neuro environment within %post ("source activate neuro"), I keep getting an error message saying that the command "source" is not found.
• I'd like to run the commands in %post with bash, but am not sure where to specify it.

Answer 1

The issue comes down to the way you are activating the environment. Normally, you would do:

source /opt/conda/bin/activate neuro

but in the case that the Singularity container post is building in a shell (sh) environment, the expectation is that you won't find the source command. Instead you want to do:

. /opt/conda/bin/activate neuro

and then you don't need to fuss with the $PATH. You also don't need to specify an interpreter at the top of the file. So the entire recipe should look like:

Bootstrap: docker
From: nipype/nipype:latest

# This is the adjusted (fixed) build recipe for the issue above.
# sudo singularity build swist Singularity.swist

%labels
    Version v1.0

%environment
    . /opt/conda/bin/activate neuro

%post
    # Install nano
    apt-get update && apt-get install -y nano

    # Install into conda environment
    . /opt/conda/bin/activate neuro &&
    /opt/conda/bin/conda install --name neuro -y seaborn &&
    /opt/conda/envs/neuro/bin/pip install pybids

And then usage would be:

sudo singularity build swist Singularity.swist

singularity/swist_fmri_image>   . /opt/conda/bin/activate neuro

(neuro) Singularity swist:~/swist-> python
Python 3.6.5 | packaged by conda-forge | (default, Apr  6 2018, 13:39:56) 
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import seaborn
>>> import bids
>>>

I've put the entire record of this in a gist

Helpful Debugging Tips

Here are a few helpful debugging tips! The way I figured out the above was to do a build commenting out the last lines with conda that triggered the error. Then I could build a writable sandbox:

sudo singularity build --sandbox swist-box Singularity.swist

and shell in with writable. This wiil allow me to make changes and test.

sudo singularity shell --writable swist-box
$ whoami
root

Since the container would have writable, this means that changes would persist, so you can exit from being root, and then edit in user space to test if your root changes indeed fixed the issue!

singularity shell swist-box
$ whoami
neuro

Then when you think all is well, delete the image and build from scratch and test.

rm -rf swist-box swist
sudo singularity build swist Singularity.swist