to access the scratch folder of a SLURM cluster node

Question

I would appreciate your suggestions and advise on the following please :

I am using a SLURM cluster and my colleagues have advised to run a singularity container on the cluster, and re-direct the output of the singularity container to a folder that is hosted in the /scratch folder of each computing node.

for example :

singularity exec --bind /local/scratch/bt:/output \
singularity_latest.sif run  \
-o /output

i would like to ask please : how can i access the "output" folder in the "scratch" of the computing node ? Thanks a lot !

bogdan

Answer 1

You can think of --bind as a bit like a symlink. Running ls /local/scratch/bt on the host OS is equivalent to running ls /output inside the exec process.

mkdir scratch
touch scratch/file1

ls -l scratch
# total 0
# -rw-rw-r-- 1 tsnowlan tsnowlan 0 Jun  8 09:13 file1

singularity exec -B $PWD/scratch:/output my_image.sif ls -l /output
# total 0
# -rw-rw-r-- 1 tsnowlan tsnowlan 0 Jun  8 09:13 file1

# singularity also accepts relative paths
singularity exec -B scratch:/output my_image.sif touch /output/file2

ls -l scratch
# total 0
# -rw-rw-r-- 1 tsnowlan tsnowlan 0 Jun  8 09:13 file1
# -rw-rw-r-- 1 tsnowlan tsnowlan 0 Jun  8 09:16 file2