Run python subprocess inside Singularity container

812 views Asked by At

I am trying to run a python subprocess inside Singularity container like this:

singularity exec image_name python python_task.py

And within python_task.py it calls this function with cmd:

def run_shell_cmd(cmd):
    p = subprocess.Popen(
        ['/bin/bash', '-o', 'pipefail'],  # to catch error in pipe
        stdin=subprocess.PIPE,
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
        universal_newlines=True,
        preexec_fn=os.setsid)  # to make a new process with a new PGID
    pid = p.pid
    pgid = os.getpgid(pid)
    log.info('run_shell_cmd: PID={}, PGID={}, CMD={}'.format(pid, pgid, cmd))
    t0 = get_ticks()
    stdout, stderr = p.communicate(cmd)
    rc = p.returncode
    t1 = get_ticks()
    err_str = (
        'PID={pid}, PGID={pgid}, RC={rc}, DURATION_SEC={dur:.1f}\n'
        'STDERR={stde}\nSTDOUT={stdo}'
    ).format(
        pid=pid, pgid=pgid, rc=rc, dur=t1 - t0, stde=stderr.strip(), stdo=stdout.strip()
    )
    
    if rc:
        # kill all child processes
        try:
            os.killpg(pgid, signal.SIGKILL)
        except:
            pass
        finally:
            raise Exception(err_str)
    else:
        log.info(err_str)
    return stdout.strip('\n')

But I am getting this error/exception

Exception: PID=23793, PGID=23793, RC=255, DURATION_SEC=0.1
STDERR=WARNING: Could not lookup the current user's information: user: unknown userid 29931
FATAL:   Couldn't determine user account information: user: unknown userid 29931

Any suggestions on how to correct this? I think userid 29931 is not present inside the container which results into this error. But I am not sure how to fix this.

0

There are 0 answers