SLURM squeue format argument fails from subprocess.Popen

Question

I'm trying to call the SLURM squeue from a python script. The command,

/usr/bin/squeue --Format=username,jobid,name,timeleft

Works fine from the command line, but fails from subprocess.Popen with:

    p = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
  File "/n/home00/DilithiumMatrix/.conda/envs/py35/lib/python3.5/subprocess.py", line 947, in __init__
    restore_signals, start_new_session)
  File "/n/home00/DilithiumMatrix/.conda/envs/py35/lib/python3.5/subprocess.py", line 1551, in _execute_child
    raise child_exception_type(errno_num, err_msg)
FileNotFoundError: [Errno 2] No such file or directory: '/usr/bin/squeue --Format=username,jobid,name,timeleft'

MWE:

import subprocess
command = "/usr/bin/squeue --Format=username,jobid,name,timeleft"
p = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
text = p.stdout.read()
print(text)

/usr/bin/squeue works fine from both the command line or Popen.

Could it be failing because it requires some information about the user/group that's executing the squeue command and that is (somehow) lost when running via python? What else could be causing this?

Answer 1

The first argument to subprocess.Popen is either a String, or a list of Strings. If it is a single String, it will be interpreted as a filename. This is the reason for the error you get. To pass a list of Strings, it should match how a shell would pass your arguments to the process. A standard shell will split your command line by whitespace, so instead of this:

command = "/usr/bin/squeue --Format=username,jobid,name,timeleft"

You need this:

command = ["/usr/bin/squeue", "--Format=username,jobid,name,timeleft"]

Splitting the second argument at the "=" as you mentioned in your comment just confuses squeue, which would then see two arguments.