Interactively writing/reading from an external program

Question

Backstory: I'm pretty new to Python, but fairly experienced in Perl. I'm trying to diversify my scripting portfolio in the sysadmin-activities area.

I'm attempting to dynamically communicate with an external process from my python script.

What I want to do is:

• Call an executable (lets call it "cli")
• Write a line to cli
• Read the response back internally
• Iterate through the response, one by one, and write another line to the CLI
• Print the response from the cli back to the console

What I'm hoping this will yield is:

(spawn process) /usr/local/bin/cli
-> show listofobjects
<- (read back list of objects internally)
-> (one by one, write a line to the cli for each of the list of objects)
-> get objectname modifiedtime
<- (print response from above command)

Here is the code that I have so far:

import shlex, subprocess, re

clicmd = "/usr/local/bin/cli -s 10.1.213.226 -n Administrator -p password"
cliargs = shlex.split(clicmd)
cliproc = subprocess.Popen(cliargs,
                       stdin=subprocess.PIPE,
                       stdout=subprocess.PIPE,
                       stderr=subprocess.PIPE,
                      )

tmpclicmd = "LIST objects OUTPUT csv NAME"
cliret = cliproc.communicate(input=tmpclicmd)[0]
regex = re.compile('^\".*')

for line in cliret.split('\n'):
    if regex.match(line):
        procline = line.replace('"','')
        if 'NAME' not in procline:
            clideets = subprocess.Popen(cliargs,
                                        stdin=subprocess.PIPE,
                                        stdout=subprocess.PIPE,
                                        stderr=subprocess.PIPE,
                                       )
            clideetscmd = 'modify objects ' + procline
            clideets.communicate(input=clideetscmd)
            clideetscmd = 'list objectdetails'
            clideetsresp = clideets.communicate(input=clideetscmd)[0]

            print clideetsresp;

I'm probably going about this in the completely wrong way. Do I have to spawn a new Popen for every step of the way? How could I do this better? (another module, etc). At the end of the day, I can't have the cli close on me, which Popen seems to do after each step of the way.

Thanks in advance!

Answer 1

1. It is not necessary to start a new process (using Popen) for every interaction. You do though, when you use communicate for sending data to the process, because, as the documentation states:

   Interact with process: Send data to stdin. Read data from stdout and stderr, until end-of-file is reached. Wait for process to terminate.

   Instead, simply write to cliproc.stdin and read from cliproc.stdout:

   cliproc.stdin.write("LIST objects OUTPUT csv NAME")
   cliret = cliproc.stdout.readline()
   

   The process keeps alive this way.

2. I don't know why you use the shlex module here:

   clicmd = "/usr/local/bin/cli -s 10.1.213.226 -n Administrator -p password"
   cliargs = shlex.split(clicmd)
   

   The built-in str.split method will do fine:

   clicmd = "/usr/local/bin/cli -s 10.1.213.226 -n Administrator -p password"
   cliargs = clicmd.split()
   

   Or you can just write the resulting list yourself:

   cliargs = ["/usr/local/bin/cli", "-s", "10.1.213.226",
              "-n", "Administrator", "-p", "password"]
   

3. You don't need a semicolon here:

   print clideetsresp;