How to PIPE the output from a subprocess.Popen, while communicating to it at later point?

Question

Okay, here's what I need to do. I'm trying to PIPE the output from Popen, to display it in real time. However, I need to communicate with my process while it is running to send inputs.

The 1st string in the list exec_str_list contains the 1st shell command which needs to be executed, and the subsequent strings contain the inputs which need to be passed on to the same process. It contains two strings as of now, however to generalise the question lets assume it contains more, in the sequence which needs to be executed.

So, to recap here's what I need to happen, 1st string is executed, and I get the real time output from it, which I need to display. Then when the process expects an input from the user, the 2nd string needs to be passed on to the process, while displaying whatever is happening on the console in real time.

And here's my code:

from subprocess import Popen, PIPE, STDOUT

exec_once = True
for str_to_exec in exec_str_list:
    if exec_once:
        proc = Popen(str_to_exec.split(), stdin=PIPE, stdout=PIPE, stderr=STDOUT)
        exec_once = False
    else:
        proc = proc.communicate(str_to_exec)

    for c in iter(lambda: proc.stdout.read(1), ''):
        publish_in_realtime(key, c, user=session.user)
        console_dump += c

Answer 1

Popen.communicate() is a helper method that does a one-time write of data. (Even if you tried to use its timeout, you'd get "Cannot send input after starting communication" on second use.)

Instead, directly read()/write() to stdin/stdout:

import subprocess
from subprocess import PIPE, STDOUT, Popen

exec_str_list = [
    "zork.exe",
    "north",
    "east",
    "north",
]

proc = Popen(
    exec_str_list[0].split(), stdin=PIPE, stdout=PIPE, stderr=STDOUT, text=True
)
for str_to_exec in exec_str_list[1:]:
    proc.stdin.write(str_to_exec)

    # Assumes program separates its outputs with newlines and only has one
    # output per input. If not, then you need a protocol: The program could
    # output the number of lines it will write followed by those lines.
    outs = proc.stdout.readline()

    print(outs)  # a string because we passed text=True

    if proc.returncode:
        break