Is there a way to capture stdout from this example?

Question

This is for the Windows environment.

Compile the testprocess.py (using pyinstaller) and place the resulting exe in a test folder.

In the same folder, run the ptest.py.

Testprocess.py starts and never ends, writing a number every 3 seconds to stdout.

ptest.py attempts to capture this output.

This code simulates a production issue that I would like to solve. Similar to what happens in production, the stdout is not released to the ptest.py until the testprocess terminates. In production, this process never stops, yet it posts important content to stdout.

Is there a method to accomplish this?

The attached code works fine, as long as the child process terminates.

## [testprocess.py]:

import time

x = 0

while True:
    print(x)
    time.sleep(3)
    x += 1


## [ptest.py]:

import os
import sys
import subprocess

def get_script_path():
    return os.path.dirname(os.path.realpath(sys.argv[0]))

start_dir = get_script_path()

cmd = [start_dir + os.sep + 'testprocess.exe']

proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, encoding='utf-8', universal_newlines=True)

print('Subprocess started.')

capture = ""

s = proc.stdout.read(1)

print('Read Stdout')

while len(s) > 0:
    sys.stdout.write(s)
    sys.stdout.flush()
    capture += s
    s = proc.stdout.read(1)
    print(s)

print(capture)

sys.exit()

Would like to be able to capture the stdout of the child process while it is still running, not waiting until it terminates.

Answer 1

It is possible, and it's easier than you think. Once the subprocess is started, you can continuously try to read from stdout and print it if there's something to print. You'll probably have to modify testprocess.py to flush itself (adding flush = True to the print statements).

p = subprocess.Popen(command, 
                     stdout = subprocess.PIPE, 
                     stderr = subprocess.STDOUT, 
                     encoding='utf-8', 
                     universal_newlines=True)

while True:
    line = p.stdout.readline()
    if line == "" and p.poll() is not None:
        break
    if line:
        print(line.strip(), flush = True)

Edit: If your command looks like python testprocess.py, you can skip having to add the flush = True's to your print statements by instead passing -u as a command option. The -u tells the python interpreter to operate in unbuffered mode.

But, I see your command is actually calling an exe file. You might need to figure out how to tell your compiler how to compile your program as unbuffered.