stdout=Subprocess.PIPE timing differs from stdout=None

20 views Asked by At

I'm trying to create a real-time wrapper around another application (steamcmd, a command line tool to download mods from the steam workshop). When running steamcmd in its own, or through subprocess.Popen(args, stdout=None, creationflags=subprocess.CREATE_NEW_CONSOLE), the output appears in real time (eg. after each downloaded workshop item, it prints the ID of the item). Yet when I create the process with subprocess.Popen(args, stdout=subprocess.PIPE, creationflags=subprocess.CREATE_NEW_CONSOLE) (or other creationflags), I only get big chunks at a time, with predictable delineations (it always hangs at the end of "Waiting for user info...", until all items are downloaded)

I'm using the following code to read from stdout:

while True:
    out = process.stdout.read1()
    print(out.decode("utf-8", errors="ignore"), end="", flush=True)
    return_code = process.poll()
    if return_code is not None:
        break

I've also tried text mode with stdout.read(1) (read a single character at a time), or even stdout.readline (since the chunks are multiple lines) - but all have the same "chunking" behavior.

When I redirect the output of to a file and look at it with tail -f, it is similarly "chunked" - so it seems to be a general difference between "console output" and "pipe output". How could I make Popen receive the output of steamcmd as if it was attached to a console window?

An example command line would be steamcmd.exe +login anonymous +workshop_download_item 281990 899653076 +workshop_download_item 281990 1596617351 +workshop_download_item 281990 1638580371 +workshop_download_item 281990 1724144458 (just some random publicly accessible workshop items items), and the tool can be found here: https://steamcdn-a.akamaihd.net/client/installer/steamcmd.zip

0

There are 0 answers