How to capture a DLL's stdout/stderr in Python?

Question

How can you capture a DLL's stdout and/or stderr in Python on Windows? For example, this prints "hello" to stderr, but it should be possible to capture the "hello" as a string instead of printing it:

import ctypes
string = b'hello\n'
ctypes.cdll.msvcrt._write(2, string, len(string))

Here's what doesn't work:

1. Temporarily assigning sys.stderr to a StringIO (or equivalently, using contextlib.redirect_stdout) doesn't capture the output because it's from a C library function, not a Python print statement. This doesn't work on Linux either.
2. Using os.dup2() and reading from a pipe on a separate thread, as suggested here, merely suppresses the output without capturing it.
3. Using ctypes.cdll.kernel32.GetStdHandle() and ctypes.cdll.kernel32.SetStdHandle(), as suggested here, gives the error OSError: [WinError 6] The handle is invalid when attempting to print to the modified stderr.
4. This solution fails because ctypes.util.find_msvcrt() returns None in Python 3.5+, which to my understanding is because Microsoft has transitioned from the Microsoft Visual C++ Runtime (MSVCRT) to the Universal C Runtime (UCRT). Even if I change the line msvcrt = CDLL(ctypes.util.find_msvcrt()) to msvcrt = ctypes.cdll.msvcrt, it merely suppresses the output without capturing it.

My general impression is that solutions that work on Linux don't work on Windows, and solutions that used to work on Windows no longer do because of the transition to the UCRT.

Answer 1

It's not exactly what you're after, but you may be able to wrap your dll calling in subprocess

import subprocess

p = subprocess.Popen(
    ["python3"],
    stdin=subprocess.PIPE,
    stdout=subprocess.PIPE,
    stderr=subprocess.PIPE,
    text=True,  # optional avoids .encode/decode
)

data = r"""
import ctypes
string = b'hello\n'
ctypes.cdll.msvcrt._write(2, string, len(string))
"""

out, err = p.communicate(data)
if p.returncode != 0:
    print("failed!")

print(out)
print(err)

Answer 2

The solution #2 involving reading from a pipe of a duped file descriptor in a separate thread does actually work in the latest versions of Windows.

I've gone ahead to wrap the logics in a friendlier context manager and make it capture the output as bytes instead of string to better suit your test case:

import os
import sys
import threading

class capture_output:
    def __init__(self, fd=sys.stdout.fileno(), chunk_size=1024):
        self.fd = fd
        self.chunk_size = chunk_size
        self.output = b''

    def _capture(self):
        chunks = []
        while chunk := os.read(self._pipe_reader, self.chunk_size):
            chunks.append(chunk)
        self.output = b''.join(chunks)

    def __enter__(self):
        self._duped_fd = os.dup(self.fd)
        self._pipe_reader, pipe_writer = os.pipe()
        os.dup2(pipe_writer, self.fd)
        os.close(pipe_writer)
        self._capture_thread = threading.Thread(target=self._capture)
        self._capture_thread.start()
        return self

    def __exit__(self, exc_type, exc_val, exc_tb):
        os.close(self.fd)
        self._capture_thread.join()
        os.close(self._pipe_reader)
        os.dup2(self._duped_fd, self.fd)
        os.close(self._duped_fd)

so that:

import ctypes

with capture_output() as captured:
    string = b'hello\n'
    ctypes.cdll.msvcrt._write(1, string, len(string))

print(captured.output)

outputs:

b'hello\r\n'