pweave not printing value of variable in code chunk

Question

I am trying to use pweave with LaTeX for reproducible research. I am seeing weird behavior trying to display the value of a variable in a code chunk. I hope someone can explain what is happening.

When calling print() within a code chunk, depending on what is to be printed, pweave sometimes produces a verbatim section in the markup file for the code, but not for the output of the code.

The environment is Ubuntu 18.04, python 3.6.8, and python-pweave 0.25-1 from the Ubuntu repositories. (This is the latest version available.)

I have experimented with a number of variations to determine what works and what doesn't. The problem seems to occur, or not, depending on the number of lines or characters that would be printed by the code chunk.

I am using the pweave.weave() function with the noweb format as follows:

pweave.weave( filename, doctype='tex', informat='noweb', output=outfile )

The following code chunk works "correctly".

<<>>=
x = 3.14
print( x )
@

It produces the following in the .tex file:

\begin{verbatim}
x = 3.14
print( x )
\end{verbatim}
\begin{verbatim}
3.14

\end{verbatim}

There are separate verbatim sections for the code chunk and the interpreter output.

On the other hand, this code chunk doesn't work.

<<>>=
x = 3
print( x )
@

It produces the following output:

\begin{verbatim}
x = 3
print( x )
\end{verbatim}

There is no verbatim section for the output of the interpreter.

Another example that works is this.

<<>>=
x = 33
print( x )
print( "Something else" )
@

It produces this output:

\begin{verbatim}
x = 33
print( x )
print( "Something else" )
\end{verbatim}
\begin{verbatim}
33
Something else

\end{verbatim}

Again, there are two separate verbatim sections.

This however does not work.

<<>>=
X = 33
print( x )
@

It produces this output.

\begin{verbatim}
X = 33
print( x )
\end{verbatim}

Answer 1

I am not sure of the protocol for answering my own question, but I believe I now know what is happening.

The python print() function, when writing to a file, uses buffered I/O. Output is written to a buffer in memory, and only written to the file when the number of bytes in the buffer reaches a certain threshold, the program terminates, or the buffer is flushed. Apparently, the buffer is not being flushed at the end of each chunk before pweave collects the output from the file where it is being accumulated.

One could argue that this is not strictly a bug in pweave because this is the documented behavior of print(). Especially since reaching the end of a chunk is not terminating the program. But it would certainly seem that flushing the output buffer at the end of each chunk would be the most appropriate behavior.

A simple work-around is to call sys.stdout.flush() as the last statement in each chunk that calls print(). This ensures that all the output is written to the file before the chunk ends. One can also include the parameter flush=True in the last print call in each chunk.