Windows/Linux child process STDIN differences

Question

I built a simple text processing script at work to be used by another program. When I was done, someone remembered that the script needs to not block STDIN/STDOUT for the tool using it to work right, and modified the script accordingly. The script opens *nix's cat in a subprocess via IPC::Open2 and prints STDIN to it, reads it back and then processes and prints it to STDOUT. I have no idea how that makes the script non-blocking, but it apparently worked.

I wanted it to work on Windows as well, so I changed out cat for type CON, which is a simple Windows command for printing STDIN. A sample script is below:

use strict;
use warnings;
use IO::Handle;
use IPC::Open2;

my $command = ($^O eq 'MSWin32') ? 'type CON' : 'cat';

my ( $com_reader, $com_writer ) = ( IO::Handle->new, IO::Handle->new );
open2( $com_reader, $com_writer, $command );
# input
while (<STDIN>) {
    print "first line: $_";
    print $com_writer "$_";
    my $line = <$com_reader>;
    # ...process $line...

    print "next line: $line";
}

However the results are completely different. On Windows the STDIN streams for the main script and in the child script seem to be different, while on Linux they are the same. On Windows (I type 1 and 2 on separate lines of input):

>perl test.pl
>1
first line: 1
>2
next line: 2
>1
>2
first line: 2
next line: 1
>1
>2
first line: 2
next line: 1
>1
>2
first line: 2
next line: 1

On Linux (same input):

>perl test.pl
>1
first line: 1
next line: 1
>2
first line: 2
next line: 2
>1
first line: 1
next line: 1
>2
first line: 2
next line: 2

Why is the output different, and how can I make the Windows behavior match the Linux behavior? Also, why does this "open cat in subprocess and pipe input through it" trick work at all?

Answer 1

This isn't a Windows verus Linux thing. You simply picked two awful examples.

1. type con reads from the console, not from STDIN. This can be seen using type con <nul.

2. cat is extremely unusual. Buffering, on either system, is completely up to the individual application, but almost all applications work the same way, and it's different than how cat works. cat goes out of its way to make this very scenario work.

Replace cat with perl -pe1 to see the behaviour of virtually every other program:

1
first line: 1
<deadlock>

The way to convince those "normal" programs to line-buffer rather than block-buffer their output is to create a pseudo-tty. This is what Expect and unbuffer does, for example. This, of course, won't work in Windows. I'm not sure on what Windows programs base their decision to buffer, but I don't think it can be easily faked because I've never heard of a way to do it.