Unable to get intermediate output from NSTask's stdout?

Question

I have written an NSTask async exec method for a simple python script.

When then python script just prints to stdout, all is fine. When there is a raw_input in there (expecting input from the user), it sure gets the input fine, but it does NOT print the data BEFORE raw_input.

What's going on?

- (NSString*)exec:(NSArray *)args environment:(NSDictionary*)env action:(void (^)(NSString*))action completed:(void (^)(NSString*))completed
{
    _task                           = [NSTask new];
    _output                         = [NSPipe new];
    _error                          = [NSPipe new];
    _input                          = [NSPipe new];
    NSFileHandle* outputF           = [_output fileHandleForReading];
    NSFileHandle* errorF            = [_error fileHandleForReading];
    NSFileHandle* inputF            = [_input fileHandleForWriting];

    __block NSString* fullOutput    = @"";

    NSMutableDictionary* envs = [NSMutableDictionary dictionary];

    NSArray* newArgs = @[@"bash",@"-c"];

    [_task setLaunchPath:@"/usr/bin/env"];
    if (env)
        for (NSString* key in env)
            envs[key] = env[key];

    if ([envs count]) [_task setEnvironment:envs];

    NSString* cmd = @"";

    cmd = [cmd stringByAppendingString:[[[self sanitizedArgs:args] componentsJoinedByString:@" "] stringByAppendingString:@" && echo \":::::$PWD:::::\""]];

    [_task setArguments:[newArgs arrayByAddingObject:cmd]];
    [_task setStandardOutput:_output];
    [_task setStandardError:_error];
    [_task setStandardInput:_input];

    void (^outputter)(NSFileHandle*) = ^(NSFileHandle *file){
        NSData *data = [file availableData];
        NSString* str = [[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding];

        NSLog(@"Output: %@",str);

        action(str);
        fullOutput = [fullOutput stringByAppendingString:str];
    };

    void (^inputter)(NSFileHandle*) = ^(NSFileHandle *file) {
        NSLog(@"In inputter");
        NSData *data = [[_task.standardOutput fileHandleForReading] availableData];
        NSString* str = [[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding];

        NSLog(@"Output: %@",str);
    };

    [outputF setReadabilityHandler:outputter];
    [errorF setReadabilityHandler:outputter];
    //[inputF setWriteabilityHandler:inputter];

    [_task setTerminationHandler:^(NSTask* task){
        NSLog(@"Terminated: %@",fullOutput);
        completed(fullOutput);

        //[task.standardOutput fileHandleForReading].readabilityHandler = nil;
        //[task.standardError fileHandleForReading].readabilityHandler = nil;
        //[task.standardInput fileHandleForWriting].writeabilityHandler = nil;
        //[task terminate];
        //task = nil;
    }];

    [_task launch];

    //[[_input fileHandleForWriting] waitForDataInBackgroundAndNotify];

    return @"";
}

---

P.S. I've searched everywhere for a solution, but didn't seem to spot anything. It looks like there are tons of NSTask walkthroughs and tutorials, but - funny coincidence - they usually avoid dealing with any of the stdin implications

Answer 1

This doesn't have anything to do with the parent process (the one with the NSTask object). It's all about the behavior of the child process. The child process is literally not writing the bytes to the pipe (yet).

From the stdio man page:

Initially, the standard error stream is unbuffered; the standard input and output streams are fully buffered if and only if the streams do not refer to an interactive or “terminal” device, as determined by the isatty(3) function. In fact, all freshly-opened streams that refer to terminal devices default to line buffering, and pending output to such streams is written automatically whenever such an input stream is read. Note that this applies only to ``true reads''; if the read request can be satisfied by existing buffered data, no automatic flush will occur. In these cases, or when a large amount of computation is done after printing part of a line on an output terminal, it is necessary to fflush(3) the standard output before going off and computing so that the output will appear. Alternatively, these defaults may be modified via the setvbuf(3) function.

In your case, the standard input and output do not, in fact, refer to an interactive/terminal device. So, standard output is fully buffered (a.k.a. block buffered, as opposed to line buffered or unbuffered). It is only flushed when the buffer internal to the stdio library is full, when the stream is closed, or when fflush() is called on the stream.

The behavior you're expecting, where standard output is flushed automatically when input is read, only happens in the case where the streams are connected to an interactive/terminal device.

There is no way for the parent process to influence this buffering behavior of the child process except by using a pseudo-terminal device rather than a pipe for the input and output. However, that's a complex and error-prone undertaking. Other than that, the child process has to be coded to set the buffering mode of standard output or flush it regularly.