Access sub-process controlling terminal from Java process

Question

I have a long-running Java server application that launches a sub-process to perform a particular task (in this case, extract the contents of a 7zip file using the 7z command-line utility, but that particular detail shouldn't be relevant here).

• The server application runs with Java 8 under Ubuntu 14.
• The sub-process is being launched via the Java ProcessBuilder API.
• The file being accessed by the sub-process might be password-protected.
• If the file is password-protected and no password is supplied as a command-line argument, the 7z program will attempt to display a message to the terminal prompting for a password, and then read the password from the terminal.
• At this point, the sub-process hangs and will not complete unless I hit <Enter> twice in the terminal that controls the Java process.

The underlying issue seems to be that the 7z utility uses the getpass system call to display the prompt and read user input. According to the docs:

The getpass() function opens /dev/tty (the controlling terminal of the process), outputs the string prompt, turns off echoing, reads one line (the "password"), restores the terminal state and closes /dev/tty again.

Since this is a server application, it is not acceptable to have the sub-process block waiting on input from the Java terminal. What I would like to happen instead is to programmatically shut down input on the sub-process terminal, so that the getpass call returns immediately with no input and the sub-process exits with an error code. Unfortunately all of my attempts at shutting down input to the sub-process terminal have resulted in the same behavior as above:

• I tried manually closing the stream returned by Process.getOutputStream() immediately after starting the sub-process.
• I tried using ProcessBuilder.redirectInput to redirect sub-process input to read from an empty file, and from /dev/null.
• For good measure, I even tried passing Redirect.INHERIT to ProcessBuilder even though that doesn't seem like it's remotely what I want.

It seems like this should be feasible, because I've observed the desired behavior when 7z is launched with the exact same command-line as a sub-process of a daemonized socat process where the terminal input is connected to /dev/null, but I'm at a loss for how to accomplish this in Java using the tools at my disposal.

Answer 1

In order to preclude that the getpass() function opens /dev/tty (the controlling terminal of the process), we have to arrange that the utility has no controlling tty, which is achieved by calling setsid(). This C program (let's name it notty) makes this call and then executes the given utility:

int main(int argc, char *argv[])
{
    setsid();                   // get rid of the controlling tty
    close(0);                   // preclude reading STDIN as well
    execvp(argv[1], argv+1);    // execute the given program file
}

We can use it from the Java application by prepending it to the ProcessBuilder command, e. g.:

        ProcessBuilder pb = new ProcessBuilder("notty", "7z", … /*arguments*/);