heisenbug: pexpect and changing passwords on Mac OS X over ssh

Question

In my organization, we have many macs running OS X Yosemite. Each of these machines has a default account that our IT team can use to access the machine to provide IT assistance. We want to periodically change the password on this account, and, as the number of macs in the organization grows, we want to find a way to automate this task.

I've written a Python script using pexpect that will SSH into each machine and execute dscl to change the login password, then SSH into each machine to run security to change the login keychain password.

These methods are part of a class that has the old and new passwords stored in old_password and new_password attributes:

def _change_login_password(self, host):
    """Change the login password of a machine.

    Returns True on success, False on failure.
    """

    try:
        child = pexpect.spawn(
            "ssh default@{} dscl . passwd /Users/default".format(host))
        child.expect("Password:")
        child.sendline(self.old_password)
        child.expect("New Password: ")
        child.sendline(self.new_password)
        child.expect(
            "Permission denied. Please enter user's old password:")
        child.sendline(self.old_password)
        child.close()

        return not child.exitstatus

    except pexpect.TIMEOUT:
        return False

def _change_keychain_password(self, host, login_password):
    """Change the keychain password of a machine.

    Changes the keychain password to match the login password.

    Returns True on success, False on failure.
    """

    try:
        child = pexpect.spawn(
            "ssh default@{} security set-keychain-password"
            " login.keychain".format(host))
        child.expect("Password:")
        child.sendline(login_password)
        child.expect("Old Password: ")
        child.sendline(self.old_password)
        child.expect("New Password: ")
        child.sendline(login_password)
        child.expect("Retype New Password: ")
        child.sendline(login_password)
        child.close()

        return not child.exitstatus

    except pexpect.TIMEOUT:
        return False

These methods both fail in testing against my OS X Yosemite work laptop. Each command being executed over ssh returns a nonzero exit status, and the passwords on the laptop don't change.

...however, when I insert a pdb breakpoint at the top of either method, and then step through the method in the debugger without making any changes, the exit status becomes 0, and the password on the laptop changes.

Heisenbug.

Inserting print statements around the pexpect calls will sometimes also cause the method to succeed.

I have already experimented with pexepect's delaybeforesend attribute, which adds a delay before sendline sends its payload to the child process, thinking that the time delay in stepping in the debugger might have been the issue, but that did not fix the problem.

Does anyone know where I might look next? Some of my coworkers suspect it might by a tty issue. Does anyone know how pdb might be affecting the environment and causing these commands to succeed, or what about OS X is causing them to fail?

Answer 1

Ahh, so here's a thing. Upon closer examination, both processes were exiting with status code 130, which corresponds to Ctrl+C. I needed to add an expect(pexpect.EOF) into both methods before close, so that they had a chance to finish before being shut off.

def _change_login_password(self, host):
    """Change the login password of a machine.

    Returns True on success, False on failure.
    """

    try:
        child = pexpect.spawn(
            "ssh -ttt default@{} dscl . passwd /Users/default".format(host))
        child.expect("Password:")
        child.sendline(self.old_password)
        child.expect("New Password: ")
        child.sendline(self.new_password)
        child.expect(
            "Permission denied. Please enter user's old password:")
        child.sendline(self.old_password)
        child.expect(pexpect.EOF)  # Wait for EOF.
        child.close()

        return not child.exitstatus

    except pexpect.TIMEOUT:
        return False

    def _change_keychain_password(self, host, login_password):
    """Change the keychain password of a machine.

    Changes the keychain password to match the login password.

    Returns True on success, False on failure.
    """

    try:
        child = pexpect.spawn(
            "ssh -ttt default@{} security set-keychain-password"
            " login.keychain".format(host))
        child.expect("Password:")
        child.sendline(login_password)
        child.expect("Old Password: ")
        child.sendline(self.old_password)
        child.expect("New Password: ")
        child.sendline(login_password)
        child.expect("Retype New Password: ")
        child.sendline(login_password)
        child.expect(pexpect.EOF)  # Wait for EOF.
        child.close()

        return not child.exitstatus

    except pexpect.TIMEOUT:
        return False

I have also, as suggested by robertklep, used ssh -t (actually ssh -ttt, so that I darned well get that tty). The combination of both of these has fixed the issue, and I'm now happily changing passwords over ssh.