How to run vimdiff from a python daemon

Question

I've been having some problems with vimdiff, python and daemons.

The thing is, I cannot use vimdiff with python when running the program as a daemon, I don't know what happens it just doesn't generate the diff.html file. It's not a permission issue. My solution for this problem was to add a & to the end of the command, but the diff.html has not the usual vimdiff highlight, and I really want it to output with the highlight.

Does anyone knows how to run the vimdiff from python without loosing it's properties?

I'm using the daemon.DaemonContext() function from the python-daemon library to run the program as a daemon.

Snippet:

from os import system

def generate_diff(old_file, new_file):
        system("vimdiff "+ old_file +" "+ new_file +" -c TOhtml -c 'w! diff.html' -c 'qa!' &")

Answer 1

When you run things are a daemon process the environments are not set the same way as when you run in a terminal.

Your home .profile is not executed for examples. The home directory will not be set the same way, so tools like vim or vimdiff will not read in their .vimrc so all those personalized configurations will not be there.

So odds are the PATH is not setup to the point where it will find the command (aka /usr/bin is not in there).

Easy to test, create a script that will print out the env to a log file or something at call it instead.

So use the full path for the command you are trying to use. Or create a script that will setup the environment with everything your command needs and call it instead.

PS: May I also suggest using Python subprocess module instead of building a command line like you are doing, especially since you are running it in the background. https://docs.python.org/3/library/subprocess.html

Security consideration: I very much hope the old_file and new_file are not arguments controlled by a user. The daemon process is most likely running as root, and if you run your command as is, any command could be passed in as "file names" and run on your system as a privileged user. This is very dangerous. at least put single quotes around it so that the string is not interpreted. and validate that the values are valid existing files before calling your command.

Answer 2

As mentioned in @Rob's answer, you should use the subprocess library to call vimdiff directly and control the arguments instead of having the shell interpret them. (This is not the reason why it's not working for you, but the solution involves adopting subprocess to control the streams passed to the spawned vimdiff, so let's adopt it first.)

You can use subprocess to run the equivalent command with:

subprocess.check_call(
    [
        "vimdiff",
        old_file,
        new_file,
        "-c",
        "TOhtml | w! diff.html | qa!"
    ],
)

Note that I merged the many Vimscript commands passed as separate -cs into a single one using the Vimscript command separator of |, which works well in this case.

Now, this will still fail under daemon.DaemonContext(), it will generate a diff.html that doesn't seem to include the old and new files (for the most part, an empty html template.)

In order to solve this, you can explicitly redirect the streams (standard input, standard output and standard error) to /dev/null, so that vimdiff will not be waiting for user input (or trying to display output) and get stuck in trying to do so.

You can easily do that using subprocess passing additional arguments to the call:

subprocess.check_call(
    [
        "vimdiff",
        old_file,
        new_file,
        "-c",
        "TOhtml | w! diff.html | qa!"
    ],
    stdin=subprocess.DEVNULL,
    stdout=subprocess.DEVNULL,
    stderr=subprocess.DEVNULL,
)

Using this snippet, I got to produce a diff.html that was very similar to the one I produced while executing the steps manually with vimdiff. The only difference was that the one I produced had additional spaces at the end of each line in the old and new files.

It turns out that this happened because, when used interactively, vimdiff was detecting the size of my terminal and using that specific number of columns in the display, while the same was no longer possible when the streams were redirected to /dev/null and it wouldn't have access to the terminal to find its settings.

If you wish to change that, you can easily work around it by setting the 'columns' option explicitly in your Vimscript command. For example, for a width of 160 columns:

subprocess.check_call(
    [
        "vimdiff",
        old_file,
        new_file,
        "-c",
        "set columns=160 | TOhtml | w! diff.html | qa!"
    ],
    stdin=subprocess.DEVNULL,
    stdout=subprocess.DEVNULL,
    stderr=subprocess.DEVNULL,
)

One final obstacle I found while testing this was that daemon.DaemonContext() will change the current directory to / (that's usually part of what setting up a daemon context entails), so trying to write to a diff.html in the current directory will most likely fail (unless you have root privileges, and then it will write to the root directory, which is most probably not what you want.)

You can fix that by either calling os.chdir() inside the daemon context to change to the directory where you want the diff.html file produced, or by passing the path to that directory as a named cwd=... argument to subprocess.check_call().