macOS launchd fails to redirect stdout to the file after log rotation

Question

I use launchd to configure a service to write logs to the file. This is the launchd config.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>my.label</string>
    <key>ProgramArguments</key>
    <array>
        <string>/path/to/binary</string>
        <string>param</string>
    </array>
    <key>KeepAlive</key>
    <true/>
    <key>StandardErrorPath</key>
    <string>/path/to/logfile.log</string>
    <key>StandardOutPath</key>
    <string>/path/to/logfile.log</string>
    <key>UserName</key>
    <string>root</string>
</dict>
</plist> 

And I use the newsyslog to rotate the logs in the directory above. And the following it the newsyslog config line

/path/to/logfile.log 644 10 10000 * Z

newsyslog is able to rotate the logs. However, after rotation, launchd does not to write to the log file anymore. The last log line in the log file is. newsyslog[2662]: logfile turned over due to -F request. Is that a bug, or there is something wrong with the usage?

Answer 1

As far as I can tell, launchd just hooks your stdout and stderr to those two files directly, it doesn't run them through syslog or anything. Since you didn't specify the pidfile option in your newsyslog conf file, it sends HUP to syslog, which doesn't do anything. And your program is writing to a deleted file which is still growing on disk and can lead to the very surprising behaviour of a full disk filled by an unlinked but not yet deleted file!

AFAICT, there are two options:

1. use the pidfile option, and have your program respond to SIGHUP
2. use syslog for logging instead of writing to stdout

Answer 2

This may be a silly question, but do your scripts stop the service before rotating its logs?

It seems pretty obvious to me that you would need to stop the service before rotating its logs, and then restart the service after.

You might do this tweaking the start/stop times in how you schedule your services.

More robust would be to start and stop them from your log rotation scripts.