Zipping up a Web Root: Why didn't nice reduce CPU load?

Question

nice -n 19 find . -type f \( -iname "*.php" -o -iname "*.js" -o -iname "*.inc" \) \
  -exec zip live.zip '{}' \;

The above command runs on our live CentOS server as though the nice command was absent. After 60 seconds or so I can see that zip is on 'top' when using the top command. The server starts to fall over and I have to crash the command.

Answer 1

nice only sets a process' scheduling priority. It does not limit how much CPU time it consumes.

So: If a low priority process wants to consume a lot of CPU time/resources, then it will get them. Until a process with higher priority comes along to hog CPU time.

The point being: If the CPU doesn't have anything else to do, then why not provide all CPU time to the process that wants it, even if it doesn't have high priority?

If you want to limit CPU usage to a % of the maximum, then consider using something like cpulimit

---

EDIT:

Other reasons why zip might be slowing things down horribly are:

1. Disk I/O: Which you can control with ionice on some distros (not sure if CentOS has it by default) - David Schmitt pointed this out in a comment below.

2. zip might allocate a lot of memory, and swap out other processes. Then when these processes wake up (say mysqld gets a query), they are sluggish. You might be able to do something about this by reducing Swappiness. But that is a system level parameter and you probably want to leave it untouched.

Answer 2

From the comments I infer that your server is low on memory and this is not actually an CPU or I/O or prioritization problem at all.

Try replacing the zipping with a streaming solution like tar. This should reduce the required memory considerably:

find . -type f \( -iname "*.php" -o -iname "*.js" -o -iname "*.inc" \) -print0 \
    | xargs -0 tar cvzf live.tar.gz 

Using nice on this command remains a choice to reduce the impact further. It has to be balanced with possible longer runtimes and the other resources (especially memory) that are used meanwhile.