Python: How to wait until disk activity subsides?

461 views Asked by At

Hegle Jensens' wrote a great SnapBtr script that makes snapshot-based backups with smart algorithm that chooses which old backup(s) to delete, when the free space become scarce.

Unfortunately the BTRFS file system has a peculiarity that after any delete command it doesn't immediately free the disk space; instead it merely schedules the deletion of each node. The actual process of freeing the disk space happens in the background, and only after it is finished, we know how much free space became available.

That's why I'd like to improve this script so after deletion of a spare subvolume it will wait until there will be no hard drive activity in order to get the actual free disk space statistics.

The question: knowing that there is so many Python libraries around, do you know any that would return something, that I can use to get hard drive activity saturation in %?

If that helps, I've already made a Bash script wait-for-disk-idle.sh, which relies on iostat for the disk activity information. But I guess that calling external Bash process for something so simple is quite inefficient and error prone (what if the iostat is not installed?):

#! /bin/bash

USAGE="Usage: `basename $0` [-t sample time] [-p disk IO percent threshold] disk-device"

time=4
percent=10
# Parse command line options.
while getopts ":t:" OPT; do
    case "$OPT" in
        t)
            time=$OPTARG
            ;;
        :)
            # getopts issues an error message
            echo "`basename $0` version 0.1"
            echo $USAGE >&2
            exit 1
            ;;
        \?)
            # getopts issues an error message
            echo "`basename $0` version 0.1"
            echo $USAGE >&2
            exit 1
            ;;
    esac
done
while getopts ":p:" OPT; do
    case "$OPT" in
        p)
            percent=$OPTARG
            ;;
        :)
            ;;
        \?)
            # getopts issues an error message
            echo "`basename $0` version 0.1"
            echo $USAGE >&2
            exit 1
            ;;
    esac
done

# Remove the switches we parsed above.
shift `expr $OPTIND - 1`

# We want at least one non-option argument. 
# Remove this block if you don't need it.
if [ $# -eq 0 ]; then
    # getopts issues an error message
    echo "`basename $0` version 0.1"
    echo $USAGE >&2
    exit 1
fi

echo percent: $percent, time: $time, disk: $1

while [[ $(iostat -d -x $time 2 $1 | 
          sed -n 's/.*[^0-9]\([0-9][0-9]*\)[\.,][^,^\.]*$/\1/p' | tail -1) > $percent 
      ]]; do 
  echo wait
done
1

There are 1 answers

0
Adam Ryczkowski On

Here is an answer I've got from the (former) maintainer of the script:

I am no longer cleaning up using the SnapBtr.py script, however you may be able to wait for the delete to complete with a btrfs filesystem sync.