Ubuntu bash script to delete old backup files from multiple directories

Question

This is the structure of my backup:

• Backups are stored to a directory named cron_hourly_backup
• Inside that directory a directory is created each day which is named with ddmmyyyy format.
• In each of these directories there are 5-6 db backups which are dumped every hour through a cron-job, and every hour's backup files have unique name by using time stamp (ex: db1_000000.zip .... db5_000000.zip upto db1_230000.zip ... db5_230000.zip)

Now I want to programmatically delete all backup files older than 1 day (OR, keep today's and yesterday's all backup), But keep one latest db (of all 5 dbs) for each day. How can I achieve this?

Currently I'm doing this:

find . -type f \( -name "*_00*" \
-o -name "*_01*"-o -name "*_02*" \
-o -name "*_03*" -o -name "*_04*" \
-o -name "*_05*" -o -name "*_06*" \
-o -name "*_07*" -o -name "*_08*" \
-o -name "*_09*" -o -name "*_10*" \
-o -name "*_11*" -o -name "*_12*" \
-o -name "*_13*" -o -name "*_14*" \
-o -name "*_14*" -o -name "*_15*" \
-o -name "*_16*" -o -name "*_17*" \
-o -name "*_18*" -o -name "*_19*" \
-o -name "*_20*" -o -name "*_21*" \
-o -name "*_22*"  \) -delete

This works great, problem is

1. if 23rd hour backup is not available for any day, then I will lose all files of that day.
2. It will also delete today's and yesterday's backups.

Any suggestions on how to solve the above 2 issues is much appreciated.

Answer 1

Not sure what "But keep one latest db (of all 5 dbs) for each day" means. If it means "for each day keep only the last (in lexicographic order) file", and if you have the coreutils date utility, a bash script like this could work (not tested):

#/usr/bin/env bash

declare -a l                         # array of backup files
bd=cron_hourly_backup                # backup dir
td=$( date +"%d%m%Y" )               # today
yd=$( date -d yesterday +"%d%m%Y" )  # yesterday
for n in "$bd"/*; do
    if [ ! -d "$n" ]; then
        continue # skip if not a directory
    fi
    if [[ "$n" == "$bd/$td" || "$n" == "$bd/$yd" ]]; then
        continue # skip if today or yesterday
    fi
    l=( $( ls "$n" ) ) # populate array
    # loop over all backup files except the last one
    for (( i = 0; i < ${#l[@]} - 1; i += 1 )); do
        echo "rm -f $n/${l[i]}" # comment when OK
#       rm -f "$n/${l[i]}"      # uncomment when OK
    done
done

If you want to keep the last of each dbN_* with 1<=N<=6, then you can use one more loop level (not tested):

#/usr/bin/env bash

declare -a l                         # array of backup files
bd=cron_hourly_backup                # backup dir
td=$( date +"%d%m%Y" )               # today
yd=$( date -d yesterday +"%d%m%Y" )  # yesterday
for n in "$bd"/*; do
    if [ ! -d "$n" ]; then
        continue # skip if not a directory
    fi
    if [[ "$n" == "$bd/$td" || "$n" == "$bd/$yd" ]]; then
        continue # skip if today or yesterday
    fi
    for (( j = 1; j <= 6; j += 1 )); do
        l=( $( ls "$n/db${j}_"* ) ) # populate array
        # loop over all backup files except the last one
        for (( i = 0; i < ${#l[@]} - 1; i += 1 )); do
            echo "rm -f ${l[i]}" # comment when OK
#           rm -f "${l[i]}"      # uncomment when OK
        done
    done
done