Shell Script to Validate Filename

Question

I have a requirement of creating a script to validate the filename. There will be two files in the source from where Informatica job start picking up these files. One will be 'In progress File (Current day : Sysdate)' and the other will be Completed/ Closed File (previous day : i.e. with a file name aligned to Sysdate -1). Now ETL has to pick up all the files except In progress File ( File date is less than Sysdate). Assuming current day is “22/06/2015” – the following is the list of files that ETL can expect to see in the source folder

filename_22/06/2015_ready.csv – this file should be ignored filename_21/06/2015_ready.csv – this file should be processed by ETL

Please help me how I can write the shell script to fulfill this requirement.

Answer 1

Try the following:

#!/bin/bash
    path="PathToYourFiles";
    `cd $path`
    for i in `ls *.csv`;
     do
            filedate=`echo "$i" |grep -Eo '[[:digit:]]{8}'`
            #echo "Filedate => $filedate";
            today=`date '+%d%m%Y'`;
            yesterday=`date -d "1 day ago" '+%d%m%Y'`;
            case  "$filedate" in
                $today )
                    echo "ignoring file its in progres";
                    ;;
                $yesterday )
                    echo "proces the file";
                    ;;
                *)
                    echo "invalid file to process";
                    ##send it to ETL
                    ;;

                    #echo "today => $today ,yesterday => $yesterday"
            esac
    done

use this script and do changes as per your requirement i have provided the comments and debugging code also.

In above code add the command to send yesterdays file to ETL in $yesterday case. I have considered date in ddmmyyy format please do change the filename accordingly.

Answer 2

You should better use find instead of regular expressions reading the filename.

Commands like these should give you the files written the day before:

# print a list of all files older than one day expressed in minutes (60*24) 
find . -type f -mmin +$((60*24)) -exec ls -halt {} +
# print a list of all files older than one day, expressed in multiple of 86400 seconds
find . -type f -mtime +1 -exec ls -halt {} +