parsing a tail file to process commands

Question

I have found some info on this, but not enough to get me going with this project. I have limited bash experience, trying to learn.

I have a log file that I would like to read (tail) and process lines as they come in based on timestamp in the log.

Log example: from a tail -f filename.txt | grep --line-buffered RXKEY,MAIN:

20230913200618,12345,RXKEY,MAIN
20230913200620,12345,RXKEY,MAIN
20230913200627,12345,RXKEY,MAIN
20230913200629,12345,RXKEY,MAIN

(Timestamp,id,type,location)

I would like to read this file and perform an action, if the last entry was X amount of seconds before the previous entry. IE, if 2 logs hit in 4 or 5 seconds apart, run a command locally. Then keep monitoring for future logs and repeat as needed.

I have the log working but not sure how to read the timestamps and then process an action.

Answer 1

You can pipe grep into while read, which lets you loop over each line received.

#!/usr/bin/env bash

declare allow_difference=5
declare last=

# 'while read -r' to prevent backslashes from being interpreted
# https://unix.stackexchange.com/questions/192786/what-is-the-meaning-of-read-r
tail -f filename.txt | grep --line-buffered RXKEY,MAIN | while read -r line; do
    # read into array 'linearray', splitting on ',' by setting
    # the IFS (how bash chooses to split word boundaries)
    IFS=',' read -ra linearray <<< "$line"
    # [[ -z "$somevariable" ]] checks if the variable is empty
    # if its empty, then we havent set a last time yet, so set it and continue the while loop
    if [[ -z "$last" ]]; then
        last="${linearray[0]}"
        continue
    fi
    # get the difference between the current time and the last time
    # $(()) for bash arithmetic
    difference="$((linearray[0] - last))"
    # if the difference is smaller than the allowed difference, then run some command
    if ((difference < allow_difference)); then
        echo "difference between ${last} and ${linearray[0]} is ${difference}, running command"
        # could add some other command here...
    fi
    last="${linearray[0]}"
done

Tried to comment more than usual as you said you're new to bash but let me know if something isn't understandable.

 $ ./script 
difference between 20230913200618 and 20230913200620 is 2, running command
^C