How to grep multiline stack trace? Defunct threads are created with my solution

Question

I have a file (application.log) where my applications store their logs. Sometimes I've got an Exception and I'd like to store only these exceptions in another file (exception.log). Each line of logs starts from the date in such format:

[2017-28-09 10:00:00,000] Text of log number 1
[2017-28-09 10:00:05,000] Text of log number 2
[2017-28-09 10:00:10,000] Text of Exception number 1
 at bla bla bla
 at bla bla bla    
 at bla bla bla    
 at bla bla bla    
[2017-28-09 10:00:15,000] Text of log number 4

In such siutation to exception.log should be stored:

[2017-28-09 10:00:10,000] Text of Exception number 1
 at bla bla bla
 at bla bla bla    
 at bla bla bla    
 at bla bla bla  

I tried with such method:

kill $(ps -ef | grep tail | awk '{print $2}')
tail -f /path/to/my/application.log | pcregrep -M 'Exception[^\[]+' | while read line
do
(echo "$line" >> /path/to/my/exception.log)
done &

This solution successfully does the job but also it created many threads and number of total threads in my system dramtically increased. Because of that I need to use another solution or resolve "defunct issue".

Do you, guys, have any idea how can I cut, grep or copy only Exception stack trace to another file or prevent to not open defunct threads?

Answer 1

$ cat test.txt 
[2017-28-09 10:00:00,000] Text of log number 1
[2017-28-09 10:00:05,000] Text of log number 2
[2017-28-09 10:00:10,000] Text of Exception number 1
 at bla bla bla
 at bla bla bla    
 at bla bla bla    
 at bla bla bla    
[2017-28-09 10:00:15,000] Text of log number 4

$ awk '/\[.*\]/{d=0; if($0 ~ "Exception")d=1}d' test.txt
[2017-28-09 10:00:10,000] Text of Exception number 1
 at bla bla bla
 at bla bla bla    
 at bla bla bla    
 at bla bla bla

Answer 2

Tell grep to print one line before matching a line starting with a space.

grep -B1 '^ '

Answer 3

Below is the @CWLiu solution and kind of solution without "tail -f". Easy but usable and there is none zombie process or tail -f thread hanging out:

#!/bin/bash

# Below I checked if last.log file exists
if [ ! -f /path/to/my/last.log ]; then
   echo "" > /path/to/my/last.log
fi

# Thanks @CWLiu again for your below solution
awk '/\[.*\]/{d=0; if($0 ~ "Exception")d=1}d' /path/to/my/application.log > /path/to/my/now.log

# Creation differential file - now.log stores all exceptions, last.log stores all exceptions from previous check
diff /path/to/my/now.log /path/to/my/last.log > /path/to/my/tmp.log
diff /path/to/my/last.log /path/to/my/now.log | grep ">" | tr -d '>' >> /path/to/my/exception.log

Now I can add execution of this file in crontab to check i.e. every minute if some Exception occurs:

* * * * * /path/to/my/file.sh