Need wget to write to 2 files

Question

First-time user here.

• Linux spike 4.4.36-server-2.mga5 #1 SMP Tue Dec 6 17:32:56 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

• GNU Wget 1.15 built on linux-gnu

• GNU bash, version 4.3.48(1)-release (x86_64-mageia-linux-gnu)

I've got a cron job that uses wget to download a file. Simple job. I want to see verification of that job in my email. I've been partially successful in getting this to work, but the email shows up with the ouput of cron rather than the wget log information. Here's an example.

The cron job:

/usr/local/bin/update_clouds.sh ; /usr/bin/mail -s "Attempted Cloud Update" -r cron@spike root@shuttle  < /usr/share/xplanet/logs/update_clouds.txt 

The shell script (update_clouds.sh) initialized by this cron job does this:

wget --user=<usr> --password=<passwd> -O /usr/share/xplanet/images/clouds.jpg http://xplanetclouds.com/clouds/2048/clouds_2048.jpg -a /usr/share/xplanet/logs/clouds.log

That works, but my email carries the entire wget log (/usr/share/xplanet/logs/clouds.log) in the body. I just want the last transaction to appear in that email.

Initially, I thought I could use both of wget's logging options, -o and -a. I could append to the system log and overwrite the file I want in the body of my email, but that doesn't work. Wget just uses the last option in the command line, not both.

I did get a solution in which I just arbitrarily pulled the last 30 lines from wget's log and mailed that. It worked, but it's inefficient and ugly. It looks like this:

 /usr/local/bin/update_clouds.sh ; tail -30 /usr/share/xplanet/logs/clouds.log > /usr/share/xplanet/logs/update_clouds.txt ; /usr/bin/mail -s "Attempted Cloud Update" -r cron@spike root@shuttle  < /usr/share/xplanet/logs/update_clouds.txt 

I tried using "tee" but that doesn't work either. It seems as if tee is passed nothing and produces an empty file. I get the email, but there is nothing in the body. I've never used tee before and am not sure I've got a handle on it.

 wget --user=mark9118 --password=xplanetmap -O /usr/share/xplanet/images/clouds.jpg http://xplanetclouds.com/clouds/2048/clouds_2048.jpg -a /usr/share/xplanet/logs/clouds.log 2>&1 |tee /usr/share/xplanet/logs/update_clouds.txt

If you've read this far, you may have noticed that there seem to be many ways to skin this cat. My preference is to have wget download the file and parse out the log file, and then cron to just run the command and send the email.

If you think my approach to this is just wrong, please let me know if there is a simpler, more functional and economical way to do this I'm willing to look at it.

Any help appreciated.

Mark

Answer 1

I'm answering my own question after a couple more hours of trial and error.

What I did was:

1. Abandon wget's native log file function. This got output going to standard out.

2. Redirected standard error and standard out into a text file to be mailed to my admin user.

3. On the same line, I added the command to append the contents of the the text file to be mailed to the log file for update_clouds.

4. Configured the cron job to just run the script and email me.

The command in the shell script looks like this:

 wget --user=xxxxxx --password=xxxxxxxxxx -O /usr/share/xplanet/images/clouds.jpg http://xplanetclouds.com/clouds/2048/clouds_2048.jpg 2> /usr/share/xplanet/logs/update_clouds.txt ; cat /usr/share/xplanet/logs/update_clouds.txt >> usr/share/xplanet/logs/clouds.log

The cron job is this:

 /usr/local/bin/update_clouds.sh ; /usr/bin/mail -s "Updated Clouds" -r "Shuttle Cron<root@shuttle>" root@shuttle < /usr/share/xplanet/logs/update_clouds.txt ; 

I'm sorry to have jostled the community unnecessarily. I appreciate the opportunity to get some help with this stuff. I'm just a part-time hobbyist and I do this stuff to learn. Learned some stuff today for sure.

Mark