Applying AWK on Set of Files with Same Extension

Question

I want to apply the following "awk" command on files with extension "*.txt"

awk '$4 ~ /NM/{ sum += $2 } END{ print sum }'

But why this command doesn't work:

for i in *.txt do echo awk '$4 ~ /NM/{ sum += $2 } END{ print sum }' $i; done

Normally,

awk '$4 ~ /NM/{ sum += $2 } END{ print sum }' file1.txt

would work.

Answer 1

Once you've removed the echo it should work:

for i in *.txt do awk '$4 ~ /NM/{ sum += $2 } END{ print sum }' $i; done

It'll fail if there are any text files with spaces in them, you could try this:

find . -name '*.txt' -print0 | xargs --null -n 1 awk '$4 ~ /NM/{ sum += $2 } END{ print sum }'

An alternative for printing out names:

find . -name '*.txt' -print -exec awk '$4 ~ /NM/{ sum += $2 } END{ print sum }' {} \;

(Basically make find execute awk directly, so and also print out the file names.

Answer 2

Not sure if you've copy pasted or it's a typo.

for i in *.txt do echo awk '$4 ~ /NM/{ sum += $2 } END{ print sum }' $i; done

With echo corrected, the command above will echo your awk script and the filename, but not run it.

Answer 3

echo is not required.

try

for i in *.txt; do; awk '$4 ~ /NM/{ sum += $2 } END{ print sum }' $i; done

or

for i in *.txt; do awk '$4 ~ /NM/{ sum += $2 } END{ print sum }' $i; done

should work

Answer 4

for i in *.txt; do echo "$i"; awk '$4 ~ /NM/{ sum += $2 } END{ print sum }' "$i"; done

This will print the names of the processed files together with the output of the awk command.

Answer 5

Try this (use nawk or /usr/xpg4/bin/awk on Solaris):

awk 'END {
  printf "%s: %.2f\n", fn, sum
  }
FNR == 1 {
  if (fn) printf "%s: %.2f\n", fn, sum
  fn = FILENAME
  sum = 0
  }
$4 ~ /NM/ { 
  sum += $2 
  }' *.txt 

Answer 6

You need to add a ';' :

for i in *.txt; do ...

instead of

for i in *.txt do ...