Delete first line of all files in a folder (on ubuntu)

Question

I have a folder that has 2800 .txt files in it and I need to delete the first line of every files. The names of the files are all different except for that fact that they end with .txt .

would it be possible to do that while keeping the same file name (instead of sending the output (file without the first line) to another file)...

Answer 1

You could do a bash script. Something like this:

#!/bin/bash
for filename in *; 
do 
    tail -n +2 "${filename}"
done

The run it from command line: $ <script_file.sh>

Take this with a grain of salt. I'm not actually running on a *nix machine. See here for a variety of ways of deleting the first line of a file. Also note that tail is supposed to be much faster than sed, if performance is important to you.

Answer 2

something like this does the trick

sed -i '1d' *.txt

where -i is inplace editing

edit: adition

pls try also this

time sed -i '1d' *.txt

and compare with other solutions (just add time before)[trying certainly with some backup files]

Answer 3

For a small number of files, what I'd write is

for f in *.txt; do sed 1d "$f" >tmpfile; mv tmpfile "$f"; done

For a large enough number of files, however, that won't work, because the shell will expand the *.txt to an argument list which is too long.

In that case (which sounds like your case), a better approach is

ls | grep '.txt$' | while read f; do sed 1d "$f" >tmpfile; mv tmpfile "$f"; done

You should be aware, however, that there are in-principle problems with this (as @EdMorton stresses in comments on an earlier version of this answer). One is that this will have trouble if you are unfortunate enough to have a directory whose name ends in .txt (you could handle that with test -f if you really felt the need). The other is that the read could have problems if there are some odd characters (such as a \n, say, or one or other type of quotes) in one of the filenames. You can handle such things by fiddling with IFS (see the comments), but it's better to have a quick glance at the files you're working on, and fix such bad filenames first.

What you shouldn't do is ls *.txt | ..., since if the number of files is long enough that for f in *.txt; do ... won't work, then ls *.txt won't work either.

---

There are more elaborate things you can do.

find . -type f -name \*.txt | while read f; do ...

That selects specifically files ending in .txt, but I always find the options to file untidy to type or read, and feel ls+grep is neater here.

Another possibility is

find . -type f -name \*.txt -exec sed -i 1d '{}' \;

That's pretty robust, but like most non-trivial file commands, it ends up looking like a mess, and you have to have memorised find's non-obvious syntax. Also that won't work unless your sed supports a no-argument -i option (POSIX sed doesn't have -i, and the sed on OS X and other BSDs effectively requires an extension to be specified). Also, this approach is limited to a single command, so won't work in a case where you need do to more to the files.