shrink (truncate) file from beginning on linux

Question

Is it possible in Linux (and/or on other Unix) 'shrink' file from beginning? I'd like to use it for persistent queue (no existing implementation suits my needs). From end of file I guess it's possible with truncate().

Answer 1

Yes, you can use cut or tail to remove portions of a file.

cut -b 17- input_file
tail -c +17 input_file

This will output the contents of input_file starting at the 17th byte, effectively removing the first 16 bytes of the file. Note that the cut example will also add a newline to the output.

Answer 2

You can try dropping half of logs using ex, but it is not as fast as I would like (5GB of logs takes ages):

ex -s -c "1d$(( $(wc -l /var/log/messages | awk '{ print $1 }') / 2 ))|x" /var/log/messages

Answer 3

If you are using ext4, xfs or some other modern file system, since Linux Kernel 3.15 you can use:

#include <fcntl.h>

int fallocate(int fd, int mode, off_t offset, off_t len);

with the FALLOC_FL_COLLAPSE_RANGE flag.

http://manpages.ubuntu.com/manpages/disco/en/man2/fallocate.2.html