Remove footer from Vertica's Copy Script

Question

I have a vertica copy script (A.copy.vsql) which is loading data into a table from a file which has HEADER, TRAILER and DETAIL RECORDS.

Vertica Copy statement can skip 1 record which means I know how to remove the header.

I want to know if I can chop the trailer in the same way or not?

Also, if I cannot chop it like the header then can I write simple linux SED commands in the VSQL itself (A.copy.vsql) to do the job?

Answer 1

If you want to remove the first and last line from a file you can use:

sed '1d;$d' file

And used in a command, with the pattern command file, one can, with bash, use a process substitution:

command <(sed '1d;$d' file)

1 and $ are absolute addresses, 1 means first line, while $ means last. d deletes the line addressed.

<(...) is a process substitution.