How to `scp` directory preserving structure but only pick certain files?

4.9k views Asked by At

I need to secure copy (scp) to remotely copy a directory with its sub structure preserved from the UNIX command line. The sub directories have identically named files that I WANT and bunch of other stuff that I don't. Here is how the structure looks like.

directorytocopy
  subdir1
    1.wanted
    2.wanted
    ...
    1.unwanted
    2.notwanted
  subdir2
    1.wanted
    2.wanted
    ...
    1.unwanted
    2.notwanted
  ..

I just want the .wanted files preserving the directory structure. I realize that it is possible to write a shell (I am using bash) script to do this. Is it possible to do this in a less brute force way? I cannot copy the whole thing and delete the unwanted files because I do not have enough space.

3

There are 3 answers

0
QuantSolo On BEST ANSWER

rsync with and -exclude/include list follwing @Adrian Frühwirth's suggestion would be a to do this.

0
lcd047 On

Assuming GNU tar on the source machine, and assuming that filenames of the wanted files won't contain newlines and they are short enough to fit the tar headers:

find /some/directory -type f -name '*.wanted' | \
    tar cf - --files-from - | \
    ssh user@host 'cd /some/other/dir && tar xvpf -'
0
glenn jackman On

Adrian has the best idea to use rsync. You can also use tar to bundle the wanted files:

cd directorytocopy
shopt -s nullglob globstar
tar -cf - **/*.wanted | ssh destination 'cd dirToPaste && tar -xvf -'

Here, using tar's -f option with the filename - to use stdin/stdout as the archive file.

This is untested, and may fail because the archive may not contain the actual subdirectories that hold the "wanted" files.