Bash: Fail to use find -exec

Question

When using scp or rsync I often fail to deal with 'Argument list too long' error. When having to mv or rm, I have no problem to use find and xargs but I fail to understand how to use find and -exec despite all the SE posts on the subject. Consider the following issue...

I tried

$scp /Path/to/B* [email protected]:/Path/to/

-bash: /usr/bin/scp: Argument list too long

So I tried

$find . -name "/Path/to/B*" -exec scp "{}" [email protected]:/Path/to/ '\;'

find: -exec: no terminating ";" or "+"

so I tried

$find . -name "/Path/to/B*" -exec scp "{}" [email protected]:/Path/to/ ';'

find: ./.gnupg: Permission denied
find: ./.subversion/auth: Permission denied

So I tried

$sudo find . -name "/Path/to/B*" -exec scp "{}" [email protected]:/Path/to/ ';'

and nothing happen onces I enter my password

I am on Mac OSX version 10.11.3, Terminal version 2.6.1

Answer 1

R. Saban's helpful answer solves your primary problem:

• -name only accepts a filename pattern, not a path pattern.

• Alternatively, you could simply use the -path primary instead of the -name primary.

As for using as few invocations of scp as possible - each of which requires specifying a password by default:

• As an alternative, consider bypassing the use of scp altogether, as suggested in Eric Renouf's helpful answer.

• While find's -exec primary allows using terminator + in lieu of ; (which must be passed as ';' or \; to prevent the shell from interpreting ; as a command terminator) for passing as many filenames as will fit on a single command line (a built-in xargs, in a manner of speaking), this is NOT an option here, because use of + requires that placeholder {} come last on the command line, immediately before +.

• However, since you're on macOS, you can use BSD xarg's nonstandard -J option for placing the placeholder anywhere on the command line, while still passing as many arguments as possible at once (using BSD find's nonstandard -print0 option in combination with xargs's nonstandard -0 option ensures that all filenames are passed as-is, even if they have embedded spaces, for instance):

find . -path "/Path/to/B*" -print0 | xargs -0 -J {} scp {} [email protected]:/Path/to/

Now you will at most be prompted a few times: one prompt for every batch of arguments, as required to accommodate all arguments while observing the max. command-line length with the fewest number of calls possible.

Answer 2

EDIT after your update:

find "/Path/to" -maxdepth 1 -name "B*" -exec scp {} [email protected]:/Path/to/ \;

Answer 3

A solution that wouldn't require multiple scp connections (and therefore password entries) would be to tar on one side and untar on the other like:

find /Path/to -maxdepth 1 -name 'B*' -print0 | tar -c --null -T - | ssh [email protected] tar -x -C /Path/to

assuming your version of find supports -print0 and the like. It works by printing out null terminated list of files from find and telling tar to read its list of files from stdin (-T -) treating the list as null terminated (--null) and create a new archive (-c). By default, tar will write to stdout.

So then we'll pipe that archive to an ssh command to the target host. That will read the output of the previous command on its stdin, so we'll use tar there to extract (-x) the archive into the given directory (-C /Path/to)