[Trilug-ontopic] [bash] copying `find`ed files across network

[Tom Roche]

http://www.trilug.org/pipermail/trilug-ontopic/2012-May/000315.html
Tom Roche Wed, 23 May 2012 15:36:07 -0400
>> I'm trying to copy a buncha files across a network. Something is
>> horribly wrong, improvements or alternatives appreciated. I've got

>> me at remote:~ $ find /work/MOD3EVAL/nsu/boundary/BLD_ddm_saprc07tc -type f | grep -ve 'CCTM\|CVS\|~$\|\.o$' | sort | xargs du -ch | tail -n 1
>> > 36M   total

>> and I want to copy only those files.

Alan Porter Wed May 23 15:45:15 EDT 2012
> How about this?

> tar -zcf - $(find /work/MOD3EVAL/nsu/boundary/BLD_ddm_saprc07tc -type f | grep -ve 'CCTM\|CVS\|~$\|\.o$') | ssh root at remote "tar -zxvf -"

I don't have root on remote, and I don't wanna restore there ...
but making a similar adjustment adds a datapoint (starred below):

REMOT_PATH='/work/MOD3EVAL/nsu/boundary/BLD_ddm_saprc07tc'
LEAF="$(basename ${REMOT_PATH})"
LOCAL_ROOT="${HOME}/code/CMAQ/${LEAF}"
LOCAL_ARCHIVE_FP="${LOCAL_ROOT}/${LEAF}.tgz"
REMOT_FIND="find . -type f | grep -ve 'CCTM\|CVS\|~$\|\.o$'"
REMOT_TAR="pushd ${REMOT_PATH} ; tar cfvz - \$(${REMOT_FIND})"

for CMD in \
  "mkdir -p ${LOCAL_ROOT}" \
  "pushd ${LOCAL_ROOT}" \
  "ssh t \"pushd ${REMOT_PATH} ; ${REMOT_FIND} | wc -l\"" \
  "ssh t \"${REMOT_TAR}\" | tar xvfz -" \
  "find . -type f | wc -l" \
  "popd" \
; do
  echo -e "$ ${CMD}"
  eval "${CMD}"
done

> $ mkdir -p /home/me/code/CMAQ/BLD_ddm_saprc07tc
> $ pushd /home/me/code/CMAQ/BLD_ddm_saprc07tc
> ~/code/CMAQ/BLD_ddm_saprc07tc ~

# case 1: `raw find`
* $ ssh t "pushd /work/MOD3EVAL/nsu/boundary/BLD_ddm_saprc07tc ; find . -type f | grep -ve 'CCTM\|CVS\|~$\|\.o$' | wc -l"
* /work/MOD3EVAL/nsu/boundary/BLD_ddm_saprc07tc ~
* 266

# case 2: `find` inside `tar`
* $ ssh t "pushd /work/MOD3EVAL/nsu/boundary/BLD_ddm_saprc07tc ; tar cfvz - $(find . -type f | grep -ve 'CCTM\|CVS\|~$\|\.o$')" | tar xvfz -
* tar: Cowardly refusing to create an empty archive
* Try `tar --help' or `tar --usage' for more information.
>
> gzip: stdin: not in gzip format
> tar: Child returned status 1
> tar: Error is not recoverable: exiting now
> $ find . -type f | wc -l
> 0
> $ popd

Why is the raw `find` working, but the same `find` inside `tar` draws
the 'empty archive' warning (presumably due to *not* finding the files)?