There is a max-size option: –max-size=SIZE don’t transfer any file larger than SIZE So: # rsync -rv –max-size=1.5m root@tss01:/tmp/dm Will send only files less than 1.5m. Regarding sizes from man: The suffixes are as follows: “K” (or “KiB”) is a kibibyte (1024), “M” (or “MiB”) is a mebibyte (1024*1024), and “G” (or “GiB”) is a … Read more
ssh-copy-id is a pretty simple script that should be pretty easy to replicate under windows. If you ignore all the parameter handling, error handling, and so on, these are the two commands from ssh-copy-id that are actually doing the work most of the time. GET_ID=”cat ${ID_FILE}” { eval “$GET_ID” ; } | ssh ${1%:} “umask … Read more
Try to use rsync version 3 if you have to sync many files! V3 builds its file list incrementally and is much faster and uses less memory than version 2. Depending on your platform this can make quite a difference. On OSX version 2.6.3 would take more than one hour or crash trying to build … Read more
Instead of using tar to write to your local disk, you can write directly to the remote server over the network using ssh. server1$ tar -zc ./path | ssh server2 “cat > ~/file.tar.gz” Any string that follows your “ssh” command will be run on the remote server instead of the interactive logon. You can pipe … Read more
I have had very good results using tar, pigz (parallel gzip) and nc. Source machine: tar -cf – -C /path/of/small/files . | pigz | nc -l 9876 Destination machine: To extract: nc source_machine_ip 9876 | pigz -d | tar -xf – -C /put/stuff/here To keep archive: nc source_machine_ip 9876 > smallstuff.tar.gz If you want to … Read more
Ship hard drives across the ocean instead. At 11 Mbps with full utilization, you’re looking at just shy of 90 days to transfer 10 TB. 11 Mbps = 1.375 MBps = 116.015 GB/day. 10240 GB / 116.015 GB/day = ~88.3 days.
I would recommend tar. When the file trees are already similar, rsync performs very well. However, since rsync will do multiple analysis passes on each file, and then copy the changes, it is much slower than tar for the initial copy. This command will likely do what you want. It will copy the files between … Read more
See: Robocopy /? Usage : ROBOCOPY source destination [file [file]…] [options] robocopy c:\folder d:\folder transfer_this.txt
I would use rsync as it means that if it is interrupted for any reason, then you can restart it easily with very little cost. And being rsync, it can even restart part way through a large file. As others mention, it can exclude files easily. The simplest way to preserve most things is to … Read more