On Tue, Jan 05, 2010 at 04:03:44PM +0000, steve-ALUG@hst.me.uk wrote:
Chris G wrote:
On Sun, Jan 03, 2010 at 02:38:43PM +0000, Chris G wrote:
I'm using rsync to copy some files across my LAN. There are various ways I can do this and I'd like to know which is likely to be fastest.
Not entirely sure if this is helpful or not, but there is a program called Unison which (according to the blurb in Ubuntu's add/remove programs) " Unison offers several advantages over various synchronization methods such as CVS, Coda, rsync, Intellisync, etc. Unison can run on and synchronize between Windows and many UNIX platforms. Unison requires no root privileges, system access or kernel changes to function. Unison can synchronize changes to files and directories in both directions, on the same machine, or across a network using ssh or a direct socket connection. Transfers are optimised using a version of the rsync protocol, making it ideal for slower links. Unison has a clear and precise specification, and is resilient to failure due to its careful handling of the replicas and its private structures.
Homepage: http://www.cis.upenn.edu/~bcpierce/unison/"
I'm after backup rather than synchronisation so Unison doesn't really offer what I want. What I'm actually doing is to rsync files to the backup system and then rdiff-backup from the rsync image to what is effectively an incremental backup. Since only the rsync image is visible to the outside world unless an intruder actually breaks into the backup system itself they can't remove the incremental backup and thus even if the original or rsync image are destroyed I should be able to restore files still.
I have compared speed of rsync copying using 'local' rsync copy to an NFS mounted partition as opposed to rsync connecting to a remote rsync (via ssh) and copying that way. The rsync to rsync copying is *much* faster, it's almost ten times faster where there are few files that actually need copying (i.e. rsync is doing its job of only copying changed files). I guess this is because file opening to get size etc. is rather slow on NFS.
Anyway I'm finally just about sorted. I have set up an rsync daemon on the backup system (which thus avoids the need for any passwordless login) and make daily copies of the areas I want to back up in the desktop systems. A script in /etc/cron.daily does this which means it gets run by anacron soon after the system is switched on, on the desktop systems rsync running is only a very small load and isn't noticeable while using the system.
Then, on the backup system, I run rdiff-backup to do an incremental backup of the files copied by rsync. I'm using normal cron to do this as the backup system is left running all the time. It's a Western Digital My Book World Edition II NAS which consumes less than 10 watts when idle. I'm pretty impressed with it in general, it runs Linux which they have made accessible by allowing ssh login to the system, thus it's pretty easy to add utilities like rsync and rdiff-backup.