On Sat, 2006-01-07 at 12:14 +0000, Jenny Hopkins wrote:
Grief, no wonder it is slow then. That is going to be so slow as to be unusable. How do other people manage?
As you seem to have already found out, when you are doing byte level changes (as rsync does on anything that is not compressed) a surprisingly small amount of data actually changes.
We are using similar methods to back up whole (small) companies, including shared files and everyone's mailbox, plus stuff like databases etc.
Once we have the initial full copy (which is done locally via a USB drive) we tend to find that only a manageable amount of traffic needs to move overnight.
The only problem I have is that certain events can trigger a much much bigger download....people reorganising a file structure on the server for example and thus changing the paths to many files...backuppc is a bit smarter in this respect as it will not transfer a file of which it has an identical copy already in the pool...instead it will hardlink it across to the new location locally.
Oh and you have to watch compressed files. rsync will need to move the entire file from the point of the first change, because of how most compression works for every byte of uncompressed changes an awful lot can change in the compressed file.
But backuppc is probably overkill for your application and it has been a royal pain to get working as we wanted it....much better at managing multiple hosts and giving a series of backups (2 days ago 1 week ago etc) due to the way that the pool works with compression and hard linking this does not consume as much diskspace as you might expect.
Where we haven't used it yet (apart from on our own machines) which is actually what it is designed for, is for backup of local workstations. Because of the way you can set a backup opportunity window, throttle the backup operation etc. It is possible to backup local workstations without the end user even noticing. If the user reboots or otherwise drops off the network it will simply resume from where it was interrupted next time they are available. It actually works better than some expensive commercial applications (TSM for example)