Backup solutions for "not very linux" server destinations
I have just installed a 500Gb Network Server on my home network. It lives in the (very) detached garage so should be proof against even quite serious disasters to our house. However, although it runs a cut down Linux as its OS and thereby provides its services by Samba/CIFS it's not a very good or modern implementation and so it isn't easy to simply rsync files across to it. Using rsync (or the closely related rdiff-backup) results in lots of errors when the backup encounters symlinks, mixed case, etc. So I suspect I need a backup solution which packs up the files to be backed up into some sort of archive file and then just copies that file across to the server. Does anyone here use a backup solution of this sort and/or know any good ones? -- Chris Green
On 20/11/2007, Chris G <cl@isbd.net> wrote:
So I suspect I need a backup solution which packs up the files to be backed up into some sort of archive file and then just copies that file across to the server. Does anyone here use a backup solution of this sort and/or know any good ones?
Have you looked at dirvish? If you back up in bundles you may find your disk filling up very quickly, but dirvish uses an incantation of rsync runes that 'simply work'. Jenny
On Tue, Nov 20, 2007 at 10:53:59PM +0000, Jenny Hopkins wrote:
On 20/11/2007, Chris G <cl@isbd.net> wrote:
So I suspect I need a backup solution which packs up the files to be backed up into some sort of archive file and then just copies that file across to the server. Does anyone here use a backup solution of this sort and/or know any good ones?
Have you looked at dirvish? If you back up in bundles you may find your disk filling up very quickly, but dirvish uses an incantation of rsync runes that 'simply work'.
... but one of my basic issues is that rsync doesn't work for anything except very 'ordinary' files when saving to my network server. -- Chris Green
On 21/11/2007, Chris G <cl@isbd.net> wrote:
On Tue, Nov 20, 2007 at 10:53:59PM +0000, Jenny Hopkins wrote:
On 20/11/2007, Chris G <cl@isbd.net> wrote:
So I suspect I need a backup solution which packs up the files to be backed up into some sort of archive file and then just copies that file across to the server. Does anyone here use a backup solution of this sort and/or know any good ones?
Have you looked at dirvish? If you back up in bundles you may find your disk filling up very quickly, but dirvish uses an incantation of rsync runes that 'simply work'.
... but one of my basic issues is that rsync doesn't work for anything except very 'ordinary' files when saving to my network server.
I've been using dirvish over a secure network for a few years now: I'm not sure what you class as a 'non'ordinary' file but it certainly handles symlinks perfectly happily. Jen
On Wed, Nov 21, 2007 at 08:19:54AM +0000, Jenny Hopkins wrote:
On 21/11/2007, Chris G <cl@isbd.net> wrote:
On Tue, Nov 20, 2007 at 10:53:59PM +0000, Jenny Hopkins wrote:
On 20/11/2007, Chris G <cl@isbd.net> wrote:
So I suspect I need a backup solution which packs up the files to be backed up into some sort of archive file and then just copies that file across to the server. Does anyone here use a backup solution of this sort and/or know any good ones?
Have you looked at dirvish? If you back up in bundles you may find your disk filling up very quickly, but dirvish uses an incantation of rsync runes that 'simply work'.
... but one of my basic issues is that rsync doesn't work for anything except very 'ordinary' files when saving to my network server.
I've been using dirvish over a secure network for a few years now: I'm not sure what you class as a 'non'ordinary' file but it certainly handles symlinks perfectly happily.
That depends on the destination system surely. My Network Drive is a Samba server which, among other things, does not support "unix extensions". I have to disable the unix extensions for the drive to be mountable at all by doing:- echo 0 > /proc/fs/cifs/LinuxExtensionsEnabled Having done this when you do an rsync copy of (for example) my home directory to the network Drive there are lots of errors, I can reduce them by telling rsync not to try and create symlinks but I still get some other errors that I can't clear at all. No doubt rsysnc is actually copying nearly everything I want copied but I could do with a 'cleaner' solution. -- Chris Green
On Wed, Nov 21, 2007 at 12:16:40PM +0000, Chris G wrote:
My Network Drive is a Samba server which, among other things, does not support "unix extensions". I have to disable the unix extensions for the drive to be mountable at all by doing:-
echo 0 > /proc/fs/cifs/LinuxExtensionsEnabled
Having done this when you do an rsync copy of (for example) my home directory to the network Drive there are lots of errors, I can reduce them by telling rsync not to try and create symlinks but I still get some other errors that I can't clear at all. No doubt rsysnc is actually copying nearly everything I want copied but I could do with a 'cleaner' solution.
Am I missing why you can't use tar? Or if you want to use rsync so you only sync changed files, you could make a file on the network server, format it as an ext3 filesystem, loopback mount it and rsync to that. J. -- If a tree fell on a florist,would he make a sound? This .sig brought to you by the letter M and the number 46 Product of the Republic of HuggieTag
On Wed, Nov 21, 2007 at 04:21:11AM -0800, Jonathan McDowell wrote:
On Wed, Nov 21, 2007 at 12:16:40PM +0000, Chris G wrote:
My Network Drive is a Samba server which, among other things, does not support "unix extensions". I have to disable the unix extensions for the drive to be mountable at all by doing:-
echo 0 > /proc/fs/cifs/LinuxExtensionsEnabled
Having done this when you do an rsync copy of (for example) my home directory to the network Drive there are lots of errors, I can reduce them by telling rsync not to try and create symlinks but I still get some other errors that I can't clear at all. No doubt rsysnc is actually copying nearly everything I want copied but I could do with a 'cleaner' solution.
Am I missing why you can't use tar? Or if you want to use rsync so you only sync changed files, you could make a file on the network server, format it as an ext3 filesystem, loopback mount it and rsync to that.
Yes, I saw somewhere else someone who had done that (the loopback mount thing), I think that may well be my most practical option. Thanks for reminding me about it. ... where can I find information about doing this on Linux? A quick Google search only seems to show Solaris information about it. -- Chris Green
On 21-Nov-07 12:35:32, Chris G wrote:
[...] Yes, I saw somewhere else someone who had done that (the loopback mount thing), I think that may well be my most practical option. Thanks for reminding me about it.
... where can I find information about doing this on Linux? A quick Google search only seems to show Solaris information about it.
-- Chris Green
Does this look like the thing? http://edseek.com/~jasonb/articles/linux_loopback.html I got lots of Google hits on loopback linux and the above was the first. Best wishes, Ted. -------------------------------------------------------------------- E-Mail: (Ted Harding) <Ted.Harding@manchester.ac.uk> Fax-to-email: +44 (0)870 094 0861 Date: 21-Nov-07 Time: 12:55:43 ------------------------------ XFMail ------------------------------
On Wed, Nov 21, 2007 at 12:55:59PM -0000, Ted Harding wrote:
On 21-Nov-07 12:35:32, Chris G wrote:
[...] Yes, I saw somewhere else someone who had done that (the loopback mount thing), I think that may well be my most practical option. Thanks for reminding me about it.
... where can I find information about doing this on Linux? A quick Google search only seems to show Solaris information about it.
-- Chris Green
Does this look like the thing?
http://edseek.com/~jasonb/articles/linux_loopback.html
I got lots of Google hits on
loopback linux
and the above was the first.
Looks good. I had tried "lofs linux howto" and didn't get much useful. It seems that loopback is the key word rather than lofs. Thanks all. -- Chris Green
On Wed, Nov 21, 2007 at 01:01:08PM +0000, Chris G wrote:
I had tried "lofs linux howto" and didn't get much useful. It seems that loopback is the key word rather than lofs.
It turns out that 'man losetup' tells you just about all you need to know about this. Just what I need. -- Chris Green
On 21-Nov-07 13:18:39, Chris G wrote:
On Wed, Nov 21, 2007 at 01:01:08PM +0000, Chris G wrote:
I had tried "lofs linux howto" and didn't get much useful. It seems that loopback is the key word rather than lofs.
It turns out that 'man losetup' tells you just about all you need to know about this. Just what I need.
-- Chris Green
And who (not already knowing) would have suspected that?! Prior to googling, I had tried apropos loopback and drew a blank. I should have tried apropos loop Gives you a lot of other stuff, but losetup is also in there! Best wishes, Ted. -------------------------------------------------------------------- E-Mail: (Ted Harding) <ted.harding@nessie.mcc.ac.uk> Fax-to-email: +44 (0)870 094 0861 Date: 21-Nov-07 Time: 13:27:42 ------------------------------ XFMail ------------------------------
On Wed, Nov 21, 2007 at 01:27:46PM -0000, Ted Harding wrote:
On 21-Nov-07 13:18:39, Chris G wrote:
On Wed, Nov 21, 2007 at 01:01:08PM +0000, Chris G wrote:
I had tried "lofs linux howto" and didn't get much useful. It seems that loopback is the key word rather than lofs.
It turns out that 'man losetup' tells you just about all you need to know about this. Just what I need. And who (not already knowing) would have suspected that?!
Prior to googling, I had tried
apropos loopback
and drew a blank. I should have tried
apropos loop
Gives you a lot of other stuff, but losetup is also in there!
"man mount" would have been a reasonable place to look for info on mounting things, and has a section on "THE LOOP DEVICE" including a pointer to losetup. J. -- /-\ | noodles is a certified organic |@/ Debian GNU/Linux Developer | manufacturer under the biological \- | farmers of australia
Hi On 21/11/2007, Chris G <cl@isbd.net> wrote:
Yes, I saw somewhere else someone who had done that (the loopback mount thing), I think that may well be my most practical option. Thanks for reminding me about it.
... where can I find information about doing this on Linux? A quick Google search only seems to show Solaris information about it.
You can add the string 'linux' in your search input box. Surge
Chris G wrote:
On Wed, Nov 21, 2007 at 08:19:54AM +0000, Jenny Hopkins wrote:
On 21/11/2007, Chris G <cl@isbd.net> wrote:
On Tue, Nov 20, 2007 at 10:53:59PM +0000, Jenny Hopkins wrote:
On 20/11/2007, Chris G <cl@isbd.net> wrote:
So I suspect I need a backup solution which packs up the files to be backed up into some sort of archive file and then just copies that file across to the server. Does anyone here use a backup solution of this sort and/or know any good ones?
Have you looked at dirvish? If you back up in bundles you may find your disk filling up very quickly, but dirvish uses an incantation of rsync runes that 'simply work'.
... but one of my basic issues is that rsync doesn't work for anything except very 'ordinary' files when saving to my network server.
I've been using dirvish over a secure network for a few years now: I'm not sure what you class as a 'non'ordinary' file but it certainly handles symlinks perfectly happily.
That depends on the destination system surely.
My Network Drive is a Samba server which, among other things, does not support "unix extensions". I have to disable the unix extensions for the drive to be mountable at all by doing:-
echo 0 > /proc/fs/cifs/LinuxExtensionsEnabled
Having done this when you do an rsync copy of (for example) my home directory to the network Drive there are lots of errors, I can reduce them by telling rsync not to try and create symlinks but I still get some other errors that I can't clear at all. No doubt rsysnc is actually copying nearly everything I want copied but I could do with a 'cleaner' solution.
I happened to be looking at Bash scripts yesterday, as beginning the extensive learning curve from win to nix... but i remembered seeing a "simple" backup script, that archives and then dumps the archive somewhere (assuming it will be able to dump a archive on your smb mounts) anyway if it helps then all is good, if not sorry! he he! http://tldp.org/LDP/abs/html/contributed-scripts.html --> Example A-15
Hi On 21/11/2007, Chris G <cl@isbd.net> wrote:
My Network Drive is a Samba server which, among other things, does not support "unix extensions". I have to disable the unix extensions for the drive to be mountable at all by doing:-
echo 0 > /proc/fs/cifs/LinuxExtensionsEnabled
I was just wondering, can you put NFS on the samba server and then do a normal rsync/etc. ? I know you said the machine is an old thing with cut down Linux and old samba. I read recently there was a buffer overflow vulnerability on samba, and so what would be your plans for security updates for this server? If you were going to do a security update, you might be inclined to think about putting on NFS on there and have 'proper' *nix file support anyway. That reminds me, I should setup NFS tonight :) Surge
On Wed, Nov 21, 2007 at 01:45:02PM +0000, Srdjan Todorovic wrote:
Hi
On 21/11/2007, Chris G <cl@isbd.net> wrote:
My Network Drive is a Samba server which, among other things, does not support "unix extensions". I have to disable the unix extensions for the drive to be mountable at all by doing:-
echo 0 > /proc/fs/cifs/LinuxExtensionsEnabled
I was just wondering, can you put NFS on the samba server and then do a normal rsync/etc. ? I know you said the machine is an old thing with cut down Linux and old samba. I read recently there was a buffer overflow vulnerability on samba, and so what would be your plans for security updates for this server? If you were going to do a security update, you might be inclined to think about putting on NFS on there and have 'proper' *nix file support anyway.
It's not old in the sense that it's an old Linux box (of mine) running as a server. It's a brand new box which is a NAS system that happens to run a cut down Linux and Samba to provide network file storage. So it's not accessible in any way to change its setup. The command above is what I have to do at the *client* end so that the CIFS mount works given the limitations of the NAS. -- Chris Green
On Wed, Nov 21, 2007 at 03:47:23PM +0000, Chris G wrote:
It's not old in the sense that it's an old Linux box (of mine) running as a server. It's a brand new box which is a NAS system that happens to run a cut down Linux and Samba to provide network file storage. So it's not accessible in any way to change its setup.
What nas is it? some of them are surprisingly hackable. Adam -- jabberid = quinophex@jabber.earth.li
On Wed, Nov 21, 2007 at 04:55:54PM +0000, Adam Bower wrote:
On Wed, Nov 21, 2007 at 03:47:23PM +0000, Chris G wrote:
It's not old in the sense that it's an old Linux box (of mine) running as a server. It's a brand new box which is a NAS system that happens to run a cut down Linux and Samba to provide network file storage. So it's not accessible in any way to change its setup.
What nas is it? some of them are surprisingly hackable.
It's a Freecom NAS. -- Chris Green
On Wed, Nov 21, 2007 at 09:17:43PM +0000, Adam Bower wrote:
On Wed, Nov 21, 2007 at 08:08:17PM +0000, Chris G wrote:
It's a Freecom NAS.
Which model? the FSG-3 will run openwrt.
I'm not really sure if it has a model number, it's just a 500Gb Network Drive as here:- http://www.freecom.com/ecproduct_detail.asp?ID=3402&CatID=8020&sCatID=114644... -- Chris Green
On Tue, 2007-11-20 at 15:20 +0000, Chris G wrote:
So I suspect I need a backup solution which packs up the files to be backed up into some sort of archive file and then just copies that file across to the server. Does anyone here use a backup solution of this sort and/or know any good ones?
Could you not tar to a file within the samba mounted directory, perhaps cobble together some script that maintains x number of cycles worth of archives and then purges them ? I used to do something similar many moons ago to archive some servers on a regular basis and it was relatively easy to put something together that kept 2 weeks worth of archives and mailed me when there was a problem etc.
On Wed, Nov 21, 2007 at 02:31:08AM +0000, Wayne Stallwood wrote:
On Tue, 2007-11-20 at 15:20 +0000, Chris G wrote:
So I suspect I need a backup solution which packs up the files to be backed up into some sort of archive file and then just copies that file across to the server. Does anyone here use a backup solution of this sort and/or know any good ones?
Could you not tar to a file within the samba mounted directory, perhaps cobble together some script that maintains x number of cycles worth of archives and then purges them ? I used to do something similar many moons ago to archive some servers on a regular basis and it was relatively easy to put something together that kept 2 weeks worth of archives and mailed me when there was a problem etc.
Yes, that's exactly what I need to do, I was just wondering if there was a ready made utility for doing it. If there isn't then I will do what you suggest. On approach I'm considering is to use rdiff-backup to a linux file system and then to tar (or cpio or whatever) the rdiff-backup files to the Network Server. It would be relatively simple but possibly rather slow. -- Chris Green
On Tue, Nov 20, 2007 at 03:20:44PM +0000, Chris G wrote:
I have just installed a 500Gb Network Server on my home network. It lives in the (very) detached garage so should be proof against even quite serious disasters to our house.
Good plan.
So I suspect I need a backup solution which packs up the files to be backed up into some sort of archive file and then just copies that file across to the server. Does anyone here use a backup solution of this sort and/or know any good ones?
Dar. http://dar.linux.free.fr/ Adam -- jabberid = quinophex@jabber.earth.li
On Wed, Nov 21, 2007 at 10:12:54AM +0000, Adam Bower wrote:
On Tue, Nov 20, 2007 at 03:20:44PM +0000, Chris G wrote:
I have just installed a 500Gb Network Server on my home network. It lives in the (very) detached garage so should be proof against even quite serious disasters to our house.
Good plan.
So I suspect I need a backup solution which packs up the files to be backed up into some sort of archive file and then just copies that file across to the server. Does anyone here use a backup solution of this sort and/or know any good ones?
Dar.
Looks good, thank you, I'll have to decide between dar and the loopback filesystem method now. -- Chris Green
participants (8)
-
Adam Bower -
Alex Scotton -
Chris G -
Jenny Hopkins -
Jonathan McDowell -
Srdjan Todorovic -
ted.harding＠nessie.mcc.ac.uk -
Wayne Stallwood