main

main@lists.alug.org.uk

10364 discussions

Re: [ALUG] Using rsync --link-dest continued, comments welcome
by Phil Ashby 01 Apr '11

01 Apr '11

Chris G <cl(a)isbd.net> wrote: > Well here's my initial attempt at writing the incremental backup using > rsync's --link-dest :- > ...[weekly rotating backup script] I got this going on my file server (not a NAS - it's a Debian box) using the following script, which can be executed at any interval that seems appropriate*, and retains a specified quantity of previous backups rather than a time limited number. I find that so little changes between backups that I can keep 100 of them on the same size backup media as the main storage (1TB, ~60% full). This should work in ash almost as-is, but you may need to use dc for the maths (subtracting 1) instead of shell built-in capability: TGT=`echo $BKP 1 -pq|dc` Notes: - This keeps the backup device mounted read-only unless a backup is in progress, to avoid 'finger trouble' nuking important stuff. - Uses the noatime option when mounted rw, to improve performance. - Since this is a full file system backup, it excludes rather than includes folders. - It doesn't try to back up if there is <10G of free space on the backup device (empirical determination, typically I see 10-20G of change at worst per backup). - Error messages (and debug) to stderr as this is run by cron so will get emailed to root (me). - Any issues are collected and stored in the backup folder. Phil. * - currently every 4 hours during the day but not overnight since the machine may be busy using my free overnight bandwidth :) ----8<---- #! /bin/sh # Take a snapshot of the current filesystem (minus snapshot folder and caches) #DBG=--verbose BACKUP_DEV="UUID=go-look-in-dev-disk-by-uuid-xxxxxxxxxxxx" BACKUP_DIR=/var/backups/snapshot EXCLUDES="/dev /proc /sys /mnt /media /tmp /var/cache /var/lock \ /var/www/.get_iplayer $BACKUP_DIR" # Time/date records BKSTART=`date` # Only keep up to 100 snapshots MAX_BACKUP=100 # 10GB in k MIN_SPACE_K=10000000 docmd() { [ -n "$DBG" ] && echo debug: "$*" >&2 $* } NEED_REMOUNT= remount() { docmd mount -o remount,ro $BACKUP_DEV $BACKUP_DIR } barf() { echo "$*" >&2 [ -n "$NEED_REMOUNT" ] && remount exit 1 } # Mount the snapshot folder read-write docmd mount -o remount,rw,noatime $BACKUP_DEV $BACKUP_DIR [ $? -eq 0 ] || barf "unable to remount $BACKUP_DIR rw" NEED_REMOUNT=1 # Space check docmd cd $BACKUP_DIR AVAIL=`df -k . |tail -1 |(read d b u a t; echo $a)` [ $AVAIL -gt $MIN_SPACE_K ] || barf "out of backup device space" [ -n "$DBG" ] && echo "debug: available space: ${AVAIL}k" >&2 # Rotate existing backups [ -d backup.$MAX_BACKUP ] && docmd rm -rf backup.$MAX_BACKUP BKP=$MAX_BACKUP while [ $BKP -gt 0 ] do TGT=$(($BKP-1)) [ -d backup.$TGT ] && docmd mv backup.$TGT backup.$BKP BKP=$TGT done # rsync changes into fresh backup, hard-linking unchanged files for p in $EXCLUDES do echo $p >>/tmp/backup-excludes.$$ done docmd rsync $DBG -a --link-dest=$BACKUP_DIR/backup.1 --exclude-from=/tmp/backup-excludes.$$ / $BACKUP_DIR/backup.0 >/tmp/sync-log.$$ 2>&1 if [ $? -ne 0 ]; then echo "rsync into $BACKUP_DIR had some problems, see log" >&2 docmd mv /tmp/sync-log.$$ $BACKUP_DIR/backup.0/RSYNC.LOG fi docmd rm -f /tmp/backup-excludes.$$ /tmp/sync-log.$$ # Time/date records BKSTOP=`date` docmd echo "Backup started: $BKSTART, completed: $BKSTOP" >$BACKUP_DIR/backup.0/README.DATES # Remount read-only NEED_REMOUNT= remount [ $? -eq 0 ] || barf "unable to remount $BACKUP_DIR ro"

2 1

date command error
by James Bensley 31 Mar '11

31 Mar '11

Can anyone explain this? [user@machine ~]$ date Tue Mar 29 16:48:38 BST 2011 [user@machine ~]$ date --date="1 month ago" Tue Mar 1 15:48:47 GMT 2011 Shouldn't that last one say the date one month ago, not 28 days ago? (I suppose one month needs to be quantified). Not sure how 'date' has worked that out but I'm guessing the change in time zone might perhaps be to blame? Can anyone shed some light on this? -- James. http://www.jamesbensley.co.cc/ There are 10 kinds of people in the world; Those who understand Vigesimal, and J others...?

6 5

Using rsync --link-dest continued, comments welcome
by Chris G 31 Mar '11

31 Mar '11

Well here's my initial attempt at writing the incremental backup using rsync's --link-dest :- #!/bin/ash # # # Backup script run daily # dirs="chris dps laptop" today=`date +%a` case $today in "Sun") yesterday="Sat";; "Mon") yesterday="Sun";; "Tue") yesterday="Mon";; "Wed") yesterday="Tue";; "Thu") yesterday="Wed";; "Fri") yesterday="Thu";; "Sat") yesterday="Fri";; esac for dir in $dirs do # # # remove last week's backup # rm -fr /ExtendVolume/$dir/$today # # # Run rsync to create a new backup for today # rsync -a --link-dest=/ExtendVolume/$dir/$yesterday /DataVolume/$dir/ /ExtendVolume/$dir/$today done It works to the extent that it's creating the backup for 'today' on /ExtendVolume, I'll have to wait until tomorrow before I can tell if it's doing the hard links right. I get a warning error today of course:- --link-dest arg does not exist: /ExtendVolume/chris/Wed but that's fine, it's actually quite encouraging! :-) If anyone can suggest a better way of working out 'yesterday' than that horrible case sequence I'd love to see it. While the case works it grates rather when I look at it! (It has to be in ash or sh, the NAS doesn't have bash or similar) -- Chris Green

3 5

Anyone know anything about rsync --link-dest option usage?
by Chris G 31 Mar '11

31 Mar '11

I'm aiming to use rsync's --link-dest option to do some incremental backups but I'm not absolutely sure I've got its use correct. What I'm going to do is maintain incremental backups for the last seven days, so they'll be in directories named for the days of the week. The script will work out values for today and yesterday and then do (example for today):- rm -fr /backup/Wed rsync -a --link-dest=/backup/Tue fromSource /backup/Wed Will this do what's intended? I'm less interested in detailed syntax errors but want to be sure I will get 7 days of incremental backups. -- Chris Green

3 4

snd-hda-intel won't load automatically
by Jenny Hopkins 30 Mar '11

30 Mar '11

Hullo all, I've got an ubuntu (maverick) computer that is refusing to load sound modules on start-up. lspci: 00:08.0 Audio device: nVidia Corporation MCP79 High Definition Audio (rev b1) Subsystem: PC Partner Limited Device 437b Flags: bus master, 66MHz, fast devsel, latency 0, IRQ 22 Memory at fae78000 (32-bit, non-prefetchable) [size=16K] Capabilities: [44] Power Management version 2 Kernel driver in use: HDA Intel Kernel modules: snd-hda-intel The command 'sudo modprobe snd-hda-intel' loads the modules and all works, but this has to be run manually. The file /etc/modprobe.d/alsa-base.conf: # autoloader aliases install sound-slot-0 /sbin/modprobe snd-card-0 install sound-slot-1 /sbin/modprobe snd-card-1 install sound-slot-2 /sbin/modprobe snd-card-2 install sound-slot-3 /sbin/modprobe snd-card-3 install sound-slot-4 /sbin/modprobe snd-card-4 install sound-slot-5 /sbin/modprobe snd-card-5 install sound-slot-6 /sbin/modprobe snd-card-6 install sound-slot-7 /sbin/modprobe snd-card-7 # Cause optional modules to be loaded above generic modules install snd /sbin/modprobe --ignore-install snd $CMDLINE_OPTS && { /sbin/modprobe --quiet --use-blacklist snd-ioctl32 ; /sbin/modpro be --quiet --use-blacklist snd-seq ; } # # Workaround at bug #499695 (reverted in Ubuntu see LP #319505) install snd-pcm /sbin/modprobe --ignore-install snd-pcm $CMDLINE_OPTS && { /sbin/modprobe --quiet --use-blacklist snd-pcm-oss ; : ; } install snd-mixer /sbin/modprobe --ignore-install snd-mixer $CMDLINE_OPTS && { /sbin/modprobe --quiet --use-blacklist snd-mixer-oss ; : ; } install snd-seq /sbin/modprobe --ignore-install snd-seq $CMDLINE_OPTS && { /sbin/modprobe --quiet --use-blacklist snd-seq-midi ; /sb in/modprobe --quiet --use-blacklist snd-seq-oss ; : ; } # install snd-rawmidi /sbin/modprobe --ignore-install snd-rawmidi $CMDLINE_OPTS && { /sbin/modprobe --quiet --use-blacklist snd-seq-mi di ; : ; } # Cause optional modules to be loaded above sound card driver modules install snd-emu10k1 /sbin/modprobe --ignore-install snd-emu10k1 $CMDLINE_OPTS && { /sbin/modprobe --quiet --use-blacklist snd-emu10k 1-synth ; } install snd-via82xx /sbin/modprobe --ignore-install snd-via82xx $CMDLINE_OPTS && { /sbin/modprobe --quiet --use-blacklist snd-seq ; } # Load saa7134-alsa instead of saa7134 (which gets dragged in by it anyway) install saa7134 /sbin/modprobe --ignore-install saa7134 $CMDLINE_OPTS && { /sbin/modprobe --quiet --use-blacklist saa7134-alsa ; : ; } # Prevent abnormal drivers from grabbing index 0 options bt87x index=-2 options cx88_alsa index=-2 options saa7134-alsa index=-2 options snd-atiixp-modem index=-2 options snd-intel8x0m index=-2 options snd-via82xx-modem index=-2 options snd-usb-audio index=-2 options snd-usb-caiaq index=-2 options snd-usb-ua101 index=-2 options snd-usb-us122l index=-2 options snd-usb-usx2y index=-2 # Ubuntu #62691, enable MPU for snd-cmipci options snd-cmipci mpu_port=0x330 fm_port=0x388 # Keep snd-pcsp from being loaded as first soundcard options snd-pcsp index=-2 # Keep snd-usb-audio from beeing loaded as first soundcard options snd-usb-audio index=-2 options snd-hda-intel enable=1 index=0 model=PC I added the last line after perusing the internet, but it doesn't help. Can anyone see what I can't? I'm not too hot on sound and modules and t'ing. Is this wrong: Kernel driver in use: HDA Intel Kernel modules: snd-hda-intel ? Many thanks, Jenny

1 1

Re: [ALUG] date command error
by James Bensley 29 Mar '11

29 Mar '11

Hmm, yes I see there are multiple ways to deal with it. This runs in a script, it runs with %Y-%m --date="1 month ago" but today it was of course returning "2011-03" so that is what sparked my question initially. I will change it and parse a day also as Ted mentioned, thanks all for your input, it is appreciated :) --James. (This email was sent from a mobile device)

1 0

Archiving mail from an Exchange Server into something more "sensible"
by Mark Rogers 29 Mar '11

29 Mar '11

I have an "old" Exchange Server (2008), which I'm decommissioning. On it there are several mailboxes which I'd like archived so that I can access them as IMAP folders on another (Linux) machine in future if I need to. We no longer have an internal mail server of any variety to host the mailbox archive (we've moved to Google Apps) so I'll be settings something up specially (probably a virtual machine). The majority of the mailboxes are "general" ones like sales, accounts, etc, and a lot of the others are for people who no longer work for the business, so ideally I'll move the email into one new account with folders for sales/accounts/etc rather than create them as different accounts. Any suggestions as to the best way to do this? There will be several GB of email in total. My main concern is to retain as much information as possible, such as email headers and timestamps. I considered pulling all the email via POP3 into a mail client on my desktop but I'm not confident that I won't end up losing some of that information (timestamps being the main thing: I still want to see when the email "arrived", not have it look like the email arrived at the time I did the POP3 transfer). -- Mark Rogers // More Solutions Ltd (Peterborough Office) // 0844 251 1450 Registered in England (0456 0902) @ 13 Clarke Rd, Milton Keynes, MK1 1LG

4 6

ASDA online shopping -- website doesn't seem to like linux?
by James Freer 23 Mar '11

23 Mar '11

using Ubuntu 10.04 [also tried on my old machine on 8.04 and clean install on spare PC Xubuntu 10.04] Does anyone shop online with Asda. I've been using it for over a year now. End of Feb it started coming up with "our website is experiencing difficulties... please try again later". So i did... and got the same message every time i tried. I thought i'd made a breakthrough when instead of first 'booking a delivery slot', did the shopping (didn't get their message) and then after booking the slot back it came. I telephone and filled out their online form and apparently the website is working fine. It seems a] their website has been updated and doesn't like linux (which doesn't make sense using Firefox browser) b] Macintosh machines seem to get the same message It would appear they've done something on the website so linux users have problems. Does anyone else shop online with Asda by any chance... any problems? Any suggestions from a geek what could be wrong or what i could suggest in my next letter? thanks james

5 9

Media playing software - do they all build their own indexes or what?
by Chris G 22 Mar '11

22 Mar '11

I'm just beginning to dip my toe(s) into moving recorded music and other such things (e.g. radio programmes) onto a computer or computers. One thing I'd prefer is to be able to keep all the files in a directory hierarchy of my own making rather than using a media player's own way of doing things. How does most media player type software handle this? If you point them at a hierarchy full of sound files do they build their own indexes from the shape of the hierarchy and the content (as in genre information etc.) of the files or what? Are there any (all?) that work like Digikam with images for instance where you can tell it to keep *all* of the information in the image files (as well as building its own index) so that if you move the hierarchy or want to access it with other software everything works still? -- Chris Green

7 18

OT: Embedded development project anyone?
by Phil Ashby 22 Mar '11

22 Mar '11

Hi all, I've been offered a small development contract that I can't take up at the moment, and was wondering if anyone hereabouts would be interested. Details that I have at present: Porting C from AVR to Cortex M3 based SolderCore board (www.soldercore.com) Rowley Associates IDE available Please contact me directly if you are interested in more info and I'll put you in touch with the request originator. Phil.

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

main