I am trying to dig into my rsync backup problems.
It would appear that for some reason rsync is getting I/O errors when copying maildir files.
I have done some tests on one particular directory, "cp -R" succeeds but "rsync -r" fails:-
home# cp -R .in /freecom/backup/home/chris/Mail/ home# rsync -r .in /freecom/backup/home/chris/Mail/ rsync: rename "/freecom/backup/home/chris/Mail/.in/audacity/cur/.1172007150.76430_0.newred:2,.PC3iF7" -> ".in/audacity/cur/1172007150.76430_0.newred:2,": Input/output error (5) rsync: rename "/freecom/backup/home/chris/Mail/.in/audacity/cur/.1172012773.6188_0.newred:2,.P11kJR" -> ".in/audacity/cur/1172012773.6188_0.newred:2,": Input/output error (5) rsync: rename "/freecom/backup/home/chris/Mail/.in/audacity/cur/.1172032604.34523_0.newred:2,.e0feQB" -> ".in/audacity/cur/1172032604.34523_0.newred:2,": Input/output error (5) rsync: rename "/freecom/backup/home/chris/Mail/.in/audacity/cur/.1172038263.69147_0.newred:2,.W6m5Zl" -> ".in/audacity/cur/1172038263.69147_0.newred:2,": Input/output error (5) rsync: rename "/freecom/backup/home/chris/Mail/.in/freecycle/cur/.1172005922.70482_0.newred:2,S.aIqsGA" -> ".in/freecycle/cur/1172005922.70482_0.newred:2,S": Input/output error (5) rsync error: some files could not be transferred (code 23) at main.c(977) [sender=2.6.9]
If I clear out the destination directory then rsync succeeds once (when it has to copy the files) but then fails as above every time. If I add the --inplace option I get:-
home# rsync -r --inplace .in /freecom/backup/home/chris/Mail/ rsync: open "/freecom/backup/home/chris/Mail/.in/audacity/cur/1172007150.76430_0.newred:2," failed: Input/output error (5) rsync: open "/freecom/backup/home/chris/Mail/.in/audacity/cur/1172012773.6188_0.newred:2," failed: Input/output error (5) rsync: open "/freecom/backup/home/chris/Mail/.in/audacity/cur/1172032604.34523_0.newred:2," failed: Input/output error (5) rsync: open "/freecom/backup/home/chris/Mail/.in/audacity/cur/1172038263.69147_0.newred:2," failed: Input/output error (5) rsync: open "/freecom/backup/home/chris/Mail/.in/freecycle/cur/1172005922.70482_0.newred:2,S" failed: Input/output error (5) rsync error: some files could not be transferred (code 23) at main.c(977) [sender=2.6.9]
The files in the destination directory seem normal:-
home$ ls -l /freecom/backup/home/chris/Mail/.in/audacity/cur total 36 -rw------- 1 root root 8192 2007-12-10 20:40 1172007150.76430_0.newred:2, -rw------- 1 root root 8192 2007-12-10 20:40 1172012773.6188_0.newred:2, -rw------- 1 root root 12288 2007-12-10 20:40 1172032604.34523_0.newred:2, -rw------- 1 root root 8192 2007-12-10 20:40 1172038263.69147_0.newred:2,
So why is rsync having problems?
On Mon, Dec 10, 2007 at 08:45:24PM +0000, Chris G wrote:
I am trying to dig into my rsync backup problems.
It would appear that for some reason rsync is getting I/O errors when copying maildir files.
I have done some tests on one particular directory, "cp -R" succeeds but "rsync -r" fails:-
OK - have you tried removing the directory then using rsync -a rather than -r? Also, is the top level directory that you're backing up to set to have the sticky bit set (cos that'd break it)... and as it appears to be having problems opening the files for write, you *are* root when running the rsync, right? And you have tried editing one of the files to see if you get and IO failure on it, right?
Cheers,
On Wed, Dec 12, 2007 at 12:15:23AM +0000, Brett Parker wrote:
On Mon, Dec 10, 2007 at 08:45:24PM +0000, Chris G wrote:
I am trying to dig into my rsync backup problems.
It would appear that for some reason rsync is getting I/O errors when copying maildir files.
I have done some tests on one particular directory, "cp -R" succeeds but "rsync -r" fails:-
OK - have you tried removing the directory then using rsync -a rather than -r? Also, is the top level directory that you're backing up to set to have the sticky bit set (cos that'd break it)... and as it appears to be having problems opening the files for write, you *are* root when running the rsync, right? And you have tried editing one of the files to see if you get and IO failure on it, right?
I think I've done all those things:-
home# rm -fr /freecom/backup/home/chris/Mtest/.in home# rsync -a .in /freecom/backup/home/chris/Mtest home# rsync -a .in /freecom/backup/home/chris/Mtest rsync: rename "/freecom/backup/home/chris/Mtest/.in/audacity/cur/.1172007150.76430_0.newred:2,.CKE4Ss" -> ".in/audacity/cur/1172007150.76430_0.newred:2,": Input/output error (5) rsync: rename "/freecom/backup/home/chris/Mtest/.in/audacity/cur/.1172012773.6188_0.newred:2,.37xDw2" -> ".in/audacity/cur/1172012773.6188_0.newred:2,": Input/output error (5) rsync: rename "/freecom/backup/home/chris/Mtest/.in/audacity/cur/.1172032604.34523_0.newred:2,.xqFadC" -> ".in/audacity/cur/1172032604.34523_0.newred:2,": Input/output error (5) rsync: rename "/freecom/backup/home/chris/Mtest/.in/audacity/cur/.1172038263.69147_0.newred:2,.cqUMWb" -> ".in/audacity/cur/1172038263.69147_0.newred:2,": Input/output error (5) rsync: rename "/freecom/backup/home/chris/Mtest/.in/freecycle/cur/.1172005922.70482_0.newred:2,S.xnKZZz" -> ".in/freecycle/cur/1172005922.70482_0.newred:2,S": Input/output error (5) rsync error: some files could not be transferred (code 23) at main.c(977) [sender=2.6.9] home#
home# ls -al /freecom total 8 drwxrwxrwx 1 root root 0 2007-01-23 22:24 . drwxr-xr-x 32 root root 4096 2007-12-10 09:22 .. drwxrwxrwx 1 root root 0 2007-01-03 09:54 backup home#
All the subsequent directories down from /freecom/backup have nothing unusual about them either, no sticky bits etc.
I can go to /freecom/backup/home/chris/Mtest/.in/audacity/cur and edit the files there without problems.
Thinks.... looks at files which have problems.
I can now see what the problem relates to, it's the files whose names have commas in that are causing problems. Doing a find for all the mail files in that area shows:-
./audacity/cur/1172007150.76430_0.newred:2, ./audacity/cur/1172012773.6188_0.newred:2, ./audacity/cur/1172032604.34523_0.newred:2, ./audacity/cur/1172038263.69147_0.newred:2, ./cafe/new/1172050891.76512_0.newred ./cafe/new/.1172050891.76512_0.newred.Db1dJL ./chris/new/1172049386.67574_0.newred ./chris/new/1172049415.67719_0.newred ./chris/new/1172049422.67809_0.newred [snip] ./chris/new/.1172052833.87374_0.newred.ECOPzp ./chris/new/.1172052839.87385_0.newred.6zXpLZ ./freecycle/cur/1172005922.70482_0.newred:2,S ./ixion/new/1172048011.60081_0.newred ./ixion/new/1172048786.64308_0.newred ./ixion/new/1172048991.65223_0.newred [snip more]
The only files that give errors are the four in audacity with trailing commas and that one in freecycle with a comma in it. Either rsync or the file system on the network drive can't cope with commas.
Now, as to how to fix it!?