This is (distantly) related to my previous question, they're both results of my migration from mbox to maildir. I'm doing that so I can rsync my mail folder hierarchy from the system where I actually read my mail to my home system without losing old mails. So, I want to write a utility that can run as a cron job and, maybe, on demand, that will saerch down my mail hierarchy for empty maildir mailboxes and delete them. This involves finding any directory that has empty cur, tmp, and new subdirectories and deleting it. I can see all sorts of horribly clumsy ways of doing it but not any really neat way. Any quick solutions anyone? Amazingly a Google search turned up nothing useful at all, surely there must be a need for a utility like this. -- Chris Green (chris@areti.co.uk) "Never ascribe to malice that which can be explained by incompetence."
Chris Green <chris@areti.co.uk>
This is (distantly) related to my previous question, they're both results of my migration from mbox to maildir. I'm doing that so I can rsync my mail folder hierarchy from the system where I actually read my mail to my home system without losing old mails.
Can't mailsync do that without maildir? Maildir always looked to me like a third-choice format for normal users, after mbox and mh.
This involves finding any directory that has empty cur, tmp, and new subdirectories and deleting it. I can see all sorts of horribly clumsy ways of doing it but not any really neat way.
Any quick solutions anyone?
Quickest to my mind is the fugly find $(find $(find $HOME/Maildir/ -name cur -type d -empty -printf '%h\n') \ -maxdepth 1 -name tmp -type d -empty -printf '%h\n') \ -maxdepth 1 -type d -name new -empty -printf '%h\n' \ | while read dir ; do rmdir -v "$dir/cur" "$dir/new" "$dir/tmp" "$dir/" done which feeds the cur find paths to a limited tmp find and again to a limited new find, then tries to rmdir anything still found. You need to be sure you don't have newlines or other funnies in names. It's a lot more complicated than the equivalent for mboxes (find $HOME/mail/ -type f -size 0 -exec rm -v '{}' ';') Hope that helps, -- MJ Ray - personal email, see http://mjr.towers.org.uk/email.html Work: http://www.ttllp.co.uk/ irc.oftc.net/slef Jabber/SIP ask
On Thu, Nov 03, 2005 at 03:28:23PM +0000, MJ Ray wrote:
Chris Green <chris@areti.co.uk>
This is (distantly) related to my previous question, they're both results of my migration from mbox to maildir. I'm doing that so I can rsync my mail folder hierarchy from the system where I actually read my mail to my home system without losing old mails.
Can't mailsync do that without maildir? Maildir always looked to me like a third-choice format for normal users, after mbox and mh.
Can it? There aren't IMAP servers at both ends, isn't mailsync an IMAP tool? Give mail on systems A and B, where B is the backup system I want to:- Copy any message in B's ~/Mail hierarchy to the same place in A's ~/Mail hierarchy leaving all existing mail on A. I can thus delete old mail on B but have it archived on A. With mbox this is rather difficult, with maildir and rsync it's trivial.
This involves finding any directory that has empty cur, tmp, and new subdirectories and deleting it. I can see all sorts of horribly clumsy ways of doing it but not any really neat way.
Any quick solutions anyone?
Quickest to my mind is the fugly
find $(find $(find $HOME/Maildir/ -name cur -type d -empty -printf '%h\n') \ -maxdepth 1 -name tmp -type d -empty -printf '%h\n') \ -maxdepth 1 -type d -name new -empty -printf '%h\n' \ | while read dir ; do rmdir -v "$dir/cur" "$dir/new" "$dir/tmp" "$dir/" done
Yes, I'd got to the stage of thinking about multiple finds, maybe it's the only way.
which feeds the cur find paths to a limited tmp find and again to a limited new find, then tries to rmdir anything still found. You need to be sure you don't have newlines or other funnies in names.
It's a lot more complicated than the equivalent for mboxes (find $HOME/mail/ -type f -size 0 -exec rm -v '{}' ';')
I don't need to do it at all with mbox as mutt will remove empty mailboxes automatically if you tell it to. -- Chris Green (chris@areti.co.uk) "Never ascribe to malice that which can be explained by incompetence."
Chris Green <chris@areti.co.uk>
On Thu, Nov 03, 2005 at 03:28:23PM +0000, MJ Ray wrote:
Can't mailsync do that without maildir? Maildir always looked to me like a third-choice format for normal users, after mbox and mh.
Can it? There aren't IMAP servers at both ends, isn't mailsync an IMAP tool? [...]
"Mailsync's mailboxes can be on IMAP servers, on the local filesystem in UNIX, MH, or any other format c-client supports" and with sshfs and other network mounts, "local" filesystem need not be local. http://mailsync.sourceforge.net/ I thought I had it grabbing from POP, but maybe I was drunk. Sounds like you're nearly sorted with your current plan now. Please let this list know if you find cool tool for Maildir. -- MJ Ray - personal email, see http://mjr.towers.org.uk/email.html Work: http://www.ttllp.co.uk/ irc.oftc.net/slef Jabber/SIP ask
On Thu, Nov 03, 2005 at 04:30:28PM +0000, MJ Ray wrote:
Chris Green <chris@areti.co.uk>
On Thu, Nov 03, 2005 at 03:28:23PM +0000, MJ Ray wrote:
Can't mailsync do that without maildir? Maildir always looked to me like a third-choice format for normal users, after mbox and mh.
Can it? There aren't IMAP servers at both ends, isn't mailsync an IMAP tool? [...]
"Mailsync's mailboxes can be on IMAP servers, on the local filesystem in UNIX, MH, or any other format c-client supports" and with sshfs and other network mounts, "local" filesystem need not be local. http://mailsync.sourceforge.net/
Ah yes, it does work with local mailboxes but it *doesn't* do what I want. I don't want my two mailboxes to be identical and I'm pretty sure that mailsync will make the two identical. What I want is as I described previously but maybe I wasn't clear. Mail hierarchy A is my active working mail system where I read my mail, with limited storage. Mail hierarchy B is the archive system with lots of disk space. What I want to do is to copy all new messsages in hierarchy A to hierarchy B, that is anything in A but not in B gets transferred. What I *don't* want to happen is that messages deleted from A will also get deleted from B. With maildir, since every message is a unique file, I can just use rsync to copy everything from hierarchy A to hierarchy B, if it's already there rsync wastes no time copying it again. Oh, and another reason that I can't use mailsync, hierarchy A is not on a system with an IMAP server and it's the 'remote' system, I can only run/write/use clever scripts on hierarchy A. -- Chris Green (chris@areti.co.uk) "Never ascribe to malice that which can be explained by incompetence."
Chris Green <chris@areti.co.uk>
Ah yes, it does work with local mailboxes but it *doesn't* do what I want. I don't want my two mailboxes to be identical and I'm pretty sure that mailsync will make the two identical.
It depends how you use it (non-identical is possible) and what access you have to your mail on the remote system (IMAP not required). As I wrote, it sounds like you almost have your current system working for you, so please let the list know if you solve the problems with Maildir. -- MJ Ray - personal email, see http://mjr.towers.org.uk/email.html Work: http://www.ttllp.co.uk/ irc.oftc.net/slef Jabber/SIP ask
On Fri, Nov 04, 2005 at 11:10:52AM +0000, MJ Ray wrote:
Chris Green <chris@areti.co.uk>
Ah yes, it does work with local mailboxes but it *doesn't* do what I want. I don't want my two mailboxes to be identical and I'm pretty sure that mailsync will make the two identical.
It depends how you use it (non-identical is possible) and what access you have to your mail on the remote system (IMAP not required).
As I wrote, it sounds like you almost have your current system working for you, so please let the list know if you solve the problems with Maildir.
I'm taking a harder look at mailsync as my life would be easier if I can stay with mbox files. It looks as if the -d option to mailsync may do what I want and it also looksas if it may be able to work through an ssh connection. If these both do what it seems as if they do then mailsync may be a better answer than going maildir. -- Chris Green (chris@areti.co.uk) "Never ascribe to malice that which can be explained by incompetence."
On Thu, 2005-11-03 at 15:28 +0000, MJ Ray wrote:
Can't mailsync do that without maildir? Maildir always looked to me like a third-choice format for normal users, after mbox and mh.
What's so bad about maildir? Personally it seems a lot more sane to me that just having one huge file with all my messages concatenated together. I am not familiar with mh though. Why can't we have a proper database for holding our mailboxes, does this exist ? Surely it could help with advanced searches etc ? Also for multi user / Multi Mailbox systems it could do what exchange is supposed to do and only hold one copy of each attachment regardless of how many mailboxes link to it.
On Thu, Nov 03, 2005 at 09:24:26PM +0000, Wayne Stallwood wrote:
On Thu, 2005-11-03 at 15:28 +0000, MJ Ray wrote:
Can't mailsync do that without maildir? Maildir always looked to me like a third-choice format for normal users, after mbox and mh.
What's so bad about maildir? Personally it seems a lot more sane to me that just having one huge file with all my messages concatenated together.
It makes the computer/programmer side of things easier but it spoils the users' view of things to my mind. Although (I think) I'm changing from mbox to maildir it's because I want to take advantage of some of the technical advantages, I'm losing some usability in the process. Maildir is inherently more difficult to manage because a mail folder is so complex, it's a directory with three sub-directories and files with messy long names. Mbox folders are just files. So if I'm trying to do things 'manually' (i.e. from the command line) it's much, much easier with mbox. For example grep just works without any clever additions, for maildir you need special tools for searching messages. Another example is that it's easy to automatically remove empty mbox folders, a zero length file is empty. As this thread has indicated it's decidedly non-trivial to remove empty maildir mailboxes automatically.
I am not familiar with mh though.
It's a 'not so good' but simpler maildir really. Separate files for mail messages but not designed so that locking is unnecessary.
Why can't we have a proper database for holding our mailboxes, does this exist ? Surely it could help with advanced searches etc ? Also for multi user / Multi Mailbox systems it could do what exchange is supposed to do and only hold one copy of each attachment regardless of how many mailboxes link to it.
The UW (originators of IMAP) view is that the two 'best' places to be are either mbox (which is why UW IMAP works only with mbox) or a database such as Cyrus. The problem with the database approach is that there is no universal standard so you can't pick and choose your MUA if you are using a database approach. OK, Cyrus is an IMAP server with a database behind it but I'm not convinced that IMAP is the answer to everything either. I've yet to find an MUA that works well (and consitently) with IMAP. -- Chris Green (chris@areti.co.uk) "Never ascribe to malice that which can be explained by incompetence."
** Chris Green <chris@areti.co.uk> [2005-11-04 09:53]:
On Thu, Nov 03, 2005 at 09:24:26PM +0000, Wayne Stallwood wrote:
On Thu, 2005-11-03 at 15:28 +0000, MJ Ray wrote:
Can't mailsync do that without maildir? Maildir always looked to me like a third-choice format for normal users, after mbox and mh.
What's so bad about maildir? Personally it seems a lot more sane to me that just having one huge file with all my messages concatenated together.
It makes the computer/programmer side of things easier but it spoils the users' view of things to my mind. Although (I think) I'm changing from mbox to maildir it's because I want to take advantage of some of the technical advantages, I'm losing some usability in the process.
Surely the users' view of things is dependent on their mail client and totally transparant to mbox, maildir, mh, database or whatever. I'm curious to know what usability is being lost from the switch from mbox to maildir. I switched some time back because I found mbox files a pain to work with if I wasn't using a mail client, and they were also getting a bit slow and cumbersome. Maildir made things a lot easier.
Maildir is inherently more difficult to manage because a mail folder is so complex, it's a directory with three sub-directories and files with messy long names. Mbox folders are just files. So if I'm trying to do things 'manually' (i.e. from the command line) it's much, much easier with mbox. For example grep just works without any clever additions, for maildir you need special tools for searching messages. Another example is that it's easy to automatically remove empty mbox folders, a zero length file is empty. As this thread has indicated it's decidedly non-trivial to remove empty maildir mailboxes automatically.
Different view points I guess, but I find Maildir easier to do things with manually since each piece of mail is a different file and if it is new it is in a different directory. I guess I don't automate many things with mail folders (bar filtering on the way in), but then I'm generally a bit paranoid about deleting things in general! <snip>
Why can't we have a proper database for holding our mailboxes, does this exist ? Surely it could help with advanced searches etc ? Also for multi user / Multi Mailbox systems it could do what exchange is supposed to do and only hold one copy of each attachment regardless of how many mailboxes link to it.
The UW (originators of IMAP) view is that the two 'best' places to be are either mbox (which is why UW IMAP works only with mbox) or a database such as Cyrus.
The problem with the database approach is that there is no universal standard so you can't pick and choose your MUA if you are using a database approach. OK, Cyrus is an IMAP server with a database behind it but I'm not convinced that IMAP is the answer to everything either. I've yet to find an MUA that works well (and consitently) with IMAP.
Yes, Cyrus I wrote off very early on because it locked my mail away in a format I couldn't access - I tend to like local access to my mail with Mutt and secondary access if necessary through an IMAP client (although I've been primarily an IMAP user for a year or so and have just switched back to Mutt). Until I moved onto Linux and Mutt I was a Netscape/IMAP user for a long time (mid 90s iirc and Netscape Messaging server). ** end quote [Chris Green] -- Paul Tansom | Aptanet Ltd. | http://www.aptanet.com
On Fri, Nov 04, 2005 at 12:07:55PM +0000, Paul Tansom wrote:
** Chris Green <chris@areti.co.uk> [2005-11-04 09:53]:
On Thu, Nov 03, 2005 at 09:24:26PM +0000, Wayne Stallwood wrote:
On Thu, 2005-11-03 at 15:28 +0000, MJ Ray wrote:
Can't mailsync do that without maildir? Maildir always looked to me like a third-choice format for normal users, after mbox and mh.
What's so bad about maildir? Personally it seems a lot more sane to me that just having one huge file with all my messages concatenated together.
It makes the computer/programmer side of things easier but it spoils the users' view of things to my mind. Although (I think) I'm changing from mbox to maildir it's because I want to take advantage of some of the technical advantages, I'm losing some usability in the process.
Surely the users' view of things is dependent on their mail client and totally transparant to mbox, maildir, mh, database or whatever. I'm curious to know what usability is being lost from the switch from mbox to maildir. I switched some time back because I found mbox files a pain to work with if I wasn't using a mail client, and they were also getting a bit slow and cumbersome. Maildir made things a lot easier.
Well with my MUA I lose a couple of things when moving from mbox to maildir:- I can no longer ask it to automatically delete empty mailboxes, as you can see from this thread deleting an empty maildir mailbox is non trivial, so MUAs don't offer the option. I no longer get to see folder sizes in the mail folder listing, again because it's buried a few directory layers down from the MUA's point of view. I suspect that many other mail clients won't even offer the above functions, but whatever they offer I think you will find that there are subtle differences according to the underlying type of mailbox.
The problem with the database approach is that there is no universal standard so you can't pick and choose your MUA if you are using a database approach. OK, Cyrus is an IMAP server with a database behind it but I'm not convinced that IMAP is the answer to everything either. I've yet to find an MUA that works well (and consitently) with IMAP.
Yes, Cyrus I wrote off very early on because it locked my mail away in a format I couldn't access - I tend to like local access to my mail with Mutt and secondary access if necessary through an IMAP client (although I've been primarily an IMAP user for a year or so and have just switched back to Mutt). Until I moved onto Linux and Mutt I was a Netscape/IMAP user for a long time (mid 90s iirc and Netscape Messaging server).
Yes, I use mutt too for the same sort of reasons I suspect. You just don't notice/use the things that you lose with mutt and maildir! :-) I find that mutt and IMAP still isn't as easy and transparent as mutt with local files. It is one of the better IMAP clients though. -- Chris Green (chris@areti.co.uk) "Never ascribe to malice that which can be explained by incompetence."
Wayne Stallwood <ALUGlist@digimatic.plus.com>
What's so bad about maildir? [...]
Splattering one mailbox across three directories and N files seems pretty wasteful and is a bit of a pain to process. sed, mboxgrep and other tools work very fast with mbox...
I am not familiar with mh though.
It's file-per-mail in one directory. Maildir uses three to avoid causing locking problems with defective MTAs or filesystems. It has lots of collatoral damage, like dictating one set of flags onto all MUAs handling the maildir, encoded into the filenames. I'm not sure, but I think DJ Bernstein either invented it or likes it a lot, which may indicate to many which planet it's on.
Why can't we have a proper database for holding our mailboxes, does this exist ? [...]
Most people get scared if their email isn't easy to backup. A database-backed emailfs or an email-optimised search would be a cool way to do that. Hope that helps, -- MJ Ray - personal email, see http://mjr.towers.org.uk/email.html Work: http://www.ttllp.co.uk/ irc.oftc.net/slef Jabber/SIP ask
On 05-Nov-05 MJ Ray wrote:
Wayne Stallwood <ALUGlist@digimatic.plus.com>
What's so bad about maildir? [...]
Splattering one mailbox across three directories and N files seems pretty wasteful and is a bit of a pain to process. sed, mboxgrep and other tools work very fast with mbox...
I am not familiar with mh though.
It's file-per-mail in one directory.
But you can have several directories, and sub-directories too. Each file's name is a number (at any rate as implemented in my MUA, XFMail). It does make searching mail content very straightforward: a 'grep -l [-r] ..." and you get a list of matching files. (I have 84222 files in 624 directories and subdirectories.) Best wishes, Ted. -------------------------------------------------------------------- E-Mail: (Ted Harding) <Ted.Harding@nessie.mcc.ac.uk> Fax-to-email: +44 (0)870 094 0861 Date: 06-Nov-05 Time: 01:35:30 ------------------------------ XFMail ------------------------------
(Ted Harding) <Ted.Harding@nessie.mcc.ac.uk>
On 05-Nov-05 MJ Ray wrote:
Wayne Stallwood <ALUGlist@digimatic.plus.com>
I am not familiar with mh though. It's file-per-mail in one directory. But you can have several directories, and sub-directories too.
Thanks for the correction. I totally missed "per mailbox" out. -- MJR
participants (5)
-
Chris Green -
MJ Ray -
Paul Tansom -
Ted.Harding＠nessie.mcc.ac.uk -
Wayne Stallwood