I'm considering writing a python script to do some mailbox manipulation but I'm finding the documentation a little confusing, or at least a little lacking.
I'm looking at:- http://docs.python.org/lib/mailbox-maildir.html http://docs.python.org/lib/mailbox-mbox.html
What I can't quite get my mind round is why the Maildir sub-class of Mailbox has list_folders(), get_folder() etc. whereas the Mbox sub-class doesn't have these methods.
From the user's point of view maildir and mbox mail work the same, you
can have a list of mbox folders in exactly the same way as you can have a list of maildir folders.
In fact the Maildir sub-class documentation is not at all clear about what you get when you call list_folders(), a list of folders at what level in the hierarchy?
Chris G cl@isbd.net wrote:
http://docs.python.org/lib/mailbox-maildir.html http://docs.python.org/lib/mailbox-mbox.html
[...]
In fact the Maildir sub-class documentation is not at all clear about what you get when you call list_folders(), a list of folders at what level in the hierarchy?
Warning: I'm a postmaster, but my python isn't expert.
Under the current Maildir, I guess, but you're right that it's not at all clear. Maildir supports nested folders (directories in directories) which Mbox doesn't (can't have files in files, usually). I think the key section of the explanation is:
"Folders of the style introduced by the Courier mail transfer agent are also supported. Any subdirectory of the main mailbox is considered a folder if "." is the first character in its name."
Hope that explains,
On Mon, Aug 13, 2007 at 10:32:56AM +0100, MJ Ray wrote:
Chris G cl@isbd.net wrote:
http://docs.python.org/lib/mailbox-maildir.html http://docs.python.org/lib/mailbox-mbox.html[...]
In fact the Maildir sub-class documentation is not at all clear about what you get when you call list_folders(), a list of folders at what level in the hierarchy?
Warning: I'm a postmaster, but my python isn't expert.
Under the current Maildir, I guess, but you're right that it's not at all clear. Maildir supports nested folders (directories in directories) which Mbox doesn't (can't have files in files, usually). I think the key section of the explanation is:
"Folders of the style introduced by the Courier mail transfer agent are also supported. Any subdirectory of the main mailbox is considered a folder if "." is the first character in its name."
Hope that explains,
Yes, probably, though I don't think many MUAs support nested folders as you describe when using maildir. It's even a bit questionable whether it's allowed by the original maildir specification as I think you were supposed to have *only* cur, new and tmp sub-directories in a maildir folder.
For example Dan Bernstein himself says:-
Can a maildir contain more than tmp, new, cur? Yes:
* .qmail: used to do direct deliveries with qmail-local. * bulletintime: empty file, used by system-wide bulletin programs. * bulletinlock: empty file, used by system-wide bulletin programs. * seriallock: empty file, used to serialize AutoTURN.
Anyway I'll press on and see what actually happens, I may even get around to reporting the result back here! :-)
Thanks for the help.
On Mon, Aug 13, 2007 at 01:19:39PM +0100, Chris G wrote:
On Mon, Aug 13, 2007 at 10:32:56AM +0100, MJ Ray wrote:
Chris G cl@isbd.net wrote:
http://docs.python.org/lib/mailbox-maildir.html http://docs.python.org/lib/mailbox-mbox.html[...]
In fact the Maildir sub-class documentation is not at all clear about what you get when you call list_folders(), a list of folders at what level in the hierarchy?
Warning: I'm a postmaster, but my python isn't expert.
Under the current Maildir, I guess, but you're right that it's not at all clear. Maildir supports nested folders (directories in directories) which Mbox doesn't (can't have files in files, usually). I think the key section of the explanation is:
"Folders of the style introduced by the Courier mail transfer agent are also supported. Any subdirectory of the main mailbox is considered a folder if "." is the first character in its name."
Well experimenting confirms the above, sort of, to me it seems distinctly broken!
E.g. I /home/chris/Mail I have the following (maildir) folders:-
. FcompParts family garden internet kyle-mutt money riding software .. FnewScientist france hardware invites leisure mud selling telecoms .in Fos2 freecycle holidays isbd lets pending sentmail vehicles Fcds boating friends houseHome jobs mairix personal shopping work
However if I execute list_dirs() in /home/chris/Mail it just gives me [.in] and that's the lot!
Chris G cl@isbd.net wrote:
E.g. I /home/chris/Mail I have the following (maildir) folders:-
. FcompParts family garden internet kyle-mutt money riding software .. FnewScientist france hardware invites leisure mud selling telecoms .in Fos2 freecycle holidays isbd lets pending sentmail vehicles Fcds boating friends houseHome jobs mairix personal shopping workHowever if I execute list_dirs() in /home/chris/Mail it just gives me [.in] and that's the lot!
That's working exactly as specified - Maildir subfolder directory names must begin with a "."
I don't know why, but that's how it is and it's confused me several times before. It's not explained in http://www.courier-mta.org/maildir.html
Anyone know why it's like this?
Help!
On Mon, Aug 13, 2007 at 08:14:31PM +0100, MJ Ray wrote:
Chris G cl@isbd.net wrote:
E.g. I /home/chris/Mail I have the following (maildir) folders:-
. FcompParts family garden internet kyle-mutt money riding software .. FnewScientist france hardware invites leisure mud selling telecoms .in Fos2 freecycle holidays isbd lets pending sentmail vehicles Fcds boating friends houseHome jobs mairix personal shopping workHowever if I execute list_dirs() in /home/chris/Mail it just gives me [.in] and that's the lot!
That's working exactly as specified - Maildir subfolder directory names must begin with a "."
I don't know why, but that's how it is and it's confused me several times before. It's not explained in http://www.courier-mta.org/maildir.html
Anyone know why it's like this?
I don't think there is *any* requirement for maildir sub-folders to start with '.', it's a Courier thing, not a maildir thing. I run mutt with maildirs and nearly all my folders and sub-folders *don't* start with '.', the .in is just a special case which I have because I don't want it to be visible in mutt.
Chris G cl@isbd.net wrote:
On Mon, Aug 13, 2007 at 08:14:31PM +0100, MJ Ray wrote:
That's working exactly as specified - Maildir subfolder directory names must begin with a "." [...]
I don't think there is *any* requirement for maildir sub-folders to start with '.', it's a Courier thing, not a maildir thing. [...]
Think what you like, but at least Dovecot seems to share the requirement. As you note, basic maildir seems not to allow folders at all. The Python class seems to be using Courier maildir. It wouldn't be the first time mutt coped with stuff that doesn't follow specs.
Regards,
On Mon, Aug 13, 2007 at 10:30:11PM +0100, MJ Ray wrote:
Chris G cl@isbd.net wrote:
On Mon, Aug 13, 2007 at 08:14:31PM +0100, MJ Ray wrote:
That's working exactly as specified - Maildir subfolder directory names must begin with a "." [...]
I don't think there is *any* requirement for maildir sub-folders to start with '.', it's a Courier thing, not a maildir thing. [...]
Think what you like, but at least Dovecot seems to share the requirement. As you note, basic maildir seems not to allow folders at all. The Python class seems to be using Courier maildir. It wouldn't be the first time mutt coped with stuff that doesn't follow specs.
There's no reason, in this case, that it should "follow spec", in mutt you *tell* it where the mail dirs are.... unless you're using imap, at which point the imap server tells you what's available. It's not a mutt bug, it's an IMAP server bug, except, of course, it isn't. Becuase the sysadmin tells you which imap server you're running, and how the mail folders will be named on the server.
Don't pick on mutt when it's not it's fault, dear boy. It's purely the fault of IMAP servers deciding that folders and subfolders begin with ".". For instance, the dovecot server that I get my mail from has a set of folders starting with ., for convienience to my sanity, I have symlinks to those folders without the starting "." and tell mutt where the folders are, and when I use offlineimap, even INBOX ends up in it's own directory INBOX, and then the config really doesn't want starting "." because offlineimap has named them without the starting ".".
Sheesh - you tell something where to find it's Maildirs and then expect it to ignore them to follow some undefined spec?! You're all damned crazy!
Python purely follows the most wide spread "spec" of the lot, i.e. Maildir subfolders will begin with a ., and inside those folders could be more folders.
Simple, really.
You want it to know different, discount new,cur,tmp and read the directory names from the Maildir for other Maildirs hanging underneath, it's not rocket science.
Cheers,
Brett Parker iDunno@sommitrealweird.co.uk wrote:
On Mon, Aug 13, 2007 at 10:30:11PM +0100, MJ Ray wrote:
[...] It wouldn't be the first time mutt coped with stuff that doesn't follow specs.
[...] Don't pick on mutt when it's not it's fault, dear boy. [...]
I was praising it! Have another sherbet and re-read.
Sheesh - you tell something where to find it's Maildirs and then expect it to ignore them to follow some undefined spec?! You're all damned crazy! [...]
undefined spec? It's defined in the courier maildir man page. The reasons for that definition are not explained - anyone know them?
Giving up trying to compile WxPython for now,
On Tue, Aug 14, 2007 at 02:03:59AM +0100, MJ Ray wrote:
Brett Parker iDunno@sommitrealweird.co.uk wrote:
On Mon, Aug 13, 2007 at 10:30:11PM +0100, MJ Ray wrote:
[...] It wouldn't be the first time mutt coped with stuff that doesn't follow specs.
[...] Don't pick on mutt when it's not it's fault, dear boy. [...]
I was praising it! Have another sherbet and re-read.
Sheesh - you tell something where to find it's Maildirs and then expect it to ignore them to follow some undefined spec?! You're all damned crazy! [...]
undefined spec? It's defined in the courier maildir man page. The reasons for that definition are not explained - anyone know them?
Yes, but that's just Courier's implementation of Maildir which a number of other programs have followed.
On Mon, Aug 13, 2007 at 10:30:11PM +0100, MJ Ray wrote:
Chris G cl@isbd.net wrote:
On Mon, Aug 13, 2007 at 08:14:31PM +0100, MJ Ray wrote:
That's working exactly as specified - Maildir subfolder directory names must begin with a "." [...]
I don't think there is *any* requirement for maildir sub-folders to start with '.', it's a Courier thing, not a maildir thing. [...]
Think what you like, but at least Dovecot seems to share the requirement. As you note, basic maildir seems not to allow folders at all. The Python class seems to be using Courier maildir. It wouldn't be the first time mutt coped with stuff that doesn't follow specs.
The Courier way of doing things is not to have nested sub-directories at all, it just delimits them with dots and creates all the sub-directories at one level, e.g. :-
/home/isbd/Mail # root of all maildirs
/home/isbd/Mail/.folder1 /home/isbd/Mail/.folder2 /home/isbd/Mail/.folder2.sub-folder2a /home/isbd/Mail/.folder2.sub-folder2b /home/isbd/Mail/.folder2.sub-folder2b.sub-sub-folder2b1 ... etc.
This is fine when the structure is hidden behind an IMAP server (as it is with Dovecot or Courier) but it's *horribly* clumsy for direct use with an MUA. There's no way that I want my local mail hierachy to be like that, much of the point is that it can be easily navigated 'by hand' as it were and I can move stuff around, rename it, etc. myself and with scripts. Names like the ones that Courier uses make this really messy, for example if I wanted to rename folder2 to folderX it's rather difficult because, with the Courier way, I have to rename a whole lot of folders rather than just one. I expect it could be automated but why make things difficult when it's far simpler to just use real directories?
The trouble is (I suspect) that very few people now work with a live Unix style mail spool on the local system so the complexity or otherwise of what underlies their mail hierarchy doesn't affect them. I still much prefer to have my mail delivered to 'old' Unix way and, for this, the simplicity of the file naming etc. is important.
Sorry to ramble on! :-)
By the way the Python Mailbox class (in Python 5.1) does a lot of the things I want very simply. The list_folders() for Maildir isn't something I really needed to use so its odd operation isn't an issue.
In particular it's trivially easy to write a Python script to convert from mbox to Maildir (or back):-
#!/usr/bin/python # # # Mail manipulation # import mailbox import email import sys
mdir = mailbox.Maildir(sys.argv[1]) # argv[1] is maildir mbox = mailbox.mbox(sys.argv[2]) # argv[2] is mbox
for mdir_key in mdir.iterkeys(): mdir_str = mdir.get_string(mdir_key) mbox.add(mdir_str)
mbox.close()
It needs a few checks for non-existent folders and things like that but it works OK just with the above. The mbox.add() function is also just what I want for another utility I'm writing for merging mail into an archive. Previously I have used formail and mutt scripts neither of which are particularly elegant or simple.
-
Brett Parker -
Chris G -
MJ Ray