maildir vs. mbox?

Master One · 2007-03-20 18:33:46

AFAIK storing emails in maildir format should be the better option, especially if there are lots of emails in each folder. Unfortunately it get's pretty confusing, when it comes down to the various email clients, so which format they use by default and which they support (if it's true, Thunderbird and Evolution for example only support mbox, but this is not even mentioned on the their websites).

Is mbox more common than maildir?
Is maildir really superior?
Which of the two is better suited, if the email client should easily be exchanged?
Which of the major email clients use maildir or support both formats?

P.S. This is not about mailserver- or IMAP-usage, but about the client side, when you just fetch emails from your ISPs POP-mailaccounts for local storage.

nogoma · 2007-03-20 20:01:00

I personally use/prefer Maildir b/c I like having my mails in individual files. It makes stuff like backup ranges of messages and such easier. I *believe* most major mail clients would support it, but I actually run an IMAP server on my machine at home to serve it up, so I don't know from experience. Actually, from this wikipedia page, it doesn't look like most support Maildir!! Of course, that's wikipedia... Oh well, both the mail clients I use (KMail, mutt/ng) do... Oh, and if your inclined to try it out, running your own local IMAP server isn't too hard (I do it so I can access my mail from both my desktop, where it actually lives, and my notebook)

Master One · 2007-03-20 22:05:19

Why do I always forget about wikipedia when searching for info....

Very informative, didn't know that Evolution does use the maildir format. I'd say Evolution, KMail and mutt/mutt-ng are the major three clients to talk about. As already assumed, wikipedia confirms that Thunderbird uses (a variation of) mbox.

maildir definitely has the advantage, that there is only one typ of it (whereas wikipedia lists mbox, mboxo & mboxrd), individual files provide a greater chance of data recovery in case of filesystem-failure, and external indexing tools (like beagle) can be used. There seems to be no point in using mbox then (except someone has to use a client, which only supports that format).

I'm tempted to setup my own IMAP server at some point, mainly for the same reason as you mentioned.

Is is really possible to access the same maildir archive with two different clients like KMail/Evolution and mutt-ng without problems? I mean, if you fetch new mail or send new mails with the one client, and then use the other one, changes to the maildir archive are correctly recognized (or does any of these clients for example do some indexing on their own, which does not get stored in the maildir folders, and then could cause troubles)?

skymt · 2007-03-20 22:16:47

Master One wrote:

Is is really possible to access the same maildir archive with two different clients like KMail/Evolution and mutt-ng without problems? I mean, if you fetch new mail or send new mails with the one client, and then use the other one, changes to the maildir archive are correctly recognized (or does any of these clients for example do some indexing on their own, which does not get stored in the maildir folders, and then could cause troubles)?

You shouldn't have any trouble with that. The Maildir format was designed to be processed by multiple programs (say, Postfix delivering a few messages and Mutt deleting others) simultaneously.

Master One · 2007-03-20 22:21:57

You mean, it is even save, to have two email clients access the same maildir archive at the same time? Isn't there any file-locking (what happens, if both programs try to access or change the same file)?

skymt · 2007-03-21 00:42:27

Maildir does locking via the file system. Your FS won't allow two processes to write to a file at the same time. Since each message is an individual file, that amounts to message-level locking (rather than mailbox-level like mbox). Problems with race conditions can arise once in a while (see Wikipedia yet again), but they're rare in normal usage.

Damnshock · 2007-03-21 08:00:46

The maildir formart is better IMHO than the mbox.

1- Safer: if, for any reason, a file gets corrupted you'll only lose *one* email
2- Faster: if you have a lot of emails you won't need to open a huge file, just an index file to show the headers on your client and open each mail file when needed
3- As it doesn't have a general lock, you could use many mail clients at the same time ( though in my experience this may cause some problems many times because, generally, mail clients use a "modified" version of the format)
4- Requires more size

About evolution, i think it only supports maildir format while importing (not in normal usage).

Is is really possible to access the same maildir archive with two different clients like KMail/Evolution and mutt-ng without problems? I mean, if you fetch new mail or send new mails with the one client, and then use the other one, changes to the maildir archive are correctly recognized (or does any of these clients for example do some indexing on their own, which does not get stored in the maildir folders, and then could cause troubles)?

You won't have problems as far as you only read *OLD* mails (this means the files in cur directory of each mailbox). As said before, the major mail clients do some kind of inside modifications to the format (that's the reason because beagle needs two different plugins for kmail and evolution).

Damnshock

I don't really think evolution supports maildir format ( only importing).

Master One · 2007-03-21 19:05:30

Any way to find out, how Evolution now really handles emails?

This is weird, I just checked with the Gnome 2.18 VMware image, the folder .evolution/mail contains a "local" and a "pop" folder after setting up one pop-mailaccount. I received 4 emails on that account, which I left in the Inbox together with the autmatic welcome message. The "local" folder contains 18 files which indicate mbox usage, when displaying the file "Inbox" in a texteditor, it contains the 4 received emails and the welcome message. The folder "pop" has a subfolder with the name of the configured pop-email-account, which contains a "cache" folder, in which I find 4 subfolders with the names "02", "3b", "09" and "20, and each contains a single file with a cryptic name (a long hexadecimal number) with one of the 4 received emails inside.

Isn't that confusing? Why should Evolution use a mixture of mbox & maildir?

Has anybody successfully used Evolution and mutt-ng on the same local mail archive?

Damnshock · 2007-03-21 19:15:37

Evolution uses mbox format.

Master One · 2007-03-22 08:30:27

Yes, seems to be true (the wikipedia page is wrong then). Quite strange, isn't it, I mean why should one of the major email clients only support a less superior mailstorage format (same goes for Thunderbird)?

In M$ Windows I used Eudora, and it uses a form of mbox as well, the largest mailbox grew to a size if about 866 MB (that can't be good for a single file!), and the program got utterly slow (the biggest annoyance was that the program virtually froze for several seconds, every time if started looking for new mails). I didn't import this mail archive over to a Linux email client by now, because I still have not decided which client to use, but I am afraid of having such a poor performace again, if the desired email client only supports mbox as well.

Snarkout · 2007-03-22 18:26:22

I use maildir where possible - mbox is a nightmare to maintain, moreso when you're admining 20k mailboxes each 20 to 40 megs in size. One thing I noticed the other day when I had to find an email addy in my sent mail (and I wasn't at home) is that it appears that kmail uses mbox for sent mail by default, and I cannot figure out how to change it. I haven't spent more than 10 min on it, but I thought it was an odd choice. Most people do not *ever* clean out their sent folder, so I can see that becoming an issue quickly for people with attachment-itis.

Damnshock · 2007-03-22 19:37:52

@Master One: those programs support the mbox format because (i'm not sure at all) it's *the* official format. Maildir was an invention for qmail. From the wikipedia:

Daniel J. Bernstein, the author of qmail, djbdns, and various other software wrote the original and only Maildir specification [1]. There have been no followups since by Dan and no effort to turn this into a standard. The specification was written for one particular mail suite (Bernstein's qmail) and is general enough to be implemented in many programs. Over time and many independent implementations a small number of shortcomings have been discovered as documented in this article, but Bernstein has never updated the specification.

As you see, the format specs were designed by an only individual which is not best suited for something like this (furthermore, maildir is unmainteined...)

Anyway, you won't have those kind of problems under evolution nor thunderbird (neither kmail) because they make an index file which lets the program open that HUGE file at the point where it should (no need to read the hole file)

@Snarkout: my sent folder is in maildir format. There's an option in "misc" which lets you select the default format

Damnshock

Snarkout · 2007-03-23 01:56:37

Damnshock wrote:

@Snarkout: my sent folder is in maildir format. There's an option in "misc" which lets you select the default format
Damnshock

I have "By default, message folders on disk are:" set to maildir already - is that what you mean? In any case, I don't really care much, it was just something I noticed the other day and was somewhat on-topic.

lloeki · 2007-03-27 13:43:12

according to wikipedia (again), novell evolution and sylpheed|claws mailbox format is MH: http://en.wikipedia.org/wiki/MH_Message_Handling_System

getmail (fetchmail replacement) can use qmail's maildir format too.

it is easy to spot client software using mbox format, they all have a 'compact' term around (as an option, in the menus or when closing the app), which rewrites the whole mbox file without empty spaces, as when you delete a mail it is overwritten with blank spaces in the mbox file, and not removed (which would e exteremely costy, as you can notice when compacting the mbox file).

mox is really a remnant of the past, when it was not conceived that people would send anything else than short text messages (where the concept was useful: it was saving much space, because small files would be taking at least one block in size), let alone attached files (which defeats the whole purpose of mbox in a grandiose way, and even make things worse, since programatically, the mbox must be seeked around, incl. through the text-encoded attached files).

Last edited by lloeki (2007-03-27 13:53:59)

zodmaner · 2008-02-11 13:39:54

Claws Mail seems to use maildir format.

awagner · 2008-03-06 12:26:54

To warm up this old thread anew:

When I do want to convert from mbox to maildir, what do I have to take into account? I've seen a mb2md package on AUR, I suppose I'll want to use that, no?

But what about:
- folder structure (I have my hundreds of MB personal mail in several mboxes in subfolders of ~/Mail)? Will it automatically be created correspondingly?
- programs that have to be replaced/reconfigured (fetchmail->procmail->spamassassin->mutt->esmtp)? (I've seen some instructions for reconfiguring procmail recipes on the web, and I probably can cope with mutt, but I don't want to miss something. What about the system's spool dir or the cronjob reports sent to root f.ex.?)
- I have some of my mbox files bz2-compressed (and therefore use mutt's compressed-folders-patch). Anything similar for maildir?

Thanks for any insights

Last edited by awagner (2008-03-06 12:32:10)

patroclo7 · 2008-03-06 14:26:13

Virtually anything can be instructed to use maildir: in procmail it is enough to add a / at the end of the destinations.
Mutt autodetect the maildir type as a reader, and there is a configuration option for the spool: (mbox_type). Fetchmail and esmtp deliver through procmail, so there is no problem (if you turn to a real MTA, then e.g. postfix is able to deliver directly to maildir).

Mb2md should be able to do everything you want to do. See http://batleth.sapienti-sat.org/projects/mb2md/ for details.

venky80 · 2009-05-20 23:29:25

does anyone use getmail to import IMAP gmail? how can I import it into kmail?

ngoonee · 2009-05-21 01:05:08

I use offlineimap (its in community). Check out the Evolution wiki, I updated it recently with the necessary settings for a regular sync-ing of IMAP gmail to a folder in your machine. Once its set up, you'll just need to instruct kmail to use that directory. I believe the only 'non-standard' setting you need with regards to the maildir is to use '/' instead of '.' for separators, so that folders are created/used.

nogoma · 2009-05-21 13:47:13

Unfortunately, kmail is a little weird when working w/ Maildirs directly:
http://software.complete.org/software/w … ions#KMail
I just run a local IMAP server (dovecot) which servers up the Maildirs that offlineimap maintains, and connect to that from KMail. Now, to be fair, I haven't used KMail much since KDE4, so they may have fixed the Maildir issues.

Arch Linux

#1 2007-03-20 18:33:46

maildir vs. mbox?

#2 2007-03-20 20:01:00

Re: maildir vs. mbox?

#3 2007-03-20 22:05:19

Re: maildir vs. mbox?

#4 2007-03-20 22:16:47

Re: maildir vs. mbox?

#5 2007-03-20 22:21:57

Re: maildir vs. mbox?

#6 2007-03-21 00:42:27

Re: maildir vs. mbox?

#7 2007-03-21 08:00:46

Re: maildir vs. mbox?

#8 2007-03-21 19:05:30

Re: maildir vs. mbox?

#9 2007-03-21 19:15:37

Re: maildir vs. mbox?

#10 2007-03-22 08:30:27

Re: maildir vs. mbox?

#11 2007-03-22 18:26:22

Re: maildir vs. mbox?

#12 2007-03-22 19:37:52

Re: maildir vs. mbox?

#13 2007-03-23 01:56:37

Re: maildir vs. mbox?

#14 2007-03-27 13:43:12

Re: maildir vs. mbox?

#15 2008-02-11 13:39:54

Re: maildir vs. mbox?

#16 2008-03-06 12:26:54

Re: maildir vs. mbox?

#17 2008-03-06 14:26:13

Re: maildir vs. mbox?

#18 2009-05-20 23:29:25

Re: maildir vs. mbox?

#19 2009-05-21 01:05:08

Re: maildir vs. mbox?

#20 2009-05-21 13:47:13

Re: maildir vs. mbox?

Board footer