How can the maildir format accomodate flags/UIDs on recent messages?

Question

I am using the maildir (maildir++, actually) email backend format for storing messages on a custom IMAP server. I have a good understanding of the relevant email protocols and maildir format, but this is one question I've had for a while that's been bothering me a bit. The maildir specification, relatively unchanged for decades now, basically says:

When a cognizant maildir-reading process (either a POP or IMAP server, or a mail user agent acting locally) finds messages in the new directory, it must move them to cur. It is just a means to notify the user "you have X new messages".[11] This moving needs to be done using rename(), as the non-atomic link-then-unlink technique may result in duplicated messages. An informational suffix is appended to filenames at this stage. It consists of a colon (to separate the unique part of the filename from the actual information), a "2", a comma and various flags. The "2" specifies the version of the information that follows the comma. "2" is the only currently officially specified version, "1" being an experimental version. The specification defines flags that show whether the message has been read, deleted and so on: the initial (capital) letter of "Passed", "Replied", "Seen", "Trashed", "Draft", and "Flagged".[7] Dovecot uses lowercase letters to match 26 IMAP keywords,[6] which may include standardised keywords, such as $MDNSent, and user-defined flags. - https://en.wikipedia.org/wiki/Maildir#Technical_operation

When you move a file from new to cur, you have to change its name from uniq to uniq:info - http://cr.yp.to/proto/maildir.html

For the most part, this seems to work fairly well. Any mail delivering process can just dump messages in new, and the IMAP server can find these in new and move them to cur. Overall, it seems very coherent; I do not have to store the \Recent flag explicitly. It is set implicitly if the message is in new but not in cur.

However, there are a few problems I've identified over time with the maildir format:

• UIDs are not assigned until a message is moved from new to cur. In other words, messages with the \Recent flag do not have a UID. This is generally not an issue, but it does make it impossible to provide UIDs for the COPYUID response, and appended messages must be moved to cur immediately to provide the APPENDUID, causing them to lose their \Recent status.

• Flags cannot be stored for \Recent messages. Most IMAP servers, however, don't seem to have an issue with this. Appended emails (e.g. to Sent) can still be marked as seen and recent at the same time. In the maildir format, however, these would seem to be mutually exclusive: a message cannot be recent and have other flags at the same time.

• Sieve allows users to add flags to messages. But it may make sense to keep the message as "Recent", but maildir does not facilitate this.

This is all at least based on my interpretation of the base maildir specs. I'm wondering if maildir has evolved in some way to be able to handle these? The most obvious thing to me would be to simply do all the "tagging" when a message is put into new, not cur. e.g. assign the UID then and allow it to have flags then. However, this would seemingly break mail delivering applications using the maildir format that just expect to be able to dump messages there. At the very least, it seems inconsistent, but I don't know how much of a problem would be.

Is there a recommended approach of supporting flags and UIDs for \Recent messages with maildir backends? Everything I can find on maildir spec-wise is about 20 years old and doesn't really detail how to deal with these edge cases. I'm not looking for code, just a general algorithm like the original maildir spec prescribes.

Answer 1

The evolution that's happened is that clients mostly disregard \recent and various servers don't set it, ever. \recent provides a poor experience when using many modern clients, since messages lose that flag when a phone leaves a WLAN and reconnects via mobile data, which happens all the time. That looks unprovoked to people, puzzling, so clients mostly just don't show \recent.

A maildir-based server that ignores \recent can move messages to cur as soon as it sees them, assign an UID then, set flags, etc.