Subject: [Extra] IMAP4rev2 and EAI (RFC 6532, 6855)

From: "Chris Newman" <[email-masked]>
Date: Wed, 18 Jul 2018 12:48:41 -0400

I'd like IMAP4rev2 to be EAI-safe.

Summary of significant technical changes:

1. Clients MUST be able to tolerate RFC 6532 messages in the message 
store if they ENABLE IMAP4rev2. This is a big ask in some ways (although 
it depends on client design), but I think it's important and not 
unreasonable.

2. Servers with a RFC 6532-compatible message store MUST accept RFC 6532 
messages via append. I implemented this already. The extra syntax in RFC 
6855 section 4 is useless, I'd recommend dropping that syntax (but I 
would not object if it's included since I can simply treat the 'UTF8' 
append label as a no-op in my implementation). My implementation scans 
the header for 8-bit. If it sees any, it checks if it's valid UTF-8 in 
the header field value. If it's valid the header is allowed and a flag 
will be set in the store to indicate an EAI message. This took me a few 
days to implement (including nearly complete code coverage with unit 
tests).

3. quoted-strings can contain UTF-8 (but SHOULD NOT contain 8-bit that 
is not valid UTF-8). I found this straightforward to implement both on 
the server and my test client. I would not expect this to be problematic 
in other contexts.

4. LIST responses should be UTF-8 instead of MUTF-7. A tricky part here 
is dealing with RFC 5198. RFC 6855 forbids mailbox names that violate 
RFC 5198. However, I'm aware of at least one widely deployed client that 
attempts to create MUTF-7 mailbox names that violate RFC 5198. This 
creates interoperability problems. I think the RFC 5198 text in RFC 6855 
is not sufficient to get interoperability in the real world. I think we 
should require servers to perform NFC normalization on mailbox names. I 
don't have implementation experience with UTF-8 mailbox listings yet; if 
someone else does input would be welcome.

5. Clients SHOULD USE UTF-8 mailbox names in all mailbox commands, and 
servers MUST accept that. I've implemented code to convert between these 
formats, it's some effort but not terribly difficult. Again, the RFC 
5198 issues should be addressed. The tricky case is when a client does: 
"CREATE <5198-incompliant-mbox-name>". The server should either reject 
the name or normalize the name (and I'd prefer this be a MUST). I think 
mandatory server normalization is more likely to work in practice, but 
could still cause issues. Unfortunately 5198 didn't exist when RFC 3501 
was written so we got this wrong in the IMAP base spec. I've written 
code to normalize MUTF-7 using ICU; I did not find that code difficult. 
Many client environments also have simple normalization APIs, such as 
CFStringNormalize(cfMutable, kCFStringNormalizationFormC).

If WG accepts this proposal, I'm willing to do an editing pass on 
IMAP4rev2 to integrate this (although no objections if Alexey wants to 
do it instead).

		- Chris

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 18 Jul 2018 19:05:21 +0100

Hi Chris,

> IMAP4rev2 to integrate this (although no objections if Alexey wants to

How about you do a pass over the document (you can download the latest
XML from datatracker) and the WG can have a look at the result? I think
all of these changes look fine, but I personally would like to have a
look at their cumulative effect.

Best Regards,
Alexey