Subject: Re: [Extra] New Version Notification for draft-bosch-sieve-eai-01.txt

From: Ned Freed <[email-masked]>
Date: Wed, 02 Jun 2021 11:12:28 -0700 (PDT)

> It appears that Ned Freed  <[email-masked]> said:
> >I'm afraid this document has a lot of probems, some of which may not be easy to
> >resolve.
> >
> >The biggest problem by far is that the idea of invoking an extension in order to
> >be able to process an EAI message is intrinsicly nonsensical: A message
> >presented to the Sieve interpreter is either an EAI message or it's not, and
> >there's nothing the Sieve interpreter or Sieve author can do to change that. (It
> >may not even know if a given message was intended to be in EAI format - there's
> >no reliable marker for this.) ...

> Yup.  I think it might be possible to redo this draft as advice on what a sieve
> implementation has to do to handle EAI mail, but your sieve code either handles
> EAI mail, in which case it makes sense to let it process EAI messages, or it
> doesn't, so the behavior with EAI messages is undefined.

I'll be happy to suggest text if there's consensus on it being recast as a client
issue, not an extension.

I do think an extension makes some sense in order to be able to redirect to
an EAI address or initiate new EAI messages.

It might also make sense for this to be split into two documents, one dealing
with compliance, the other with any extension. Or not - I have no real
opinion on the point.

				Ned

---

From: Stephan Bosch <[email-masked]>
Date: Wed, 2 Jun 2021 21:00:14 +0200

On 26/05/2021 14:53, Bron Gondwana wrote:
> Awesome.  Are you planning to progress this document further?  I was 
> about to email the list to see if there's interest in meeting at 
> IETF111, and that probably depends if we have any work to do :)

Well, yes, although I am not sure I have time to make progress on this 
very quickly. The aftermath of Corona infection of a months ago 
definitely is not helping. And I am not a great fan of the idea of a 
whole WG waiting for me to finish this. :)

Regards,

Stephan.

> 
> Cheers,
> 
> Bron.
> 
> On Wed, May 26, 2021, at 05:28, Stephan Bosch wrote:
>> Hi,
>>
>> This is just a refresh of the document submission. Apart from my address
>> information, nothing changed.
>>
>> -------- Forwarded Message --------
>>
>> A new version of I-D, draft-bosch-sieve-eai-01.txt
>> has been successfully submitted by Stephan Bosch and posted to the
>> IETF repository.
>>
>> Name: draft-bosch-sieve-eai
>> Revision: 01
>> Title: Sieve: Internationalized Email
>> Document date: 2021-05-25
>> Group: Individual Submission
>> Pages: 8
>> URL: https://www.ietf.org/archive/id/draft-bosch-sieve-eai-01.txt 
>> <https://www.ietf.org/archive/id/draft-bosch-sieve-eai-01.txt>
>> Status: https://datatracker.ietf.org/doc/draft-bosch-sieve-eai/ 
>> <https://datatracker.ietf.org/doc/draft-bosch-sieve-eai/>
>> Html:
>> https://www.ietf.org/archive/id/draft-bosch-sieve-eai-01.html 
>> <https://www.ietf.org/archive/id/draft-bosch-sieve-eai-01.html>
>> Htmlized: https://datatracker.ietf.org/doc/html/draft-bosch-sieve-eai 
>> <https://datatracker.ietf.org/doc/html/draft-bosch-sieve-eai>
>> Diff: https://www.ietf.org/rfcdiff?url2=draft-bosch-sieve-eai-01 
>> <https://www.ietf.org/rfcdiff?url2=draft-bosch-sieve-eai-01>
>>
>> Abstract:
>>     This document defines an extension to the Sieve language called "eai"
>>     which adds full support for internationalized email.
>>
>>
>>
>>
>> The IETF Secretariat
>>
>>
>> _______________________________________________
>> Extra mailing list
>> [email-masked] <mailto:Extra%40ietf.org>
>> https://www.ietf.org/mailman/listinfo/extra 
>> <https://www.ietf.org/mailman/listinfo/extra>
>>
> 
> --
>    Bron Gondwana, CEO, Fastmail Pty Ltd
>    [email-masked]
> 
> 
> 
> _______________________________________________
> Extra mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/extra
>

---

From: Stephan Bosch <[email-masked]>
Date: Wed, 2 Jun 2021 21:28:31 +0200

Hi Ned,

(Ugh.. why does Thunderbird make a mess of quoting your message...)

On 26/05/2021 15:27, Ned Freed wrote:
> no reliable marker for this.)
> 
> And now things start to get sticky: There's no telling what will happen 
> if you
> feed an EAI message into a Sieve implementation that's not itself EAI 
> aware.
> Will it simply process the UTF-8 it finds by leaving it alone (likely)? 
> Or will
> it treat the UTF-8 it finds in headers as illegal - whatever that means? 
> Will it
> know that message/global contains a message, and if so, will it be able 
> to look
> inside of an encoded message/global (probably not)?

> 
> The ineluctable conclusion here is that an email system that supports 
> both EAI
> and Sieve in a compliant way MUST support EAI in Sieve. This is a 
> compliance
> matter, plain and simple, and cannot be handled as an extension. So the 
> first
> task of this document needs to be to define what this form of compliance 
> means.
> It definitely includes saying that the header and address tests must 
> allow for
> unencoded UTF-8, proper handling of message/global, and probably a bunch of
> other stuff.
> 

Yes, that is all true and I agree. But wouldn't it also be necessary to 
know whether the script author is expecting message/global messages? The 
eai extension would indicate that the script was written with the 
possibility of receiving message/global in mind. I haven't read the 
relevant RFCs in a while (and not going to right now), but what is 
supposed to happen when the server has EAI support but the mail 
retrieval agent (IMAP client) does not? I remember something about 
downgrade or placeholder messages. Most solutions I'd imagine would be 
far from ideal. The most crude solution could be to make the recipient 
server reject message/global outright when it somehow knows the 
destination user doesn't have the software installed to read the 
messages. Our software doesn't do much of the EAI stuff yet, so I can't 
say I have implementation/deployment experience. My point is: forcing 
Sieve scripts to handle EAI messages when they were not written with 
such messages in mind could pose problems. What to do? I don't know...

Also, the ManageSieve client would need to know somehow that EAI is 
supported by the server. It could connect to IMAP first and find out, 
but I think that is not the way to handle this. Of course this could be 
a ManageSieve capability then and not a Sieve extension.

> But now things get even stickier: A-labels. EAI extends the message 
> format to
> allow, among other things, U-Labels in domains, but AFAIK doesn't 
> explicitly say
> that A-Labels cannot be used. (Except in the case of domains in trace 
> fields,
> and as a recent discussion has shown, it appears this requirement is being
> widely ignored.)
> 
> Domains appear in a lot more header fields than addresses, and it would be
> asking a lot of a Sieve implementation to know how to parse them all and 
> convert
> A-labels to U-labels. FWIW, I think the right answer is to only perform the
> conversion in the address test, but I could be persuaded  otherwise.
I'd say all Sieve bits that compare addresses (address test, vacation 
:addresses etc.) or domains should normalize them in several ways, 
including the A-label/U-label format.

> 
> Redirect, vacation, notify, and any other action that sends a message 
> are where
> an smtputf8, as opposed to an eai, Sieve extension makes sense. And note 
> that
> all of these actions can send an EAI even if the original message wasn't 
> EAI.
> 

Agreed.

> A careful review of every Sieve extension with an eye to how it interacts
> with EAI needs to be done, but that should be deferred until we're settled
> on the overall approach.
> 
> Finally, I note that there's some odd language in the current document
> about not interpreting encoded-words in EAI messages. This makes no sense
> to me as messsages are often Frankensteinian in origin and can easily
> end up with a mix of UTF-8 and encoded-words.

I guess you're right. It's been a long time since I wrote this, but I 
think I must have picked this up from one of the RFCs, as I find it 
unlikely that I came up with this restriction myself. I'll look that up 
when I continue writing.

Regards,

Stephan.

>> URL:            
>> https://www.ietf.org/archive/id/draft-bosch-sieve-eai-01.txt
>> Htmlized:       
>> https://datatracker.ietf.org/doc/html/draft-bosch-sieve-eai
>> Diff:           
>> https://www.ietf.org/rfcdiff?url2=draft-bosch-sieve-eai-01
> 
>> Abstract:

---

From: Ned Freed <[email-masked]>
Date: Wed, 02 Jun 2021 12:41:43 -0700 (PDT)

> > The ineluctable conclusion here is that an email system that supports both EAI
> > and Sieve in a compliant way MUST support EAI in Sieve. This is a compliance
> > matter, plain and simple, and cannot be handled as an extension. So the first
> > task of this document needs to be to define what this form of compliance means.
> > It definitely includes saying that the header and address tests must allow for

> know whether the script author is expecting message/global messages?

I don't see why. Sieve interpreters by default decode encoded words and
translate them to UTF-8, which effectively changes the message format from
message/rfc822 to message/global. The one place where UTF-8 can occur that it
couldn't previously is in addresses and domains, but since you can already
compare UTF-8 strings to addresses and domain in Sieve (they won't match but
won't produce an error), and variables can contain UTF-8 values, I'm not seeing
a reason why there's any need for an extension to announce this.

And perhaps more to the point, what's an interpreter supposed to do with an EAI
message when the script says it can't handle it? Downgrade? We've tried twice to
define a viable downgrade format, and I can pretty much guarantee that use of
either one is a cure that's much worse than the disease. I for one would not
even consider implementing such a thing.

> The
> possibility of receiving message/global in mind.

And therein lies the problem. Given the way Sieve is defined the overwhelming 
majority of scripts will work fine with EAI messages. (Weren't we clever?) Your
proposal, taken to the logical conclusion, will penalize all of those scripts
for the sake of a few edge cases.

> I haven't read the
> retrieval agent (IMAP client) does not?

You either deny access or downgrade, although in practice I suspect "just send
EAI" happens a lot.

> I remember something about
> far from ideal.

Yes they are.

> The most crude solution could be to make the recipient

Well, I *can* speak from experience. Making an MTA EAI-aware in a way that
actually provides a somewhat satisfactory experience is a real PITA, because you
have to figure out *why* the message was marked as EAI in order to hanle it
optimally.

IMAP is also difficult because of the downgrade issue.

In contrast, converting our core Sieve interpreter to support EAI was easy. It's
entirely a matter of how you handle previously illegal 8bit in a few contexts. 
You probably already have code for that - and if you don't and are passing 8bit
through unchanged it will mostly work - so that's where the changes go.

> a ManageSieve capability then and not a Sieve extension.

Again, I don't see why. If there's no EAI extension there's nothing
to detect. An smtputf8 extension is another matter, of course.

> including the A-label/U-label format.

Much as I wish it were otherwise, there's no defined normalization for
local-parts, so applying a default normalization may actually produce bogus
results. The good news is Sieve supports comparators, and comparators can
specify normalization, including on the address test. Not ideal, but given how
EAI is defined, I don't know what else could be done.

				Ned

---

From: "Bron Gondwana" <[email-masked]>
Date: Thu, 03 Jun 2021 09:59:14 +1000

> whole WG waiting for me to finish this. :)

Oh, we're going to stay open regardless - there will be other documents in future - it's more "is it worth meeting at IETF111" which depends if we have any work to do this cycle.  It sounds like the answer is no.

Cheers,

Bron.

P.S. sorry to hear about the Corona - I hope you recover OK.  I hear that it can have ongoing issues.