Subject: [imapext] General Request for Assignment (imap-keywords) (was: [JMAP] SMIME Attachments)

From: vaibhav singh <[email-masked]>
Date: Mon, 13 Nov 2017 20:40:03 +0530

Hello,

In order to optimize the way attachments inside encrypted mail are treated
by the MUA, I am suggesting the registration of a "$HasEncryptedAttachment"
to the IMAP Keywords registry[RFC5788].

Please find details of the original proposal below:

> Type of Assignment:
>  Requesting for addition of a keyword "$HasEncryptedAttachment" to the
> IMAP Keywords registry[RFC5788].
>
> Registry:
> The registration request is being made for IMAP Keywords
> registry[RFC5788].
>
> Description:
>     This flag can be useful so that the MUA can display an attachment
> icon when displaying
>    messages in the folder, without it having to decrypt the message.
> This flag can also be used to filter
>    encrypted emails with attachments.
>
> Additional Info:
>  The $HasEncryptedAttachment IMAP keyword will be used by IMAP/JMAP
> MUA to
>    specify that the marked encrypted message contains an attachment.
> A MUA-with-keys
>    sets this keyword when it sees a message having an attachment
> before encryption.
>     This flag can be useful so that the MUA can display an attachment
> icon when displaying
>    messages in the folder, without it having to decrypt the message.
> This flag can also be used to filter
>    encrypted emails with attachments.
>
> Once set, any entity (be it the MDA or receiver's MUA) SHOULD not edit
> the flag.
>
> JMAP Message Stores SHOULD be able to store the
> $HasEncryptedAttachment keyword.
> They MUST preserve it on the COPY operation.  The servers MUST
> support the SEARCH KEYWORD $HasEncryptedAttachment.

Please find comments by Mr. Barry Leiba, and my answers below (marked as
<vs>):

1.) There should also be a “has no attachment” keyword, so the MUA doesn’t
have to check either way and only needs to check if neither is set.

 <vs>  Agreed. </vs>

2.) The keyword is poorly named: it’s not about an encrypted attachment,
but about any attachment.  I suppose that in the end there no reason to
mention encryption at all, because it could be used for any message, though
it’s less important for unencrypted ones.

 <vs> Not really sure. </vs>

3.) How will the server know to set the keyword?  The server also can’t
decrypt an incoming message, and so it can’t check.  Where does the
knowledge come from?  It seems that it could only work for messages that
are created locally, with the keyword set at creation.

  <vs> A better use case for the flag could be: the server could set this
flag to true when it sees the mime being decrypted the first time, so that
any scans later on would not have to decrypt the mail again to know the
presence of the attachment inside it </vs>

Will this be useful?

-- 

Regards,
Vaibhav Singh

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Tue, 14 Nov 2017 08:25:23 +0000

Hi,

sorry about the late response. I suck at the moment.

I don't understand the semantics here.

If present, it means that someone has checked and found an encrypted 
attachment. And if absent, it means nothing. There may or may not be an 
encrypted attachment.

So my question is, what advantage does this provide over checking the 
bodystructure?

Arnt

---

From: Bron Gondwana <[email-masked]>
Date: Wed, 15 Nov 2017 01:06:58 +1100

On Tue, 14 Nov 2017, at 19:25, Arnt Gulbrandsen wrote:
> attachment. And if absent, it means nothing. There may or may
> not be an> encrypted attachment.
> 
> So my question is, what advantage does this provide over checking the> bodystructure?

The usecase is that the entire message is encrypted as a single blob,
but it contains an attachment.  The client wants to mark that so that
other clients know it has an attachment.
There are questions about that being a metadata leak that haven't been
addressed.
There are questions that Barry and I spoke about briefly on Saturday
about pairing it with a $HasNoEncryptedAttachment or something so you
can tell that it's been checked already, which led to a general
discussion about paired keywords and what it means if they're both set
and wouldn't it be nice if we had tristate keywords and what about per-message-
annotations and doesn't everything suck.
But yeah, I'm pretty sure the intention was just that clients would use
it to communicate to each other something about the content of the
message inside the related opaque blob that is the message, in a case
where a user has multiple clients that all know how to decrypt the
message, but the server doesn't.
I have no idea where that most  idea of the server setting the flag in
Vaibhav's most recent email came from - the whole point in the original
scenario is that the server doesn't know anything about the content of
the email.
Anyway, I don't think the whole way this would work has been well
thought out yet.
Bron.

--
  Bron Gondwana, CEO, FastMail Pty Ltd
  [email-masked]

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Tue, 14 Nov 2017 14:25:06 +0000

Bron Gondwana writes:
> The usecase is that the entire message is encrypted as a single 
> blob, but it contains an attachment.  The client wants to mark 
> that so that other clients know it has an attachment.

Oh, right.

> There are questions about that being a metadata leak that 
> haven't been addressed.

I thought about those and it seems insignificant. The size of the encrypted 
blob is a good proxy for that information anyway. Big? Attachment. Small? 
No.

> Anyway, I don't think the whole way this would work has been 
> well thought out yet.

AOL. It would seem to serve the case where the user uses two clients, both 
of which display whether a message has an attachment and neither of which 
displays any other information from inside the blob. In particular, neither 
can display an excerpt of the text, or any information about the kind of 
attachment.

Arnt

---

From: Cyrus Daboo <[email-masked]>
Date: Tue, 14 Nov 2017 09:58:42 -0500

Hi Arnt,

--On November 14, 2017 at 2:25:06 PM +0000 Arnt Gulbrandsen 
<[email-masked]> wrote:

> AOL. It would seem to serve the case where the user uses two clients,
> both of which display whether a message has an attachment and neither of
> which displays any other information from inside the blob. In particular,
> neither can display an excerpt of the text, or any information about the
> kind of attachment.
>

I think a much more useful option would be for the first client to decrypt 
the blob and generate an IMAP-like BODYSTRUCTURE which it then encrypts to 
the user's public key and uploads as metadata attached to the original 
message. Then other clients which have the private key can grab the 
metadata and decrypt it to get the full structure of the encrypted blob. 
That way the clients get to learn a lot more about the message than the 
mere fact it contains an attachment.

Note, that it might also be useful to send the encrypted BODYSTRUCTURE data 
as a separate part/header in the encrypted message. That way recipients 
would also gain the benefit of being able to quickly determine what the 
message contains. Of course that additional data does potentially leak some 
information about the message, but that might be acceptable.

-- 
Cyrus Daboo

---

From: Bron Gondwana <[email-masked]>
Date: Wed, 15 Nov 2017 11:17:50 +1100

On Wed, 15 Nov 2017, at 01:58, Cyrus Daboo wrote:
>> AOL. It would seem to serve the case where the user uses two clients,>> both of which display whether a message has an attachment and
>> neither of>> which displays any other information from inside the blob. In
>> particular,>> neither can display an excerpt of the text, or any information
>> about the>> kind of attachment.
>> 
> 
> the blob and generate an IMAP-like BODYSTRUCTURE which it then
> encrypts> to
> the user's public key and uploads as metadata attached to the original> message. Then other clients which have the private key can grab the
> metadata and decrypt it to get the full structure of the
> encrypted blob.> That way the clients get to learn a lot more about the message
> than the> mere fact it contains an attachment.
> 
> Note, that it might also be useful to send the encrypted BODYSTRUCTURE> data
> as a separate part/header in the encrypted message. That way
> recipients> would also gain the benefit of being able to quickly determine
> what the> message contains. Of course that additional data does potentially leak> some

The main value of any non-encrypted facts about the message (like a
keyword about the attachment state) is to allow server-side search.
I agree with Cyrus - the sender could easily attach the BODYSTRUCTURE as
a separate encrypted part, though again that leaks information about the
structure based on the size of the encrypted part - all sorts of side
channel/metadata stuff needs to be thought through, and different people
have different tradeoffs/risk profiles - but then most people don't want
to think that deeply about every message they send, so if you make it
too complex nobody will use it.
Regardless of what gets stored encrypted or who does it however, the
value of a keyword is search/sort.
I still remain unconvinced that the usecase is common enough that any
client would use it, particularly since the first client still needs to
download the message so it can set the flag.
Bron.

---

From: Barry Leiba <[email-masked]>
Date: Wed, 15 Nov 2017 10:43:25 +0800

Vaibhav left off one significant bit of my review:

4.  ...the term “attachment” is a bit odd in today’s usage.  Messages
may have multiple parts.  Whether some or all of those parts appear to
the user as “attachments” depends upon the rendering.  For example, a
two-part message that is multipart/alternative... is there an
“attachment”?  How about a single-part message that is image/jpeg,
with no text at all?  What about if I forward a message and the
forwarded message is included as a separate message/rfc822 part rather
than being included in the main text?

Barry

On Mon, Nov 13, 2017 at 11:10 PM, vaibhav singh
<[email-masked]> wrote:
> 2.) The keyword is poorly named: it’s not about an encrypted attachment, but
> about any attachment.  I suppose that in the end there no reason to mention
> encryption at all, because it could be used for any message, though it’s
> less important for unencrypted ones.
>
> knowledge come from?  It seems that it could only work for messages that are
> created locally, with the keyword set at creation.
>
> _______________________________________________
> imapext mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/imapext
>

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 15 Nov 2017 10:55:09 +0800

> than being included in the main text?

I think "Content-Disposition: attachment" is a good indicator of whether
something is an "attachment". In absence of Content-Disposition any
decision is an heuristic.

---

From: Barry Leiba <[email-masked]>
Date: Wed, 15 Nov 2017 10:58:40 +0800

> decision is an heuristic.

Sure, but the use of Content-Disposition is sporadic and unreliable.

I think that, basically, this issue speaks to a preference for the
suggestion of storing a BODYPART parse on first use, and then being
able to use that for optimization later.

Barry

---

From: Neil Jhaveri <[email-masked]>
Date: Wed, 15 Nov 2017 22:13:39 -0700

> On Nov 14, 2017, at 5:17 PM, Bron Gondwana <[email-masked]> wrote:
> 
> I still remain unconvinced that the usecase is common enough that any client would use it, particularly since the first client still needs to download the message so it can set the flag.

I agree, I just don’t see the value of this being significant enough, since it only helps the second+ client that comes along.

If you are dealing with messages encrypted as a single blob, and want to have a good user experience, you probably have to venture into thick-client territory and proactively download/decrypt the message locally anyways. Otherwise, how can you provide full-text body search, or a UI with a message preview, as many clients seem to be doing nowadays?

Doesn’t that also diminish the need for this keyword?

Have I missed some important use-case?

---

From: "Chris Newman" <[email-masked]>
Date: Thu, 16 Nov 2017 01:54:42 -0600

> other clients know it has an attachment.

Typically "has attachment" is a UI flag. For encrypted mail, the server 
can't provide that information so the client must if it's going to be 
provided. So clients wanting to present that UI flag have to parse 
encrypted messages.

> There are questions about that being a metadata leak that haven't been

It is a deliberate leak of metadata. The nice thing about a leak of 1 or 
2 bits of information is that the scope of the leak is largely 
understood. With an encrypted body structure, I'm more concerned about 
the scope of the information leak as it leaks non-obvious information.

> annotations and doesn't everything suck.

This problem can be addressed by changing the definition as follows:

$ClientSawAttachment => at least one client believes this message has an 
attachment.
$ClientSawNoAttachment => at least one client believes this message does 
not have an attachment.

If both are set, that now has a meaning (specifically that the end-user 
uses at least two different clients that have different opinions on what 
constitutes an attachment and whether the message has one :-).

Basic guidelines for "has an attachment" calculations are:
* Content-Disposition: attachment anywhere in the message means the 
message has an attachment.
* If all parts have Content-Disposition: inline, that means the message 
has no attachment.
For the following four guidelines, implementation-defined exceptions are 
permitted:
* If the client can usefully render all MIME parts in a single view (and 
none are explicitly labelled an attachment), there's probably no 
attachment.
* If there's a MIME part the client can't usefully render, there 
probably is an attachment.
* multipart/related tends to imply that MIME subtree has no attachment 
unless the top-level type of the multipart/related is one the client 
can't render.
* multipart/alternative tends to imply that MIME subtree has no 
attachment if at least one part is a text part the client can render.

This is clearly somewhat heuristic when Content-Disposition is missing, 
but if the keywords are defined as above, I don't see a problem with 
that.

I don't find this issue particularly important given how rarely 
encrypted email is used, but since there seems to be interest from more 
than one implementer, I don't see a compelling reason to block 
registration.

		- Chris

> Anyway, I don't think the whole way this would work has been well

I can't say I find this a particularly useful extension but I think this 
can be documented with sufficient clarity that it serves the intended 
function well enough.