Subject: Re: [Jmap] the large email (attachment) problem

From: Jamey Sharp <[email-masked]>
Date: Sun, 1 Aug 2021 09:43:26 -0700

On Sat, Jul 31, 2021 at 01:00:47PM -0400, John Levine wrote:
>It appears that Jamey Sharp  <[email-masked]> said:
>>In the text (plain or HTML) parts of the email, large attachments are
>>still referenced by a URL provided by the sender, as has become common
>>practice today. However, we add new attachment-like parts which serve
>>several purposes: to indicate that a given URL should be treated like an
>>attachment at the recipient, to provide metadata hints, and to provide
>>alternate URLs for retrieving the attachment. ...
>
>Take a look at message/external-body in RFC 2017, 2046 and 2912.  It already has size,
>URL, expiration, and content-features parameters.
>
>We could add a few new attributes to external-body like an optional encryption key.

I'd never heard of message/external-body, and wow, that would be perfect 
if everyone implemented it. The 20 years of personal email archives I've 
managed to keep don't have a single instance of that type in them 
though, which is disappointing: I'm guessing there are few, if any, 
implementers today.

I assume the "normal" way to use this in an HTML email would be to 
assign a Content-ID to the multipart/alternative containing the 
external-body, if there is one, or to the external-body part itself, and 
then reference it using "cid:" links, right? But that doesn't work if 
recipients don't understand message/external-body.

Would this work instead? Encourage recipients to collect the URLs from 
any external-body parts and treat references to those URLs as equivalent 
to "cid:" references to the corresponding part. Then a sender can still 
link to e.g. Google Drive in the HTML and plain-text parts, so that 
recipients that don't understand external-body can at least follow the 
link if they want to, but supporting implementations can also choose to 
treat it fully like an attachment.

If that seems reasonable, then I think this is indeed most of the way 
there, and just needs new stuff for integrity and confidentiality.

Jamey

---

From: "John R Levine" <[email-masked]>
Date: 1 Aug 2021 13:43:30 -0400

> I assume the "normal" way to use this in an HTML email would be to assign a 
> Content-ID to the multipart/alternative containing the external-body, if 
> there is one, or to the external-body part itself, and then reference it 
> using "cid:" links, right? But that doesn't work if recipients don't 
> understand message/external-body.

External-body parts are just like any other MIME part, and can go 
anywhere.  It is my strong impression that the vast majortity of giant 
MIME parts are attached videos or other opaque formats so they appear as 
attachments.  I don't see any reaspn a Content-ID wouldn't work but it 
rarely would be needed.

I agree there is a bootstrap problem.  Some mail programs implemented it 
back in the 1990s when people still dialed in to download their mail, then 
it went away when we all got permanently online, now we need it again as 
the size of bloatware has pulled away from what mail systems do.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

---

From: "Neil Jenkins" <[email-masked]>
Date: Mon, 02 Aug 2021 11:37:47 +1000

> the size of bloatware has pulled away from what mail systems do.

I *think* the bootstrap problem can be solved if most clients support multiple MIME parts correctly (which I think they probably must, because Apple's clients often produce bodies like that). You have something like this:

multipart/mixed
    text/html *(The message body)*
    multipart/alternative
        text/html *(HTML representation of attachments, just like current G Drive etc.)*
        message/external-body *(For external attachment aware clients)*

Neil.

---

From: "Bron Gondwana" <[email-masked]>
Date: Mon, 02 Aug 2021 12:42:26 +1000

>         message/external-body *(For external attachment aware clients)*

We've collected a bunch of great ideas here.  I think the next sensible step is probably some running code and maybe try to get a group of hackathon people trying things at the next IETF.

I'm particularly interested in anyone who knows any clients which currently implement `message/external-body` at all, so we can check that they handle any changes we make to the format.

Bron.
--
  Bron Gondwana, CEO, Fastmail Pty Ltd
  [email-masked]

---

From: Richard Clayton <[email-masked]>
Date: Mon, 2 Aug 2021 10:16:30 +0100

In message <[email-masked]>,
Bron Gondwana <[email-masked]> writes

>    I'm particularly interested in anyone who knows any clients which 
>    currently implement message/external-body at all, so we can check 
>    that they handle any changes we make to the format.

my client (Turnpike) will handle incoming message/external-body but does
not generate it

        https://en.wikipedia.org/wiki/Turnpike_(software)

I still use this software every day (see headers), but I haven't seen an
example of this MIME type for many years (the IETF used to use it for
their notifications of new RFCs and drafts)

BTW: there are surely other WGs where any MIME extension work (if you
want to add content hashes perhaps, to assist in abuse prevention) will
be better handled. If you just want to encourage the use of this MIME
type then I doubt it matters where you do it !

- -- 
richard                                                   Richard Clayton

Those who would give up essential Liberty, to purchase a little temporary 
Safety, deserve neither Liberty nor Safety. Benjamin Franklin 11 Nov 1755

---

From: "John Levine" <[email-masked]>
Date: 2 Aug 2021 15:58:33 -0400

It appears that Neil Jenkins  <[email-masked]> said:
>I *think* the bootstrap problem can be solved if most clients support multiple MIME parts correctly (which I think they probably must, because Apple's clients often
>produce bodies like that). You have something like this:
>
>multipart/mixed

I suppose, but if the HTML fits in the message, what's the point of the external version other
than to debug your MUA?

R's,
John

---

From: "Neil Jenkins" <[email-masked]>
Date: Tue, 03 Aug 2021 06:49:40 +1000

> I suppose, but if the HTML fits in the message, what's the point of the external version other

Sorry, I wasn't clear. The HTML version is a (prettier version) of something like this:

<h2>Attachments</h2>
<a href="https://example.com/download/large/file">whatididonmyholiday.mp4</a>

So if your client doesn't support the external body, it will instead show the sender-generated links to download the attachment (which is exactly what Google, Microsoft etc. currently do). But if it does support the external body, it can show it like any real attachment and offer the advantages we've discussed (immutability, privacy-preserving caching on delivery etc.). This provides a seamless upgrade path to offer the feature.

Neil.

---

From: "John R Levine" <[email-masked]>
Date: 2 Aug 2021 17:48:23 -0400

> So if your client doesn't support the external body, it will instead show the sender-generated links to download the attachment (which is exactly what Google, Microsoft etc. currently do). But if i…

Ah.  Of course.

Regards,