Subject: [Extra] SNIPPET (now PREVIEW) draft update

From: Michael Slusarz <[email-masked]>
Date: Mon, 5 Nov 2018 09:30:19 -0700 (MST)

I've submitted an update to the (formerly snippet, now) preview draft incorporating feedback received from this list and other implementers.
 
Main change is the name - this is now "PREVIEW" instead of "SNIPPET".  Also, increased recommended preview size to 200 characters (previously: 100).
 
Will not be able to remotely join IETF 103 meeting as I am traveling, but wanted to make sure this draft was updated before then.

michael

---

From: "Bron Gondwana" <[email-masked]>
Date: Mon, 05 Nov 2018 21:44:23 -0500

Thanks for submitting this! I'll make sure it gets discussed in the meeting. I'm just putting together the agenda this morning. 
 
Cheers, 
 Bron. 
 
On Tue, Nov 6, 2018, at 03:30, Michael Slusarz wrote: 
> _______________________________________________ 
> Extra mailing list 
> [email-masked] 
> https://www.ietf.org/mailman/listinfo/extra 
>  
 
-- 
 Bron Gondwana, CEO, FastMail Pty Ltd 
 [email-masked]

---

From: "Bron Gondwana" <[email-masked]>
Date: Thu, 08 Nov 2018 07:03:58 -0500

Hi Michael, 
 Thanks again for submitting this, and apologies that I haven't done a thorough review yet - it's on my todo list and I expect to do so soon.
 The ONLY remaining question was length. 200 is good, but JMAP says 255 and consistency will make it better. Unless you have a strong reason to lock it down to 200, would you consider changing it to "no more than 255 octets" with an understanding that there's no requirement that servers have to store that much, just that clients must be willing to accept that much. That will keep everything consistent.
 
Other than this, we believe we're ready to take this document to working group last call, which I intend to do once I've heard back from you - it's easy to fix this up during last call. 
 
Thanks for your work and for getting us the update in time for this meeting. 
 Cheers,
 Bron.

---

From: Timo Sirainen <[email-masked]>
Date: Thu, 8 Nov 2018 16:08:00 +0200

Hmm. You're talking about octets and so far Dovecot (and Michael's draft) has been talking about characters. So with a maximum of 4 bytes per UTF-8 character and Dovecot's current 100 character limit, it could already be sending 400 octets..

We chose characters rather than octets because it makes more sense from the client's point of view to show approximately the same amount of text no matter what characters are being used.

> On 8 Nov 2018, at 14.03, Bron Gondwana <[email-masked]> wrote:
> 
> Hi Michael,
> 
>   [email-masked] <mailto:[email-masked]>
> 
> 
> [email-masked] <mailto:[email-masked]>
> https://www.ietf.org/mailman/listinfo/extra <https://www.ietf.org/mailman/listinfo/extra>

---

From: "Bron Gondwana" <[email-masked]>
Date: Thu, 08 Nov 2018 09:42:34 -0500

The arguments for octets are storage space and network traffic, which is octets, but I do see your point. Should we say 255 characters maximum then - obviously servers can still do less, and clients can choose to truncate. 
 
Either way, we'll make sure JMAP aligns. 
 Bron.

---

From: Stuart Brandt <[email-masked]>
Date: Thu, 8 Nov 2018 19:50:18 -0500

I know I brought this up years ago, but I'm not sure I ever saw solid 
resolution. If, as proposed, the Preview is mutable, then are we not 
addressing how to efficiently identify and sync changes that might have 
been made to it? If I were a client dev, I'd want to have the most 
recent Preview for a message because a new version likely means:
a) server fixed a defective preview
b) server improved preview's value-to-user

Mutable data without an efficient way to sync leads to ugly polling.

- Stu

On 11/8/2018 7:03 AM, Bron Gondwana wrote:
> Hi Michael,
>
> Thanks again for submitting this, and apologies that I haven't done a
> thorough review yet - it's on my todo list and I expect to do so soon.
>
> The ONLY remaining question was length.  200 is good, but JMAP says 255
> and consistency will make it better.  Unless you have a strong reason to
> lock it down to 200, would you consider changing it to "no more than 255
> octets" with an understanding that there's no requirement that servers
> have to store that much, just that clients must be willing to accept
> that much.  That will keep everything consistent.
>
> Other than this, we believe we're ready to take this document to working
> group last call, which I intend to do once I've heard back from you -
> it's easy to fix this up during last call.
>
>> I've submitted an update to the (formerly snippet, now) preview draft
>> incorporating feedback received from this list and other implementers.
>>
>> Main change is the name - this is now "PREVIEW" instead of "SNIPPET".
>> Also, increased recommended preview size to 200 characters
>> (previously: 100).
>>
>> Will not be able to remotely join IETF 103 meeting as I am traveling,
>> but wanted to make sure this draft was updated before then.
>>
>> michael
>>

---

From: Brandon Long <[email-masked]>
Date: Fri, 9 Nov 2018 11:14:49 -0800

Do we think this is likely to be used for full-sync clients?
Theoretically, those could just generate their own snippets locally from
the rest of the data.

For online clients, the preview would just be part of what they download to
display the message list.

Without QRESYNC, we still have some ugly polling, though obviously PREVEIWS
are larger than UIDs.

OTOH, I feel we already have part of this type of problem today, where
improvements to our parsers or encoding detection might change our response
to any
given message based request, but obviously we won't rev the UIDVALIDITY
folders on every bi-weekly release when its only ever going to affect a
small number of messages.

Brandon

---

From: Timo Sirainen <[email-masked]>
Date: Sat, 10 Nov 2018 04:14:22 +0200

I think from IMAP protocol purity point of view it would be nice if we didn't have fields that could suddenly change without any indication. But from a practical point of view: Who cares, really? The previews are mainly intended for clients that don't have a local cache. Even if we implemented modseq changes for previews, would any client (that has more than a couple of users) ever really use them? I find it doubtful. More likely these additional requirements would make it more difficult for servers to implement this feature, so the clients wouldn't have it at all.

> On 9 Nov 2018, at 21.14, Brandon Long <blong=[email-masked]> wrote:
> 
> Do we think this is likely to be used for full-sync clients?  Theoretically, those could just generate their own snippets locally from the rest of the data.
> 
> For online clients, the preview would just be part of what they download to display the message list.
> 
> Without QRESYNC, we still have some ugly polling, though obviously PREVEIWS are larger than UIDs.
> 
> OTOH, I feel we already have part of this type of problem today, where improvements to our parsers or encoding detection might change our response to any
> given message based request, but obviously we won't rev the UIDVALIDITY folders on every bi-weekly release when its only ever going to affect a small number of messages.
> 
> Brandon
> 
> On Thu, Nov 8, 2018 at 4:52 PM Stuart Brandt <stujenerin=[email-masked]..org <mailto:[email-masked]>> wrote:

---

From: "Bron Gondwana" <[email-masked]>
Date: Sat, 10 Nov 2018 00:15:15 -0500

That's my attitude to previews too, they don't "change" but they might not be byte identical if fetched again later due to changing server algorithms. The old value is never "wrong" though, short of buggy servers. 
 
Bron.

---

From: Stuart Brandt <[email-masked]>
Date: Sun, 11 Nov 2018 18:38:15 -0500

If this is the case, then it might be best to include some level of 
recommendation in the draft that clients *not* make an effort to stay up 
to date with any changes that the server might make. I sure would hate 
to see <tag> FETCH 1:* (FLAGS) become <tag> FETCH 1:* (FLAGS PREVIEW), 
adding up to 255bytes per item to the FETCH response just because 
PREVIEW *might* change.

- Stu

On 11/10/2018 12:15 AM, Bron Gondwana wrote:
> That's my attitude to previews too, they don't "change" but they might
> not be byte identical if fetched again later due to changing server
> algorithms. The old value is never "wrong" though, short of buggy servers.
>
> Bron.
>
>> I think from IMAP protocol purity point of view it would be nice if we
>> didn't have fields that could suddenly change without any indication.
>> But from a practical point of view: Who cares, really? The previews
>> are mainly intended for clients that don't have a local cache. Even if
>> we implemented modseq changes for previews, would any client (that has
>> more than a couple of users) ever really use them? I find it doubtful.
>> More likely these additional requirements would make it more difficult
>> for servers to implement this feature, so the clients wouldn't have it
>> at all.
>>
>>> On 9 Nov 2018, at 21.14, Brandon Long
>>> <blong=[email-masked]
>>> <mailto:blong=[email-masked]>> wrote:
>>>
>>> Theoretically, those could just generate their own snippets locally
>>> from the rest of the data.
>>>
>>> OTOH, I feel we already have part of this type of problem today,
>>> where improvements to our parsers or encoding detection might change
>>> our response to any
>>> given message based request, but obviously we won't rev the
>>> UIDVALIDITY folders on every bi-weekly release when its only ever
>>> going to affect a small number of messages.
>>>
>>> Brandon
>>>
>>> On Thu, Nov 8, 2018 at 4:52 PM Stuart Brandt
>>> <stujenerin=[email-masked]..org
>>> <mailto:[email-masked]>> wrote:
>>>

---

From: "Neil Jenkins" <[email-masked]>
Date: Sun, 11 Nov 2018 21:48:42 -0500

> adding up to 255bytes per item to the FETCH response just because  
 
That sounds reasonable. In the JMAP spec, the "preview" field is marked as immutable (as are all fields except keywords and mailboxes, to match IMAP). I would agree that in practice it might change if the server updates its algorithm for excluding salutations, or fixes a MIME parsing bug, (just as the latter might affect the body structure returned), but this previous value is not considered "wrong" and clients are not informed of the changes, nor expected to resync them. 
 
Neil.