Subject: [Extra] Robert Wilton's Discuss on draft-ietf-extra-imap-fetch-preview-09: (with DISCUSS and COMMENT)

From: Robert Wilton via Datatracker <[email-masked]>
Date: Wed, 23 Sep 2020 02:26:14 -0700

Robert Wilton has entered the following ballot position for
draft-ietf-extra-imap-fetch-preview-09: Discuss

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.

The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-extra-imap-fetch-preview/

----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------

Hi,

Thanks for this document.  I found it easy to read and understand, but have one
potential issue that probably warrants a bit of discussion.  I don't have any
great knowledge of IMAP, so this may already be handled elsewhere, but I had a
concern about returning zero-length strings under error conditions.

   It is possible that the server has determined that no meaningful
   preview text can be generated for a particular message, and that
   decision won't change later.  Examples of this involve encrypted
   messages, content types the server does not support previews of, and
   other situations where the server is not able to extract information
   for a preview.  In such cases, the server MUST return a zero-length
   string.  Clients SHOULD NOT send another FETCH for a preview for such
   messages.  (As discussed previously, preview data is not immutable so
   there is chance that at some point in the future the server would be
   able to generate meaningful text.  However, this scenario is expected
   to be rare so a client should not continually send out requests to
   try to capture this infrequent occurrence.)

   ... A server MUST NOT return NIL
   to a FETCH PREVIEW request made without the LAZY modifier.

When the LAZY modifier is not being used, then what would be returned if the
server was transiently unable to return the preview for any reason?  Does it
still have to return a zero-length string in this error case?  Is there some
way that the server can indicate that it cannot satisfy the request now but
without indicating that no preview is available?

----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

One minor comment:

   This standardized display can reduce user confusion when using
   multiple clients, as abbreviated message representations in clients
   will show identical message contents.

I didn't really find this reasoning to be compelling, and perhaps it could be
removed.  In my experience the amount of preview displayed depends on client
behaviour or settings (e.g., how much space to allow for previews).

Regards,
Rob

---

From: Barry Leiba <[email-masked]>
Date: Wed, 23 Sep 2020 07:58:44 -0400

> without indicating that no preview is available?

The idea here is that there are three conditions:

1. The server determines that there's no preview available for this
message, and it returns an empty string: "" or the equivalent literal
{0}.

2. The server returns a non-empty preview.

3. The client uses LAZY, indicating that it's OK to defer the
generation of a preview and the server decides that deferral is the
best approach, so it returns NIL.  At some later time, the client
might re-send the FETCH without LAZY.  If it does, the server will
respond with (1) or (2).

So, NIL does not mean that no preview is available for the message; it
means that the server is invoking the option given it by the LAZY
modifier, and is deferring the response.

> behaviour or settings (e.g., how much space to allow for previews).

Perhaps some slight re-phrasing of this text will address your comment
℗rhaps changing the word "identical"), but the point here isn't that
all clients will always present the preview in exactly the same way:
indeed, one might present less text than another due to space
limitations in the UI.  The point here is that if a message says that
we have to discuss document status in the EXTRA working group and we
should do it over lunch tomorrow and should we go for Chinese or
Indian food and at what time... we won't wind up with one client's
preview saying that the message is about the EXTRA working group's
document status while another's says it's about lunch plans tomorrow
and still another's that it's about China and India.  I think this is
useful text making a useful point, and that we shouldn't remove it.

Barry

---

From: Barry Leiba <[email-masked]>
Date: Wed, 23 Sep 2020 10:49:34 -0400

> useful text making a useful point, and that we shouldn't remove it.

On thinking about re-phrasing, how does this sound?:

NEW
While different clients might generate quite different preview text,
having common preview text generated by the server can give a more
consistent user experience to those who use multiple clients.
END

Barry

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Wed, 23 Sep 2020 16:50:35 +0000

Hi Barry,

Please see inline ...

> -----Original Message-----
> From: iesg <[email-masked]> On Behalf Of Barry Leiba
> Sent: 23 September 2020 12:59
> To: Rob Wilton (rwilton) <[email-masked]>
> Cc: [email-masked]; Bron Gondwana <[email-masked]>; The IESG
> <[email-masked]>; [email-masked]; draft-ietf-extra-imap-fetch-
> [email-masked]
> Subject: Re: Robert Wilton's Discuss on draft-ietf-extra-imap-fetch-
> preview-09: (with DISCUSS and COMMENT)
> 
[RW] 

Sure.

My concern is, for the non LAZY case, what does the server return if a preview should be available but the server cannot construct it at that point in time, for any reason?  Is it possible for the server to return an error? E.g., should it just fail the entire request?  If the only option is to return an empty string that they seems to imply not only that it cannot return a preview now, but it may well never be able to return a preview (which may not true).  I.e., the client has no way to distinguish between a transient failure at generating the preview and a semi-permanent indication that the preview cannot be generated for that particular message.

I have memories of using IMAP clients where they effectively get stuck/broken and seem to have to slowly resync all the data from the server.  Hence my fear is that the server will fail to produce the preview at one point in time, and then the client is stuck without any previews for a subset of messages because a zero length preview string implies that there is no preview available and it SHOULD NOT ask again.

Of course, I don't know IMAP, and hence I might just be missing something basic about how IMAP works ...

> useful text making a useful point, and that we shouldn't remove it.
[RW] 

Okay.  Will look/respond to your subsequent email.

Thanks,
Rob

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Wed, 23 Sep 2020 16:51:17 +0000

Hi Barry,

Yes, your proposed text is fine.

Regards,
Rob

> From: Barry Leiba <[email-masked]>
> Sent: 23 September 2020 15:50
> Cc: The IESG <[email-masked]>; [email-masked]; Bron Gondwana
> <[email-masked]>; [email-masked]; draft-ietf-extra-imap-
> [email-masked]

---

From: Barry Leiba <[email-masked]>
Date: Wed, 23 Sep 2020 13:04:53 -0400

Well, there's a general thing with IMAP: What does a server do if you
FETCH ENVELOPE or FETCH BODYSTRUCTURE and the server has a problem
generating the response (say, some transient error in parsing the
message because of a back-end database glitch?  There are two choices:
fill in NIL for strings and 0 for numbers and just fake up a response,
which can really screw up clients... or reply NO and hope that the
client will handle that well enough, and will maybe try again later --
which can really screw up clients.  In retrospect, maybe it would have
been nice to have the 3xx/4xx/5xx sort of mechanism, allowing for
temporary errors with semantics of "try again later, for some
unspecified value of 'later'."  But it's not that way, and there's
really nothing that can be done about it now.

Barry

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 23 Sep 2020 18:17:04 +0100

> really nothing that can be done about it now.
Indeed. And in general NO response (that fails the whole FETCH command) 
when the request covers multiple messages is worse than a faked response 
for a particular message.
> _______________________________________________
> Extra mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/extra

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Wed, 23 Sep 2020 17:24:42 +0000

Thanks.  I think that was the background explanation that I was missing.

But given this is a new extension, I presume that it is possible that the error handling behaviour for this extension could be defined a bit more tightly.

At the moment, my interpretation of the error handling is:  The server has one shot at generating this, and if it hits a transient error then the client ends up with no preview, with the justification that errors should be rare.

An alternative might be to weaken the following text, so that clients should not immediately and repeatedly send another FETCH, but they may query again in future if appropriate.  Perhaps you see that the SHOULD NOT already allows this?  But I'm not sure that it is particularly clear.

Clients SHOULD NOT send another FETCH for a preview for such

Regards,
Rob

> Sent: 23 September 2020 18:05
> > > > Thanks for this document.  I found it easy to read and understand,
> but
> > > have one
> > > > great knowledge of IMAP, so this may already be handled elsewhere,
> but I
> > > had a
> > > > When the LAZY modifier is not being used, then what would be
> returned if
> > > the
> > > > still have to return a zero-length string in this error case?  Is
> there
> > > some
> > > > way that the server can indicate that it cannot satisfy the request
> now
> > > but
> > My concern is, for the non LAZY case, what does the server return if a
> preview should be available but the server cannot construct it at that
> point in time, for any reason?  Is it possible for the server to return an
> error? E.g., should it just fail the entire request?  If the only option
> is to return an empty string that they seems to imply not only that it
> cannot return a preview now, but it may well never be able to return a
> preview (which may not true).  I.e., the client has no way to distinguish
> between a transient failure at generating the preview and a semi-permanent
> indication that the preview cannot be generated for that particular
> message.
> >
> > I have memories of using IMAP clients where they effectively get
> stuck/broken and seem to have to slowly resync all the data from the
> server.  Hence my fear is that the server will fail to produce the preview
> at one point in time, and then the client is stuck without any previews
> for a subset of messages because a zero length preview string implies that
> there is no preview available and it SHOULD NOT ask again.
> >
> > > > removed.  In my experience the amount of preview displayed depends
> on
> > > client

---

From: Barry Leiba <[email-masked]>
Date: Wed, 23 Sep 2020 14:17:54 -0400

Sorry, Rob, but I really think it would be bad for this to be different to
the rest of IMAP.  In practice, we don’t seem to have issues with transient
errors in IMAP back-end stores, so it really has not been a long-term
issue.  It would be much worse to define an extension (and a simple,
straightforward extension, at that) that attempts to significantly overhaul
how IMAP returns errors.

Please, let’s consider this discussion to have been had, and not try to go
there.

Barry

On Wed, Sep 23, 2020 at 1:25 PM Rob Wilton (rwilton) <[email-masked]>
wrote:

> But given this is a new extension, I presume that it is possible that the
> error handling behaviour for this extension could be defined a bit more
> tightly.
>
>
>
> At the moment, my interpretation of the error handling is:  The server has
> one shot at generating this, and if it hits a transient error then the
> client ends up with no preview, with the justification that errors should
> be rare.
>
>
>
> An alternative might be to weaken the following text, so that clients
> should not immediately and repeatedly send another FETCH, but they may
> query again in future if appropriate.  Perhaps you see that the SHOULD NOT
> already allows this?  But I'm not sure that it is particularly clear.
>
>
>
> > On Wed, Sep 23, 2020 at 12:50 PM Rob Wilton (rwilton) <[email-masked]
> >
>
> > wrote:
>
> > >
>
> > > Hi Barry,
>
> > >
>
> > > Please see inline ...
>
> > >
>
> > > > > potential issue that probably warrants a bit of discussion.  I
> don't
>
> > > > have any
>
> > > > >    messages, content types the server does not support previews of,
>
> > and
>
> > > > >    other situations where the server is not able to extract
>
> > information
>
> > > > >    for a preview.  In such cases, the server MUST return a zero-
>
> > length
>
> > > > >    string.  Clients SHOULD NOT send another FETCH for a preview for
>
> > such
>
> > > > >    messages.  (As discussed previously, preview data is not
>
> > immutable so
>
> > > > >    there is chance that at some point in the future the server
> would
>
> > be
>
> > > > >    able to generate meaningful text.  However, this scenario is
>
> > expected
>
> > > > >    to be rare so a client should not continually send out requests
>
> > to
>
> > > > > server was transiently unable to return the preview for any reason?
>
> > > > Does it
>
> > > Of course, I don't know IMAP, and hence I might just be missing
>
> > something basic about how IMAP works ...
>
> > >
>
> > >
>
> > > >
>
> > > > > One minor comment:
>
> > > > >
>
> > > > >    multiple clients, as abbreviated message representations in
>
> > clients
>
> > > > > I didn't really find this reasoning to be compelling, and perhaps
> it
>
> > > > could be
>

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Wed, 23 Sep 2020 20:18:47 +0000

Hi Barry,

Sorry, but I still find the existing paragraph is unclear and under specified.

It states what to do for the case that the server both (i) cannot generate the preview AND (ii) this is not a transient error.
But because of the “and that decision won't change later”) text, which seems to be somewhat contradicted by the final example, then my reading of this paragraph is that it does not apply for transient errors, and hence the server could either return “NO” or perhaps it could return an empty preview.

Is the document intended to be ambiguous in this regard?  I.e., the server can choose what to do in the case that it hits a transient error?

Note that “4.2.  Client Implementation Advice” indicates that such a transient error could be reasonably expected to occur on the server and explicitly recommends that clients limit the size of the requests to avoid them …

Regards,
Rob

From: Barry Leiba <[email-masked]>
Sent: 23 September 2020 19:18
To: Rob Wilton (rwilton) <[email-masked]>
Cc: Bron Gondwana <[email-masked]>; The IESG <[email-masked]>; [email-masked]; [email-masked]; [email-masked]
Subject: Re: Robert Wilton's Discuss on draft-ietf-extra-imap-fetch-preview-09: (with DISCUSS and COMMENT)

Sorry, Rob, but I really think it would be bad for this to be different to the rest of IMAP.  In practice, we don’t seem to have issues with transient errors in IMAP back-end stores, so it really has not been a long-term issue.  It would be much worse to define an extension (and a simple, straightforward extension, at that) that attempts to significantly overhaul how IMAP returns errors.

Please, let’s consider this discussion to have been had, and not try to go there.

Barry

On Wed, Sep 23, 2020 at 1:25 PM Rob Wilton (rwilton) <[email-masked]<mailto:[email-masked]>> wrote:
Thanks.  I think that was the background explanation that I was missing.

Regards,

Rob

> From: Barry Leiba <[email-masked]<mailto:[email-masked]>>

> To: Rob Wilton (rwilton) <[email-masked]<mailto:[email-masked]>>

> Cc: [email-masked]<mailto:[email-masked]>; Bron Gondwana <[email-masked]<mailto:[email-masked]>>; The IESG

> <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>; draft-ietf-extra-imap-fetch-

> [email-masked]<mailto:[email-masked]>

> On Wed, Sep 23, 2020 at 12:50 PM Rob Wilton (rwilton) <[email-masked]<mailto:[email-masked]>>

> > > From: iesg <[email-masked]<mailto:[email-masked]>> On Behalf Of Barry Leiba

> > > To: Rob Wilton (rwilton) <[email-masked]<mailto:[email-masked]>>

> > > Cc: [email-masked]<mailto:[email-masked]>; Bron Gondwana <[email-masked]<mailto:[email-masked]>>; The IESG

> > > <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>; draft-ietf-extra-imap-fetch-

> > > [email-masked]<mailto:[email-masked]>

> > > > potential issue that probably warrants a bit of discussion.  I don't

> > > >    there is chance that at some point in the future the server would

> > > > I didn't really find this reasoning to be compelling, and perhaps it

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Wed, 23 Sep 2020 22:47:18 +0200

On Wednesday 23 September 2020 22:18:47 CEST, Rob Wilton (rwilton) wrote:
> It states what to do for the case that the server both (i) 
> cannot generate the preview AND (ii) this is not a transient 
> error.

IMAP doesn't report transient errors (which is perhaps wrong, but that's 
how it is). The server has no way to report transient errors for the other 
dozen or more FETCH items. Why makes this one different?

IMO, the document shouldn't mention transient errors at all, since they're 
alien to IMAP and orthogonal to this extension.

Arnt

---

From: Michael Slusarz <[email-masked]>
Date: Thu, 24 Sep 2020 00:30:14 -0600 (MDT)

The case study we are analysing is expected to be rare.

Real world: a server handling large amounts of incoming mail is going to deal with preview generation at delivery, rather than display. Any delays at delivery are not normally apparent to the end user. For a server supporting this extension, preview data will exist in almost every access, so generation issues will be irrelevant purposes of UX.

If a client is worried about the tiny fraction of requests where a preview can not be generated by the server, that client can have code to fallback to attempting to generating the preview itself (I.e. the current behavior for many clients). The situation of a single email where a preview cannot be generated is going to be interpreted by the end user as an exception... And the lack of preview has no bearing on whether an end user can view the full contents of the message.

michael

On 09/23/2020 2:47 PM Arnt Gulbrandsen <[email-masked]> wrote:

error.

Arnt

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Thu, 24 Sep 2020 09:09:18 +0000

In case it helps reach a quicker resolution, one solution could be to delete the clause “and that decision won’t change later”:

I.e. old:

   It is possible that the server has determined that no meaningful
   preview text can be generated for a particular message, and that
   decision won't change later. ...

New:
   It is possible that the server has determined that no meaningful
   preview text can be generated for a particular message. ...

Full paragraph (new):
   It is possible that the server has determined that no meaningful
   preview text can be generated for a particular message.  Examples
   of this involve encrypted messages, content types the server does
   not support previews of, and other situations where the server is
   not able to extract information for a preview.  In such cases, the
   server MUST return a zero-length string.  Clients SHOULD NOT send
   another FETCH for a preview for such messages.  (As discussed
   previously, preview data is not immutable so there is chance that at
   some point in the future the server would be able to generate
   meaningful text.  However, this scenario is expected to be rare so
   a client should not continually send out requests to try to capture
   this infrequent occurrence.)

Regards,
Rob

From: Michael Slusarz <michael.slusarz=[email-masked]>
Sent: 24 September 2020 07:30
To: Arnt Gulbrandsen <[email-masked]>; Rob Wilton (rwilton) <[email-masked]>
Cc: Barry Leiba <[email-masked]>; [email-masked]; Bron Gondwana <[email-masked]>; The IESG <[email-masked]>; [email-masked]; [email-masked]
Subject: Re: [EXT] Re: [Extra] Robert Wilton's Discuss on draft-ietf-extra-imap-fetch-preview-09: (with DISCUSS and COMMENT)

michael
On 09/23/2020 2:47 PM Arnt Gulbrandsen <[email-masked]<mailto:[email-masked]>> wrote:

Arnt

---

From: Barry Leiba <[email-masked]>
Date: Thu, 24 Sep 2020 09:26:40 -0400

I believe Michael has already said that he's deleting that text.

Barry