Subject: Re: [Extra] AD review of draft-ietf-extra-imap-status-size-01

From: Stephan Bosch <[email-masked]>
Date: Fri, 4 May 2018 00:31:41 +0200

Op 26/04/2018 om 19:10 schreef Alexey Melnikov:
> Hi,
>
> I've been following discussions on this document and it is ready for
> IETF Last Call, which I just initiated.
>
> One nit: I believe the following reference is now Normative, due to
> added SHOULD level requirement:
>
>   [QUOTA]
>
> Please update the document to reflect this next time the document is
> revised.

Applied in Git:

https://github.com/ietfextra/draft-ietf-extra-imap-status-size/commit/0bbbfb5c25b3cec18b16e4c58f3aaa11208a3c10

Regards,

Stephan.

>
> Best Regards,
> Alexey, as AD.
>
> _______________________________________________
> Extra mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/extra

---

From: "Chris Newman" <[email-masked]>
Date: Fri, 04 May 2018 16:42:00 -0700

I just noticed we implemented this in 2005 (under the "X-SUN-IMAP" 
capability). Looking at the implementation, there's a significant bug in 
the specification. Mailbox sizes can easily exceed 4GB; so the ABNF for 
status-att-list needs to be replaced as follows:

    status-att-list = status-att SP number63 *(SP status-att SP 
number63)
         ;; number63 is only permitted with extension status attributes
         ;; that explicitly allow it; such as SIZE

    number63  = 1*DIGIT
                           ;; Positive unsigned 63-bit integer
                           ;; (0 <= n <= 9,223,372,036,854,775,807).

		- Chris

On 26 Apr 2018, at 10:10, Alexey Melnikov wrote:

> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_extra&d=DwICAg&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=K_BObr5Kfkr3rxt1oBPF9KFiEU3xl9LcD2OOJG3TXfI&m=B73C8TOrpzVRz_O5v_4esOIRwnbdcZoZaulZWKyZwBY&s=Av_oW8rLHO6EMocPg9SJXED-Ta7xK6PRHohZ6tFZmmM&e=

---

From: "Chris Newman" <[email-masked]>
Date: Fri, 04 May 2018 16:46:17 -0700

On 4 May 2018, at 16:42, Chris Newman wrote:

> capability). Looking at the implementation, there's a significant bug 
> in the specification. Mailbox sizes can easily exceed 4GB; so the ABNF 
> for status-att-list needs to be replaced as follows:
>

Oh, just noticed RFC 4466 redefines that, so you instead need a 
normative reference to RFC 4466 and then do:

    status-att-val /= "SIZE" SP number63

		- Chris

> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_extra&d=DwICAg&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=K_BObr5Kfkr3rxt1oBPF9KFiEU3xl9LcD2OOJG3TXfI&m=ErU-4WHaHwqAH1nltCBHaf7zbhatpz2gGlB_BH0n3uI&s=HPU9C98qqq9ZOoEpMPOpUKits0ILuqWjGwmCx6_ZCvo&e=

---

From: Stephan Bosch <[email-masked]>
Date: Thu, 10 May 2018 19:30:35 +0200

Op 05/05/2018 om 01:46 schreef Chris Newman:
>> in the specification. Mailbox sizes can easily exceed 4GB; so the 
>> ABNF for status-att-list needs to be replaced as follows:
>>
>>    status-att-list = status-att SP number63 *(SP status-att SP number63)

A 4Gb mailbox is still quite huge these days, but you're right. The 
QUOTA capability has the same problem: how did you address that? That 
would have been solved by the 64BIT capability, but that one's on hold now

Bron, Alexey, opinions?

Regards,

Stephan

---

From: "Chris Newman" <[email-masked]>
Date: Thu, 10 May 2018 15:40:31 -0700

On 10 May 2018, at 10:30, Stephan Bosch wrote:
>>> capability). Looking at the implementation, there's a significant 
>>> bug in the specification. Mailbox sizes can easily exceed 4GB; so 
>>> the ABNF for status-att-list needs to be replaced as follows:
>>>
> QUOTA capability has the same problem: how did you address that?

Quota Storage partially dodges the problem by using 1K rather than 1 
byte units. However, the fact it doesn't allow 63-bit storage numbers is 
a specification bug (due to proof-by-example: the IMAP data model allows 
a mailbox with 4G messages each of 4G size). Implementations should 
ignore specification bugs like that (ours does). However, the 32-bit 
limit on UIDs, message size, and message sequence numbers is part of the 
fundamental IMAP data model. Changes to those values require an 
incompatible change to the IMAP data model that can break previously 
standards-compliant implementations. The IETF generally doesn't make 
that sort of change without a negotiated extension and implementations 
that ignore fundamental data models in the standards without negotiation 
are usually bad Internet citizens.

> That would have been solved by the 64BIT capability, but that one's on 
> hold now

There's no reason we can't have this spec use a 63-bit number at the 
time we write it. Clients won't get this data unless they ask for it and 
servers won't send it unless they've advertised it. This requires no 
change to the IMAP data model so no separate negotiation is needed to 
use 63-bit numbers here. It's just an extra line of ABNF and with no 
impact on data model and very little impact on implementations. A 
normative dependency on a draft that changes the IMAP data model is a 
change to SIZE that I would oppose. But use of a 32-bit integer is a bug 
in the SIZE draft (again via proof-by-example).

		- Chris

>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_extra&d=DwIDaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=K_BObr5Kfkr3rxt1oBPF9KFiEU3xl9LcD2OOJG3TXfI&m=5HJofT5ocTAB71bioSeN2ztVNEW92Tx14L2kdo_8_PQ&s=VsTTjiVSxN0fQkrv_AoqVm9VLp5Rw5qTjsqJJEODbT4&e=

---

From: Bron Gondwana <[email-masked]>
Date: Fri, 11 May 2018 12:01:03 +1000

My opinion is - just define this response as a 63 bit number.

Crappy thing here with RFC4466 is:

   tagged-ext-simple   = sequence-set / number

And "number" is defined in RFC3501:

number          = 1*DIGIT ; Unsigned 32-bit integer ; (0 <= n <
4,294,967,296)

Which means that the response SHOULD be:

"SIZE" SP "(" number63 ")"  rather than "SIZE" SP number63 to be
tagged-ext.
Bron.

On Fri, 11 May 2018, at 08:40, Chris Newman wrote:
>>>> I just noticed we implemented this in 2005 (under the "X-SUN-IMAP">>>> capability). Looking at the implementation, there's a significant
> byte units. However, the fact it doesn't allow 63-bit storage
> numbers is> a specification bug (due to proof-by-example: the IMAP data
> model allows> a mailbox with 4G messages each of 4G size). Implementations should
> limit on UIDs, message size, and message sequence numbers is
> part of the> fundamental IMAP data model. Changes to those values require an
> that sort of change without a negotiated extension and implementations> that ignore fundamental data models in the standards without
> negotiation> are usually bad Internet citizens.
> 
>> That would have been solved by the 64BIT capability, but that
>> one's on>> hold now
> 
> time we write it. Clients won't get this data unless they ask
> for it and> servers won't send it unless they've advertised it. This requires no
> change to the IMAP data model so no separate negotiation is needed to> use 63-bit numbers here. It's just an extra line of ABNF and with no
> normative dependency on a draft that changes the IMAP data model is a> change to SIZE that I would oppose. But use of a 32-bit integer
> is a bug> in the SIZE draft (again via proof-by-example).
> 
> - Chris
> 
>> Bron, Alexey, opinions?
>> 
>> Regards,
>> 
>> Stephan
>> 
>>> 
>>>         - Chris
>>> 
>>>>    number63  = 1*DIGIT
>>>>                           ;; Positive
>>>> unsigned 63-bit integer
>>>>                           ;; (0 <= n <=
>>>>> One nit: I believe the following reference is now Normative,
>>>>> due to>>>>> added SHOULD level requirement:
>>>>> 
>>>>>  [QUOTA]
>>>>> 
>>>>> Please update the document to reflect this next time the document>>>>> is
>>>>> revised.
>>>>> 
>>>>> Best Regards,
>>>>> Alexey, as AD.
>>>>> 
>>>>> _________________________________________________
>>>>> Extra mailing list
>>>>> [email-masked]
>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_extra&d=DwICAg&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=K_BObr5Kfkr3rxt1oBPF9KFiEU3xl9LcD2OOJG3TXfI&m=B73C8TOrpzVRz_O5v_4esOIRwnbdcZoZaulZWKyZwBY&s=Av_oW8rLHO6EMocPg9SJXED-Ta7xK6PRHohZ6tFZmmM&e=>>>> 
>>>> _________________________________________________
>>>> Extra mailing list
>>>> [email-masked]
>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_extra&d=DwICAg&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=K_BObr5Kfkr3rxt1oBPF9KFiEU3xl9LcD2OOJG3TXfI&m=ErU-4WHaHwqAH1nltCBHaf7zbhatpz2gGlB_BH0n3uI&s=HPU9C98qqq9ZOoEpMPOpUKits0ILuqWjGwmCx6_ZCvo&e=>>> 
>>> _________________________________________________
>>> Extra mailing list
>>> [email-masked]
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_extra&d=DwIDaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=K_BObr5Kfkr3rxt1oBPF9KFiEU3xl9LcD2OOJG3TXfI&m=5HJofT5ocTAB71bioSeN2ztVNEW92Tx14L2kdo_8_PQ&s=VsTTjiVSxN0fQkrv_AoqVm9VLp5Rw5qTjsqJJEODbT4&e=> 
> _________________________________________________
> Extra mailing list
> [email-masked]

--
  Bron Gondwana, CEO, FastMail Pty Ltd
  [email-masked]

---

From: "Chris Newman" <[email-masked]>
Date: Mon, 14 May 2018 16:54:56 -0700

> number          = 1*DIGIT ; Unsigned 32-bit integer ; (0 <= n <

Clients have a parser. That parser will either support 32-bit numbers or 
63-bit numbers. Clients implementing SIZE should just support 63-bit 
numbers; that's less client code than supporting 32-bit numbers outside 
parens and 63-bit numbers inside parens; thus less risk of bugs. The 
extra parentheses make the parser more complex for no good reason. The 
extra parentheses make the extension incompatible with an already 
implemented extension in my server for no good reason (yes, I'm willing 
to make incompatible changes to private extensions, but only if there is 
a technical reason to do so). I object to adding the extra parentheses 
for those two reasons.

		- Chris

>>>>> I just noticed we implemented this in 2005 (under the 
>>>>> "X-SUN-IMAP">>>> capability). Looking at the implementation, 
>>>>> there's a significant
>> that sort of change without a negotiated extension and 
>> implementations> that ignore fundamental data models in the standards 
>> without
>> change to the IMAP data model so no separate negotiation is needed 
>> to> use 63-bit numbers here. It's just an extra line of ABNF and with 
>> no
>> normative dependency on a draft that changes the IMAP data model is 
>> a> change to SIZE that I would oppose. But use of a 32-bit integer
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_extra&d=DwICaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=K_BObr5Kfkr3rxt1oBPF9KFiEU3xl9LcD2OOJG3TXfI&m=iuOZ5F20PkM5wdQEwZxEPtbn6Zkqct206gpiSREDRcM&s=tFJXEAphLbOeNohg_H0fX9hOKWGsxGtr8rozXS687OA&e=
>
> --

> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_extra&d=DwICAg&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=K_BObr5Kfkr3rxt1oBPF9KFiEU3xl9LcD2OOJG3TXfI&m=iuOZ5F20PkM5wdQEwZxEPtbn6Zkqct206gpiSREDRcM&s=tFJXEAphLbOeNohg_H0fX9hOKWGsxGtr8rozXS687OA&e=

---

From: Bron Gondwana <[email-masked]>
Date: Tue, 15 May 2018 11:14:18 +1000

On Tue, May 15, 2018, at 09:54, Chris Newman wrote:
> Clients have a parser. That parser will either support 32-bit
> numbers or> 63-bit numbers. Clients implementing SIZE should just support 63-bit
> numbers; that's less client code than supporting 32-bit numbers
> outside> parens and 63-bit numbers inside parens; thus less risk of bugs. The
> extra parentheses make the parser more complex for no good reason. The> extra parentheses make the extension incompatible with an already
> implemented extension in my server for no good reason (yes, I'm
> willing> to make incompatible changes to private extensions, but only if
> there is> a technical reason to do so). I object to adding the extra parentheses> for those two reasons.

I'm totally fine with all those reasons.  Do we need to do a 4466bis for
this, or do we let that get folded with the 64bit extension work in
general and IMAP4rev2?
I'm pretty keen to say that "IMAP4rev2 will fix it" about irrelevant
things like this, but I think it's good to have the discussion and have
a clear answer for why we're going with a number without parens -
particularly since OBJECTID is doing parans everywhere.
Bron.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Tue, 15 May 2018 11:46:19 +0200

On Tuesday 15 May 2018 03:14:18 CEST, Bron Gondwana wrote:
> I'm totally fine with all those reasons.  Do we need to do a 
> 4466bis for this, or do we let that get folded with the 64bit 
> extension work in general and IMAP4rev2?
>
> I'm pretty keen to say that "IMAP4rev2 will fix it" about 
> irrelevant things like this, but I think it's good to have the 
> discussion and have a clear answer for why we're going with a 
> number without parens - particularly since OBJECTID is doing 
> parans everywhere.

Try this:

If this extension uses 63-bit unsigned numbers, then that complicates the 
grammar, which needs an additional sentence to say that the number 
production comes from RFCxxx instead of 3501,  and the RFC, which needs to 
explain that the size is potentially big. But it simplifies the clients, 
which only need to parse and handle one case instead of two, and which only 
need to test one thing instead of two, and breaks nothing.

This is similar to the history of "" for UTF8=ACCEPT. There was separate 
syntax for quoting, it was dropped because it seemed to offer no 
advantages, and instead the "" was extended to allow UTF8. A success. As 
far as I know, none of the implementations have reported problems with 
that.

IMHO (note the H, that's h for humble) objectid doesn't have the same 
problem. The parens aren't necessary. They have no function, they're just 
there because some other parens add clarity in some other cases. But my 
personal opinion is that if unnecessary syntax is mandatory, then it adds 
no unit tests, and the required number of unit tests are an effective proxy 
for simplicity and reliability.

Arnt

---

From: Alexey Melnikov <[email-masked]>
Date: Tue, 15 May 2018 11:04:55 +0100

On 15/05/2018 02:14, Bron Gondwana wrote:
> I'm totally fine with all those reasons.

I am fine with increasing the numeric range for "tagged-ext-simple".

>   Do we need to do a 4466bis for this,

Maybe :-).

> or do we let that get folded with the 64bit extension work in general 
> and IMAP4rev2?

I am happy to do this in IMAP4rev2. If people also want to have 
rfc4466bis, this can be done as well.

> things like this, but I think it's good to have the discussion and 
> have a clear answer for why we're going with a number without parens -

---

From: Stephan Bosch <[email-masked]>
Date: Thu, 17 May 2018 21:43:30 +0200

Hi,

So, in summary, what should be done here for the draft?

Regards,

Stephan.

Op 15/05/2018 om 12:04 schreef Alexey Melnikov:
>> have a clear answer for why we're going with a number without parens 
>> - particularly since OBJECTID is doing parans everywhere.
>
>
>

---

From: Bron Gondwana <[email-masked]>
Date: Sat, 26 May 2018 22:17:17 +1000

> So, in summary, what should be done here for the draft?

Use number64 for the SIZE.  You'll probably have to define it as
well, since 64bit hasn't been standardised and we don't want to have
to wait on it:
  number64    = 1*DIGIT ; Unsigned 63-bit integer ; (0 <= n <=
  9,223,372,036,854,775,807)

Bron.

---

From: Alexey Melnikov <[email-masked]>
Date: Tue, 29 May 2018 21:58:58 +0300

On 26/05/2018 15:17, Bron Gondwana wrote:
> Use number64 for the SIZE.  You'll probably have to define it as well,
> since 64bit hasn't been standardised and we don't want to have to wait
> on it:
> 
>   number64    = 1*DIGIT
>                 ; Unsigned 63-bit integer
>                 ; (0 <= n <= 9,223,372,036,854,775,807)

I will also add it to IMAP4rev2.

---

From: Stephan Bosch <[email-masked]>
Date: Tue, 29 May 2018 21:19:52 +0200

>                  ; (0 <= n <= 9,223,372,036,854,775,807)

Done:

https://github.com/ietfextra/draft-ietf-extra-imap-status-size/commit/cab94c5a9216da5ac168b3fc6bbaff31ba6f940e

Regards,

Stephan.