Subject: [Gen-art] Gen-ART Last Call review of draft-ietf-tsvwg-rfc4960-errata-06

From: Paul Kyzivat <[email-masked]>
Date: Sun, 3 Jun 2018 14:59:54 -0400

[[INCOMPLETE, NOT READY TO SEND. PLEASE IGNORE]]

I am the assigned Gen-ART reviewer for this draft. The General Area 
Review Team (Gen-ART) reviews all IETF documents being processed by the 
IESG for the IETF Chair. Please treat these comments just like any other 
last call comments. For more information, please see the FAQ at 
<​http://wiki.tools.ietf.org/area/gen/trac/wiki/GenArtfaq>.

Document: draft-ietf-tsvwg-rfc4960-errata-06
Reviewer: Paul Kyzivat
Review Date: 2018-06-03
IETF LC End Date: 2018-06-04
IESG Telechat date: ?

Summary:

This draft is on the right track but has open issues, described in the 
review.

Issues:

Major: 1
Minor: 2
Nits:  1

1) MAJOR:

The format of this document disturbs me. According to the abstract:

    ... This
    document provides deltas to RFC4960 and is organized in a time
    ordered way.  The issues are listed in the order they were brought
    up.  Because some text is changed several times the last delta in the
    text is the one which should be applied.

This format makes the document hard to deal with. A developer who wants 
to implement sctp with some or all of the errata fixes will want to work 
from a variant of 4960 that incorporates all of those fixes - a bis. But 
it isn't clear how this document helps with that. I don't think you can 
start with 4960 and simply apply all the deltas sequentially, because 
overlapping changes won't work right.

A developer won't be interested in the order in which errata were 
reported. An actual bis document would be more useful to a developer 
than this format. Is that not being done because doing so would be more 
difficult? Or because it isn't yet certain that these are the correct fixes?

I think you should give some serious consideration of the most suitable 
form for this document, in the context of how it is intended to be used.

2) MINOR (maybe MAJOR):

Discovering where one change is impacted by another change is hard.

I dug into the details of the document to understand how many places 
there are actually overlaps between the changes in multiple sections. 
(It took a lot of work to do this.) I found five of these:

- 3.1 / 3.23
- 3.3 / 3.43
- 3.5 / 3.10
- 3.6 / 3.23
- 3.24 / 3.32

(I don't guarantee that this list is exhaustive.)

Of these, I think only one (3.1/3.23) explicitly indicates the conflict, 
and it only indicates it within 3.23.

Most of the changes don't have any conflicts. And some of the conflicts 
could be removed by being more precise in indicating the change being 
made. In cases where this isn't possible, the presence of the conflict 
should be indicated in each section that has a conflict, with cross 
references. IOW, shift the burden of detecting conflicts from the reader 
to the document.

3) MINOR:

Errata Tracking: Apparently each subsection of section 3 covers one 
erratum. But the errata numbers are not mentioned. Each section ought to 
reference the errata number it responds to.

4) NIT:

In section 3.35 (DSCP Changes) the change to section 10.1 isn't properly 
indicated. It shows 'Old text' twice rather than 'Old text' and 'New text'.

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 4 Jun 2018 09:15:36 +0000

Hi,

I have also looked at this document, and there are things that I have
think are unclear:

Q1: It is Informational, and it does not update RFC 4960. Instead, it just
seems to list the erratas (but without even referencing them, as noted by
Paul). I think that it should be made very clear that this document is
only for guidance, and that implementers shall use the actual erratas for
the actual updates.

Q2: Unless I’ve missed it, there is no indication whether the draft only
includes the Verified erratas, or also others - in which case the modified
text in one or more erratas may still be changed (erratas may even be
rejected).

Q3: While the draft name contains “-errata-“, it is unclear whether the
draft only covers issues for which erratas has been filed, or whether
other issues (e.g., issues that have been discussed on the list) are also
included.

Q4: When looking at the changes, at least in one case I can’t find an
associated errata. For example, section 3.34 is associated with Section
10.1. I only find one errata (#5003) associated with Section 10.1, but the
changes in that errata does not match what is in the draft. A reference to
the actual errata would help.

Q5: The text says that the draft includes issues found since publication.
Now, there may be more issues after this draft has been published, so it
should say something like “at the time of publishing this document”.

Regards,

Christer

On 03/06/18 21:59, "Gen-art on behalf of Paul Kyzivat"
<[email-masked] on behalf of [email-masked]> wrote:

>_______________________________________________
>Gen-art mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/gen-art

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 4 Jun 2018 09:17:59 +0000

Re-sent due to wrong e-mail address.

---

From: Gorry Fairhurst <[email-masked]>
Date: Mon, 4 Jun 2018 10:39:12 +0100

Hi Christer,

As document shepherd for this SCTP process, I'll have a first go at 
responding. I think that for RFC4960 the Errata as filed still apply.

See below. The document authors can of course also propose answers to 
these questions....

Gorry

>> only for guidance, and that implementers shall use the actual erratas for
The document write-up describes the process that has been used with SCTP 
updates, and this document follows this plan - this is information to 
help the WG prepare a bis document, that we plan to populate with these 
edits to the spec and adopt as a WG item in Montreal.

I'm OK with adding text to say this - and see later.
>> other issues (e.g., issues that have been discussed on the list) are also
There are other issues discussed on the list, and the text proposed here 
is what the WG has discussed and finally agreed upon to appear in the 
.bis document. It would be OK to add something about that.
>> the actual errata would help.
I think it would be good to check that Errata refs are included where 
applicable, if we added the text you suggest in Q3.
>> should say something like “at the time of publishing this document”.
I agree, and I really should have spotted that- Sorry and thanks for 
noting this. I suggest as this ID now approaches publication to be 
clearer about the intended plans, by adding a para of text (to be 
discussed and finalised with my AD and Co-Chairs) to say something like:

The information in this document does not update RFC4640 or the Errata 
to that specification.  The document is instead provided as input to 
preparation of a new document that is expected to be a standards-track 
replacement for RFC4960. If approved, the replacement document will 
incorporate the updates described here and any other changes needed to 
allow this to progress this specification along the standards track.

- and to add a corresponding note in the abstract.

Gorry

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 4 Jun 2018 10:13:45 +0000

Hi Gorry,

...

>allow this to progress this specification along the standards track.

I am ok with the two first sentences.

But, I don’t think you can make the last sentence. This document cannot
normatively define text for the replacement document, or assume that
everything will be incorporated: the WG will have to agree on what goes
into the replacement document once it has been added to the charter etc,
using normal IETF procedures.

Regards,

Christer

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 4 Jun 2018 11:06:24 +0000

Not a comment on the document, but a question/suggestion:

If you want to have a place holder for changes to be done in the bis
(which seems to be the main purpose of the errata document), why not
create a GitHub repo for the bis, and then document everything as GitHub
issues? Then, when you start working on the bis, you can map each issue to
a pull request etc.

Regards,

Christer

On 04/06/18 13:13, "Gen-art on behalf of Christer Holmberg"
<[email-masked] on behalf of [email-masked]>

---

From: Gorry Fairhurst <[email-masked]>
Date: Mon, 4 Jun 2018 12:51:29 +0100

Hi

> using normal IETF procedures.
And it wouldn't say those exact words of course!

If I carefully composed actual text, it would be IETF-compliant;-)

Gorry

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 4 Jun 2018 12:18:23 +0000

Hi,

>If I carefully composed actual text, it would be IETF-compliant;-)

Great! :)

All I want is to avoid a situation where someone laters suggests some
modified text for the bis, and the reply is that the text in the errata
draft has already been agreed upon by the WG to be included in the bis.

Regards,

Christer

>>>>>>      up.  Because some text is changed several times the last delta
>>>>>>in
>>>>>> the
>>>>>> difficult? Or because it isn't yet certain that these are the
>>>>>>correct
>>>>>> fixes?
>>>>>>
>>>>>> erratum. But the errata numbers are not mentioned. Each section
>>>>>>ought
>>>>>> to

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Mon, 4 Jun 2018 09:55:23 -0500

Just chiming in as AD ...

> >If I carefully composed actual text, it would be IETF-compliant;-)

Your AD agrees with adding words like the ones you're discussing here, once
you figure out exactly what words should be added.

Spencer

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 4 Jun 2018 12:39:52 -0400

> edits to the spec and adopt as a WG item in Montreal.

In that case, do you actually intend for this document to progress to an 
informational RFC, or just leave it as a draft and let it expire once 
the bis has captured all it contains?

(My understanding is that there is now a preference not to waste 
resources publishing transient documents like this.)

> .bis document. It would be OK to add something about that.

So is this really just a capture of agreements the wg has made about 
planned changes to 4960, where many of them have been motivated by errata?

	Thanks,
	Paul

---

From: Michael Tuexen <[email-masked]>
Date: Wed, 6 Jun 2018 18:14:22 -0400

Hi Christer,

thank you very much for the review. Sorry about the late response, I'm
currently traveling.

See my replies in-line.

Best regards
Michael

> On 4. Jun 2018, at 05:17, Christer Holmberg <[email-masked]> wrote:
> 
> 
Please note that the documents covers the following processed erratas and
also mentions this:

* https://tools.ietf.org/html/draft-ietf-tsvwg-rfc4960-errata-06#section-3.1
  covers Errata ID 1440.
* https://tools.ietf.org/html/draft-ietf-tsvwg-rfc4960-errata-06#section-3.2
  covers Errata ID 1574.
* https://tools.ietf.org/html/draft-ietf-tsvwg-rfc4960-errata-06#section-3.3
  covers Errata ID 2592.
* https://tools.ietf.org/html/draft-ietf-tsvwg-rfc4960-errata-06#section-3.4
  covers Errata ID 3291 and Errata ID 3804.
* https://tools.ietf.org/html/draft-ietf-tsvwg-rfc4960-errata-06#section-3.5
  covers Errata ID 3423.
* https://tools.ietf.org/html/draft-ietf-tsvwg-rfc4960-errata-06#section-3.6
  covers Errata ID 3788.
* https://tools.ietf.org/html/draft-ietf-tsvwg-rfc4960-errata-06#section-3.7
  covers Errata ID 4071.
* https://tools.ietf.org/html/draft-ietf-tsvwg-rfc4960-errata-06#section-3.8
  covers Errata ID 4400.
* https://tools.ietf.org/html/draft-ietf-tsvwg-rfc4960-errata-06#section-3.9
  covers Errata ID 5003.

Other sections deal with problems not necessarily reported as erratas using
the IETF errata tool.
>> text in one or more erratas may still be changed (erratas may even be
See above.
>> other issues (e.g., issues that have been discussed on the list) are also
See above.
>> the actual errata would help.
See above.
>> should say something like “at the time of publishing this document”.
We can add such wording.
>>> form for this document, in the context of how it is intended to be used.
As indicated by Gorry, the plan is to build a 4960bis once this document is
done. Then a developer has only a single RFC to implement. If he wants to
know which changes have been done and why it an consult this informative
document.
>>> (It took a lot of work to do this.) I found five of these:
In the past it was not that critical. Especially, once RFC 4960bis is out.
>>> reference the errata number it responds to.
Please note that every subsection of section 3 covers an issue, but not
one errata listed in https://www.rfc-editor.org/errata_search.php?rfc=4960
>>> indicated. It shows 'Old text' twice rather than 'Old text' and 'New
Fixed in the repository:
https://github.com/sctplab/rfc4960bis/commit/650416a74049578aeea6c0e6098815f746ecc778

---

From: Michael Tuexen <[email-masked]>
Date: Wed, 6 Jun 2018 18:18:17 -0400

> On 4. Jun 2018, at 07:06, Christer Holmberg <[email-masked]> wrote:
> 
We did use a github report using issues which working on this document.

Replacing this document with an github issue tracker doesn't seem
attraktive to me. Github can go away at any time or gets replaced
by other tools and than the information would not be accessible
anymore. Please note that we document the changes and the reasoning
not for us, but for developers which are interested in it in the
future.

The only alternative would be to document everything in the IETF
errata system...

However, we have used this way already in the past and it worked fine.

Best regards
Michael

---

From: Christer Holmberg <[email-masked]>
Date: Thu, 7 Jun 2018 06:40:17 +0000

Hi Michael,

Please see inline.

>* 
>https://tools.ietf.org/html/draft-ietf-tsvwg-rfc4960-errata-06#section-3.1
>  covers Errata ID 1440.
>* 
>https://tools.ietf.org/html/draft-ietf-tsvwg-rfc4960-errata-06#section-3.2
>  covers Errata ID 1574.
>* 
>https://tools.ietf.org/html/draft-ietf-tsvwg-rfc4960-errata-06#section-3.3
>  covers Errata ID 2592.
>* 
>https://tools.ietf.org/html/draft-ietf-tsvwg-rfc4960-errata-06#section-3.4
>* 
>https://tools.ietf.org/html/draft-ietf-tsvwg-rfc4960-errata-06#section-3.5
>  covers Errata ID 3423.
>* 
>https://tools.ietf.org/html/draft-ietf-tsvwg-rfc4960-errata-06#section-3.6
>  covers Errata ID 3788.
>* 
>https://tools.ietf.org/html/draft-ietf-tsvwg-rfc4960-errata-06#section-3.7
>  covers Errata ID 4071.
>* 
>https://tools.ietf.org/html/draft-ietf-tsvwg-rfc4960-errata-06#section-3.8
>  covers Errata ID 4400.
>* 
>https://tools.ietf.org/html/draft-ietf-tsvwg-rfc4960-errata-06#section-3.9
>  covers Errata ID 5003.

I did not find a reference to Errata ID 5003 for the changes to section
10.1. But, you are right that the other erratas are mentioned. I should
have double checked that.

Regards,

Christer

---

From: Christer Holmberg <[email-masked]>
Date: Thu, 7 Jun 2018 06:43:24 +0000

Hi,

>not for us, but for developers which are interested in it in the

Sure, but my understanding is that the future, i.e., the bis document, is
coming soon, and I guess the bis document will anyway describe the changes
(and the reasons) compared to RFC 4960.

Anyway, since I haven’t been involved in the work, I don’t want to argue
about the way the WG is working. It was just a question/suggestion :)

Regards,

Christer

>>>>>>> from a variant of 4960 that incorporates all of those fixes - a
>>>>>>>bis.
>>>>>> But
>>>>>>> indicated. It shows 'Old text' twice rather than 'Old text' and
>>>>>>>'New
>>>>>>> text'.
>>>>>>>

---

From: Michael Tuexen <[email-masked]>
Date: Thu, 7 Jun 2018 06:19:54 -0400

> On 7. Jun 2018, at 02:43, Christer Holmberg <[email-masked]> wrote:
> 
> 
> Hi,
> 
Hi Christer,

no it doesn't. It will be basically RFC 4960 + the diff from the errata
document applied.

We did this in the past. See
RFC 2960 as the initial specification of SCTP.
RFC 4460 as an errata document
RFC 4960 as the updated specification of SCTP.

As you see, RFC 4960, does not tell you what has changed or why.
If a developer is not interested in that and just wants to
implement SCTP, only the reading on RFC 4960 is needed. If
someone wants to understand the changes, he can read RFC 4460.

Best regards
Michael

---

From: Michael Tuexen <[email-masked]>
Date: Thu, 7 Jun 2018 06:23:06 -0400

> On 7. Jun 2018, at 02:40, Christer Holmberg <[email-masked]> wrote:
> 
> Hi Michael,
> 
> Please see inline.
> 
Hi Christer,

sorry for not be precise. By double checking that the erratas where covered
I figured out that 5003 was not mentioned (one of the typos) and I added
it yesterday:
https://github.com/sctplab/rfc4960bis/commit/b8e4e8c6c558c66dab4530cce1b0b5af8eea9b0f

Sorry for the confusion.

Best regards
Michael

---

From: Gorry Fairhurst <[email-masked]>
Date: Thu, 07 Jun 2018 11:32:51 +0100

+1, as Chair. I see we have caused a little confusion here - The WG will 
not repeat this list of changes again as a part of the new .bis document.

There could always be potentially be further changes as the .bis 
document passes through the WG - of course - but we'd rather expect this 
spec is mature -- indeed there was suggestion the final Spec could 
progress to Full Standard (to be confirmed of course).

Gorry

On 07/06/2018, 11:19, Michael Tuexen wrote:
>> On 7. Jun 2018, at 02:43, Christer Holmberg<[email-masked]>  wrote:
>>
>>
>> Hi,
>>

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Thu, 7 Jun 2018 05:52:31 -0500

FWIW,

> progress to Full Standard (to be confirmed of course).

Thanks, Michael and Gorry, for clarifying this.

I think everyone who lived through 4460 and 4960 knew this, and no one one
who didn't live through that knew it, so that's very helpful.

Spencer

> >>>> issues? Then, when you start working on the bis, you can map each
> issue
> >>>> to
> >>>> a pull request etc.
> >>>> <[email-masked] on behalf of [email-masked]
> >
> >>>> wrote:
> >>>>
> >>>>> Hi Gorry,
> >>>>>
> >>>>> ...
> >>>>>
> >>>>>>>>> Review Team (Gen-ART) reviews all IETF documents being processed
> by
> >>>>>>>>> the
> >>>>>>>>> IESG for the IETF Chair. Please treat these comments just like
> any
> >>>>>>>>> other
> >>>>>>>>> This draft is on the right track but has open issues, described
> in
> >>>>>>>>> the
> >>>>>>>>> review.
> >>>>>>>>>
> >>>>>>>>> Issues:
> >>>>>>>>>
> >>>>>>>>> Major: 1
> >>>>>>>>> Minor: 2
> >>>>>>>>> Nits:  1
> >>>>>>>>>
> >>>>>>>>> 1) MAJOR:
> >>>>>>>>>
> >>>>>>>>> to implement sctp with some or all of the errata fixes will want
> to
> >>>>>>>>> work
> >>>>>>>>> it isn't clear how this document helps with that. I don't think
> you
> >>>>>>>>> can
> >>>>>>>>> than this format. Is that not being done because doing so would
> be
> >>>>>>>>> more
> >>>>>>>>> form for this document, in the context of how it is intended to
> be
> >>>>>>>>> used.
> >>>>>>>>>
> >>>>>>>>> 2) MINOR (maybe MAJOR):
> >>>>>>>>>

---

From: Christer Holmberg <[email-masked]>
Date: Thu, 7 Jun 2018 10:58:23 +0000

Hi,

>not repeat this list of changes again as a part of the new .bis document.

In my opinion a bis should always describe (at least on a high level) what
changes have been done since the previous version. If for no other reason,
to help companies decide upon, and plan, updates to their products.

>progress to Full Standard (to be confirmed of course).

I guess that is my main question: if you already know how the bis is going
to look like, why not simply make the bis with the changes? :)

As you said above, once the bis work start, things may change in the WG
(or in the later reviews), so you cannot assume that the errata document
will be an 100% accurate description of what will go into the bis.

Anyway, I respect the decisions made by the WG, so as long as the
clarifications I originally requested are implemented I’m happy :)

Regards,

Christer

>>> On 7. Jun 2018, at 02:43, Christer
>>>Holmberg<[email-masked]>  wrote:
>>>
>>>
>>> Hi,
>>>
>>>>>>>>>> Review Team (Gen-ART) reviews all IETF documents being
>>>>>>>>>>processed by
>>>>>>>>>> the
>>>>>>>>>> This format makes the document hard to deal with. A developer
>>>>>>>>>>who
>>>>>>>>>> wants
>>>>>>>>>> to implement sctp with some or all of the errata fixes will
>>>>>>>>>>want to
>>>>>>>>>> work
>>>>>>>>>> references. IOW, shift the burden of detecting conflicts from
>>>>>>>>>>the
>>>>>>>>>> reader
>>>>>>>>>> to the document.
>>>>>>>>>>
>>>>>>>>>> 3) MINOR:
>>>>>>>>>>