Subject: [Detnet] Alvaro Retana's Discuss on draft-ietf-detnet-ip-06: (with DISCUSS and COMMENT)

From: Alvaro Retana via Datatracker <[email-masked]>
Date: Tue, 23 Jun 2020 08:41:46 -0700

Alvaro Retana has entered the following ballot position for
draft-ietf-detnet-ip-06: Discuss

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.

The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-detnet-ip/

----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------

Several sections mandate actions to be taken where the specific mechanisms are
not clearly defined.  This lack of specifics leaves me at a loss about what is
being mandated.

Specifically:

(1) §4.3.2 (Quality of Service): "Quality of Service...MUST be provided
    locally by the DetNet-aware hosts and routers supporting DetNet flows.
    The traffic control mechanisms used to deliver QoS...are expected to be
    defined in a future document."

(2) §5.2 (Forwarding Procedures): "Specifically...SHALL use management and
    control information to select the one or more outgoing interfaces and
    next hops..."  This sentence sounds very generic to me: using management
    and control information is what every forwarder does -- regardless of
    DetNet.  Not only is it a generic statement, but the management and
    control functions are not defined...

(3) §5.3 (DetNet IP Traffic Treatment Procedures): "MUST ensure that a
    DetNet flow receives the traffic treatment that is provisioned for
    it...Typical mechanisms used to provide different treatment to different
    flows includes the allocation of system resources...and provisioning or
    related parameters...Other mechanisms than the ones used in the TSN case
    are outside the scope of this document."

It is ok to define the mechanisms in a different document -- but there are no
specific references.  What exactly is this document requiring (MUST)?  If there
are no specifics on the mechanisms (or where they are defined), how can an
implementation comply with this document?  What are the interoperability
consequences if not all nodes comply with the same set of (undefined)
mechanisms?

I am balloting DISCUSS because I believe that this omission makes the
specification incomplete.   Adding details will satisfy my concern.

----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

(1) §4.2 (DetNet Domain-Specific Considerations):

   As a general rule, DetNet IP domains need to be able to forward any
   DetNet flow identified by the IP 6-tuple.  Doing otherwise would
   limit the number of 6-tuple flow ID combinations that could be used
   by the end systems.  From a practical standpoint this means that all
   nodes along the end-to-end path of DetNet flows need to agree on what
   fields are used for flow identification.

There is text in §5.1 requiring the ability to forward based on the
6-tuple...but there is nothing related to the "need to agree on what fields are
used for flow identification" -- it seems to me that this agreement should also
be required.  §6 does say:

   It is the responsibility of the DetNet controller plane to properly
   provision both flow identification information and the flow specific
   resources needed to provided the traffic treatment needed to meet
   each flow's service requirements.  This applies for aggregated and
   individual flows.

I guess that "to properly provision" can be a standin for the "need to agree on
what fields are used".  I would still like to see something about the effect of
not agreeing on/provisioning the information correctly/consistently.

(2) §4.3.1 (Class of Service): "...DetNet nodes...MUST ensure that the CoS
service classes do not impact the congestion protection and latency control
mechanisms used to provide DetNet QoS."

(a) What enforceable action are the DetNet nodes expected to take to comply
with this statement?  What does ensuring include?

(b) Maybe it's just me, but I would think that the requirement is not at a node
level (each node doing something), but within a DetNet domain; IOW, this sounds
more like a provisioning requirement (even probably related to the text from §6
I quoted above).

(3) §5.1 (DetNet IP Flow Identification Procedures): s/The configuration and
control information used to identify an individual DetNet flow MUST be ordered
by an implementation./An implementation MUST support ordering of the
configuration and control information used to identify an individual DetNet
flow.

The current wording seems to imply that the implementation may order the
information any way it wants...but proper identification through the domain
depends on consistent ordering.

(4) §5.1.1.3 (IPv4 Protocol and IPv6 Next Header Fields):

   Implementations of this document MUST support DetNet flow
   identification based on the IPv4 Protocol field when processing IPv4
   packets, and the IPv6 Next Header Field when processing IPv6 packets.
   An implementation MUST support flow identification based on the next
   protocol values defined in Section 5.1.2.  Other, non-zero values,
   MUST be used for flow identification.

The first sentence represents a superset of the second: in general, using the
Protocol field includes more options than just what is defined in §5.1.2.

The third sentence, again, goes back to a superset: anything else except 0. 
BTW, 0 is the IPv6 HbH option -- do you really want to exclude it?

Can that paragraph be summarized by just the first sentence?

(5) "Implementations SHOULD allow for this field to be ignored for a specific
DetNet flow."   This sentence, or a "MUST allow" version, are used throughout
the document.  Is there a strong reason (that I'm obviously missing) to
sometimes use SHOULD and others MUST?

(6) A reference for ICMP is needed.

(7) §5.1.2.2 (ICMP): "Note that ICMP type is not included in the flow
definition."  Does this mean that the ICMP type should not (SHOULD NOT/MUST NOT
??) be used in the flow definition?   That's what the text sounds to me as
saying.

I found a thread in the archive [1] which talks about this sentence being added
with the intention of supporting ICMP types...but even with that added context
the text just sounds as if the opposite was intended.

[1] https://mailarchive.ietf.org/arch/msg/detnet/wVA93y2cO2nt9ddFcDaTw-eHKK8

(8) I believe that these references should be Normative:
I-D.ietf-detnet-data-plane-framework and RFC8655.

(9) Editorial nits:

s/uses "6-tuple" based flow identification, where 6-tuple refers to/uses
6-tuple based flow identification, where "6-tuple" refers to

s/6-tuple is (destination/6-tuple is destination

s/Same can be valid/The same can be valid

s/using smaller tuples are appropriate/using smaller tuples is appropriate

s/can communicate over DetNet IP network with IP End System./can communicate
over a DetNet IP network with IP End Systems.

s/from data plane perspective/from [a|the] data plane perspective

s/end-system/end system/g

s/multiple methods that MAY be used...including but not limited to/multiple
methods that may be used...including but not limited to

s/matching based on this field MUST allow for this field to be ignored/matching
based on this field MUST allow for it to be ignored

s/found in the [I-D.ietf-detnet-data-plane-framework]/found in
[I-D.ietf-detnet-data-plane-framework]/g

s/provisioning or related parameters/provisioning of related parameters

---

From: Lou Berger <[email-masked]>
Date: Thu, 25 Jun 2020 10:25:52 -0400

Hi Alvaro,

Thank you for the comments/discuss.  Please see responses (as co-author) 
in line below

> https://datatracker.ietf.org/doc/draft-ietf-detnet-ip/

I think you have two points here:

1) What are the traffic control (queuing, shaping policing) mechanisms 
required by this document and

2) What are the required forwarding/node implementation behavior

right?

WRT 1)

As far as I can tell the IETF doesn't specify internal node behavior but 
rather just the required node behavior.  This is even true back to 
RSVP/Int-Serv days.  To be consistent with that, the closest we felt we 
could go is covered in section 5.3:

    5.3.  DetNet IP Traffic Treatment Procedures

    Implementations of this document MUST ensure that a DetNet flow
    receives the traffic treatment that is provisioned for it via
    configuration or the controller plane, e.g., via
    [I-D.ietf-detnet-yang].  General information on DetNet service can be
    found in [I-D.ietf-detnet-flow-information-model].  Typical
    mechanisms used to provide different treatment to different flows
    includes the allocation of system resources (such as queues and
    buffers) and provisioning or related parameters (such as shaping, and
    policing).  Support can also be provided via an underlying network
    technology such as MPLS [I-D.ietf-detnet-ip-over-mpls] or IEEE802.1
    TSN [I-D.ietf-detnet-ip-over-tsn].  Other mechanisms than the ones
    used in the TSN case are outside the scope of this document.

As a WG we're also working on additional *informative* documents that 
relate to a node providing a DetNet service (see 
https://tools.ietf.org/html/draft-ietf-detnet-bounded-latency).

If you have a proposal on what more we should do, it would be most 
appreciated.

WRT 2)

The specific node behavior is specified in sections 5, with 
management/control requirements specified in section 6.  If you feel 
there's not much there, I think that's because there really isn't much 
to the DetNet IP data plane that is more than can be summarized by "IP 
DetNet=PBR+QoS".  Now the document does have specific traffic 
classification and forwarding behaviors -- and this is what is found in 
section 5 and controlled via section 6.

Again, If you have a proposal on what more we should do or what things 
we can fix (beyond what is addressed below), it would be most appreciated.

Please see the specific response below and if your 'discuss'-level 
concerns are still present.

>      defined in a future document."

while likely not diminishing your larger point, I think this is left 
over from older version.  It should read:

it should read:

         The node-internal traffic control mechanisms used to deliver 
QoS for
           IP encapsulated DetNet flows are outside the scope of this
           document.

>      control functions are not defined...

Management and control information is summarized in section 6 and as 
stated just below the quoted text:

    The above implies that management and control functions will be
    defined to support this requirement, e.g., see
    [I-D.ietf-detnet-yang].

> implementation comply with this document?
This is a fair point, but I don't see where we have specific queuing 
mechanisms defined. Now we certainly have on-wire behavior defined, e,g, 
RFC  2212, and others have defined ones IEEE 802.1Qch.  Our expectation 
is that the management (YANG) and future control plane documents will 
specific which forwarding treatment is expected for specific flows. We 
also expect that the market (users/providers and vendors) will decide 
ehcih are appropriate for different use cases and products just as is 
done for MPLS-TE in general.
> What are the interoperability
The traffic delivery won't be what's expected, and those vendors with 
behavior not suited to particular applications won't be selected for 
such networks.  Again, taking a page from the MPLS-TE market place...
> specification incomplete.   Adding details will satisfy my concern.

okay, how about:

           Possible consequences of not having such an agreement include
           having some flows interfering with other flows, and for a
           the traffic treatment expected for a service not being
           provided.

> with this statement?
This is an internal implementation matter.
> What does ensuring include?

That the the traffic treatment of other flows are not impacted -- as 
observed on the wire.

> more like a provisioning requirement (even probably related to the text from §6

I certainly agree there is a large interaction here.

> depends on consistent ordering.

agreed.  to be consistent with the section:

             An implementation MUST support ordering of the
             set of information information used to identify an
             individual DetNet flow.

> Protocol field includes more options than just what is defined in §5.1.2.
Yes, but the second is a refinement and has additional implementation 
implications.
> BTW, 0 is the IPv6 HbH option -- do you really want to exclude it?
I think so, but this was not extensively discussed.  Do you think it's 
the wrong call?
> sometimes use SHOULD and others MUST?

It came down to complexity of new hardware and allowing for support for 
existing hardware as much as possible.

> (6) A reference for ICMP is needed.
Done!
> ??) be used in the flow definition?   That's what the text sounds to me as
This came from a question -- we don't want to preclude a future addition 
of looking ICMP type, but right now it's not included.
> I-D.ietf-detnet-data-plane-framework and RFC8655.
the data plane framework is informative so generally references to 
informative docs are informative.
> 6-tuple based flow identification, where "6-tuple" refers to
yes.
> s/6-tuple is (destination/6-tuple is destination
thanks
> s/Same can be valid/The same can be valid
this text was replaced in an earlier response.
> s/using smaller tuples are appropriate/using smaller tuples is appropriate

the text now reads

      In such cases, using fewer fields is appropriate,

> over a DetNet IP network with IP End Systems.
yes
> s/from data plane perspective/from [a|the] data plane perspective
thanks
> s/end-system/end system/g
done
> methods that may be used...including but not limited to
thanks
> based on this field MUST allow for it to be ignored
okay
> [I-D.ietf-detnet-data-plane-framework]/g
done
> s/provisioning or related parameters/provisioning of related parameters
thanks

Changes covered here can be previewed at:
https://github.com/detnet-wg/data-plane-drafts/tree/working/lb/iesg-0625

https://xml2rfc.tools.ietf.org/cgi-bin/xml2rfc.cgi?url=https://raw.githubusercontent.com/detnet-wg/data-plane-drafts/working/lb/iesg-0625/ip/draft-ietf-detnet-ip.xml

Thank you!

Lou

>
> _______________________________________________
> detnet mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/detnet

---

From: Alvaro Retana <[email-masked]>
Date: Thu, 25 Jun 2020 11:41:07 -0700

On June 25, 2020 at 10:25:59 AM, Lou Berger wrote:

Lou:

Hi!

...

Thanks for the explanation!

> > It is ok to define the mechanisms in a different document -- but there
> > are no specific references. What exactly is this document requiring
> > (MUST)? If there are no specifics on the mechanisms (or where they are
> > defined), how can an implementation comply with this document?
>

My main concern is the combination of normative language (MUST/SHALL)
+ no specifics.  To solve it, I need you to either add specifics or
not use normative language.

Defining mechanisms elsewhere, or letting the market decide, is ok.
If that is where the WG is, then simply don't use normative language
in these descriptions.

If you come up with alternate text, then I'll clear my DISCUSS.

> > What are the interoperability consequences if not all nodes comply with
> > the same set of (undefined) mechanisms?
>

Please add something like that to the text.  There are several places
(e.g., the 3 examples in the DISCUSS and some comments below) that
need that type of disclaimer/guidance.

I wish there was a Deployment Considerations section, but that is
probably too much to ask for a few additional sentences.

...
...
> the traffic treatment expected for a service not being

Sure.  As mentioned above, the same type of text is needed in multiple places.

> > (2) §4.3.1 (Class of Service): "...DetNet nodes...MUST ensure that the
> > CoS service classes do not impact the congestion protection and latency
> > control mechanisms used to provide DetNet QoS."
...
> > (b) Maybe it's just me, but I would think that the requirement is not at
> > a node level (each node doing something), but within a DetNet domain;
> > IOW, this sounds more like a provisioning requirement (even probably
> > related to the text from §6 I quoted above).
>

Then this is another opportunity for text related to the need to have
the same/similar behavior/agreement/provisioning...even if the general
expectation is per node and not explicitly defined.

...
> > (4) §5.1.1.3 (IPv4 Protocol and IPv6 Next Header Fields):
...
> I think so, but this was not extensively discussed. Do you think it's

I don't know.  But if all the other EHs are not excluded there must be
a good reason.  I can imagine ways to use HbH to maybe carry other
flow identification parameter...for later use.  Just thinking out
loud...

...
> informative docs are informative.

Nope.

A normative reference is one that "must be read to understand or
implement the technology"...regardless of the status.

https://www.ietf.org/about/groups/iesg/statements/normative-informative-references/

Thanks!

Alvaro.

---

From: Lou Berger <[email-masked]>
Date: Thu, 25 Jun 2020 15:57:20 -0400

Hi Alvaro,

> not use normative language.

can you point to an RFC that has language that you'd like to see? 
RFC3290 has some nice language, but is informational.

Either way, I'll work with the co authors on wording to address this 
comment (which I think is limited to traffic treatment.)

Lou

---

From: Lou Berger <[email-masked]>
Date: Thu, 25 Jun 2020 17:24:17 -0400

Alvaro

to be precise, you are looking for:

section 4.3.1: s/MUST/must

section 4.3.2: s/IP MUST/IP must

section 5.3: s/MUST/must

Right?

I think that's fine as it translates to conformance being judged based 
on (a) this document, (b) a yang or control plane document, and (c) 
whichever traffic treatment RFCs/references the yang or control plane 
document reference.  This is reasonable (at least to me).

I think you have two additional points below...

> On June 25, 2020 at 3:57:22 PM, Lou Berger ([email-masked] 
> <mailto:[email-masked]>) wrote:
>
...
>> > flow identification parameter...for later use.  Just thinking out

okay, I'll make the change to allow for zero values (assuming no 
objections for others.)

>> https://www.ietf.org/about/groups/iesg/statements/normative-informative-references/ 

okay then.

All,

please look at the repo to see if you object/approve of the changes:

https://github.com/detnet-wg/data-plane-drafts/compare/working/lb/iesg-0625

Thank you!

  Lou

---

From: Alvaro Retana <[email-masked]>
Date: Fri, 26 Jun 2020 12:48:25 -0700

Lou:

There’s also a “SHALL” in §5.2.

The rest looks good to me.

I trust you with the changes, so I’m clearing my DISCUSS.

Thanks!

Alvaro.

On June 25, 2020 at 5:24:22 PM, Lou Berger ([email-masked]) wrote:

Alvaro

section 5.3: s/MUST/must

Right?

...

okay then.

All,

Thank you!

 Lou

---

From: Lou Berger <[email-masked]>
Date: Fri, 26 Jun 2020 17:55:17 -0400

Alvaro,

> There’s also a “SHALL” in §5.2.
I think this shall is right, how about, if we add:

        Specific management and control
           information will be defined in future documents, e.g., <xref
           target="I-D.ietf-detnet-yang"/>.

> I trust you with the changes, so I’m clearing my DISCUSS.
Thanks;-)  I'll upload once we close on the above
> Thanks!
>
> Alvaro.
>
> On June 25, 2020 at 5:24:22 PM, Lou Berger ([email-masked] 
>> <mailto:[email-masked]>
>> > <mailto:[email-masked] <mailto:[email-masked]>>) wrote:
>> >
>> ...
>> >> > implement the technology"...regardless of the status