Subject: [Detnet] DetNet IP Data Plane draft review pass

From: "Grossman, Ethan A." <[email-masked]>
Date: Wed, 23 Oct 2019 02:07:29 +0000

Hi Balazs et al,
I went over the data-plane-ip draft in detail and here are my observations. I was working from ver 02 before ECN was expunged, but I am duly ignoring ECN references per Lou's update.
Ethan.
----------------------------------------------

  1.  "The DetNet Architecture models the DetNet related data plane functions decomposed into two sub-layers: functions into two sub- layers: a service sub-layer and a forwarding sub-layer." Typo, maybe you want "The DetNet Architecture models the DetNet-related data plane functions as two sub-layers: a service sub-layer and a forwarding sub-layer." Funny though how we interchange "layer" and "plane". "Forwarding sub-plane" sounds odd I suppose?
  2.  Are "Service layer" and "Forwarding layer" proper names, i.e. s/b capitalized? They are capitalized in the figures.
  3.  "DetNet specific" - hyphenate "DetNet-specific". Same for "DetNet-aware".
  4.  "Since no DetNet specific headers are added to support DetNet IP flows, only the forwarding sub-layer functions are supported using the DetNet IP defined by this document." I would like to see additional background text before this so that in total it would say something like:
"The Service layer requires additional fields to provide its service; for example to provide service protection via PREOF, each packet must be assigned a sequence number. Since no DetNet-specific fields (such as sequence number) are added to support DetNet IP flows, the DetNet Service layer for IP cannot support PREOF. Thus only the forwarding sub-layer functions are supported using the DetNet IP defined by this document."
  5.  I don't see any reference to adding a sequence number and doing PREOF with IPv6  header extension - did that concept get dropped?
  6.  "The service sub-layer is used to provide DetNet service protection" - As the first time this is introduced this is where it should be followed by "(packet replication and packet elimination functions)" as done later in the text, and as done for Forwarding layer in the same paragraph.
  7.  "This document uses the terminology and concepts established in the DetNet architecture [I-D.ietf-detnet-architecture], and the reader is assumed to be familiar with that document and its terminology. 2.2.  Abbreviations..." - then the whole set of Abbreviations from the Arch draft is copy/pasted here, so now there are two copies, including at least one typo (PREOF - Packet Replication, Ordering and Elimination Function, s/b ordered Elimination and Ordering to match abbreviation, right?). Presumably they will then be pasted into the other DP drafts? But maybe it is common to list all Abbrev's locally?
  8.  "This document describes how IP is used by DetNet nodes, i.e., hosts and routers, identify DetNet flows and provide a DetNet service using an IP data plane." - missing a word there - "routers, to identify DetNet flows"
  9.  Sec 3 DetNet IP Data Plane Overview: "Common data plane procedures and control information for all DetNet data planes can be found in the [I-D.ietf-detnet-data-plane-framework]." - this was already said in the Intro, but maybe that duplication is OK?
  10. "Generally the fields used in flow identification are forward unmodified but modification is allowed, for example to a DSCP value, when required by the DetNet service." - by this do you mean "Generally the fields used in flow identification are forwarded unmodified, however modification of these fields is allowed, for example to a DSCP value, when required by the DetNet service." - If we are saying that, shouldn't we use normative language instead of "is allowed"?
  11. "DetNet service assurance" - this phrase is used exactly once in the Framework draft (capitalized) and exactly once here (not capitalized). I think we should replace this with "DetNet Service Protection" then be consistent in using it (i.e. also change in Framework). I.e. eliminate use of word "assurance".
  12. "The DetNet enabled end systems originate IP encapsulated traffic that is identified as DetNet flows" - the phrase "is identified" - does this mean that the application identifies the flow to the Controller plane? What does this mean at the data plane? I mean, the flows are identified by 6-tuple, but there is no other "is identified", right? Anyway please clarify this, perhaps by making an "active voice" statement like "the application identifies the flow to the Controller plane which allocates resources such that when the forwarding plane forwards the packet it is guaranteed to meet its QoS requirements".
  13. "IEEE 802.1 TSN is an example sub-network type which can provide support for DetNet flows and service.
Note: The sub-network can represent a TSN, MPLS or IP network segment." - Unify these two statements (either in the note or in the text).
  14. "Note, that Figure 1 and Figure 2 can be combined, so IP DetNet End Systems can communicate over DetNet IP network with IP End System." - I find "combined " to be somehow ambiguous - can we say "Note: An IP DetNet End System can communicate over a DetNet IP network terminating at an IP End System, in effect a hybrid of figure 1 and figure 2." Maybe that isn't any better, but I think that putting the "specific" clause before the "generalization" clause would be clearer.
  15. "Non-DetNet and DetNet IP packets are identical on the wire..." - Can we move this paragraph earlier in the section? Somehow it answers a lot of questions and should be known sooner. I think it would be better to put it just before "DetNet flow aggregation may be enabled via the use of wildcards", just before Fig 1.
  16. "Data-flows" - no hyphen here.
  17. "Data-flows requiring DetNet service are generated and terminated on end systems.  This document deals only with IP end systems." - this seems choppy. Can we say "IP Data-flows requiring DetNet service are generated and terminated on IP end systems."
  18. "The protocols used by an IP end system are specific to an application and end systems peer with end systems using the same application encapsulation format." - Is "protocol specific to an application" referring to the same thing as "application encapsulation format"? Somehow I understand that any given Application must somehow communicate to the DetNet End System that packets with the given flow ID must be given DetNet treatment, e.g. bandwidth reservation. Then at the destination end system, there is some symmetrical operation, but what is that? Presumably there is a UDP socket there that gets the packets, what else would have to be done there? Thus far we have said nothing about this whole process, until we suddenly imply it here. Can we somehow enumerate what is supposed to happen between the Application and End System, and then at the other side of the network, what is supposed to happen between the End System there and the Application at the far end?
  19. "This said, DetNet's use of 6-tuple IP flow identification means that DetNet must be aware of not only the format of the IP header, but also of the next protocol carried within an IP packet." - I think this would benefit from a forward reference to [5.1.1.3.  IPv4 Protocol and IPv6 Next Header Fields].
  20. "when processing packets associated with a DetNet flow" - above we said "identified" - can we use "identified as belonging to a DetNet flow" for consistency here? As in "Flow ID". Or is there something clever, like "associated with" is supposed to also include in-band OAM packets?
  21. "appropriately shaped on transmission and received appropriate traffic treatment" - I think "received" should be "receive".
  22. "receive and process a DetNet flow packets" - maybe ""receive and process the packets of that DetNet flow"
  23. "Doing otherwise would limit end system encapsulation format." - I think you are saying "Doing otherwise would limit the number of 6-tuple flow ID combinations that could be used by the end systems."
  24. "end system is directly connected to an edge node or end system is behind a sub-network" - to clarify, "The end system is directly connected to an edge node, or the end system is behind a sub-network".
  25. "2.  DN integrated: end system" - "...The end system"
  26. "DetNet enabled IP Routers" - "DetNet-enabled IP routers"
  27. "system specific" - "system-specific"
  28. "service protections ... are not" - "service protection ... is not"
  29. "As noted earlier, the Service Protection is done within each link / sub-network independently using the domain specific mechanisms (due the lack of a unified end to end sequencing information that would be available for intermediate nodes)." - Hmm, SP might be done in other ways e.g. network coding? "As noted earlier, service protection is implemented locally within each link / sub-network independently, using domain-specific mechanisms, for example, but not limited to, PREOF. Implementing service protection between domains is not possible with DetNet IP because there is no mechanism for ensuring uniqueness of the sequencing information between domains." - or maybe just stick closer to the original "As noted earlier, service protection must be implemented within each link / sub-network independently, using the domain specific mechanisms. This is due to the lack of unified end-to-end sequencing information that could be used by the intermediate nodes (see Fig 4)."
  30. "simpler 5-tuple filters to be reused at the edges" - "simpler 5-tuple filters to be used (or re-used) at the edges"
  31. "non-congestion responsive" - "non-congestion-responsive"
  32. "This requirement is similar to requirement for MPLS LSRs to that CoS LSPs do not impact the resources allocated to TE LSPs via [RFC3473]" - "This requirement is similar to the requirement for MPLS LSRs that CoS LSPs must not impact the resources allocated to TE LSPs [RFC3473]".
  33. Sec 4.3.2 "the network nodes of a DetNet network must" - is that a MUST? Review use of "must" generally?
  34. "a aggregate" - "an aggregate"
  35. "in future" - "in the future"
  36. "when the implementation is acting as a DetNet end systems, a relay node or as an edge node" - "when the implementation is acting as a DetNet end system, a relay node, or an edge node"
  37. "An implementation MUST support flow identification based based the next protocol values defined" - "An implementation MUST support flow identification based on the next protocol values defined"
  38. "belonging to a DetNet flow" - I suppose here is another alternative to "identified" or "associated". Again this is the only time this phrase is used in this draft, and is not used in the Framework draft.
  39. "such as MPLS [I-D.ietf-detnet-ip-over-mpls].  And IEEE802.1 TSN" - "such as MPLS [I-D.ietf-detnet-ip-over-mpls]  or IEEE802.1 TSN"
  40. "When used, can be exclusive of matching against the next header field." - I think you mean "When used, can be used instead of matching against the Next Header field." But I'm not sure.
----------------------------------------------

---

From: Balázs Varga A <[email-masked]>
Date: Thu, 24 Oct 2019 20:40:57 +0000

Hi Ethan,

Thanks for the great comments. Resolution inline below.
New Pull Request (#62) created on Github.

Cheers
Bala'zs

From: Grossman, Ethan A. <[email-masked]>
Sent: Wednesday, October 23, 2019 4:07 AM
To: [email-masked]; Balázs Varga A <[email-masked]>
Subject: DetNet IP Data Plane draft review pass

Hi Balazs et al,

OK.

  1.  Are "Service layer" and "Forwarding layer" proper names, i.e. s/b capitalized? They are capitalized in the figures.
NOT OK. Architecture (RFC8655) defines then with no capitalization. Let's remain consequent.

  1.  "DetNet specific" - hyphenate "DetNet-specific". Same for "DetNet-aware".
OK.

  1.  "Since no DetNet specific headers are added to support DetNet IP flows, only the forwarding sub-layer functions are supported using the DetNet IP defined by this document." I would like to see additional background text before this so that in total it would say something like:
"The Service layer requires additional fields to provide its service; for example to provide service protection via PREOF, each packet must be assigned a sequence number. Since no DetNet-specific fields (such as sequence number) are added to support DetNet IP flows, the DetNet Service layer for IP cannot support PREOF. Thus only the forwarding sub-layer functions are supported using the DetNet IP defined by this document."
OK.

  1.  I don't see any reference to adding a sequence number and doing PREOF with IPv6  header extension - did that concept get dropped?
YES it was dropped, We preferred to have similar DetNet IP for both IPv4 and IPv6.

  1.  "The service sub-layer is used to provide DetNet service protection" - As the first time this is introduced this is where it should be followed by "(packet replication and packet elimination functions)" as done later in the text, and as done for Forwarding layer in the same paragraph.
OK.

  1.  "This document uses the terminology and concepts established in the DetNet architecture [I-D.ietf-detnet-architecture], and the reader is assumed to be familiar with that document and its terminology. 2.2.  Abbreviations..." - then the whole set of Abbreviations from the Arch draft is copy/pasted here, so now there are two copies, including at least one typo (PREOF - Packet Replication, Ordering and Elimination Function, s/b ordered Elimination and Ordering to match abbreviation, right?). Presumably they will then be pasted into the other DP drafts? But maybe it is common to list all Abbrev's locally?
OK. Typo corrected. It is easier to read if all abbrevs are local.

  1.  "This document describes how IP is used by DetNet nodes, i.e., hosts and routers, identify DetNet flows and provide a DetNet service using an IP data plane." - missing a word there - "routers, to identify DetNet flows"
OK.

  1.  Sec 3 DetNet IP Data Plane Overview: "Common data plane procedures and control information for all DetNet data planes can be found in the [I-D.ietf-detnet-data-plane-framework]." - this was already said in the Intro, but maybe that duplication is OK?
OK, I would keep the duplicate for now.

  1.  "Generally the fields used in flow identification are forward unmodified but modification is allowed, for example to a DSCP value, when required by the DetNet service." - by this do you mean "Generally the fields used in flow identification are forwarded unmodified, however modification of these fields is allowed, for example to a DSCP value, when required by the DetNet service." - If we are saying that, shouldn't we use normative language instead of "is allowed"?
OK. Modification done. We are still in "... overview" so "is allowed is fine here".

  1.  "DetNet service assurance" - this phrase is used exactly once in the Framework draft (capitalized) and exactly once here (not capitalized). I think we should replace this with "DetNet Service Protection" then be consistent in using it (i.e. also change in Framework). I.e. eliminate use of word "assurance".
OK in principle. I have deleted "assurance". Intention was to note that DetNet service is provided for the aggregate. Protection is not the right word here as it is just a function.

  1.  "The DetNet enabled end systems originate IP encapsulated traffic that is identified as DetNet flows" - the phrase "is identified" - does this mean that the application identifies the flow to the Controller plane? What does this mean at the data plane? I mean, the flows are identified by 6-tuple, but there is no other "is identified", right? Anyway please clarify this, perhaps by making an "active voice" statement like "the application identifies the flow to the Controller plane which allocates resources such that when the forwarding plane forwards the packet it is guaranteed to meet its QoS requirements".
Not OK. But agree some clarification is needed here. Intention was to say that DetNet nodes recognize DetNet flows based on the native encapsulation (without adding additional fields). I will change to "The DetNet enabled end systems originate IP encapsulated traffic those are identified within the DetNet domain as DetNet flows."

  1.  "IEEE 802.1 TSN is an example sub-network type which can provide support for DetNet flows and service.
Note: The sub-network can represent a TSN, MPLS or IP network segment." - Unify these two statements (either in the note or in the text).
OK. Note was kept.

  1.  "Note, that Figure 1 and Figure 2 can be combined, so IP DetNet End Systems can communicate over DetNet IP network with IP End System." - I find "combined " to be somehow ambiguous - can we say "Note: An IP DetNet End System can communicate over a DetNet IP network terminating at an IP End System, in effect a hybrid of figure 1 and figure 2." Maybe that isn't any better, but I think that putting the "specific" clause before the "generalization" clause would be clearer.
OK in principle. I have changed combined to collapsed.

  1.  "Non-DetNet and DetNet IP packets are identical on the wire..." - Can we move this paragraph earlier in the section? Somehow it answers a lot of questions and should be known sooner. I think it would be better to put it just before "DetNet flow aggregation may be enabled via the use of wildcards", just before Fig 1.
OK.

  1.  "Data-flows" - no hyphen here.
OK.

  1.  "Data-flows requiring DetNet service are generated and terminated on end systems.  This document deals only with IP end systems." - this seems choppy. Can we say "IP Data-flows requiring DetNet service are generated and terminated on IP end systems."
OK.

  1.  "The protocols used by an IP end system are specific to an application and end systems peer with end systems using the same application encapsulation format." - Is "protocol specific to an application" referring to the same thing as "application encapsulation format"? Somehow I understand that any given Application must somehow communicate to the DetNet End System that packets with the given flow ID must be given DetNet treatment, e.g. bandwidth reservation. Then at the destination end system, there is some symmetrical operation, but what is that? Presumably there is a UDP socket there that gets the packets, what else would have to be done there? Thus far we have said nothing about this whole process, until we suddenly imply it here. Can we somehow enumerate what is supposed to happen between the Application and End System, and then at the other side of the network, what is supposed to happen between the End System there and the Application at the far end?
OK. I have changed text to "Encapsulation of application data on the IP end system is application specific and it has to match the encapsulation format of the peer IP end system."

  1.  "This said, DetNet's use of 6-tuple IP flow identification means that DetNet must be aware of not only the format of the IP header, but also of the next protocol carried within an IP packet." - I think this would benefit from a forward reference to [5.1.1.3.  IPv4 Protocol and IPv6 Next Header Fields].
OK.

  1.  "when processing packets associated with a DetNet flow" - above we said "identified" - can we use "identified as belonging to a DetNet flow" for consistency here? As in "Flow ID". Or is there something clever, like "associated with" is supposed to also include in-band OAM packets?
OK. I change to "identified". No OAM was meant here.

  1.  "appropriately shaped on transmission and received appropriate traffic treatment" - I think "received" should be "receive".
OK.

  1.  "receive and process a DetNet flow packets" - maybe ""receive and process the packets of that DetNet flow"
OK.

  1.  "Doing otherwise would limit end system encapsulation format." - I think you are saying "Doing otherwise would limit the number of 6-tuple flow ID combinations that could be used by the end systems."
OK.

  1.  "end system is directly connected to an edge node or end system is behind a sub-network" - to clarify, "The end system is directly connected to an edge node, or the end system is behind a sub-network".
OK.

  1.  "2.  DN integrated: end system" - "...The end system"
OK.

  1.  "DetNet enabled IP Routers" - "DetNet-enabled IP routers"
OK.

  1.  "system specific" - "system-specific"
OK.

  1.  "service protections ... are not" - "service protection ... is not"
OK.

  1.  "As noted earlier, the Service Protection is done within each link / sub-network independently using the domain specific mechanisms (due the lack of a unified end to end sequencing information that would be available for intermediate nodes)." - Hmm, SP might be done in other ways e.g. network coding? "As noted earlier, service protection is implemented locally within each link / sub-network independently, using domain-specific mechanisms, for example, but not limited to, PREOF. Implementing service protection between domains is not possible with DetNet IP because there is no mechanism for ensuring uniqueness of the sequencing information between domains." - or maybe just stick closer to the original "As noted earlier, service protection must be implemented within each link / sub-network independently, using the domain specific mechanisms. This is due to the lack of unified end-to-end sequencing information that could be used by the intermediate nodes (see Fig 4)."
OK. I used your "stick closer" text.

  1.  "simpler 5-tuple filters to be reused at the edges" - "simpler 5-tuple filters to be used (or re-used) at the edges"
OK.

  1.  "non-congestion responsive" - "non-congestion-responsive"
OK.

  1.  "This requirement is similar to requirement for MPLS LSRs to that CoS LSPs do not impact the resources allocated to TE LSPs via [RFC3473]" - "This requirement is similar to the requirement for MPLS LSRs that CoS LSPs must not impact the resources allocated to TE LSPs [RFC3473]".
OK.

  1.  Sec 4.3.2 "the network nodes of a DetNet network must" - is that a MUST? Review use of "must" generally?
OK.

  1.  "a aggregate" - "an aggregate"
OK.

  1.  "in future" - "in the future"
OK

  1.  "when the implementation is acting as a DetNet end systems, a relay node or as an edge node" - "when the implementation is acting as a DetNet end system, a relay node, or an edge node"
OK.

  1.  "An implementation MUST support flow identification based based the next protocol values defined" - "An implementation MUST support flow identification based on the next protocol values defined"
OK.

  1.  "belonging to a DetNet flow" - I suppose here is another alternative to "identified" or "associated". Again this is the only time this phrase is used in this draft, and is not used in the Framework draft.
OK.

  1.  "such as MPLS [I-D.ietf-detnet-ip-over-mpls].  And IEEE802.1 TSN" - "such as MPLS [I-D.ietf-detnet-ip-over-mpls]  or IEEE802.1 TSN"
OK.

  1.  "When used, can be exclusive of matching against the next header field." - I think you mean "When used, can be used instead of matching against the Next Header field." But I'm not sure.
OK.