Subject: [Detnet] Benjamin Kaduk's No Objection on draft-ietf-detnet-ip-06: (with COMMENT)

From: Benjamin Kaduk via Datatracker <[email-masked]>
Date: Wed, 24 Jun 2020 09:39:49 -0700

Benjamin Kaduk has entered the following ballot position for
draft-ietf-detnet-ip-06: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.

The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-detnet-ip/

----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

The other ADs caught a bunch of editorial stuff that I'll try to not
duplicate, though I do list a bunch of editorial stuff as well.

Section 1

   and routers that provide DetNet service to IP encapsulated data.  No
   DetNet-specific encapsulation is defined to support IP flows, instead
   the existing IP and higher layer protocol header information is used
   to support flow identification and DetNet service delivery.  Common

nit: comma splice

   This document provides an overview of the DetNet IP data plane in
   Section 3, considerations that apply to providing DetNet services via
   the DetNet IP data plane in Section 4.  Section 5 provides the

nit: this looks like it's trying to be a list but failing, so it's an
incomplete sentence.

Section 3

Do we want to have a note anywhere in here about how information about
which flows are DetNet flows and what behavior(s) they get (e.g., the
controller plane, or perhaps a forward reference to section 5), to
remind the reader about the architecture?

   Non-DetNet and DetNet IP packets are identical on the wire.

I suggest a different word than "identical", perhaps
"indistinguishable" (or maybe "the non-DetNet and DetNet IP packet
layouts")

   DetNet flow aggregation may be enabled via the use of wildcards,
   masks, lists, prefixes and ranges.  IP tunnels may also be used to

side note: I'd consider rewording so this doesn't sound like an
exhaustive list.

   Figure 1 illustrates a DetNet enabled IP network.  The DetNet enabled
   end systems originate IP encapsulated traffic that is identified
   within the DetNet domain as DetNet flows.  Relay nodes understand the

nit: I think the description of "identified" could be tightened up -- at
present, one could read this as either "the DetNet ingress classifies IP
traffic as DetNet or non DetNet, thus identifying the flows that are
DetNet flows", or "within the DetNet domain, these flows (originated
from the specific end systems) are identifiable as being DetNet flows".

   As non-DetNet and DetNet IP packets are identical on the wire, from
   data plane perspective, the only difference is that there is flow-

[whatever is done for "identical" above could be done here as well]
nit: missing article before "data-plane perspective"

Section 4.1

   IP flow identification means that DetNet must be aware of not only
   the format of the IP header, but also of the next protocol carried
   within an IP packet (see Section 5.1.1.3).

The way this is written it seems to bind as "(must be aware of) (not
only the format of) (the IP header) (but also of [the format of] the next
protocol)", but following the reference, it seems to only require
inspecting the value in the "next protocol" IP header field.
There would, of course, *also* be a requirement to look into TCP/UDP
headers to extract the ports needed for the 6-tuple, which is where I
originally thought this was going, but given the apparent intent, my
suggestion is to just s/next protocol/next protocol value/.

   End systems need to ensure that DetNet service requirements are met
   when processing packets associated to a DetNet flow.  When forwarding

We just said in the previous paragraph that end systems can be
DetNet-unaware; how would such DetNet-unaware systems ensure that DetNet
service requirements are met?

   packets, this means that packets are appropriately shaped on

Also, are end systems going to be *forwarding* packets?  I thought that
it was DetNet edge nodes (and transit/relay nodes) that would do so.

Section 4.2

   aware.  Such routers identify DetNet flows based on the IP 6-tuple,
   and ensure that the DetNet service required traffic treatment is
   provided both on the node and on any attached sub-network.

nit: I suggest hyphenating the compound adjective(s?) in "DetNet service
required traffic treatment" or otherwise rewording it, as I'm not
entirely clear on how to parse it.

   This solution provides DetNet functions end to end, but does so on a
   per link and sub-network basis.  Congestion protection and latency

We should probably note that there is a single point of failure at the
routers/etc. that remove and reapply these functions.

   Note that not mixing DetNet and non-DetNet traffic within a single
   5-tuple, as described above, enables simpler 5-tuple filters to be
   used (or re-used) at the edges of a DetNet network to prevent non-
   congestion-responsive DetNet traffic from escaping the DetNet domain.

I'd consider s/described/recommended/

Also, does the "simpler filters" also apply to determining what inbound
traffic should receive DetNet treatment?

Section 4.3.2

   future document.  From an encapsulation perspective, the combination
   of the 6-tuple i.e., the typical 5-tuple enhanced with the DSCP and
   previously mentioned optional field (i.e., flow label), uniquely
   identifies a DetNet IP flow.

I agree with the directorate reviewer that the language here needs to
get cleaned up.

   queue or shaper, is used by such flows.  There are multiple methods
   that MAY be used by an implementation to defend service delivery to
   reserved DetNet flows, including but not limited to:

nit: this doesn't seem like a normative MAY.

Section 4.3.3

   next hops.  As mentioned above, the DetNet IP data plane identifies
   flows using "6-tuple" header information as well as the additional
   optional header field.  DetNet generally allows for both flow-

Please just name the "optional header field" field directly; there's no
need to be oblique about it.

   Care should be taken when using different next hops for the same
   5-tuple.  As discussed in [RFC7657], unexpected behavior can occur
   when a single 5-tuple application flow experience reordering due to
   being split across multiple next hops.  Understanding of the

For some reason I thought the detnet architecture doc also had some
discussion of potential consequences of having a single flow split
across multiple paths, but all I can find is Section 3.2.2.2 that points
out there may be need to have additional buffering in such cases, in
order to equalize delays.  (Maybe I'm thinking of a different detnet
document, though I'm not sure which one that would be.)

   application and transport protocol impact of using different next
   hops for the same 6-tuple is required.  Again, this impacts path

I don't think I understand why we went from "5-tuple" to "6-tuple" in
the same paragraph.

Section 4.4

   flow identification.  DetNet IP flows can be aggregated using any of
   the 6-tuple, and an additional optional field (i.e., flow label).

Similarly to the previous comments, perhaps this should be "any of the
6-tuple and flow label"?  There's no need to add "optional" when already
subject to "using any of".

Section 5.1

   IP and higher layer protocol header information is used to identify
   DetNet flows.  All DetNet implementations that support this document
   MUST identify individual DetNet flows based on the set of information
   identified in this section.  Note, that additional flow

Does this requirement apply even to those implementations that have no
need to act at flow-level granularity, e.g., a router that only provide
DetNet service for aggregated flows, as recommended in Section 4.4?

   The configuration and control information used to identify an
   individual DetNet flow MUST be ordered by an implementation.
   Implementations MUST support a fixed order when identifying flows,
   and MUST identify a DetNet flow by the first set of matching flow
   information.

I'm a little surprised that this type of determinism is only an
implementation-level requirement.  If I'm understanding correctly, this
requirement would still allow for two different implementations to use
different rules for flow identification, and thus, potentially, to map a
single end-to-end flow to different classes of service.

Section 5.1.1.3

   An implementation MUST support flow identification based on the next
   protocol values defined in Section 5.1.2.  Other, non-zero values,
   MUST be used for flow identification.  Implementations SHOULD allow

I don't understand what distinction we're trying to draw between flow
identification based on the values defined in Section 5.1.2 and flow
identification based on other (non-zero) values.  Aren't they both
MUST-level requirements, as written?

   for these fields to be ignored for a specific DetNet flow.

side note: it seems like we're using "flow" to mean a couple different
(but similar) things in this paragraph (e.g., "considering IPv4
Protocol/IPv6 Next Header to be distinct or not").  Perhaps that's
unavioidable, though.

Section 5.1.1.4

   Traffic Class Field when processing IPv6 packets.  Implementations
   MUST support list based matching of DSCP values, where the list is

nit: hyphenate "list-based matching"

Section 6

   Information identifying a DetNet flow is ordered and implementations
   use the first match.  This can, for example, be used to provide a

[this bit might perhaps be affected if my previous comment about
implementation-specific results in any changes]

   DetNet service for a specific UDP flow, with unique Source and
   Destination Port field values, while providing a different service
   for the aggregate of all other flows with that same UDP Destination
   Port value.

[and this part actually seems to be setting at least an expectation that
the order is tied down more tightly than just "implementation specific".
Is this order perhaps something that the controller plane is expected to
provision?]

Section 7

We discuss elsewhere the need for end systems to (paraphrasing) "not
shoot themself in the foot", i.e., actually provide the resources
necessary to meet DetNet traffic guarantees.  For the case of DN
attached systems that are behind a sub-network (e.g., ES2 in Figure 3),
are there also considerations on the ability of that sub-network to
provide the necessary level of service?

Is there a need for synchronization of information about DetNet flows?
For example, would the controller plane need to ensure that information
about a new flow is fully propagated to the entire DetNet domain before
any traffic on that flow starts to flow?

If an attacker knows the 6-tuple used by a DetNet flow, isn't there a
risk that they could use IP-level spoofing to generate traffic that gets
classified as a DetNet flow, potentially causing that flow to exceed its
reservation and get erminated?  This may be a slightly different risk
than the one discussed in the last paragraph as requiring policing at
the input of the DetNet domain.  (This is related to the "source address
spoofing" risk that Bob Briscoe raised.)

   Security considerations for DetNet are described in detail in
   [I-D.ietf-detnet-security].  General security considerations are
   described in [RFC8655].  This section considers exclusively security

side note: the wording of these first two sentences feels a little
strange, as if the two documents should be somehow the same but are not.
Perhaps:

% Detailed security considerations for DetNet are catalogued in
% [I-D.ietf-detnet-security], and a more general (abbreviated) treatment
% is provided in [RFC8655].

Also, it looks like the comments I made on 2020-04-23 regarding
draft-ietf-detnet-security (as a side note in my ballot position on
draft-ietf-detnet-data-plane-framework-04) remain applicable:

% The referenced document (now at -09) seems significantly improved from
% when I previously made an in-passing review of the -03
% (https://mailarchive.ietf.org/arch/msg/detnet/jZLodXBmQa7ZFDbBIvDO_xCDD0E/),
% however, some of the issues I mentioned there remain, at least in part
% (e.g., mentioning the use of HMAC without a colocated discussion of
% the need for key distribution and having a taxonomy of threats titled
% "threat model").

   Security aspects which are unique to DetNet are those whose aim is to
   provide the specific quality of service aspects of DetNet, which are
   primarily to deliver data flows with extremely low packet loss rates
   and bounded end-to-end delivery latency.

I recognize that this is essentially verbatim from RFC 8655, but it
still reads somewhat strangely to me: specifically, it seems to say that
there are "security aspects [...] whose aim is to provide [QoS, low
packet loss, and bounded latency]", and I don't think it's the security
aspects specifically that aim to provide those.  Rather, I think it's
the overall goal of DetNet to provide those attributes, and there are
some security aspepcts specific to attempting to provide those
attributes.

   The primary considerations for the data plane is to maintain
   integrity of data and delivery of the associated DetNet service
   traversing the DetNet network.  Application flows can be protected
   through whatever means is provided by the underlying technology.  For
   example, encryption may be used, such as that provided by IPSec
   [RFC4301] for IP flows and/or by an underlying sub-net using MACSec
   [IEEE802.1AE-2018] for IP over Ethernet (Layer-2) flows.

It's good that you call this out, so thank you for that.
I'd suggest a slight addition, prefacing the sentence that currently
starts with "Application flows" with a lead-in that "Since no
DetNet-specific fields are available for DetNet IP, data integrity must
be provided by a lower (or, potentially, higher) layer; application
flows can [...]".  I'd also consider noting that providing guaranteed
delivery is impossible in the face of a sufficiently powerful attacker
(that can drop all traffic, cut fiber, etc.), so the attacker in the
DetNet threat model is necessarily slightly weaker than a typical BCP 72
threat model.

Section 11.1

The way in which RFC 3473 is used does not seem to require a normative
reference, to me.

(RFC 4301 is perhaps in a similar situation, though given that we need
normative references to 4302 and 4303, it doesn't stick out as much.)

Section 11.2

Was it considered to make draft-ietf-detnet-data-plane-framework a
normative reference?  The reference in Section 3 that says "common data
plane procedures and control information for all DetNet data planes" can
be found in it makes me wonder whether it contains information that I
need to know in order to use detnet-ip properly.

---

From: Lou Berger <[email-masked]>
Date: Thu, 25 Jun 2020 11:38:41 -0400

Hi Benjamin,

Thank you for the comments/discuss.  Please see responses (as co-author)
in line below

> https://datatracker.ietf.org/doc/draft-ietf-detnet-ip/
The above comments relate to text that has been modified based on 
earlier responses.
> "indistinguishable" (or maybe "the non-DetNet and DetNet IP packet

how about

           Non-DetNet and DetNet IP packets can have the same protocol
             header format on the wire.

> from the specific end systems) are identifiable as being DetNet flows".
added "...based on IP header information"
> nit: missing article before "data-plane perspective"
sure
> suggestion is to just s/next protocol/next protocol value/.
thanks
> service requirements are met?
I'll rearrange the sentences to make clear what applies to aware end 
systems.
> it was DetNet edge nodes (and transit/relay nodes) that would do so.

s/forwarding/sending

> I'd consider s/described/recommended/
I think I'll defer to the RFC editor on the above.
> traffic should receive DetNet treatment?
I don't think so as 6-tuple support is required in the detnet domain.
> I agree with the directorate reviewer that the language here needs to

Do you have a specif recommendation?

> nit: this doesn't seem like a normative MAY.
agreed
> need to be oblique about it.
sure
> document, though I'm not sure which one that would be.)
we're not really supporting such splits in this document, which is more 
specific than the architecture document.
> I don't think I understand why we went from "5-tuple" to "6-tuple" in
good catch.  This should be 5-tuple
> subject to "using any of".
I think I prefer the current phrasing as it won't lead someone to get 
confused on if the field is required.
> DetNet service for aggregated flows, as recommended in Section 4.4?
Yes.
> single end-to-end flow to different classes of service.

This is based on the management (yang)  or control plane protocol 
semantics.  We would expect each to cover such unambiguously.

> MUST-level requirements, as written?

This paragraph has morphed as the doc evolved.  How about:

               Implementations of this document MUST support DetNet flow
               identification based on the IPv4 Protocol field when
               processing IPv4 packets, and the IPv6 Next Header Field
               when processing IPv6 packets. This includes
               the next protocol values defined in Section 5.1.2 and other
               non-zero values.

(Alvaro may be happier with this change too)

> nit: hyphenate "list-based matching"

Sure

> implementation-specific results in any changes]

here's the new phrasing from the discussion with alvaro:

             An implementation MUST support ordering of the
             set of information information used to identify an
             individual DetNet flow

> provision?].

yes -- as stated above...

> provide the necessary level of service?

Yes -- although this is a consideration below IP - e.g., addressed via 
802.1TSN

> any traffic on that flow starts to flow?

For the detnet service to be delivered end to end - certainly. From a 
traffic protection/isolation standpoint, I don't think so as this is no 
differ4ent than an IP flow existing on a network without any detnet state

> reservation and get erminated?
sure, but how would it enter the detnet network?  I'd expect the traffic 
to be dropped at the domain boundary.
> This may be a slightly different risk
sure.
> some security aspepcts specific to attempting to provide those

I do agree.  The current text came about through lots of discussions 
with other security minded folks, so would prefer to leave it unless you 
strongly object.

> flows can [...]".

how about:

         Since no DetNet-specific fields are available in the DetNet IP
         data plane, the integrity and confidentiality of application 
flows ...

> I'd also consider noting that providing guaranteed
Do you have a specific text proposal?
> The way in which RFC 3473 is used does not seem to require a normative
nice catch!
> normative reference?
yes, but it's an informational document so didn't want the downref.
> The reference in Section 3 that says "common data
Changes covered here can be previewed at:

https://github.com/detnet-wg/data-plane-drafts/tree/working/lb/iesg-0625

https://xml2rfc.tools.ietf.org/cgi-bin/xml2rfc.cgi?url=https://raw.githubusercontent.com/detnet-wg/data-plane-drafts/working/lb/iesg-0625/ip/draft-ietf-detnet-ip.xml

Thank you!

Lou

---

From: Benjamin Kaduk <[email-masked]>
Date: Thu, 25 Jun 2020 19:28:46 -0700

Hi Lou,

Also in-line.

> The above comments relate to text that has been modified based on 

Fair enough.  (And I do see "The DetNEt controller plane [...] is
responsible for providing each node [...]" which should do the trick.
Thanks!)

>              header format on the wire.

Seems reasonable.  Maybe s/can//, though?  I don't know of a case where the
format is actually different.

> added "...based on IP header information"

Okay, thanks.

> I think I'll defer to the RFC editor on the above.

Okay.

> I don't think so as 6-tuple support is required in the detnet domain.

Hmm, okay.

> Do you have a specif recommendation?

How about:

  From an encapsulation perspective, the combination of the 6-tuple (the
  typical 5-tuple enhanced by the DSCP) and potentially the flow label
  uniquely identifies a DetNet IP flow.

> specific than the architecture document.

Okay.

> confused on if the field is required.

How would you feel about "and optionally also by the flow label"?
(I'm just thinking about ways to avoid having to say "optional field" and
the name of that field in close succession.)

> > DetNet service for aggregated flows, as recommended in Section 4.4?

Okay.

> semantics.  We would expect each to cover such unambiguously.

I think Alvaro made the same point as me in a more direct manner, and this
got updated to be " An implementation MUST support ordering of the set of
information information used to identify an individual DetNet flow."  I had
to read that a couple times to understand how it provided the property I
was expecting (i.e., that there is an external source of order and the
implementation has to cope with whatever such order it gets), but it seems
like it will do the job.
Or, hmm, was that down in section 6 and not here?

> (Alvaro may be happier with this change too)

That looks much better, thanks!
(Though didn't we say we would also allow zero values based on ...
Éric(?)'s review?)

> Yes -- although this is a consideration below IP - e.g., addressed via 

It might still be worth noting the dependency in these cases, since we do
mention the possibility of such configurations.

> differ4ent than an IP flow existing on a network without any detnet state

Okay, thanks.

> to be dropped at the domain boundary.

Based on ... physical interface?  I feel like in the general case you can
spoof a packet that is indistinguishable from a legitimate one, if you're
in the right part of the local topology to pass any BCP38-style checks, so
presumably there's some extra context that I'm missing.

> with other security minded folks, so would prefer to leave it unless you 

I don't strongly object (and it may be partially my fault anyway); just
noting it in case it was easy to change.
>          data plane, the integrity and confidentiality of application 

Sure, that does the trick.

> Do you have a specific text proposal?

Just tossing some ideas out there, maybe:

The DetNet quality assurance goals of extremely low packet loss and bounded
end-to-end latency are not achievable in the face of a highly capable
adversary, such as the one envisioned by the Internet Threat Model of BCP
72 that can arbitrarily drop or delay any or all traffic.  In order to
present meaningful security considerations, we consider a somewhat weaker
attacker who does not control the physical links of the DetNet domain, but
may have the ability to control a network node within the boundary of the
DetNet domain.

> yes, but it's an informational document so didn't want the downref.

[Alvaro already corrected this so I will not repeat it]

> https://xml2rfc.tools.ietf.org/cgi-bin/xml2rfc.cgi?url=https://raw.githubusercontent.com/detnet-wg/data-plane-drafts/working/lb/iesg-0625/ip/draft-ietf-detnet-ip.xml

Thanks for the explanations and updates!

-Ben

---

From: Lou Berger <[email-masked]>
Date: Fri, 26 Jun 2020 10:14:55 -0400

Hi Benjamin,

See below.

> format is actually different.

Done in two places.

>    uniquely identifies a DetNet IP flow.

Slight tweak:

s/potentially/optionally

> the name of that field in close succession.)
works for me.
> mention the possibility of such configurations.

okay, added "or within an edge IEEE802.1 TSN domain" to:

To protect against DOS attacks, excess traffic due to
         malicious or malfunctioning devices can be prevented or mitigated,
         for example through the use of existing mechanism such as 
policing and shaping applied at
         the input of a DetNet domain or within an edge IEEE802.1 TSN 
domain.

> may have the ability to control a network node within the boundary of the

The second paragraph now reads:

        Security aspects which are unique to DetNet are those whose aim 
is to
        provide the specific quality of service aspects of DetNet, which are
        primarily to deliver data flows with extremely low packet loss rates
        and bounded end-to-end delivery latency.
        Achieving such loss rates and bounded latency may not be possible
        in the face of a highly capable adversary, such as the one
        envisioned by the Internet Threat Model of BCP 72 that can
        arbitrarily drop or delay any or all traffic.  In order to
        present meaningful security considerations, we consider a
        somewhat weaker attacker who does not control the physical links
        of the DetNet domain, but may have the ability to control a
        network node within the boundary of the DetNet domain.
       </t>

> Thanks for the explanations and updates!

Thank you for the comments and helpful text suggestions!

Lou

PS Repo has been updated...