Subject: [Detnet] Last Call: <draft-ietf-detnet-architecture-08.txt> (Deterministic Networking Architecture) to Proposed Standard

From: The IESG <[email-masked]>
Date: Wed, 19 Sep 2018 13:07:56 -0700

The IESG has received a request from the Deterministic Networking WG (detnet)
to consider the following document: - 'Deterministic Networking Architecture'
  <draft-ietf-detnet-architecture-08.txt> as Proposed Standard

The IESG plans to make a decision in the next few weeks, and solicits final
comments on this action. Please send substantive comments to the
[email-masked] mailing lists by 2018-10-03. Exceptionally, comments may be
sent to [email-masked] instead. In either case, please retain the beginning of
the Subject line to allow automated sorting.

Abstract

   This document provides the overall Architecture for Deterministic
   Networking (DetNet), which provides a capability to carry specified
   unicast or multicast data flows for real-time applications with
   extremely low data loss rates and bounded latency.  Techniques used
   include: 1) reserving data plane resources for individual (or
   aggregated) DetNet flows in some or all of the intermediate nodes
   (e.g., bridges or routers) along the path of the flow; 2) providing
   explicit routes for DetNet flows that do not immediately change with
   the network topology; and 3) distributing data from DetNet flow
   packets over time and/or space to ensure delivery of each packet's
   data in spite of the loss of a path.  DetNet operates at the IP layer
   and delivers service over sub-network technologies such as MPLS and
   IEEE 802.1 TSN.

The file can be obtained via
https://datatracker.ietf.org/doc/draft-ietf-detnet-architecture/

IESG discussion can be tracked via
https://datatracker.ietf.org/doc/draft-ietf-detnet-architecture/ballot/

The following IPR Declarations may be related to this I-D:

   https://datatracker.ietf.org/ipr/2710/

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 24 Sep 2018 16:56:08 +1200

Hi,

The term "Flow-ID" is used frequently but is not defined or explained.
Section 4.7 "Flow identification at technology borders" doesn't really
help on this. 

In fact, it isn't entirely clear what the word "flow" itself means.
You might think it's obvious, but when designing the rules for the
IPv6 Flow Label we stumbled until we got to this:

   From the viewpoint of the network layer, a flow is a sequence of
   packets sent from a particular source to a particular unicast,
   anycast, or multicast destination that a node desires to label as a
   flow. 

In other words, a flow is whatever set of packets the source chooses
to assert is a flow. Is that the DetNet definition too?

Also:
> The Flow-ID must be unique inside a given domain.

It isn't clear from the discussion of Flow-ID mapping whether
this implies that the value of the upper layer (the deepest
encapsulated) Flow-ID is is intended to be preserved end-to-end.
It appears so from the examples in Figure 10 and Figure 11, but
based on the experience that this has been a contentious point
for the IPv6 Flow Label, I recommend that this should be clearly
defined in the architecture.

> 3.2.1.1.  "Eliminate congestion loss"

Shouldn't there be some discussion here about buffer bloat,
which serves to reduce congestion loss but does terrible
things to latency? The discussion at the beginning of
Section 4.5 "Queuing, Shaping, Scheduling, and Preemption"
doesn't make it clear how buffer bloat is avoided at large
scale, where many DetNet flows are going through the same
choke point. Having reserved buffers per flow doesn't prevent
the total buffer space creeping up, and clever queueing algorithms
can't completely save you when that happens.

>    Security must cover:
> 
>    o  the protection of the signaling protocol
> 
>    o  the authentication and authorization of the controlling systems
> 
>    o  the identification and shaping of the DetNet flows

What about the identification and authorization of the nodes
allowed to generate DetNet flows within a domain, and
of the border nodes at the edges of the domain?

Regards
   Brian Carpenter

---

From: Norman Finn <[email-masked]>
Date: Mon, 24 Sep 2018 05:56:01 +0000

Thanks for the comments, Brian.  I'll try to respond; others will have opinions, I'm sure.  See <NWF>
________________________________________
From: Brian E Carpenter [[email-masked]]
Sent: Sunday, September 23, 2018 9:56 PM
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Re: Last Call: <draft-ietf-detnet-architecture-08.txt> (Deterministic Networking Architecture) to Proposed Standard

Hi,

<NWF>  In 2.1 we have, "A DetNet flow is a sequence of packets to
 which the DetNet service is to be provided."  This could be expanded to, "A DetNet flow is
a sequence of packets from one source to one or more destinations, which conform uniquely
to a flow identifier, and to which the DetNet service is to be provided."  The idea is that
the process of creating a reservation for a DetNet flow always provides a means for
unambiguously identifying to which DetNet flow, if any, a packet belongs.

<NWF> A better way to phrase the uniqueness requirement is that any given
relay node must be able to unambiguously assign to a packet the resources
reserved for that packet's DetNet flow.  That's easiest if the identifiers are
unique over domain, of course.  I'd like to hear from others, on this.

<NWF> As we are trying to make more clear in the bounded-latency draft, one
of the characteristics of any queuing strategy that is useful for DetNet is that
one can compute no later than flow reservation time how much buffer space is
required to guarantee zero congestion loss.  If you can't, then the queuing strategy
is unsuitable.  Different strategies make different trade-offs regarding buffer space,
latency, data plane and control plane complexity, etc.  Generally speaking, buffer
space will grow with the number of flows visible to a given relay node.  Flow
aggregation is helpful, here, but one can expect a DetNet relay node to require
more buffer space than a non-DetNet node.  And, if you calculate that you haven't
enough buffers to ensure zero congestion loss, you refuse the reservation.

<NWF> I'll let others comment on that one.  I'm not clear about the boundaries
and the amount of overlap between network security and application security,
in this case.

Regards
   Brian Carpenter

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Mon, 24 Sep 2018 06:27:20 +0000

> What about the identification and authorization of the nodes allowed to
> generate DetNet flows within a domain, and of the border nodes at the edges
> of the domain?
> 
> <NWF> I'll let others comment on that one.  I'm not clear about the
> boundaries and the amount of overlap between network security and
> application security, in this case.

I agree with Brian; the source of the flow must be authenticated. I believe that a secure identification of the flow involves that, but it could make sense to emphasize this.
What about:

> >    o  the identification and shaping of the DetNet flows, including authorization of the packet source

Note: we have imposed that a flow has a single source and one or mane end points. There are MP2MP use cases in industrial. Unsure how to fully characterize them but the most basic methods are source A or source B, and source A and source B. The latter can be reserved as 2 independent DetNet flows, but the former needs methods that we do not have.

Cheers;

Pascal

> -----Original Message-----
> From: Norman Finn <[email-masked]>
> Sent: lundi 24 septembre 2018 07:56
> To: Brian E Carpenter <[email-masked]>
> Subject: RE: Last Call: <draft-ietf-detnet-architecture-08.txt> (Deterministic
> Networking Architecture) to Proposed Standard
> 
> Thanks for the comments, Brian.  I'll try to respond; others will have opinions,
> I'm sure.  See <NWF> ________________________________________
> Subject: Re: Last Call: <draft-ietf-detnet-architecture-08.txt> (Deterministic
> Networking Architecture) to Proposed Standard
> 
> Hi,
> 
> Section 4.7 "Flow identification at technology borders" doesn't really help on
> this.
> 
> In other words, a flow is whatever set of packets the source chooses to assert
> is a flow. Is that the DetNet definition too?
> 
> <NWF>  In 2.1 we have, "A DetNet flow is a sequence of packets to  which the
> DetNet service is to be provided."  This could be expanded to, "A DetNet flow
> is a sequence of packets from one source to one or more destinations, which
> conform uniquely to a flow identifier, and to which the DetNet service is to be
> provided."  The idea is that the process of creating a reservation for a DetNet
> flow always provides a means for unambiguously identifying to which DetNet
> flow, if any, a packet belongs.
> 
> Also:
> It isn't clear from the discussion of Flow-ID mapping whether this implies that
> the value of the upper layer (the deepest
> It appears so from the examples in Figure 10 and Figure 11, but based on the
> experience that this has been a contentious point for the IPv6 Flow Label, I
> recommend that this should be clearly defined in the architecture.
> 
> Shouldn't there be some discussion here about buffer bloat, which serves to
> reduce congestion loss but does terrible things to latency? The discussion at
> the beginning of Section 4.5 "Queuing, Shaping, Scheduling, and Preemption"
> doesn't make it clear how buffer bloat is avoided at large scale, where many
> DetNet flows are going through the same choke point. Having reserved buffers
> per flow doesn't prevent the total buffer space creeping up, and clever
> queueing algorithms can't completely save you when that happens.
> 
> one can compute no later than flow reservation time how much buffer space
> is required to guarantee zero congestion loss.  If you can't, then the queuing
> strategy is unsuitable.  Different strategies make different trade-offs regarding
> buffer space, latency, data plane and control plane complexity, etc.  Generally
> speaking, buffer space will grow with the number of flows visible to a given
> relay node.  Flow aggregation is helpful, here, but one can expect a DetNet
> relay node to require more buffer space than a non-DetNet node.  And, if you
> calculate that you haven't enough buffers to ensure zero congestion loss, you
> refuse the reservation.
> 
> >    Security must cover:
> >
> What about the identification and authorization of the nodes allowed to
> generate DetNet flows within a domain, and of the border nodes at the edges
> of the domain?
> 
> <NWF> I'll let others comment on that one.  I'm not clear about the
> boundaries and the amount of overlap between network security and
> application security, in this case.
> 
> Regards
>    Brian Carpenter

---

From: Balázs Varga A <[email-masked]>
Date: Tue, 25 Sep 2018 09:09:41 +0000

Hi,

> It isn't clear from the discussion of Flow-ID mapping whether this implies that the value of the upper layer (the deepest
> It appears so from the examples in Figure 10 and Figure 11, but based on the experience that this has been a contentious 
> point for the IPv6 Flow Label, I recommend that this should be clearly defined in the architecture.
>
> <NWF> A better way to phrase the uniqueness requirement is that any given relay node must be able to unambiguously 
> assign to a packet the resources reserved for that packet's DetNet flow.  That's easiest if the identifiers are unique over 
> domain, of course.  I'd like to hear from others, on this.

Agree with Norm comment, we simple intend to ensure that flows are identified correctly. A flow may cross several technology 
domains, and within a domain it must be identified correctly. "The upper layer (deepest encapsulated) Flow-ID" may not be
visible in a given technology domain. 

Cheers
Bala'zs

-----Original Message-----
From: detnet <[email-masked]> On Behalf Of Norman Finn
Sent: 2018. szeptember 24. 7:56
To: Brian E Carpenter <[email-masked]>
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Re: [Detnet] Last Call: <draft-ietf-detnet-architecture-08.txt> (Deterministic Networking Architecture) to Proposed Standard

Thanks for the comments, Brian.  I'll try to respond; others will have opinions, I'm sure.  See <NWF> ________________________________________

Hi,

The term "Flow-ID" is used frequently but is not defined or explained.
Section 4.7 "Flow identification at technology borders" doesn't really help on this.

In other words, a flow is whatever set of packets the source chooses to assert is a flow. Is that the DetNet definition too? 

<NWF>  In 2.1 we have, "A DetNet flow is a sequence of packets to  which the DetNet service is to be provided."  This could be expanded to, "A DetNet flow is a sequence of packets from one source to one or more destinations, which conform uniquely to a flow identifier, and to which the DetNet service is to be provided."  The idea is that the process of creating a reservation for a DetNet flow always provides a means for unambiguously identifying to which DetNet flow, if any, a packet belongs.

> The Flow-ID must be unique inside a given domain.

It isn't clear from the discussion of Flow-ID mapping whether this implies that the value of the upper layer (the deepest
encapsulated) Flow-ID is is intended to be preserved end-to-end.
It appears so from the examples in Figure 10 and Figure 11, but based on the experience that this has been a contentious point for the IPv6 Flow Label, I recommend that this should be clearly defined in the architecture.

<NWF> A better way to phrase the uniqueness requirement is that any given relay node must be able to unambiguously assign to a packet the resources reserved for that packet's DetNet flow.  That's easiest if the identifiers are unique over domain, of course.  I'd like to hear from others, on this.

> 3.2.1.1.  "Eliminate congestion loss"

Shouldn't there be some discussion here about buffer bloat, which serves to reduce congestion loss but does terrible things to latency? The discussion at the beginning of Section 4.5 "Queuing, Shaping, Scheduling, and Preemption"
doesn't make it clear how buffer bloat is avoided at large scale, where many DetNet flows are going through the same choke point. Having reserved buffers per flow doesn't prevent the total buffer space creeping up, and clever queueing algorithms can't completely save you when that happens.

<NWF> As we are trying to make more clear in the bounded-latency draft, one of the characteristics of any queuing strategy that is useful for DetNet is that one can compute no later than flow reservation time how much buffer space is required to guarantee zero congestion loss.  If you can't, then the queuing strategy is unsuitable.  Different strategies make different trade-offs regarding buffer space, latency, data plane and control plane complexity, etc.  Generally speaking, buffer space will grow with the number of flows visible to a given relay node.  Flow aggregation is helpful, here, but one can expect a DetNet relay node to require more buffer space than a non-DetNet node.  And, if you calculate that you haven't enough buffers to ensure zero congestion loss, you refuse the reservation.

>    o  the identification and shaping of the DetNet flows

What about the identification and authorization of the nodes allowed to generate DetNet flows within a domain, and of the border nodes at the edges of the domain?

<NWF> I'll let others comment on that one.  I'm not clear about the boundaries and the amount of overlap between network security and application security, in this case.

Regards
   Brian Carpenter

_______________________________________________
detnet mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/detnet

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Tue, 25 Sep 2018 09:24:03 +0000

Hello Balázs:

A DetNet track is a constrained collection of pipes with a complex overall shape aimed at providing a very precise service. Any packet injection means at least a DOS since the shapers may discard another packet instead in order to provide the required service. Since DetNet makes a decision to drop a packet, it is also DetNet 's responsibility to select the appropriate packet to discard and in particular avoid this DOS.

IOW, it MUST be possible at each hop to validate that what is found in the track is appropriate, and I agree with the IESG review comments that this should be mentioned in the Architecture Security Section. 

This means:
* On the one hand, the DetNet ingress MUST validate the authorization of the source to post in the DetNet track before injecting the packet in the track.
* On the other hand, a DetNet relay MUST ensure that the packet is appropriate in the track. This can be achieved for instance by validating the source authorization, or indirectly if the ingress has tagged the packet in a way that the relay can validate.

I do hope it makes sense;

Pascal

> From: Balázs Varga A <[email-masked]>
> Sent: mardi 25 septembre 2018 11:10
> To: Norman Finn <[email-masked]>; Brian E Carpenter
> <[email-masked]>
> > It isn't clear from the discussion of Flow-ID mapping whether this
> > implies that the value of the upper layer (the deepest
> > It appears so from the examples in Figure 10 and Figure 11, but based
> > on the experience that this has been a contentious point for the IPv6 Flow
> Label, I recommend that this should be clearly defined in the architecture.
> >
> > <NWF> A better way to phrase the uniqueness requirement is that any
> > given relay node must be able to unambiguously assign to a packet the
> > resources reserved for that packet's DetNet flow.  That's easiest if the
> identifiers are unique over domain, of course.  I'd like to hear from others, on
> this.
> 
> Agree with Norm comment, we simple intend to ensure that flows are
> identified correctly. A flow may cross several technology domains, and within
> a domain it must be identified correctly. "The upper layer (deepest
> encapsulated) Flow-ID" may not be visible in a given technology domain.
> 
> Cheers
> Bala'zs
> 
> 
> 
> Subject: Re: [Detnet] Last Call: <draft-ietf-detnet-architecture-08.txt>
> (Deterministic Networking Architecture) to Proposed Standard
>