Subject: Re: [Detnet] DetNet Use Case Statements: Link Aggregation

From: "Norman Finn (nfinn)" <[email-masked]>
Date: Tue, 12 Apr 2016 21:25:28 +0000

Ethan,

Don’t lose this use case.  Whether it is a problem for DetNet is an open question.  What your problem and the “Packet Replication and Deletion (PRD)” (or Seamless Redundancy) have in common is that they can both be solved by a sequence number in every packet.  But, there are differences.

PRD:    Duplicate every packet in a single stream.  Send on two paths.  Delete the duplicates.  Sequence numbers are essential to identify duplicate packets.  Restoring packet order is an option, but many are reluctant to make this an objective.

Yours:  Split a single stream, sending some packets on one path, and other packets on another path.  Bring them back together.  Sequence numbers are needed only if restoring packet order.

Normally, today, when one has multiple links between two devices, one tries very hard to ensure that any given flow is all put on one of the links, in order to minimize out-of-order delivery.  (Using multiple links for one stream generally shuffles packets.)  Of course, this is exactly what you’re trying to fix.  Tell me — do the packets need to be delivered in order?

IMO, the problem you’re bringing up is important, but is only related to the DetNet problem by having a user (video studio) that needs solutions to both problems.

— Norm

From: detnet <[email-masked]<mailto:[email-masked]>> on behalf of "Grossman, Ethan A." <[email-masked]<mailto:[email-masked]>>
Date: Thursday, April 7, 2016 at 13:44 PM
To: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: [Detnet] DetNet Use Case Statements: Link Aggregation

Statement:
Link Aggregation
Discussion:
I used the term “link aggregation” without understanding that it is a reserved word in this context, so I need to figure out what the correct term is. The use case is that the user has a high bandwidth stream such as video but there is no single network link between talker and listener which has available bandwidth (for whatever reason) and the question is can DetNet “seamlessly” split the stream and send parts of it across different paths to the listener. It seemed like the feel of the room was that we shouldn’t be involved in doing this, however it was brought up that this function could be implemented in a similar way as packet replication and deletion, except that the packet sequence numbers are not duplicates so they would be accepted and it might “just work”. It sounds to me like it might be a useful thing to keep in there since it doesn’t seem like it would be that hard, but perhaps this is a DetNet “future feature” that we don’t have to address now as long as we don’t do anything that would preclude developing it later.

---

From: "Grossman, Ethan A." <[email-masked]>
Date: Tue, 12 Apr 2016 21:31:04 +0000

Thanks Norm,
Yes the idea is that the packets would be delivered to the end user in their original order, so sequence numbers would be required. I think of this as a “nice to have” feature, not a core requirement. Or maybe a “DetNet 2.0” feature. I would probably have just dropped it except for Pascal having noted the overlap with PRD which made it sound “not hard”.
Ethan.

From: Norman Finn (nfinn) [mailto:[email-masked]]
Sent: Tuesday, April 12, 2016 2:25 PM
To: Grossman, Ethan A.; [email-masked]
Subject: Re: [Detnet] DetNet Use Case Statements: Link Aggregation

Ethan,

— Norm

Statement:
Link Aggregation
Discussion:

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Tue, 12 Apr 2016 21:47:29 +0000

Hello Norm

This is also related to 1) low jitter, and 2) to network defragmentation.

Regarding 1) If the use case requires low jitter, then in-order delivery is, well, in order. The re-sequencing is a side effect of that need. Industrial control networks want that. In TSCH networks, different packets of a same flow will take different paths, not necessarily to distribute load but also to route around interference.

Then, about 2) being able to split a stream in micro streams is one cool way to avoid ending up with the fragmentation problem. This is the case where all the paths are used to some percentage such that an additional flow cannot fit, whereas the overall bandwidth is present. Defragmentation needs to take to allow the new flow in. This means moving existing flows and it hurts.

My read is that some detnet properties are more or less difficult to achieve depending on the technology and the position in the network. We can describe them all, but not necessarily expect them from all devices.

Regards,

Pascal

Le 12 avr. 2016 ? 14:25, Norman Finn (nfinn) <[email-masked]<mailto:[email-masked]>> a ?crit :

Ethan,

Don't lose this use case.  Whether it is a problem for DetNet is an open question.  What your problem and the "Packet Replication and Deletion (PRD)" (or Seamless Redundancy) have in common is that they can both be solved by a sequence number in every packet.  But, there are differences.

Normally, today, when one has multiple links between two devices, one tries very hard to ensure that any given flow is all put on one of the links, in order to minimize out-of-order delivery.  (Using multiple links for one stream generally shuffles packets.)  Of course, this is exactly what you're trying to fix.  Tell me - do the packets need to be delivered in order?

IMO, the problem you're bringing up is important, but is only related to the DetNet problem by having a user (video studio) that needs solutions to both problems.

- Norm

Statement:
Link Aggregation
Discussion:
I used the term "link aggregation" without understanding that it is a reserved word in this context, so I need to figure out what the correct term is. The use case is that the user has a high bandwidth stream such as video but there is no single network link between talker and listener which has available bandwidth (for whatever reason) and the question is can DetNet "seamlessly" split the stream and send parts of it across different paths to the listener. It seemed like the feel of the room was that we shouldn't be involved in doing this, however it was brought up that this function could be implemented in a similar way as packet replication and deletion, except that the packet sequence numbers are not duplicates so they would be accepted and it might "just work". It sounds to me like it might be a useful thing to keep in there since it doesn't seem like it would be that hard, but perhaps this is a DetNet "future feature" that we don't have to address now as long as we don't do anything that would preclude developing it later.

_______________________________________________
detnet mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/detnet

---

From: Balázs Varga A <[email-masked]>
Date: Mon, 18 Apr 2016 13:55:47 +0000

Hi,

Some more aspects related to PRD (Packet Replication and Deletion) and LA (Link Aggregation) for the "in-order delivery" context.
I am sure there are scenarios where in-order delivery is a must, but not for all the scenarios.

Let's call as "merging node", where PRD or LA merges the detnet packets arriving over different paths. Let's assume that the
merging node is somewhere in the middle of the detnet network. The same function can be used for both PRD and LA to ensure
the correct packet order. The different paths from the source to the merging node have different delays and jitters.
The path with the longest delay will determine packet delivery.

            B
           / \
 Source--A   D--E--Sink
           \ /
            C

NodeA: replication point (PRD) or distribution point (LA)
NodeD: merging node
pathA-B-D: fast path
pathA-C-D: slow path
Let's assume we lost a packet over path A-B-D and analyze the impact.

Adding a function (to "D") to ensure in-order delivery may have some negative impacts:
- it needs more resources: on the merging node as you need buffering.
- it can increase delay: if on nodeD we have sent packetN-1 to nodeE than we expect to receive packetN. However
if  we receive packetN+1, than we have to wait until packetN is received. If packetN is lost than we may have to wait
until we are sure it is lost (determine waiting time depends on traffic characteristics and network scenario, just as an
example "waiting-time=maximum delay differences of used paths from source-to-nodeD"). Therefore forwarding
of packetN+1 could be delayed.
- it can increase flow burstyness: Due to the additional waiting of packetN+1 (and maybe also N+2, etc.) burstyness
and jitter is increased.
- it can increase loss: if we do not wait long enough (e.g., miscalculation) and have sent already packetN+1 towards
nodeE when packetN arrives, than it will be dropped otherwise out-of-order delivery happens.

Therefore from networking perspective the optimal place to ensure correct packet order is at the endpoint (sink)
of the detnet flow. If the endpoint can handle out-of-order delivery, than we do not need an "put-in-order" function
and detnet node implementations become more simple. There are also some network scenarios what makes
in-order-delivery and the related nodal function obsolete (e.g. CBR source with 1packet/sec and a network with
100msec max latency).

So I agree that we should target in-order delivery as a requirement, but I would argue to have it as an optional requirement
(or DetNet 2.0 as formulated by Ethan).

Cheers
Bala'zs

From: detnet [mailto:[email-masked]] On Behalf Of Pascal Thubert (pthubert)
Sent: Tuesday, April 12, 2016 11:47 PM
To: Norman Finn (nfinn)
Cc: Grossman, Ethan A.; [email-masked]
Subject: Re: [Detnet] DetNet Use Case Statements: Link Aggregation

Hello Norm

Regards,

Pascal

Le 12 avr. 2016 à 14:25, Norman Finn (nfinn) <[email-masked]<mailto:[email-masked]>> a écrit :
Ethan,

- Norm

Statement:
Link Aggregation
Discussion: