Subject: [LOOPS] draft-li-tsvwg-loops-problem-opportunities and use-cases.

From: Gorry Fairhurst <[email-masked]>
Date: Mon, 27 Jul 2020 16:21:35 +0100

Thank you for the scenarios update in 
draft-li-tsvwg-loops-problem-opportunities, I see you have updated the 
text relating to satellite network segments. I now have some more 
questions on scenarios based on the what is said in the most recent ID:

(1) The text says:

"By continuously measuring the
    delay of path segments and use them as metrics for path selection,
    when the number of overlay nodes is sufficiently large, there is a
    high chance that a better path could be found
    [DOI_10.1109_ICDCS.2016.49] [DOI_10.1145_3038912.3052560].
    [DOI_10.1145_3038912.3052560] further shows all cloud providers
   experience random loss episodes and random loss accounts for more
    than 35% of total loss."

- I think this use case was presented in previuous meetings, and one of 
the questions was how to react appropiately to route changes and traffic 
engineering by the physical network operator. This doesn't come across 
to me in the latest text.

(2) I will argue there is not a satellite-specific use case really, 
there is a use case for any reliable long-haul path (large RTT) that is 
concatenated with another segment with a lossy segment with lower RTT.

  “Enhanced error recovery at the satellite link layer
    helps for the loss on the satellite link but doesn't help for the
    terrestrial links.  Even when the terrestrial segments are short, any
    loss must be recovered across the satellite link delay.”
-  I think the argument is really that the end-to-end internet path 
using a satellite network segment can be long (large RTT), and therefore 
that loss experienced at any point outside the satellite network segment 
still suffers performance because of the larger RTT.
- If that is the case, then I think a GEO satellite path is an 
**example** of such a path.
is this a “long terrestrial” path? or a long Internet path?
- “Faster recovery over such
    long terrestrial segments is desirable.”

(3) The previous revision mentioned the specific case where the “network 
is long-haul”, but I think this aspect was removed in the latest rev?. 
Maybe I don’t fully see - is it expected (or somehow known) that the 
delay for the LOOPS path segment is much less than the end-to-end path 
(in Section 1)? How does the LOOPS path segment know this?

(4) I see you also mention in the latest rev adds a use case for a 
"fleet of drones" - I'm not quite sure what this is describing, maybe 
you could expand how this works, and add some text to the draft.

Gorry

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Tue, 28 Jul 2020 13:10:06 -0500

Speaking as an individual,

> concatenated with another segment with a lossy segment with lower RTT.

I've probably been using "satellite" as a mental shorthand for "long
haul/large RTT" without meaning to, for years (plus Gorry's mention of
another path segment with shorter RTT and losses). I think that's an
important thing to remember when we're talking about use cases - we need to
focus on the problematic aspects of the path, rather than focusing on why a
specific path has those problems, in order to identify the problems we're
trying to solve.

There are many more kinds of problematic paths than kinds of problems with
paths :-)

Thanks, Gorry, for the comment.

Best,

Spencer

> --
> LOOPS mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/loops
>

---

From: <[email-masked]>
Date: Wed, 29 Jul 2020 06:14:29 +0000

Hi Gorry, 

I guess you are referring to this text: 

   Similar issues and optimizations
   may be helpful in other deployments such as enhancing the reliability
   of data transfer when a fleet of drones are used for specific
   missions (e.g., site inspection, live streaming, and emergency
   service).  Captured data should be reliably transmitted via paths
   involving radio connections.

Actually, this is an old text. The main point we wanted to convey here is that when a swarm of drones is collaborating for example to provide connectivity, multiple drones (seen ad as forwarders) will be solicited till packets are (hopefully) delivered to a network where the fleet of drones is grafted (can be a 3GPP core network). The idea we had in mind is that LOOPS can be enabled in such mutli-hop wireless network (composed of drones) for the sake of performance optimization. 

Cheers,
Med

> -----Message d'origine-----
> De : LOOPS [mailto:[email-masked]] De la part de Gorry
> Fairhurst
> Envoyé : lundi 27 juillet 2020 17:22
> À : loops <[email-masked]>
> Objet : [LOOPS] draft-li-tsvwg-loops-problem-opportunities and use-
> cases.
> 

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: Gorry Fairhurst <[email-masked]>
Date: Wed, 29 Jul 2020 12:25:00 +0100

Hi med,

> Actually, this is an old text. The main point we wanted to convey here is that when a swarm of drones is collaborating for example to provide connectivity, multiple drones (seen ad as forwarders) w…

So this use case probably needs more text. I think you're saying drones 
are an example of a network with continuously changing paths. To me, 
mitigating the impacts of handover likely does expose some interesting 
constraints on the designs for LOOPS as well as on the performance.  I 
haven't though about what that means wrt to using ECN and path information.

Gorry

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 29 Jul 2020 23:52:01 +0200

Hi Gorry,

thank you for these comments.

One of the shortcomings that this document has is that, in describing scenarios, it does not always make perfectly clear which part is solved by LOOPS and which parts need to be solved by other means so the scenario works and LOOPS is effective.

On 2020-07-27, at 17:21, Gorry Fairhurst <[email-masked]> wrote:
> 
> Thank you for the scenarios update in draft-li-tsvwg-loops-problem-opportunities, I see you have updated the text relating to satellite network segments. I now have some more questions on scenarios based on the what is said in the most recent ID:
> 
> (1) The text says:
> 
> - I think this use case was presented in previuous meetings, and one of the questions was how to react appropiately to route changes and traffic engineering by the physical network operator. This doesn't come across to me in the latest text.

The document is not explicit about that, but path selection is not something LOOPS would be doing.  The SDN controller that sets up these paths would add LOOPS support to those segments that need it (which is what the last sentence is about: These segments exist!).  LOOPS is really just about loss recovery.  Some of the measurements that LOOPS carries out will be useful for the SDN controller to make path selection decisions; we would expect a management interface will make these measurements available to the SDN controller.

> (2) I will argue there is not a satellite-specific use case really, there is a use case for any reliable long-haul path (large RTT) that is concatenated with another segment with a lossy segment with lower RTT.

Yes!
(For a definition of “reliable”.)

> -  I think the argument is really that the end-to-end internet path using a satellite network segment can be long (large RTT), and therefore that loss experienced at any point outside the satellite network segment still suffers performance because of the larger RTT.
> - If that is the case, then I think a GEO satellite path is an **example** of such a path.

A long Internet path (that is still not at the level of RTT a GEO sat gives you.)

> (3) The previous revision mentioned the specific case where the “network is long-haul”, but I think this aspect was removed in the latest rev?. Maybe I don’t fully see - is it expected (or somehow known) that the delay for the LOOPS path segment is much less than the end-to-end path (in Section 1)?

That is the configuration where you get the most benefit.

> How does the LOOPS path segment know this?

It can’t.  The aggregate that LOOPS is repairing may be mixed out of quite different end-to-end paths.  LOOPS does not have a way to measure the RTT of those paths (*).  A recommendation for the proper use of LOOPS would be that it is not very useful to run LOOPS on a segment that covers most of the span of the end-to-end path, but LOOPS cannot know that this was done, and it is not going to dramatically break traffic that has such a mix.  If LOOPS does little good, due to its cost (additional reverse packets, processing/memory, tunneling if that wasn’t already in use) it is a pessimization; many optimizations have regions of operation in which they don't.

> (4) I see you also mention in the latest rev adds a use case for a "fleet of drones" - I'm not quite sure what this is describing, maybe you could expand how this works, and add some text to the draft.

I think Med answered that — I don’t think that is a use case the operators interested in this have today :-)

Grüße, Carsten

(*) Well, vendors could add some secret sauce to actually do this, for certain cases.  But it is not the LOOPS protocol that will do that for them, and LOOPS does not depend on it.

---

From: Gorry Fairhurst <[email-masked]>
Date: Thu, 30 Jul 2020 09:24:38 +0100

Hi Carston,

> One of the shortcomings that this document has is that, in describing scenarios, it does not always make perfectly clear which part is solved by LOOPS and which parts need to be solved by other mea…

I think this is exactly my feeling, and to me this is important to 
understand any WG BoF. It helps to be able to separate specific 
applicability for general use and focussed on specific problems; and 
separate the proposed technology, from dependent technology and 
separable technologies.

> The document is not explicit about that, but path selection is not something LOOPS would be doing.  The SDN controller that sets up these paths would add LOOPS support to those segments that need it (which is what the last sentence is about: These segments exist!).
See below [GF].
> LOOPS is really just about loss recovery.  Some of the measurements that LOOPS carries out will be useful for the SDN controller to make path selection decisions; we would expect a management interface will make these measurements available to the SDN controller.
>
OK.
> (For a definition of “reliable”.)
OK.
> That is the configuration where you get the most benefit.
And the converse appears also the case where LOOPS has the potential for 
most negative interactions with end-to-end recovery?
> It can’t.
That's an issue.
> The aggregate that LOOPS is repairing may be mixed out of quite different end-to-end paths.
See (a).
> LOOPS does not have a way to measure the RTT of those paths (*).  A recommendation for the proper use of LOOPS would be that it is not very useful to run LOOPS on a segment that covers most of the span of the end-to-end path, but LOOPS cannot know that this was done, and it is not going to dramatically break traffic that has such a mix.  If LOOPS does little good, due to its cost (additional reverse packets, processing/memory, tunneling if that wasn’t already in use) it is a pessimization; many optimizations have regions of operation in which they don't.
I would like to understand the potential for harm... to.
> I think Med answered that — I don’t think that is a use case the operators interested in this have today :-)
That thread was interesting, but it seemed to create additional issues - 
it might be worth deciding whether that is included in the next rev of 
the document.
> (*) Well, vendors could add some secret sauce to actually do this, for certain cases.  But it is not the LOOPS protocol that will do that for them, and LOOPS does not depend on it.

:-).

[GF] I'm now trying to understand how the "SDN" controller can know 
which flows in the aggregates support ECN:  At what point can an 
aggregated tunnel egress assume that it is safe to translate a tunnel 
loss into a flow ECN CE-mark?

Example: (Would only established ECN-capable flows added to the 
aggregate? ... Translating loss at the start of a flow into a CE-mark 
has a strange effect when the sender is probing for support, but a 
recipient doesn't report CE-marks?)

Gorry

---

From: <[email-masked]>
Date: Thu, 30 Jul 2020 09:06:26 +0000

Hi Gorry, 

Please see inline.

Cheers,
Med

> De : Gorry Fairhurst [mailto:[email-masked]]
> Envoyé : mercredi 29 juillet 2020 13:25
> À : BOUCADAIR Mohamed TGI/OLN <[email-masked]>; loops
> <[email-masked]>
> Objet : Re: [LOOPS] draft-li-tsvwg-loops-problem-opportunities and
> use-cases.
> 
> Hi med,
> 
> > Actually, this is an old text. The main point we wanted to convey
> here is that when a swarm of drones is collaborating for example to
> provide connectivity, multiple drones (seen ad as forwarders) will be
> solicited till packets are (hopefully) delivered to a network where
> the fleet of drones is grafted (can be a 3GPP core network). The idea
> we had in mind is that LOOPS can be enabled in such mutli-hop wireless
> network (composed of drones) for the sake of performance optimization.
> >
> > Cheers,
> > Med
> 
> So this use case probably needs more text. I think you're saying
> drones are an example of a network with continuously changing paths.

[Med] Yes...and a particularity of that case is that the performance of the sub-paths is not known beforehand (because the paths themselves are not known nor the "physical" environment where these paths are established. 

> To me, mitigating the impacts of handover likely does expose some
> interesting constraints on the designs for LOOPS as well as on the
> performance. 

[Med] Fully agree. Aligned with the scope set for the initial LOOPS work, we can remove that part of the draft but set it as a future perspective if LOOPS proves to be beneficial for "stable" topologies.   

 I haven't though about what that means wrt to using ECN
> and path information.
> >> Fairhurst Envoyé : lundi 27 juillet 2020 17:22 À : loops
> >> <[email-masked]> Objet : [LOOPS]
> >> draft-li-tsvwg-loops-problem-opportunities and use- cases.
> >>
> >> draft-li-tsvwg-loops-problem-opportunities, I see you have updated
> >> the text relating to satellite network segments. I now have some
> more
> ______________________________________________________________________
> > ___________________________________________________
> >
> > Ce message et ses pieces jointes peuvent contenir des informations
> > confidentielles ou privilegiees et ne doivent donc pas etre
> diffuses,
> > exploites ou copies sans autorisation. Si vous avez recu ce message
> > par erreur, veuillez le signaler a l'expediteur et le detruire ainsi
> que les pieces jointes. Les messages electroniques etant susceptibles
> d'alteration, Orange decline toute responsabilite si ce message a ete
> altere, deforme ou falsifie. Merci.
> >
> > This message and its attachments may contain confidential or
> > privileged information that may be protected by law; they should not
> be distributed, used or copied without authorisation.
> > If you have received this email in error, please notify the sender
> and delete this message and its attachments.
> > As emails may be altered, Orange is not liable for messages that
> have been modified, changed or falsified.
> > Thank you.
> >

---

From: Jeremy Harris <[email-masked]>
Date: Thu, 30 Jul 2020 14:50:46 +0100

> (For a definition of “reliable”.)

Seems you'd do well, for that scenario, to have a separate LOOPS-like
reliability segment for the lower-RTT but lossy segment - to better
respond given the segment RTT being lower than the full path RTT?

But we're regressing to the 1950's era of separate link-reliability
mechanisms, and a sequence of such links.  Something the e2e principle
discarded very successfully.  Are we really better off, provably?
-- 
Cheers,
  Jeremy

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 30 Jul 2020 17:16:39 +0200

Hi Jeremy,

> On 2020-07-30, at 15:50, Jeremy Harris <[email-masked]> wrote:
> 

Yes.

> discarded very successfully.  Are we really better off, provably?

I do not think there is a conflict with the e2e argument here (*).
LOOPS is not replacing TCP/QUIC/etc., it is assisting the e2e protocol.
All links that do not have ridiculous signal/noise budgets have some form of error recovery; that may just not very visible or well known (e.g., Trellis Coding for Gigabit Ethernet).  In theory, that would always be working well and be enough so the rest can be done in the e2e protocol.  In practice, there is a difference between theory and practice.

Grüße, Carsten

(*) Please see Section 2.3 of https://dl.acm.org/doi/10.1145/357401.357402 if you need an appeal to the original reference.

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 30 Jul 2020 17:32:51 +0200

> And the converse appears also the case where LOOPS has the potential for most negative interactions with end-to-end recovery?

I’m lost.  Where is (a)?

> I would like to understand the potential for harm... to.

Pessimization is “harm”.  Not really a lot of harm.

> That thread was interesting, but it seemed to create additional issues - it might be worth deciding whether that is included in the next rev of the document.
>> Grüße, Carsten
>> 
> [GF] I'm now trying to understand how the "SDN" controller can know which flows in the aggregates support ECN:  At what point can an aggregated tunnel egress assume that it is safe to translate a tunnel loss into a flow ECN CE-mark?

In the MVP, the tunnel ingress can only provide recovery to ECT-marked packets.
Any non-ECT traffic needs to be handled without LOOPS support (until we go beyond the MVP, which in the current proposal requires rechartering).

> Example: (Would only established ECN-capable flows added to the aggregate? ... Translating loss at the start of a flow into a CE-mark has a strange effect when the sender is probing for support, but a recipient doesn't report CE-marks?)

The tunnel that implements LOOPS segment might also move non-ECT traffic, but LOOPS cannot help that in the MVP.  The SDN controller could be better off to configure the tunnel ends to direct ECT traffic to the LOOPS segment, not so much non-ECT traffic (unless the tunneling needs to happen anyway).

Grüße, Carsten

---

From: Gorry Fairhurst <[email-masked]>
Date: Fri, 31 Jul 2020 08:33:02 +0100

See below.

> On 30 Jul 2020, at 16:33, Carsten Bormann <[email-masked]> wrote:
> 
> 
>> 
>>> 

Ah sorry my mistake, I truncates my message to focus on talking about the scope.

> Pessimization is “harm”.  Not really a lot of harm.
On the contrary. Pessimism is important in protecting infrastructure. Please supply data and discuss operational practice.

> The tunnel that implements LOOPS segment might also move non-ECT traffic, but LOOPS cannot help that in the MVP.  The SDN controller could be better off to configure the tunnel ends to direct ECT t…

I’ve appreciated the discussion, and I wish this was more clear in the drafts. 

To me, the scope and intention expressing in  the problem statement is itself problematic to understanding the proposed work, and the impact of this service.

GOrry

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 31 Jul 2020 10:14:01 +0200

Hi Gorry,

>>>> If LOOPS does little good, due to its cost (additional reverse packets, processing/memory, tunneling if that wasn’t already in use) it is a pessimization; many optimizations have regions of operation in which they don't.

The tunneling overhead will be on the order of 40 to 60 bytes per packet.
I have made some proposals how to reduce the reverse overhead in early May, we haven’t discussed this to the end, but we are talking about single-digit percentage or less overheads here.  So much for the pessimization (inverse of optimization).

Pessimism is also a necessary component of protocol development.
By choosing reasonable upper bounds for LOOPS behavior (e.g., limiting the retransmission time window to something not much more than a couple RTTs, defining bounds for the FEC/retransmission overhead), we can limit any harm from unforeseen interactions.

> I’ve appreciated the discussion, and I wish this was more clear in the drafts. 

Certainly, more work is needed on the drafts, but that can be done in the WG.

> To me, the scope and intention expressing in  the problem statement is itself problematic to understanding the proposed work, and the impact of this service.

The document that we put most energy into is the charter, and that should be what we discuss today.  The I-Ds are background and illustrations, and we certainly weren't perfect in thinning them in line with the narrowing of the charter to the MVP.  I still think the background/illustration (proof of concept) provided by the I-Ds is useful to evaluate the charter.

Grüße, Carsten