Subject: [LOOPS] LOOPS Comments

From: Martin Duke <[email-masked]>
Date: Wed, 13 May 2020 09:01:43 -0700

High-level comments about LOOPS:

This work will be valuable in several use cases, in particular encrypted
transports (or IPSec) running over links with RF losses. I think it would
benefit from a more detailed consideration of what problem(s) you are
trying to solve. A generalized solution will require lots of additional
mitigation mechanisms, which would scale up the work significantly.

The gen-info and problem-opportunities drafts mention a lot of these
concerns, and imply several different mechanisms.

For example:
- Detecting congestion losses vs. RF losses: this is hard, but certain
network architectures can eliminate the problem by, e.g. rate limiting at
the ingress point.
- ECT vs non-ECT traffic: a general solution will have to provide different
mechanisms for both; if it is sufficient to initially support one or the
other, that may be half as much algorithm to design.
- Relatedly, initially requiring the entire LOOPS path to support ECN
marking would reduce the problem space.
- Varying transport behaviors: different flows will have different e2e
latencies, reordering thresholds, and reactions to delay variation and
delay spikes. These will be hard to detect! It may be unavoidable to do a
lot of work here, but limiting the cases where LOOPS can be applied (e.g.
the added delay is a small fraction of the LOOPS segment min delay, which
IIUC would require FEC) would simplify as much as possible. Alternatively,
if there is a means for endpoints to explicitly request the service, then
one can assume that they have configured themselves correctly to leverage
LOOPS.
- Identifying one, or a handful, of tunneling protocols with a good
deployment story before chartering will reduce the risk there is not a
critical mass to do any of them.

One possible minimal increment of work would be: FEC-based LOOPS with a
Geneve encapsulation to be applied only to ECT-marked traffic over a
segment with ECN-enabled routers and a rate limiter at ingress.

I emphasize that the above is only example of a use case that is
constrained enough to present a small group of problems to solve. I'd
encourage the group to zoom in on a similarly constrained use case that
meets demand in the real internet.

A similarly constrained used case would allow you to put in the basic
protocol elements and then recharter to tackle more difficult problems.

Martin

---

From: Michael Welzl <[email-masked]>
Date: Wed, 13 May 2020 23:20:49 +0200

Hi,

Thank you very much for your comments.
FWIW, I very much agree with them - your ECT comment in particular: that would make things a lot easier for LOOPS, and who knows, maybe it could even help incentivize ECN deployment?

Cheers,
Michael

> On May 13, 2020, at 6:01 PM, Martin Duke <[email-masked]> wrote:
> 
> This work will be valuable in several use cases, in particular encrypted transports (or IPSec) running over links with RF losses. I think it would benefit from a more detailed consideration of what problem(s) you are trying to solve. A generalized solution will require lots of additional mitigation mechanisms, which would scale up the work significantly..
> 
> The gen-info and problem-opportunities drafts mention a lot of these concerns, and imply several different mechanisms.
> 
> For example:
> - Detecting congestion losses vs. RF losses: this is hard, but certain network architectures can eliminate the problem by, e.g. rate limiting at the ingress point.
> - ECT vs non-ECT traffic: a general solution will have to provide different mechanisms for both; if it is sufficient to initially support one or the other, that may be half as much algorithm to design.
> - Relatedly, initially requiring the entire LOOPS path to support ECN marking would reduce the problem space.
> - Varying transport behaviors: different flows will have different e2e latencies, reordering thresholds, and reactions to delay variation and delay spikes. These will be hard to detect! It may be unavoidable to do a lot of work here, but limiting the cases where LOOPS can be applied (e.g. the added delay is a small fraction of the LOOPS segment min delay, which IIUC would require FEC) would simplify as much as possible. Alternatively, if there is a means for endpoints to explicitly request the service, then one can assume that they have configured themselves correctly to leverage LOOPS.
> - Identifying one, or a handful, of tunneling protocols with a good deployment story before chartering will reduce the risk there is not a critical mass to do any of them.
> 
> One possible minimal increment of work would be: FEC-based LOOPS with a Geneve encapsulation to be applied only to ECT-marked traffic over a segment with ECN-enabled routers and a rate limiter at ingress.
> 
> I emphasize that the above is only example of a use case that is constrained enough to present a small group of problems to solve. I'd encourage the group to zoom in on a similarly constrained use case that meets demand in the real internet.
> 
> A similarly constrained used case would allow you to put in the basic protocol elements and then recharter to tackle more difficult problems.
> 
> Martin
> -- 
> LOOPS mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/loops

---

From: wangjl50 <[email-masked]>
Date: Thu, 14 May 2020 13:07:54 +0800

Hi,
 
I agree with starting with the specific transport behavior.
 
LOOPS enabled in-network recovery is most likely to be enable for certain applications, not all traffic. As operator, we can identify the traffic from those applications and  direct them to LOOPS enabled tunnel ingress.
 
Talking about means for endpoints to aware the LOOPS enabled service somewhere in the middle and be correctly configured, I believe it is highly feasible for the direction from the server to the client. We can assume specific behaviors at the server side for, e.g. ECN enablement, DUPACK based loss detection or time-based loss detection.
For applications like gaming ( e.g. Arena Of Valor, a local popular game here), traffic from server to client direction highly affects the experience of customers. In-network recovery would be good to help in this case.
 
A slightly different case is payment. There are very small number of packets in both directions. The host sender (no matter which direction) does not necessarily care about the window adjustment as it is very likely a one-time sending. What matters is simply to save the potential tail loss as soon as possible. In-network recovery could help here without worrying too much about how the e2e sender would detect or react to a loss. As long as the packet is recovered quickly, we are good.  

Best regards
Jianglong Wang

From: Martin Duke
Date: 2020-05-14 00:01
To: loops
Subject: [LOOPS] LOOPS Comments
High-level comments about LOOPS:

For example:

Martin

---

From: Michael Welzl <[email-masked]>
Date: Thu, 14 May 2020 09:07:04 +0200

Hi,

A few answers below:

> On May 14, 2020, at 7:07 AM, wangjl50 <[email-masked]> wrote:
> 
>  Hi,
>  
> Talking about means for endpoints to aware the LOOPS enabled service somewhere in the middle and be correctly configured, I believe it is highly feasible for the direction from the server to the client.

I said before that I agree with Martin’s comments - but now I see that I should have excluded this particular one. I agree that things can work better "if there is a means for endpoints to explicitly request the service”, but this is also an extension of the solution space, working against the plan to narrow it down and keep things simple. I think, at least at this point in time, we should not explicitly involve end systems.

> We can assume specific behaviors at the server side for, e.g. ECN enablement, DUPACK based loss detection or time-based loss detection.

The problem with them is that they wouldn’t necessarily react to congestion in a correct way - which may not be a problem because their traffic volume is so tiny, but how is a LOOPS system supposed to know? Requiring them to ECN-mark their (perhaps UDP) packets requires developers to at least think about a congestion reaction, so that’s not a bad direction.

>  A slightly different case is payment. There are very small number of packets in both directions. The host sender (no matter which direction) does not necessarily care about the window adjustment as it is very likely a one-time sending. What matters is simply to save the potential tail loss as soon as possible. In-network recovery could help here without worrying too much about how the e2e sender would detect or react to a loss. As long as the packet is recovered quickly, we are good..  

This is probably just a silly little idea, but: maybe we could make an exception to the “require ECN” rule for TCP packets carrying a FIN flag?
(because there’s no point to require a correct congestion reaction upon the last packet of a connection).

…that’s just a little trick, though, and it goes against the “don’t look, don’t touch” premise that I liked about LOOPS from the start. IMO, we shouldn’t make such tricks part of our design, but resulting documents could perhaps hint at these possibilities.

Cheers,
Michael

> 
> Best regards
> Jianglong Wang
> 
>  
> From: Martin Duke <mailto:[email-masked]>
> Date: 2020-05-14 00:01
> To: loops <mailto:[email-masked]>

---

From: Liyizhou <[email-masked]>
Date: Thu, 14 May 2020 08:37:58 +0000

Hi Michael,

---snip----

My guess here. It was not to imply LOOPS should include the protocol work involving the end systems for now. It is more like, in specific applicable scenarios, it would be safe to make some assumptions, e.g.  always ECT-traffic, so that LOOPS does not need to worry about non-ECT traffic. Non-ECT traffic would not be directed to LOOPS ingress.

The payment case at the end of the email gives another type of applications that ECT or non-ECT traffic does not matter. LOOPS works with both.

In short, 2 specific applicable scenarios are the traffic directed to LOOPS segment ingress either ECT-traffic (ECN-marking required over LOOPS segments), or short flows aggregation with no much worry on cwnd adjustment caused by loss.

Rgds,
Yizhou

Cheers,
Michael

Best regards
Jianglong Wang

From: Martin Duke<mailto:[email-masked]>
Date: 2020-05-14 00:01
To: loops<mailto:[email-masked]>
Subject: [LOOPS] LOOPS Comments
High-level comments about LOOPS:

For example:

Martin
--
LOOPS mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/loops

---

From: Michael Welzl <[email-masked]>
Date: Thu, 14 May 2020 12:55:42 +0200

> On May 14, 2020, at 10:37 AM, Liyizhou <[email-masked]> wrote:
> 
> Hi Michael,
>  
> --------snip-------
>  

Oh, ECT, yes, that IS a form of end system involvement - no further signaling is what I meant.

> The payment case at the end of the email gives another type of applications that ECT or non-ECT traffic does not matter. LOOPS works with both.

Well… you need to detect this traffic somehow. If it’s TCP, how would a LOOPS ingress know it doesn’t matter if you hide loss from it because it’s only sending so little anyway?  That doesn’t sound like a good direction. But, we could maybe make an exception for the very last packet (which is detectable due to its FIN flag). Anyway, if the receiver gets even only one packet at the end delivered, good loss recovery such as RACK can repair things.

>  In short, 2 specific applicable scenarios are the traffic directed to LOOPS segment ingress either ECT-traffic (ECN-marking required over LOOPS segments), or short flows aggregation with no much w…

So I’m sceptical about the second scenario unless you have at least a clear idea on how to detect that traffic.
Even then… I don’t like the premise of “this traffic, e.g. towards this destination port, is known to send very little so we can safely hide loss from it”. You can build a system that does that, the Internet won’t break because of it, but it’s a bad direction for a standard.

Cheers,
Michael