Subject: Re: [LOOPS] BOF co-chairs thinking on LOOPS next steps

From: Carsten Bormann <[email-masked]>
Date: Thu, 1 Aug 2019 09:03:14 +0200

On Aug 1, 2019, at 01:28, Marie-Jose Montpetit <[email-masked]> wrote:
> 
> Actually in FEC you do not process every packet. In what is called “systematic mode” you send all the packets in the buffer (window) followed by the coded packet(s).

To generate the coded packets, you need to process every packet (ingress node).  Obviously, you can send on the unchanged packets before you do that (see also Appendix B of gen-info why that may be a very good thing).

> If everything goes well you never process anything.

At the egress node.

> And if something is lost then you use the recovery packets.

Right.  So, at the egress node, you have to provide the processing capability for this worse [sic] case, even if you don’t need it in the better case.  (Of course, you can always decide to start losing badly if there is a cluster of losses that overwhelms insufficient egress processing capacity, but that is not normally what you’ll want to do.)

This is not a theoretical concern; processing speed is a make-or-break consideration for many applications of LOOPS.

Grüße, Carsten

> 
> 
> mjm
> 
> Marie-José Montpetit, Ph.D.
> Research Affiliate, MIT Media Laboratory
> [email-masked]
> [email-masked]
> 
> On July 31, 2019 at 7:11:41 PM, Carsten Bormann ([email-masked]) wrote:
> 
>> On Aug 1, 2019, at 00:18, Michael Welzl <[email-masked]> wrote:
>> > 
>> > if the packet loss ratio is known
>> 
>> Conversely, retransmission can be better to handle random, bursty losses.
>> (For a definition of “better”, which may depend on your needs.)
>> 
>> But more importantly, for retransmission, the ingress node needs to store copies of packets for a while (and retransmit from this cache), and the egress node needs to send ACKs. This is much less work per packet than doing FEC, which needs processing of the contents of each packet. So nodes may be able to do retransmission LOOPS at a higher rate than they would be able to do FEC LOOPS.
>> 
>> Grüße, Carsten
>> 
>> -- 
>> LOOPS mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/loops
> -- 
> LOOPS mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/loops

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 20 Aug 2019 11:47:14 +0200

Hi Spencer,

It is probably not too easy to generate a lot of excitement on this list while most people are still on vacation, but let me try to answer your questions.

> On Jul 30, 2019, at 16:22, Spencer Dawkins at IETF <[email-masked]> wrote:
> 
> […]
> So, my suggestion for the interested community is to nail down
> 	• In order to “do LOOPS", what already exists, that can be used without changes?

Most of IP :-)

Seriously, the idea behind LOOPS is to use existing tunnel encapsulation protocols with few additions, capitalizing on that work, and also making use of ASIC support already present in devices — it helps to be encapsulation-agnostic for this.

It is unlikely that we’ll have to invent another FEC scheme for LOOPS, so existing schemes can probably be used unchanged.  I’m not so sure the WG won’t want to improve on the FECFRAME framework, though.  I don’t want to design much of the protocol while we are still in a WG-forming stage, so I’ll refrain from delving too much on why I think that may be a good thing.  Basically, think about packet size distributions in general aggregated IP flows vs. those in the RTP flows or even single transport flows that FECFRAME schemes are being used with.

Most likely, the time stamp formats (and even some algorithms around those) we need can be stolen from existing work.

Not much else can probably be used unchanged.  

> 	• what already exists, but needs to be extended for LOOPS?

The encapsulation schemes may need to have their extension points exercised to actually send the forward and reverse information in the best way.  Some of the encapsulation protocols already define how to carry a sequence number, but other elements (such as block 1 and block 2 reverse information [1]) may need some encoding decisions.

> 	• what needs to be created, because nothing exists that meets the needs?

The actual protocol, i.e., the rules of action. (A best-effort, not fully reliable retransmit scheme; FEC parameter control loops; measurement schemes that go into these and into congestion feedback mechanisms.)

> I'm not a LOOPS proponent (the ADs asked me to chair the BOF about a month before IETF 105), but speaking as someone who hasn't been involved in depth, I wonder about
> 	• How a sender tunes FEC dynamically - is that automatic, based on FEC mechanisms people are thinking about, or is there work to do there?

I’m not aware of such a control loop in existing IETF standards, but I would love to be surprised.

> 	• How a sender knows whether to do FEC, retransmission, or both FEC and retransmission, dynamically?

In most cases, this will be setup information (controller model).

> 	• How a sender knows that it shouldn’t be doing anything, because anything it does won't help ("first, do no harm")?

Again, probably setup information.  The protocol could also define some thresholds for switching components on an off, on a relatively long time scale.

> Does everyone know how to do those three things, except me? :-)

I hope the WG can find out once it exists…

I would also like to avoid the FEC tail wagging with the LOOPS dog too much.
(I know, if you are standing very close to the tail, it may seem larger than the dog :-)

Yes, the sheer bulk and complexity of the specifications is larger in the FEC part than in the rest of the specifications.  But then, FEC is relatively well understood, a lot of (re-)usable specifications are out there, and we even have a whole RG for that which we can tap.  On the other hand, doing best-effort retransmissions (and FEC) on aggregate flows while we are experiencing increased presence of new-style end-to-end transports (encrypted, more reordering-tolerant) is a new engineering problem that needs a bit more focus from the WG.

			.oOo.

By the way, I have generated a post-ietf105 branch on the charter text proposal [charter-0.3] that explicitly calls out focusing on retransmission initially and adding FEC later (with or without a rechartering needed); this version also adds milestones for a (shim-layer-in-)host-to-network component that was requested so vividly at the BOF.

Grüße, Carsten

[1]: https://tools.ietf.org/html/draft-welzl-loops-gen-info-00#section-4.3
[2]: https://github.com/loops-wg/charter/tree/post-105

---

From: Marie-Jose Montpetit <[email-masked]>
Date: Tue, 20 Aug 2019 10:42:50 +0000

Not “excitement" but some of the FEC control mechanisms as well as the
encapsulation and aggregated flow protocols that you want to investigate
are part of the work in the nwcrg (and yes not the IETF) and I would hope
that they will be reused by LOOPS.

mjm

Marie-José Montpetit

On August 20, 2019 at 5:47:24 AM, Carsten Bormann ([email-masked]) wrote:

Hi Spencer,

It is probably not too easy to generate a lot of excitement on this list
while most people are still on vacation, but let me try to answer your
questions.

Most of IP :-)

Seriously, the idea behind LOOPS is to use existing tunnel encapsulation
protocols with few additions, capitalizing on that work, and also making
use of ASIC support already present in devices — it helps to be
encapsulation-agnostic for this.

It is unlikely that we’ll have to invent another FEC scheme for LOOPS, so
existing schemes can probably be used unchanged. I’m not so sure the WG
won’t want to improve on the FECFRAME framework, though. I don’t want to
design much of the protocol while we are still in a WG-forming stage, so
I’ll refrain from delving too much on why I think that may be a good thing.
Basically, think about packet size distributions in general aggregated IP
flows vs. those in the RTP flows or even single transport flows that
FECFRAME schemes are being used with.

Most likely, the time stamp formats (and even some algorithms around those)
we need can be stolen from existing work.

> • what already exists, but needs to be extended for LOOPS?

The encapsulation schemes may need to have their extension points exercised
to actually send the forward and reverse information in the best way. Some
of the encapsulation protocols already define how to carry a sequence
number, but other elements (such as block 1 and block 2 reverse information
[1]) may need some encoding decisions.

> • what needs to be created, because nothing exists that meets the needs?

The actual protocol, i.e., the rules of action. (A best-effort, not fully
reliable retransmit scheme; FEC parameter control loops; measurement
schemes that go into these and into congestion feedback mechanisms.)

> I'm not a LOOPS proponent (the ADs asked me to chair the BOF about a
month before IETF 105), but speaking as someone who hasn't been involved in
depth, I wonder about
> • How a sender tunes FEC dynamically - is that automatic, based on FEC
mechanisms people are thinking about, or is there work to do there?

I’m not aware of such a control loop in existing IETF standards, but I
would love to be surprised.

> • How a sender knows that it shouldn’t be doing anything, because
anything it does won't help ("first, do no harm")?

Again, probably setup information. The protocol could also define some
thresholds for switching components on an off, on a relatively long time
scale.

I would also like to avoid the FEC tail wagging with the LOOPS dog too much.
(I know, if you are standing very close to the tail, it may seem larger
than the dog :-)

Yes, the sheer bulk and complexity of the specifications is larger in the
FEC part than in the rest of the specifications. But then, FEC is
relatively well understood, a lot of (re-)usable specifications are out
there, and we even have a whole RG for that which we can tap. On the other
hand, doing best-effort retransmissions (and FEC) on aggregate flows while
we are experiencing increased presence of new-style end-to-end transports
(encrypted, more reordering-tolerant) is a new engineering problem that
needs a bit more focus from the WG.

.oOo.

By the way, I have generated a post-ietf105 branch on the charter text
proposal [charter-0.3] that explicitly calls out focusing on retransmission
initially and adding FEC later (with or without a rechartering needed);
this version also adds milestones for a (shim-layer-in-)host-to-network
component that was requested so vividly at the BOF.

Grüße, Carsten

---

From: Colin Perkins <[email-masked]>
Date: Tue, 20 Aug 2019 11:50:32 +0100

One of the things we’d need to decide is what’s the path from NWCRG into IETF to standardise such things before use by LOOPS.

FECFRAME seemed to work quite well. Is the goal to revive it and move some work from NWCRG? Or to bring work from NWCRG into TSVWG to do FECFRAME extensions for LOOPS to reference? Or to bring work from NWCRG into LOOPS? 

I’d suggest either of the former two options make more sense than the latter. 

Colin

> On 20 Aug 2019, at 11:42, Marie-Jose Montpetit <[email-masked]> wrote:
> 
> Not “excitement" but some of the FEC control mechanisms as well as the encapsulation and aggregated flow protocols that you want to investigate are part of the work in the nwcrg (and yes not the IETF) and I would hope that they will be reused by LOOPS. 
> 
> mjm
> 
> Marie-José Montpetit
> 
> On August 20, 2019 at 5:47:24 AM, Carsten Bormann ([email-masked] <mailto:[email-masked]>) wrote:
> 
>> Hi Spencer,
>> 
>> > On Jul 30, 2019, at 16:22, Spencer Dawkins at IETF <[email-masked] <mailto:[email-masked]>> wrote:
>> > 
>> > […]
>> [1]: https://tools.ietf.org/html/draft-welzl-loops-gen-info-00#section-4.3 <https://tools.ietf.org/html/draft-welzl-loops-gen-info-00#section-4.3>
>> [2]: https://github.com/loops-wg/charter/tree/post-105 <https://github.com/loops-wg/charter/tree/post-105>
>> 
>> -- 
>> LOOPS mailing list
>> [email-masked] <mailto:[email-masked]>
>> https://www.ietf.org/mailman/listinfo/loops <https://www.ietf.org/mailman/listinfo/loops>

-- 
Colin Perkins
https://csperkins.org/

---

From: Marie-Jose Montpetit <[email-masked]>
Date: Tue, 20 Aug 2019 10:56:02 +0000

I think within the nwcrg we could discuss what is the best path. Just
bringing the work I agree is not the best way forward.

On August 20, 2019 at 6:50:47 AM, Colin Perkins ([email-masked]) wrote:

One of the things we’d need to decide is what’s the path from NWCRG into
IETF to standardise such things before use by LOOPS.

FECFRAME seemed to work quite well. Is the goal to revive it and move some
work from NWCRG? Or to bring work from NWCRG into TSVWG to do FECFRAME
extensions for LOOPS to reference? Or to bring work from NWCRG into LOOPS?

I’d suggest either of the former two options make more sense than the
latter.

Colin

On 20 Aug 2019, at 11:42, Marie-Jose Montpetit <[email-masked]>
wrote:

mjm

Marie-José Montpetit

Hi Spencer,

Most of IP :-)

.oOo.

Grüße, Carsten

-- 
Colin Perkins
https://csperkins.org/

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Wed, 21 Aug 2019 23:23:56 -0500

Hi, Colin,

> extensions for LOOPS to reference? Or to bring work from NWCRG into LOOPS?

It seemed to me that doing the sliding window FECFRAME extension in TSVWG
worked well enough (modulo a really amazing kerfluffle on getting a PRNG
implementation that we could publish in an IETF RFC), although that
decision would be up to Magnus and Mirja, at this point in the process. But
the TSVWG co-chairs are already trained on what would be required from
them.

Spencer

> Most likely, the time stamp formats (and even some algorithms around
> those) we need can be stolen from existing work.
>
> The encapsulation schemes may need to have their extension points
> exercised to actually send the forward and reverse information in the best
> way. Some of the encapsulation protocols already define how to carry a
> sequence number, but other elements (such as block 1 and block 2 reverse
> information [1]) may need some encoding decisions.
>

---

From: Colin Perkins <[email-masked]>
Date: Thu, 22 Aug 2019 09:28:57 +0100

Makes sense – although as you say, it’s for Magnus and Mirja to decide. 
Colin

> On 22 Aug 2019, at 05:23, Spencer Dawkins at IETF <[email-masked]> wrote:
> 
> Hi, Colin, 
> 
> On Tue, Aug 20, 2019 at 5:50 AM Colin Perkins <[email-masked] <mailto:[email-masked]>> wrote:
> It seemed to me that doing the sliding window FECFRAME extension in TSVWG worked well enough (modulo a really amazing kerfluffle on getting a PRNG implementation that we could publish in an IETF RFC), although that decision would be up to Magnus and Mirja, at this point in the process. But the TSVWG co-chairs are already trained on what would be required from them. 
> 
> Spencer
>  
>> On 20 Aug 2019, at 11:42, Marie-Jose Montpetit <[email-masked] <mailto:[email-masked]>> wrote:
>> 
> https://csperkins.org/ <https://csperkins.org/>
> 
> 
> 
> 

-- 
Colin Perkins
https://csperkins.org/

---

From: Marie-Jose Montpetit <[email-masked]>
Date: Thu, 22 Aug 2019 07:22:50 -0400

Just an add on:

Besides the code that was presented within FECFRAME, NWCRG is also working
on API for sliding window codes:
https://datatracker.ietf.org/doc/draft-roca-nwcrg-generic-fec-api/
And of course we have a hackaton to develop an open source codec and the
LOOPS community is more that welcome to participate.

mjm

On August 22, 2019 at 4:29:11 AM, Colin Perkins ([email-masked]) wrote:

On 22 Aug 2019, at 05:23, Spencer Dawkins at IETF <
[email-masked]> wrote:

Hi, Colin,

Spencer

-- 
Colin Perkins
https://csperkins.org/