Subject: [Lsr] Congestion (flow) control thoughts.

From: Christian Hopps <[email-masked]>
Date: Wed, 29 Apr 2020 08:24:09 -0400

[As WG member]

I like the start these drafts are making, I have a few thoughts.

- I think its worthwhile work and as a WG we should end up adopting a draft or drafts that try to solve the problem.
- Congestion control is hard, and a very well studied problem, as such whatever we end up should reference prior-art to justify it's claims.
- Data, whatever solution we end up with needs to have data to back up it's claims.

** draft-ginsberg-lsr-isis-flooding-scale

Having just recently offered a, "CC algorithm is local so we aren't specifying it", in some unrelated work, I discovered (correctly so) that it was not sufficient (see [*] for the resulting changes if curious). One needs to provide (doesn't need to be normative) a working algorithm to prove that the protocol is providing enough information to the transmitter to actually implement a viable CC algorithm.

Now draft-ginsberg does suggest an algorithm, however only for P2P. We need to address LAN as well. This also needs references on how this algorithm relates to existing studied work, and perhaps most importantly some data showing that it actually works.

Without this it will almost for sure not clear the IESG (in particular Transport AD review), and rightly so. :)

** draft-decraene-lsr-isis-flooding-speed-03

I read this draft as providing feedback to transmitters on ways to modify the parameters of their CC algorithm. It doesn't actually seem in conflict with draft-ginsberg (or doesn't need to be) to me. I believe it's following the path of least resistance in utilizing existing IS-IS parameters. I wonder if we could be more adventurous here.

** Both

Both drafts mention trying to increase the ACK rate. I think we need to look deeply into this, as this is critical information for CC algorithms. It very well could be that we do not need protocol changes as the protocol provides an ACK (or NACK for LANs), but I also don't think changes or additions should be verboten either if they could provide significant improvements.

In the algorithms we propose or envision we need to be considering the affect of larger RTT on the algorithm for slow or long distant links (radios, satellites, long-haul fibers, etc).

Thanks,
Chris.
[As WG member]

[*] - https://tools.ietf.org/html/draft-hopps-ipsecme-iptfs-01#section-3
      https://tools.ietf.org/html/draft-hopps-ipsecme-iptfs-01#appendix-B),

---

From: "Gengxuesong (Geng Xuesong)" <[email-masked]>
Date: Thu, 30 Apr 2020 02:38:20 +0000

Hi,

Thank you Chris for emphasizing that congestion control is hard and more reference may be necessary.

And as the the follow-up of my comments in the interim meeting, I would like to make my question more clear:

In congestion control of layer 4, it is assumed that there is a bottleneck in the network, and the ideal rate of the transmitters equals to a fair share of the bandwidth in the bottleneck. The flows in the network change all the time and so as to the ideal transmitting rate. There are some methods to detect the bottleneck, for example detecting packet loss, setting ECN, RTT and so on. Considering that the goal of cc is to maximize the throughput and minimize the queuing delay, the throughput and delay could be used to compare different cc algorithms.

The problem of mine is that for CC of ISIS flooding, where is the bottleneck?(maybe the receiver capability) What method of detecting could be used? (In my understanding, we have two options at this stage: one from the receiver side and the other from the transmitter side) What is the criteria of comparing? (still a little confusing for me )

Best Regards

Xuesong

-----Original Message-----

From: Lsr [mailto:[email-masked]] On Behalf Of Christian Hopps

Sent: Wednesday, April 29, 2020 8:24 PM

To: [email-masked]

Cc: [email-masked]

Subject: [Lsr] Congestion (flow) control thoughts.

[As WG member]

** Both

Thanks,

Chris.

[As WG member]

---

From: [email-masked]
Date: Thu, 30 Apr 2020 09:15:55 -0700

Hi Xuesong,

> In congestion control of layer 4, it is assumed that there is a bottleneck in the network, and the ideal rate of the transmitters equals to a fair share of the bandwidth in the bottleneck. The flow…

I consider flow control and congestion control to be separate, but similar problems.

Flow control is about creating a single control loop between a single transmitter and single receiver.

Congestion control is about creating multiple interacting control loops between multiple transmitters and multiple receivers.

> The problem of mine is that for CC of ISIS flooding, where is the bottleneck?(maybe the receiver capability) What method of detecting could be used? (In my understanding, we have two options at thi…

In the case of modern IS-IS, we have gigabit links everywhere, the bandwidth greatly exceeds our needs, and flooding only happens between peers, so link resources are never the issue.  Thus, to my mind, congestion only happens when a node is engaged in flooding on multiple interfaces concurrently. Effectively, congestion control can be seen as flow control with shared resources, or as flow control with varying resources.

The resources of concern vary depending on the internals of both the sender and receiver.  Some that come to mind are:

- CPU. Both transmitter and receiver have finite cycles available. This is typically shared across multiple interfaces, so can congest.

- Process level buffer space. There may be a fixed number of packet buffers within the IS-IS process.  These can be depleted.

- Kernel buffers. 

- Internal control fabric. In some multi-chassis designs, the line card chassis is remote from the CPU running IS-IS. The network between the two can congest.

- Line card CPU. Again in the case of a multi-chassis system, control packets may flow through a CPU on the line card.  This can congest due to cycles or buffers.

- Forwarding plane buffers. Modern design for many systems has a forwarding plane ASIC doing a great deal of our packet handling. 
  One of its responsibilities is to extract control plane traffic from incoming traffic and pass it upwards. As a single ASIC, it is finite and can only buffer
  so many packets before loss.  The buffers are only cleared when the packets are read out by the line card or central CPU.  IMHO, this is a very likely point of congestion.

All of these can be instrumented to determine whether they are congesting.  It seems very unlikely that the transmitter ever congests.  Bruno’s data as presented supports this: the transmitter can outstrip the receiver. Thus, we tend to focus on congestion at the receiver.

The other feedback mechanisms that we have available are the ones that we’ve discussed: PSNPs provide acknowledgment of received packets. From that, and their timing, we may be able to infer more useful data, but we are discussing changing their behavior to make things more useful.

In our draft, we are proposing other feedback mechanisms.

Tony