Subject: [PWE3] PWE congestion draft

From: Yaakov Stein <[email-masked]>
Date: Sun, 3 Aug 2008 16:35:32 +0300

Stewart / Bruce

In order to progress this draft, I would like to break down my comments
into small pieces and see if we can resolve them one at a time.

Before raising teleological questions, I would like to get an answer to an ontological one.

In the draft you state
  In these environments the operator needs to make a judgement call on
  a fair estimate of the number of equivalent flows that the PW
  represents.
and
   In the PW environment some estimate (measured or configured) of the
   number flows in the PW needs to be applied to the estimate of fair
   throughput as part of the process of determining appropriate
   congestion behavior.
When I first read these sentences, I was relatively sure that I knew what they meant,
but afterwards I realized that there are several possibilities.
Upon discussing the matter with Stewart,
I discovered that perhaps I did not guess correctly which of these possibilities was intended.

I can think of at least three possibilities as to the meaning of these sentences:.

1) When a packet is lost, the egress PE notifies the ingress PE (similar to TCP),
but rather than the ingress decreasing the (maximum) sending rate to 1/2,
the rate is decreased by 1 / 2N. For example, if N=10, instead of reducing by 50 %,
we reduce by 5%.
(We won't discuss the mechanism for enforcing this rate reduction yet,
as this will be the subject of another email.)
As discussed in previous emails (and in the slides that I didn't get to present in Dublin)
this is what would happen automatically is the PW carries TCP/IP flows with
one TCP/IP packet per PW packet.
This is how I initially understood the draft.
Note that thuis mechanism does not require change to intermediate nodes,
but necessitates a new mechanism for feedback from egress PE to ingress PE.

2) When a packet is lost, the ingress is notified, but no rate decrease is enforced.
Only after N packets are lost (within some time span) is the rate decreased to 1/2.
This has the advantage of forestalling the policing of the PW,
but the disadvantage of a rather sudden drop in capacity later on.
>From a chat with Stewart, I believe this is what he had in mind.

3) When intermediate nodes become congested and need to drop a packet,
they drop a packet belonging to the PW with probability 1/N of that of an individual TCP/IP flow.
Thereafter, the egress informs the ingress, and policing is carried out according to method 1.

There are several ways of carrying out the packet dropping :
3a) intermediate nodes maintain state as to PW weightings
3b) ingress PEs encode the value of N in the packet header (no state in intermediate nodes)
3c) ingress PEs indicate drop precedence of every N'th packet (no state in intermediate nodes).

I assume that 3a is ruled out due to the memory requirements.
3b requires generating random numbers and performing a comparison per packet
(but only when there is congestion), while 3c is the simplest to implement,
and furthermore leaves the decision on which packets to drop to the ingress PE,
which may be in the best situation to know which packets should be dropped.

Can I ask the authors to reply as to which of the proposed possibilities (1, 2, 3a, 3b, 3c)
they intended - or did they intend some option 4 ?

Y(J)S

_______________________________________________
pwe3 mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/pwe3

---

From: Yaakov Stein <[email-masked]>
Date: Mon, 4 Aug 2008 19:30:27 +0300

> Yaakov,
 >  In our quest to make sure we're on the same page, let me start by trying to decipher the above sentence.
 >  I assume you mean you want to be sure you understand our meaning before arguing with us about our intentions?

I couldn't put it better - I need to understand what your are proposing,
before attempting to determine if I agree that it solves the problem at hand.
Once we agree on the meaning of your proposal, we can decide how useful/effective/efficient/etc it is.

>  I think your email actually indicates that you misunderstand the intent of our draft. It is meant to be a *framework* draft (hence the title)
>  and as such it is not trying to say what mechanism should be deployed. I think you are having trouble understanding what we mean
>  because you assume that we have a mechanism in mind when we describe only a high level policy.
>  In fact we are really trying NOT to propose a mechanism here. (Separating policy from mechanism is, I'm sure you know,
>  an age-old principle in computer science.) To the extent that the current draft talks about mechanisms,
>  we're mostly trying to illustrate that all mechanisms that we know of have shortcomings,
>  and hence this is a hard problem space to solve correctly.

First, as you admit here, you discuss quite a few mechanisms. You mostly reject them
and propose instead some principles. Even in a framework draft these principles should
be compared to the existing mechanisms, or at least described in sufficient detail that we can understand
possible side-effects.
I quite understand that you don't want to go into the precise formats of messages, etc. -
but at least I can't understand what you propose if we do not have at least "some" mechanism in mind.
>  In fact none of your guesses is correct, because, as I've said, we're not trying to propose a mechanism.

Or, perhaps all of them ?

> We're trying to describe a high-level objective for PW congestion.
>  Let me try to put this another way that may be easier to understand.

>  Assume there is a PW running over a network that is congested enough to occasionally experience loss,
>  and that the loss occasionally affects packets of the PW.
>  (I think you have argued in some prior emails that such networks should not exist, but I'll ask you to suspend disbelief for a moment).

No, I never said that there will not be congestion for PWs.
I just don't think that the most important case is PWs over the public Internet
or PWs in parallel with individual TCP/IP flows.

>  The main objective of this PW congestion work is to ensure that such loss is not ignored,
>  that is, the PW does not just keep on sending packets into the network at the same rate or, even worse, send more packets.

>  Now there is a well-known equation for the long term throughput of a TCP flow for a given loss rate, RTT, and MSS (packet size).
>  (See RFC 3448 for a quick summary.)  Call this the "single flow TCP fair rate". If a PW were carrying a single flow,
>  and experiencing a certain loss rate, and its long term average throughput where much higher than the the single flow TCP fair rate,
>  we could argue that it is not responding correctly to the fact that packets are getting lost.
>  However, if the PW happened to be carrying 100 flows, then of course we would expect that the long term average throughput
>  of the PW should be 100 times the single flow TCP fair rate.

Yes, I understood this reasoning. I just want to delve a bit deeper into how you are proposing
to enforce this importance weighting, and what its effect on the PW and the surrounding will be.

< snip>

>  Now I have some mechanisms in mind for how I would build this stuff.
>  But that is not the job of the framework to describe those mechanisms.
>  I will say however that anything involving help from the interior nodes is a non-starter,
>  because we are very much trying to avoid congestion collapse on existing IP networks.

Once again, that is understandable for the PW over the general Internet case.
While a valid case in principle, and perhaps one that the IETF as a whole really wants us to address,
I don't believe that it is the most important one for us to solve.

> So we can rule out option 3 as a feasible mechanism. But like I said, the above paragraph is about policy, not mechanism.

But you see how fast the difference between policy and mechanism becomes blurred ?

You talk about considering the PW to be equivalent to some number of flows.
I must assume that in addition to yourself, you imagine that some network element is cognizant of this fact,
as otherwise the statement does not lead to the PW being treated operationally as N flows.
That network element takes some action that uses the information (i.e. its knowledge of "N").
I think that which network elements you consider to be relevant and what the kinds of actions you have in mind
are completely relevant to a framework draft - indeed they are part of what you call "high level policy".

Y(J)S

---

From: Bruce Davie <[email-masked]>
Date: Mon, 4 Aug 2008 11:52:09 -0400

> answer to an ontological one.

Yaakov,
  In our quest to make sure we're on the same page, let me start by  
trying to decipher the above sentence. (What good is a PhD if you  
can't tell your teleology from your ontology?) I assume you mean you  
want to be sure you understand our meaning before arguing with us  
about our intentions?

I think your email actually indicates that you misunderstand the  
intent of our draft. It is meant to be a *framework* draft (hence the  
title) and as such it is not trying to say what mechanism should be  
deployed. I think you are having trouble understanding what we mean  
because you assume that we have a mechanism in mind when we describe  
only a high level policy. In fact we are really trying NOT to propose  
a mechanism here. (Separating policy from mechanism is, I'm sure you  
know, an age-old principle in computer science.) To the extent that  
the current draft talks about mechanisms, we're mostly trying to  
illustrate that all mechanisms that we know of have shortcomings, and  
hence this is a hard problem space to solve correctly.

> possibilities was intended.

In fact none of your guesses is correct, because, as I've said, we're  
not trying to propose a mechanism. We're trying to describe a high- 
level objective for PW congestion. If the Wg ever gets to the point  
of designing a mechanism or set of mechanisms, then they will need to  
meet this high level objective.

So, the short answer to your question is "none of the above".

Assume there is a PW running over a network that is congested enough  
to occasionally experience loss, and that the loss occasionally  
affects packets of the PW. (I think you have argued in some prior  
emails that such networks should not exist, but I'll ask you to  
suspend disbelief for a moment). The main objective of this PW  
congestion work is to ensure that such loss is not ignored, that is,  
the PW does not just keep on sending packets into the network at the  
same rate or, even worse, send more packets.

Now there is a well-known equation for the long term throughput of a  
TCP flow for a given loss rate, RTT, and MSS (packet size). (See RFC  
3448 for a quick summary.)  Call this the "single flow TCP fair  
rate". If a PW were carrying a single flow, and experiencing a  
certain loss rate, and its long term average throughput where much  
higher than the the single flow TCP fair rate, we could argue that it  
is not responding correctly to the fact that packets are getting  
lost. However, if the PW happened to be carrying 100 flows, then of  
course we would expect that the long term average throughput of the  
PW should be 100 times the single flow TCP fair rate.

So what we are trying to say here is this:
  - A PW should not keep sending traffic at an arbitrary rate in the  
presence of loss
  - To understand if a PW is sending more traffic than is reasonable  
in the presence of loss, we want to see if it is sending a lot more  
than would be sent if the PW traffic were actually TCP, or TCP- 
friendly as per the definition in RFC 3448
  - to calculate what it means to be TCP-friendly for a PW, you need  
some estimate of the number of flows, which, in the para you quoted,  
we suggested might be something that the operator configures.

Note that if you get the number of flows wrong, you will cause the PW  
to either send more traffic or less traffic than is strictly "fair"  
during congestion. But at least the PW won't be completely  
unresponsive to congestion in either case.  And of course in the  
absence of congestion (the regime that you and many others consider  
the norm) the TCP fair rate is infinitely high, so there will be  
nothing to do to ensure that the PW does not exceed the TCP fair rate.

Now I have some mechanisms in mind for how I would build this stuff.  
But that is not the job of the framework to describe those  
mechanisms. I will say however that anything involving help from the  
interior nodes is a non-starter, because we are very much trying to  
avoid congestion collapse on existing IP networks. So we can rule out  
option 3 as a feasible mechanism. But like I said, the above  
paragraph is about policy, not mechanism.

Bruce

---

From: Bruce Davie <[email-masked]>
Date: Tue, 5 Aug 2008 15:40:03 -0400

On Aug 4, 2008, at 12:30 PM, Yaakov Stein wrote:

> >  (I think you have argued in some prior emails that such networks  
> should not exist, but I'll ask you to suspend disbelief for a moment).
>

Well, you're certainly entitled to your opinion, but I think the case  
of PWs interacting with non-PW traffic is the main area of concern  
from the Transport area, and the main thing we're trying to do here  
is meet the requests of the Transport folks to avoid unleashing a non- 
congestion-controlled form of traffic into the wild. FWIW, most  
deployments of PWs that I am aware of run on networks that carry lots  
of TCP-IP traffic that is not inside PWs (e.g. combined L3VPN and  
L2VPN networks).

The case of PWs running in networks that only support PWs is  
definitely NOT the focus of this draft, and I guess we should try to  
make that point more clearly in the next rev (if there is one.)

> mechanisms.

See above. Feel free to go and work on the problem that you think is  
important, but I don't think this framework needs to tackle it.
Note however that I am not just talking about the "general Internet"  
but any network where there is a mix of PW and non-PW traffic.

>
> > So we can rule out option 3 as a feasible mechanism. But like I  
> said, the above paragraph is about policy, not mechanism.
>

Only because I have graciously decided to comment on your mechanisms.

> I must assume that in addition to yourself, you imagine that some  
> network element is cognizant of this fact,
> I think that which network elements you consider to be relevant and  
> what the kinds of actions you have in mind

We clearly have somewhat different ideas about where the line lies  
between policy and mechanism.

However, to satisfy your curiosity about the mechanisms I have in  
mind, I will say that I think TFRC enforced at the ingress PE by  
using a shaper (not a dropping policer) is the best combination of  
mechanisms I have thought of so far. The use of both these mechanisms  
is discussed in Section 7 of the draft.

Bruce