Subject: Re: PWE3 congestion

From: [email-masked]
Date: Wed, 12 Feb 2003 12:53:01 +0000

All,

Is the implicit assumption here that you cannot characterise the PW 
traffic sufficiently to provision the PSN so that it behaves in a way that 
the end-to-end layer 2 service can be sustained?  For example, many 
protocols e.g. ATM, were originally designed with the assumption that 
packet loss in the underlying network (usually SONET/SDH) is very low. 
Even if you built in some form of closed loop congestion control for the 
PW, you still need to provision the PSN to allow the QoS objectives of the 
end-to-end service to be met. 

Cheers,

Matthew
-----------------------------------------
Matthew Bocci
Office of the CTO
Alcatel Fixed Networks Division
-----------------------------------------

Chris Metz <[email-masked]>
Sent by: [email-masked]
12/02/2003 11:58

        To:     Luca Martini <[email-masked]>
        cc:     [email-masked], "[email-masked]" <[email-masked]>
        Subject:        Re: [PWE3] PWE3 congestion

At 06:42 PM 2/11/2003 -0700, Luca Martini wrote:
>it sounds like a reasonable approach.
>It would be nice to  point out somewhere that the several PW layer2 
>payloads assume 100% delivery , and therefore PW should probably be run 
>over a PSN that can guarantee 100% frame delivery for the pw class of 
service.

Does such a PSN operating a today's speeds exist :-)

Getting back to the notion of PWE3 congestion control I too am intrigued. 
The approach with TCP-style traffic operating over the PW is the correct 
one. Regarding traffic of unknown provenance the question would be how to 
implement the feedback loop so that the ingress PE is make aware of 
congestion and can take appropriate action. I am presuming we are not 
talking about encapsulating PWs in TCP/SCTP sessions between the PW 
end-points :-) so it seems we need what some sort of ABR-style closed 
loop?

Cheers ...

>Luca
>
>
>Stewart Bryant wrote:
>>I am looking at the congestion section of the arch
>>revision (we need one). It seems to me that we can divide the
>>problem up into two traffic classes:
>>o Traffic known (perhaps by SLA, perhaps by audit) to be TCP
>>friendly
>>o Traffic of unknown provenance
>>In the former case we need do nothing. Natural packet discard
>>will feed back to the transport layer, and the right things
>>will happen. I include in this case IP over packet PW, and IP
>>over ATM PW. I propose that the term "TCP friendly" be left
>>sufficiently general that it covers any IETF transport (they
>>are all supposed to be TCP friendly), and other similar
>>transports such as TP4 (if it still exists).
>>For the "traffic of unknown provenance", it seems to me that
>><draft-ietf-tsvwg-tfrc-05.txt>
>>could be used to calculate the maximum offered load, and
>>the PE would then be required to maintain it's load below
>>this rate (if necessary by shutting down PWs). The
>>statistics needed to operate this algorithm look a bit
>>daunting for a forwarding system, but presumably
>>some stastical view of the counters, a priori knowledge
>>of the bit rate etc would simplify things.
>>Has anyone elso looked at this, and does it seem like the
>>right way to go?
>>Stewart
>>_______________________________________________
>>pwe3 mailing list
>>[email-masked]
>>https://www1.ietf.org/mailman/listinfo/pwe3
>
>
>_______________________________________________
>pwe3 mailing list
>[email-masked]
>https://www1.ietf.org/mailman/listinfo/pwe3

_______________________________________________
pwe3 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pwe3

---

From: Stewart Bryant <[email-masked]>
Date: Wed, 12 Feb 2003 19:00:29 +0000

There are two halves to the congestion problem:

1) Does the existence of the PW cause congestion on the PSN, and
hence break the PSN.

2) Does the existence of congestion on the PSN cause the PW
to break.

The comments and questions (from me) that Luca quoted were as
a result of the ADs specifying that we MUST have a solution
to (1). The text (below from ...arch-02) is a re-write of the
text that was added to RTP to cover case (1). There is no
requirement use TCP or SCTP. The requirement is that if the
payload will not automatically react like TCP to congestion
loss in the PSN, then a method must me provided whereby an
equivalent reduction in the offered traffic reduces in response
to PSN congestion "just like TCP". In the case of a bit or
cell stream, then I would assume that this must be done by
closing one or more PWs when the packet loss reaches some
threshold, but any other method that better suits the PW
is fine.

(2) can only be achieved in practice by a mixture of provisioning
and QoS management. If the underlying PSN cannot deliver the necessary
QoS performance, then it is not suitable to carry the PW. To some
extent the deficiencies of the underlying PSN can be disguised
by an NSP function, but this is not always possible.
We could run the PW over a reliable transport to improve the
QoS of the underlying PSN, but I think that most people agree that
the resultant cost in processing resources is unacceptable, and
a QoS based approach is the best practical approach.

Regards

Stewart

-------------------------------------------------------------------------
6.5  Congestion Considerations

    The PSN carrying the PW may be subject to congestion. The congestion
    characteristics will vary with the PSN type, the network architecture
    and configuration, and the loading of the PSN.

    Where the traffic carried over the PW is known to be TCP friendly
    (by, for example, packet inspection), packet discard in the PSN will
    trigger the necessary reduction in offered load, and no additional
    congestion avoidance action is necessary.

    If the PW is operating over a PSN that provides enhanced delivery,
    the PEs SHOULD monitor packet loss to ensure that the service that
    was requested is actually being delivered. If it is not, then the PE
    SHOULD assume that the PSN is providing a best-effort service, and
    SHOULD use the best-effort service congestion avoidance measures
    described below.

    If best-effort service is being used and the trafic is not known to
    be TCP friendly, the PEs SHOULD monitor packet loss to ensure that
    the packet loss rate is within acceptable parameters. Packet loss is
    considered acceptable if a TCP flow across the same network path and
    experiencing the same network conditions would achieve an average
    throughput, measured on a reasonable timescale, that is not less than
    the PW flow is achieving. This condition can be satisfied by
    implementing a rate-limiting measure in the NSP, or by shutting down
    one or more PWs.  The choice of which approach to use depends upon
    the type of traffic being carried.  Where congestion is avoided by
    shutting down a PW, a suitable mechanism MUST be provided to prevent
    it immediately returning to service, causing a series of congestion
    pulses.

    The comparison to TCP cannot be specified exactly, but is intended as
    an "order-of-magnitude" comparison in timescale and throughput. The
    timescale on which TCP throughput is measured is the round-trip time
    of the connection. In essence, this requirement states that it is not
    acceptable to deploy an application (using PWE3 or any other
    transport protocol) on the best-effort Internet which consumes
    bandwidth arbitrarily and does not compete fairly with TCP within an
    order of magnitude.  One method of determining an acceptable PW
    bandwidth is described in [TFRC].

---

From: Sasha Vainshtein <[email-masked]>
Date: Thu, 13 Feb 2003 12:26:15 +0200

Stewart and all,
Please see a brief comment inline.

With best regards,
                                   Sasha Vainshtein
email:     [email-masked] <mailto:[email-masked]> 
tel:       +972-3-7659993 (office)
           +972-8-9254948 (res.)
           +972-58-674833 (cell.)

> -----Original Message-----
> From: Stewart Bryant [mailto:[email-masked]]
> Sent: Wednesday, February 12, 2003 9:00 PM
> To: [email-masked]
> Cc: Chris Metz; Luca Martini; [email-masked]; [email-masked]
I do not think that proposed behavior is "just like TCP". 
TCP connections do not usually disconnect themselves 
under congestion (even if a frustrated user may do so 
if the connection looks stuck). On the other hand, I
do not see any other way to treat congestion in PWs
that are associated with CBR-like traffic.

---

From: Stewart Bryant <[email-masked]>
Date: Thu, 13 Feb 2003 11:22:50 +0000

Sasha,

> that are associated with CBR-like traffic. 

It is "TCP like" in that the transport (PW) significantly
reduces the offered load in the presence of network congestion
(as detected by packet loss), and that it refrains from trying
to increase the offered load until a suitable cooling off
period has elapsed. In the case of TCP the offered load is
regulated by the window size and the acknowledgment delay.
In the PW case it is regulated by the PW bandwidth.

Stewart

---

From: Philip Chimento <[email-masked]>
Date: Thu, 13 Feb 2003 09:14:17 -0500

One thing you have to remember is that TCP is in some sense 'greedy' in 
that
it will keep probing to find the max load that it can offer (i.e. the 
max
rate at which it can transmit). It seems to me that with PWs, 
especially if
you want to do circuit emulation, that the following points hold: 

1) Your options for squeezing the rate down are very limited, otherwise 
you lose
the value of the service; i.e. you start to underrun your playout 
buffer severely.
2) You don't have a continuum of rates to work with, again I am 
referring mostly to circuit emulation, in the sense that you cannot 
reduce the bandwidth by arbitrary amounts. You cannot reduce the 
bandwidth in this case by 1/2 for example (which, of course, TCP can). 
This argues that you congestion control, at least for circuit emulation 
PWs, should be removal of some wires.
3) You cannot re-introduce load in small increments. If you want to 
re-introduce a PW after congestion has cleared, you will have to know 
either that flows that you share the bottleneck link with will adjust 
to the bandwidth that the PW re-introduces or that there is sufficient 
slack (i.e. underutilization) that will accomodate the new PW.

Non-trivial problems....

Regards, Phil Chimento
 
-- 
Phil Chimento             Ericsson IPI
Phone: 240-314-3597	  7301 Calhoun Place
[email-masked]   Rockville, Md. 20855
-----------------------------------------------------

---

From: [email-masked]
Date: Thu, 13 Feb 2003 16:32:35 -0000

Philip...I totally agree with you, these are indeed 'non-trivial problems'.
However, speaking as an operator, and considering *if* it will be possible
to migrate technologies like ATM and FR to a common MPLS core, I see it as
far worse than this.  Consider these factors:

We should be defining proper client/server layer network relationships such
that they are functionally independent (and we sure need this for arbitrary
XoverMPLS), ie both the client layer network and the server layer network
must be functionally self-sufficient and decoupled.  This is an obvious
requirement IMO to allow flexible/arbitrary client/server relationships to
exist, and to allow layer network technologies to be developed/sunsetted
independently.

I consider even the adding of sequence numbers to the 'control word' to be a
error of judgement.....it strikes me that this is standardising a broken
implementation of the server layer (esp in case of MPLS).  If these are
retained then I would expect to see some defect detection/handling
procedures specified for SN aberrations.....which seems, in one respect (but
not all), to be a concern of the ADs here (though I note they are actually
more worried about protecting the public Internet, which is fair enough from
their perspective.....but its not sufficient from mine as an operator).

Similarly, if anyone adds anything else to the control-word (or anything
that can be considered as lying somewhere between the client and the server
layer) which is *not* simply client/server adaptation (see that dirty-word
G.805), then this too will need 'managing'.  I even hear a rumour that a
large router vendor wants to add an OAM function here....if true, this would
really be stupid.

PWE3 are in danger of setting off on the road (in fact they are already on
it) of defining new layer networks.  I would not advise going there....we
have enough layer networks as it is (and some of these are not well
specified functionally) without adding more.

If the server layer is IP then we should make sure IP can be managed
properly.  If the server layer is MPLS make sure this can be managed
properly.  This is the right answer.  I don't think its a good idea to keep
adding 'fixes' to this 'in-between' layer......this should really only be a
simple adaptation function IMO.

regards, Neil

> From: Philip Chimento [mailto:[email-masked]]
> Sent: 13 February 2003 14:14
> To: [email-masked]
> Cc: Sasha Vainshtein; [email-masked]

---

From: Stewart Bryant <[email-masked]>
Date: Thu, 13 Feb 2003 16:49:40 +0000

> simple adaptation function IMO.

What exactly do you mean by "managed properly"?

The requirement that MUST be addressed is to have a mechnanism that in
real time protects the network core from congestion collapse in a
fair way.

In the case of non-TCP friendly traffic what is your technical proposal
for the design of an alternative congestion back-off mechnism?

Stewart

---

From: [email-masked]
Date: Thu, 13 Feb 2003 17:23:18 -0000

Stewart, please see below, regards, Neil

Stewart Bryant wrote 13 February 2003 16:50

<snip>
> What exactly do you mean by "managed properly"?
NH=> Simply this: If there is a client/server relationship between 2 layer
network technologies  X/Y say, then its important that both X and Y can be
operated/managed as layer networks in their own right.  So this embraces all
the functional components one requires to do this, eg addressing, routing,
signalling (not cnls obviously), fault-management, traffic management,
performance management, etc.  In particular one should not be asking a given
layer network (be it client or server to some other layer network) to proxy
for functional deficiences in the other layer network....and in the specific
case here there is an 'in-between' layer network that seems to be being
created that looks like its taking on some of these proxying functions.
> for the design of an alternative congestion back-off mechnism?
NH=>The solution for protecting the network (get the source to slow down) is
not trivial as Philip pointed out. How do you tell an ATM connection to slow
down?  How do you tell a circuit emulation T1/E1 to slow down?

One way of addressing this is via signalling and CAC for known client
demands on the server.....but of course it assumes these functions are
available/specified in the server.....if they are not then perhaps one
should not be doing this?

---

From: Shahram Davari <[email-masked]>
Date: Thu, 13 Feb 2003 11:18:27 -0800

> for the design of an alternative congestion back-off mechnism?

Hi,

A PW may be carrying a fixed BW traffic (CBR, like TDM)
or it may be carrying variable BW traffic that requires 
committed rate and excess rate such as FR. In the former case
Neil is correct to suggest that CAC, policing and resource
reservation should be used and that if these are done properly
these types of PW won't effectively cause any congestion in the PSN.
This is what Luca and Sasha were referring to. 

The second type of traffic, requires resource reservation, CAC and policing
for the committed rate, but not necessarily for the excess rate. This type
of PW may in fact cause congestion in the PSN.

The most logical approach is to inform the ingress PE (and not the customer)
about the congestion and let the ingress PE either cut the excess traffic or
may be even reroute it.

Mechanisms such as those in RFC 3168 (IP ECN), may be used, but it only identifies the
congestion to the egress (i.e., forward congestion notification). 

Two other method are at least possible:

1) Define congestion notification packets and generate them at the interior nodes
toward the egress. For example if the PSN is MPLS, then interior nodes may generate
a packet that has the same PSN header as the user packet (ie, same labels) but has
a PW or IP header that identifies it as a congestion notification packet. In fact this packet may look very similar to MPLS-ping (with different UDP port #). The egress, upon receiving the EFCN packet may generate a BECN packet toward the ingress. 

2) Use the NMS and the MIBs counters and have a threshold that triggers a message to all
ingress nodes and informs them of congestion in that node. The ingress nodes could then
use that info to either cut the excess traffic or reroute them. This method requires TE
be used but is more scalable than (1).

My 2c,

-Shahram

---

From: Stewart Bryant <[email-masked]>
Date: Thu, 13 Feb 2003 20:18:30 +0000

Philip Chimento wrote:
> Actually, I wasn't pointing out that these problems are non-trivial to 
> say that they can't
> be solved, just that they are hard, and that (I think) a different 
> approach from that of
> TCP needs to be considered. You can't make circuit emulation 
> "TCP-friendly" because of the characteristics that they have due to the 
> service they provide.
> 
> Neil's points about the management of different layers are well-taken, 
> but rather different points.
> I was just trying to start a discussion on the order of "what would a 
> good solution look like?"

I put out the original posting to solicit just that feedback, and then
when none was received put a baseline proposal in the arch draft.

Stewart

---

From: "White, Craig (Level 3)" <[email-masked]>
Date: Fri, 14 Feb 2003 13:58:26 -0700

Stewart, looks like you got your discussion :) I know it wasn't exactly what you're looking for, but hey....

Seems my way of viewing this issue is backwards from conventional views so bear with me here. By this I mean that I view congest avoidance as something that occurs "only" on the senders side, or in our case could potentially only effect 1 direction of the PW.

As you stated, we have to address the issue of Internet congestion if for no other reason than a procedural one. So....

My first question is "what exactly happens when "the Internet" congests"? 

When the "Internet congests", that means that some layer 1 has clocked out some transitions (that we may think of as containing our packet) and this packet will never again be clocked out by any other layer 1. This is not to say that all loss occurs on links, obviously there are issues that occur in a box (buffer overrun, (W)RED dropping) but in anycase, those packets never get clocked out again. 

Because that packet never gets clocked out again, then, obviously in certain cases, the higher layers (in that intermediate node) aren't necessarily alerted to this loss. Granted, some layer 2's framing methods may be able to detect and even ask for retransmission if the loss if it occurs on links of these certain types, but this is not true in all cases. 

Since the loss can occur in several places, which may even occur at several different layers, where does that leave us?

The important part about the implications of the above, is not exactly what happens, the important part is where something happens, or rather where it doesn't happen. The router that clocked out the packet that never arrives doesn't do something like send "back pressure" towards the edge, rather it relies on the receiver to somehow "signal" the sender  to adjust it's output rate. And while you could argue that (w)red is a "kind" of signaling, it effect is still on the edges rather than box/link that's experiencing the problem.

Furthermore, the TCP based transmitter doesn't require a positive message (I lost packet X) to cause it to stop transmitting, but also non-receipt will cause the same effect.

So for a tcp type connection, the sender "figures out" that further packet transmission is useless and then, several things occur, the most basic one being that the transmission stops, except for the periodic (re)sending of the packet that hasn't been ack'ed. Once ack'ed, TCP will "slow start" so as to not "reintroduce" the congestion, on the odd chance that this stream was causing the congestion.

So, the basic behavior that current Internet congestion mechanisms exhibit is that the *edge* stops sending traffic towards the congestion until the cognition event is over.

So, under "Internet congestion", it would seem that the behavior we're after is to stop transmitting towards the "Internet". After all, it makes no sense to even put the packets into flight if they are only going to get dropped before reaching the destination.

If we can somehow manage to figure out when the packets aren't reaching the other end and stop the engress PE from sending, the we would have the same behavior as TCP in the beginning, but being slow to start back doesn't seem to be an option for us. 

Further more, having a round trip based (probe?) restart method also doesn't seem to be particularly good. 

Seems like we have to come up with a way to tell the sending PE  to stop sending for some *small* amount of time, then blast away again. 

It would seems like a bad idea to just stop transmitting for 5-10 microseconds, but maybe not. What is the duration of  "Internet congestion"? We know that PW will not work with chronic packet loss (congestion),  so I'm thinking that a "fast start" mechanism is acceptable.

I'm thinking right now that only the PE has to stop sending towards the Internet, rather than push the congestion towards the CE. Although, given certain PW types, it might be nice to try to use the "native" layer 2 methods to push the congestion back towards the CE, but I don't think this is a requirement.

My questions today seem to revolve around the reaction of the customer application that's generating the PW packets on which the loss occurs. By this I mean if it's a TCP stream as PW payload would the TCP behavior be compromised/made worse if the PE quit sending towards the congestion.

Although, It would also seem that in the case that if it's a TCP stream, it's backoff behavior may have acted soon enough so that the PE would only have to stops the flow under severe congestion. If it's not a tcp stream, for this I tend to think in terms of a UDP or even ethernet based "broken" video app that basically just spews packets, in this case, the PE sending or not sending doesn't effect the end application. 

I would like to come up with a PW congestion behavior that works the same regardless of payload type (TCP/UDP, raw ethernet, etc)  thereby freeing us from the "payload" inspection idea. 

I also think that the "prior arrangement" method of classifying a stream "type" would be foolhardy at best, and most likely just simply futile, as least as a congestion avoidance .

So, how do we determine if packets aren't making it to the other end?

-craig

> From:	Stewart Bryant [SMTP:[email-masked]]
> Sent:	Thursday, February 13, 2003 1:07 PM
> To:	[email-masked]
> Cc:	[email-masked]; [email-masked]; [email-masked]
> ...and I think that the architecture uses this functional partitioning
> because the only sub-system that is allowed to modulate (by any method)
> the payload flow is the NSP, which is part of what you call the client
> network.
> 
> You don't. You set a congestion threshold and if that threshold is
> exceeded you tear down the PW.
> 
> Remember the IESG is not going to approve PW unless there is a method
> of protecting the Internet from congestion.
> 
> Stewart
> 
> 
>

---

From: Shahram Davari <[email-masked]>
Date: Fri, 14 Feb 2003 13:37:29 -0800

> So, how do we determine if packets aren't making it to the other end?

The obvious way to do it is to use seq #. But IP/MPLS have no seq #.
As I said in an earlier email, why not ask the node that is dropping
a packet to generate a Congestion notification packet for that flow?

Yours,
-Shahram

---

From: Stewart Bryant <[email-masked]>
Date: Fri, 14 Feb 2003 22:03:55 +0000

Shahram

> The obvious way to do it is to use seq #. But IP/MPLS have no seq #.

That is why the L2TPv3 and the MPLS CW have an optional sequence number,

> a packet to generate a Congestion notification packet for that flow?

Because this feature is not generally available, particularly
in the Internet core, and the PW has to run over the Internet.

Stewart

---

From: Shahram Davari <[email-masked]>
Date: Fri, 14 Feb 2003 14:10:45 -0800

Stewart,

> Sent: Friday, February 14, 2003 5:04 PM
> To: Shahram Davari
> Cc: 'White, Craig (Level 3)'; [email-masked];
> [email-masked]; [email-masked]; [email-masked]

In that case the seq # must become mandatory !

> in the Internet core, and the PW has to run over the Internet.

OK, I accept your argument that we don't want to retrofit all routers only
the edge ones.

Yours,
-Shahram

---

From: Stewart Bryant <[email-masked]>
Date: Fri, 14 Feb 2003 22:18:15 +0000

Shahram Davari wrote:
>>>>So, how do we determine if packets aren't making it to the 
>>>
>>other end?
>>
>>>

No, not if you have a well behaved payload.

---

From: Philip Chimento <[email-masked]>
Date: Fri, 14 Feb 2003 17:43:31 -0500

Craig: 

> When the "Internet congests", that means that some layer 1 has clocked out some transitions (that we may think of as containing our packet) and this packet will never again be clocked out by any ot…

It is important to note that the queue control mechanisms take 
advantage of TCP's "built-in" feedback mechanism which PWs don't 
necessarily have. RED is not particularly effective on uncontrolled UDP.

> So, under "Internet congestion", it would seem that the behavior we're after is to stop transmitting towards the "Internet". After all, it makes no sense to even put the packets into flight if they…

If you have no way of reducing the rate, then you had better stop 
transmitting.

> If we can somehow manage to figure out when the packets aren't reaching the other end and stop the engress PE from sending, the we would have the same behavior as TCP in the beginning, but being sl…

Historically, there are lots of ways to do this. The issue to discuss 
is how, and what overhead/additional traffic are you willing to 
tolerate.

> Seems like we have to come up with a way to tell the sending PE  to stop sending for some *small* amount of time, then blast away again. 

No, actually, you don't want to do this. This is just the behavior that 
slow start is trying to avoid. You don't want to make it worse...
 
> It would seems like a bad idea to just stop transmitting for 5-10 microseconds, but maybe not. What is the duration of  "Internet congestion"? We know that PW will not work with chronic packet loss…

Basically, the duration is on the order of RTTs of the TCP sessions on 
the bottleneck link. (That is assuming that the traffic on any link is 
composed mostly of TCP, which is mostly true, I think.) That can be far 
far more than a few microseconds.

> I'm thinking right now that only the PE has to stop sending towards the Internet, rather than push the congestion towards the CE. Although, given certain PW types, it might be nice to try to use th…

How much loss can your service tolerate and for how long ? Sometimes it 
is better just to terminate the connection....

> I would like to come up with a PW congestion behavior that works the same regardless of payload type (TCP/UDP, raw ethernet, etc)  thereby freeing us from the "payload" inspection idea. 

I think that the behavior should be pretty much service-dependent and 
that you do have to classify. Different types of sources can have very 
different requirements and characteristics...

Regards, Phil Chimento

---

From: Sasha Vainshtein <[email-masked]>
Date: Sun, 16 Feb 2003 10:09:42 +0200

Shahram and all,

With best regards,
                                   Sasha Vainshtein

> From: Shahram Davari [mailto:[email-masked]]
> Sent: Friday, February 14, 2003 11:37 PM
> To: 'White, Craig (Level 3)'; [email-masked]; 
> [email-masked]
> Cc: [email-masked]; Sasha Vainshtein; [email-masked]
> Subject: RE: [PWE3] PWE3 congestion
> 
> 
Mainly because the router dropping a packet due to congestion does not know 
if it is a PW packet or something else (does not recognize any "flows" in
your
terms.)  This knowledge is only in the PEs. 
To the best of my knowledge recognition of flows by intermediate routers
has been the original IntServ idea - and it is its proven non-scalability
that
prevented usage of IntServ. 
I believe that we have discussed domething like that before on this list...

---

From: Shahram Davari <[email-masked]>
Date: Mon, 17 Feb 2003 06:10:34 -0800

Sasha,

> From: Sasha Vainshtein [mailto:[email-masked]]
> Sent: Sunday, February 16, 2003 3:10 AM
> To: Shahram Davari; 'White, Craig (Level 3)'; [email-masked];
> [email-masked]
> Cc: [email-masked]; [email-masked]

Agree. But the egress has knowledge of the payload type and could
ignore congestion notification packets that don't belong to a PW.

Yours,
-Shahram

---

From: Sasha Vainshtein <[email-masked]>
Date: Mon, 17 Feb 2003 16:17:45 +0200

Shahram,
Looks interesting. 
If I understand your suggestion correctly:
1) An intermediate router must know how to send a congestion notification
packet 
    back to the ingess PE:
    a) Trivial over IP
    b) May be not so trivial over MPLS (LSPs are unidirectional)
2) The congestion notification packet must contain the information that
would
    allow the ingress PE to find the associated PW. This may be problematic
in case
    of hierarchical tunnels (IP or MPLS). I am not sure it is possible at
all in case
    of an IPSec tunnel (but then I am not an IPSec expert - rather, the
opposite).
3) If we solve (1) and (2), your suggestion will work.
 
With best regards,
                                   Sasha Vainshtein

> Sent: Monday, February 17, 2003 4:11 PM
> To: Sasha Vainshtein; 'White, Craig (Level 3)'; 
> [email-masked]; [email-masked]

---

From: Shahram Davari <[email-masked]>
Date: Mon, 17 Feb 2003 06:24:32 -0800

Sasha,

> Sent: Monday, February 17, 2003 9:18 AM
> To: Shahram Davari
> Cc: [email-masked]; [email-masked]; 'White, Craig (Level 3)';
> [email-masked]; [email-masked]

Unfortunately you have not understood it correctly. What I said was that an interior node
may generate in the forward direction a forward congestion notification toward the egress PE, by appending the user-data header (e.g., MPLS labels) . The egress
PE could then generate a backward congestion notification toward ingress PE.

Yours,
-Shahram

---

From: Sasha Vainshtein <[email-masked]>
Date: Mon, 17 Feb 2003 16:30:42 +0200

Shahram,
Now I see.
Thank you for the explanation.

With best regards,
                                   Sasha Vainshtein

> Sent: Monday, February 17, 2003 4:25 PM
> To: Sasha Vainshtein
> Cc: [email-masked]; [email-masked]; 'White, Craig 
> (Level 3)'; [email-masked]; [email-masked]
> may generate in the forward direction a forward congestion 
> notification toward the egress PE, by appending the user-data 
> header (e.g., MPLS labels) . The egress

---

From: Stewart Bryant <[email-masked]>
Date: Mon, 17 Feb 2003 14:53:44 +0000

> ignore congestion notification packets that don't belong to a PW.

What generates these CN packets?

Stewart

---

From: Shahram Davari <[email-masked]>
Date: Mon, 17 Feb 2003 06:57:03 -0800

> What generates these CN packets?

An interior node that detects congestion.

-Shahram

---

From: Stewart Bryant <[email-masked]>
Date: Mon, 17 Feb 2003 14:57:02 +0000

> 3) If we solve (1) and (2), your suggestion will work.

Let's stay real. Routers don't do this today and a solution that requires
a redesign of the Internet is not going to fly.

Stewart

>>>>The obvious way to do it is to use seq #. But IP/MPLS 
>>>
>>have no seq #.
>>
>>>>As I said in an earlier email, why not ask the node that 
>>>
>>is dropping
>>
>>>>a packet to generate a Congestion notification packet for 
>>>
>>that flow?
>>

---

From: Shahram Davari <[email-masked]>
Date: Mon, 17 Feb 2003 07:21:54 -0800

Hi,

Stewart is correct. The forward/backward congestion notification method
that I suggested requires retrofitting the internet. The sequence number
seems to be the best method to detect congestion loss without needing to
change anything in the interior nodes. 

May I suggest then that a single flag in the PW header be used to indicate
congestion in the forward direction. In other words the egress, upon detection 
of persistent packet loss, set a flag in the reverse PW direction header (assuming
bidirectional PW).

Yours,
-Shahram

> From: Shahram Davari 
> Sent: Monday, February 17, 2003 9:57 AM
> To: [email-masked]'
> Cc: 'Sasha Vainshtein'; 'White, Craig (Level 3)';
> [email-masked]; [email-masked]; [email-masked]

---

From: Alia Atlas <[email-masked]>
Date: Mon, 17 Feb 2003 10:44:50 -0500

Hi,

What does the ingress do when it receives an indication that the PW has 
experienced congestion?

Also, what level over what time period of missing sequence numbers would 
trigger such an indication?

Alia

---

From: Shahram Davari <[email-masked]>
Date: Mon, 17 Feb 2003 07:50:50 -0800

Hi Alia,

Good questions. Please see below.

Yours,
-Shahram

> From: Alia Atlas [mailto:[email-masked]]
> Sent: Monday, February 17, 2003 10:45 AM
> To: Shahram Davari
> Cc: [email-masked]'; 'Sasha Vainshtein'; 'White, Craig 
> (Level 3)';

I think the ingress should either cut the excess traffic (the traffic
that is not guaranteed), or reroute the traffic. What do you think
should be done?

> trigger such an indication?

Good question. This should be debated on the list. I have no
particular suggestion on this at the moment.

> >of persistent packet loss, set a flag in the reverse PW 
> direction header 
> >(assuming
> >bidirectional PW).
> >
> >Yours,
> >-Shahram
> >

---

From: "Daniel Feldman" <[email-masked]>
Date: Mon, 17 Feb 2003 17:57:49 +0200

Hi Shahram and all,
	sorry for the silly question but: isn't forward/backward
congestion notification the idea behind IP ECN?

> Sent: Monday, February 17, 2003 5:22 PM
> To: [email-masked]'
> direction header (assuming
> bidirectional PW).
> 
> Yours,
> -Shahram
>

---

From: Shahram Davari <[email-masked]>
Date: Mon, 17 Feb 2003 08:03:18 -0800

Daniel,
 
Your question is very valid (not silly). You are correct, the IP ECN was designed to do only the forward
congestion notification (not backward), but unfortunately is not a standard. I guess, if ECN is implemented, it could be used
for the forward congestion indication.
 
Yours,
-Shahram

-----Original Message-----
From: Daniel Feldman [mailto:[email-masked]]
Sent: Monday, February 17, 2003 10:58 AM
To: Shahram Davari; [email-masked]
Cc: Sasha Vainshtein; White, Craig (Level 3); [email-masked]; [email-masked]; [email-masked]
Subject: RE: [PWE3] PWE3 congestion

        Hi Shahram and all, 
        sorry for the silly question but: isn't forward/backward congestion notification the idea behind IP ECN? 

> From: Shahram Davari [ mailto:[email-masked] <mailto:[email-masked]> ] 
> > https://www1.ietf.org/mailman/listinfo/pwe3 <https://www1.ietf.org/mailman/listinfo/pwe3>  
> > 
> https://www1.ietf.org/mailman/listinfo/pwe3 <https://www1.ietf.org/mailman/listinfo/pwe3>  
>

---

From: Sasha Vainshtein <[email-masked]>
Date: Mon, 17 Feb 2003 18:44:48 +0200

Shahram and Daniel,
RFC 3168 "The Addition of Explicit Congestion Notification (ECN) to IP" is a
Proposed Standard. And it is supported, e.g., by the Cisco IOS 12.2(8) for
quite a bunch of platforms. (I do not know to what extent it is actually
deployed).
All the standing proposals for TDM circuits emulation support a dedicated
flag that allows reporting back lost packets. It can be also used to report
packets received with the EC (experienced congestion) indication. The
ingress could then use this information.
So it looks like Shahram's proposal for PWs carrying TDM circuits over
ECN-enabled IP networks is quite realistic.
This still leaves a problem of non-TDM PWs (there is no flag and, what's
more, there may be no packet to send back to carry the backward
notification) and PWs over MPLS (no ECN - you can only look at the actually
dropped packets). 
 
With best regards,
                                   Sasha Vainshtein

-----Original Message-----
From: Shahram Davari [mailto:[email-masked]]
Sent: Monday, February 17, 2003 6:03 PM
To: 'Daniel Feldman'; [email-masked]
Cc: Sasha Vainshtein; White, Craig (Level 3); [email-masked];
[email-masked]; [email-masked]
Subject: RE: [PWE3] PWE3 congestion

Daniel,
 
Your question is very valid (not silly). You are correct, the IP ECN was
designed to do only the forward
congestion notification (not backward), but unfortunately is not a standard.
I guess, if ECN is implemented, it could be used
for the forward congestion indication.
 
Yours,
-Shahram

-----Original Message-----
From: Daniel Feldman [mailto:[email-masked]]
Sent: Monday, February 17, 2003 10:58 AM
To: Shahram Davari; [email-masked]
Cc: Sasha Vainshtein; White, Craig (Level 3); [email-masked];
[email-masked]; [email-masked]
Subject: RE: [PWE3] PWE3 congestion

        Hi Shahram and all, 
        sorry for the silly question but: isn't forward/backward congestion
notification the idea behind IP ECN? 

> > An interior node that detects congestion. 
<https://www1.ietf.org/mailman/listinfo/pwe3>  
<https://www1.ietf.org/mailman/listinfo/pwe3>

---

From: Shahram Davari <[email-masked]>
Date: Mon, 17 Feb 2003 08:52:02 -0800

Sasha,
 
Agree with you.
 
-Shahram

-----Original Message-----
From: Sasha Vainshtein [mailto:[email-masked]]
Sent: Monday, February 17, 2003 11:45 AM
To: Shahram Davari; 'Daniel Feldman'
Cc: White, Craig (Level 3); [email-masked]; [email-masked]; [email-masked]; [email-masked]
Subject: RE: [PWE3] PWE3 congestion

Shahram and Daniel,
RFC 3168 "The Addition of Explicit Congestion Notification (ECN) to IP" is a Proposed Standard. And it is supported, e.g., by the Cisco IOS 12.2(8) for quite a bunch of platforms. (I do not know to what extent it is actually deployed).
All the standing proposals for TDM circuits emulation support a dedicated flag that allows reporting back lost packets. It can be also used to report packets received with the EC (experienced congestion) indication. The ingress could then use this information.
So it looks like Shahram's proposal for PWs carrying TDM circuits over ECN-enabled IP networks is quite realistic.
This still leaves a problem of non-TDM PWs (there is no flag and, what's more, there may be no packet to send back to carry the backward notification) and PWs over MPLS (no ECN - you can only look at the actually dropped packets). 
 
With best regards,
                                   Sasha Vainshtein

-----Original Message-----
From: Shahram Davari [mailto:[email-masked]]
Sent: Monday, February 17, 2003 6:03 PM
To: 'Daniel Feldman'; [email-masked]
Cc: Sasha Vainshtein; White, Craig (Level 3); [email-masked]; [email-masked]; [email-masked]
Subject: RE: [PWE3] PWE3 congestion

Daniel,

Yours,
-Shahram

-----Original Message-----
From: Daniel Feldman [mailto:[email-masked]]
Sent: Monday, February 17, 2003 10:58 AM
To: Shahram Davari; [email-masked]
Cc: Sasha Vainshtein; White, Craig (Level 3); [email-masked]; [email-masked]; [email-masked]
Subject: RE: [PWE3] PWE3 congestion

        Hi Shahram and all,

---

From: "Daniel Feldman" <[email-masked]>
Date: Mon, 17 Feb 2003 18:53:54 +0200

Sasha, Shahram, Lloyd et al.,
    I think we all agreed ECN for forward congestion notification is
beneficial, even though not a solution for all the cases. I am for
adopting Sasha's proposal for TDM circuits over IP PSNs. Regarding MPLS,
anyone knows whatever happened to the experimental bits ;-).
    For asymmetric flows...

-----Original Message-----
From: Sasha Vainshtein [mailto:[email-masked]]
Sent: Monday, February 17, 2003 6:45 PM
To: 'Shahram Davari'; Daniel Feldman
Cc: White, Craig (Level 3); [email-masked];
[email-masked]; [email-masked]; [email-masked]
Subject: RE: [PWE3] PWE3 congestion

Shahram and Daniel,
RFC 3168 "The Addition of Explicit Congestion Notification (ECN) to IP"
is a Proposed Standard. And it is supported, e.g., by the Cisco IOS
12.2(8) for quite a bunch of platforms. (I do not know to what extent it
is actually deployed).
All the standing proposals for TDM circuits emulation support a
dedicated flag that allows reporting back lost packets. It can be also
used to report packets received with the EC (experienced congestion)
indication. The ingress could then use this information.
So it looks like Shahram's proposal for PWs carrying TDM circuits over
ECN-enabled IP networks is quite realistic.
This still leaves a problem of non-TDM PWs (there is no flag and, what's
more, there may be no packet to send back to carry the backward
notification) and PWs over MPLS (no ECN - you can only look at the
actually dropped packets). 
 
With best regards,
                                   Sasha Vainshtein

-----Original Message-----
From: Shahram Davari [mailto:[email-masked]]
Sent: Monday, February 17, 2003 6:03 PM
To: 'Daniel Feldman'; [email-masked]
Cc: Sasha Vainshtein; White, Craig (Level 3); [email-masked];

Daniel,
 
Your question is very valid (not silly). You are correct, the IP ECN was
designed to do only the forward
congestion notification (not backward), but unfortunately is not a
standard. I guess, if ECN is implemented, it could be used
for the forward congestion indication.
 
Yours,
-Shahram

-----Original Message-----
From: Daniel Feldman [mailto:[email-masked]]
Sent: Monday, February 17, 2003 10:58 AM
To: Shahram Davari; [email-masked]
Cc: Sasha Vainshtein; White, Craig (Level 3); [email-masked];

        Hi Shahram and all,

---

From: Chris Metz <[email-masked]>
Date: Mon, 17 Feb 2003 12:16:32 -0500

Couple of observations/thoughts on the notion of PWE3 congestion:

- "realness" principle states no change to interior nodes for the purpose 
of detecting/signaling explicit congestion of the PSN. This implies that 
any feedback loop is directly between the PEs with no explicit input from 
the interior nodes

- implication that PWE3 CC be applied based on a priori knowledge of PW 
payload ("is it carrying TCP or not?") seems problematic. Is it verifiable? 
I don't think we want to get into classifying packets on the ingress PE 
based on transport layer port # to determine if/when PWE3 CC should be applied.

- At PW provisioning time (or via CE signaling) we can get a pretty good 
idea about the type of traffic that the PW is carrying. If it is a UBR or 
ABR-style PW then we know that we can reduce the sending rate with the 
impact on the application being within the expectation level made known to 
the customer. If it is a CBR-style connection then we need to engineer the 
network accordingly. If we can't then one should not attempt to 
sell/support this service. What happens to CBR-style PWs if congestion 
occurs? Something is broken (as far as the customer is concerned) and we 
should tear it down and engage in damage control with the customer.

That said is there some way to re-use existing (and to date effective) TCP 
congestion control machinery to provide PWE3 CC feedback?

- Is it possible to utilize TCP-style CC to determine if there is 
congestion somewhere between the PEs? There is in the case of LDP a TCP 
stack on the PEs. One could inject/interleave TCP packets along the PW path 
(same label stack for example) at a some rate proportional to the 
configured data rate of the PW. Analysis of the TCP feedback by ingress PE 
would indicate if congestion was present. From there appropriate actions on 
the part of the ingress PE could be taken.

Just thinking outside the box :-)

Cheers ...

---

From: Alia Atlas <[email-masked]>
Date: Mon, 17 Feb 2003 12:15:02 -0500

The mechanisms for ECN might help as far as deciding to indicate
congestion, but PWs don't necessarily have an IP header... and if they
did, it's buried underneath a random number of MPLS labels and, more
critically, a control word.

Currently, I don't see that ECN is defined to work with MPLS, and we only
have 3 bits for EXP.

We could clearly define an extension to a DIFFSERV object to specify that
one of the EXP bits is for ECN...  Then this would need to be
signaled on the LSPs used in the PSN, not the pseudo-wire.

Alia

At 08:03 AM 2/17/2003 -0800, Shahram Davari wrote:

Daniel,

Your question is very valid
(not silly). You are correct, the IP ECN was designed to do only the
forward

congestion notification (not
backward), but unfortunately is not a standard. I guess, if ECN is
implemented, it could be used

for the forward congestion
indication.

Yours,

-Shahram

-----Original
Message-----

From: Daniel Feldman
[mailto:[email-masked]]

Cc: Sasha Vainshtein; White, Craig (Level 3);
[email-masked]; [email-masked]; [email-masked]

Hi Shahram
and all,

sorry for the
silly question but: isn't forward/backward congestion notification the
idea behind IP ECN?

> From: Shahram Davari
[mailto:[email-masked]]

> Stewart is correct. The forward/backward congestion

> notification method

> that I suggested requires retrofitting the internet.
The

> sequence number

> seems to be the best method to detect congestion loss
without

> needing to

> May I suggest then that a single flag in the PW header
be

> used to indicate

> congestion in the forward direction. In other words the

> egress, upon detection

> > > > Agree. But the egress has knowledge of the payload type

> and could

> > > > ignore congestion notification packets that don't

> belong to a PW.

---

From: Alia Atlas <[email-masked]>
Date: Mon, 17 Feb 2003 12:23:28 -0500

Hi Shahram,

>that is not guaranteed), or reroute the traffic. What do you think

Well, it seems to me that one wants levels on how much the excess traffic 
is cut.  Otherwise, you don't have the equivalent of a slow-start...
So, I could see having a policer with a rate and burst.  The rate could be 
the minimum committed.  The burst could be adjusted among a set of values, 
going down when a congestion indication is received in a set time window, 
and up when one isn't...  Of course, determining the window to detect in 
would be a little entertaining...
Conceptually, say we have an array of bursts[16], ordered in size from 
lowest to highest.  If a congestion indication is received, then the 
burstIndex is divided by 2.  If no congestion indication is received, then 
the burstIndex is incremented by 1.

The question is what time-frame this would have to be done on and for how 
many PWs.

What do you think?

>particular suggestion on this at the moment.

The trick is finding something fast enough to deal with congestion and not 
so fast that it can't be handled for thousands of PWs.  Regrettably, to 
interact and respond similarly to TCP streams, this might mean a window of 
about 100ms... but, like everything else, I'm sure it could be configurable.

Alia

---

From: Sasha Vainshtein <[email-masked]>
Date: Mon, 17 Feb 2003 19:42:37 +0200

Chris and all,
Just a single brief comment inline...

With best regards,
                                   Sasha Vainshtein

> From: Chris Metz [mailto:[email-masked]]
> Sent: Monday, February 17, 2003 7:17 PM
> To: Lloyd Wood
> Cc: Shahram Davari; [email-masked]'; [email-masked]
... snippedm ...
> If it is a CBR-style connection then we need to 
> engineer the 
My impression has been that the ADs are not ready to accept this
approach. I assume that they consider the situation when a pair of
PEs actually belongs to the "end customer" who has decided to deploy
a CBR-like PW between them over the public Internet without explicitly
"buying" this service or "engineering" any network. And the problem 
is not the impact on the CBR-like PW (yes, it will not work!), but on 
the public Internet (lots of other things will stop working as well). 
... snipped to the end ...

---

From: Shahram Davari <[email-masked]>
Date: Mon, 17 Feb 2003 11:58:42 -0800

Stewart,

> 
> > Therefore a more generic
> > method that applies to all types of payloads should be used.
> 
> False - only to those payloads that don't have the TCP friendly
> property.
> 

Even if a PE knows that the payload is TCP, how do you guarantee that
the customer has intentionally or unintentionally implemented the TCP
back off. 

Yours,
-Shahram

---

From: Stewart Bryant <[email-masked]>
Date: Mon, 17 Feb 2003 20:20:24 +0000

> An interior node that detects congestion.

But very few of the nodes in the Internet have this property,
and holding off deploying this service until they do is
unrealistic.

Stewart

---

From: Shahram Davari <[email-masked]>
Date: Mon, 17 Feb 2003 12:24:15 -0800

Agree.

-Shahram

> Sent: Monday, February 17, 2003 3:20 PM
> To: Shahram Davari

---

From: Stewart Bryant <[email-masked]>
Date: Mon, 17 Feb 2003 20:33:14 +0000

> What does the ingress do when it receives an indication that the PW has 

A suitable strategy might be:

1) Notify the NSP that the PW is experiencing congestion
2) Wait whilst the NSP takes some action to reduce the offered load
3) If congestion still experienced, shut down the PW

Step 2 gives PW services that are in a position to reduce offered load
some time to take action.

Step 3 ensured that PW services that are unable to back off their offered
load do eventually back off for the good of the other users.

> trigger such an indication?

I guess that we take guidance from the TCP control loop experts.

Stewart

---

From: Shahram Davari <[email-masked]>
Date: Mon, 17 Feb 2003 12:40:53 -0800

Lloyd,

> From: Lloyd Wood [mailto:[email-masked]]
> Sent: Monday, February 17, 2003 3:38 PM
> To: Shahram Davari
> Cc: [email-masked]'
> On Mon, 17 Feb 2003, Shahram Davari wrote:
> 
> > > > Therefore a more generic
> (TCP != TCP friendly congestion control, but you knew that, right?)

Right.

> 
> unintentionally implemented??

Sorry I meant unintentionally not implemented.

> 
> There's no guarantee that the TCP sender is doing TCP backoff etc. and
> playing friendly (and if it's an ancient purely RFC793-compliant
> pre-Jacobson-hack TCP it won't be) - this is the area of having to
> protect your receiver against Savage attacks.
> 

This was exactly my point. We should not rely on customer back off, even if
their traffic is TCP.

Thanks,
-Shahram

> L.
> 
> hey! I'm actually reading the shorter, edited, mails!
> 
> <http://www.ee.surrey.ac.uk/Personal/L.Wood/><[email-masked]>
>

---

From: Sasha Vainshtein <[email-masked]>
Date: Tue, 18 Feb 2003 16:30:16 +0200

Stewart and all,
It looks like we have two different cases to consider.

1. On one hand, we have the TDM PWs. This case can be described as
   following:
   a) Definitely TCP-like traffic behavior in the payload
   b) It is easy to detect lost packets at the PW egress, because:
      * All the proposed encapsulations assume sequencing
      * All the proposed encapsulations assume packets of fixed size
        and hence a constant average packet arrival rate (in other
        words, you do not have to wait for the next received packet
        - which may well happen once the congestion condition is already
        over -  to guess that you have meanwhile lost thousands of them)
   c) It is easy to send backward notification - all the proposed
      encapsulations reserve a dedicated flag in the control word
      exactly for these purposes
   d) It is easy to decide how to handle the congestion (since you 
      cannot back off gradually, all that remains is to shut down
      the service and restart it after some timeout)

2. On the other hand, we have all the rest of PWs. This case
   can be described like following:
   a) They may carry TCP-like traffic - and then, they may not  
      (BTW, is, say, SNA TCP-like? it is carried over Ethernet 
      and FR, and these can be carried over PWs). And, BTW, nothing
      prevents a mix of different types of traffic over a single
      FR DLCI being carried over a PW - that is what RFC 1490 and
      its updates are about... 
   b) It is difficult to detect lost packets, because:
      * Some of the proposed encapsulations do not use sequencing
      * Even with sequencing, you must wait for the next received
        packet to decide that you have lost some packets (and, if
        you use re-ordering, you may wait some more after that)
   c) None of the proposed encapsulations have flags for reporting
      lost packets back. And, what is even more important, you may
      not have an appropriate packet at hand to piggi-back such a flag;
      nor would you really know how to interpret it (i.e., you would
      probably like to know how many packets have been lost)
   d) Finally, you have a wide range of options for handling the detected
      congestion (provided, of course, that you have some deduced
      that a congestion is taking place), e.g.:
      * Relegate the responsibility to the NSP (if you have one
        and if you assume that it can handle the congestion)
      * Do some gradual back-off at the PW level and then try to
        come back
      * Shut down the service completely and then try to restart it
      (I do not know if this list is complete, but it is surely
      long enough to make the selection complicated).

Please note that, until now, the discussion for congestion detection and
handling concentrated on the capabilities of the PW data path. I believe
that this was partially due to the fact that, for TDM PWs (that natively
are the prime suspects for creating congestion) this seems an easy task;
and, partly, because PWE3 explicitly allows manual setup and teardown
of PWs.

However, shifting the congestion-related activities to the PWE3 control
plane seems to me worth considering for the following reasons:

1. Congestion condition could be easily deduced e.g., from the information
   about the nubmer of bytes send during a certain interval and
   information about the number of bytes received by the "remote"
   side for the same interval. The control protocol could allow exchange
   of this information in quite a generic way.
2. The congestion handling actions could be coordinated betwen the ends of
   the PW. E.g., if one PE decides to shut down a certain PW and then
restart 
   it after a certain period, such an action should be better synchronized
   between the two PW ends. I do not even want to think of possible
consequences
   of one PE taking one type of action (e.g., relegate to NSP( while the
other
   takes another action (e.g., shut down the connection). The control
   protocol could, hopefully, take care of this process also
3. Last but not least, should the control protocol session (be it LDP or
   L2TPv3) fail due to congestion, all the associated PWs would be 
   shut down as well.

Of course, congestion handling and detection using the control protocol
would be probably slower than handling it in the data path - but, maybe,
this could be tolerated at least in the cases when we do not see any clear
way for handling it in the data path? And the problem of manually configured
PWs remains in any case...

Hopefully these notes will be of some interest.

With best regards,
                                   Sasha Vainshtein

> Sent: Tuesday, February 18, 2003 10:14 AM
> To: Keith Knightson
> Cc: [email-masked]'
> Keith
> You are correct(ish) for TDM. Some TDM emulations can hid stream
> damage due to packet loss. However the congestion loss is the
> Internet's way of crying for help, and if the packet loss continues
> then the PEs have to reduce the offered load, which they can only do
> by shutting down one of more PWs.
> 
> Quite the opposite is true in the case of packet. Here packet
> loss in the PSN translates to packet loss in the emulated service
> and if the service is carrying TCP, TP4 or similar, then PSN loss
> will trigger exactly the right behaviour in the hosts ulimately
> generating the traffic. In such casses shutting down the service
> would be OTT.
> 
> Then there is ATM, which could be carrying anything. If we KNEW it
> was carrying packets, then we could be sure that the right things
> will happen, otherwise we have to start shutting down PWs. Perhaps
> we can operate packet strategy in STU mode and TDM strategy in cell
> mode?
> 
> Regards
> 
> Stewart
> 
> Keith Knightson wrote:
> > Guys,
> > 
> > If you are emulating TDM you cannot surely make any 
> assumptions at all 
> > about the protocols being used by the customer over the TDM.
> > 
> > Conversely, if the protocol on the CE-PE is known then TDM 
> across the 
> > core is not necessary, since this protocol can be extracted 
> from the TDM 
> > on the access and sent as IP payload across the core.
> > 
> > Surely the reason for a TDM PWE in the first place is to 
> provide a means 
> > of conveying a transparent TDM stream all the way across an IP core?
> > 
> > If the PSN underneath a TDM PWE becomes congested, such 
> that blocks of 
> > bits are lost, then the PEs will have to detect it, at the 
> PWE layer. I 
> > do not think that the underlying PSN can be relied upon to 
> perform that 
> > function. The PEs have to detect it, and take action, like 
> presumably 
> > turn off the clock towards the CE, or use OAM framing bits 
> or something 
> > similar. I would also have thought that "normal" TDM OAM 
> would have to 
> > be emulated across the core for this purpose.
> > 
> > All this talk about about customer back-off at the TCP level eludes 
> > me,..., why on earth would any  customer who is already operating 
> > already TCP over IP, want some emulated TDM PWE across an 
> IP core? Why 
> > is IP in IP not a better solution?
> > 
> > It seems to be that an emulated TDM PWE should be just that, ie an 
> > emulated TDM pipe. TDM is the PWE payload, not the payload 
> of the TDM. 
> > The payload of the TDM is unknown. Conversely, if it is 
> known then the 
> > TDM is not required, since it can be handled directly.
> > 
> > I must be missing something, brain cells probably.
> > 
> > Regards
> > 
> > Keith
> > 
> > 
> > 
> > 
> > At 11:58 AM -0800 17/2/03, Shahram Davari wrote:
> > 
> >> Stewart,
> >>
> >>>
> >>>  > Therefore a more generic

---

From: Chris Metz <[email-masked]>
Date: Tue, 18 Feb 2003 10:34:00 -0500

>    a) Definitely TCP-like traffic behavior in the payload

Sasha, do you mean definitely NOT TCP-like traffic behavior in the payload? 
I am not clear as to how one could make the assumption that a TDM circuit 
is carrying TCP.

Thanks for the clarification

<snip>

---

From: Sasha Vainshtein <[email-masked]>
Date: Tue, 18 Feb 2003 17:37:43 +0200

Chris,
Thanl you for correction. You are right, of course, 
the TDM payload is definitely NOT TCP-like.

With best regards,
                                   Sasha Vainshtein

> Sent: Tuesday, February 18, 2003 5:34 PM
> To: Sasha Vainshtein
> Cc: [email-masked]'; [email-masked]'; Keith Knightson; 
> Mark Townsley (E-mail); Luca (E-mail); Shahram Davari (E-mail)

---

From: "Busschbach, Peter B (Peter)" <[email-masked]>
Date: Tue, 18 Feb 2003 11:05:25 -0500

Keith, et.al.

Every time I read statements about "shutting down PWs" I shudder. In my view this whole discussion is based on the wrong premise, which is that congestion is bad and the network needs to solve that situation autonomously.

Why would congestion be bad? Isn't that up to the Service Provider to decide? In general, a Service Provider will use PWs to carry revenue generating services. Packet loss may be acceptable for certain services and not for others. If  packet loss is to be avoided, the Service Provider will need to make sure that it doesn't occur, e.g. through bandwidth reservation and rate limiting at the ingress of his network.

The network should never autonomously decide to throttle or shut down PWs.

Regards,

Peter 

-----Original Message-----
From: Stewart Bryant [mailto:[email-masked]]
Sent: Tuesday, February 18, 2003 3:14 AM
To: Keith Knightson
Cc: [email-masked]'
Subject: Re: [PWE3] PWE3 congestion

Keith

Regards

Stewart

Keith Knightson wrote:
> Guys,
> 
> If you are emulating TDM you cannot surely make any assumptions at all 
> Conversely, if the protocol on the CE-PE is known then TDM across the 
> core is not necessary, since this protocol can be extracted from the TDM 
> Surely the reason for a TDM PWE in the first place is to provide a means 
> If the PSN underneath a TDM PWE becomes congested, such that blocks of 
> bits are lost, then the PEs will have to detect it, at the PWE layer. I 
> do not think that the underlying PSN can be relied upon to perform that 
> function. The PEs have to detect it, and take action, like presumably 
> turn off the clock towards the CE, or use OAM framing bits or something 
> similar. I would also have thought that "normal" TDM OAM would have to 
> already TCP over IP, want some emulated TDM PWE across an IP core? Why 
> emulated TDM pipe. TDM is the PWE payload, not the payload of the TDM. 
> The payload of the TDM is unknown. Conversely, if it is known then the

---

From: "Busschbach, Peter B (Peter)" <[email-masked]>
Date: Tue, 18 Feb 2003 12:22:10 -0500

Lloyd,

I am still curious about your views on service delivery. Take a service provider who receives thousands of dollars a months for delivering a TDM service. How would you explain to him that the service is shut down, because "the NSP judges likely services using the PW can't be usefully delivered"

I still maintain that shutting down a service is not an option for many services carried over PWs. A service provider will need to take action beforehand (i.e. reserve bandwidth for that service) to ensure that congestion does not occur.

But if you ignore the commercial aspects of telecommunication, I agree that "Tuning specific NSP types for the traffic/services they're carrying could be a whole ball of implementation-experience fun."

Peter

-----Original Message-----
From: Lloyd Wood [mailto:[email-masked]]
Sent: Tuesday, February 18, 2003 11:36 AM
To: Busschbach, Peter B (Peter)
Cc: [email-masked]'
Subject: RE: [PWE3] PWE3 congestion

On Tue, 18 Feb 2003, Busschbach, Peter B (Peter) wrote:

> Every time I read statements about "shutting down PWs" I shudder. In
> my view this whole discussion is based on the wrong premise, which
> is that congestion is bad and the network needs to solve that
> situation autonomously.
>
> Why would congestion be bad? Isn't that up to the Service Provider
> to decide?

This is, alas, the Internet Engineering Taskforce.

Not the Service Provider Engineering Taskforce (aka the mpls forum).

> In general, a Service Provider will use PWs to carry
> revenue generating services. Packet loss may be acceptable for
> certain services and not for others. If packet loss is to be
> avoided, the Service Provider will need to make sure that it doesn't
> occur, e.g. through bandwidth reservation and rate limiting at the
> ingress of his network.

shutting down PWs *is* very simple rate limiting at ingress. Ingress
to the Internet...

> The network should never autonomously decide to throttle or shut down PWs.

The (inter)network doesn't make the decision; throttling/loss happens
under congestion as a byproduct anyway. The decision to shut down a PW
gets made by the NSP if the NSP judges likely services using the PW
can't be usefully delivered (e.g. TDM rates not being met).

It's not a network decision, because the network is stupid. It's an
NSP decision, once you've set your NSP for the PW type and the
expected traffic/services over that.

Tuning specific NSP types for the traffic/services they're carrying
could be a whole ball of implementation-experience fun.

L.

snipped four list sigs.

---

From: Stewart Bryant <[email-masked]>
Date: Tue, 18 Feb 2003 18:56:40 +0000

Busschbach, Peter B (Peter) wrote:
> my view this whole discussion is based on the wrong premise, which is 
> that congestion is bad and the network needs to solve that situation 
> autonomously.

Well congestion is certainly not good and can lead to the collapse of
the network if it is not addressed. At some level you need a safetly
valve.

> 
> Why would congestion be bad? 

Because the Internet has a single data-plane used to carry both control
and user traffic. Aside from any issues of fairness, there is the need
to get routing packets through the network. A network that is so congested
that the routing messages get discarded rapidly becomes unstable and
collapses.

> Isn't that up to the Service Provider to 
> decide? In general, a Service Provider will use PWs to carry revenue
> generating services. Packet loss may be acceptable for certain services 
> and not for others. If  packet loss is to be avoided, the Service Provider 
> will need to make sure that it doesn't occur, e.g. through bandwidth 
> reservation and rate limiting at the ingress of his network.

The SP can try their best, but what happens when there is a failure, or
a missconfiguration?

> The network should never autonomously decide to throttle or shut down PWs.

But it always will. Eventually congestion will get to the point where the
routers discard packets.

Stewart

---

From: Stewart Bryant <[email-masked]>
Date: Tue, 18 Feb 2003 19:10:10 +0000

Busschbach, Peter B (Peter) wrote:
> Lloyd,
> 
> I am still curious about your views on service delivery. Take a service 
> provider who receives thousands of dollars a months for delivering a TDM 
> service. How would you explain to him that the service is shut down, 
> because "the NSP judges likely services using the PW can't be usefully delivered"

Better to shut down the PW and save the network than loose the whole
network.

> 
> I still maintain that shutting down a service is not an option for many 
> services carried over PWs. A service provider will need to take action 
> beforehand (i.e. reserve bandwidth for that service) to ensure that congestion 
 > does not occur.

I guess that I could accept that no congestion avoidance scheme is needed
in a TE network, but the majority of networks are not TE. In a non-TE network,
having a congestion discard scheme is an important safety valve.

> 
> But if you ignore the commercial aspects of telecommunication, I agree 
> that "Tuning specific NSP types for the traffic/services they're 
> carrying could be a whole ball of implementation-experience fun."
> 

Stewart

---

From: [email-masked]
Date: Tue, 18 Feb 2003 22:15:57 -0000

Stewart Bryant wrote 18 February 2003 18:57
<snip>
> and user traffic.
NH=> Stewart, OK.....fine for me with a cnls mode, but can you therefore
please explain what GMPLS is about?  This is a co cct-sw mode.  I assume
from this statement GMPLS has no place in the "Internet" because the
traffic's data-plane and the control's (and/or management) data-plane are
disjoint networks in all functional senses.

regards, Neil

---

From: [email-masked]
Date: Tue, 18 Feb 2003 22:39:03 -0000

Opinion or fact?

neil

> Sent: 18 February 2003 22:32
> To: Harrison,N,Neil,IKL2 R
> Cc: [email-masked]; ; 
> On Tue, 18 Feb 2003 [email-masked] wrote:
> > I assume
> > from this statement GMPLS has no place in the "Internet" because
> 
> ...it's not IP.
> 
> L.
>

---

From: "Busschbach, Peter B (Peter)" <[email-masked]>
Date: Tue, 18 Feb 2003 18:00:14 -0500

Lloyd,

The charter of the PWE3 WG talks about service emulation over a PSN. Therefore, I thought that this WG was focused on the creation of the mechanism and protocols required for an adequate delivery of these services. I hope this answers your question about what I think the IETF _does_

>From an end-user perspective, shutting down a service is highly undesirable. I agree that excessive packet loss is undesirable also, but I would think that the consequent actions should be focused on restoring an acceptable service level, e.g. through some form of rerouting or protection switching.

Stewart wrote in an earlier email: packet "loss is the Internet's way of crying for help" and he argued for the need of a safety valve. This might be a valid point for the Internet at large. And to your point: if a Service Provider decides to offer PW services over The Internet, then he might need to accept that the service can be shut down.

However, the charter of the WG doesn't mention The Internet. It mentions PSNs. I think you need to be very careful to mandate such safety valves for PSNs in general. E.g. a PSN might be under complete control of one single Service Provider and hence, for whatever reason, a certain level of congestion might be perfectly fine. Also, for whatever reason, a Service Provider might prefer to keep a PW up, even if packet loss is too high and rerouting is not possible, rather than to shut it down.

To come back to my original point when I joined this email threat: I think the discussion should not be: how can we lower the traffic load in the network in case of congestion, e.g. by shutting down PWs. It should be focused on: what tools does a Service Provider need to manage the total portfolio of services he supports over his PSN in a manner that meets his business needs.

Peter

-----Original Message-----
From: Lloyd Wood [mailto:[email-masked]]
Sent: Tuesday, February 18, 2003 1:03 PM
To: Busschbach, Peter B (Peter)
Cc: [email-masked]'
Subject: RE: [PWE3] PWE3 congestion

On Tue, 18 Feb 2003, Busschbach, Peter B (Peter) wrote:

> I am still curious about your views on service delivery. Take a
> service provider who receives thousands of dollars a months for
> delivering a TDM service. How would you explain to him that the
> service is shut down, because "the NSP judges likely services using
> the PW can't be usefully delivered"

Well, I'd explain that if the service hadn't been shut down, it would
be unuseable for its intended traffic. I'd explain that the network
path the service was carried across had not been dimensioned/selected
appropriately for its intended traffic -- or was being used for
entirely other traffic than agreed in the SLA with the customer and
which the NSP was configured for.

Such explanations and implementation specifics are outside the scope
of PWE3, and outside the scope of the IETF.

The service provider's job is to understand the tools he uses to
deliver the solution to his customers: their uses, their limitations,
so that he knows that the service he is selling can be reasonably
provided and guaranteed. (Which is why they build test networks, etc.)
This is why the service provider receives those thousands of dollars a
month.

I am curious about what you think the IETF _does_.

L.

> I still maintain that shutting down a service is not an option for
> many services carried over PWs. A service provider will need to take
> action beforehand (i.e. reserve bandwidth for that service) to
> ensure that congestion does not occur.
>
> But if you ignore the commercial aspects of telecommunication, I
> agree that "Tuning specific NSP types for the traffic/services
> they're carrying could be a whole ball of implementation-experience
> fun."
>
> Peter
>

---

From: Stewart Bryant <[email-masked]>
Date: Wed, 19 Feb 2003 09:24:18 +0000

I completely agree with this position

Stewart

Eduard Metz wrote:
> <snipped>
> 
>>From an end-user perspective, shutting down a service is
>>highly undesirable. I agree that excessive packet loss is
>>undesirable also, but I would think that the consequent
>>actions should be focused on restoring an acceptable service
>>level, e.g. through some form of rerouting or protection switching.
>>
> 
> 
> For some services, e.g. TDM, the user-perception of excessive packet-loss
> and a shut down is likely to be the same or similar. Given that some PW
> services do not tolerate loss very well, shutting down some of these in a
> PSN may be the only way to restore acceptable service levels for the
> remaining PW services of the same nature.
> 
> <snipped>
> 
>>However, the charter of the WG doesn't mention The Internet.
>>It mentions PSNs. I think you need to be very careful to
>>mandate such safety valves for PSNs in general. E.g. a PSN
>>might be under complete control of one single Service
>>Provider and hence, for whatever reason, a certain level of
>>congestion might be perfectly fine. Also, for whatever
>>reason, a Service Provider might prefer to keep a PW up, even
>>if packet loss is too high and rerouting is not possible,
>>rather than to shut it down.
>>
> 
> 
> Would you prefer a situation in which the average packet loss per PW is low,
> but the service degradation of all PW may be high, over a situation in which
> some PW are shut, and the remaining are operating normally?
> 
> 
>>To come back to my original point when I joined this email
>>threat: I think the discussion should not be: how can we
>>lower the traffic load in the network in case of congestion,
>>e.g. by shutting down PWs. It should be focused on: what
>>tools does a Service Provider need to manage the total
>>portfolio of services he supports over his PSN in a manner
>>that meets his business needs.
>>
> 
> 
> IMHO congestion in the end will conflict with business needs, and thus needs
> to be relieved. [I think there is at least some basic agreement here ;-)]
> 
> Ofcourse measures should be taken to prevent congestion in the first place,
> however, in case of network failures (link, node) congestion may still
> occur. In these situations mechanisms need to be availabe to relieve
> congestion (aka restore acceptable service levels, meet business needs),
> also for PW.
> 
> Which mechanism to choose for which PW type, when/how these mechanisms
> kick-in (and 'kick-out' for that matter) are IMHO the set of "tools" that
> are within the scope PWE3 (tools for the remainder service portfolio are
> clearly not). I'm curious to which other 'tools' you envision.
> 
> cheers,
> 	Eduard
> 
> 
>>Peter
>>

<snip>

---

From: [email-masked]
Date: Wed, 19 Feb 2003 11:35:32 -0000

Stewart,

Stewart Bryant wrote 19 February 2003 09:18

<snipped>
> >>From an end-user perspective, shutting down a service is highly 
> > undesirable. I agree that excessive packet loss is 
> undesirable also, 
> > but I would think that the consequent actions should be focused on 
>  > restoring an acceptable service level, e.g. through some form of
> > rerouting or protection switching.
> 
> So are you advocating that the only PSN that we specify is MPLS-TE?

NH= Yes and no.  There are only certain client/server relationships between
the 3 network modes of cnls, co pkt-sw and co cct-sw that make sense.
-	a cnls client can use any of the modes as server
-	a co pkt-sw client should only use a co pkt-sw or a co cct-sw mode
as server
-	a co cct-sw client should only use a co cct-sw server.
Any other relationships lead to one or more functional management problems,
eg timing recovery, performance-management, fault-management,
traffic-management, etc.  However, even though our architectural policy
states might state this I would not stop anyone else from doing crazy things
if they want....we all have choices and sometimes learning from (bad)
experience is the only way.  Those who have children will immediately
recognise what I mean here.

So the trite answer is 'yes' but whether one would use a given technology
would be an operators choice....it would depend on the functional 'quality'
of the technolgy and fit for purpose....some are better than others.

<snip>
> But the concern of the IESG is that PWE3 can and will get connected to
> the Internet, and if and when that happens, it must not break 
> the Internet.
NH=> Which particular 'Internet' are you/IESG talking about here Stewart?
Is it:
-	the classical IP cnls variety?
-	the MPLS co pkt-sw variety?
-	or the GMPLS (ie SDH, OTN) co cct-sw variety?
Seems to me the 'Internet' means different things to different people.....or
have I not grasped the purpose of CCAMPs?

regards, Neil
<snipped to end NH>

---

From: [email-masked]
Date: Wed, 19 Feb 2003 12:42:48 -0000

Lloyd Wood wrote 19 February 2003 12:03
> Fact.
NH=> Thanks Lloyd.....but how do you square this remark with CCAMPS
activities?

---

From: "Eduard Metz" <[email-masked]>
Date: Wed, 19 Feb 2003 13:46:08 +0100

> Any other relationships lead to one or more functional

Maybe so, but unfortunately "sense" is not only a function of functional
management. It involves a whole lot of other forces that apparently have
created a demand for "non-sense" solutions. If the problems associated with
these solutions cannot be solved (or not sufficiently solved), maybe the
demand goes away (or the solution may not be offered). Anyway the current
problem is "PWE3 congestion" .... do you think this is unsolvable when
running a co pkt-sw or co cct-sw client (PW) over a clns or co pkt-sw
server?

<snip>

> -	or the GMPLS (ie SDH, OTN) co cct-sw variety?

Cannot speak for anybody else, but my vote goes to IP as the technology that
successfully implements internetworking functionality. On the other hand, an
unfortunate case of PW congestion could break all three.

> have I not grasped the purpose of CCAMPs?

As far as I understand CCAMP it defines extensions to IP/MPLS control plane
technologies so that these can be applied to other type of networks. But
does that make the other networks "Internet". If I would modify PSTN control
plane technologies for use on IP networks, that does not make my IP network
a telephony network does it? (but this is well besides the point of the
discussion)

cheers,
	Eduard

---

From: [email-masked]
Date: Wed, 19 Feb 2003 14:31:55 -0000

Eduard....please see below.  regards, Neil

Eduard Metz wrote 19 February 2003 12:46
> running a co pkt-sw or co cct-sw client (PW) over a clns or co pkt-sw
NH=> Unfortunately you snipped the remarks in my preceding mail so I'll
repeat one of them:
"However, even though our architectural policy might state this I would not
stop anyone else from doing crazy things if they want....we all have choices
and sometimes learning from (bad) experience is the only way."
I would not say they are unsolvable, but they may not be 'effectively'
solvable....and by this I mean by whatever metric of cost/quality one wants
to apply to judge this.  Those who do want to do this stuff need to make a
call on this.  

> technologies so that these can be applied to other type of 
> networks.
NH=> For what purpose?

> But
> does that make the other networks "Internet".
NH=> So why are IETF doing this then?

> If I would 
> modify PSTN control
NH=> No it does not....you are quite right there.  So why are IETF working
on this stuff then if its nothing to do with the Internet?  However, you are
right that this discussion does not belong here even though it is pretty
well a fundamantal point to everything else.

regards, Neil

---

From: "Eduard Metz" <[email-masked]>
Date: Wed, 19 Feb 2003 16:41:51 +0100

> From: [email-masked] [mailto:[email-masked]]On Behalf Of
> [email-masked]
> Sent: woensdag 19 februari 2003 15:32
> To: [email-masked]; [email-masked]; [email-masked]
> Cc: [email-masked]

Okay good, though I'm not sure how to relate the remark to the issue (who is
the owner of "our policy" and why are things not compliant with that policy
"crazy"?). However, pointing out the inefficiencies of particular solutions
and their impact on the PW service delivery and operation is indeed
important.

cheers,
	Eduard

<snip>

---

From: "Bob Barrett" <[email-masked]>
Date: Wed, 19 Feb 2003 13:59:11 -0000

Snip

> It  mentions PSNs. I think you need to be very careful to mandate such
> safety valves for PSNs in general. E.g. a PSN might be under complete
> control of one single Service Provider and hence, for whatever reason,
> a certain level of congestion might be perfectly fine.

But the concern of the IESG is that PWE3 can and will get connected to
the Internet, and if and when that happens, it must not break the Internet.

Snip

But it is OK if the internet affects the performance of the PWE3 to such a
degree that the PWE3 is effectively broken. Users (SPs) with control of
their own PSNs are specific cases, the internet (with all of it's
uncertainties) is the general case, and PWE3 must address the general case.

The user base for PWE3 is very broad in terms of type and performance of
PSN. A lot of the issue comes down to how implementations handle late and
lost frames in the most part caused by congestion. How much of this process
is written into the PWE3 rfcs and how much is left to implementers is the
issue. Any mechanism that requires that both ends are using the same method
really needs to be in the PWE3 to ensure interoperability. Any mechanism
that can work within a PWE3 edge device without any knowledge of the partner
does not need to be in the PWE3 drafts.

The feedback that I have had from the market is that whilst operators expect
their PSNs to be near perfect, they still want Circuit Emulation kit to be
able to cope with imperfect PSNs. The profile of variable packet latency and
packet loss is the key issue in deciding how complex to make the replay
mechanisms and clock mechanism. For example a PSN that drops 1 in 25 of the
packets on a regular basis needs special handling for signalling, and
network that is 100% most of the time and then has a 50ms outage will not be
help by this. Even customers with perfect networks will require the
functionality to cope with imperfections, even if they hope it will never be
'used' in anger.

Thanks

Bob

---

From: "Busschbach, Peter B (Peter)" <[email-masked]>
Date: Wed, 19 Feb 2003 13:09:15 -0500

Eduart,

The tools that I was talking about are probably the same as the ones you have in mind: the ability to detect packet loss on a PW, the ability to reroute traffic, the ability to change bandwidth reservation parameters and the ability to shut down a PW. 

I agree with Stewart that not all tools are available in all networks. You use what you have. But a Service Provider should have flexibility. In case of congestion, he might decide to shut down one or more PWs, but not necessarily the ones that are experiencing packet loss. It all depends on the SLA agreements that are in place.

I agree with statements made in other emails that the way a Service Provider responds to certain network conditions is outside the scope of the WG. I was not proposing to bring these things into the WG. My point was: why is congestion avoidence in the scope of the WG? Shutting down a PW if you detect congestion is one of many different actions and why would that one be standardized if the others are outside the scope of the WG?

The answer is apparently: to protect the Internet. I can accept that. But if we standardize this "PW Back-off mechanism" a Service Provider should be allowed to disable it if the traffic stays within his own network or if the traffic is guaranteed to traverse the networks of other Service Providers that don't require this back-off mechanism.

Peter

-----Original Message-----
From: Eduard Metz [mailto:[email-masked]]
Sent: Wednesday, February 19, 2003 7:46 AM
To: [email-masked]; [email-masked]; [email-masked]
Cc: [email-masked]
Subject: RE: [PWE3] PWE3 congestion

<snip>

cheers,
	Eduard

---

From: Stewart Bryant <[email-masked]>
Date: Thu, 20 Feb 2003 10:03:45 +0000

Busschbach, Peter B (Peter) wrote:
> Eduart,
> 
> The tools that I was talking about are probably the same as the ones 
> you have in mind: the ability to detect packet loss on a PW, the ability to 
 > reroute traffic, the ability to change bandwidth reservation parameters and
> the ability to shut down a PW. 
> 
> I agree with Stewart that not all tools are available in all networks. You    > use what you have. But a Service Provider should have flexibility. In case of 
> congestion, he might decide to shut down one or more PWs, but not necessarily 
> the ones that are experiencing packet loss. It all depends on the SLA agreements 
> that are in place.
> 
> I agree with statements made in other emails that the way a Service Provider 
> responds to certain network conditions is outside the scope of the WG. I was 
> not proposing to bring these things into the WG. My point was: why is 
> congestion avoidence in the scope of the WG? Shutting down a PW if you 
> detect congestion is one of many different actions and why would that one 
> be standardized if the others are outside the scope of the WG?
> 
> The answer is apparently: to protect the Internet. I can accept that. But if 
> we standardize this "PW Back-off mechanism" a Service Provider should be 
> allowed to disable it if the traffic stays within his own network or if the 
> traffic is guaranteed to traverse the networks of other Service Providers 
> that don't require this back-off mechanism.

I agree with most of what you say except the idea of wiring shut the safety
valve (although I acknowledge that if customers demand it, then vendors
will build it). A better approach would perhaps be to set the protective
shutdown at a higher level than that at which the other more selective
mechanisms cut in. After all, even well engineered "private" networks may
have an unanticipated failure.

Stewart

>>>But the concern of the IESG is that PWE3 can and will get
>>
>>connected to
>>

---

From: "Busschbach, Peter B (Peter)" <[email-masked]>
Date: Thu, 20 Feb 2003 09:36:12 -0500

I agree

-----Original Message-----
From: Stewart Bryant [mailto:[email-masked]]
Sent: Thursday, February 20, 2003 5:04 AM
To: Busschbach, Peter B (Peter)
Cc: 'Eduard Metz'; [email-masked]; [email-masked]
Subject: Re: [PWE3] PWE3 congestion

Stewart