Subject: [PWE3] Transport Area Review: Congestion Control Framework

From: [email-masked]
Date: Mon, 27 Oct 2008 10:46:00 -0400

Transport Area Review
                Pseudowire Congestion Control Framework
                draft-ietf-pwe3-congestion-frmwk-01.txt

				David L. Black, EMC
				 October 27, 2008
		---------------------------------------------

Ordinarily, a review like this would focus on requirements.  Due to the
unusual circumstances surrounding congestion control for pseudowires,
this review's primary purpose is to state a number of non-requirements,
three in particular:

(1) The Fibre Channel pseudo wire is special.  The congestion control
framework for the Fibre Channel pseudowire (cf.
draft-ietf-pwe3-fc-encap-08.txt) is specific to that encapsulation and
is well beyond what appears to be needed and/or appropriate for
pseudowires in general.

Fibre Channel (FC) implementations are effectively lossless in practice;
FC frames are dropped under only the most extreme circumstances.  For
the most common usage of Fibre Channel, carrying SCSI-based I/O via the
FCP protocol, a dropped FC frame will typically result in the entire I/O
being retried after a timeout (30 seconds and 60 seconds are common
values for this timeout).  Typical SCSI I/O latencies are a small
fraction of a second, and hence this sort of frame loss requiring an
expensive and delayed I/O retry needs to be a very rare event.  For this
reason, the FC pseudowire incorporates an reliable delivery protocol
that includes
retransmission, and the FC pseudowire's congestion control measures are
based on that protocol.  In essence, this is an FC-specific adaptation
to make the FC pseudowire tolerant of (and hence usable over) a broader
range of network conditions.

In contrast, most pseudowires do not have requirements to avoid losses
beyond those of the underlying network, and in fact pass through such
losses to the pseudowire service provided, avoiding retransmission
entirely at the pseudowire level.  In the absence of retransmission, the
congestion impact of such a pseudowire can be estimated from its packet
or frame size, transmission rate, losses experienced at that
transmission rate and the latency between the endpoints.  The simplified
TCP throughput equation in Section 3.1 of RFC 5348 indicates the rate at
which TCP would transmit under such conditions, so the difference
between the pseudo-wire's actual transmission rate and the corresponding
TCP transmission rate is an initial indication of the pseudowire's
congestion impact.  The word "initial" needs to be emphasized - there
are numerous other considerations, starting with non-requirements (2)
and (3).

(2) There is no single always-right answer for TCP equivalence of a
pseudowire, i.e., the number of TCP-equivalent flows represented by a
pseudowire's traffic.

In the absence of other information, it is reasonable to assume that a
pseudowire is equivalent to one TCP flow.  On the other hand, other
information is often available about a pseudo-wire, often indicating
that it carries traffic equivalent to multiple flows.  Examples include,
the nature and composition of traffic on a network pseudowire, number of
hosted or hostable voice connections for a TDM pseudowire, etc.
Considering possible application use of a pseudowire at one extreme and
the common service provider use of pseudowires for network provisioning
and traffic engineering at the other extreme, it does not appear
possible to span this range of applicability with a single specification
of the number of TCP-equivalent flows that an individual pseudowire
should be considered to represent.

(3) There isn't even a single goal for pseudo-wire congestion control.
The purposes of congestion control appear to vary from fairness wrt
other traffic to basic safety protection of the network, depending on
the deployment.

This range of goals follows from a similar premise as the range of
TCP-equivalent flows in the previous item.  If an application's
pseudowire is mixed with TCP traffic on the same links, fairness to that
other TCP traffic is very important.  By contrast, in a
well-controlled-and-managed service provider network that is provisioned
entirely with pseudowires, inter-pseudo-wire fairness is a network
management and traffic engineering issue for OAM systems and the like,
and the appropriate goal of congestion control is safety (network
protection), as the network design and traffic engineering will be
tolerant of some failure scenarios, but unanticipated failures may cause
situations in which congested traffic has to be removed in order to
preserve the remainder of the network.

-----------------------

Overall, the pseudowire congestion framework draft is a good starting
point, but it appears (in 20/20 hindsight) that the TCP flow equivalence
issue has been raised prematurely.

Based on discussion of the congestion framework draft, there does not
appear to be a single congestion control problem or solution for
pseudowires that can span the spectrum from possible application use of
pseudowires to core network provisioning based entirely on pseudowires.
Rather, there are likely to be multiple problems and hence multiple
solutions involved, making determination of applicability important.

The next step suggested by this review is to focus on the problem
structure. In particular, it has been strongly asserted that the use of
pseudowires to provision and traffic engineer service provider networks
has fundamentally different congestion control characteristics, goals
and requirements from the Internet at large.  The author of this review
and the Transport Area Directors are inclined to agree, but want to see
the applicability details.

We also want to put the pwe3 WG on notice that we do not believe
assertions to the effect that all service provider networks that use
pseudowires are completely immune from the disruptive effects of:
	- congestion
	- lightning bolts
	- backhoes (e.g., JCB)
	- ship anchor damage to submarine cables
	- etc.
The words "all" and "completely" are crucial in the above sentence.

With the support of the Transport ADs, the transport area advisor to the
pwe3 WG (David L. Black) is interested in working with the pwe3 WG on a
draft that describes the nature of a
well-controlled-and-managed-service-provider-network (shorter name
clearly needed) for which pseudowire congestion control mechanisms
should be focused on network protection from unanticipated drastic
failures.  This could be productively viewed as an applicability
statement for a basic set of congestion control requirements that may
cover the vast majority of current pseudowire deployments.  Outside this
scope of applicability, a more demanding set of congestion requirements
may be appropriate for unconstrained use of pseudowires.

Thanks,
--David (IETF Transport Directorate Member)
----------------------------------------------------
David L. Black, Distinguished Engineer
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
[email-masked]        Mobile: +1 (978) 394-7754
----------------------------------------------------
_______________________________________________
pwe3 mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/pwe3

---

From: <[email-masked]>
Date: Tue, 28 Oct 2008 10:05:38 -0000

Interesting mail David,

I agree with your remarks but IMO they are not sufficient....one also
has to take performance inheritance into account.

Let me call a "well-controlled-and-managed-service-provider-network" a
"transport network" here, for both brevity and because this has a
well-understood meaning to those au fait with the unified modelling (of
the functional architecture) of networks - see G.800.

The topology of a layer N network (any mode/technology - see note) is
composed of layer N links and layer N nodes.  Performance impairments
can be sourced from both these components. {Note - There are 3 generic
network modes:  cl-ps, co-ps, co-cs.  All technologies belong to one of
these.  Mode differences arise due to how resource (time/freq/space) is
partitioned and labelled.  See G.800}.

The layer N links are actually end-end network connections in layer N-1,
which itself is composed of layer N-1 nodes and layer N-1 links.  And
this behaviour is recursive until we hit the duct/physicals.

The consequence of this is the performance experienced by the clients of
layer N is (to a very good 1st ~) the cumulative sum of the impairments
generated in the layer N nodes traversed and the impairments *inherited*
by the layer N links traversed....which is a recursive impairment
inheritance behaviour to the duct.

In other words, it is *not* sufficient to define a 'transport network'
at layer N only, but this must also be a 'transport network' at all
lower levels down to the duct/physicals.

A network must have (at a minimum) the following characteristics to be
deemed a transport network from performance inheritance considerations:
-	reachability isolation, ie the access points of client X cannot
communicate with the access points of client Y;
-	performance isolation, ie the traffic behaviour of client X
cannot impact the performance experienced by client Y.

In order to satisfy the latter requirement, a 'transport network' must:
-	be based on connections (these have a very precise meaning - see
G.800)...a necessary condition;
-	not have overbooked resources...a full-sufficiency condition.

A further consequence of the recursive nature of network topology
creation is that the disjoint connectivity of layer N cannot be greater
than that of the duct/physicals...which of course impacts the layer N
availability performance.

I think the implications of what I am pointing out should be clear, and
IMO the above should also be factored-in to your deliberations as well
as the other points you made.

regards, Neil

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of [email-masked]
> Sent: 27 October 2008 14:46
> To: [email-masked]
> Subject: [PWE3] Transport Area Review: Congestion Control Framework
> 
> 				Transport Area Review
> Ordinarily, a review like this would focus on requirements.  
> Due to the unusual circumstances surrounding congestion 
> control for pseudowires, this review's primary purpose is to 
> state a number of non-requirements, three in particular:
> 
> (1) The Fibre Channel pseudo wire is special.  The congestion 
> control framework for the Fibre Channel pseudowire (cf.
> draft-ietf-pwe3-fc-encap-08.txt) is specific to that 
> encapsulation and is well beyond what appears to be needed 
> and/or appropriate for pseudowires in general.
> 
> Fibre Channel (FC) implementations are effectively lossless 
> in practice; FC frames are dropped under only the most 
> extreme circumstances.  For the most common usage of Fibre 
> Channel, carrying SCSI-based I/O via the FCP protocol, a 
> dropped FC frame will typically result in the entire I/O 
> being retried after a timeout (30 seconds and 60 seconds are 
> common values for this timeout).  Typical SCSI I/O latencies 
> are a small fraction of a second, and hence this sort of 
> frame loss requiring an expensive and delayed I/O retry needs 
> to be a very rare event.  For this reason, the FC pseudowire 
> incorporates an reliable delivery protocol that includes 
> retransmission, and the FC pseudowire's congestion control 
> measures are based on that protocol.  In essence, this is an 
> FC-specific adaptation to make the FC pseudowire tolerant of 
> (and hence usable over) a broader range of network conditions.
> 
> In contrast, most pseudowires do not have requirements to 
> avoid losses beyond those of the underlying network, and in 
> fact pass through such losses to the pseudowire service 
> provided, avoiding retransmission entirely at the pseudowire 
> level.  In the absence of retransmission, the congestion 
> impact of such a pseudowire can be estimated from its packet 
> transmission rate and the latency between the endpoints.  The 
> simplified TCP throughput equation in Section 3.1 of RFC 5348 
> indicates the rate at which TCP would transmit under such 
> conditions, so the difference between the pseudo-wire's 
> actual transmission rate and the corresponding TCP 
> transmission rate is an initial indication of the 
> pseudowire's congestion impact.  The word "initial" needs to 
> be emphasized - there are numerous other considerations, 
> starting with non-requirements (2) and (3).
> 
> (2) There is no single always-right answer for TCP 
> equivalence of a pseudowire, i.e., the number of 
> TCP-equivalent flows represented by a pseudowire's traffic.
> 
> In the absence of other information, it is reasonable to 
> assume that a pseudowire is equivalent to one TCP flow.  On 
> the other hand, other information is often available about a 
> pseudo-wire, often indicating that it carries traffic 
> equivalent to multiple flows.  Examples include, the nature 
> and composition of traffic on a network pseudowire, number of 
> Considering possible application use of a pseudowire at one 
> extreme and the common service provider use of pseudowires 
> for network provisioning and traffic engineering at the other 
> extreme, it does not appear possible to span this range of 
> applicability with a single specification of the number of 
> TCP-equivalent flows that an individual pseudowire should be 
> considered to represent.
> 
> The purposes of congestion control appear to vary from 
> fairness wrt other traffic to basic safety protection of the 
> network, depending on the deployment.
> 
> This range of goals follows from a similar premise as the 
> range of TCP-equivalent flows in the previous item.  If an 
> application's pseudowire is mixed with TCP traffic on the 
> same links, fairness to that other TCP traffic is very 
> important.  By contrast, in a well-controlled-and-managed 
> service provider network that is provisioned entirely with 
> pseudowires, inter-pseudo-wire fairness is a network 
> management and traffic engineering issue for OAM systems and 
> the like, and the appropriate goal of congestion control is 
> safety (network protection), as the network design and 
> traffic engineering will be tolerant of some failure 
> scenarios, but unanticipated failures may cause situations in 
> which congested traffic has to be removed in order to 
> Overall, the pseudowire congestion framework draft is a good 
> starting point, but it appears (in 20/20 hindsight) that the 
> TCP flow equivalence issue has been raised prematurely.
> 
> Based on discussion of the congestion framework draft, there 
> does not appear to be a single congestion control problem or 
> solution for pseudowires that can span the spectrum from 
> possible application use of pseudowires to core network 
> provisioning based entirely on pseudowires.
> Rather, there are likely to be multiple problems and hence 
> multiple solutions involved, making determination of 
> applicability important.
> 
> The next step suggested by this review is to focus on the 
> problem structure. In particular, it has been strongly 
> asserted that the use of pseudowires to provision and traffic 
> engineer service provider networks has fundamentally 
> different congestion control characteristics, goals and 
> requirements from the Internet at large.  The author of this 
> review and the Transport Area Directors are inclined to 
> agree, but want to see the applicability details.
> 
> We also want to put the pwe3 WG on notice that we do not 
> believe assertions to the effect that all service provider 
> networks that use pseudowires are completely immune from the 
> disruptive effects of:
> 	- congestion
> 	- lightning bolts
> 	- backhoes (e.g., JCB)
> pwe3 WG (David L. Black) is interested in working with the 
> pwe3 WG on a draft that describes the nature of a 
> well-controlled-and-managed-service-provider-network (shorter 
> name clearly needed) for which pseudowire congestion control 
> mechanisms should be focused on network protection from 
> unanticipated drastic failures.  This could be productively 
> viewed as an applicability statement for a basic set of 
> congestion control requirements that may cover the vast 
> majority of current pseudowire deployments.  Outside this 
> scope of applicability, a more demanding set of congestion 
> requirements may be appropriate for unconstrained use of pseudowires.
> 
> 
> Thanks,