Subject: RE: POS flow control: draft-tsenevir-ppp-flow-00.txt reply to Ver non

From: Vernon Schryver <[email-masked]>
Date: Mon, 30 Apr 2001 19:40:30 -0600 (MDT)

> From: Tissa Senevirathne <[email-masked]>
> To: "'Vernon Schryver'" <[email-masked]>, [email-masked]
> Cc: [email-masked], [email-masked]

> This is similar to flow control like XON/XOFF. Do you think it was bad for
> original designers to have XON/XOFF. 

XON/XOFF dates from at least mechanical teletypes ("tty's") as a mechanism
to start and stop a remote paper tape reader.  In its realm it was the
only and clearly necessary and appropriate flow control mechanism.  IP
applications already have a flow control mechanism in the form of TCP and
various UDP mechanisms.

Consider when and why XON/XOFF or RTS/CTS flow control is useful with
PPP over modems.  When the PPP link is connected to a host that is a
TCP endpoint, the TCP state machine can notice when the local PPP/modem
transmit queue is too full and stop the application.  That does not
apply to any router, including one with a SONET link.  On the other
hand, if a router drops a packet, a distant TCP sender will notice
and stop sending so much.

If any of the protocols or applications used above PPP needed help
with flow control, would the reliable link form of PPP defined in RFC
1663 be dead and forgotten?  If something other than just dropping
packets that don't fit in queues were good for PPP, wouldn't we be
doing that for modems?  What is different about SONET except speed?

>                                      Please also see the reply to James.
> What is your opinion on Ethernet PAUSE frames. Do you think that is not
> useful and affects IP flows ?

TCP/IP reacts badly to link layers that delay data.  TCP/IP wants to know
about congestion on any link or connection among links.  To tell TCP about
congestion in a link, you must signal it to the distant hosts that have
the TCP state machines.  You can do that by setting bits in IP or TCP
headers (for an example, look for "ECN"), sending new kinds of packets
(look for "ICMP source quench") or by deleting packets entirely.  There
is no fourth possiblity.  A link layer that adds buffering or retransmits
is failing to tell TCP about congestion, and that is usually a bad thing.
Yes, if the congestion is very short term, TCP does not need to know, but
the easiest way to do that is to do what network hardware has done since
before TCP, Ethernet, and even the ARPANet, and that is to have a FIFO
(or other kind of) quiet buffer.

I don't know anything about Ethernet PAUSE frames, and given history
including 100VG-AnyLAN 802.12, I'm afraid to learn.  Maybe Ethernet PAUSE
frames are good things that restore the link layer flow control lost with
so called "switches" (really multi-port bridges) and then with FDX.  The
TCP state machines in hosts connected to classic Ethernet could notice
when the wire was too full (i.e. collisions) and stop pushing more data.
Switches and FDX broke that and can cause TCP retransmissions, but only
when for hosts connected directly to the Ethernet.

Does your proposal involve hosts or routers with SONET links?

Vernon Schryver    [email-masked]

---

From: Vernon Schryver <[email-masked]>
Date: Tue, 1 May 2001 11:58:49 -0600 (MDT)

Why is my sense of lurking patents tingling?

> To: "'Vernon Schryver'" <[email-masked]>, [email-masked],
>    [email-masked]
> Cc: [email-masked]

>  Another point with the flow control is it really helps transmitting end to
> set ECN bits on TCP flows. Without buffer buildup this will not happen. SO I
> feel having flow control as proposed will help in generating ECN etc..

How does your proposal affect routers setting the ECN bits?  Why have
the authors of the ECN RFC's not mentioned the utility of equivalents
to XON/XOFF?

What does your proposal communicate between PPP peers anything that migh
be used to set ECN bits and that is not already known by the router that
shoudl be setting the ECN bits?  The router with the full buffers (either
output or input) knows it has full buffers.  It does not need to be told
its buffers are full in order to set ECN bits, and neither does it need
to tell some other router so that the other router can set ECN bits.

] From: Tissa Senevirathne <[email-masked]>
] To: "'Vernon Schryver'" <[email-masked]>, [email-masked],
]    [email-masked]
] Cc: [email-masked]

] The proposal involve connections betwen routers. Almost allways POS is
] between routers. On the other hand applications we are focusing is not
] necessaraly TCP/IP. It can be any variation of Layer 2, MPLS or IP tunnels.

Given that this is the IETF and your proposal is for PPP, what
applications do you anticipate?  (Note that MPLS and IP tunnels
generally must be counted as "TCP" because that's what is usually
what is happening at the top.)

] What we are trying to do here is when limted input buffer over flow occurs
] use the large Egress buffers to hold on packets during temporay ingress
] buffer overuns.

Is it reasonable to expect routers to shuffle free buffer space pools
at SONET speeds?  From what little I've seen, the answer to that
question is emphatically no.  Consider why router vendors are talking
about including 100-200 MByte of buffering in OC 192 interface cards
in order to have 10 milliseconds of buffering (i.e. buffer efficiency
of <15% due to fragmentation, such as putting a 200 byte packet in a
1534 byte buffer).

] What our stdudies indicated was, ability to buffer temporary congestion
] infact help TCP flows. It maintained a smooth flow rather than quinching
] windows.

Did your studies compare the relative costs of statically allocating
more RAM in each port as opposed to making it dynamic?

]          On the other hand use of ECN is still not so wide spread and
] applications are required to implement that.

That is mistaken.  Applications have nothing to do with, for, or to ECN.
Second, ECN will be widely implemented far sooner than a new PPP-SONET
XON/XOFF protocol.

]                                              On the other hand when we were
] tunneling non IP packets we need to provide as smooth flow as possible
] between router.

What non-IP packets do you anticipate tunnelling above PPP?

]                 IN theory receive router appears as a DCE to the sending
] router since they are directly connected via a POS (SONET) link, Hence all
] RTS/CTS and XON/XOFF philosphies apply directly. Once you start thinking to
] routers are DCE/DTE pair all what we presented may appear as logical. May be
] that is something we should have highlighetd. If that is the case please let
] us know we would include that in the next revision

Again, please explain why your proposal has not been required for all
other serial links among routers.  Why aren't RTS/CTS or XON/XOFF with
buffer management signalling such as you propose used for all other serial
links?  How does SONET differ from an old fashioned CSU/DSU leased line
link, except that your proposal might be implementable at low speeds?

---

From: Vernon Schryver <[email-masked]>
Date: Tue, 1 May 2001 18:52:02 -0600 (MDT)

> When you are tail dropping at the remote end there is no way that the
> buffers over flow, hence there is no ECN bit set. As I said earlier POS
> peers appears like DCE/DTE hence, hence flow control help to build the
> egress buffers at the transmit end and hence kick in ECN algorithms.

You did say that, but I didn't understand.  Your document says
"Packet Over SONET (POS)" which implies to me that each "POS peer"
does not "appear lke DCE/DTE".  The PPP universe is about complete
routers that are DTE's.  The DCE stuff is below PPP.

Why can't any PPP peer that is connected to a PPP peer by SONET,
modems, leased lines or wet string and that is dropping packets
from heads, tails or elsewhere in its queues set the ECN bits?
Why can't any PPP/SONET router watch its own ingress queue and mark
packets that it doesn't drop with the ECN bits?

>  In older implementations, DSU and DCU were sitting between. Now with POS
> two routesr appears like they are back to back connected. I presume that you
> would atleast agree on that....

Either I don't understand that or I disagree.

If you envision pairs of boxes connected by SONET links where the pair
of boxes acts as if it were a single PPP router containing a medium
bandwidth (10 Gbits/bit) internal bus that is very long (up to many
miles) with very high latency (packet serialization plus speed of
light), then I don't see what your proposal has to do with PPP and
the IETF.  "Half-bridges" are not the concern of the IETF but are in
the territory of one of the offices down the hall, such as IEEE 802.

On the other hand, there are plenty of SONET routers that use PPP
and know better than any other box whether they are dropping packets
from any of their queues and that can and (realsoonnow) do their
own ECN marking.

How is SONET different from an ancient 56 kbit/sec leased line
using CSU/DSU's or modern v.90 modem?  For your proposal to be needed,
there must be something about SONET that is very different from those
case, because in neither of those cases does one PPP peer need to tell
the other PPP about XON/XOFF, RTS/CTS, or other flow control.

PPP implementations using modems do care about RTS/CTS or XON/XOFF
flow control, but those cares are outside the domain of the IETF.
The EIA, TIA, or some such standards organization is in charge of
those DCE/DTE flow control issues, not the IETF.

To put it yet another way, I don't understand or disagree with the
this statement in your document:

]                                                                    PPP
]  [RFC 1661] protocol depends on the lower layer to provide required flow
]  control

The word "flow" does not appear in RFC 1661.  Consider the implications
of the single instance of "flow" in RFC 1662 (PPP in HDLC-like Framing).
Notice that "flow" does not appear in RFC 1618 (PPP over ISDN).  "Flow
control" is not mentioned in RFC 2823 (PPP over Simple Data Link (SDL)
using SONET/SDH with ATM-like framing) or RFC 2615 (PPP over SONET/SDH).

In still other words, how do existing PPP/SONET implementations
work without IEEE 802 Ethernet-PAUSE frames?