Subject: Re: When non TCP/IP literate people design cable modem systems

From: Reiner Ludwig <[email-masked]>
Date: Wed, 23 Jun 1999 11:06:38 -0700

At 09:50 AM 6/23/99 , Perry E. Metzger wrote:
>
>Reiner Ludwig <[email-masked]> writes:
>> I don't work with cable modems in any way but ...
>> 
>> ... fully-reliable link layer error recovery really is a Great Idea (not
>> necessarily the large queues, though) provided that the link layer 
>> 1. discriminates between TCP and UDP flows and performs fully-reliable link
>> layer error recovery _only_ for TCP flows (maybe also for ICMP ?), and 
>
>Ah, no. Its really unfortunate when you have two layers both doing
>retransmits, 

Where does it say that?

1. Unreliable or semi-reliable links require too much FEC (wasting too much
precious spectrum) to raise the packet loss rate to an acceptable level for
TCP-compatible congestion control.

2. The MTU is simply too big as a retransmission unit on some wireless links
where the size of the optimal retransmission unit can be way below 300 bytes.
This gets particularly painful with IPv6 which has a minimum MTU of 1280 bytes
(check out e.g.,
http://iceberg.cs.berkeley.edu/papers/Ludwig-Mobicom99/index.html).

=> you need (segmenting) link layer error recovery.

>especially when the upper layer starts shoving
>retransmitted packets in to a queue 

This can be avoided as I said in my last mail ("Eifel" algorithm). 

>that then is hell bent on reliably 
>delivering them and never letting them drop.

I did not say "never letting them drop". I just proposed a much higher "packet
discard timeout" than usually implemented in link layer error recovery
protocols.

>Ignoring the trouble it causes, it takes considerable effort to
>produce this living horror.

True because TCP's retransmission timer is so overly conservative. Also shown
in http://iceberg.cs.berkeley.edu/papers/Ludwig-FlowAdaptive/index.html -
Section 4.3 and 4.5.

>> (2) might sound odd but consider what happens over paths that include links
>> exhibiting temporary link outages (e.g., when a user roams into (and back
out
>> of) an area without wireless connectivity in a packet radio network). In
this
>> case all of the flow’s unacked packets will eventually be discarded by an
>> unreliable or less persistent (semi-reliable) link layer sender.
>
>Thank goodness. What you seem to think is a bad idea is actually of
>considerable value.
>
>All you can do by reimplementing TCP inside the data link layer is
>breeding fascinating sorts of interactions that lead to congestive
>collapses.

Where does it say that? Why would a fully-reliable link having the
properties I
mentioned cause a congestion collapse?

>TCP is already there for you. Let it do its job instead of screwing up 
>every algorithm it has to try to do its job.

Don't agree. In some cases TCP needs a little help (fully-reliable link layer
error recovery) to achieve good end-to-end perfromance. 

By the way, this is absolutely compatible with the e2e argument if you read
the
intro of that paper where it talks about 
a. "[...] functions each of which might be implemented in any of several ways:
by the communication subsystem, by its clients, as a joint venture, or perhaps
redundantly, each doing its own version. [...]", and 
b. "[...] Sometimes an incomplete version of the function provided by the
communication system may be useful as a performance enhancement. [...]"

That's the point. Fully-reliable link layer error recovery having the
properties I mentioned is such an "incomplete version of the function".

Clearly the communication subsystem needs to know the application
requirements,
but it *can*, by reading the IP header. If there just was more information in
there apart from discriminating between protocol-types ....

Please don't respond by telling me about layer violations. A) there are other
such examples and B) a layer conformant implementation is simple: it just
requires a capable network-layer/ link-layer interface.

>If we've learned anything over the past twenty years, it is that
>router queues should not expand forever, and that packet drops,
>properly understood, are your *friend*, not your enemy.

I agree that they are our *friends*. I did not argue in favour of large
queues.

///Reiner

---

From: "Perry E. Metzger" <[email-masked]>
Date: 23 Jun 1999 14:17:37 -0400

Reiner Ludwig <[email-masked]> writes:
> [1  <text/plain; us-ascii (7bit)>]

You said a *FULLY* RELIABLE (that is, 100% reliable) link layer with
error recovery, only for TCP packets and not for UDP packets, provided
by the network layer. If you can explain to me how you do that without 
having retransmissions for dropped packets in the link layer, please do.

> where the size of the optimal retransmission unit can be way below 300 bytes.

I've heard this song and dance before....

> This can be avoided as I said in my last mail ("Eifel" algorithm). 

Without touching TCP? Really? Explain how in a paragraph or so. I'm
all ears.

> discard timeout" than usually implemented in link layer error recovery

That's effectively the same thing. Frankly, link layer error recovery
is the enemy of a good TCP/IP based internet, but if the link layer
trys recovering for more than a fraction of a second, you REALLY are
asking to have things fall flat on their faces.

> properties I mentioned cause a congestion collapse?

I leave it to you to try doing the simulation. Figure it out for
yourself.

> error recovery) to achieve good end-to-end perfromance. 

Yeah, right. And I have a bridge to sell you, too.

I started this largely to inform people here of the fact that ignorant
people doing more or less what you advocate to "help things out" had
rendered a giant metropolitan based network totally unusable --
precisely because of the mechanism you were advocating, in fact. I
wanted to warn people of the fact that the ignorant were out and about 
and implementing cable systems. I didn't wish to get into a debate
with someone over why an observed problem couldn't possibly be
happening.

Perry

---

From: John Leong <[email-masked]>
Date: Wed, 23 Jun 1999 12:37:24 -0700

>> Ah, no. Its really unfortunate when you have two layers both doing

That may be a fact of life.  Ethernet can retry on the CD part for up to
16 times and V.42 modem also does error correction.  One can argue that
they are not designed specifically for TCP/IP and other protocols may
enjoy such error correction service.  It is up to the user of the link
layer (in this case, TCP/IP) to understand the capability of the
services being offered.

On the other hand, given that cable modem is designed mostly for TCP/IP
traffic, the persistant retry characteristic is more puzzling.

Regards,
John Leong
--
---------------------------------------------------------
Bell Labs Research       [email-masked]
4995 Patrick Henry Dr.                  Tel: 408-567-4459
Santa Clara, CA 95054                   Fax: 408-567-4448

---

From: Kumar Balachandran <[email-masked]>
Date: Wed, 23 Jun 1999 16:30:58 -0400 (EDT)

TCP is an end-to-end connection based transport layer, whose
reliability mechanisms are geared towards losses due to network
congestions (slow start is the problem). For certain short hop link
layers, such as on either side of a cable TV line, or say, a wireless
network, where the packet losses may be due to the physical channel or
external interference, there are benefits to having reliability
mechanisms on the link layer protocols. There is no way you can
guarantee 100 % reliability, but you can have retransmissions within
temporal windows that do not cause TCP timers to expire.

The long ping times are probably due to improper implementation. It
has no bearing on where reliability mechanisms must be implemented.

Ciao,
Kumar.

---

From: "Perry E. Metzger" <[email-masked]>
Date: 23 Jun 1999 19:29:50 -0400

> temporal windows that do not cause TCP timers to expire.

Trying to lower error rates on a very noisy line by retrying for a
fraction of a second or using FEC is just fine. Retrying for hundreds
of seconds with giant sized queues is just plain braindead, and I
don't care how you slice it. Unfortunately, others don't seem to have
gotten this message.

> has no bearing on where reliability mechanisms must be implemented.

On the contrary, the whole thing is an excellent example of people not 
understanding TCP/IP as a whole building link layers that are so smart 
as to be dangerous. This happens all the time. One wonders WHY people
spend so much time and effort designing link layers that actively
damage TCP stacks, but they seem to be doing it more and more often.

They start out by not understanding how TCP congestion avoidance
works, and they compound the damage by building mechanisms that fight
actively with it (at great expense).

Perry

---

From: Cannara <[email-masked]>
Date: Wed, 23 Jun 1999 22:00:44 -0700

I like what Barney said:  

  "Do all you rigid no-arq folks actually turn off V.42 on your modem when   
you dial?  Have you in fact demonstrated superior performance thereby?

   Barney Wolff, who uses V.42 and V.42bis"

Facts:

1) TCP can't yet distinguish error loss from congestion loss, so it slows    
down on errors -- poor idea, unless we're sure errors are highly          
congestion related.  Almost 21 years after the first TCP state machine     was
done at DEC/BB&N, we have no way to tell why a pkt was lost and yet    we have
only one algorithmic recourse in available TCPs, all based on      assumed
congestion at some overwhelmed router somewhere.  TCP's feedback    loop is
incomplete, because the key signal returned to the sender is       ambiguous.

2) Ethernet protects TCP from this at the millisecond level, far from TCP's   
ken.

3) Commercial TCPs' behaviors on various WAN paths is easily driven to low   
performance by very low error rates, as others here have pointed out,      and
many of us often witness.  

4) The WAN links being discussed here probably can't, or shouldn't, protect   
TCP from all this.  Why?  Because TCP is not a finished, fully-
   engineered transport yet.  

TCP is not to be protected, it is to be improved or replaced.  Like the
emperor, this old horse may need new shoes, and it may be in need of more.

Alex