Subject: RE: Some comments to the draft-ietf-pilc-error-00.txt

From: Nitin Vaidya <[email-masked]>
Date: Tue, 22 Jun 1999 15:32:57 -0700

> From: Lloyd Wood [mailto:[email-masked]]
  ...
  > Local ARQ duplicates TCP checksum functionality and is against
  > end-to-end, as I point out above. TCP already does the checking; the
  > added ARQ delay messes up TCP's RTT estimates, in any case.

This may be true, but not necessarily harmful, if link level (local)
ARQ delay is not large relative to end-to-end delay.

  > FEC and ARQ are orthogonal; TCP effectively duplicates ARQ
  > functionality and makes ARQ redundant, but it doesn't duplicate FEC
  > functionality. It can't. TCP can check the integrity of what is
  > delivered by the transmission process, but it can't improve on it.

Why is ARQ at link layer redundant? Think of ARQ as a form of
adaptive FEC - both make delays variable. 
So long as delay variations are not too large, no
harm in using ARQ at link layer.
Just make sure that the local ARQ does not make packet delivery "too" 
out-of-order. 

- nitin

----
Disclaimer: Speaking for myself, not my employer.

---

From: Reiner Ludwig <[email-masked]>
Date: Tue, 22 Jun 1999 17:44:12 -0700

Just to add to Nitin's arguments that oppose Lloyd's statement "the added
ARQ delay messes up TCP's RTT estimates" ...

... the RTT is a property of the end-to-end path - as we know. If it
happens that some link on that path exhibits variable delays caused by link
layer ARQ (to improve end-to-end performance) then that must be accounted
for in the RTT. Just like any other delay variations caused by sources like
route "flapping", routing table updates, etc. as described in a paper by V.
Paxson ("End-to-End Routing Behavior in the Internet"). 

///Reiner

---

From: "Bora Akyol" <[email-masked]>
Date: Wed, 23 Jun 1999 09:29:41 -0700

1)Try doing an ARQ-based procedure on satellite links.
Even in LEO constellations, the ARQ delay almost always
causes a TCP RTO and completely messes up the throughput of the
link.

2) I did not say FEC was bad, all I said is that in some links such as wide
area packet data where the channel (radio channel) is rapidly changing
having an FEC that is not adaptive does not make any sense, since most of
the time you don't need it and when you do need it, it is usually
insufficient protection. What you really need is an adaptive FEC that is
fast and robust and also does not require you to change the framing of data
in the physical link.

Bora

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]On
> Behalf Of Reiner Ludwig
> Sent: Tuesday, June 22, 1999 5:44 PM
> To: Nitin Vaidya
> Cc: 'Lloyd Wood'; pilc
> Subject: RE: Some comments to the draft-ietf-pilc-error-00.txt
>
>

---

From: Lloyd Wood <[email-masked]>
Date: Wed, 23 Jun 1999 17:50:26 +0100 (BST)

> ARQ delay is not large relative to end-to-end delay.

...and hey, finer-grained measurement and RTO handling are never going
to be deployed anyway, so messing up measurement where such messing
can be avoided must therefore be acceptable? Sorry, can't agree.

> adaptive FEC - both make delays variable.

Adaptive FEC adds rather less coding/decoding delay variation than
multiple mini roundtrips do. Yes, ARQ on the local link _can_ allow
the link to achieve the channel capacity of a binary erasure channel -
if infinite delay is allowed for ARQ to work, of course, which rather
messes up TCP delay measurement.

ARQ's granularity compared to coding is coarse, to say the least;
coding is IMO by far the better per-link solution.

> harm in using ARQ at link layer.

Well, so much for achieving that channel capacity, then. Limiting
delay variation means limiting local roundtrips means limiting ARQ
resends, so then what's the point of ARQ if it never gets to perform
well? Severely errored link conditions then degrade link performance
badly as a result of limited permitted retries. Bad in itself, but the
knock-on system effects are worse...

Please see Perry's recent end2end email, appended below, for a 
clear illustration of why use of ARQ is misguided, and how
ARQ use messes up end-to-end communication.

The resulting delays are a product of the ARQ process; limiting ARQ to
prevent 'too large' variations just adds more complexity while
limiting the intended benefits of the ARQ approach. You'd be
better off junking ARQ completely as inappropriate to the overall
system - which is what application of the end-to-end principle tells
us anyway.

> Just make sure that the local ARQ does not make packet delivery "too" 

That's another bizarre compromise/specification/requirement in itself,
IMO, crossing layers to boot. All these 'so long as' and 'just make
sure that' are intended to ameliorate the fundamental mismatch between
ARQ and end-to-end requirements, when clean design involves
avoiding the complication of ARQ in the first place.

What may be appropriate for a single link for communication between
its two dumb endpoints in isolation can be extremely inappropriate for
an internetwork or higher layers utilising that link, where
the functionality is duplicated.

ARQ can really get in the way when you have asymmetric/variable
routing/topology, too; local ARQ'd resends of routing updates strike
me as being particularly unhelpful. ARQ and asymmetry don't mix.

A PILC-generated draft explaining why ARQ is bad would
IMO be a laudable goal. The current errors draft doesn't discuss
FEC or ARQ at all, and:

http://www.ietf.org/internet-drafts/draft-ietf-pilc-error-00.txt

 2.4 Delayed Duplicate Acknowlegements [MV97, VMPM99]
 [..]
   It is preferrable [sic] to allow a local mechanism
   to resolve a local problem, instead of invoking TCP's end-to-end
   mechanism and incurring the associated costs, both in terms of
   wasted bandwidth and in terms of its effect on TCP's window
   behavior.

needs careful qualification in terms of the local mechanisms that can
be safely applied without impacting TCP's behaviour, and the scope of
the problems that can be resolved; distinguishing between errors as
part of the link characteristic and unusual errors symptomatic of a
local problem that can be resolved is important. Or just reference
that not-yet-written draft...

cheers,

L.

the less said about various bizarre FEC/ARQ combinations, the better.

> Disclaimer: Speaking for myself, not my employer.

<[email-masked]>PGP<http://www.ee.surrey.ac.uk/Personal/L.Wood/>

Date: Wed, 23 Jun 1999 08:52:02 -0400 (EDT)
From: Perry E. Metzger <[email-masked]>
To: [email-masked]
Subject: When non TCP/IP literate people design cable modem systems
 
Some folks at Unnamed Cablemodem Provider have a lovely cable modem
system that rarely drops packets, but which does appear to have
astoundingly large queues and reliable retransmits in the data link
layer. I'm sure they thought this was a Great Idea. I provide a sample
unretouched ping sample below of what using this system is like.
 
(my personal favorite was
64 bytes from 206.1.51.15: icmp_seq=243 ttl=246 time=212681.418 ms
but I've heard stories of 300,000ms and more. The losses appear to
occur only later in the network path, not within the provider's net):
 
64 bytes from XXX.X.XX.XX: icmp_seq=213 ttl=246 time=167892.549 ms
64 bytes from XXX.X.XX.XX: icmp_seq=214 ttl=246 time=166909.097 ms
64 bytes from XXX.X.XX.XX: icmp_seq=215 ttl=246 time=165921.709 ms
64 bytes from XXX.X.XX.XX: icmp_seq=217 ttl=246 time=166285.507 ms
64 bytes from XXX.X.XX.XX: icmp_seq=218 ttl=246 time=165302.197 ms
64 bytes from XXX.X.XX.XX: icmp_seq=219 ttl=246 time=164312.156 ms
64 bytes from XXX.X.XX.XX: icmp_seq=220 ttl=246 time=163328.912 ms
64 bytes from XXX.X.XX.XX: icmp_seq=221 ttl=246 time=162344.836 ms
64 bytes from XXX.X.XX.XX: icmp_seq=222 ttl=246 time=161355.764 ms
64 bytes from XXX.X.XX.XX: icmp_seq=223 ttl=246 time=174627.732 ms
64 bytes from XXX.X.XX.XX: icmp_seq=224 ttl=246 time=173649.189 ms
64 bytes from XXX.X.XX.XX: icmp_seq=225 ttl=246 time=172664.074 ms
64 bytes from XXX.X.XX.XX: icmp_seq=226 ttl=246 time=178160.748 ms
64 bytes from XXX.X.XX.XX: icmp_seq=227 ttl=246 time=177176.078 ms
64 bytes from XXX.X.XX.XX: icmp_seq=228 ttl=246 time=176190.470 ms
64 bytes from XXX.X.XX.XX: icmp_seq=230 ttl=246 time=174203.410 ms
64 bytes from XXX.X.XX.XX: icmp_seq=231 ttl=246 time=175363.403 ms
64 bytes from XXX.X.XX.XX: icmp_seq=232 ttl=246 time=174380.950 ms
64 bytes from XXX.X.XX.XX: icmp_seq=233 ttl=246 time=173398.209 ms
64 bytes from XXX.X.XX.XX: icmp_seq=234 ttl=246 time=172413.667 ms
64 bytes from XXX.X.XX.XX: icmp_seq=239 ttl=246 time=196856.713 ms
64 bytes from XXX.X.XX.XX: icmp_seq=240 ttl=246 time=195878.520 ms
64 bytes from XXX.X.XX.XX: icmp_seq=241 ttl=246 time=194906.446 ms
64 bytes from XXX.X.XX.XX: icmp_seq=242 ttl=246 time=193907.888 ms
64 bytes from XXX.X.XX.XX: icmp_seq=243 ttl=246 time=212681.418 ms
64 bytes from XXX.X.XX.XX: icmp_seq=245 ttl=246 time=210696.110 ms
64 bytes from XXX.X.XX.XX: icmp_seq=246 ttl=246 time=209721.004 ms
64 bytes from XXX.X.XX.XX: icmp_seq=247 ttl=246 time=208735.515 ms
64 bytes from XXX.X.XX.XX: icmp_seq=248 ttl=246 time=207753.737 ms
64 bytes from XXX.X.XX.XX: icmp_seq=249 ttl=246 time=206764.453 ms
64 bytes from XXX.X.XX.XX: icmp_seq=250 ttl=246 time=205798.128 ms
64 bytes from XXX.X.XX.XX: icmp_seq=251 ttl=246 time=204803.277 ms
64 bytes from XXX.X.XX.XX: icmp_seq=252 ttl=246 time=203810.780 ms
64 bytes from XXX.X.XX.XX: icmp_seq=253 ttl=246 time=202825.129 ms
64 bytes from XXX.X.XX.XX: icmp_seq=258 ttl=246 time=197842.556 ms