Subject: Re: COMMENTS for your COMMENT - ARQ ID

From: Dr G Fairhurst <[email-masked]>
Date: Tue, 06 Mar 2001 11:31:30 +0000

Reiner Ludwig wrote:
> 
> At 11:44 06.03.01, Lloyd Wood wrote:
> >On Mon, 5 Mar 2001, Reiner Ludwig wrote:
> >
> > > >Provided that we're talking about individual
> > > >(src/dst/port|flowid) flows here, and not aggregated (TCP/UDP) flow
> > > >*types* or the assumptions based on type.
> > >
Sometimes the term "flow" doesn't map to TCP session/connection
The classifier that is used to determine which flow a packet
belongs to may in some cases be an aggregate - finding the (ip src, ip dst)
is easy, but find the end-to-end flow is difficult with VPNs,
tunnels, etc. 

> > > You can allow multiple flows across the same link layer ARQ protocol as
> > > long as that link layer ARQ protocol implements inter-flow out-of-order
> > > delivery, i.e., the link only ensures the order between packets of the
> > same
> > > flow.
> >
> >Yes; per flow state; effectively interleaving per-flow ARQ.
> 
> It does not require "per flow state". It only requires "flow
> identification" at the link layer. A subtle but potentially important
> difference.
> 
> The link layer receiver only needs to decide whether it can allow one
> packet to "bypass" the other and for that only needs to check whether those
> packets belong to the same flow or not.
> 
> ///Reiner

I don't actually see this.  I've tried to do this, and each time
I ended having to define per-flow state at the link layer
receiver to indicate whether each flow was "forwarding" 
- i.e. had no segments pending retransmission
at the link layer, or "blocking" had one of more segments where a
fragment was awaiting link layer retransmission.

Have you designed another way of doing this? 

Gorry

-- 
------------------------------
http://www.erg.abdn.ac.uk/users/gorry

---

From: Reiner Ludwig <[email-masked]>
Date: Fri, 09 Mar 2001 19:06:22 +0100

>fragment was awaiting link layer retransmission.

Let's discuss at the IETF meeting.

///Reiner

---

From: Gorry Fairhurst <[email-masked]>
Date: Fri, 09 Mar 2001 19:12:17 +0000

I include some comments on the new LINK ID (-05):

-------
link-design-05 says:
>  However, when a link layer can identify separate flows, the LL ARQ
>  persistency should be high for flows using reliable transport
>  protocols (e.g., TCP) and low for the other flows.

I think this is a somewhat simplistic conclusion for links in general.
There are issues here:

(i) Individual TCP flows, considered in isolation, benefit
from a low probability of individual segment loss.
Each lost segment leads to a reduction in cwnd,
which takes many path round trip times to recover.  
Too much loss forces cwnd towards one segment. 
(This isn't saying anything about overall shared link/network
utilisation, which is rather more complex.)   

High persistence on a lossy link offers a remedy from TCP 
perspective, by reducing the loss rate experienced by the 
TCP flow to near zero.   With appreciable frame loss rate, on 
the link, this high persistence leads to intermittent (variable) 
delay.  This delay may also
slows TCP transmission rate (by causing pauses in ACK clocking).
If the link round trip delay is high (e.g. for satellite links
- which provide a useful counterpoint to cellular links),
then these pauses may be sufficiently long to introduce short 
periods during in which the link receiver forwards no DATA
segments - and thus negatively impacts the throughput of the TCP flow.

The increased jitter is also felt by non-bulk flows, such as
streaming sessions, leading to degradation here too.

Low persistence techniques accept some probability of loss. They
therefore may see segment loss, and cwnd reduction with frame loss.
Throughput therefore varies with time, at moderate to high error
rates.

(ii) I remain unconvinced by the argument that reliable-transport flows 
need high-persistence ARQ over all links.  For THIN links, at low 
speed, i.e. where the  bandwidth delay product is very low, the TCP 
DATA segment serialisation delay (the total time to serialise all 
the frames for a segment) can be important, and out-of-order 
delivery and/or low persistence may be important.

(iii) At high speed, a link will be ***shared*** by a large number of TCP
flows.  It is, I suggest, important to look at the impact of the
ARQ on the shared link.  Recommending high-persistent ARQ,
Recommending high-persistence can require all flows sharing the 
same link to suffer, (if you're not able 
to do clear per-flow separation within the ARQ retransmission procedure)
whenever a loss event causes one or many link ARQ 
retransmissions.  One could argue this is unfair
to flows which have just started (e.g. small cwnd) when sharing
with established (bulk) flows.

Using a more complex ARQ scheme working per-flow, where individual
flows each have their own protocol machinery,  may also be possible,
and would help promote isolation.

(iv)  When the link transitions from a low loss rate, to a high loss
rate, high persistence introduces a sudden increase in delay, possibly
triggering time out (RTO expiry) of flows sharing the link.  For this
to happen, will require a significant change in path RTT (given a 
0.5s TCP timer granularity).

Low persistence ARQ would observe loss of some (but not necessarily
all segments) in this case.

---------
link-design-05 continues to say:

> The LL ARQ
> persistency for a reliable flow should be as high as the largest link
> outage period expected on a particular link, but no higher than the
> transport protocol's maximum retransmission timer (64 seconds for
> TCP).

Why no higher than 64 seconds? The link in question may be one of many
along the path from the source to the receiver and back again.  64
seconds is an end system parameter, it should not be used as a design 
parameter for a link.  The value could even be changed in future TCP
variants: It's bad to presume it.

"Should" seems very strong in this case. A high persistency may overcome
the effects of the link outage (as may other possible mitigations,
see existing text on "Recovery from Subnetwork Outages" which
does not prescribe full ARQ retransmission).
It may also be difficult to provide an upper bound on the duration
of an outage, and therefore I'm not sure of the value of saying this.

With 64 seconds of persistency, a high error rate in
both forward and return link directions, the ARQ protocol could
introduce 64 seconds of delay to TCP DATA segments, and 64 seconds of
delay to TCP ACKs for each (and every) link using this ARQ strategy
(64+64+64... in the worst case).

Total path RTT now > 64 seconds. What is the impact of
this? - I question that the event doesn't cause a protocol deadlock
or a significant penalty for being even slightly larger than 64 seconds.

I am seriously worried that this number is NOT a link parameter,
and that we should NOT be prescribing a solution which anticipates
only one wireless link along a path.

--------
link-design-05 says:

> High LL ARQ persistencies for reliable (e.g., TCP) flows uses
> maximizes end-to-end throughput in the face of transient link
> outages
> (see Section "Recovery from Subnetwork Outages"). It also improves
> the utilization of a link with statically allocated resources.
> [LKJK01].

I think it would be wise to also mention the down-sides of the
approach, to allow designers to decide. In a shared
medium, high persistence continues to consume transmission resource
for the duration. Many protocols will simply transmit until they
get through - this may be undesirable, since it prevents other
users of the medium using the link for more productive transfers.
In this case, an ARQ back-off algorithm (or network rerouting)
algorithm may be desirable.

There is also an issue of just how much backlogged data the link
should hold. The current text appears to advocate holding all
data pending transmission. This could be a vast volume of data
on high speed links - particularly if there are many flows with
large cwnds, not to mention the need to also transfer the
back-log of UDP traffic (which could be CBR). 

Class-based queuing helps (assuming the link has access to 
classification information - implicit or explicit, and is
therefore able to reliably decide which flows are which).

----
link-design-05 says in the introduction:
>    Optimizing a subnetwork for IP involves three complementary
>   considerations:

>   1. Providing functionality sufficient to carry IP.

>   2. Eliminating unnecessary functions that increase cost or
>   complexity.

>   3. Choosing subnetwork parameters that maximize the performance of
>   the Internet protocols.

link-design-05 again says:
> "as much flexibility as possible in the implementation of these
> mechanisms, and to provide access to them as discussed above in
> the section on Quality of Service."

link-design-05 advocates a similar flexible approach in the section
on reliability and error control.

----

In summary, the new text in the ARQ section seems very prescriptive,
and goes against the approach advocated at various other places 
in the draft. I like the general style of the draft, and so think
the ARQ section therefore needs more work.

Gorry Fairhurst

P.S. Thank you lloyd for your many comments and feedback which
helped me to think this through.