Subject: RE: QoS and adaptive end protocols

From: "Manfredi, Albert E" <[email-masked]>
Date: Tue, 20 Jun 2000 18:06:47 -0400

> -----Original Message-----
> From: demir [mailto:[email-masked]]
> 
> Does it make sense to use adaptive end protocols, requesting services
> othe than best-effort service, on a QoS-network?

This is an interesting question that I've also pondered. It seems that
protocols which adapt at Layer 4, mainly TCP, do very poorly when the lower
layers adapt on their own. What results seem to show is that allowing TCP do
do its thing over a best effort network is preferable to TCP over something
fancier, like ABR ATM.

UDP flows would benefit, though. If TCP tends to fill the pipe by adapting
itself, UDP which cannot adapt itself would be most impacted. Since much
QoS-demanding traffic will be UDP, what to do?

Some of the techniques mentioned in diffserv, such as RED and all its
cousins, do seem to work well with TCP. They allow TCP to remain responsive,
but to be a better neighbor on a multimedia network. The goal is to allow a
single network to provide different levels of service for TCP and UDP flows.

So the adaptive protocols like TCP would use services different from
non-apaptive protocols like UDP, on the same network.

Bert
[email-masked]

---

From: demir <[email-masked]>
Date: Tue, 20 Jun 2000 16:35:59 -0700 (PDT)

> do its thing over a best effort network is preferable to TCP over something

One of the reasons why I asked this question is because there is a
research on TCP modifications over QoS-network (I'm aware of Diffserv). To
me it does not make sense to modify TCP ad-hoc (depending on different
cases) cause adaptation of TCP works over best-effort networks. It makes
sense to evaulate TCP performance on a QoS-network cause TCP is a
widely-deployed protocol. May be I should ask my question more
explicit. If a network is offering a service other than best effort, does
an end-protocol need to be adaptive? If a network offers a service it
better satisfies it otherwise the network needs to be re-engineered. I
can't think of any service (again other than best-effort) will require
adaptation from users of the service (am I being too restirictive?)

> QoS-demanding traffic will be UDP, what to do?
To me what to do is not use any adaptive protocol if I am asking a service
other than best-effort from the network. TCP eventually full its pipe and
inevitably perform congestion avoidance, results with poorer service than
the one being asked. 

> single network to provide different levels of service for TCP and UDP flows.
Again, to me, RED is designed to provide fair BW-share (also network
support for TCP's adaptation) to different flows where network provides
only best-effort service. In Diffserv network, RED makes sense when
network is under-provisioned and services related to this. When network is
well-provisioned, does it make sense to use RED? It should be
end-applications' responsibility to be in-profile.

> non-apaptive protocols like UDP, on the same network.
Can you name or give an example for such a service?

Alper K. Demir

---

From: Carlos Alberto Kamienski <[email-masked]>
Date: Tue, 20 Jun 2000 23:26:43 -0300 (EST)

Demir,

> adaptation from users of the service (am I being too restirictive?)

Your idea is that TCP has to adapt because a lot of TCP connections share
links in a best-effort network and it is necessary to avoid congestion at
the same time it is desirable to keep links' utilization high.
However, if applications are using a QoS-enabled-service with certain
guarantees, why would they adapt ? 
I think this is true for "quantitative services", where applications have
strict guarantees about bandwidth, packet loss, etc.
But, and about "qualitative services", like for example those can be
provided by Diffserv's AF PHB ? In these cases it is expected that
applications exceeds their subscribed rate, even though exceeding traffic
won't get the same QoS guarantees. If applications don't adapt, how can
they manage this situation properly ?
And if we don't know so far if TCP adaptation is suitable for a
QoS-network,
than maybe it is better to change its adaptation mechanism to get a higher
performance of a qos-service.

Carlos

---

From: Dan Grossman <[email-masked]>
Date: Wed, 21 Jun 2000 10:51:27 -0400

> do its thing over a best effort network is preferable to TCP over something
This seems to be an urban myth.  I did a quick search through netbib a few months ago on this subject, and found that there was no consensus in the published literature.   A lot depended on whether the ATM network did explicit rate or EFCI control, how much buffering was present in the edge router/ATM end system (or host/ATM end system) devices (I did not see any papers that showed the effects of active queue management in these devices) and the usual simulation issues. I don't think that there is any basis for conclusion based on the existing literature.  

I have been told that there are large deployments of IP over ATM with ABR, and that they have excellent goodput (sorry, that's about as much as I can talk about).  Unfortunately, I don't expect to see any published measurement results from these networks.

My own hypothesis:  
Perhaps the best way to think of ABR and its interactions with TCP is that the ATM network provides a  near-constant delay, low loss subnet that deals with competing flows by varying the available cell rate.  TCP is looking, ideally, for a fixed bandwidth, near-constant delay Internet that deals with congestion by varying the packet loss rate.  Edge routers can fairly easily adapt the characteristic of the ABR subnet to match the behavior desired by TCP.  One could argue that by pushing congestion to the edge devices (and thus keeping goodput high in the ATM core), system goodput increases.  Also, since ABR **with ER**  is very fair,  pushing packet dropping to the one queue in the edge device is likely to be more fair than the net effect of active queue management in every device in the core.

Dan

---

From: Lloyd Wood <[email-masked]>
Date: Wed, 21 Jun 2000 17:02:34 +0100 (BST)

On Wed, 21 Jun 2000, Dan Grossman wrote:

> I have been told that there are large deployments of IP over ATM
> with ABR, and that they have excellent goodput (sorry, that's
> about as much as I can talk about).  

You'd expect excellent goodput - provided the ABR
links are generally of fixed dimension and of low loss.

(Are those ATM networks running anything else other than ABR?)

old, but relevant here:

J. Crowcroft, 'Why lossy internetworking and lossless ABR
services do not go together', discussion paper, April 17, 1994.

http://www.cs.ucl.ac.uk/staff/J.Crowcroft/jons-papers.html

L.

is not sure why this has to be across three lists, and has Reply-To:
set accordingly.

<[email-masked]>PGP<http://www.ee.surrey.ac.uk/Personal/L.Wood/>

> Unfortunately, I don't expect
> to see any published measurement results from these networks.
>
> My own hypothesis:  
> Perhaps the best way to think of ABR and its
> interactions with TCP is that the ATM network provides a
> near-constant delay, low loss subnet that deals with competing
> flows by varying the available cell rate.  TCP is looking,
> ideally, for a fixed bandwidth, near-constant delay Internet that
> deals with congestion by varying the packet loss rate.  Edge
> routers can fairly easily adapt the characteristic of the ABR
> subnet to match the behavior desired by TCP.  One could argue that
> by pushing congestion to the edge devices (and thus keeping
> goodput high in the ATM core), system goodput increases.  Also,
> since ABR **with ER** is very fair, pushing packet dropping to the
> one queue in the edge device is likely to be more fair than the
> net effect of active queue management in every device in the core.

---

From: Kathleen Nichols <[email-masked]>
Date: Wed, 21 Jun 2000 09:15:15 -0700

I think you make some very good points
that are often overlooked.

---

From: Fred Baker <[email-masked]>
Date: Wed, 21 Jun 2000 09:06:41 -0700

At 04:35 PM 6/20/00 -0700, demir wrote:
>If a network is offering a service other than best effort, does

Isn't this the question Parekh and Gallagher tried to put to bed in 
INFOCOMM '93?

Prior to that, there was an argument that to have end to end QoS, one 
either needed a circuit-switch network (which provides mathematical 
predictability for the wire) from the bellheads, or that it could all be 
done from the host, from the dataheads. Parekh basically proved that you 
needed two things:

   - you need the end systems to keep a predictable amount of data in the
     network
   - you need the network elements, including the end systems, to use
     appropriate mechanisms to assure that delay at each point en route
     is predictable.

He assumed a leaky bucket rate controller and per-vc WFQ - he assumed ATM - 
and proved that such a network could deliver a predictable round trip delay.

Bringing that into the protocols we use in the Internet, it turns out we 
have fundamentally three kinds:

   - voice/video, the same things we run in ATM networks
         (that would be Napster as well as VoIP)
   - transaction exchanges (short HTTP, ftp control, SMTP, etc)
   - long adaptive streams (long HTTP, FTP data, perhaps SCTP, etc)

The first two don't particularly adapt. Voice/Video is what brought up this 
whole question of QoS, and it *does* rate-control at the input side. The 
codec has some fairly predictable parameters, especially for voice. The 
transaction exchanges don't have any means to adapt and hardly have 
information on which to adapt.

When you talk about "adaptive end protocols", therefore, you are talking 
about TCP and TCP-like data streams. If you want to build something 
equivalent to ATM-CBR all the way from host to host, Parekh's proof would 
argue that particular part of the network was predictable and your down to 
asking whether the host OS is predictable (which is b no means a given). 
There are several gotchas in that - for one thing, you have no idea what to 
set the rate to, so the transactions (a huge percentage of the traffic) 
would wind up underutilizing their capacity and the net would run 
relatively idle. You would also get exactly what you paid for out of the 
long streams - if you asked for 10 KBPS per stream because you didn't want 
to fritter away all of your network's capacity on CBR streams carrying no 
transactions, when the file you asked for turned out to be long, you would 
get it at 10 KBPS.

That's not a world I would want to live in. Far better, IMHO, to let the 
adaptive traffic and the transactions mix it up, and then give them a QoS 
service more appropriate to an adaptive transport. That, in a nutshell, is 
what we have tried to do in diff-serv's AF service (EF being CBR-like). We 
have tried to let the TCP stuff mix it up, and when we need to interwork 
with TCP to rate control someone, we do it in a way that the adaptive 
protocol will adapt to.

---

From: demir <[email-masked]>
Date: Wed, 21 Jun 2000 14:45:20 -0700 (PDT)

That's right. We know that TCP's adaptation does not perform well on
Diffserv AF PHB. There is research trying to make TCP work better on AF
PHB. I think it is just refinement to have TCP get its desired service. Is
it possible to higher performance with end-protocol adaptation? If not,
why do I need adaptation? It is network's responsiblity to provide
service. Within the same class of service, network should have mechanisms
to control it (I know there is TCP and will be there).

Alper K. Demir

---

From: Grenville Armitage <[email-masked]>
Date: Wed, 21 Jun 2000 15:39:14 -0700

demir wrote:
	[..]
> There is research trying to make TCP work better on AF
> PHB.

This thread is so full of ambiguous statements like the above
I'm lost as to what is actually being discussed. Would anyone
care to clarify?

cheers,
gja

---

From: demir <[email-masked]>
Date: Wed, 21 Jun 2000 16:23:35 -0700 (PDT)

> This thread is so full of ambiguous statements like the above

Sorry for the ambuguity. What I meant by research is just to give an
example. You can refer:
http://www.aciri.org/floyd/tcp_diff.html
Do they really differ from AF PHB of Diffserv? I guess not. My
question was: Is it necessary/does it help to use adaptive-end-protocols
on a QoS-network (not for best-effort-like service on this QoS-network, 
exlude best-effort-like service). If so, why and how?

Alper K. Demir

---

From: Fred Baker <[email-masked]>
Date: Wed, 21 Jun 2000 16:16:42 -0700

>I'm lost as to what is actually being discussed. Would anyone

I have repeatedly asked Denir for pointers to the research he is 
discussing. I haven't gotten a response. I'd really like to understand. I 
suspect it has to do with "strict" token buckets or token buckets with 
excess bursts of zero bytes, and not using WRED.

---

From: demir <[email-masked]>
Date: Wed, 21 Jun 2000 17:09:17 -0700 (PDT)

> excess bursts of zero bytes, and not using WRED.

When I received your email, I replied for pointers ASAP. AF PHB gives
reference to:
[Clark]   Clark, D. and Fang, W., Explicit Allocation of Best Effort
             Packet Delivery Service.  IEEE/ACM Transactions on
             Networking, Volume 6, Number 4, August 1998, pp. 362-373.

This paper also talkes about the problems that TCP face with. 

Alper K. Demir

---

From: Grenville Armitage <[email-masked]>
Date: Wed, 21 Jun 2000 21:08:56 -0700

demir wrote:
	[..]
> My

This question is imprecise. Refine "a QoS-network" and perhaps we
can all start answering the question. There is no such thing as a
simple QoS-network, there are however networks (or paths) with
certain QoS characteristics. Define the characteristics you have
in mind (characteristics, not mechanisms) and perhaps we can discuss
whether an adaptive protocol is useful or not over a path with the
characteristics you have in mind.

cheers,
gja

---

From: Panos GEVROS <[email-masked]>
Date: Thu, 22 Jun 2000 20:55:29 +0100

Demir, 

if the QoS network is supposed to provide "guaranteed" (hard) QoS (in terms of 
transmission rate, burst size) then adaptation is not required as long as the 
end-system keeps its flows within the specifications
(loss can be dealt by reliability mechanisms - not rate control ones where i 
think adaptation refers)

if the guaranntees provided are probabilistic (or relative or "softer") by any 
means then intuitively adaptation is needed for optimal performance,

Why? 
because you cannot be sure when you get the best out of your network and you 
always want to hunt for more, you always hope for the better :)

How?
- not as easy to answer , it depends iow
what sort of adaptation is needed (the best strategy) it will be determined by 
what sort of guarantees are provided, *and* what sort of information becomes 
available to the end-system

the fact that TCP congestion control is great for Best effort Internet does 
not necessarily make it suitable for networks that offer other types of service

hope it helps
Panos

Grenville Armitage writes:
 |
 |
 |demir wrote:
 |	[..]
 |> My
 |> question was: Is it necessary/does it help to use adaptive-end-protocols
 |> on a QoS-network (not for best-effort-like service on this QoS-network,
 |> exlude best-effort-like service). If so, why and how?

 |
 |This question is imprecise. Refine "a QoS-network" and perhaps we
 |can all start answering the question. There is no such thing as a
 |simple QoS-network, there are however networks (or paths) with
 |certain QoS characteristics. Define the characteristics you have
 |in mind (characteristics, not mechanisms) and perhaps we can discuss
 |whether an adaptive protocol is useful or not over a path with the
 |characteristics you have in mind.
 |
 |cheers,
 |gja

---

From: Henning Schulzrinne <[email-masked]>
Date: Thu, 22 Jun 2000 09:50:20 -0400

Papers on the relationship of QoS guarantees and diff-serv will be
appearing in the Journal of Communications and Networks (JCN) in the
June issue, due out soon.

Papers on the interaction of TCP and ABR can be found in work by Sisalem
and myself, and the references therein, all at
http://www.cs.columbia.edu/~hgs/netbib. I believe there have been a
number of papers at Infocom on this topic as well
(http://www.ieee-infocom.org)

---

From: Dan Grossman <[email-masked]>
Date: Fri, 23 Jun 2000 11:11:58 -0400

> http://www.cs.columbia.edu/~hgs/netbib.
The Netbib search engine does not hit any of these papers on queries on 
"Schulzrinne"  but does hit a few on "Sisalem". I'm quite sorry to say I 
missed Dorgham's thesis when I did a literature search a few months ago with 
Netbib on ABR and TCP.    

His results were about what I'd expect.  Selecting the better algorithms when 
there was a choice and reasonable parameter values resulted in excellent 
fairness and utilization for TCP over ABR.   Selecting legacy algorithms and 
bad parameter values resulted in poor performance.  TCP and IP over an 
end-to-end ABR service offered much better results than a heterogenious 
network.     The one thing I wish he'd done differently was in his 
heterogenious environment simulation would be to put RED  into the routers at 
the edges of the ATM cloud:  I suspect he'd have seen better fairness and 
utilization.

Dan