Subject: RE: PILC ASYM: Wording of Motivation (section 3).

From: "Mandin, Jeff" <[email-masked]>
Date: Sun, 23 Sep 2001 19:08:20 +0300

Dan, Gorry:

See my comments inline.

>> 3. Motivation
>> 
>> Asymmetric characteristics are exhibited by several network
>> technologies, including Hybrid Fiber Coax (HFC) [CR98] and DOCSIS cable

>Probably some misunderstandings here (I don't have the [CR98] reference).  
>First, there are two standards (plus a significant deployed base of 
>non-standard stuff) for data over CATV.  The dominant one is DOCSIS, which 
>also has a European variant for use over plants with 8MHz downstream
channels 
>and other differences.  There is a smaller base which uses the DAVIC
1.2/DVB 
>standard.  I believe the issues apply to both the standards, and at least
to 
>the proprietary systems that I'm familiar with.  Second, both DOCSIS and
DAVIC 
>1.2 can run over purely coax systems, as long as the reverse path is
enabled;  
>in practice, transmission performance over long cascades is usually poor,
but 
>it is occasionally done (leading to worst case assymetries).  More proper
to 
>say CATV rather than HFC. 

Yes, "CATV" is probably the best term here as the Fiber Node conversions are
transparent(and not absolutely necessary) to Docsis.  And the CR98 
reference can be eliminated since it describes a type of CATV network that
is 
AFAIK no longer in use (CR98 is concerned with TCP adjustments to cope with 
HFC upstream bit errors - whereas Docsis uses FEC to deal with bit errors).

>> In these
>> schemes the transmission of ACKs may be as costly as transmission of
>> data packets. These MAC interactions may result in significant
    ^
>I don't understand this sentence.  The MAC and below don't care whether a
PDU 
>is TCP data or TCP ACK.  

The point being made is that transmitting minimal length packets has the 
same overhead as long packets(precisely because the MAC doesn't care what 
the packet is).

> Are you really trying to say that upstream 
>transmission is more costly than downstream transmission?

No.  The intent is that upstream ACKs are almost as expensive as upstream
data.

Perhaps the phrase "upstream transmission" should be used instead of
transmission -
but I think the intent is understood from the preceding sentences.

>> In DOCSIS/HFC cable networks, the
>> analogue channels assigned for upstream communication may be narrower
                                                         ^
                                                         are
>> and more noisy than those assigned for the downstream; as a consequence,
>      ^
>      usually more

>> the upstream and down stream links differ in their transmission rate and
                                                       ^
>                                                     possible symbol rate
(or 
>you could say "transmission rate" and omit modulation format).
>> modulation format. 

Agreed.  The original text made reference to the "symbol rate" and
"constellation";
the text was changed presumably to eliminate PHY-specific terms.  I think
its 
worthwhile to mention both the symbol rate and the modulation.

>> Cable service providers typically therefore offer a
>> downstream transmission rate (e.g. 27-52 Mbps) which is much higher
>> (e.g. of 2-200 times) than that used for the upstream direction. The

>Well, no, not quite.  They offer the best rates they can get out of their 
>plants, divided by the number of active subscriber.  

It also depends on the bandwidth consumption of each active subscriber of
course.  Providers these days typically put a per-user upstream cap of 128K
(this was introduced by @home 2 yrs. ago as a "service enhancement").  

Per-user downstream rate of course depends on plant capacity and user 
activity, but providers claim rates up to "100 times faster than a 
dialup modem"(ie. 5.6 Mbps).

Providers also associate multiple upstream channels to a single downstream
channel(hence the MAP message transmitted on the downstream includes an
Identifier of an upstream channel).

>Try:
>In DOCSIS 1.0 and 1.1,  rates of upstream data bursts
>(including Physical Layer overhead)  can range from 166.4 kbit/s to 9.022 
>Mbit/s, while downstream rates are either 26.97 Mbit/s or 38.81 Mbit/s.
The 
>rates available in each direction depend upon analog parameters in the
cable 
>plant.  Various complexities related to burst operation and the MAC further

>add to upstream to downstream assymetry.  

I would suggest focus on the typical per-user provisioning currently offered
by providers.

>> The asymmetry in capacity may be substantially increased when TCP ACKs
>> share the available upstream capacity with higher priority traffic (e.g.
>> support of Voice over IP using the Unsolicited Grant service in DOSCSIS
>> [DS00], CBR cells in ATM over ADSL, or IP QoS link scheduling).  When
>> upstream capacity is shared by a number of end hosts, the asymmetry may
>> be mitigated in some cases by using multiple upstream links for each
>> downstream link.

>A few corrections (a bit more than nits):  the problem occurs with TCP ACKs

>over a best effort service with no reserved capacity;  if there's reserved 
>upstream capacity, the problem is mitigated or goes away.  

Docsis requires that an Unsolicited Grant Service class transmit a valid
packet at each grant.  This means that "reserved capacity"/UGS isn't usable 
for a TCP flow(or a group of TCP flows) which are naturally bursty.

Regards,

Jeff Mandin
Software System Architect
Texas Instruments - Cable Broadband Communications
email: [email-masked]

---

From: Dan Grossman <[email-masked]>
Date: Mon, 24 Sep 2001 12:21:52 -0400

> but I think the intent is understood from the preceding sentences.
So perhaps it could be state more clearly as:  
In these schemes, the amount of overhead per upstream transmission is so 
significant that transmitting short packets (including TCP ACKs) is almost as 
costly as transmitting MTU-sized packets.

> course.  
I suppose it's a question of how you define "active".  What I was thinking 
about is something along the lines of "when TCP has packets to send AND the 
send window is open."  That is, active on a small enough timescale that it 
isn't quite right to be talking about sending at different rates.  Obviously, 
you're right if you calculate rate by the number of bytes sent over longer 
timescales.  

> Providers these days typically put a per-user upstream cap of 128K
YMMV.  
 
> dialup modem"(ie. 5.6 Mbps).
Marketing claims are necessarily oversimplified to aid comprehension, and tend 
to be optimistic.   Some providers have a pretty good handle on their plant 
and traffic engineering, and others don't.  Probably best not to include 
claims like this in the draft.
> Identifier of an upstream channel).
The draft should mention this.

> I would suggest focus on the typical per-user provisioning currently offered
These are maximums, not minimums, although, yes they're worth mentioning.  
Part of the problem is balancing Gorry's desire not to get too detailed on 
this stuff with the sheer complexity of how data-over-CATV and ADSL interact 
with TCP.
> for a TCP flow(or a group of TCP flows) which are naturally bursty.
True, but I was also thinking about the minimum reserved traffic rate 
parameter in the best effort and the two polling services, possibly in 
combination with the nominal polling interval and transmitted poll jitter.

---

From: Gorry Fairhurst <[email-masked]>
Date: Tue, 25 Sep 2001 13:38:16 +0100

Gorry Fairhurst: 
A new rev of the draft should be out soon. I have tried to take
on-board the issues raised here. Please see in-line comments.

> costly as transmitting MTU-sized packets.

OK. I'll work something like this into the wording.

> > (this was introduced by @home 2 yrs. ago as a "service enhancement").

That's interesting.
128 kbps x 200 = 25 Mbps.... 
To me, that means the current text seems to set the right order of magnitude?
The reason this was introduced was to provide the reader with some
feel for the NORMALISED BANDWIDTH RATIO, k, defined in section 2.

> YMMV.

I now have an ITU reference to traffic engineering of DOCSIS/HFC for 
IP telephony from Waisum, (see separate mail thread) so I'll add that too.

I've also replaced HFC with CATV, since it seems a more generic term.

> claims like this in the draft.

I do agree, there are also quite a number of varients of modulation/rates
in DOCSIS, and different service offerings, so I'd rather steer away from
operational issues, and details of current deployments in this ID.

> The draft should mention this.

This I do not understand - please tell me more about the MAP message and
how this works, and how much it has been deployed. ---- Can you explain more?

> this stuff with the sheer complexity of how data-over-CATV and ADSL interact

That's exactly what concerns me. I am actually very interested in the IP-level
stuff and how the two layers interact. I've always thought this was exactly
why PILC was formed to look at this sort of interaction. It's also seems to
be something which is generally not well understood. There may be scope
to document/dicsuss/recommend about this in a separate document? I note
the ITU are looking at this!

The ASYM ID, about TCP interactions and mitigations.

> combination with the nominal polling interval and transmitted poll jitter.

Interesting too... so there seems to be some advice we can offer designers
of such links advice on IP service to reduce the impact of link asymmetry?

---

From: Dan Grossman <[email-masked]>
Date: Tue, 25 Sep 2001 10:22:04 -0400

Gorry, 
> on-board the issues raised here. Please see in-line comments.
Thanks.   Further inline.
 
> feel for the NORMALISED BANDWIDTH RATIO, k, defined in section 2.
Remember that the 128k bit/s number is an upper bound;  the actual upstream 
and downstream rate (and thus the instantaneous value of k) will vary 
dynamically.  That's the key point of all of this nattering about the fine 
points of DOCSIS

Which is not to say that the draft shouldn't include at least the tops of the 
waves... most consumers with broadband Internet access have either DOCSIS or 
ADSL, making the ASYM issues surrounding these technologies very important to 
the Internet community.  I'd encourage the draft to err in favor of being more 
rather than less informative.
> operational issues, and details of current deployments in this ID.
I think you can summarize this stuff in a few sentences, like the ones below, 
and Jeff's sentence about maximum rates.  This to help readers have some 
intuition on the bounds and dyanamics of k.

> how this works, and how much it has been deployed. ---- Can you explain more?
The point is that one of the mitigations to the ASYM problem in DOCSIS (and I 
think also DAVIC 1.2/DVB) has been to allow multiple upstream channels to be 
associated with a single downstream channel.  Cable modems can be assigned to 
transmit on any upstream channel which is associated with the downstream 
channel.   The business about MAPs is an implementation detail which doesn't 
need to be included in the draft.
> stuff and how the two layers interact.
Actually, the interactions are between the subnet and TCP.  Subnet dynamics 
has a significant effect on TCP dynamics.  
> I've always thought this was exactly
> be something which is generally not well understood. 
Yes...

> There may be scope
That's TCP interactions with subnet dynamics, and mitigations at both layers 
to help TCP work better.

Really, you don't need two documents.  Jeff and I have provided what amounts 
to a few paragraphs that describe the ASYM dynamics of two particularly 
important technologies.  I think that with a little editing, those paragraphs 
are necessary (to the purpose of the draft) and sufficient (to understanding 
the issues).
> of such links advice on IP service to reduce the impact of link asymmetry?
Jeff and I were talking at a level of detail which doesn't need to be 
reflected in the draft.  The point here is that both DOCSIS and ATM/ADSL 
provide optional rate reservation for the upstream (which some operators may 
offer, maybe at extra cost).  These can be used as a mitigation at the subnet 
layer, to limit the value of k.

Dan

---

From: Gorry Fairhurst <[email-masked]>
Date: Wed, 26 Sep 2001 16:53:48 +0100

Thanks to all who provided input.

Following Lloyd Wood's comments and those from other people in PILC,
we have revised section 3.  The suggestion to remove specifics of
under-lying link technologies (e.g. satellite, DVB) and instead 
focus on the generic issues, seems much better.

Mahesh and I have just been through the recent mail which were sent,
we've included the ADSL references too (thank's Dan) amd the ITU-T
reference to E.681 - Waisum, can you tell us when this is approved
and formally published and we'll update the reference?

We now have text which we hope points to the DOCSIS/ADSL MAC issues
from Dan, Jeff. This also clarifies a number of other issues in
section 3.

We've also added some text on QoS interactions/shared upstream 
links found in DOCSIS/EURO-DOCSIS, etc and their impact.

Thanks are also due to Andrew Valentine (chair IPCDN WG) 
here for reviewing text and making suggestions on what not to say.

The recommendations and conclusions have been revised along
the lines described at the IETF meeting in London.

Thanks also to all the other people who suggested things to 
be deleted and added.

As agreed, a new ID (-06) will be posted today with the hope 
that this may soon move to WG last call.

Gorry