Subject: Re: Draft on Advice for Internet Subnetwork Designers

From: Dan Grossman <[email-masked]>
Date: Wed, 01 Mar 2000 17:17:27 -0500

Comments:

1) Shortly before the Washington meeting, I sent a suggestion to change the 
Introduction/Overview section as follows.  

> ...seminal paper "End-to-End Arguments in System Design" [SRC81]. This paper 
> argued that many -- if not most -- network functions are best implemented on 
> an end-to-end basis, i.e., at the higher protocol layers.  Duplicating these 
> functions at the lower levels can be needlessly redundant or harmful. 
> However,  there are also cases where duplication can be justified by 
> considerations such  as performance, availability or security.  Examples 
> might include link layer  retransmission on an unusually lossy channel, e.g., 
> mobile radio, encryption  of the entire link layer protocol over a shared 
> medium, or  redundant paths at the link layer for fault recovery.  
> Duplication of functions should be done only based on an understanding of 
> system level implications, including any side-effects on the higher layers.

I still believe rather strongly that this is more appropriate than the current 
wording.  Since it has been quite a while since the last discussion of the 
document on the mailing list,  a reminder seems appropriate.

2) Some initial thoughts on what we should do with the empty section on 
buffering, congestion and flow control.  Let's discuss on the list, and I'll 
draft text after the discussion:
a) [Following owes a great deal to a side-discussion I had with Mark Allman] 
TCP needs lots of buffering.  An ability to buffer bursts of up to (fixed 
delay)*(bottleneck rate) per TCP connection in each node is ideal if 
impractical.  Problem is that neither the fixed delay nor the bottleneck rate 
is necessarily known at any node.  A good approximation is fixed delay of the 
link * link bandwidth.  For links that have variable delays, the approximation 
should be based on the max.  Also, the maximum number of TCP connections per 
node is very difficult to determine a priori, and is not subject to any kind 
of admission control (except perhaps at a PEP).  A SWAG has to suffice.  Some 
kind of mechanism to measure delay on the link can be useful in autotuning 
buffer sizes.  In general, more buffers are better than fewer.  For example, 
TCP performance in very early ATM switch implementations suffered horribly 
from the small buffer sizes.

b) There is a rough consensus (not unanimous) in the research community that 
active queue management is important to improving fairness, link utilization 
and throughput. Reference RFC 2309.  Followed by a brief introduction to 
active queue management, with RED as an example. An important part of the 
concept is that active queue management attempts to keep average queue depths 
short, while accommodating short term bursts.  It may be advisable for subnets 
that perform queueing and discarding to consider implementing some form of 
active queue management, especially if large aggregates of TCP connections are 
likely to share the same queue.  It may be less effective in the case, for 
example, of an ATM switch that implements per-VC queueuing.  Note, 
incidentally, that active queue management algorithms in general are highly 
sensitive to parameter settings and reference Borden and  Fioriu paper.  

c) If the subnet does fragmentation and reassembly, make sure that if it 
becomes necessary to discard, then discard entire frames, not just scattered 
fragments. For ATM, this specifically means using EPD/PPD, and reference ATM 
Forum TM 4.0 specification.   Reference Romanow paper. 

d) If the subnet includes flow control that shapes the rate at the edge of the 
subnet to match a share of the bottleneck bandwidth in the subnet (e.g., ATM 
ABR), this has the effect of pushing congestion to the edge of the subnet.  In 
such a case, adequate buffering is needed at the edge of the subnet (see a 
above).  Whether or not there is benefit in this kind of flow control is 
controversial, and there have been simulation studies that go both ways.  
Anecdotally, some large networks have used ABR sucessfully to carry IP 
traffic, but no published results.

e) Flow control mechanisms that mark but do not shape frames in excess of fair 
share of the bottleneck rate (e.g., frame relay FECN and BECN) are at best 
useful only in the context of IP ECN (RFC 2481).  New subnets would do well to 
have flow control mechanisms that have that semantics (i.e., feedback signal 
based on EWMA buffer size, feedback set probabilistically proportional to 
difference between two thresholds, or always set when second threshold 
exceeded) so that feedback could be 'bubbled up' from the subnet layer to the 
IP header.  

3) I have some ideas on QoS, which I'll try to organize and send out over the 
next few days.

---

From: Dan Grossman <[email-masked]>
Date: Mon, 06 Mar 2000 16:58:11 -0500

My thoughts on buffers and queueing didn't seem to generate much as much 
discussion as I'd hoped.  However, since the deadline is upon us, here is some 
text that Phil can insert into the draft.

-----------------------------------------

Buffering, flow control and congestion control
Many kinds of subnets have points at which there is contention for link 
transmission resources.  At some of these points, the rate of the link may 
vary.  At such points, there must be a queueing system, including buffering, 
scheduling and a capability to discard excess subnet packets.  These queueing 
points may also be part of a subnet flow control or congestion control scheme.

For the purpose of this discussion, a layered model of the interaction between 
the IP layer and the subnet layer is not helpful.  Instead, we adopt the frame 
of reference of a packet.  At each queueing point -- whether that point is at 
the IP layer or at the subnet layer -- the packet experiences delay for a 
period of time (which depends on competing traffic and the scheduling 
discipline), and is subjected to a local discarding policy.   If there are 
flow control or congestion control mechanisms in the subnet that do not rely 
on TCP, these must operate on the scheduling discipline and/or the discarding 
policy in the subnet or in the routers at the edges of the subnet, since TCP 
has no way to operate on their semantics and IP has no way to  be able to pass 
their semantics to TCP;  however, see below for one exception.

TCP traffic, and especially aggregated TCP traffic, is bursty.  As a result, 
instantanious queue depth at a queueing point can vary dramatically, even in 
nominally stable networks.  For optimal performance, it is desirable that 
packets be dropped in a controlled fashion, rather than due to running out of 
buffers.  As a rule of thumb, nodes should be engineered to have buffering 
available for each TCP connection equal to the product of the maximum 
instantanious rate at which packets can be transmitted on the link (which is 
usually the link rate, but may be further limited by a shaper) and the 
propagation delay of the link.  For links that have variable propagation 
delays (such as having diverse paths, or variable depth interleavers), the 
longest likely propagation delay should be used.   

Unfortunately, since the maximum number of TCP connections at a queueing point 
can neither be known with certainty a priori, nor limited by an admission 
control policy (except, perhaps, at a PEP), buffer sizes in subnet devices can 
at best be engineered based on crude estimates.   In general, it is wise to 
err in favor of too much buffering rather than too little.  It may also be 
useful for subnets to incorporate mechanisms for measuring propagation delay, 
to assist in buffer sizing calculations.

There is a rough consensus in the research community that  active queue 
management is important to improving fairness, link utilization  and 
throughput [RFC2309].   Although there are questions and concerns about the 
efficacy of active queue management (e.g., see [MBDL99]), it is widely 
considered an improvement over tail-drop discard policies.   

One well known example of an active queue management the Random Early 
Detection (RED) algorithm [RED93].  RED maintains an exponential weighted 
moving average of the queue depth.  When this average queue depth is between a 
maximum threshold max_th, and a minimum threshold min_th, packets are dropped 
with a probability which is proportional to the amount by which the average 
queue depth exceeds min_th.  When this average queue depth is equal to max_th, 
the drop probability is equal to a configurable parameter max_p.  When this 
average queue depth is greater than max_th, packets are always dropped.  
Numerous variants on RED appear in the literature, and there are other active 
queue management algorithms which claim advantages over RED in various 
dimensions.

Active queue management algorithms forms a control regime wherein dropped 
packets are treated as a feedback signal.  Randomization of dropping tends to 
break up the observed tendency of TCP windows belonging to different TCP 
connections to become synchronized by correlated drops, and also imposes a 
degree of fairness on those connections which properly implement TCP 
congestion avoidance.    Another property of active queue management 
algorithms which is particularly important to subnet designers is that they 
attempt to keep average queue depths short, while accommodating large short 
term bursts. 

Since TCP neither knows nor cares whether packet loss due to congestive 
discard is due to congestion at the IP layer or in a subnet, it may be 
advisable for subnets  that perform queueing and discarding to consider 
implementing some form of active queue management.  This especially holds if 
large aggregates of TCP connections are likely to share the same queue.  
However, active queue management may be less effective in the case of many 
queues carrying smaller aggregates of TCP connections, as for  example, in an 
ATM switch that implements per-VC queueuing.  

Note,  incidentally, that the performance of active queue management 
algorithms is highly sensitive to settings of configurable parameters, and 
also to factors such as RTT [MBB00][FB00].  

Some subnets, most notably ATM, perform segmentation and reassembly at the 
edge of the subnet, so as to be able to forward subnet packets of size less 
than an MTU.  There are advantages and disadvantages to doing this.  However, 
if this is done, care should be taken in designing discard policies. Subnet 
packets with missing fragments must be destroyed by the subnet, as they are of 
no use to  TCP.  Thus, if the subnet discards random and uncorrelated 
fragments of IP  packets, then the balance of these packets constitute an 
unproductive load on the subnet and can markedly degrade end-to-end 
performance. [RF95]  Therefore, subnets should attempt to discard entire IP 
packets.  If a portion of an IP packet has been forwarded and discarding of 
subnet packets which are fragments of the IP packet becomes unavoidable, then 
either all remaining fragments or all but the fragment marking the end of the 
packet should be discarded.   For ATM subnets, this specifically means using 
Early Packet Discard and Partial Packet Discard [ATMFTM]. 

Some subnets might includes flow control mechanisms that effectively require 
that the rate of traffic flows be shaped as they enter the subnet.  One 
example of such a subnet mechanism is in the ATM Available Bitrate (ABR) 
service category [ATMFTM].   Such flow control mechanisms have the effect of 
making the subnet nearly lossless by pushing congestion into the IP routers 
edges of the subnet.  In such a case, adequate buffering and discard policies 
are needed in these routers to deal with a subnet which appears to have 
dynamically varying bandwidth.   Whether or not there is benefit in this kind 
of flow control is controversial, and there have been numerous simulation and 
analytical studies that go both ways.  It appears that some of the issues that 
lead to such different results include sensitivity to ABR parameters, use of 
binary rather than explicit rate feedback, use (or not) of per-VC queueing, 
and the specific ATM switch algorithms selected for the study.   Anecdotally, 
some large networks have used IP over ABR to carry TCP traffic, and have 
claimed it to be sucessful, but have published no results.
 
Another possible approach to flow control in the subnet would be to work with 
TCP Explicit Congestion Notification (ECN) semantics [RFC2481].  Routers at 
the edges of the subnet, rather than shaping, would set the ECN bit in those 
IP packets that are received in subnet packets that have an ECN indication.  
Nodes in the subnet would need to implement an  active queue management 
protocol which marks subnet packets rather than dropping. However, RFC2481 is 
presently experimental, and TCPs which can use ECN are not widely deployed. 

------------
References 
 
[ATMFTM] The ATM Forum, "Traffic Management Specification, Version 4.0", April 
1996, document af-tm-0056.000 (www.atmforum.com).

[FB00] Firoiu V., and Borden M., "A Study of Active Queue Management for 
Congestion Control" to appear in Infocom 2000

[MBB00] May, M., Bonald, T., and Bolot, J-C., "Analytic Evaluation of RED 
Performance" to appear INFOCOM 2000

[MBDL99] May, M., Bolot, J., Diot, C., and Lyles, B., Reasons not to deploy 
RED, technical report, June 1999.

[RF95] Romanow, A., and Floyd, S., Dynamics of TCP Traffic over ATM Networks. 
IEEE JSAC, V. 13 N. 4, May 1995, p. 633-641.

[RFC2481] Ramakrishan, K. and Floyd S.,  "A Proposal to add Explicit Congestion
          Notification (ECN) to IP" RFC2481 January 1999

---

From: Grenville Armitage <[email-masked]>
Date: Mon, 06 Mar 2000 14:24:24 -0800

> text that Phil can insert into the draft.

Nice text.

cheers,
gja
________________________________________________________________________
Grenville Armitage                            http://mh005.infi.net/~gja
Bell Labs Research Silicon Valley

---

From: Phil Karn <[email-masked]>
Date: Wed, 8 Mar 2000 00:00:18 -0800

>document on the mailing list,  a reminder seems appropriate.

Okay, here's my current proposed version of this paragraph. I've
adopted your language with a few minor changes. What do you think?

A major consideration in the efficient design of any layered
communication network are the appropriate layer(s) in which to
implement a given feature. This issue was first addressed in the
seminal paper "End-to-End Arguments in System Design" [SRC81]. This
paper argued that many -- if not most -- network functions are best
implemented on an end-to-end basis, i.e., at the higher protocol
layers.  Duplicating these functions at the lower levels can be
needlessly redundant or harmful. However, there are cases where
duplication can be justified by performance, availability or security
considerations. Examples include link layer retransmission on an
unusually lossy channel, e.g., mobile radio; link level encryption
intended to thwart traffic analysis; and redundant transmission
links to improve availability. Duplication of functions should be
done only with an understanding of system level implications, including
possible interactions with higher layer mechanisms.

--Phil

---

From: Phil Karn <[email-masked]>
Date: Wed, 8 Mar 2000 00:28:33 -0800

>If there are 

I am editing your text into the Buffering, Flow Control and Congestion Control
section, but I am having trouble parsing the above paragraph. Can you explain
and/or clarify it?

Phil

---

From: Phil Karn <[email-masked]>
Date: Wed, 8 Mar 2000 01:00:55 -0800

>PAGE 4: MTU Size

>It may be useful to note that using IPSEC in tunnel mode can increase
>the size of a packet as it travels across the network.  This can result
>in a smaller MTU.

I'm not sure what to do about this one. I'm inclined to agree with the
comments that say this is not unique to IPSEC but is a consideration with any
IP-in-IP encapsulation scheme -- all of which are at higher layers than the subnet.

Perhaps some text would be appropriate that says that while 1500 bytes
is an important MTU in the Internet because of the prevalence of
Ethernet, it would be useful to support a somewhat larger size to avoid IP fragmentation
when tunneling (which would include IPSEC) is used. Problem is, this still doesn't
avoid the fragmentation that would occur when the path between the tunnel endpoints
includes one or more Ethernets. So the benefits of a larger MTU are apt to be limited anyway.

Phil

---

From: Dan Grossman <[email-masked]>
Date: Wed, 08 Mar 2000 08:40:54 -0500

Phil,
Looks good.  Thanks.
Dan

---

From: Dan Grossman <[email-masked]>
Date: Wed, 08 Mar 2000 09:34:40 -0500

> section, but I am having trouble parsing the above paragraph. Can you explain
Yeah, that is a little awkward, and run-on.  Try:

In addition, some subnets may have flow control or congestion control 
mechanisms in addition to packet dropping and reliance on TCP behavior.  Such 
mechanisms can operate on components in the subnet layer, such as schedulers, 
shapers or discarders, and can  affect the operation of IP forwarders at the 
edges of the subnet.  However, with the exception of  RFC2481 explicit 
congestion notification (which will be discussed below), IP has no way to pass 
explicit congestion or flow control signals to TCP, and TCP would not react to 
such signals if they were available. 

Dan

---

From: Phil Karn <[email-masked]>
Date: Wed, 8 Mar 2000 08:13:48 -0800

>such signals if they were available. 

Ah, much better! I'll stick it in.

Phil

---

From: Phil Karn <[email-masked]>
Date: Wed, 8 Mar 2000 08:51:23 -0800

Gorry,

>Reference: MPEG-2 or DVB reference.

I'm editing in your text. Can you provide cites to these reference(s)?

Phil

---

From: Cannara <[email-masked]>
Date: Wed, 08 Mar 2000 09:49:01 -0800

Perhaps we should be considering at least the FDDI MTU of 4500B as a basis for
future statements?  This would cover all Ethernet-sourced pkts that get
tunnelled.  Routers from the major manuf. have interface specs for which this
is no problem.  Of course, the Ethernet folks are talking of larger pkts as
well.

Alex

---

From: Cannara <[email-masked]>
Date: Wed, 08 Mar 2000 09:55:51 -0800

FYI, a related suggestion was for IP to communicate info TCP could use by
passing up the IP ID (v4) via TCP's Urgent Pointer (with URG bit off) and the
receiving TCP would so get the ID of the packet sent, then return that ID when
it Acked that pkt.  The sender keeps a list of IDs vs TCP pkts and then knows
exactly what TCP packet was acked, even if there is a false retransmission
(due to a short timer, etc.) and can avoid going into backoffs simply because
of misjudgments of RTT, etc.  IPv6, of course, has messed this up (no ID).

Alex

---

From: Joe Touch <[email-masked]>
Date: Wed, 08 Mar 2000 10:10:07 -0800

> > >> system level implications, including any side-effects on the higher layers.

The paper does not exactly argue that these fns are best implemented
E2E;
it argues that:

	(I call this the E2E Principle:)
	End-to-end services cannot be provided solely by the composition 
	of hop-by-hop services.

	(I call this the E2E Corrolary:)
	Hop-by-hop services may be required for performance, but they enhance, 
	rather than replace, the corresponding end-to-end service.

The issue of whether they are needlessly redundent or harmful stems only
from these.
A HBH implementation is harmful only where it is used to incorrectly
replace an E2E one 
(violating E2E Princ.) or where it fails to enhance the concurrent E2E
version (violating
E2E Corrol.)

"best implemented as E2E" isn't exactly the point; the point is that HBH
is insufficient,
not 'relatively worse'. Duplication at lower layers is strictly a
performance
issue, by my read.

Is there a way of morphing that into the above??

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 08 Mar 2000 10:17:46 -0800

> >the size of a packet as it travels across the network.  This can result

Many protocols other than IPSEC can reduce the E2E MTU.
IPSEC is one, IPIP tunneling is another (it need not 
include or refer to IPSEC), as are any protocols that augment
the IP header with options.
 
> IP-in-IP encapsulation scheme -- all of which are at higher layers than the subnet.

Any protocol that augments the IP header can cause this, whether by 
encapsulation, option augmentation, etc.
 
> includes one or more Ethernets. So the benefits of a larger MTU are apt to be limited anyway.

A useful point may be "don't assume the MTU at all anymore," i.e., it
might be 
worthwhile to have designers of links _and_ upper protocols both drop
such preconceptions
they have about the link.

Joe

---

From: Dan Grossman <[email-masked]>
Date: Wed, 08 Mar 2000 14:05:44 -0500

> 	rather than replace, the corresponding end-to-end service.
Not just for performance, but otherwise agree thus far.
> not 'relatively worse'. 
I agree up to this point.  The old text was a bit of an oversimplification of 
the end-to-end paper.  The underlying issue in end-to-end was the historical 
battle between X.25 with Class 0 transport protocol and IP with TCP (or CLNP 
and Class 4 transport protocol).  X.25 attempted to provide an end-to-end 
reliable service using network layer (hop-by-hop) mechanisms, so that no 
retransmission mechanisms were required in the end systems.  The point was 
that although those kinds of mechanisms could produce an _almost_ reliable 
service at the cost of expensive mechanisms in the network, a _truly_ reliable 
end-to-end service could never be supported using only hop-by-hop mechanisms.  
The end-to-end paper then extends its exposition of this flaw to further argue 
for statelessness as a way of providing end-to-end availability despite 
network failures ("fate sharing"), and in general to call for extremely 
lightweight networks. 

>Duplication at lower layers is strictly a
> performance
> issue, by my read.
Here's where I disagree.  There are hop-by-hop (or, rather) subnet functions 
which are in some sense redundant with end-to-end functions, but which are 
still needed for security and availability, as well as performance.  The 
examples in the new text are fairly generic examples.  Consider the case of 
availability:  various kinds of subnets -- SONET/SDH, ATM, MPLS, optical -- 
provide some kind of failover protection switching.  These are  stateful and 
redundant with IP layer re-routing, and could if improperly used result in 
harmful interactions with the IP layer.  On the other hand, they can 
accomplish a failover in 50 ms, rather than the 30 seconds or more that it 
takes for OSPF to reconverge;  if Five-9s availability is a network objective, 
then these additional mechanisms are needed.

> Is there a way of morphing that into the above??
...[SRC81].   This paper argued that certain kinds of end-to-end services 
cannot be provided solely by the composition of hop-by-hop mechanisms, and 
instead require mechanisms that operate end-to-end. Duplicative presence of 
hop-by-hop mechanisms in the network can be needlessly redundant or even cause 
harmful interactions with end-to-end mechanisms. However...

---

From: Phil Karn <[email-masked]>
Date: Wed, 8 Mar 2000 11:29:10 -0800

>Is there a way of morphing that into the above??

I agree with your presentation. I'll try to come up with the appropriate language.

Phil

---

From: Phil Karn <[email-masked]>
Date: Thu, 9 Mar 2000 15:47:12 -0800

>it Acked that pkt.  The sender keeps a list of IDs vs TCP pkts and then knows

Ugh, what a layer-violating hack! I think a better way already exists
to deal with the false retransmission problem: TCP header timestamps.

Phil

---

From: Phil Karn <[email-masked]>
Date: Thu, 9 Mar 2000 15:48:46 -0800

>might be worthwhile to have designers of links _and_ upper protocols
>both drop such preconceptions they have about the link.

That's a good idea. Not only does tunneling mess up assumptions about
MTUs and the distribution of packet sizes, but so do the newer TCP
header options (timestamps, etc).

phil

---

From: Phil Karn <[email-masked]>
Date: Thu, 9 Mar 2000 15:45:17 -0800

Okay, here's the current wording of the end-to-end principle section.
Any more comments? --Phil

A major consideration in the efficient design of any layered
communication network are the appropriate layer(s) in which to
implement a given feature. This issue was first addressed in the
seminal paper "End-to-End Arguments in System Design" [SRC81]. This
paper argued that many functions can be implemented properly *only* on
an end-to-end basis, i.e., at the higher protocol layers, outside the
subnetwork. These functions include ensuring the reliable delivery of
data and the protection of information through cryptography.
Duplicating these functions at the lower protocol layers (i.e., within
the subnetwork) can be needlessly redundant or even harmful to cost
and performance.

However, partial duplication of functionality in a lower layer
can *sometimes* be justified by performance, security or availability
considerations. Examples include link layer retransmission on an
unusually lossy channel, e.g., mobile radio; link level encryption
intended to thwart traffic analysis; and redundant transmission links
to improve availability. Duplication of protocol function should be done only
with an understanding of system level implications, including possible
interactions with higher layer mechanisms.

---

From: Cannara <[email-masked]>
Date: Thu, 09 Mar 2000 19:40:29 -0800

Phil, this wasn't suggested as a "thing to do", but an illustration of what
could be done, even just experimentally or by simulation, to demonstrate the
utility of addressing TCP's blind side.  If timestamps can be trusted, then
maybe that's a way.  In any case, considerable performance in real situations
is sacrificed because we've gone years without taking significant action to
help TCP avoid making costly mistakes.  "Ugh" is indeed the right word for
those behaviors.

Alex

---

From: Cannara <[email-masked]>
Date: Thu, 09 Mar 2000 19:44:30 -0800

Sounds good Phil, and I'd add something a bit stronger in relation to
higher-layers that, in fact, are not fully capable of offering reliability and
performance under circumstances that the lower layers can do better on.  This
relates to the problems TCP has with noncongestive loss, etc.

Alex

---

From: Dan Grossman <[email-masked]>
Date: Fri, 10 Mar 2000 11:25:33 -0500

> implement a given feature. This issue was first addressed in the
Suggest changing 'feature' to 'function'.
I don't believe that it was first addressed in the Saltzer, Reed and Clark 
paper.  If I remember, this was an ongoing debate, and Saltzer, Reed and Clark 
captured one set of views.  
> subnetwork. These functions include ensuring the reliable delivery of
I think Joe's point was that certain end-to-end services such as reliable 
delivery and
confidentiality cannot be provided solely by composition of hop-by hop 
services.
> data and the protection of information through cryptography.
try... confidentiality and message integrity
> can *sometimes* be justified by performance, security or availability
why the extra emphasis?

---

From: Joe Touch <[email-masked]>
Date: Fri, 10 Mar 2000 09:09:51 -0800

> relates to the problems TCP has with noncongestive loss, etc.

Is this saying that the lower layer is required to help the higher,
or that the lower layer replaces the higher?

WRT the latter, E2E asserts that HBH may be required for performance, 
but not correctness. Is there a counterexample?

Joe

---

From: Joe Touch <[email-masked]>
Date: Fri, 10 Mar 2000 09:06:01 -0800

> captured one set of views.

IMO, "captured" == "addressed" (or close enough).

> confidentiality cannot be provided solely by composition of hop-by hop

The *only* above appears to suffice; it might be more clear to
include the caveat. Further, SRC81 didn't say 'some functions'
so much as 'many _end-to-end_ functions'.

> > can *sometimes* be justified by performance, security or availability

These two paragraphs appear to contradict each other. It may be
useful to unify them into a single statement that:

	Where HBH is implemented without consideration of E2E,
	it can be needlessly redundent or even harmful to cost and performance.
	However, HBH services can be designed to assist, rather
	than replace or ignore, their E2E counterparts. In this latter
	case, they can improve the performance of the E2E function.

Joe

---

From: Phil Karn <[email-masked]>
Date: Fri, 10 Mar 2000 09:29:42 -0800

Wow, I never thought the wording of a section on the end-to-end principle
would generate so much discussion!

I worded the current sections the way I did because of my observation
that many subnetwork designers do not understand that services such as
reliable delivery can only be properly implemented on an end-to-end basis,
so they go ahead and put reliability into the subnet without any regard
to whether it hurts or helps overall performance -- because they're thinking
about reliability, not performance.

With the ID deadline coming up in a few hours, I can't spend much more time
on this point. I'll maybe make one more pass to try to handle your comments,
but I have to move on. Now if we want to pick it up again after the deadline,
I suppose we could...

Phil

---

From: Cannara <[email-masked]>
Date: Fri, 10 Mar 2000 11:57:33 -0800

Since there've long been many experiences in networking outside the Internet
protocols, even before them, it's often been assumed, in my engineering
experience, that each layer does its every best to assure delivery to the next
node, without sacrificing available performance (note lack of saying: 
"bandwidth", which is erroneously used these days).

So, Enet for example, makes better use of a shared medium than does Token, at
most common usage levels, but it guarantees (within 32-bits of CRC) that
received data are right.  Its CSMA/CD is faster at resolving contention
without loss delays than Token and Token is better than any transport,
including TCP.  Token Ring even had "assured delivery" in the IBM spec, but
not implemented.  Imagine TCP on Enet -- if a collision were to go unsensed,
then we'd be back to Aloha Net, waiting 100s of milliseconds for TCP (NCP,
SPP...) to recover the loss that Enet recovers in a millisecond.  This
obviously makes no sense, which is what Abramson and Metcalfe determined
decades ago and solved by adding swift, effective contention handling for
layer 1.

The e-to-e principles are interesting, but in a real sense, bureaucratic: 
"work with TCP, it's our old buddy and we worked on it for all these
years..."  No layer is fully independent of those below, and those below have
the purpose of delivering services to layers above that are useful to these
upper layers.  There's no 3rd tablet from the mount that says "Thou shalt not
communicate between protocol layers", especially when things can be made to
perform better and more reliably by doing so.  Service to Mankind is
engineering's purpose, right?

So, since TCP counts and tracks individual packets (ok, segments), and has
algorithms pasted on over the years to help router (ok, gateway) queues avoid
embarassment, why should anyone balk at giving it more help from any layer
below -- its backoff algorithm is there to help the network-layer queues, not
just the end-station's slowness.  We've already seen a good deal of mucking
about among TCP/IP layers and even removal of useful services, such as the IP
ID in v6, apparently because no one really understood all the implications. 
We don't need many np simulations to understand that TCP needs help to perform
reasonably in a busy, lossy network. 

Oh, what's "HBH" -- should we buy or sell?  Thanks,

Alex

---

From: Phil Karn <[email-masked]>
Date: Fri, 10 Mar 2000 13:37:11 -0800

>below -- its backoff algorithm is there to help the network-layer queues, not

That's fine, as long as the added lower-level functionality really does
help performance overall. I've seen too many overly complicated subnetwork
designs where this isn't the case, though.

Phil

---

From: Patrick Cipiere <[email-masked]>
Date: Sun, 12 Mar 2000 09:36:35 -0800

Phil,

> Reference: MPEG-2 or DVB reference.

I did'nt see Gorry's answer, and did'nt see the references either in
draft-ietf-pilc-link-design-02.txt
so if you still need that

DVB reference is

  ETSI EN 301 192 V1.2.1 (1999-06)
 
MPEG-2 reference is

  ISO/IEC 13818

Patrick.

---

From: Cannara <[email-masked]>
Date: Mon, 13 Mar 2000 14:43:14 -0800

Yes, that goes without saying Phil.

Alex