Subject: [Tsvwg] Re: Why use Slow Start when destination is local?

From: Joe Touch <[email-masked]>
Date: Sun, 07 Oct 2001 18:04:03 -0700

Hi, Tim,

First, can we move this thread to the TSVWG mailing list? I thought 
TCPIMPL was closed as a WG, and this is definitely relevant to ongoing 
work in TSVWG.

Tim Moors wrote:

> Richard Stevens wrote in RFC 2001 and p. 311 of TCP Ill. vol.1 that the
> Tahoe release of TCP "performed slow start only if the other end was on a
> different network", but that this was changed with the Reno release of TCP
> so that slow start was always performed.
> 
> Does anyone know why Tahoe didn't perform slow start for local
> communication?  

Slow start is designed to avoid overloading router queues; for a local 
network, there is (presumably) no router to overload.

> Was there a reason for this, or was it just a mistake that
> was corrected in Reno?  Is the following justification valid?: MAC protocols
> (e.g. Ethernet) often control congestion for link-local communication,
> making transport layer congestion control unnecessary for local

> communication.

Ethernet doesn't control congestion. Its backoff deals only with 
collisions, which shouldn't (or can't?) occur on full-duplex links anyway.

> (I searched the archives, and the only discussion I found of this topic was
> in Mar 1997, where the definition of "local" was queried: local to a LAN, or
> local to a subnet as defined by the address class.  That discussion didn't
> address WHY there should be no SS for local communication.)

This was raised at the Minneapolis IETF in a hallway discussion with 
Reiner Ludwig and Andrei Gurtov, who posted the attached note to the 
freebsd-net mailing list; I think Andrei's goal was to find the origin 
of the code, though.

Joe

----------------

Date: Tue, 17 Apr 2001 18:04:38 +0300 (EET DST)
From: Andrei Gurtov <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>
Subject: initial congestion window
Sender: [email-masked]

Hi folks,

 At the last IETF meeting there were some debates around FreeBSD using a
16-KB initial congestion window in TCP when destination IP address is from
the local subnet. Does anybody remember when it was introduced into the
code and what kind of ideas were behind?

Some reasons were given why it may not be a good idea:

-the benefit of not having slow start on LANs is very small, i.e. some
milliseconds

-it is not a conformant TCP feature, i.e. not allowed by TCP Congestion
Control (RFC2581) and is explicitly given in Known TCP Implementation
Problems (RFC2525) "2.1 No initial slow start" and "2.3 Uninitialized
CWND"

-people may have the same subnet mask also over a slow PPP link. In this
case the effect of the huge initial window is quite bad, see for example
http://www.cs.Helsinki.FI/u/gurtov/papers/effect_of_delays_on_tcp_performance.pdf

-in case of congestion on Ethernet, packets queues build up at the
network interfaces in hosts and agressive TCP start-up behaviour can
further increase congestion losses

What are your thoughts on this?

Andrei

tcp_output.c:

int ss_fltsz = 1;
SYSCTL_INT(_net_inet_tcp, OID_AUTO, slowstart_flightsize, CTLFLAG_RW,
        &ss_fltsz, 1, "Slow start flight size");

int ss_fltsz_local = TCP_MAXWIN;               /* something large */
SYSCTL_INT(_net_inet_tcp, OID_AUTO, local_slowstart_flightsize,
CTLFLAG_RW,
        &ss_fltsz_local, 1, "Slow start flight size for local networks");

[...]
                if (
                     in_localaddr(tp->t_inpcb->inp_faddr)
                    )
                        tp->snd_cwnd = tp->t_maxseg * ss_fltsz_local;
                else
                        tp->snd_cwnd = tp->t_maxseg * ss_fltsz;

To Unsubscribe: send mail to [email-masked]
with "unsubscribe freebsd-net" in the body of the message

---

From: "Andrei Gurtov" <[email-masked]>
Date: Mon, 8 Oct 2001 15:06:07 +0300

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]On Behalf Of Joe
> Touch
> Sent: 8. lokakuuta 2001 4:04
> To: Tim Moors
> Cc: [email-masked]; [email-masked]
> Subject: [Tsvwg] Re: Why use Slow Start when destination is local?
>
>
> Hi, Tim,
>

...

> freebsd-net mailing list; I think Andrei's goal was to find the origin

Yes, and I tried to convince FreeBSD folks that blasting a full receive
window instead of doing a slow start is not a good idea.  The discussion in
freebsd-net can be found at
http://docs.freebsd.org/mail/archive/2001/freebsd-net/20010422.freebsd-net.h
tml

I agree with David that having the same subnet mask does not necessary mean
the hosts are connected with Ethernet. It could be WLAN or something else
slower than Ethernet. Also, the receiver can be busy or slow and sending a
huge burst to it can leads to packets being discarded.

Some other reasons were given why a huge initial window may not be a good
idea:

-people may have the same subnet mask also over a slow PPP link. In this
case the effect of the huge initial window is quite bad, see for example
http://www.cs.Helsinki.FI/u/gurtov/papers/effect_of_delays_on_tcp_performanc
e.pdf

-in case of congestion on Ethernet, packets queues build up at the
network interfaces in hosts and aggressive TCP start-up behaviour can
further increase congestion losses

Andrei

_______________________________________________
tsvwg mailing list
[email-masked]
http://www1.ietf.org/mailman/listinfo/tsvwg

---

From: Alan Cox <[email-masked]>
Date: Mon, 8 Oct 2001 15:43:24 +0100 (BST)

> http://docs.freebsd.org/mail/archive/2001/freebsd-net/20010422.freebsd-net.h

I agree - it isnt a good idea for a variety of reasons.

Firstly not every switch or bridge is exactly 100% quality
Secondly your slow start congestion avoidance is avoiding on wire congestion
Thirdly your local "network" on an interface can be many weird and wonderful
things - ATM lane, frame cloud, SMDS even PPP bridging - none of which react
well to being blasted with data causing congestion.

Its one of those ideas that only looks good in a paper

Alan

---

From: Joe Touch <[email-masked]>
Date: Mon, 08 Oct 2001 09:12:00 -0700

Andrei Gurtov wrote:

...
> -the benefit of not having slow start on LANs is very small, i.e. some

Milliseconds on a link with microsecond delays are enormous, BTW.

> Problems (RFC2525) "2.1 No initial slow start" and "2.3 Uninitialized

FWIW, RFC2525 refers to the complete omission of slow start 
specifically. It never refers to disabling it on specific links, even 
though this was in the code at the time.

> http://www.cs.Helsinki.FI/u/gurtov/papers/effect_of_delays_on_tcp_performanc

There are, BTW, a few other bits of code that depend on 'local' vs. 
'remote', as interpreted by the subnet and the interfaces, notably:

	- MSS default

	- SYN/SYNACK piggybacking for T/TCP
		(delaying the SYN/SYNACK when there was data in
		the SYN and the connection is local)

> further increase congestion losses

That's an odd interpretation of congestion. If all this is happening 
inside the host, it seems like something much more explicit should be 
happening between the NIC and the OS, rather than dragging TCP into the 
fray.

Joe

---

From: Joe Touch <[email-masked]>
Date: Mon, 08 Oct 2001 09:27:09 -0700

> well to being blasted with data causing congestion.

The number of ways some of those break the Internet architecture is 
long, starting with the lack of true broadcast (defeating ARP, RIP, etc.)

> Its one of those ideas that only looks good in a paper

The same could be said about ATM lane, frame clouds, SMDS or PPP 
bridging :-)

Joe

---

From: Joe Touch <[email-masked]>
Date: Mon, 08 Oct 2001 09:31:13 -0700

FWIW, I'm not advocating for retaining the "avoid slowstart on local 
links" code.

Just pointing out that:

	- it assumes links are different from paths
		as do many Internet protocols

	- the number of 'tricks' it defeats is not justification to
	  change it
		most of the tricks are as bad if not worse, and
		break in too many other ways

We need to be more careful about what we think is local, which might 
also include more careful use of a notion of direct connection when it 
isn't in fact direct.

Joe

---

From: RJ Atkinson <[email-masked]>
Date: Mon, 08 Oct 2001 14:04:56 -0400

At 12:27 08/10/01, Joe Touch wrote:
>The number of ways some of those break the Internet architecture is long, starting with the lack of true broadcast (defeating ARP, RIP, etc.)

Irrelevant.  

        The issue at hand is what should be implemented given the 
realities of the deployed Internet, rather than what should be done 
in some theoretic ideal world.  In the real world, Proxy ARP and other 
things exist -- so slow-start should always be implemented even on what 
might appear to be a link.  

        Nor should we stop there, we ought to be conservative (in the 
avoiding breakage sense of the meaning) in other areas as well, 
not just with TCP slow-start.

Ran

---

From: Joe Touch <[email-masked]>
Date: Mon, 08 Oct 2001 11:23:41 -0700

> might appear to be a link.  

Then we should support NATs throughout the architecture as well, since 
they are part of the realities of the deployed Internet.

But then, ignoring slow start on local links is _also_ part of the 
realities of the deployed Internet. This isn't a thought exercise - the 
code has been out there for a fairly long time.

At some point we have to consider what the basic assumptions and 
requirements are, and realize that the Internet isn't optimized for all 
parameters.

> not just with TCP slow-start.

Backing off to complete protocol agnosticism and conservatism renders 
all protocols impotent. At some point there are assumptions - about how 
much data can be sent, MTU, etc. The notion of a subnet has meaning; if 
it doesn't, then the netmask on the interface is incorrect.

Joe

---

From: Pablo Molinero Fernandez <[email-masked]>
Date: Mon, 08 Oct 2001 11:36:58 -0700

> Milliseconds on a link with microsecond delays are enormous, BTW.

If we take into account several user space/kernel space switches of some 
milliseconds in both the sender and the receiver, that the transmission 
time of a full frame in each 10-Mbit segment is 1.2 ms, and the fact 
that we need one SYN/SYN-ACK handshake before we can pass the 
information to the end-user, then the increase of some ms in the 
completion time has a smaller impact than suggested.

Pablo Molinero

---

From: Joe Touch <[email-masked]>
Date: Mon, 08 Oct 2001 11:45:57 -0700

Pablo Molinero Fernandez wrote:

> milliseconds in both the sender and the receiver, 

That's a very slow switch. Should be in the 10's of microseconds.

> that the transmission 
> time of a full frame in each 10-Mbit segment is 1.2 ms, 

That's for data...

> information to the end-user

SYN/SYN-ACKs don't have full frames of data :-)

Joe

---

From: "Andrei Gurtov" <[email-masked]>
Date: Tue, 9 Oct 2001 10:31:51 +0300

Joe Touch wrote:

> The notion of a subnet has meaning; if

On the first cover of TCP/IP Illus. v.1 there is a picture of class B
network 140.252 used for all examples in the book. A fragment of it includes
hosts connected via SLIP and Ethernet and having the same netid and subnet
id:

(.13.65)      (.13.66) (.13.65)         (.13.33)
 slip ----SLIP---- bsdi ----Ethernet---- sun

Thus, if the slow start is disabled on sun then an initial blast to slip has
to travel through the Ethernet, the router, and the slow serial link.

Andrei

---

From: Markus Peuhkuri <[email-masked]>
Date: 09 Oct 2001 10:49:57 +0300

> might appear to be a link.  

For example, the computer which I'm typing on is connected with 155
Mbit/s ATM connection.  The computer (laptop), which runs this emacs,
is in same IP subnet connected with WLAN, which has some 5 Mbit/s
throughput at my room.  If I move the laptop to another room, the link
throughput drops below 1 Mbit/s.  There used to be 28,8 kbit/s modems
at the same subnet.

If a computer with fast connection just sends flow control window
worth of data, some point in network must queue a few packets.

Given future of wireless computing, varying link speeds within same
subnets are future, not just "realities of the deployed Internet".  If
computer must change subnet (if subnet == "link characteristics
equal") if it's access speed changes, we would see quite much of
handovers.

-- 
Markus Peuhkuri        ! http://www.iki.fi/puhuri/

---

From: Joe Touch <[email-masked]>
Date: Tue, 09 Oct 2001 15:21:56 -0700

> hosts connected via SLIP and Ethernet and having the same netid and subnet

Don't judge a book by it's cover :-)

Though I pause to note that the diagram doesn't indicate the subnet mask 
used. The above diagram is appropriate if the BSDI machine uses a 32-bit 
subnet mask for the slip interface. All pt-pt interfaces should have 
32-bit subnet masks, since there is no subnet, by definition.

Joe

Joe

---

From: "Tim Moors" <[email-masked]>
Date: Wed, 10 Oct 2001 08:58:07 -0400

> TCPIMPL was closed as a WG, and this is definitely relevant to ongoing

Thanks.  Greatly appreciated.

> > MAC protocols

I disagree/don't understand, perhaps because I'm abstracting too far:
Considering first the multiple-access case (not full-duplex links). Ethernet
regulates access to a shared resource (the transmission medium), just as
TCP's congestion control regulates access to the shared resource of buffer
space in routers (demand for which depends on the availability of another
shared resource: link capacity).  Just like TCP's congestion control,
Ethernet is concerned with the fair sharing of this resource, and its
carrier-sensing forces sources to reduce their transmission rate when the
aggregate load on the shared resource is high (like reducing cwnd when TCP
detects signs of congestion).

Multi-access Ethernet may not be as popular as it used to be, but the real
motivation behind my question came from wireless LANs, where I thought it to
be unnecessarily slow and problematic to implement congestion control at the
transport level for communication that is local to the wireless LAN
(directly between mobiles), when the only shared resource that I could see
was the transmission medium, whose access is already regulated by the MAC
protocol.  In this case, there are physical layer indicators of locality
(signal strength from a certain MAC address) that are stronger than IP's
subnet notion of locality (how the transport layer gets to see them is
another matter).  I've also been thinking about transport protocols for
optical networks, where there may be an end-to-end circuit.  There is no
issue of router buffer overflow there either, and TCP's congestion control
takes unnecessary RTTs.

MAC protocols do nothing to prevent overflow of Network Interface buffers,
which may be lower in the stack than the implementation of flow control
(e.g. TCP).  After reading Joe's original reply & its attached email, this
suggests to me one good reason for transport-layer "congestion control" (at
least, backing off in response to loss), even when there are MAC protocols
to regulate access to the medium: You still have to deal with the overflow
of buffers that are below the flow control.  (Though see Joe's email
Message-ID: <[email-masked]> for arguments that TCP shouldn't be
involved in this.)  This would presumably also be necessary in the
full-duplex Ethernet case (although, doesn't GigE provide MAC-layer flow
control?).

Tim Moors
___________________________________
Web: http://uluru.poly.edu/~tmoors/

---

From: Joe Touch <[email-masked]>
Date: Wed, 10 Oct 2001 14:13:37 -0700

Tim Moors wrote:

> Ethernet is concerned with the fair sharing of this resource,

I wouldn't say ethernet does anything fairly, in that regard. It is 
designed to make sure that once granted, the resource isn't usurped. The 
capture effect is evidence that fairness has little to do with it :-)

> and its

Ethernet doesn't reduce offered load when sensed load is high; it keeps 
on trying. It's a matter of statistics; when it senses use, it avoids 
usurping it, but otherwise the backoff occurs only on collisions.

> issue of router buffer overflow there either, and TCP's congestion control

TCP makes sense when there is a shared resource AND when offered load 
results in congestive failure. While this is the case for paths (loss at 
routers), it doesn't appear relevant to links per se.

Joe