Subject: [homegate] Amount of packet buffering in home gateways

From: Iljitsch van Beijnum <[email-masked]>
Date: Wed, 29 Jul 2009 10:34:38 +0200

[posted to homegate and tsv area, prune as necessary]

I'm currently in the congestion control research group session and the  
issue of excessive buffering in home gateways came up. (I'm posting to  
tsv area rather than iccrg or ledbat to avoid subscribing to a bunch  
of new lists for a single discussion, I would prefer to discuss this  
on homegate but I'm open to redirects if necessary.)

So what is the right thing to do? As I mentioned in the homegate bar  
bof, last year I was in a place where there was a 128 kbps uplink with  
10 seconds (160 kbyte) of buffering. This made the connection almost  
unusable until I manually set my send/receive buffers to 8k.

The question is, what do we want to tell home gateway builders?  
Obviously it would be nice to give priority to VoIP and gaming traffic  
and so on, but how do you easily do this without an extensive packet  
classification system? We also wouldn't want applications to game the  
system by sending more small packets that get better service than  
fewer large packets.

What I'm thinking is that at 64 kbps or slower, you probably want to  
buffer 4 packets and no more, but also no less. At higher speeds, I'm  
thinking 10 packets is really all you need, assuming a single fifo /  
tail drop queue. Does anyone have data or arguments that conflict with  
this?

But do we want to recommend RED? I seem to remember it has fallen  
somewhat out of favor, but is it worse than a simple tail drop queue?  
With RED it's necessary to have more buffering in order to let it work  
without tail drops happening anyway. So how much buffering would that  
require? 20 packets? 40?

Bob Briscoe just made the argument that weighted fair queuing isolates  
flows from each other so they can't interact in useful ways. So  
recommend against it? On the other hand, WFQ would make VoIP etc work  
much better in the presence of some heavy flows. But not as well as in  
the case where we specifically prioritize the real-time traffic.

I'm guessing that we don't want to specify any default diffserv  
behavior...

Iljitsch

---

From: David R Oran <[email-masked]>
Date: Wed, 29 Jul 2009 10:49:35 +0200

On Jul 29, 2009, at 10:34 AM, Iljitsch van Beijnum wrote:

> I'm currently in the congestion control research group session and  
> the issue of excessive buffering in home gateways came up. (I'm  
> posting to tsv area rather than iccrg or ledbat to avoid subscribing  
> to a bunch of new lists for a single discussion, I would prefer to  
> discuss this on homegate but I'm open to redirects if necessary.)
>
It isn't actually home gateways that appear to be the problem, but  
either independent (or combined) L2 devices like cable modems or DSL  
modems that are the big offenders. In my case as a user I tend to  
ameliorate the problem by rate limiting the output of my home gateway  
to avoid building up a queue on the cable modem. At least the open- 
source firmware loads allow you to do this on some devices.

This does bring up a potential work area in the IETF, which is to  
develop or re-purpose some protocol for coordinating L3 and L2 devices  
so that the L3 box can learn the link characteristics of the L2 device  
and not act as if the ethernet between them is actually the capacity  
of the (probably bottleneck) next hop.

I don't think is this is an appropriate topic for the homegate group,  
as it jumps way over the line into protocol development.

> bof, last year I was in a place where there was a 128 kbps uplink  
> with 10 seconds (160 kbyte) of buffering. This made the connection  
> almost unusable until I manually set my send/receive buffers to 8k.
>
> Obviously it would be nice to give priority to VoIP and gaming  
> traffic and so on, but how do you easily do this without an  
> extensive packet classification system?
Extensive is in the mind of the beholder. If you simply let the hosts  
do the marking, then all the home gateway has to do is police/shape.  
Right now we're in a chicken/egg situation where most hosts don't  
bother to mark because the home gateways either don't obey the  
markings, or worse, remark or drop the traffic.

HGI and others (e.g. DLNA) do in fact have specs for this kind of  
stuff. I'll refrain from commenting on what I think of their quality.  
I think the group might consider taking some of this on, however I  
urge caution for a number of reasons including:
- focus for the group
- bleed-over into net neutrality issues
- service definition and provisioning of device queues by the service  
provider versus the consumer.

> We also wouldn't want applications to game the system by sending  
> more small packets that get better service than fewer large packets.
>
All the reasonable QoS implementations I know of count bytes and  
account for headers. Do have a counter example?

> buffer 4 packets and no more, but also no less. At higher speeds,  
> I'm thinking 10 packets is really all you need, assuming a single  
> fifo / tail drop queue. Does anyone have data or arguments that  
> conflict with this?
>
> somewhat out of favor, but is it worse than a simple tail drop  
> queue? With RED it's necessary to have more buffering in order to  
> let it work without tail drops happening anyway. So how much  
> buffering would that require? 20 packets? 40?
>
RED doesn't help in the split-box configuration since the big queues  
aren't in the home gateway, as I noted above.

> Bob Briscoe just made the argument that weighted fair queuing  
> isolates flows from each other so they can't interact in useful  
> ways. So recommend against it? On the other hand, WFQ would make  
> VoIP etc work much better in the presence of some heavy flows. But  
> not as well as in the case where we specifically prioritize the real- 
> time traffic.
>
Interesting, but orthogonal in my view.

> _______________________________________________
> homegate mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/homegate

---

From: james woodyatt <[email-masked]>
Date: Wed, 29 Jul 2009 10:59:12 +0200

> The question is, what do we want to tell home gateway builders?

We should tell them that forwarding IP flows from higher bandwidth  
links into much lower bandwidth links generally produces better  
results when queues are only large enough to handle common burst sizes  
without loss and no larger.  Unnecessarily large transmission queues  
may introduce persistently and perversely high path latency on fully  
loaded low-bandwidth links, producing severe negative effects on  
application throughput.

Beyond that, I don't know what else we should say.  Maybe it would  
help to give some guidance for various common sub-IP links.

--
james woodyatt <[email-masked]>
member of technical staff, communications engineering

---

From: Lars Eggert <[email-masked]>
Date: Wed, 29 Jul 2009 11:05:56 +0200

Hi,

> The question is, what do we want to tell home gateway builders?

one half-baked idea is to tell them to provision buffering for some  
number of milliseconds of traffic (I'm using ms here instead of bytes,  
because that factors out link speeds) *and* use something better than  
FIFO (e.g., some AQM scheme) that you configure so that the target  
queue is a small fraction of that buffer.

> somewhat out of favor, but is it worse than a simple tail drop queue?

I believe we want a queueing scheme that results in a short standing  
queue independent of how much buffer there is. Many AQM schemes (like  
RED) have that property, FIFO doesn't.

We also do want ECN, so that whatever AQM scheme we recommend doesn't  
need to induce unnecessary losses in order to drive the queue size down.

> the case where we specifically prioritize the real-time traffic.

I don't think basic WFQ has the property that it builds short standing  
queues (but my queue fu is ten years old).

> I'm guessing that we don't want to specify any default diffserv

Actually, I think recommending support for less-than-best-effort  
(e.g., RFC 3662) would be good.

Lars

---

From: [email-masked]
Date: Wed, 29 Jul 2009 23:14:58 +0800

Question of interest: Does the homegate take the frequency/time 
synchronization between devices into account ?

Fei Su

---

From: [email-masked]
Date: Wed, 29 Jul 2009 23:15:12 +0800

Fei Su

---

From: David R Oran <[email-masked]>
Date: Wed, 29 Jul 2009 17:17:41 +0200

> synchronization between devices into account ?
Are you asking on whether it is important to write down a requirement  
to run NTP?

---

From: Fred Baker <[email-masked]>
Date: Wed, 29 Jul 2009 17:29:18 +0200

> synchronization between devices into account ?

Let me return the question - does it need to? Routers, like other  
equipment, have the option of fairly fine-grain synchronization of  
their clocks using NTP, or more coarse synchronization using SNTP, and  
can go crazy with IEEE 1588. Most home routers today do none of those,  
and the assertion that there is no time-synchronous log or other  
reason to worry about it. Do you have a requirement for the home  
router to do that?

---

From: Lars Eggert <[email-masked]>
Date: Wed, 29 Jul 2009 18:20:13 +0200

Hi,

On 2009-7-29, at 17:29, Fred Baker wrote:
> their clocks using NTP, or more coarse synchronization using SNTP,  
> and can go crazy with IEEE 1588. Most home routers today do none of  
> those, and the assertion that there is no time-synchronous log or  
> other reason to worry about it.

actually, every home router I had during the last 10 years at least  
did NTP, and the D-Link issue (see http://en.wikipedia.org/wiki/NTP_server_misuse_and_abuse) 
  also indicates that NTP support is being implemented.

That said, it does mostly seem to be used for correct timestamps in  
whatever log is provided. (All those boxes had some sort of weg log  
and often even syslog.)

> Do you have a requirement for the home router to do that?

Agreed, it'd be interesting what this would be for. Timestamped  
logging in my personal view isn't essential enough to weigh down an  
IETF recommendation on home gateways with NTP support.

Lars

---

From: [email-masked]
Date: Thu, 30 Jul 2009 00:32:21 +0800

The motivation of the requirement might be from potential services 
provided for home network such as gaming, VoIP or IPTV. If the NTP can 
meet the requirement or more advanced sync method e.g. 1588v2 could be 
used?

Lars Eggert <[email-masked]> 
2009-07-30 00:20

收件人
Fred Baker <[email-masked]>
抄送
"[email-masked]" <[email-masked]>, "[email-masked]" 
<[email-masked]>
主题
Re: [homegate] Timing and synchronization needed in home gateways

Hi,

actually, every home router I had during the last 10 years at least 
did NTP, and the D-Link issue (see 
http://en.wikipedia.org/wiki/NTP_server_misuse_and_abuse) 
  also indicates that NTP support is being implemented.

Lars

---

From: Henk Uijterwaal <[email-masked]>
Date: Wed, 29 Jul 2009 18:49:11 +0200

Lars Eggert wrote:

> Agreed, it'd be interesting what this would be for. Timestamped logging 
> in my personal view isn't essential enough to weigh down an IETF 
> recommendation on home gateways with NTP support.

Some years ago, I was involved in a project where code was added to
home gateways to do performance measurements.  The idea was to
store the data on a $5 memory USB stick in the box (one can store
a lot on that).  If a user complained about issues with his connection,
the ISP would have some real data to look at rather than having to
deal with vague statements from users.  It could also be used to
exclude a whole bunch of typical problems on the spot, without
the helpdesk having to go through all kinds of exercises with the
user.  (For example, the user cannot get his mail, but the data
says that the port on the mailserver responds, that excludes a
bunch of things).  The project never got anywhere, though I still
think that this is a good idea.

Anyway, setups like this does require that the box has an idea about global
time and this requires the ability to run NTP or similar.

Henk

-- 
------------------------------------------------------------------------------
Henk Uijterwaal                           Email: henk.uijterwaal(at)ripe.net
RIPE Network Coordination Centre          http://www.xs4all.nl/~henku
P.O.Box 10096          Singel 258         Phone: +31.20.5354414
1001 EB Amsterdam      1016 AB Amsterdam  Fax: +31.20.5354445
The Netherlands        The Netherlands    Mobile: +31.6.55861746
------------------------------------------------------------------------------

Belgium: an unsolvable problem, discussed in endless meetings, with no
          hope for a solution, where everybody still lives happily.

---

From: "Erichsen, Kirk" <[email-masked]>
Date: Wed, 29 Jul 2009 12:58:27 -0400

Homegate Scope question: - How far should we go? 
 
Do we incorporate all typical cases and leave room for specific delivery models that go above and beyond what constitutes as core functionality? My needs for performance measurement may not be the same as another ISP, should we thus: 
 
1) Specify such functionality as optional (and other features that while useful aren't necessarily "universally" so)
2) refer to such functionality with informational text without specifying anything, or x-ref to a spec that covers these use cases
3) remain silient on the topic and allow the ISP and their choice of vendor to add such functionality as a product spec vs. a generic industry spec. 
 
We might start by defining "core" functionality and then "advanced" functionality that straps to that core and adds useful features. Whether "advanced" is the same as "optional" (in my view, it may well be) should be discussed.
 
Thoughts? This frames, in general, where I'd like to start seeing us knock some stakes into the ground, defining some boundaries on what we do within the HomeGate and gel the charter's scope a bit around that.
 
-KE

________________________________

From: [email-masked] on behalf of Henk Uijterwaal
Sent: Wed 7/29/2009 10:49 AM
To: Lars Eggert
Cc: [email-masked]
Subject: Re: [homegate] Timing and synchronization needed in home gateways

Henk

This E-mail and any of its attachments may contain Time Warner
Cable proprietary information, which is privileged, confidential,
or subject to copyright belonging to Time Warner Cable. This E-mail
is intended solely for the use of the individual or entity to which
it is addressed. If you are not the intended recipient of this
E-mail, you are hereby notified that any dissemination,
distribution, copying, or action taken in relation to the contents
of and attachments to this E-mail is strictly prohibited and may be
unlawful. If you have received this E-mail in error, please notify
the sender immediately and permanently delete the original and any
copy of this E-mail and any printout.

---

From: David R Oran <[email-masked]>
Date: Wed, 29 Jul 2009 19:30:55 +0200

On Jul 29, 2009, at 6:58 PM, Erichsen, Kirk wrote:

> Do we incorporate all typical cases and leave room for specific  
> delivery models that go above and beyond what constitutes as core  
> functionality? My needs for performance measurement may not be the  
> same as another ISP, should we thus:
>
> 1) Specify such functionality as optional (and other features that  
> while useful aren't necessarily "universally" so)
> 2) refer to such functionality with informational text without  
> specifying anything, or x-ref to a spec that covers these use cases
> 3) remain silient on the topic and allow the ISP and their choice of  
> vendor to add such functionality as a product spec vs. a generic  
> industry spec.
>
> We might start by defining "core" functionality and then "advanced"  
> functionality that straps to that core and adds useful features.  
> Whether "advanced" is the same as "optional" (in my view, it may  
> well be) should be discussed.
>
> Thoughts? This frames, in general, where I'd like to start seeing us  
> knock some stakes into the ground, defining some boundaries on what  
> we do within the HomeGate and gel the charter's scope a bit around  
> that.
>
My personal view is that we should start with a goal to specify as  
little as possible - those things that are crucially needed to ensure  
that the Internet works for the home user.

In order to do this I think (again personal hat on) we should:

- avoid inventing protocols, unless absolutely needed to do a critical  
function for *using the Internet*. I would be personally very  
surprised if there were crucial functions for which we don't have  
adequate protocols today

- avoid writing long feature lists, but refer to existing RFCs,  
clarifying the requirement where we believe things have gone off the  
rails in the past, or are likely to do off the rails in the future. At  
the BoF I think we did identify a few of these in the areas of IPv6  
support, ECN operation and DNS. A "SOHO-edge Router Requirements"  
document could be a decent vehicle to accomplish this.

- try to agree on a set of topologies that are expected to work if you  
follow the BCP specification (perhaps as part of the specification) so  
that we are all on the same page with respect to the need for things  
like:
	- multiple subnets on a single L2 multi-switch or single switch LAN
	- Cascaded routers/NATs
	- parallel OHO-edge boxes with uplinks to different providers

- not re-invent the work done in other places like BBF, HGI, DLNA,  
etc. especially for features/services/provisioning of the connection  
between the SOHO and the service provider at the other end of an  
access network link (eg. DSL, DOCSIS, WiMAX, FTTH).

- attempt to meet the goal James Woodyatt articulated of having zero- 
configuration needed to come up in a predictable *operational* state  
(again apart from the pieces mandated by the serice provider to  
connect to their network). Also nie would be to somehow require that  
any manufacturer-specified "factory reset" function return the box to  
this state.

There are likely other similar goals/requirements that I've missed,  
and conversely things I should have thought to exclude but didn't.

How about we have people produce strawman requirements of like the  
ones we wrote above. Then we can gather them and discuss them, perhaps  
by setting up a Wiki somewhere?

DaveO.

>> logging

---

From: [email-masked]
Date: Wed, 29 Jul 2009 19:38:27 +0200

> IETF recommendation on home gateways with NTP support.

According to a very recent meeting I had with UK government and law 
enforcement people it would be very useful for forensic purposes.  Not 
only that, but for certain log information (DHCP leases, etc) it would be 
desirable for that information to persist between reboots.

That said, IMHO I think this falls under the HGI / BBF sphere of 
requirements (and not IETF's) since it is not related to interoperability 
nor needed for correct operation of client machines.

Ray

---

From: Mikael Abrahamsson <[email-masked]>
Date: Wed, 29 Jul 2009 20:15:03 +0200 (CEST)

>> IETF recommendation on home gateways with NTP support.

I'm not sure if this is the IETF that should say this or not, but I'd 
prefer that it *somewhere* was recommended to vendors that any timestamp 
produced should be clearly marked whether it had a traceable time back to 
NTP or alike, or not. Producing timestamps that you have no idea whether 
they are correct or not is almost worse than nothing at all. I prefer 
"time since boot" or alike if you have no idea what the time is.

It might be ok with just SNTP or other mechanism more lightweight than NTP 
that is done on a regular basis.

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: Richard Bennett <[email-masked]>
Date: Wed, 29 Jul 2009 14:21:36 -0700

> provider versus the consumer.
I'm not sure what Homegate has to do with "net neutrality issues." If 
net neutrality means anything at all, it's the demand by certain private 
network operators, most notably Google, to have unfettered access to 
first/last mile residential broadband networks. Google wants to bypass 
the public Internet and install edge caches directly on the L2 network, 
in other words. Aren't private network-to-private network issues are 
outside the scope of IETF?

Under other definitions of net neutrality, ISP behavior is circumscribed 
in various ways, but end-user behavior isn't. Given that home gateways 
are (at least to some extent) end-user controlled devices, NN also 
wouldn't apply to them under this more fuzzy definition.

NN is in any case a US issues, so I don't see why the international 
community should care about it. What am I missing?

RB

-- 
Richard Bennett
Research Fellow
Information Technology and Innovation Foundation
Washington, DC