Subject: Re: [aqm] [Bloat] [iccrg] AQM deployment status?

From: Michael Richardson <[email-masked]>
Date: Fri, 11 Oct 2013 23:18:05 -0400

Eggert, Lars <[email-masked]> wrote:
    > I've heard that statement [that RED is supported] from many different
    > people. I wonder if it is
    > actually true. Is there any hard data on this?

The issue I've had is that while RED is supported on a routing device,
and I can enable it on a per-VLAN basis, that does me little good, because
there are bottlenecks in layer-2 devices (ethernet, but also DSL,GPON, etc.)
which are out of my control.

What I would like is a standard layer-3 way to measure bandwidth across
my layer-3 hop, and autoconfigure RED (or CoDel) to the measured bandwidth.
My understanding is that some equipment can use 802.1ad CCP and/or Loopback
frames to measure bandwidth across bridged layer-2 ethernet networks.

I'm unclear if this is a feature of that equipment, or a part of the
standard.  My understanding is that the math involved is pretty
straightforward, but getting it right in code can involve dealing with a
number of numerical issues which makes it slightly non-trivial to
implement. (And you need good low-latency time stamping of packets)

--
Michael Richardson <[email-masked]>, Sandelman Software Works

---

From: Anoop Ghanwani <[email-masked]>
Date: Sun, 13 Oct 2013 10:47:31 -0700

> which are out of my control.

Most Ethernet switches that I'm aware of do already implement WRED.   Can't
speak for other devices (DSL, etc.) since I don't have experience with
those.

> frames to measure bandwidth across bridged layer-2 ethernet networks.

You probably mean 802.1ag.

> standard.

I don't believe bandwidth measurement is part of the standard, but
certainly one can choose to use these protocols to measure the bandwidth.
 However, it's an expensive process.  However, bandwidth will variable if
the L2 network is shared by multiple L3 devices, and this means the
measurements would have to be made continuously, making it
counter-productive.  Unless, some mechanism is used to pre-provision
bandwidth between L3 peers.

Anoop

---

From: Curtis Villamizar <[email-masked]>
Date: Mon, 14 Oct 2013 13:07:58 -0400

In message <[email-masked]>
 
> implement. (And you need good low-latency time stamping of packets)

We need to keep context of deployment in mind: core, edge, access,
enterprise, data center, home or soho, etc.

In network core RED implementations or things claiming to be RED
started coming out as early as about 1995.  Some of the early hardware
that supported RED didn't get it even close to right, most notably
early Cisco RED, and gave RED a bad name among service providers.
With the faulty implementations there was no way to get good results
but the vendors just claimed "you didn't configure it right".

In the core there are no bottlenecks at layer-2 devices.  There are
almost no layer-2 devices to be found and a responsible service
provider, knowing layer-2 devices are dumb will make sure not to
congest them if used at all.

Layer-2 devices are used in core to edge sometimes and in edge
sometimes, but again a responsible service provider, knowing layer-2
devices are dumb will make sure not to congest them if used at all.

We would have to poll or do an informal anonymous poll of a few major
service providers to see if they use RED at all today.  For the most
part the goal is to keep at least the core if not the core and edge
sufficiently overprovisioned as to always be uncongested.  This is
where in the decade gone by large numbers of parallel 10G ports were
used and today that is moving toward parallel 100G.  With a Tb or
multiple Tb between routers (DWDM makes this expensive but
sufficiently cost effective), congestion is very unlikely.  Sometimes
growth can temporarily outpace deployment, so perhaps RED is available
for that situation.

In enterprise and data center there is also very good control over
what equipment is used and how it is used.  However, clue density
decreases exponentially farther from the core and approaches zero in
some data centers and in some enterprise IT departments.  This is also
true of the part of service provider organizations that pick stuff for
consumer access.  Where clue density is lowest, you'll find ignorance
of AQM and buffer issues and lack of consideration for buffering and
AQM when picking equipment for deployment and configuring it.

Where consumer networking is concerned, cost is often the deciding
factor, often on the SP side as well.  DSLAMs went from no buffering
in the 1990s to today where DSLAM and CMTS have too much and don't
ship with buffers configured to reasonable values and AQM enabled by
default, and SP often don't change the defaults yielding bloat.

As for AQM in home networking and SOHO or small business, there may be
bottlenecks elsewhere.  RED will only help if the bottleneck is at the
link between you and the service provider.  One way to make that the
case on the inbound direction is to create a queue (using something
like pf and altq) with less than your downlink speed and apply RED to
that.  That covers a bottleneck between you and the SP in either
direction (or both).  AFAIK no consumer "home router" allows this sort
of pacing.  If the bottleneck is further in the SP infrastructure you
can't do much if anything from your home network to affect it.

Its also worth noting that PC hosts don't always make good routers or
switches and the limits of hardware (like PCIe x1 cards, or too many
cards) or CPU processing can become the bottleneck.  OTOH, if you are
just front ending an SP link on the order of 10 Mb/s things it should
work with all but an old PC and NICs.

There can also be bottlenecks internal to a LAN.  An example would be
large transfers between hosts.  This can cause a bottleneck at a
switch within the LAN and maybe what you (Michael) are referring to.
Again, about the only thing you can do is configure the interfaces
down to FastE speeds (100Mb/s) on a low end GbE switch, or get a
higher end switch with buffering and RED - they exist but may cost 2-3
binary orders of magnitude more that a dumb and/or unbuffered GbE
switch with same port count (dumb and overbuffered also exists).

So my first question to the AQM WG is "what is the scope of AQM WG
work in terms of where in the network this WG wants to focus?"  If the
answer to that question is "everywhere", then we have to be aware that
conditions in core and conditions in home or enterprise are very
different.  If the focus is on home, soho, and small business, then
the charter should say so (I don't think it is).

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Mon, 14 Oct 2013 13:55:16 -0400

Not sure either 802.1ad (QinQ) or 802.1ag (CFM, Y.1771 like OAM) is
going to help.

What might help is if the switch does 802.1Qbb (priority based flow
control).  This only helps of the thing being flow controlled (the
host or other device) does 802.1Qbb and does something reasonable with
its backlog of traffic and not just keep buffering seconds of traffic.

802.1Qav might also help (Forwarding and Queuing Enhancements for
Time-sensitive Streams) for LAN congestion.  This would require
802.11Q priority tagging by the hosts or other devices for best
results.  Otherwise the traffic shaping (leaky bucket) applied to the
BE traffic class could also help.

Any 802.11Q tagging would likely get lost over the first router it
crosses.  I don't think highly of this solution, except prehaps to
keep the wifi speakers and the wifi streaming video and the downloads
from stepping on each other in the home or soho LAN.  A good wifi
router can also solve this without extensions to Ethernet, but usually
with configuration.

Curtis

In message <[email-masked]>
Anoop Ghanwani writes:

--===============7512874271394803300==
Content-Type: multipart/alternative; boundary=f46d043be28624734e04e8a2f27d

--f46d043be28624734e04e8a2f27d
Content-Type: text/plain; charset=ISO-8859-1

Anoop

--f46d043be28624734e04e8a2f27d
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><br><div class=3D"gmail_extra"><br><br><div class=3D"gmail=
_quote">On Fri, Oct 11, 2013 at 8:18 PM, Michael Richardson <span dir=3D"lt=
r">&lt;<a href=3D"mailto:[email-masked]" target=3D"_blank">mcr+ietf@=
sandelman.ca</a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><br>
Eggert, Lars &lt;<a href=3D"mailto:[email-masked]">[email-masked]</a>&gt;=
 wrote:<br>
=A0 =A0 &gt; I&#39;ve heard that statement [that RED is supported] from man=
y different<br>
<div class=3D"im">=A0 =A0 &gt; people. I wonder if it is<br>
=A0 =A0 &gt; actually true. Is there any hard data on this?<br>
<br>
</div>The issue I&#39;ve had is that while RED is supported on a routing de=
vice,<br>
and I can enable it on a per-VLAN basis, that does me little good, because<=
br>
there are bottlenecks in layer-2 devices (ethernet, but also DSL,GPON, etc.=
)<br>
which are out of my control.<br></blockquote><div><br></div><div>Most Ether=
net switches that I&#39;m aware of do already implement WRED. =A0 Can&#39;t=
 speak for other devices (DSL, etc.) since I don&#39;t have experience with=
 those.=A0</div>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
<br>
What I would like is a standard layer-3 way to measure bandwidth across<br>
my layer-3 hop, and autoconfigure RED (or CoDel) to the measured bandwidth.=
<br>
My understanding is that some equipment can use 802.1ad CCP and/or Loopback=
<br>
frames to measure bandwidth across bridged layer-2 ethernet networks.<br></=
blockquote><div><br></div><div>You probably mean 802.1ag.=A0</div><blockquo=
te class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc so=
lid;padding-left:1ex">

<br>
I&#39;m unclear if this is a feature of that equipment, or a part of the<br=
standard. =A0</blockquote><div><br></div><div>I don&#39;t believe bandwidth=
 measurement is part of the standard, but certainly one can choose to use t=
hese protocols to measure the bandwidth. =A0However, it&#39;s an expensive =
process. =A0However, bandwidth will variable if the L2 network is shared by=
 multiple L3 devices, and this means the measurements would have to be made=
 continuously, making it counter-productive. =A0Unless, some mechanism is u=
sed to pre-provision bandwidth between L3 peers.</div>
<div><br></div><div>Anoop</div></div></div></div>

--f46d043be28624734e04e8a2f27d--

--===============7512874271394803300==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
aqm mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/aqm

--===============7512874271394803300==--

---

From: Wesley Eddy <[email-masked]>
Date: Tue, 15 Oct 2013 21:00:16 -0400

> the charter should say so (I don't think it is).

The charter says:
"""
  It is expected that some classes of algorithms will focus on software
  implementations, while others on existing or new hardware
  deployments, and algorithms may be specific to distinct scenarios.
"""

I would say anywhere that AQM algorithms can have a positive impact
is in scope, and that it's understood and accepted that particular
algorithms or tuning rules may not be ideal across different
environments (or may not be easily implemented in different kinds of
platforms).  There was at least some talk of "applicability
statements" for things that wind up being recommended by the working
group.

In my opinion, the home broadband router is one well-known case that
may hold a lot of the initial attention in the working group, because
the barriers to implementing/testing/deploying new algorithms for this
case are less than for many others.  We definitely did not want to
limit the charter to this scenario though, and it is intentionally
open to others.

I hope this clears it up!

-- 
Wes Eddy
MTI Systems