From mcr@sandelman.ca Fri Oct 11 20:18:27 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CB12D21E80BD for ; Fri, 11 Oct 2013 20:18:27 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.067 X-Spam-Level: X-Spam-Status: No, score=-2.067 tagged_above=-999 required=5 tests=[AWL=-0.295, BAYES_00=-2.599, J_CHICKENPOX_34=0.6, SARE_SUB_OBFU_Q1=0.227] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IPLo-MdpcQWY for ; Fri, 11 Oct 2013 20:18:22 -0700 (PDT) Received: from tuna.sandelman.ca (tuna.sandelman.ca [IPv6:2607:f0b0:f:3::184]) by ietfa.amsl.com (Postfix) with ESMTP id 2C25821E8120 for ; Fri, 11 Oct 2013 20:18:14 -0700 (PDT) Received: from sandelman.ca (desk.marajade.sandelman.ca [209.87.252.247]) by tuna.sandelman.ca (Postfix) with ESMTP id 50C5920273; Sat, 12 Oct 2013 00:28:25 -0400 (EDT) Received: by sandelman.ca (Postfix, from userid 179) id E813163B88; Fri, 11 Oct 2013 23:18:05 -0400 (EDT) Received: from sandelman.ca (localhost [127.0.0.1]) by sandelman.ca (Postfix) with ESMTP id D7A1863B87; Fri, 11 Oct 2013 23:18:05 -0400 (EDT) From: Michael Richardson To: "Eggert\, Lars" In-Reply-To: References: X-Mailer: MH-E 8.2; nmh 1.3-dev; GNU Emacs 23.4.1 X-Face: $\n1pF)h^`}$H>Hk{L"x@)JS7<%Az}5RyS@k9X%29-lHB$Ti.V>2bi.~ehC0; <'$9xN5Ub# z!G,p`nR&p7Fz@^UXIn156S8.~^@MJ*mMsD7=QFeq%AL4m Sender: mcr@sandelman.ca Cc: "Akhtar, Shahid \(Shahid\)" , "iccrg@irtf.org" , "aqm@ietf.org" , bloat Subject: Re: [aqm] [Bloat] [iccrg] AQM deployment status? X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Oct 2013 03:18:27 -0000 --=-=-= Eggert, Lars wrote: > I've heard that statement [that RED is supported] from many different > people. I wonder if it is > actually true. Is there any hard data on this? The issue I've had is that while RED is supported on a routing device, and I can enable it on a per-VLAN basis, that does me little good, because there are bottlenecks in layer-2 devices (ethernet, but also DSL,GPON, etc.) which are out of my control. What I would like is a standard layer-3 way to measure bandwidth across my layer-3 hop, and autoconfigure RED (or CoDel) to the measured bandwidth. My understanding is that some equipment can use 802.1ad CCP and/or Loopback frames to measure bandwidth across bridged layer-2 ethernet networks. I'm unclear if this is a feature of that equipment, or a part of the standard. My understanding is that the math involved is pretty straightforward, but getting it right in code can involve dealing with a number of numerical issues which makes it slightly non-trivial to implement. (And you need good low-latency time stamping of packets) -- Michael Richardson , Sandelman Software Works --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQCVAwUBUli/bYqHRg3pndX9AQItSQQA4LSYBoUP5+xivrh2v0A8hW8qNB9ULmJX hXGK3TSYv5dXslVbxt90uFv3dyaOgxltpuT2pP4uxcH6LBk6m/jXraL3lBthqWfF pUKkq60iZYmCRYLaYBzMA3BWtVq7Vt9nY+GcWZtOUODQJvJnXQdAcSfLLY7r73N/ X2e24Y8eZaI= =FnNZ -----END PGP SIGNATURE----- --=-=-=-- From ghanwani@gmail.com Sun Oct 13 10:47:34 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9C66921F9F34 for ; Sun, 13 Oct 2013 10:47:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.15 X-Spam-Level: X-Spam-Status: No, score=-1.15 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, J_CHICKENPOX_34=0.6, NO_RELAYS=-0.001, SARE_SUB_OBFU_Q1=0.227] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6OOvwaAO1Bq8 for ; Sun, 13 Oct 2013 10:47:34 -0700 (PDT) Received: from mail-we0-x22c.google.com (mail-we0-x22c.google.com [IPv6:2a00:1450:400c:c03::22c]) by ietfa.amsl.com (Postfix) with ESMTP id 8E30121E80F8 for ; Sun, 13 Oct 2013 10:47:32 -0700 (PDT) Received: by mail-we0-f172.google.com with SMTP id q58so6245853wes.3 for ; Sun, 13 Oct 2013 10:47:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=o9FOWchf3ALELIjWGSrzIO1Hu16JKT2EevkiIfH8iSY=; b=iWo5wgY5FEQn0S2vCun1ZVtzs0tKOo5h28afrt9K5hv7JixZ3EYa2oDhLZ/AiFmETM d1zFgQWksE72Orfvq2fHHwiUmXrdDPP33kY0lbbvLh+HpnXjPyhyM7m6udpDJxC+5zLI JT386NPRAP4UVlq0WZcpWNaDZksMjVglnQkVPQj0XaYWYb70wnXkhwpAIWDGyuxBb3rJ ZhpTQ5ouxTlwSG72N1Ep63lP7s7FHTc3wHbfjahidKuMFKdjoATEcoYnm9H2bmGRw+n9 tvbJYd30se6QpgjZvfvoPI2AnAEW5nOF3eYZ4Ps97RYvalUapXoHLGvNUFQzm0Omwl5b cyLQ== MIME-Version: 1.0 X-Received: by 10.180.72.207 with SMTP id f15mr462914wiv.60.1381686451661; Sun, 13 Oct 2013 10:47:31 -0700 (PDT) Sender: ghanwani@gmail.com Received: by 10.217.54.6 with HTTP; Sun, 13 Oct 2013 10:47:31 -0700 (PDT) In-Reply-To: <12845.1381547885@sandelman.ca> References: <12845.1381547885@sandelman.ca> Date: Sun, 13 Oct 2013 10:47:31 -0700 X-Google-Sender-Auth: 6xqxioVkutxt39t4jSvY6hZvR7k Message-ID: From: Anoop Ghanwani To: Michael Richardson Content-Type: multipart/alternative; boundary=f46d043be28624734e04e8a2f27d Cc: "Akhtar, Shahid \(Shahid\)" , "iccrg@irtf.org" , "Eggert, Lars" , "aqm@ietf.org" , bloat Subject: Re: [aqm] [Bloat] [iccrg] AQM deployment status? X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Oct 2013 17:47:34 -0000 --f46d043be28624734e04e8a2f27d Content-Type: text/plain; charset=ISO-8859-1 On Fri, Oct 11, 2013 at 8:18 PM, Michael Richardson wrote: > > Eggert, Lars wrote: > > I've heard that statement [that RED is supported] from many different > > people. I wonder if it is > > actually true. Is there any hard data on this? > > The issue I've had is that while RED is supported on a routing device, > and I can enable it on a per-VLAN basis, that does me little good, because > there are bottlenecks in layer-2 devices (ethernet, but also DSL,GPON, > etc.) > which are out of my control. > Most Ethernet switches that I'm aware of do already implement WRED. Can't speak for other devices (DSL, etc.) since I don't have experience with those. > > What I would like is a standard layer-3 way to measure bandwidth across > my layer-3 hop, and autoconfigure RED (or CoDel) to the measured bandwidth. > My understanding is that some equipment can use 802.1ad CCP and/or Loopback > frames to measure bandwidth across bridged layer-2 ethernet networks. > You probably mean 802.1ag. > > I'm unclear if this is a feature of that equipment, or a part of the > standard. I don't believe bandwidth measurement is part of the standard, but certainly one can choose to use these protocols to measure the bandwidth. However, it's an expensive process. However, bandwidth will variable if the L2 network is shared by multiple L3 devices, and this means the measurements would have to be made continuously, making it counter-productive. Unless, some mechanism is used to pre-provision bandwidth between L3 peers. Anoop --f46d043be28624734e04e8a2f27d Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable



On Fri, Oct 11, 2013 at 8:18 PM, Michael Richardson <mcr+ietf@= sandelman.ca> wrote:

Eggert, Lars <lars@netapp.com>= wrote:
=A0 =A0 > I've heard that statement [that RED is supported] from man= y different
=A0 =A0 > people. I wonder if it is
=A0 =A0 > actually true. Is there any hard data on this?

The issue I've had is that while RED is supported on a routing de= vice,
and I can enable it on a per-VLAN basis, that does me little good, because<= br> there are bottlenecks in layer-2 devices (ethernet, but also DSL,GPON, etc.= )
which are out of my control.

Most Ether= net switches that I'm aware of do already implement WRED. =A0 Can't= speak for other devices (DSL, etc.) since I don't have experience with= those.=A0

What I would like is a standard layer-3 way to measure bandwidth across
my layer-3 hop, and autoconfigure RED (or CoDel) to the measured bandwidth.=
My understanding is that some equipment can use 802.1ad CCP and/or Loopback=
frames to measure bandwidth across bridged layer-2 ethernet networks.

You probably mean 802.1ag.=A0

I'm unclear if this is a feature of that equipment, or a part of the standard. =A0

I don't believe bandwidth= measurement is part of the standard, but certainly one can choose to use t= hese protocols to measure the bandwidth. =A0However, it's an expensive = process. =A0However, bandwidth will variable if the L2 network is shared by= multiple L3 devices, and this means the measurements would have to be made= continuously, making it counter-productive. =A0Unless, some mechanism is u= sed to pre-provision bandwidth between L3 peers.

Anoop
--f46d043be28624734e04e8a2f27d-- From curtis@ipv6.occnc.com Mon Oct 14 10:08:06 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 924F621E80DB; Mon, 14 Oct 2013 10:08:06 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.773 X-Spam-Level: X-Spam-Status: No, score=-1.773 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, J_CHICKENPOX_34=0.6, NO_RELAYS=-0.001, SARE_SUB_OBFU_Q1=0.227] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Q3Au3GvyxgGv; Mon, 14 Oct 2013 10:08:06 -0700 (PDT) Received: from gateway1.orleans.occnc.com (gateway1.orleans.occnc.com [IPv6:2001:470:1f07:1545::1:132]) by ietfa.amsl.com (Postfix) with ESMTP id B016F21E809F; Mon, 14 Oct 2013 10:08:05 -0700 (PDT) Received: from harbor1.ipv6.occnc.com (harbor1.ipv6.occnc.com [IPv6:2001:470:1f07:1545::2:819]) (authenticated bits=0) by gateway1.orleans.occnc.com (8.14.5/8.14.5) with ESMTP id r9EH7wC7068478; Mon, 14 Oct 2013 13:07:58 -0400 (EDT) (envelope-from curtis@ipv6.occnc.com) Message-Id: <201310141707.r9EH7wC7068478@gateway1.orleans.occnc.com> To: Michael Richardson From: Curtis Villamizar In-reply-to: Your message of "Fri, 11 Oct 2013 23:18:05 EDT." <12845.1381547885@sandelman.ca> Date: Mon, 14 Oct 2013 13:07:58 -0400 Cc: "Akhtar, Shahid \(Shahid\)" , "iccrg@irtf.org" , "Eggert, Lars" , "aqm@ietf.org" , bloat Subject: Re: [aqm] [Bloat] [iccrg] AQM deployment status? X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: curtis@ipv6.occnc.com List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Oct 2013 17:08:06 -0000 In message <12845.1381547885@sandelman.ca> Michael Richardson writes: > > Eggert, Lars wrote: > > I've heard that statement [that RED is supported] from many different > > people. I wonder if it is > > actually true. Is there any hard data on this? > > The issue I've had is that while RED is supported on a routing device, > and I can enable it on a per-VLAN basis, that does me little good, because > there are bottlenecks in layer-2 devices (ethernet, but also DSL,GPON, etc.) > which are out of my control. > > What I would like is a standard layer-3 way to measure bandwidth across > my layer-3 hop, and autoconfigure RED (or CoDel) to the measured bandwidth. > My understanding is that some equipment can use 802.1ad CCP and/or Loopback > frames to measure bandwidth across bridged layer-2 ethernet networks. > > I'm unclear if this is a feature of that equipment, or a part of the > standard. My understanding is that the math involved is pretty > straightforward, but getting it right in code can involve dealing with a > number of numerical issues which makes it slightly non-trivial to > implement. (And you need good low-latency time stamping of packets) > > -- > Michael Richardson , Sandelman Software Works We need to keep context of deployment in mind: core, edge, access, enterprise, data center, home or soho, etc. In network core RED implementations or things claiming to be RED started coming out as early as about 1995. Some of the early hardware that supported RED didn't get it even close to right, most notably early Cisco RED, and gave RED a bad name among service providers. With the faulty implementations there was no way to get good results but the vendors just claimed "you didn't configure it right". In the core there are no bottlenecks at layer-2 devices. There are almost no layer-2 devices to be found and a responsible service provider, knowing layer-2 devices are dumb will make sure not to congest them if used at all. Layer-2 devices are used in core to edge sometimes and in edge sometimes, but again a responsible service provider, knowing layer-2 devices are dumb will make sure not to congest them if used at all. We would have to poll or do an informal anonymous poll of a few major service providers to see if they use RED at all today. For the most part the goal is to keep at least the core if not the core and edge sufficiently overprovisioned as to always be uncongested. This is where in the decade gone by large numbers of parallel 10G ports were used and today that is moving toward parallel 100G. With a Tb or multiple Tb between routers (DWDM makes this expensive but sufficiently cost effective), congestion is very unlikely. Sometimes growth can temporarily outpace deployment, so perhaps RED is available for that situation. In enterprise and data center there is also very good control over what equipment is used and how it is used. However, clue density decreases exponentially farther from the core and approaches zero in some data centers and in some enterprise IT departments. This is also true of the part of service provider organizations that pick stuff for consumer access. Where clue density is lowest, you'll find ignorance of AQM and buffer issues and lack of consideration for buffering and AQM when picking equipment for deployment and configuring it. Where consumer networking is concerned, cost is often the deciding factor, often on the SP side as well. DSLAMs went from no buffering in the 1990s to today where DSLAM and CMTS have too much and don't ship with buffers configured to reasonable values and AQM enabled by default, and SP often don't change the defaults yielding bloat. As for AQM in home networking and SOHO or small business, there may be bottlenecks elsewhere. RED will only help if the bottleneck is at the link between you and the service provider. One way to make that the case on the inbound direction is to create a queue (using something like pf and altq) with less than your downlink speed and apply RED to that. That covers a bottleneck between you and the SP in either direction (or both). AFAIK no consumer "home router" allows this sort of pacing. If the bottleneck is further in the SP infrastructure you can't do much if anything from your home network to affect it. Its also worth noting that PC hosts don't always make good routers or switches and the limits of hardware (like PCIe x1 cards, or too many cards) or CPU processing can become the bottleneck. OTOH, if you are just front ending an SP link on the order of 10 Mb/s things it should work with all but an old PC and NICs. There can also be bottlenecks internal to a LAN. An example would be large transfers between hosts. This can cause a bottleneck at a switch within the LAN and maybe what you (Michael) are referring to. Again, about the only thing you can do is configure the interfaces down to FastE speeds (100Mb/s) on a low end GbE switch, or get a higher end switch with buffering and RED - they exist but may cost 2-3 binary orders of magnitude more that a dumb and/or unbuffered GbE switch with same port count (dumb and overbuffered also exists). So my first question to the AQM WG is "what is the scope of AQM WG work in terms of where in the network this WG wants to focus?" If the answer to that question is "everywhere", then we have to be aware that conditions in core and conditions in home or enterprise are very different. If the focus is on home, soho, and small business, then the charter should say so (I don't think it is). Curtis From curtis@ipv6.occnc.com Mon Oct 14 10:55:47 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5CDBE11E814C for ; Mon, 14 Oct 2013 10:55:47 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.773 X-Spam-Level: X-Spam-Status: No, score=-1.773 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, J_CHICKENPOX_34=0.6, NO_RELAYS=-0.001, SARE_SUB_OBFU_Q1=0.227] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XmsrM3Bgg6yd for ; Mon, 14 Oct 2013 10:55:46 -0700 (PDT) Received: from gateway1.orleans.occnc.com (gateway1.orleans.occnc.com [IPv6:2001:470:1f07:1545::1:132]) by ietfa.amsl.com (Postfix) with ESMTP id 0B54A11E8153 for ; Mon, 14 Oct 2013 10:55:30 -0700 (PDT) Received: from harbor1.ipv6.occnc.com (harbor1.ipv6.occnc.com [IPv6:2001:470:1f07:1545::2:819]) (authenticated bits=0) by gateway1.orleans.occnc.com (8.14.5/8.14.5) with ESMTP id r9EHtG1T068656; Mon, 14 Oct 2013 13:55:17 -0400 (EDT) (envelope-from curtis@ipv6.occnc.com) Message-Id: <201310141755.r9EHtG1T068656@gateway1.orleans.occnc.com> To: Anoop Ghanwani From: Curtis Villamizar In-reply-to: Your message of "Sun, 13 Oct 2013 10:47:31 PDT." Date: Mon, 14 Oct 2013 13:55:16 -0400 Cc: Michael Richardson , "iccrg@irtf.org" , "Akhtar, Shahid \(Shahid\)" , "Eggert, Lars" , bloat , "aqm@ietf.org" Subject: Re: [aqm] [Bloat] [iccrg] AQM deployment status? X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: curtis@ipv6.occnc.com List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Oct 2013 17:55:47 -0000 Not sure either 802.1ad (QinQ) or 802.1ag (CFM, Y.1771 like OAM) is going to help. What might help is if the switch does 802.1Qbb (priority based flow control). This only helps of the thing being flow controlled (the host or other device) does 802.1Qbb and does something reasonable with its backlog of traffic and not just keep buffering seconds of traffic. 802.1Qav might also help (Forwarding and Queuing Enhancements for Time-sensitive Streams) for LAN congestion. This would require 802.11Q priority tagging by the hosts or other devices for best results. Otherwise the traffic shaping (leaky bucket) applied to the BE traffic class could also help. Any 802.11Q tagging would likely get lost over the first router it crosses. I don't think highly of this solution, except prehaps to keep the wifi speakers and the wifi streaming video and the downloads from stepping on each other in the home or soho LAN. A good wifi router can also solve this without extensions to Ethernet, but usually with configuration. Curtis In message Anoop Ghanwani writes: --===============7512874271394803300== Content-Type: multipart/alternative; boundary=f46d043be28624734e04e8a2f27d --f46d043be28624734e04e8a2f27d Content-Type: text/plain; charset=ISO-8859-1 On Fri, Oct 11, 2013 at 8:18 PM, Michael Richardson wrote: > > Eggert, Lars wrote: > > I've heard that statement [that RED is supported] from many different > > people. I wonder if it is > > actually true. Is there any hard data on this? > > The issue I've had is that while RED is supported on a routing device, > and I can enable it on a per-VLAN basis, that does me little good, because > there are bottlenecks in layer-2 devices (ethernet, but also DSL,GPON, > etc.) > which are out of my control. > Most Ethernet switches that I'm aware of do already implement WRED. Can't speak for other devices (DSL, etc.) since I don't have experience with those. > > What I would like is a standard layer-3 way to measure bandwidth across > my layer-3 hop, and autoconfigure RED (or CoDel) to the measured bandwidth. > My understanding is that some equipment can use 802.1ad CCP and/or Loopback > frames to measure bandwidth across bridged layer-2 ethernet networks. > You probably mean 802.1ag. > > I'm unclear if this is a feature of that equipment, or a part of the > standard. I don't believe bandwidth measurement is part of the standard, but certainly one can choose to use these protocols to measure the bandwidth. However, it's an expensive process. However, bandwidth will variable if the L2 network is shared by multiple L3 devices, and this means the measurements would have to be made continuously, making it counter-productive. Unless, some mechanism is used to pre-provision bandwidth between L3 peers. Anoop --f46d043be28624734e04e8a2f27d Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable



On Fri, Oct 11, 2013 at 8:18 PM, Michael Richardson <mcr+ietf@= sandelman.ca> wrote:

Eggert, Lars <lars@netapp.com>= wrote:
=A0 =A0 > I've heard that statement [that RED is supported] from man= y different
=A0 =A0 > people. I wonder if it is
=A0 =A0 > actually true. Is there any hard data on this?

The issue I've had is that while RED is supported on a routing de= vice,
and I can enable it on a per-VLAN basis, that does me little good, because<= br> there are bottlenecks in layer-2 devices (ethernet, but also DSL,GPON, etc.= )
which are out of my control.

Most Ether= net switches that I'm aware of do already implement WRED. =A0 Can't= speak for other devices (DSL, etc.) since I don't have experience with= those.=A0

What I would like is a standard layer-3 way to measure bandwidth across
my layer-3 hop, and autoconfigure RED (or CoDel) to the measured bandwidth.=
My understanding is that some equipment can use 802.1ad CCP and/or Loopback=
frames to measure bandwidth across bridged layer-2 ethernet networks.

You probably mean 802.1ag.=A0

I'm unclear if this is a feature of that equipment, or a part of the standard. =A0

I don't believe bandwidth= measurement is part of the standard, but certainly one can choose to use t= hese protocols to measure the bandwidth. =A0However, it's an expensive = process. =A0However, bandwidth will variable if the L2 network is shared by= multiple L3 devices, and this means the measurements would have to be made= continuously, making it counter-productive. =A0Unless, some mechanism is u= sed to pre-provision bandwidth between L3 peers.

Anoop
--f46d043be28624734e04e8a2f27d-- --===============7512874271394803300== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ aqm mailing list aqm@ietf.org https://www.ietf.org/mailman/listinfo/aqm --===============7512874271394803300==-- From dave.taht@gmail.com Mon Oct 14 21:36:10 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DD80221E819F for ; Mon, 14 Oct 2013 21:36:10 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.6 X-Spam-Level: X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, NO_RELAYS=-0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xgO16WCxqZPk for ; Mon, 14 Oct 2013 21:36:06 -0700 (PDT) Received: from mail-wi0-x232.google.com (mail-wi0-x232.google.com [IPv6:2a00:1450:400c:c05::232]) by ietfa.amsl.com (Postfix) with ESMTP id 63A2221F9697 for ; Mon, 14 Oct 2013 21:36:02 -0700 (PDT) Received: by mail-wi0-f178.google.com with SMTP id hn9so932839wib.17 for ; Mon, 14 Oct 2013 21:35:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; bh=cYItL6WjmU6OBcSF3MuMvWrHTcapOYMmZ60NzNpWpnc=; b=ct9Dr6PIq683+XJZIFBiJCFKJrOkyEzz0yAnbnCxK17VegPIQFvxzdO5/gttvgmo3m j7tp5HwlPVT6LDboRxRsv7s0YZeXgdduBLTP7RyfFaZ4aXo2wfLNBnmBrupYNArD0BZb KuerNcdKhw1YVuzxF47RQcTbcGgu/YBPV6wYahOQfOoqEBKug3wrtCfN7IA7O/vmwRDl Bh0WZrELoaNJJoduy9iY34YoLct9Nh3462O+BZgTI/QqpnRYU8RpYsWLqwb0Mzp//sdv t0hsLhX3BhMTa9g5RwytjZlkwtu/S3z+4hgckeaH+XhpQ59DhYwSOKigHxVbjey5K4C1 ku0g== MIME-Version: 1.0 X-Received: by 10.194.119.132 with SMTP id ku4mr90027wjb.51.1381811758139; Mon, 14 Oct 2013 21:35:58 -0700 (PDT) Received: by 10.217.67.202 with HTTP; Mon, 14 Oct 2013 21:35:58 -0700 (PDT) In-Reply-To: <201309171233.r8HCXKkV003952@bagheera.jungle.bt.co.uk> References: <201309171233.r8HCXKkV003952@bagheera.jungle.bt.co.uk> Date: Mon, 14 Oct 2013 21:35:58 -0700 Message-ID: From: Dave Taht To: "aqm@ietf.org" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: [aqm] Fwd: [tsvwg] Guidelines for Adding Congestion Notification to Protocols that Encapsulate IP X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Oct 2013 04:36:11 -0000 It is my general assumption that membership in the aqm wg is a subset of tsvwg, but in case it isn't... ---------- Forwarded message ---------- From: Bob Briscoe Date: Tue, Sep 17, 2013 at 5:33 AM Subject: [tsvwg] Guidelines for Adding Congestion Notification to Protocols that Encapsulate IP To: tsvwg IETF list Cc: Gorry Fairhurst TSV folks, We've just posted an update to the guidelines for adding congestion notification to L2 and tunnelling protocols, e.g. Ethernet, L2TP, GRE, PPTP, GTP etc. Especially of current interest in mobile and separately in data centres. Pls read/review and send any comments to the list. It's currently still an individual draft; intended status: BCP. We're looking for support for adoption as a item of work in tsvwg. We've dealt with most of the outstanding issues from previous reviews, esp. those from Gorry (see summary of diffs below). -------------------------------- Summary of Diffs from -02 to -03 * Scope section: + Added dependence on correct propagation of traffic class information + For the feed-backward mode, deemed multicast and anycast out of scope * Ensured all guidelines referring to subnet technologies also refer to tunnels and vice versa by adding applicability sentences at the start of sections 4.1, 4.2, 4.3, 4.4, 4.6 and 5. * Added Security Considerations on ensuring congestion signal fields are classed as immutable and on using end-to-end congestion signal integrity technologies rather than hop-by- hop. Bob > From: > To: John Kaippallimalil , > Bob Briscoe > , > "Bob J.Briscoe" , > Pat Thaler > , > Patricia Thaler > Subject: New Version Notification for > draft-briscoe-tsvwg-ecn-encap-guidelines-03.txt > > > A new version of I-D, draft-briscoe-tsvwg-ecn-encap-guidelines-03.txt > has been successfully submitted by Bob Briscoe and posted to the > IETF repository. > > Filename: draft-briscoe-tsvwg-ecn-encap-guidelines > Revision: 03 > Title: Guidelines for Adding Congestion Notification to Protocol= s that Encapsulate IP > Creation date: 2013-09-17 > Group: Individual Submission > Number of pages: 29 > URL: http://www.ietf.org/internet-drafts/draft-briscoe-tsvwg-ecn-encap-gu= idelines-03.txt > Status: http://datatracker.ietf.org/doc/draft-briscoe-tsvwg-ecn-encap-gui= delines > Htmlized: http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidel= ines-03 > Diff: http://www.ietf.org/rfcdiff?url2=3Ddraft-briscoe-tsvwg-ecn-encap-gu= idelines-03 > > Abstract: > The purpose of this document is to guide the design of congestion > notification in any lower layer or tunnelling protocol that > encapsulates IP. The aim is for explicit congestion signals to > propagate consistently from lower layer protocols into IP. Then the > IP internetwork layer can act as a portability layer to carry > congestion notification from non-IP-aware congested nodes up to the > transport layer (L4). Following these guidelines should assure > interworking between new lower layer congestion notification > mechanisms, whether specified by the IETF or other standards bodies. > > > > > > Please note that it may take a couple of minutes from the time of submiss= ion > until the htmlized version and diff are available at tools.ietf.org. > > The IETF Secretariat ________________________________________________________________ Bob Briscoe, BT --=20 Dave T=E4ht Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.= html From wes@mti-systems.com Tue Oct 15 18:00:26 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7F75421F9C8B for ; Tue, 15 Oct 2013 18:00:26 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.429 X-Spam-Level: X-Spam-Status: No, score=-2.429 tagged_above=-999 required=5 tests=[AWL=-0.057, BAYES_00=-2.599, SARE_SUB_OBFU_Q1=0.227] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8tISyVx2tFz9 for ; Tue, 15 Oct 2013 18:00:20 -0700 (PDT) Received: from atl4mhob14.myregisteredsite.com (atl4mhob14.myregisteredsite.com [209.17.115.52]) by ietfa.amsl.com (Postfix) with ESMTP id 6569121F99DE for ; Tue, 15 Oct 2013 18:00:20 -0700 (PDT) Received: from mailpod.hostingplatform.com ([10.30.71.203]) by atl4mhob14.myregisteredsite.com (8.14.4/8.14.4) with ESMTP id r9G10J02032288 for ; Tue, 15 Oct 2013 21:00:19 -0400 Received: (qmail 18297 invoked by uid 0); 16 Oct 2013 01:00:19 -0000 X-TCPREMOTEIP: 69.81.143.143 X-Authenticated-UID: wes@mti-systems.com Received: from unknown (HELO ?192.168.1.129?) (wes@mti-systems.com@69.81.143.143) by 0 with ESMTPA; 16 Oct 2013 01:00:19 -0000 Message-ID: <525DE520.7040501@mti-systems.com> Date: Tue, 15 Oct 2013 21:00:16 -0400 From: Wesley Eddy Organization: MTI Systems User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.0.1 MIME-Version: 1.0 To: curtis@ipv6.occnc.com, Michael Richardson References: <201310141707.r9EH7wC7068478@gateway1.orleans.occnc.com> In-Reply-To: <201310141707.r9EH7wC7068478@gateway1.orleans.occnc.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: "Akhtar, Shahid \(Shahid\)" , "iccrg@irtf.org" , "Eggert, Lars" , "aqm@ietf.org" , bloat Subject: Re: [aqm] [Bloat] [iccrg] AQM deployment status? X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Oct 2013 01:00:26 -0000 On 10/14/2013 1:07 PM, Curtis Villamizar wrote: > So my first question to the AQM WG is "what is the scope of AQM WG > work in terms of where in the network this WG wants to focus?" If the > answer to that question is "everywhere", then we have to be aware that > conditions in core and conditions in home or enterprise are very > different. If the focus is on home, soho, and small business, then > the charter should say so (I don't think it is). The charter says: """ It is expected that some classes of algorithms will focus on software implementations, while others on existing or new hardware deployments, and algorithms may be specific to distinct scenarios. """ I would say anywhere that AQM algorithms can have a positive impact is in scope, and that it's understood and accepted that particular algorithms or tuning rules may not be ideal across different environments (or may not be easily implemented in different kinds of platforms). There was at least some talk of "applicability statements" for things that wind up being recommended by the working group. In my opinion, the home broadband router is one well-known case that may hold a lot of the initial attention in the working group, because the barriers to implementing/testing/deploying new algorithms for this case are less than for many others. We definitely did not want to limit the charter to this scenario though, and it is intentionally open to others. I hope this clears it up! -- Wes Eddy MTI Systems From internet-drafts@ietf.org Thu Oct 17 12:42:22 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 099E111E82E3; Thu, 17 Oct 2013 12:42:22 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.462 X-Spam-Level: X-Spam-Status: No, score=-102.462 tagged_above=-999 required=5 tests=[AWL=-0.089, BAYES_00=-2.599, NO_RELAYS=-0.001, SARE_SUB_OBFU_Q1=0.227, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id c532qarBlkWx; Thu, 17 Oct 2013 12:42:21 -0700 (PDT) Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id E844811E82E6; Thu, 17 Oct 2013 12:42:17 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: internet-drafts@ietf.org To: i-d-announce@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 4.80.p3 Auto-Submitted: auto-generated Precedence: bulk Message-ID: <20131017194217.32613.1808.idtracker@ietfa.amsl.com> Date: Thu, 17 Oct 2013 12:42:17 -0700 Cc: aqm@ietf.org Subject: [aqm] I-D Action: draft-ietf-aqm-recommendation-00.txt X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Oct 2013 19:42:22 -0000 A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the Active Queue Management and Packet Schedu= ling Working Group of the IETF. Title : IETF Recommendations Regarding Active Queue Management Author(s) : Fred Baker Godred Fairhurst Filename : draft-ietf-aqm-recommendation-00.txt Pages : 21 Date : 2013-10-17 Abstract: This memo presents recommendations to the Internet community concerning measures to improve and preserve Internet performance. It presents a strong recommendation for testing, standardization, and widespread deployment of active queue management (AQM) in network devices, to improve the performance of today's Internet. It also urges a concerted effort of research, measurement, and ultimate deployment of AQM mechanisms to protect the Internet from flows that are not sufficiently responsive to congestion notification. The note largely repeats the recommendations of RFC 2309, updated after fifteen years of experience and new research. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-aqm-recommendation There's also a htmlized version available at: http://tools.ietf.org/html/draft-ietf-aqm-recommendation-00 Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From rs@netapp.com Sun Oct 20 03:31:25 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 74F6211E83A5 for ; Sun, 20 Oct 2013 03:31:25 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.599 X-Spam-Level: X-Spam-Status: No, score=-10.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KXtgiPZaLgdU for ; Sun, 20 Oct 2013 03:31:21 -0700 (PDT) Received: from mx1.netapp.com (mx1.netapp.com [216.240.18.38]) by ietfa.amsl.com (Postfix) with ESMTP id E63FE11E83A4 for ; Sun, 20 Oct 2013 03:31:20 -0700 (PDT) X-IronPort-AV: E=Sophos;i="4.93,533,1378882800"; d="scan'208";a="286884151" Received: from vmwexceht01-prd.hq.netapp.com ([10.106.76.239]) by mx1-out.netapp.com with ESMTP; 20 Oct 2013 03:31:19 -0700 Received: from SACEXCMBX02-PRD.hq.netapp.com ([169.254.1.86]) by vmwexceht01-prd.hq.netapp.com ([10.106.76.239]) with mapi id 14.03.0123.003; Sun, 20 Oct 2013 03:31:19 -0700 From: "Scheffenegger, Richard" To: "aqm@ietf.org" Thread-Topic: IETF 88 Agenda Thread-Index: Ac7NfreyzmbcQnE1TPe/Yata1hOExA== Date: Sun, 20 Oct 2013 10:31:18 +0000 Message-ID: <012C3117EDDB3C4781FD802A8C27DD4F25E1AC01@SACEXCMBX02-PRD.hq.netapp.com> Accept-Language: de-AT, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.104.60.117] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: [aqm] IETF 88 Agenda X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Oct 2013 10:31:25 -0000 Hi, A tentative agenda has been uploaded to http://www.ietf.org/proceedings/88/= agenda/agenda-88-aqm We plan to spend the majority of the 1st session (Tuesday) on the adopted A= QM recommendations draft, and the 2nd slot (Friday) mostly on the AQM evalu= ation criteria discussion. We would also want to highlight the accompanying talks in ICCRG (Tuesday af= ter AQM): * Shahid Akhtar, An Evaluation of Various AQM techniques on Access Networks= with Realistic Internet Traffic * Naeem Khademi, Evaluating CoDel, FQ_CoDel and PIE: how good are they real= ly? =20 Please send Additions and Objections to aqm-chairs@ietf.org We could probably also carter for one or two additional presentations, idea= lly on one of the two subjects above. Richard Scheffenegger From bob.briscoe@bt.com Mon Oct 21 16:41:30 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E990711E8834; Mon, 21 Oct 2013 16:41:29 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.749 X-Spam-Level: X-Spam-Status: No, score=-2.749 tagged_above=-999 required=5 tests=[AWL=-0.546, BAYES_00=-2.599, MIME_QP_LONG_LINE=1.396, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pOniafac-Zzd; Mon, 21 Oct 2013 16:41:25 -0700 (PDT) Received: from hubrelay-by-04.bt.com (hubrelay-by-04.bt.com [62.7.242.140]) by ietfa.amsl.com (Postfix) with ESMTP id 8F91311E8837; Mon, 21 Oct 2013 16:40:49 -0700 (PDT) Received: from EVMHR72-UKRD.domain1.systemhost.net (10.36.3.110) by EVMHR04-UKBR.bt.com (10.216.161.36) with Microsoft SMTP Server (TLS) id 8.3.297.1; Tue, 22 Oct 2013 00:40:41 +0100 Received: from EPHR02-UKIP.domain1.systemhost.net (147.149.100.81) by EVMHR72-UKRD.domain1.systemhost.net (10.36.3.110) with Microsoft SMTP Server (TLS) id 8.3.279.1; Tue, 22 Oct 2013 00:40:42 +0100 Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR02-UKIP.domain1.systemhost.net (147.149.100.81) with Microsoft SMTP Server id 14.2.347.0; Tue, 22 Oct 2013 00:40:41 +0100 Received: from BTP075694.jungle.bt.co.uk ([10.109.40.34]) by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id r9LNee5p025162; Tue, 22 Oct 2013 00:40:40 +0100 Message-ID: <201310212340.r9LNee5p025162@bagheera.jungle.bt.co.uk> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Tue, 22 Oct 2013 00:40:39 +0100 To: Dave Taht From: Bob Briscoe In-Reply-To: References: <201309171233.r8HCXKkV003952@bagheera.jungle.bt.co.uk> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1"; format=flowed Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158 Cc: "aqm@ietf.org" , tsvwg IETF list Subject: [aqm] Immediate and Explicit Congestion Notification (was: Guidelines for Adding Congestion Notification to Protocols that Encapsulate IP) X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Oct 2013 23:41:30 -0000 Dave, Thx for cross-posting. It's quite possible there are aqm people not on= tsvwg. 1) Did you have views on the ecn-encap-guidelines I-D itself? Is it helpful? I think it's fairly non-controversial. 2) In case I'm misunderstood, I don't see much=20 latency gain from ECN as currently defined=20 (RFC3168). I'm still banging on about ECN in=20 lower layers 'cos early tests show promise that=20 we may have found a way to make the ultra-low=20 queuing delay of data centre TCP incrementally=20 deployable on the public Internet We =3D Mirja Kuehlewind, David Wagner, Juan Manuel Reyes Espinosa= and I. It can be deployed with most existing kit,=20 without any new AQM implementation needed. We use=20 WRED in an unusual configuration. The idea in a nutshell: * For ECN-capable packets, shift the job of hiding bursts from network to= host: - ECN-capable packets map to a WRED policy with queue smoothing turned= off - so the network signals ECN with zero smoothing delay - then the transport can hide bursts of ECN=20 signals from itself (like DCTCP does) - the transport knows o that it's ECN-capable o whether it's TCP or RTP etc, o whether its in congestion avoidance or slow-start, o and it knows its RTT, o so it can know whether to respond immediately or to smooth the= signals, o and if so, over what time - then short RTT flows can smooth the signals=20 with only the delay of their /own/ RTT o so they can fill troughs and absorb peaks that longer RTT flows= cannot - a TCP only needs to smooth the signals if in congestion avoidance o in slow start, it can respond immediately, thus reducing overshoot There's more to it - I've asked to present the whole story in Vancouver. Bob At 05:35 15/10/2013, Dave Taht wrote: >It is my general assumption that membership in the aqm wg is a subset >of tsvwg, but in case it isn't... > > >---------- Forwarded message ---------- >From: Bob Briscoe >Date: Tue, Sep 17, 2013 at 5:33 AM >Subject: [tsvwg] Guidelines for Adding Congestion Notification to >Protocols that Encapsulate IP >To: tsvwg IETF list >Cc: Gorry Fairhurst > > >TSV folks, > >We've just posted an update to the guidelines for adding congestion >notification to L2 and tunnelling protocols, e.g. Ethernet, L2TP, GRE, >PPTP, GTP etc. Especially of current interest in mobile and separately >in data centres. > >Pls read/review and send any comments to the list. > > >It's currently still an individual draft; intended status: BCP. >We're looking for support for adoption as a item of work in tsvwg. > >We've dealt with most of the outstanding issues from previous reviews, >esp. those from Gorry (see summary of diffs below). > >-------------------------------- >Summary of Diffs from -02 to -03 > > * Scope section: > > + Added dependence on correct propagation of traffic class > information > > + For the feed-backward mode, deemed multicast and anycast out > of scope > > * Ensured all guidelines referring to subnet technologies also > refer to tunnels and vice versa by adding applicability > sentences at the start of sections 4.1, 4.2, 4.3, 4.4, 4.6 and > 5. > > * Added Security Considerations on ensuring congestion signal > fields are classed as immutable and on using end-to-end > congestion signal integrity technologies rather than hop-by- > hop. > > >Bob > > > > From: > > To: John Kaippallimalil , > > Bob Briscoe > > , > > "Bob J.Briscoe" , > > Pat Thaler > > , > > Patricia Thaler > > Subject: New Version Notification for > > draft-briscoe-tsvwg-ecn-encap-guidelines-03.txt > > > > > > A new version of I-D, draft-briscoe-tsvwg-ecn-encap-guidelines-03.txt > > has been successfully submitted by Bob Briscoe and posted to the > > IETF repository. > > > > Filename: draft-briscoe-tsvwg-ecn-encap-guidelines > > Revision: 03 > > Title: Guidelines for Adding=20 > Congestion Notification to Protocols that Encapsulate IP > > Creation date: 2013-09-17 > > Group: Individual Submission > > Number of pages: 29 > > URL:=20 >= http://www.ietf.org/internet-drafts/draft-briscoe-tsvwg-ecn-encap-guideline= s-03.txt > > Status:=20 > http://datatracker.ietf.org/doc/draft-briscoe-tsvwg-ecn-encap-guidelines > > Htmlized:=20 > http://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-encap-guidelines-03 > > Diff:=20 >= http://www.ietf.org/rfcdiff?url2=3Ddraft-briscoe-tsvwg-ecn-encap-guidelines= -03 > > > > Abstract: > > The purpose of this document is to guide the design of congestion > > notification in any lower layer or tunnelling protocol that > > encapsulates IP. The aim is for explicit congestion signals to > > propagate consistently from lower layer protocols into IP. Then the > > IP internetwork layer can act as a portability layer to carry > > congestion notification from non-IP-aware congested nodes up to the > > transport layer (L4). Following these guidelines should assure > > interworking between new lower layer congestion notification > > mechanisms, whether specified by the IETF or other standards bodies. > > > > > > > > > > > > Please note that it may take a couple of=20 > minutes from the time of submission > > until the htmlized version and diff are available at tools.ietf.org. > > > > The IETF Secretariat > > >________________________________________________________________ >Bob Briscoe, BT > > >-- >Dave T=E4ht > >Fixing bufferbloat with cerowrt:=20 >http://www.teklibre.com/cerowrt/subscribe.html >_______________________________________________ >aqm mailing list >aqm@ietf.org >https://www.ietf.org/mailman/listinfo/aqm ________________________________________________________________ Bob Briscoe, BT=20 From ford@isoc.org Tue Oct 22 03:56:17 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C11C621F9E8B for ; Tue, 22 Oct 2013 03:56:14 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.372 X-Spam-Level: X-Spam-Status: No, score=-102.372 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, SARE_SUB_OBFU_Q1=0.227, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YIqnb2bAQuAY for ; Tue, 22 Oct 2013 03:56:09 -0700 (PDT) Received: from smtp70.iad3a.emailsrvr.com (smtp70.iad3a.emailsrvr.com [173.203.187.70]) by ietfa.amsl.com (Postfix) with ESMTP id 9CA1011E8185 for ; Tue, 22 Oct 2013 03:56:07 -0700 (PDT) Received: from localhost (localhost.localdomain [127.0.0.1]) by smtp1.relay.iad3a.emailsrvr.com (SMTP Server) with ESMTP id 3F996601CE; Tue, 22 Oct 2013 06:56:07 -0400 (EDT) X-Virus-Scanned: OK Received: by smtp1.relay.iad3a.emailsrvr.com (Authenticated sender: ford-AT-isoc.org) with ESMTPSA id 568276056D; Tue, 22 Oct 2013 06:56:06 -0400 (EDT) Mime-Version: 1.0 (Mac OS X Mail 6.6 \(1510\)) Content-Type: text/plain; charset=iso-8859-1 From: Matthew Ford In-Reply-To: <201310212340.r9LNee5p025162@bagheera.jungle.bt.co.uk> Date: Tue, 22 Oct 2013 11:56:04 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <657D1484-1B02-41F5-9FD1-0921439DF754@isoc.org> References: <201309171233.r8HCXKkV003952@bagheera.jungle.bt.co.uk> <201310212340.r9LNee5p025162@bagheera.jungle.bt.co.uk> To: Bob Briscoe X-Mailer: Apple Mail (2.1510) Cc: tsvwg IETF list , Dave Taht , "aqm@ietf.org" Subject: Re: [aqm] Immediate and Explicit Congestion Notification (was: Guidelines for Adding Congestion Notification to Protocols that Encapsulate IP) X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Oct 2013 10:56:23 -0000 On 22 Oct 2013, at 00:40, Bob Briscoe wrote: > There's more to it - I've asked to present the whole story in = Vancouver. Cool, looking forward to it. I'll prime the question pump now if I may: Does the transport need to know that it is connected to a network that = will signal ECN in this fashion? Mat From bclaise@cisco.com Thu Oct 24 09:36:22 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5DBF811E8374 for ; Thu, 24 Oct 2013 09:36:09 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.392 X-Spam-Level: X-Spam-Status: No, score=-10.392 tagged_above=-999 required=5 tests=[AWL=-0.020, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, SARE_SUB_OBFU_Q1=0.227] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GZeUG8mXoH-K for ; Thu, 24 Oct 2013 09:35:34 -0700 (PDT) Received: from ams-iport-4.cisco.com (ams-iport-4.cisco.com [144.254.224.147]) by ietfa.amsl.com (Postfix) with ESMTP id 592F511E81AD for ; Thu, 24 Oct 2013 09:35:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=788; q=dns/txt; s=iport; t=1382632511; x=1383842111; h=message-id:date:from:mime-version:to:subject:references: in-reply-to:content-transfer-encoding; bh=UcUf9BkX2nz4yKq3iiS4owayajUEwSahjffP3E+qi54=; b=RZDnJ4eh3gS/+aYMWpOxlzGY7y5vvKqKnJJmqlYbhilCojDOVjSogbmn 9LUvni8mK+CNtB7xlwvdzyNrmnhpaQvyaxzxHS/uLVxmO2DptOmnDmMHO wGXNmF9jVuWYfe8lCeyDFhLhx/Tl/+SdFP4Wl4tumGJiOrqyg90bOOOef k=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgIFAIpLaVKQ/khN/2dsb2JhbABZgwe/Y4EdFnSCJgEBBDg6BhEsFg8JAwIBAgFFEwgBAYgDukuPVBaEFgOYCoY7i0yBZoFAOg X-IronPort-AV: E=Sophos;i="4.93,563,1378857600"; d="scan'208";a="18988353" Received: from ams-core-4.cisco.com ([144.254.72.77]) by ams-iport-4.cisco.com with ESMTP; 24 Oct 2013 16:35:10 +0000 Received: from [10.60.67.89] (ams-bclaise-8918.cisco.com [10.60.67.89]) by ams-core-4.cisco.com (8.14.5/8.14.5) with ESMTP id r9OGZ5pN008578 for ; Thu, 24 Oct 2013 16:35:07 GMT Message-ID: <52694C39.3020606@cisco.com> Date: Thu, 24 Oct 2013 18:35:05 +0200 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130801 Thunderbird/17.0.8 MIME-Version: 1.0 To: aqm@ietf.org References: <52694092.9020809@cisco.com> In-Reply-To: <52694092.9020809@cisco.com> X-Forwarded-Message-Id: <52694092.9020809@cisco.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [aqm] on the relation between draft-ietf-tsvwg-byte-pkt-congest-11 and draft-ietf-aqm-recommendation X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Oct 2013 16:36:36 -0000 Dear all, After reviewing draft-ietf-tsvwg-byte-pkt-congest-11 (part of the IESG telechat) and draft-ietf-aqm-recommendation, I was wondering what the connection was between the two drafts. draft-ietf-tsvwg-byte-pkt-congest will update RFC 2309, while draft-ietf-aqm-recommendation will obsolote RFC 2309. I now understand that this is not an issue. I believe that draft-ietf-aqm-recommendation should contain the recommendations from draft-ietf-tsvwg-byte-pkt-congest (basically the section 2). This is the case now AFAICT. Then draft-ietf-tsvwg-byte-pkt-congest could be referenced, informatively, if someone wants to read the justifications behind the recommendations. Right now, the reference is normative. Regards, Benoit (as a contributor) From john@jlc.net Thu Oct 24 13:10:44 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 501E211E81A0 for ; Thu, 24 Oct 2013 13:10:44 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.297 X-Spam-Level: X-Spam-Status: No, score=-106.297 tagged_above=-999 required=5 tests=[AWL=0.075, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, SARE_SUB_OBFU_Q1=0.227, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9e9But7Kp-Tc for ; Thu, 24 Oct 2013 13:10:39 -0700 (PDT) Received: from mailhost.jlc.net (mailhost.jlc.net [199.201.159.4]) by ietfa.amsl.com (Postfix) with ESMTP id 009BE11E835E for ; Thu, 24 Oct 2013 13:10:38 -0700 (PDT) Received: by mailhost.jlc.net (Postfix, from userid 104) id 56772C94EA; Thu, 24 Oct 2013 16:10:36 -0400 (EDT) Date: Thu, 24 Oct 2013 16:10:36 -0400 From: John Leslie To: Benoit Claise Message-ID: <20131024201036.GG65952@verdi> References: <52694092.9020809@cisco.com> <52694C39.3020606@cisco.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <52694C39.3020606@cisco.com> User-Agent: Mutt/1.4.1i Cc: aqm@ietf.org Subject: Re: [aqm] on the relation between draft-ietf-tsvwg-byte-pkt-congest-11 and draft-ietf-aqm-recommendation X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Oct 2013 20:10:44 -0000 Benoit Claise wrote: > > I believe that draft-ietf-aqm-recommendation should contain the > recommendations from draft-ietf-tsvwg-byte-pkt-congest (basically the > section 2). This is the case now AFAICT. Then > draft-ietf-tsvwg-byte-pkt-congest could be referenced, informatively, if > someone wants to read the justifications behind the recommendations. +1 -- John Leslie From bob.briscoe@bt.com Fri Oct 25 14:16:30 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8947811E81EC; Fri, 25 Oct 2013 14:16:30 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.408 X-Spam-Level: X-Spam-Status: No, score=-3.408 tagged_above=-999 required=5 tests=[AWL=-0.037, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1, SARE_SUB_OBFU_Q1=0.227] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mQh62ZfZC6IT; Fri, 25 Oct 2013 14:16:25 -0700 (PDT) Received: from hubrelay-by-04.bt.com (hubrelay-by-04.bt.com [62.7.242.140]) by ietfa.amsl.com (Postfix) with ESMTP id 9EC7511E8213; Fri, 25 Oct 2013 14:16:22 -0700 (PDT) Received: from EVMHR02-UKBR.domain1.systemhost.net (193.113.108.41) by EVMHR04-UKBR.bt.com (10.216.161.36) with Microsoft SMTP Server (TLS) id 8.3.297.1; Fri, 25 Oct 2013 22:16:17 +0100 Received: from EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) by EVMHR02-UKBR.domain1.systemhost.net (193.113.108.41) with Microsoft SMTP Server (TLS) id 8.3.297.1; Fri, 25 Oct 2013 22:16:18 +0100 Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) with Microsoft SMTP Server id 14.2.347.0; Fri, 25 Oct 2013 22:16:12 +0100 Received: from BTP075694.jungle.bt.co.uk ([10.111.172.244]) by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id r9PLGAFp007348; Fri, 25 Oct 2013 22:16:10 +0100 Message-ID: <201310252116.r9PLGAFp007348@bagheera.jungle.bt.co.uk> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Fri, 25 Oct 2013 22:16:09 +0100 To: Matthew Ford From: Bob Briscoe In-Reply-To: <657D1484-1B02-41F5-9FD1-0921439DF754@isoc.org> References: <201309171233.r8HCXKkV003952@bagheera.jungle.bt.co.uk> <201310212340.r9LNee5p025162@bagheera.jungle.bt.co.uk> <657D1484-1B02-41F5-9FD1-0921439DF754@isoc.org> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="=====================_552357903==.ALT" X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158 Cc: "aqm@ietf.org" , tsvwg IETF list , Dave Taht Subject: Re: [aqm] Immediate and Explicit Congestion Notification (was: Guidelines for Adding Congestion Notification to Protocols that Encapsulate IP) X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Oct 2013 21:16:30 -0000 --=====================_552357903==.ALT Content-Type: text/plain; charset="us-ascii"; format=flowed Mat, At 11:56 22/10/2013, Matthew Ford wrote: >On 22 Oct 2013, at 00:40, Bob Briscoe wrote: > > > There's more to it - I've asked to present the whole story in Vancouver. > >Cool, looking forward to it. I'll prime the question pump now if I may: > >Does the transport need to know that it is connected to a network >that will signal ECN in this fashion? Here's the outline of the talk I'm planning, that I sent to Martin originally. Most of the second half addresses your question. Exec summary * Early tests show promise that we may have found a way to make the ultra-low queuing delay of data centre TCP incrementally deployable on the public Internet * For rtcweb, we need to address a) cc for r-t media [rmcat w-g in progress] b) Making TCP nicer c) minimise ability of TCP to bloat queues [AQM w-g now in progress] This addresses b) & c) The problem * All AQMs delay dropping for about one (hard-coded) worst-case RTT, in case a burst dissipates (allegedly a 'good queue' according to Van Jacobson) * For a flow with 1/10 or 1/100 of this RTT (e.g. from a CDN or your home media server), any congestion signal is delayed tens or hundreds of its own RTTs by these AQMs. * A TCP flow in slow-start doesn't need the burst smoothed anyway - delaying the signal just makes slow-start overshoot more - a TCP in slow-start knows that it won't allow the burst to dissipate anyway The solution: make ECN also mean "Immediate Congestion Notification"? * For ECN-capable packets, shift the job of hiding bursts from network to host: - the network signals ECN with no smoothing delay - then the transport can hide bursts of ECN signals from itself - the transport knows o whether it's TCP or RTP etc, o whether its in congestion avoidance or slow-start, o and it knows its RTT, o so it can know whether to respond immediately or to smooth the signals, o and if so, over what time - then short RTT flows can smooth the signals with only the delay of their /own/ RTT o so they can fill troughs and absorb peaks that longer RTT flows cannot - a TCP only needs to smooth the signals if in congestion avoidance o in slow start, it can respond immediately, thus reducing overshoot Incremental Deployment: * Immediate congestion notification doesn't need new AQM implementation - it can use the widely implemented WRED algorithm with an unexpected configuration * The network classifies packets for this AQM treatment based on their ECN-capability - Without ECN, it smoothes the queue before signalling drops - With ECN, it signals immediately, without any smoothing delay - (as today, the operator can still use WRED with the Diffserv field too) * For TCP apps, the stack will use 'DCTCP' (we've tweaked it), if the ends negotiate ECN with the accurate feedback capability. * It should 'just work' if an RTP app or a Reno TCP uses ECN. The request: * Much more evaluation to do, but first we want to know: - if the idea works, would the IETF have an appetite for tweaking the definition of ECN so it is merely equivalent to drop in the long term, but the dynamics need not be equivalent. Much better than the ECN that didn't get deployed * This is Explicit and Immediate Congestion Notification (EICN?) - same wire protocol, much greater benefits * The advantage of the original ECN (avoiding congestive loss) was too small to be worth the deployment hassle * Predictable ultra-low latency without loss too (similar to DCTCP-ECN) would be worth deploying * But we all thought DCTCP could only be deployed in isolation (e.g. data centres) - we all thought DCTCP traffic would starve alongside today's TCP traffic - because in a DCTCP queue, the ECN threshold is lower than you would trigger drop - and we thought ECN & drop had to be equivalent. * We believe we've found a way to ensure DCTCP-ECN traffic doesn't starve - we still make DCTCP-ECN equivalent to drop in the long-run, but not in its dynamics More info A paper with early results is under submission (available on request). It shows neither this approach nor legacy traffic can starve the other under a wide range of conditions, but we have a lot more work to test the limits. Bob >Mat > >_______________________________________________ >aqm mailing list >aqm@ietf.org >https://www.ietf.org/mailman/listinfo/aqm ________________________________________________________________ Bob Briscoe, BT --=====================_552357903==.ALT Content-Type: text/html; charset="us-ascii" Mat,

At 11:56 22/10/2013, Matthew Ford wrote:

On 22 Oct 2013, at 00:40, Bob Briscoe <bob.briscoe@bt.com> wrote:

> There's more to it - I've asked to present the whole story in Vancouver.

Cool, looking forward to it. I'll prime the question pump now if I may:

Does the transport need to know that it is connected to a network that will signal ECN in this fashion?

Here's the outline of the talk I'm planning, that I sent to Martin originally. Most of the second half addresses your question.

Exec summary
* Early tests show promise that we may have found a way to make the ultra-low queuing delay of data centre TCP incrementally deployable on the public Internet
* For rtcweb, we need to address
  a) cc for r-t media [rmcat w-g in progress]
  b) Making TCP nicer
  c) minimise ability of TCP to bloat queues [AQM w-g now in progress]
  This addresses b) & c)

The problem
* All AQMs delay dropping for about one (hard-coded) worst-case RTT, in case a burst dissipates (allegedly a 'good queue' according to Van Jacobson)
* For a flow with 1/10 or 1/100 of this RTT (e.g. from a CDN or your home media server), any congestion signal is delayed tens or hundreds of its own RTTs by these AQMs.
* A TCP flow in slow-start doesn't need the burst smoothed anyway
  - delaying the signal just makes slow-start overshoot more
  - a TCP in slow-start knows that it won't allow the burst to dissipate anyway

The solution: make ECN also mean "Immediate Congestion Notification"?
* For ECN-capable packets, shift the job of hiding bursts from network to host:
  - the network signals ECN with no smoothing delay
  - then the transport can hide bursts of ECN signals from itself
  - the transport knows
    o whether it's TCP or RTP etc,
    o whether its in congestion avoidance or slow-start,
    o and it knows its RTT,
    o so it can know whether to respond immediately or to smooth the signals,
    o and if so, over what time
  - then short RTT flows can smooth the signals with only the delay of their /own/ RTT
    o so they can fill troughs and absorb peaks that longer RTT flows cannot
  - a TCP only needs to smooth the signals if in congestion avoidance
    o in slow start, it can respond immediately, thus reducing overshoot

Incremental Deployment:
* Immediate congestion notification doesn't need new AQM implementation
  - it can use the widely implemented WRED algorithm with an unexpected configuration
* The network classifies packets for this AQM treatment based on their ECN-capability
  - Without ECN, it smoothes the queue before signalling drops
  - With ECN, it signals immediately, without any smoothing delay
  - (as today, the operator can still use WRED with the Diffserv field too)
* For TCP apps, the stack will use 'DCTCP' (we've tweaked it), if the ends negotiate ECN with the accurate feedback capability.
* It should 'just work' if an RTP app or a Reno TCP uses ECN.

The request:
* Much more evaluation to do, but first we want to know:
  - if the idea works, would the IETF have an appetite for tweaking the definition of ECN so it is merely equivalent to drop in the long term, but the dynamics need not be equivalent.


Much better than the ECN that didn't get deployed
* This is Explicit and Immediate Congestion Notification (EICN?)
  - same wire protocol, much greater benefits
* The advantage of the original ECN (avoiding congestive loss) was too small to be worth the deployment hassle
* Predictable ultra-low latency without loss too (similar to DCTCP-ECN) would be worth deploying
* But we all thought DCTCP could only be deployed in isolation (e.g. data centres)
  - we all thought DCTCP traffic would starve alongside today's TCP traffic
  - because in a DCTCP queue, the ECN threshold is lower than you would trigger drop
  - and we thought ECN & drop had to be equivalent.
* We believe we've found a way to ensure DCTCP-ECN traffic doesn't starve
  - we still make DCTCP-ECN equivalent to drop in the long-run, but not in its dynamics

More info
A paper with early results is under submission  (available on request). It shows neither this approach nor legacy traffic can starve the other under a wide range of conditions, but we have a lot more work to test the limits.



Bob


Mat

_______________________________________________
aqm mailing list
aqm@ietf.org
https://www.ietf.org/mailman/listinfo/aqm

________________________________________________________________
Bob Briscoe,                                                  BT --=====================_552357903==.ALT-- From john@jlc.net Sat Oct 26 05:16:43 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1D27811E824B; Sat, 26 Oct 2013 05:16:43 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.304 X-Spam-Level: X-Spam-Status: No, score=-106.304 tagged_above=-999 required=5 tests=[AWL=0.068, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, SARE_SUB_OBFU_Q1=0.227, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id k5UaVVpGG6Vl; Sat, 26 Oct 2013 05:16:38 -0700 (PDT) Received: from mailhost.jlc.net (mailhost.jlc.net [199.201.159.4]) by ietfa.amsl.com (Postfix) with ESMTP id C8D0B11E8199; Sat, 26 Oct 2013 05:16:34 -0700 (PDT) Received: by mailhost.jlc.net (Postfix, from userid 104) id D78C2C94C2; Sat, 26 Oct 2013 08:16:32 -0400 (EDT) Date: Sat, 26 Oct 2013 08:16:32 -0400 From: John Leslie To: Bob Briscoe Message-ID: <20131026121632.GJ65952@verdi> References: <201309171233.r8HCXKkV003952@bagheera.jungle.bt.co.uk> <201310212340.r9LNee5p025162@bagheera.jungle.bt.co.uk> <657D1484-1B02-41F5-9FD1-0921439DF754@isoc.org> <201310252116.r9PLGAFp007348@bagheera.jungle.bt.co.uk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201310252116.r9PLGAFp007348@bagheera.jungle.bt.co.uk> User-Agent: Mutt/1.4.1i Cc: "aqm@ietf.org" , tsvwg IETF list Subject: Re: [aqm] [tsvwg] Immediate and Explicit Congestion Notification (was: Guidelines for Adding Congestion Notification to Protocols that Encapsulate IP) X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Oct 2013 12:16:43 -0000 (If this continues, we've got to fix the title and cross-posting...) Bob Briscoe wrote: > > Exec summary > * Early tests show promise that we may have found a way to make the > ultra-low queuing delay of data centre TCP incrementally deployable > on the public Internet > * For rtcweb, we need to address > a) cc for r-t media [rmcat w-g in progress] > b) Making TCP nicer > c) minimise ability of TCP to bloat queues [AQM w-g now in progress] > This addresses b) & c) > > The problem > * All AQMs delay dropping for about one (hard-coded) worst-case RTT, > in case a burst dissipates (allegedly a 'good queue' according to Van > Jacobson) This assertion is going to need a lot of support. Bob is a man after my own heart suggesting that an ECN notification may be sent earlier than a packet drop would be indicated. I don't know if we can get there; but IMHO that is essential to getting ECN deployed and used. I don't think I agree with Bob that what's hard-coded is necessarily a "worst-case" RTT -- and I'm quite sure I'm not willing to make any pronouncement about "all AQMs". I suggest the talk might be more useful if Bob outlined the AQMs currently in widespread use and detailed _how_ they delay dropping for an estimated RTT. > * For a flow with 1/10 or 1/100 of this RTT (e.g. from a CDN or your > home media server), any congestion signal is delayed tens or hundreds > of its own RTTs by these AQMs. Clearly, RTTs differing by a factor of ten are quite common at most nodes traversed in a typical path; and it seems _very_ suboptimal to have the responsibility for guessing the RTT at the node which must drop packets. > * A TCP flow in slow-start doesn't need the burst smoothed anyway > - delaying the signal just makes slow-start overshoot more > - a TCP in slow-start knows that it won't allow the burst to > dissipate anyway A critical point! (It seems obvious to me, but is it obvious to everyone?) > The solution: make ECN also mean "Immediate Congestion Notification"? > * For ECN-capable packets, shift the job of hiding bursts from network to > host: > - the network signals ECN with no smoothing delay > - then the transport can hide bursts of ECN signals from itself But can we get there from here? The node doing the ECN notification _can't_ know how the transport will react; and the transport receiving and ECN notification can't know whether the forwarding node has "smoothed" the signal. (It is truly a shame we haven't left any bits for signals like this!) > - the transport knows > o whether it's TCP or RTP etc, > o whether its in congestion avoidance or slow-start, > o and it knows its RTT, > o so it can know whether to respond immediately or to smooth the > signals, > o and if so, over what time Yes, but it can't know what smoothing may already have been applied. > - then short RTT flows can smooth the signals with only the delay > of their /own/ RTT > o so they can fill troughs and absorb peaks that longer RTT flows cannot > - a TCP only needs to smooth the signals if in congestion avoidance > o in slow start, it can respond immediately, thus reducing overshoot This would, IMHO, improve "slow start". > Incremental Deployment: > * Immediate congestion notification doesn't need new AQM implementation > - it can use the widely implemented WRED algorithm with an > unexpected configuration Bob is beginning to lose me here. Does he mean that a forwarding node would apply WRED for both drop and ECN, but with different parameters? > * The network classifies packets for this AQM treatment based on > their ECN-capability > - Without ECN, it smoothes the queue before signalling drops Bob has lost me now -- apparently he doesn't mean different parameters... and I don't recognize this "smoothing" step in WRED. > - With ECN, it signals immediately, without any smoothing delay > - (as today, the operator can still use WRED with the Diffserv field too) (Do we need to confuse this discussion by adding diffserv?) > * For TCP apps, the stack will use 'DCTCP' (we've tweaked it), if the > ends negotiate ECN with the accurate feedback capability. Have we settled on "accurate feedback" already? I thought that was still under discussion. (I don't follow exactly what it adds...) > * It should 'just work' if an RTP app or a Reno TCP uses ECN. I don't see any way for a Reno transport using ECN to avoid being starved if ECN arrives earlier (without notice). > The request: > * Much more evaluation to do, but first we want to know: > - if the idea works, would the IETF have an appetite for tweaking > the definition of ECN so it is merely equivalent to drop in the long > term, but the dynamics need not be equivalent. There's a good question there; but I don't think we're ready for it. I'd really like to discuss the dynamics of responding more quickly but perhaps less drastically for almost any real-time flow. But proving "equivalence in the long term" seems too hard. > Much better than the ECN that didn't get deployed > * This is Explicit and Immediate Congestion Notification (EICN?) > - same wire protocol, much greater benefits > * The advantage of the original ECN (avoiding congestive loss) was > too small to be worth the deployment hassle Actually, I don't agree that was the problem -- instead I believe the code has been deployed but administratively suppressed because the operators don't trust the transports. There _is_ a significant improvement from one-RTT reaction instead of several (to detect a drop), but the whole process is just too complicated, while the opportunity for abuse remains obvious. > * Predictable ultra-low latency without loss too (similar to > DCTCP-ECN) would be worth deploying I'm optimistic that latency will become an easier argument. > * But we all thought DCTCP could only be deployed in isolation (e.g. > data centres) > - we all thought DCTCP traffic would starve alongside today's TCP traffic > - because in a DCTCP queue, the ECN threshold is lower than you > would trigger drop > - and we thought ECN & drop had to be equivalent. (I'm not sure we'll succeed at breaking that "equivalence"...) > * We believe we've found a way to ensure DCTCP-ECN traffic doesn't starve > - we still make DCTCP-ECN equivalent to drop in the long-run, but > not in its dynamics (I'm still not sure it's worth arguing the "long-run".) -- John Leslie From rs@netapp.com Sat Oct 26 15:42:29 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A303221F9F0A; Sat, 26 Oct 2013 15:42:29 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.448 X-Spam-Level: X-Spam-Status: No, score=-10.448 tagged_above=-999 required=5 tests=[AWL=-0.076, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, SARE_SUB_OBFU_Q1=0.227] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vLn6SNHxAA4v; Sat, 26 Oct 2013 15:42:25 -0700 (PDT) Received: from mx2.netapp.com (mx2.netapp.com [216.240.18.37]) by ietfa.amsl.com (Postfix) with ESMTP id 8D9CA21F8319; Sat, 26 Oct 2013 15:42:25 -0700 (PDT) X-IronPort-AV: E=Sophos;i="4.93,578,1378882800"; d="scan'208";a="53101918" Received: from vmwexceht03-prd.hq.netapp.com ([10.106.76.241]) by mx2-out.netapp.com with ESMTP; 26 Oct 2013 15:42:24 -0700 Received: from SACEXCMBX02-PRD.hq.netapp.com ([169.254.1.86]) by vmwexceht03-prd.hq.netapp.com ([10.106.76.241]) with mapi id 14.03.0123.003; Sat, 26 Oct 2013 15:42:24 -0700 From: "Scheffenegger, Richard" To: John Leslie , Bob Briscoe Thread-Topic: [aqm] [tsvwg] Immediate and Explicit Congestion Notification (was: Guidelines for Adding Congestion Notification to Protocols that Encapsulate IP) Thread-Index: AQHO0kWRNdZgBsOVl06FWzWNER6BB5oHkc6Q Date: Sat, 26 Oct 2013 22:42:23 +0000 Message-ID: <012C3117EDDB3C4781FD802A8C27DD4F25E42144@SACEXCMBX02-PRD.hq.netapp.com> References: <201309171233.r8HCXKkV003952@bagheera.jungle.bt.co.uk> <201310212340.r9LNee5p025162@bagheera.jungle.bt.co.uk> <657D1484-1B02-41F5-9FD1-0921439DF754@isoc.org> <201310252116.r9PLGAFp007348@bagheera.jungle.bt.co.uk> <20131026121632.GJ65952@verdi> In-Reply-To: <20131026121632.GJ65952@verdi> Accept-Language: de-AT, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.104.60.115] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "aqm@ietf.org" , tsvwg IETF list Subject: Re: [aqm] [tsvwg] Immediate and Explicit Congestion Notification (was: Guidelines for Adding Congestion Notification to Protocols that Encapsulate IP) X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Oct 2013 22:42:29 -0000 HI John, (aqm chair hat off, and accecn author hat on) >> * For TCP apps, the stack will use 'DCTCP' (we've tweaked it), if the >> ends negotiate ECN with the accurate feedback capability. >=20 > Have we settled on "accurate feedback" already? I thought that was > still under discussion. (I don't follow exactly what it adds...) The current requirements draft give a concise list of what benefits it shou= ld bring. There is also the (individual) draft on the TCP ECN modifications, that try= to deliver these benefits within the confines of the TCP header without re= quiring additional bits. The main issue with traditional ECN/TCP is, that you only get one bit of in= formation per RTT (delivered over N segments sent in an RTT), thus the exte= nt of congestion (number of CE marks per this flows RTT), and the fine-grai= ned sequencing of the marks is lost. AccECN aims to deliver these two signals to the sender - but what the sende= r is then doing exactly with this much more detailed information is out of = scope for the AccECN drafts so far. http://tools.ietf.org/html/draft-ietf-tcpm-accecn-reqs http://tools.ietf.org/html/draft-kuehlewind-tcpm-accurate-ecn ( http://tools.ietf.org/html/draft-kuehlewind-tcpm-accurate-ecn-option ) Any feedback on these drafts during the TCPM session is appreciated. Richard Scheffenegger From bob.briscoe@bt.com Sun Oct 27 18:36:53 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C997011E82EA; Sun, 27 Oct 2013 18:36:53 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.107 X-Spam-Level: X-Spam-Status: No, score=-2.107 tagged_above=-999 required=5 tests=[AWL=-1.335, BAYES_50=0.001, RCVD_IN_DNSWL_LOW=-1, SARE_SUB_OBFU_Q1=0.227] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Clns4-sk8n3n; Sun, 27 Oct 2013 18:36:48 -0700 (PDT) Received: from hubrelay-by-04.bt.com (hubrelay-by-04.bt.com [62.7.242.140]) by ietfa.amsl.com (Postfix) with ESMTP id EF21111E82DF; Sun, 27 Oct 2013 18:36:45 -0700 (PDT) Received: from EVMHR01-UKBR.domain1.systemhost.net (193.113.108.40) by EVMHR04-UKBR.bt.com (10.216.161.36) with Microsoft SMTP Server (TLS) id 8.3.297.1; Mon, 28 Oct 2013 01:36:40 +0000 Received: from EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) by EVMHR01-UKBR.domain1.systemhost.net (193.113.108.40) with Microsoft SMTP Server (TLS) id 8.3.297.1; Mon, 28 Oct 2013 01:36:43 +0000 Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) with Microsoft SMTP Server id 14.2.347.0; Mon, 28 Oct 2013 01:36:43 +0000 Received: from BTP075694.jungle.bt.co.uk ([10.109.144.88]) by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id r9S1afJ0018887; Mon, 28 Oct 2013 01:36:41 GMT Message-ID: <201310280136.r9S1afJ0018887@bagheera.jungle.bt.co.uk> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Mon, 28 Oct 2013 01:36:39 +0000 To: John Leslie From: Bob Briscoe In-Reply-To: <20131026121632.GJ65952@verdi> References: <201309171233.r8HCXKkV003952@bagheera.jungle.bt.co.uk> <201310212340.r9LNee5p025162@bagheera.jungle.bt.co.uk> <657D1484-1B02-41F5-9FD1-0921439DF754@isoc.org> <201310252116.r9PLGAFp007348@bagheera.jungle.bt.co.uk> <20131026121632.GJ65952@verdi> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158 Cc: "aqm@ietf.org" , tsvwg IETF list Subject: Re: [aqm] [tsvwg] Immediate and Explicit Congestion Notification X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Oct 2013 01:36:53 -0000 John, inline... At 12:16 26/10/2013, John Leslie wrote: >Bob Briscoe wrote: > > > > Exec summary > > * Early tests show promise that we may have found a way to make the > > ultra-low queuing delay of data centre TCP incrementally deployable > > on the public Internet > > * For rtcweb, we need to address > > a) cc for r-t media [rmcat w-g in progress] > > b) Making TCP nicer > > c) minimise ability of TCP to bloat queues [AQM w-g now in progress] > > This addresses b) & c) > > > > The problem > > * All AQMs delay dropping for about one (hard-coded) worst-case RTT, > > in case a burst dissipates (allegedly a 'good queue' according to Van > > Jacobson) > > This assertion is going to need a lot of support. > > Bob is a man after my own heart suggesting that an ECN notification >may be sent earlier than a packet drop would be indicated. I don't know >if we can get there; but IMHO that is essential to getting ECN deployed >and used. > > I don't think I agree with Bob that what's hard-coded is necessarily >a "worst-case" RTT -- and I'm quite sure I'm not willing to make any >pronouncement about "all AQMs". > > I suggest the talk might be more useful if Bob outlined the AQMs >currently in widespread use and detailed _how_ they delay dropping >for an estimated RTT. You're right. The 'about' in my sentence was meant to indicate some leeway. The specifics depend on each AQM... The only AQM I know of that doesn't smooth over some nominal RTT is DCTCP itself. * CoDel was designed for 'interval' to be a worst-case (largest) RTT, which it recommends to be set to 100ms. One the queue has exceeded threshold, CoDel delays for time 'interval' before starting to signal congestion. - I already said how sluggish CoDel would be for a flow with a much shorter RTT than CoDel (others have made this point, e.g. for data centres : https://lists.bufferbloat.net/pipermail/codel/2012-August/000448.html). - And in the other direction, we already know that utilisation suffers fairly badly for flows with RTT significantly larger than 100ms. * PIE suppresses all drops for time max_burst (set to 100ms by default) from when the drop probability it calculates (but doesn't necessarily use) first rises above zero. This is very similar to CoDel, and similar comments are applicable. * RED requires the constant for its exponentially weighted moving average (w_q) to be set taking into account how many packets are likely to arrive at the link in a 'typical' RTT. Reverse engineering the values recommended by Sally Floyd in the RED paper and in her famous RED parameters Web page , she recommended a 'typical' RTT of about 130ms. [BTW, I know of people who don't calculate w_q, but just use the value of "0.002" that Sally recommended for her 45Mb/s link in the original RED paper simulations (and repeated at ). This was calculated assuming about 500 packets arrive at a link (from all flows) in a typical RTT. Links have got a lot faster since 1993. Nonetheless, she was considering 45Mb/s for an aggregated link in those days, and it happens to be about right for a single user today.] > > * For a flow with 1/10 or 1/100 of this RTT (e.g. from a CDN or your > > home media server), any congestion signal is delayed tens or hundreds > > of its own RTTs by these AQMs. > > Clearly, RTTs differing by a factor of ten are quite common at most >nodes traversed in a typical path; and it seems _very_ suboptimal to >have the responsibility for guessing the RTT at the node which must >drop packets. For packets that do not support ECN, the dropping node has to make a guess at the RTT, so as not to drop packets unnecessarily, because drop is an impairment as well as a congestion signal. So a transport cannot 'undrop' packets. Our point though is that a network node doesn't have to mimic this behaviour for ECN packets, because ECN is not an impairment. So a transport can un-ECN-mark packets (by smoothing out bursts itself). > > * A TCP flow in slow-start doesn't need the burst smoothed anyway > > - delaying the signal just makes slow-start overshoot more > > - a TCP in slow-start knows that it won't allow the burst to > > dissipate anyway > > A critical point! (It seems obvious to me, but is it obvious to >everyone?) > > > The solution: make ECN also mean "Immediate Congestion Notification"? > > * For ECN-capable packets, shift the job of hiding bursts from network to > > host: > > - the network signals ECN with no smoothing delay > > - then the transport can hide bursts of ECN signals from itself > > But can we get there from here? > > The node doing the ECN notification _can't_ know how the transport >will react; and the transport receiving and ECN notification can't know >whether the forwarding node has "smoothed" the signal. (It is truly a >shame we haven't left any bits for signals like this!) Well, we do have ECT(1) still only assigned experimentally and never used, which we could decide to use for this immediate ECN. However, first I want to see whether people think it might be feasible to just redefine the meaning of CE. Rationale: So few buffers have ECN support turned on anyway that we should be able to redefine ECN so that many more will want to turn it on. For those AQMs that already support ECN, we believe this retrospective change will make them only a little worse than they are already (and the operator can update them by simple reconfiguration anyway, and is more likely to do so, given these are clearly early-adopter networks). > > - the transport knows > > o whether it's TCP or RTP etc, > > o whether its in congestion avoidance or slow-start, > > o and it knows its RTT, > > o so it can know whether to respond immediately or to smooth the > > signals, > > o and if so, over what time > > Yes, but it can't know what smoothing may already have been applied. Yes. If this is a problem, we will have to consider using ECT(1) not CE. But it's pretty academic when so few buffers support ECN. The tiny proportion that do support ECN will already smooth by a 'typical RTT' of about 100ms. If a 20ms RTT flow adds smoothing over its own RTT to this, it will be smooth over 120ms. The main problem there is not the extra 20ms, it's the original 100ms, which we won't lose unless we make this change somehow. > > - then short RTT flows can smooth the signals with only the delay > > of their /own/ RTT > > o so they can fill troughs and absorb peaks that longer RTT > flows cannot > > - a TCP only needs to smooth the signals if in congestion avoidance > > o in slow start, it can respond immediately, thus reducing overshoot > > This would, IMHO, improve "slow start". > > > Incremental Deployment: > > * Immediate congestion notification doesn't need new AQM implementation > > - it can use the widely implemented WRED algorithm with an > > unexpected configuration > > Bob is beginning to lose me here. Does he mean that a forwarding node >would apply WRED for both drop and ECN, but with different parameters? > > > * The network classifies packets for this AQM treatment based on > > their ECN-capability > > - Without ECN, it smoothes the queue before signalling drops > > Bob has lost me now -- apparently he doesn't mean different >parameters... and I don't recognize this "smoothing" step in WRED. I do mean that a forwarding node would apply WRED for both drop and ECN, but with different parameters. Each WRED policy-map includes a setting for this smoothing parameter, which Cisco calls the exponential-weighting-constant. Many people don't notice it's there and they just leave it at the default. For instance, Cisco set it to 2^(-9) ~ 0.002 by default for each of the WRED policy-maps (see http://www.cisco.com/en/US/docs/ios/12_0s/feature/guide/fswfq26.html#wp1039982). > > - With ECN, it signals immediately, without any smoothing delay > > - (as today, the operator can still use WRED with the Diffserv field too) > > (Do we need to confuse this discussion by adding diffserv?) A non-Diffserv network still doesn't need to worry about Diffserv. I put this in parentheses because, if WRED is used today, it is usually used with Diffserv, and I didn't want anyone to worry that they wouldn't be able to continue to do this (e.g. BT use WRED with Diffserv in enterprise networks, as do many other carriers). > > * For TCP apps, the stack will use 'DCTCP' (we've tweaked it), if the > > ends negotiate ECN with the accurate feedback capability. > > Have we settled on "accurate feedback" already? I thought that was >still under discussion. (I don't follow exactly what it adds...) See response from Richard Scheffenegger. Essentially the TCPM WG has accepted the requirements doc, but not decided between the mechanisms on offer. > > * It should 'just work' if an RTP app or a Reno TCP uses ECN. > > I don't see any way for a Reno transport using ECN to avoid being >starved if ECN arrives earlier (without notice). We haven't tested legacy Reno with ECN yet (we figured legacy Reno without ECN is a lot more prevalent, so focused on this first). Nonetheless, Reno-ECN is unlikely to starve, because starvation is about long-running behaviour, and once a flow has run for more than a couple of 100ms RTTs, the immediate ECN signals should be no different from a smoothed ECN. I suspect Reno-ECN might be worse in its short-term dynamics. But remember Reno-ECN is likely to be a tiny corner-case. > > The request: > > * Much more evaluation to do, but first we want to know: > > - if the idea works, would the IETF have an appetite for tweaking > > the definition of ECN so it is merely equivalent to drop in the long > > term, but the dynamics need not be equivalent. > > There's a good question there; but I don't think we're ready for it. At this stage, even we haven't got many answers. So I'm not asking the IETF to answer the question right now. I'm merely saying, /if/ our idea works, is there at least an /appetite/ in the IETF for reconsidering the definition of ECN? We wanted to make the IETF aware of this research early, because it might want to at least hold off on any actions that would otherwise close off this option. And if we find that any change is completely out of the question, we have to try a different tack (e.g. ECT(1)). > I'd really like to discuss the dynamics of responding more quickly >but perhaps less drastically for almost any real-time flow. > > But proving "equivalence in the long term" seems too hard. This should be the easy part, because the longer that conditions are stable, a smoothed signal should tend towards an unsmoothed signal, all other factors being equal. Equivalence during dynamics is the hard part, and I'm suggesting we don't sweat too much about that, as long as the performance evaluations are not too far apart. > > Much better than the ECN that didn't get deployed > > * This is Explicit and Immediate Congestion Notification (EICN?) > > - same wire protocol, much greater benefits > > * The advantage of the original ECN (avoiding congestive loss) was > > too small to be worth the deployment hassle > > Actually, I don't agree that was the problem -- instead I believe >the code has been deployed but administratively suppressed because >the operators don't trust the transports. There _is_ a significant >improvement from one-RTT reaction instead of several (to detect a >drop), but the whole process is just too complicated, while the >opportunity for abuse remains obvious. I agree. That's the 'deployment hassle' side of my sentence - the extra trust-enhancing mechanisms that seemed necessary were too much pain for the small gain. > > * Predictable ultra-low latency without loss too (similar to > > DCTCP-ECN) would be worth deploying > > I'm optimistic that latency will become an easier argument. > > > * But we all thought DCTCP could only be deployed in isolation (e.g. > > data centres) > > - we all thought DCTCP traffic would starve alongside today's TCP traffic > > - because in a DCTCP queue, the ECN threshold is lower than you > > would trigger drop > > - and we thought ECN & drop had to be equivalent. > > (I'm not sure we'll succeed at breaking that "equivalence"...) > > > * We believe we've found a way to ensure DCTCP-ECN traffic doesn't starve > > - we still make DCTCP-ECN equivalent to drop in the long-run, but > > not in its dynamics > > (I'm still not sure it's worth arguing the "long-run".) I mean competing long-running ECN & non-ECN flows stabilise at predictable rates, rather than one ratchetting itself down to nothing over time (starvation). That's the primary concern of congestion control 'fairness', before anyone starts worrying about what the relative rates are. Given apps get different relative rates with different RTTs, with different size objects or by opening multiple flows, we don't need to sweat so much about precisely equal flow rates; but we must sweat about stable convergence. Results so far show that the proposed idea is at least very robust against starvation. Bob >-- >John Leslie ________________________________________________________________ Bob Briscoe, BT From dave.taht@gmail.com Sun Oct 27 21:22:32 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8EE9D11E814B; Sun, 27 Oct 2013 21:22:32 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.97 X-Spam-Level: X-Spam-Status: No, score=-1.97 tagged_above=-999 required=5 tests=[AWL=0.403, BAYES_00=-2.599, NO_RELAYS=-0.001, SARE_SUB_OBFU_Q1=0.227] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ut-lDglbL0Em; Sun, 27 Oct 2013 21:22:30 -0700 (PDT) Received: from mail-we0-x232.google.com (mail-we0-x232.google.com [IPv6:2a00:1450:400c:c03::232]) by ietfa.amsl.com (Postfix) with ESMTP id A8B5611E80F2; Sun, 27 Oct 2013 21:22:29 -0700 (PDT) Received: by mail-we0-f178.google.com with SMTP id q59so5924944wes.23 for ; Sun, 27 Oct 2013 21:22:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=RITAgogiFx/sU7NzueW0IH7j4Ape3mjwuyKPuD1JUTU=; b=vf5htJyzkc7DvZxEy21VcQRJfkrqXB2KkSMOOV6BmVXeBY4pDK8e1qmgqOlV1bFQuh u6NW4Bv9GPyuu1D2exk9ib7+sNxecDd8PKnsnISxGZNq3SN+KCqC6GGOhTLHzf2ynAzo t2o6En80n8VdiiP3thJxgxArc/JjUyNDtsxcXfz/PdF6oLSR6w3MQcY/hkYCYU7aqd1P P2H1s+s9K5Hl1Vul3RhDUMlc4LKCmO/MsKunwklQ/t3SpPrPlDeBIBmlErHlu8OHLbam eIZnOlln7YprfCIdmQS13qBD/PnIE+o9MTO7ruToVsVln1qlJvUz7PKoYHBpoclT8lZN ujZg== MIME-Version: 1.0 X-Received: by 10.180.13.113 with SMTP id g17mr7337187wic.19.1382934148568; Sun, 27 Oct 2013 21:22:28 -0700 (PDT) Received: by 10.217.67.202 with HTTP; Sun, 27 Oct 2013 21:22:28 -0700 (PDT) In-Reply-To: <201310280136.r9S1afJ0018887@bagheera.jungle.bt.co.uk> References: <201309171233.r8HCXKkV003952@bagheera.jungle.bt.co.uk> <201310212340.r9LNee5p025162@bagheera.jungle.bt.co.uk> <657D1484-1B02-41F5-9FD1-0921439DF754@isoc.org> <201310252116.r9PLGAFp007348@bagheera.jungle.bt.co.uk> <20131026121632.GJ65952@verdi> <201310280136.r9S1afJ0018887@bagheera.jungle.bt.co.uk> Date: Sun, 27 Oct 2013 21:22:28 -0700 Message-ID: From: Dave Taht To: Bob Briscoe Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: tsvwg IETF list , John Leslie , "aqm@ietf.org" Subject: Re: [aqm] [tsvwg] Immediate and Explicit Congestion Notification X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Oct 2013 04:22:32 -0000 As is probably well known at this point, I make a clear distinction between networking problems in the data center and ones in the wild and wooly world. The data center portion of the universe is a couple hundred meters in diameter, the other, I dunno, let's say 8x from here to the moon (3,075,200,000m). There are all sorts of things that work and are needed in the data center that probably won't work outside it. People run straighter cables, do microwave, use pause frames at layer 2, etc, etc, in order to wring out the last nanosecond of performance, and certainly running without loss is important in that world. On Sun, Oct 27, 2013 at 6:36 PM, Bob Briscoe wrote: > John, inline... > > At 12:16 26/10/2013, John Leslie wrote: >> >> Bob Briscoe wrote: >> > >> > Exec summary >> > * Early tests show promise that we may have found a way to make the I'm failing to get excited lacking paper, testable code, and other proofs. >> > ultra-low queuing delay of data centre TCP incrementally deployable >> > on the public Internet At the moment the edge of the internet is using things like cable and gpon which have 2ms or more of inherent latency built into their grant/response structures. >> > * For rtcweb, we need to address >> > a) cc for r-t media [rmcat w-g in progress] >> > b) Making TCP nicer >> > c) minimise ability of TCP to bloat queues [AQM w-g now in progress] >> > This addresses b) & c) I do strongly feel that webrtc needs good aqm and packet scheduling in order to succeed, and that the webrtc folk should be testing out what has already been developed (red, sfqred, codel, fq_codel, pie) and the aqm folk testing the webrtc code. I look forward to some interesting conference calls. I gave the webrtc codebases a whirl against what aqm and packet scheduling techniques we have already this past summer, the initial results were encouraging, but I was able to easily crash most of the browsers sooner than I could take a good, repeatable set of measurements. That said, things over there are moving along smartly. >> > >> > The problem >> > * All AQMs delay dropping for about one (hard-coded) worst-case RTT, >> > in case a burst dissipates (allegedly a 'good queue' according to Van >> > Jacobson) >> >> This assertion is going to need a lot of support. >> >> Bob is a man after my own heart suggesting that an ECN notification >> may be sent earlier than a packet drop would be indicated. I don't know >> if we can get there; but IMHO that is essential to getting ECN deployed >> and used. >> >> I don't think I agree with Bob that what's hard-coded is necessarily >> a "worst-case" RTT -- and I'm quite sure I'm not willing to make any >> pronouncement about "all AQMs". >> >> I suggest the talk might be more useful if Bob outlined the AQMs >> currently in widespread use and detailed _how_ they delay dropping >> for an estimated RTT. > > > You're right. The 'about' in my sentence was meant to indicate some leewa= y. > The specifics depend on each AQM... > > The only AQM I know of that doesn't smooth over some nominal RTT is DCTCP > itself. DCTCP fits uncomfortably into the aqm catagory. > > * CoDel was designed for 'interval' to be a worst-case (largest) RTT, whi= ch > it recommends to be set to 100ms. One the queue has exceeded threshold, Which is 5ms of delay. > CoDel delays for time 'interval' before starting to signal congestion. Which it then decreases until it finds something approximating the RTT. For a system in a reasonably steady state, it will find a decent value and stay there. For bursty traffic it is not necessarily helpful, and slow start is interesting... > - I already said how sluggish CoDel would be for a flow with a much shor= ter But you've never published a measurement, unless there is one in your new p= aper? > RTT than CoDel (others have made this point, e.g. for data centres : > https://lists.bufferbloat.net/pipermail/codel/2012-August/000448.html). You might want to read the full thread... a stumbling block on fixing the srtt was the stochastic hashing... so now there is a full blown non-approximated fq scheduler that implements pacing. http://www.ietf.org/mail-archive/web/aqm/current/msg00259.html And I look forward to a certain upcoming presentation in ICCRG with great anticipation as to additional fallout from this.... > - And in the other direction, we already know that utilisation suffers > fairly badly for flows with RTT significantly larger than 100ms. "target" and "interval" have always been variables in the fq_codel and codel codebase and ecn is supported. tc qdisc add dev your_device root fq_codel target 500us interval 10ms ecn I don't think the pain of adding that one line of configuration would affect a data center deployment any. I would certainly like to see benchmarks in a data center environment. I personally lack 10Gig hw. I note that once you get below 1ms on a typical box today, you start hitting other bottlenecks in (for example) the cpu scheduler. > * PIE suppresses all drops for time max_burst (set to 100ms by default) f= rom > when the drop probability it calculates (but doesn't necessarily use) fir= st > rises above zero. This is very similar to CoDel, and similar comments are > applicable. This too has a variable target value, although there are several other magic constants that I don't fully understand. I would certainly like to see some benchmarks in a data center environment. The pie code I have for linux shoots for a target of 20ms by default, for some reason. > * RED requires the constant for its exponentially weighted moving average > (w_q) to be set taking into account how many packets are likely to arrive= at > the link in a 'typical' RTT. Reverse engineering the values recommended b= y > Sally Floyd in the RED paper and in her famous RED parameters Web page > , she recommended a 'typical= ' > RTT of about 130ms. > > [BTW, I know of people who don't calculate w_q, but just use the value of > "0.002" that Sally recommended for her 45Mb/s link in the original RED pa= per > simulations (and repeated at ). > This was calculated assuming about 500 packets arrive at a link (from all > flows) in a typical RTT. Links have got a lot faster since 1993. > Nonetheless, she was considering 45Mb/s for an aggregated link in those > days, and it happens to be about right for a single user today.] > > >> > * For a flow with 1/10 or 1/100 of this RTT (e.g. from a CDN or your >> > home media server), any congestion signal is delayed tens or hundreds >> > of its own RTTs by these AQMs. Congestion takes time to occur. Consider tcp dynamics as flows also only increase relative to RTT, as well. A core question is what do you want response time to congestion to be under= ? 100ms? 5ms? 1ms? 500us? >> Clearly, RTTs differing by a factor of ten are quite common at most >> nodes traversed in a typical path; and it seems _very_ suboptimal to 2.4 billion folk connect to the internet generally at RTTs between 20 and 8= 0ms. In a data center environment I have no data. >> have the responsibility for guessing the RTT at the node which must >> drop packets. I certainly favor the ongoing development of end to end cc capable of monitoring the RTT and doing the right thing. > For packets that do not support ECN, the dropping node has to make a gues= s > at the RTT, so as not to drop packets unnecessarily, because drop is an > impairment as well as a congestion signal. So a transport cannot 'undrop' > packets. > > Our point though is that a network node doesn't have to mimic this behavi= our > for ECN packets, because ECN is not an impairment. So a transport can > un-ECN-mark packets (by smoothing out bursts itself). > > >> > * A TCP flow in slow-start doesn't need the burst smoothed anyway >> > - delaying the signal just makes slow-start overshoot more >> > - a TCP in slow-start knows that it won't allow the burst to >> > dissipate anyway The end of slow start is an ECN notification or a packet drop. What am I missing here? >> A critical point! (It seems obvious to me, but is it obvious to >> everyone?) >> >> > The solution: make ECN also mean "Immediate Congestion Notification"? >> > * For ECN-capable packets, shift the job of hiding bursts from network >> > to >> > host: >> > - the network signals ECN with no smoothing delay >> > - then the transport can hide bursts of ECN signals from itself >> >> But can we get there from here? >> >> The node doing the ECN notification _can't_ know how the transport >> will react; and the transport receiving and ECN notification can't know >> whether the forwarding node has "smoothed" the signal. (It is truly a >> shame we haven't left any bits for signals like this!) > > > Well, we do have ECT(1) still only assigned experimentally and never used= , > which we could decide to use for this immediate ECN. However, first I wan= t > to see whether people think it might be feasible to just redefine the > meaning of CE. A lot of the NONCE logic has been discussed in other rfcs. > > Rationale: So few buffers have ECN support turned on anyway that we shoul= d > be able to redefine ECN so that many more will want to turn it on. > > For those AQMs that already support ECN, we believe this retrospective > change will make them only a little worse than they are already (and the > operator can update them by simple reconfiguration anyway, and is more > likely to do so, given these are clearly early-adopter networks). > > >> > - the transport knows >> > o whether it's TCP or RTP etc, >> > o whether its in congestion avoidance or slow-start, >> > o and it knows its RTT, >> > o so it can know whether to respond immediately or to smooth the >> > signals, >> > o and if so, over what time >> >> Yes, but it can't know what smoothing may already have been applied. > > > Yes. If this is a problem, we will have to consider using ECT(1) not CE. > But it's pretty academic when so few buffers support ECN. > > The tiny proportion that do support ECN will already smooth by a 'typical > RTT' of about 100ms. > > If a 20ms RTT flow adds smoothing over its own RTT to this, it will be > smooth over 120ms. > The main problem there is not the extra 20ms, it's the original 100ms, wh= ich > we won't lose unless we make this change somehow. > > >> > - then short RTT flows can smooth the signals with only the delay >> > of their /own/ RTT >> > o so they can fill troughs and absorb peaks that longer RTT flows >> > cannot >> > - a TCP only needs to smooth the signals if in congestion avoidance >> > o in slow start, it can respond immediately, thus reducing oversho= ot >> >> This would, IMHO, improve "slow start". >> >> > Incremental Deployment: >> > * Immediate congestion notification doesn't need new AQM implementatio= n >> > - it can use the widely implemented WRED algorithm with an >> > unexpected configuration >> >> Bob is beginning to lose me here. Does he mean that a forwarding node >> would apply WRED for both drop and ECN, but with different parameters? >> >> > * The network classifies packets for this AQM treatment based on >> > their ECN-capability >> > - Without ECN, it smoothes the queue before signalling drops >> >> Bob has lost me now -- apparently he doesn't mean different >> parameters... and I don't recognize this "smoothing" step in WRED. > > > I do mean that a forwarding node would apply WRED for both drop and ECN, = but > with different parameters. > > Each WRED policy-map includes a setting for this smoothing parameter, whi= ch > Cisco calls the exponential-weighting-constant. Many people don't notice > it's there and they just leave it at the default. For instance, Cisco set= it > to > 2^(-9) ~ 0.002 by default for each of the WRED policy-maps (see > http://www.cisco.com/en/US/docs/ios/12_0s/feature/guide/fswfq26.html#wp10= 39982). > > >> > - With ECN, it signals immediately, without any smoothing delay >> > - (as today, the operator can still use WRED with the Diffserv field >> > too) >> >> (Do we need to confuse this discussion by adding diffserv?) > > > A non-Diffserv network still doesn't need to worry about Diffserv. > > I put this in parentheses because, if WRED is used today, it is usually u= sed > with Diffserv, and I didn't want anyone to worry that they wouldn't be ab= le > to continue to do this (e.g. BT use WRED with Diffserv in enterprise > networks, as do many other carriers). Perhaps we need a taxonomy so we can all talk to areas of the network we care about? RED as currently defined will not work correctly on variable rate networks, like wireless... I have never thought we'd end up with one aqm or tcp to rule them all.. DC: Data Center WI: Wireless FL: Fixed Line > >> > * For TCP apps, the stack will use 'DCTCP' (we've tweaked it), if the >> > ends negotiate ECN with the accurate feedback capability. >> >> Have we settled on "accurate feedback" already? I thought that was >> still under discussion. (I don't follow exactly what it adds...) > > > See response from Richard Scheffenegger. Essentially the TCPM WG has > accepted the requirements doc, but not decided between the mechanisms on > offer. > > >> > * It should 'just work' if an RTP app or a Reno TCP uses ECN. >> >> I don't see any way for a Reno transport using ECN to avoid being >> starved if ECN arrives earlier (without notice). > > > We haven't tested legacy Reno with ECN yet (we figured legacy Reno withou= t > ECN is a lot more prevalent, so focused on this first). Nonetheless, > Reno-ECN is unlikely to starve, because starvation is about long-running > behaviour, and once a flow has run for more than a couple of 100ms RTTs, = the > immediate ECN signals should be no different from a smoothed ECN. I suspe= ct > Reno-ECN might be worse in its short-term dynamics. But remember Reno-ECN= is > likely to be a tiny corner-case. > > >> > The request: >> > * Much more evaluation to do, but first we want to know: >> > - if the idea works, would the IETF have an appetite for tweaking >> > the definition of ECN so it is merely equivalent to drop in the long >> > term, but the dynamics need not be equivalent. >> >> There's a good question there; but I don't think we're ready for it. > > > At this stage, even we haven't got many answers. So I'm not asking the IE= TF > to answer the question right now. I'm merely saying, /if/ our idea works,= is > there at least an /appetite/ in the IETF for reconsidering the definition= of > ECN? I wouldn't mind it, but frankly my own concern was addressing the security issue, not the current definition. > We wanted to make the IETF aware of this research early, because it might > want to at least hold off on any actions that would otherwise close off t= his > option. > > And if we find that any change is completely out of the question, we have= to > try a different tack (e.g. ECT(1)). > > >> I'd really like to discuss the dynamics of responding more quickly >> but perhaps less drastically for almost any real-time flow. >> >> But proving "equivalence in the long term" seems too hard. > > > This should be the easy part, because the longer that conditions are stab= le, > a smoothed signal should tend towards an unsmoothed signal, all other > factors being equal. > > Equivalence during dynamics is the hard part, and I'm suggesting we don't > sweat too much about that, as long as the performance evaluations are not > too far apart. > > >> > Much better than the ECN that didn't get deployed >> > * This is Explicit and Immediate Congestion Notification (EICN?) >> > - same wire protocol, much greater benefits >> > * The advantage of the original ECN (avoiding congestive loss) was >> > too small to be worth the deployment hassle >> >> Actually, I don't agree that was the problem -- instead I believe >> the code has been deployed but administratively suppressed because >> the operators don't trust the transports. There _is_ a significant >> improvement from one-RTT reaction instead of several (to detect a >> drop), but the whole process is just too complicated, while the >> opportunity for abuse remains obvious. > > > I agree. That's the 'deployment hassle' side of my sentence - the extra > trust-enhancing mechanisms that seemed necessary were too much pain for t= he > small gain. I > > >> > * Predictable ultra-low latency without loss too (similar to >> > DCTCP-ECN) would be worth deploying >> >> I'm optimistic that latency will become an easier argument. >> >> > * But we all thought DCTCP could only be deployed in isolation (e.g. >> > data centres) >> > - we all thought DCTCP traffic would starve alongside today's TCP >> > traffic >> > - because in a DCTCP queue, the ECN threshold is lower than you >> > would trigger drop >> > - and we thought ECN & drop had to be equivalent. I'm looking forward to being convinced otherwise. >> (I'm not sure we'll succeed at breaking that "equivalence"...) >> >> > * We believe we've found a way to ensure DCTCP-ECN traffic doesn't >> > starve >> > - we still make DCTCP-ECN equivalent to drop in the long-run, but >> > not in its dynamics >> >> (I'm still not sure it's worth arguing the "long-run".) > > > I mean competing long-running ECN & non-ECN flows stabilise at predictabl= e > rates, rather than one ratchetting itself down to nothing over time > (starvation). > > That's the primary concern of congestion control 'fairness', before anyon= e > starts worrying about what the relative rates are. Given apps get differe= nt > relative rates with different RTTs, with different size objects or by > opening multiple flows, we don't need to sweat so much about precisely eq= ual > flow rates; but we must sweat about stable convergence. > > Results so far show that the proposed idea is at least very robust agains= t > starvation. > > > Bob > > >> -- >> John Leslie > > > ________________________________________________________________ > Bob Briscoe, BT > _______________________________________________ > aqm mailing list > aqm@ietf.org > https://www.ietf.org/mailman/listinfo/aqm --=20 Dave T=E4ht Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.= html From bob.briscoe@bt.com Mon Oct 28 00:17:17 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1127411E813D; Mon, 28 Oct 2013 00:17:17 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.638 X-Spam-Level: X-Spam-Status: No, score=-2.638 tagged_above=-999 required=5 tests=[AWL=-0.662, BAYES_00=-2.599, MIME_QP_LONG_LINE=1.396, RCVD_IN_DNSWL_LOW=-1, SARE_SUB_OBFU_Q1=0.227] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id occOPEMkaL9R; Mon, 28 Oct 2013 00:17:10 -0700 (PDT) Received: from hubrelay-by-04.bt.com (hubrelay-by-04.bt.com [62.7.242.140]) by ietfa.amsl.com (Postfix) with ESMTP id B7A2A11E81BF; Mon, 28 Oct 2013 00:17:06 -0700 (PDT) Received: from EVMHR72-UKRD.domain1.systemhost.net (10.36.3.110) by EVMHR04-UKBR.bt.com (10.216.161.36) with Microsoft SMTP Server (TLS) id 8.3.297.1; Mon, 28 Oct 2013 07:17:04 +0000 Received: from EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) by EVMHR72-UKRD.domain1.systemhost.net (10.36.3.110) with Microsoft SMTP Server (TLS) id 8.3.279.1; Mon, 28 Oct 2013 07:17:04 +0000 Received: from bagheera.jungle.bt.co.uk (132.146.168.158) by EPHR01-UKIP.domain1.systemhost.net (147.149.196.177) with Microsoft SMTP Server id 14.2.347.0; Mon, 28 Oct 2013 07:16:54 +0000 Received: from BTP075694.jungle.bt.co.uk ([10.109.144.88]) by bagheera.jungle.bt.co.uk (8.13.5/8.12.8) with ESMTP id r9S7GoZf020045; Mon, 28 Oct 2013 07:16:51 GMT Message-ID: <201310280716.r9S7GoZf020045@bagheera.jungle.bt.co.uk> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Mon, 28 Oct 2013 07:16:49 +0000 To: Dave Taht From: Bob Briscoe In-Reply-To: References: <201309171233.r8HCXKkV003952@bagheera.jungle.bt.co.uk> <201310212340.r9LNee5p025162@bagheera.jungle.bt.co.uk> <657D1484-1B02-41F5-9FD1-0921439DF754@isoc.org> <201310252116.r9PLGAFp007348@bagheera.jungle.bt.co.uk> <20131026121632.GJ65952@verdi> <201310280136.r9S1afJ0018887@bagheera.jungle.bt.co.uk> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1"; format=flowed Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.56 on 132.146.168.158 Cc: tsvwg IETF list , John Leslie , "aqm@ietf.org" Subject: Re: [aqm] [tsvwg] Immediate and Explicit Congestion Notification X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Oct 2013 07:17:20 -0000 Dave, At 04:22 28/10/2013, Dave Taht wrote: >As is probably well known at this point, I make a clear distinction >between networking problems in the data center and ones in the wild >and wooly world. The two are converging. It will gradually become=20 less sensible to think of the two as separate. >The data center portion of the universe is a couple hundred meters in >diameter, the other, I dunno, let's say 8x from here to the moon >(3,075,200,000m). DCTCP was fixed ages ago for global RTTs.=20 Download the Linux version from Stanford. >There are all sorts of things that work and are needed in the data >center that probably won't work outside it. People run straighter >cables, do microwave, use pause frames at layer 2, etc, etc, in order >to wring out the last nanosecond of performance, and certainly running >without loss is important in that world. Not relevant. Don't be confused by the name=20 DCTCP. It's only a few lines of code different=20 from TCP, and it's not only relevant in data=20 centres. It's for low queuing delay. It's only confined to a data centre while we=20 can't work out how it co-exists with current=20 Internet traffic - that's what we're doing. >On Sun, Oct 27, 2013 at 6:36 PM, Bob Briscoe wrote: > > John, inline... > > > > At 12:16 26/10/2013, John Leslie wrote: > >> > >> Bob Briscoe wrote: > >> > > >> > Exec summary > >> > * Early tests show promise that we may have found a way to make the > >I'm failing to get excited lacking paper, testable code, and other proofs. I'll send the paper, but agreed we've only just=20 got started, so you don't have to be excited yet. A lot more testing to do. > >> > ultra-low queuing delay of data centre TCP incrementally deployable > >> > on the public Internet > >At the moment the edge of the internet is using things like cable and >gpon which have 2ms or more of inherent latency built into their >grant/response structures. You're saying you're not interested in low=20 queuing delay solutions because there's often=20 fluff in badly designed access networks anyway?=20 That sounds like a poor excuse for not aspiring=20 to lower delay. What excuse is there for the=20 100ms of signalling delay on ECN packets in CoDel=20 then? That's 50 times more than this 2ms=20 (currently) inherent access network delay (that=20 some of us are working to remove as well). If you put an AQM in a host that is the=20 bottleneck for a transfer to your media server in=20 the next room of your house, at the same time as=20 transfers to the wide area, you really don't want=20 to introduce a nominal global RTT into both feedback loops. > >> > * For rtcweb, we need to address > >> > a) cc for r-t media [rmcat w-g in progress] > >> > b) Making TCP nicer > >> > c) minimise ability of TCP to bloat queues [AQM w-g now in= progress] > >> > This addresses b) & c) > >I do strongly feel that webrtc needs good aqm and packet scheduling in >order to succeed, and that the webrtc folk should be testing out what >has already been developed (red, sfqred, codel, fq_codel, pie) and the >aqm folk testing the webrtc code. The AQM for DCTCP was developed 2 years before=20 all of those (except RED). Why decide to build on=20 CoDel in 2012, when DCTCP had already been out=20 there and deployed for 2 years with excellent=20 results? And tcp-rcv-cheat had been out there for 5 years. >I look forward to some interesting conference calls. > >I gave the webrtc codebases a whirl against what aqm and packet >scheduling techniques we have already this past summer, the initial >results were encouraging, but I was able to easily crash most of the >browsers sooner than I could take a good, repeatable set of >measurements. That said, things over there are moving along smartly. > > >> > > >> > The problem > >> > * All AQMs delay dropping for about one (hard-coded) worst-case RTT, > >> > in case a burst dissipates (allegedly a 'good queue' according to Van > >> > Jacobson) > >> > >> This assertion is going to need a lot of support. > >> > >> Bob is a man after my own heart suggesting that an ECN notification > >> may be sent earlier than a packet drop would be indicated. I don't know > >> if we can get there; but IMHO that is essential to getting ECN deployed > >> and used. > >> > >> I don't think I agree with Bob that what's hard-coded is necessarily > >> a "worst-case" RTT -- and I'm quite sure I'm not willing to make any > >> pronouncement about "all AQMs". > >> > >> I suggest the talk might be more useful if Bob outlined the AQMs > >> currently in widespread use and detailed _how_ they delay dropping > >> for an estimated RTT. > > > > > > You're right. The 'about' in my sentence was meant to indicate some= leeway. > > The specifics depend on each AQM... > > > > The only AQM I know of that doesn't smooth over some nominal RTT is= DCTCP > > itself. > >DCTCP fits uncomfortably into the aqm catagory. Oh dear. Is this Dave Taht saying "I don't like=20 it cos it's different to tradition"? DCTCP shows just how low you can get queuing=20 delay even with a brain-dead simple step AQM as=20 long as you fix the right problem (smooth in TCP,=20 not in the AQM). Doesn't this tell you something=20 about where the problem probably was? > > > > * CoDel was designed for 'interval' to be a worst-case (largest) RTT,= which > > it recommends to be set to 100ms. One the queue has exceeded threshold, > >Which is 5ms of delay. 105ms of delay on top of RTT delay before it can possibly do anything. By far the most common case on an access link is=20 a flow arriving when the link's empty. > > CoDel delays for time 'interval' before starting to signal congestion. > >Which it then decreases until it finds something approximating the >RTT. For a system in a reasonably steady state, it will find a decent >value and stay there. Until the queue empties (which it should if the=20 control system is working), then it has to start=20 over, because another flow may have a different RTT. >For bursty traffic it is not necessarily helpful, and slow start is >interesting... > > > - I already said how sluggish CoDel would be=20 > for a flow with a much shorter > >But you've never published a measurement, unless=20 >there is one in your new paper? Lots of experiments in the paper, but not the=20 dynamics yet - we deliberately focused on the=20 long-running behaviour first, to check the critical starvation issues first. > > RTT than CoDel (others have made this point, e.g. for data centres : > > https://lists.bufferbloat.net/pipermail/codel/2012-August/000448.html). > >You might want to read the full thread... a stumbling block on fixing >the srtt was the stochastic hashing... so now there is a full blown >non-approximated fq scheduler that implements pacing. > >http://www.ietf.org/mail-archive/web/aqm/current/msg00259.html As I understand it, this only works for an AQM on=20 a host that has the benefit of being told the RTT from higher up the stack. >And I look forward to a certain upcoming presentation in ICCRG with >great anticipation as to additional fallout from this.... > > > - And in the other direction, we already know that utilisation suffers > > fairly badly for flows with RTT significantly larger than 100ms. > >"target" and "interval" have always been variables in the fq_codel and >codel codebase It's not use having variables you don't know how=20 to set. That was the lesson from RED that DCTCP=20 started to solve (concerning variable RTT), and=20 CoDel continued to solve (concerning line=20 variable rate). However, unfortunately CoDel chose the not invented here= route. >and ecn is supported. > >tc qdisc add dev your_device root fq_codel target 500us interval 10ms ecn AFAICT, ECN is only supported in CoDel in the=20 sense that it is treated as equivalent to drop.=20 The factors where ECN is not equivalent to drop have not been catered for. And ECN certainly hasn't been considered as=20 something to exploit beyond what drop can do. For=20 instance, to take the approach I'm suggesting,=20 but using CoDel not RED, for ECN-capable packets=20 you would use interval as zero. Then the=20 end-system would smooth out bursts of ECN marks instead of CoDel doing it. >I don't think the pain of adding that one line of configuration would >affect a data center deployment any. I would certainly like to see >benchmarks in a data center environment. I personally lack 10Gig hw. > >I note that once you get below 1ms on a typical box today, you start >hitting other bottlenecks in (for example) the cpu scheduler. As above, not a good reason. We have to knock=20 down each source of delay, one after the other. > > * PIE suppresses all drops for time max_burst=20 > (set to 100ms by default) from > > when the drop probability it calculates (but doesn't necessarily use)= first > > rises above zero. This is very similar to CoDel, and similar comments= are > > applicable. > >This too has a variable target value, although there are several other >magic constants that I don't fully understand. It takes a few minutes of reading to understand=20 all the PIE variables (about the same complexity=20 as CoDel). I think PIE has the following constants (defaults in []): * target_del (the target queuing delay [20ms] as you say) * max_burst (has a similar role to interval in CoDel [100ms]) * deq_threshold (there's enough packets in the=20 queue to measure the line-rate [10KB]) * Tupdate, which determines how often the line=20 rate is tested (which shouldn't affect=20 performance unless the machine can't cope with a high enough update rate). All the other variables are dependents of these,=20 or autotuned (in the case of alpha and beta). >I would certainly like >to see some benchmarks in a data center environment. Just to be clear, for this conversation I'm not=20 focused on a data centre environment - I'm making=20 DCTCP applicable to the public Internet. >The pie code I >have for linux shoots for a target of 20ms by default, for some >reason. > > > * RED requires the constant for its exponentially weighted moving= average > > (w_q) to be set taking into account how many=20 > packets are likely to arrive at > > the link in a 'typical' RTT. Reverse engineering the values recommended= by > > Sally Floyd in the RED paper and in her famous RED parameters Web page > > , she recommended a= 'typical' > > RTT of about 130ms. > > > > [BTW, I know of people who don't calculate w_q, but just use the value= of > > "0.002" that Sally recommended for her 45Mb/s=20 > link in the original RED paper > > simulations (and repeated at=20 > ). > > This was calculated assuming about 500 packets arrive at a link (from= all > > flows) in a typical RTT. Links have got a lot faster since 1993. > > Nonetheless, she was considering 45Mb/s for an aggregated link in those > > days, and it happens to be about right for a single user today.] > > > > > >> > * For a flow with 1/10 or 1/100 of this RTT (e.g. from a CDN or your > >> > home media server), any congestion signal is delayed tens or hundreds > >> > of its own RTTs by these AQMs. > >Congestion takes time to occur. Consider tcp dynamics as flows also >only increase relative to RTT, as well. Er no. As a queue builds, it causes delay.=20 Regular TCP makes this worse than it has to be by=20 using large saw-teeth, so they become a large=20 inherent variable source of delay. VJ calls this=20 a good queue. But if a variant of TCP can remove=20 this variable source of delay, it can't be good. >A core question is what do you want response time to congestion to be= under? > >100ms? 5ms? 1ms? 500us? Wrong question. Answer depends on RTT. That's the=20 whole point. Consider that you may have been brain-washed. > >> Clearly, RTTs differing by a factor of ten are quite common at most > >> nodes traversed in a typical path; and it seems _very_ suboptimal to > >2.4 billion folk connect to the internet=20 >generally at RTTs between 20 and 80ms. The Internet includes your home network,=20 potentially a cache in your home gateway, I/O=20 lanes within your own machine, etc etc. No-one=20 "connects to the Internet at an RTT of" anything.=20 Today, all these 2.4B folk use connections over=20 the Internet with a very wide range of RTTs. >In a data center environment I have no data. Typically 100us or 200us base RTT. And all that=20 cheap data centre kit can be and will be re-used in the public Internet too. > >> have the responsibility for guessing the RTT at the node which must > >> drop packets. > >I certainly favor the ongoing development of end to end cc capable of >monitoring the RTT and doing the right thing. Good. > > For packets that do not support ECN, the dropping node has to make a= guess > > at the RTT, so as not to drop packets unnecessarily, because drop is an > > impairment as well as a congestion signal. So a transport cannot= 'undrop' > > packets. > > > > Our point though is that a network node=20 > doesn't have to mimic this behaviour > > for ECN packets, because ECN is not an impairment. So a transport can > > un-ECN-mark packets (by smoothing out bursts itself). > > > > > >> > * A TCP flow in slow-start doesn't need the burst smoothed anyway > >> > - delaying the signal just makes slow-start overshoot more > >> > - a TCP in slow-start knows that it won't allow the burst to > >> > dissipate anyway > >The end of slow start is an ECN notification or a packet drop. What am >I missing here? A traditional AQM (RED, CoDel, PIE) suppresses=20 any signal (ECN or drop) for another 100ms or so=20 after slow start has pushed the queue over the=20 delay threshold. That's on top of the feedback=20 delay of 1RTT, because it doesn't know this is=20 slow-start, so it waits to see if the queue goes away. The transport knows whether it's in slow-start,=20 so it doesn't need to do any smoothing here - it=20 can drop straight out of slow-start without smoothing. The transport knows when it's in congestion=20 avoidance too, of course, then it can smooth out=20 bursts of ECN by doing its own EWMA over the=20 actual RTT (that it knows properly). Ie the AQM can't be selective about when it=20 smooths bursts, because it doesn't know what=20 phase each transport flow is in, but each transport does and therefore can. > >> A critical point! (It seems obvious to me, but is it obvious to > >> everyone?) > > >> > >> > The solution: make ECN also mean "Immediate Congestion Notification"? > >> > * For ECN-capable packets, shift the job of hiding bursts from= network > >> > to > >> > host: > >> > - the network signals ECN with no smoothing delay > >> > - then the transport can hide bursts of ECN signals from itself > >> > >> But can we get there from here? > >> > >> The node doing the ECN notification _can't_ know how the transport > >> will react; and the transport receiving and ECN notification can't know > >> whether the forwarding node has "smoothed" the signal. (It is truly a > >> shame we haven't left any bits for signals like this!) > > > > > > Well, we do have ECT(1) still only assigned experimentally and never= used, > > which we could decide to use for this immediate ECN. However, first I= want > > to see whether people think it might be feasible to just redefine the > > meaning of CE. > >A lot of the NONCE logic has been discussed in other rfcs. > > > > > Rationale: So few buffers have ECN support turned on anyway that we= should > > be able to redefine ECN so that many more will want to turn it on. > > > > For those AQMs that already support ECN, we believe this retrospective > > change will make them only a little worse than they are already (and the > > operator can update them by simple reconfiguration anyway, and is more > > likely to do so, given these are clearly early-adopter networks). > > > > > >> > - the transport knows > >> > o whether it's TCP or RTP etc, > >> > o whether its in congestion avoidance or slow-start, > >> > o and it knows its RTT, > >> > o so it can know whether to respond immediately or to smooth the > >> > signals, > >> > o and if so, over what time > >> > >> Yes, but it can't know what smoothing may already have been applied. > > > > > > Yes. If this is a problem, we will have to consider using ECT(1) not CE. > > But it's pretty academic when so few buffers support ECN. > > > > The tiny proportion that do support ECN will already smooth by a= 'typical > > RTT' of about 100ms. > > > > If a 20ms RTT flow adds smoothing over its own RTT to this, it will be > > smooth over 120ms. > > The main problem there is not the extra 20ms,=20 > it's the original 100ms, which > > we won't lose unless we make this change somehow. > > > > > >> > - then short RTT flows can smooth the signals with only the delay > >> > of their /own/ RTT > >> > o so they can fill troughs and absorb peaks that longer RTT flows > >> > cannot > >> > - a TCP only needs to smooth the signals if in congestion avoidance > >> > o in slow start, it can respond immediately, thus reducing= overshoot > >> > >> This would, IMHO, improve "slow start". > >> > >> > Incremental Deployment: > >> > * Immediate congestion notification doesn't need new AQM= implementation > >> > - it can use the widely implemented WRED algorithm with an > >> > unexpected configuration > >> > >> Bob is beginning to lose me here. Does he mean that a forwarding= node > >> would apply WRED for both drop and ECN, but with different parameters? > >> > >> > * The network classifies packets for this AQM treatment based on > >> > their ECN-capability > >> > - Without ECN, it smoothes the queue before signalling drops > >> > >> Bob has lost me now -- apparently he doesn't mean different > >> parameters... and I don't recognize this "smoothing" step in WRED. > > > > > > I do mean that a forwarding node would apply=20 > WRED for both drop and ECN, but > > with different parameters. > > > > Each WRED policy-map includes a setting for this smoothing parameter,= which > > Cisco calls the exponential-weighting-constant. Many people don't notice > > it's there and they just leave it at the=20 > default. For instance, Cisco set it > > to > > 2^(-9) ~ 0.002 by default for each of the WRED policy-maps (see > >=20 >= http://www.cisco.com/en/US/docs/ios/12_0s/feature/guide/fswfq26.html#wp1039= 982). > > > > > >> > - With ECN, it signals immediately, without any smoothing delay > >> > - (as today, the operator can still use WRED with the Diffserv= field > >> > too) > >> > >> (Do we need to confuse this discussion by adding diffserv?) > > > > > > A non-Diffserv network still doesn't need to worry about Diffserv. > > > > I put this in parentheses because, if WRED is=20 > used today, it is usually used > > with Diffserv, and I didn't want anyone to worry that they wouldn't be= able > > to continue to do this (e.g. BT use WRED with Diffserv in enterprise > > networks, as do many other carriers). > >Perhaps we need a taxonomy so we can all talk to areas of the network >we care about? RED as currently defined will not work correctly on >variable rate networks, like wireless... > >I have never thought we'd end up with one aqm or tcp to rule them all.. > >DC: Data Center >WI: Wireless >FL: Fixed Line Again, this immediate ECN stuff is for public=20 Internet (not just data centres). > > > >> > * For TCP apps, the stack will use 'DCTCP' (we've tweaked it), if the > >> > ends negotiate ECN with the accurate feedback capability. > >> > >> Have we settled on "accurate feedback" already? I thought that was > >> still under discussion. (I don't follow exactly what it adds...) > > > > > > See response from Richard Scheffenegger. Essentially the TCPM WG has > > accepted the requirements doc, but not decided between the mechanisms on > > offer. > > > > > >> > * It should 'just work' if an RTP app or a Reno TCP uses ECN. > >> > >> I don't see any way for a Reno transport using ECN to avoid being > >> starved if ECN arrives earlier (without notice). > > > > > > We haven't tested legacy Reno with ECN yet (we figured legacy Reno= without > > ECN is a lot more prevalent, so focused on this first). Nonetheless, > > Reno-ECN is unlikely to starve, because starvation is about long-running > > behaviour, and once a flow has run for more=20 > than a couple of 100ms RTTs, the > > immediate ECN signals should be no different from a smoothed ECN. I= suspect > > Reno-ECN might be worse in its short-term=20 > dynamics. But remember Reno-ECN is > > likely to be a tiny corner-case. > > > > > >> > The request: > >> > * Much more evaluation to do, but first we want to know: > >> > - if the idea works, would the IETF have an appetite for tweaking > >> > the definition of ECN so it is merely equivalent to drop in the long > >> > term, but the dynamics need not be equivalent. > >> > >> There's a good question there; but I don't think we're ready for it. > > > > > > At this stage, even we haven't got many answers. So I'm not asking the= IETF > > to answer the question right now. I'm merely=20 > saying, /if/ our idea works, is > > there at least an /appetite/ in the IETF for=20 > reconsidering the definition of > > ECN? > >I wouldn't mind it, but frankly my own concern was addressing the >security issue, not the current definition. It seems strange that you weren't aware of solutions to this then. > > We wanted to make the IETF aware of this research early, because it= might > > want to at least hold off on any actions that=20 > would otherwise close off this > > option. > > > > > > And if we find that any change is completely=20 > out of the question, we have to > > try a different tack (e.g. ECT(1)). > > > > > >> I'd really like to discuss the dynamics of responding more quickly > >> but perhaps less drastically for almost any real-time flow. > >> > >> But proving "equivalence in the long term" seems too hard. > > > > > > This should be the easy part, because the=20 > longer that conditions are stable, > > a smoothed signal should tend towards an unsmoothed signal, all other > > factors being equal. > > > > Equivalence during dynamics is the hard part, and I'm suggesting we= don't > > sweat too much about that, as long as the performance evaluations are= not > > too far apart. > > > > > >> > Much better than the ECN that didn't get deployed > >> > * This is Explicit and Immediate Congestion Notification (EICN?) > >> > - same wire protocol, much greater benefits > >> > * The advantage of the original ECN (avoiding congestive loss) was > >> > too small to be worth the deployment hassle > >> > >> Actually, I don't agree that was the problem -- instead I believe > >> the code has been deployed but administratively suppressed because > >> the operators don't trust the transports. There _is_ a significant > >> improvement from one-RTT reaction instead of several (to detect a > >> drop), but the whole process is just too complicated, while the > >> opportunity for abuse remains obvious. > > > > > > I agree. That's the 'deployment hassle' side of my sentence - the extra > > trust-enhancing mechanisms that seemed necessary were too much pain for= the > > small gain. > >I > > > > > > >> > * Predictable ultra-low latency without loss too (similar to > >> > DCTCP-ECN) would be worth deploying > >> > >> I'm optimistic that latency will become an easier argument. > >> > >> > * But we all thought DCTCP could only be deployed in isolation (e.g. > >> > data centres) > >> > - we all thought DCTCP traffic would starve alongside today's TCP > >> > traffic > >> > - because in a DCTCP queue, the ECN threshold is lower than you > >> > would trigger drop > >> > - and we thought ECN & drop had to be equivalent. > >I'm looking forward to being convinced otherwise. Yup, I don't want to take up too much of anyone's=20 time with this until we've proved it. But we=20 decided a heads up was important. That's all. Bob > >> (I'm not sure we'll succeed at breaking that "equivalence"...) > >> > >> > * We believe we've found a way to ensure DCTCP-ECN traffic doesn't > >> > starve > >> > - we still make DCTCP-ECN equivalent to drop in the long-run, but > >> > not in its dynamics > >> > >> (I'm still not sure it's worth arguing the "long-run".) > > > > > > I mean competing long-running ECN & non-ECN flows stabilise at= predictable > > rates, rather than one ratchetting itself down to nothing over time > > (starvation). > > > > That's the primary concern of congestion control 'fairness', before= anyone > > starts worrying about what the relative rates are. Given apps get= different > > relative rates with different RTTs, with different size objects or by > > opening multiple flows, we don't need to=20 > sweat so much about precisely equal > > flow rates; but we must sweat about stable convergence. > > > > Results so far show that the proposed idea is at least very robust= against > > starvation. > > > > > > Bob > > > > > >> -- > >> John Leslie > > > > > > ________________________________________________________________ > > Bob Briscoe, BT > > _______________________________________________ > > aqm mailing list > > aqm@ietf.org > > https://www.ietf.org/mailman/listinfo/aqm > > > >-- >Dave T=E4ht > >Fixing bufferbloat with cerowrt:=20 >http://www.teklibre.com/cerowrt/subscribe.html ________________________________________________________________ Bob Briscoe, BT=20 From sk.anirudh@gmail.com Mon Oct 28 21:07:34 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E41C311E820C for ; Mon, 28 Oct 2013 21:07:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.6 X-Spam-Level: X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, NO_RELAYS=-0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tRBsBeSTKYm4 for ; Mon, 28 Oct 2013 21:07:34 -0700 (PDT) Received: from mail-vb0-x22d.google.com (mail-vb0-x22d.google.com [IPv6:2607:f8b0:400c:c02::22d]) by ietfa.amsl.com (Postfix) with ESMTP id 495D311E80DC for ; Mon, 28 Oct 2013 21:07:34 -0700 (PDT) Received: by mail-vb0-f45.google.com with SMTP id p6so4150327vbe.32 for ; Mon, 28 Oct 2013 21:07:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:cc:content-type; bh=PSztR23yiQPOn2FkiDsJWe+/LRgyUADoKhE6kH11gBg=; b=Mvwa2o2Kwam0G5sFdVwMejvIZpRCQdwX5s6hr8Pu4ePM1KdzlXhQ5EowXqjCT4kU5X X4Z52O2OL47LOflsWLEo0FBKzvh8Sq7EwRxYHYJCjEi+MpgGhgo/NHJ1k6ntk2lYe590 CurKhNf3JWNYlVKvqTiqQlgOP/s4C3F8/18ayGjlbrFUP8WF6U39D+8nyU6/ZNYrxpVu cc/cTAP7OkzjudguKrcRZb2ir/0qlRca25zv/SiChEfsVTdy5a3aCK+Azs3v2IXgNfe1 U3TIcjLzpsrHvYtY/2k5Fn54HAPvrZh2FpQuVbEjCoSVOreU08F2QYqqg8Mrm6Yw/nCm zGsQ== MIME-Version: 1.0 X-Received: by 10.58.19.195 with SMTP id h3mr8696vee.48.1383019651778; Mon, 28 Oct 2013 21:07:31 -0700 (PDT) Received: by 10.52.165.204 with HTTP; Mon, 28 Oct 2013 21:07:31 -0700 (PDT) Date: Tue, 29 Oct 2013 00:07:31 -0400 Message-ID: From: Anirudh Sivaraman To: aqm@ietf.org Content-Type: text/plain; charset=UTF-8 Cc: Hari Balakrishnan , Suvinay Subramanian , Keith Winstein Subject: [aqm] Work on absence of a universal queue management / scheduling scheme X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Oct 2013 04:07:35 -0000 Hello, I would be grateful for the list's feedback re: our new paper that attempts to show that there is no universal AQM or scheduling scheme, and proposes extending SDN to the data plane to allow a switch to implement different scheduling and queue management algorithms, even after being installed. The paper, along with instructions to reproduce its results, is available here: http://web.mit.edu/anirudh/www/sdn-data-plane.html We compared three queue-management schemes on a bottleneck gateway in simulation: CoDel running on a single queue (as described in "man tc-codel"), CoDel with per-flow queues (fq_codel, as described in "man tc-fq_codel"), and per-flow queueing with long DropTail queues. It turns out that depending on the objectives desired by the traffic running across the gateway, any one of these schemes can be better than any other -- i.e. A>B>C>A and A>C>B>A. In other words, we don't think there is likely to be a 'best' AQM scheme Our proposal to address the absence of a universal in-network configuration is a switch data plane that's flexible enough to implement new scheduling and queue management schemes. To be clear, this work is merely a position paper at this point, and needs a lot more thought before such a design is feasible at line rates on switches with an aggregate capacity reaching 1 Terabit/sec. We are presenting this next month at HotNets, so we are grateful for any feedback from the list. Anirudh From wes@mti-systems.com Tue Oct 29 07:08:35 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AEC7511E815F for ; Tue, 29 Oct 2013 07:08:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.5 X-Spam-Level: X-Spam-Status: No, score=-0.5 tagged_above=-999 required=5 tests=[AWL=-0.501, BAYES_50=0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id v7Yl5X-sGJt3 for ; Tue, 29 Oct 2013 07:08:31 -0700 (PDT) Received: from atl4mhob10.myregisteredsite.com (atl4mhob10.myregisteredsite.com [209.17.115.48]) by ietfa.amsl.com (Postfix) with ESMTP id 54E1921F9D9C for ; Tue, 29 Oct 2013 07:08:31 -0700 (PDT) Received: from mailpod.hostingplatform.com ([10.30.71.205]) by atl4mhob10.myregisteredsite.com (8.14.4/8.14.4) with ESMTP id r9TE8Uv5000429 for ; Tue, 29 Oct 2013 10:08:30 -0400 Received: (qmail 24983 invoked by uid 0); 29 Oct 2013 14:08:30 -0000 X-TCPREMOTEIP: 107.45.98.20 X-Authenticated-UID: wes@mti-systems.com Received: from unknown (HELO ?192.168.43.65?) (wes@mti-systems.com@107.45.98.20) by 0 with ESMTPA; 29 Oct 2013 14:08:30 -0000 Message-ID: <526FC15B.3020004@mti-systems.com> Date: Tue, 29 Oct 2013 10:08:27 -0400 From: Wesley Eddy Organization: MTI Systems User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.0.1 MIME-Version: 1.0 To: "aqm@ietf.org" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: [aqm] remote participation for IETF88 X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Oct 2013 14:08:36 -0000 This is just a reminder that for remote participation in the AQM meetings at IETF 88, you can access the streaming mp3 and the jabber rooms via the links for AQM at: http://tools.ietf.org/agenda/88/ MP3 stream links (based on current room assignments: Tuesday: http://ietf88streaming.dnsalias.net/ietf/ietf882.m3u Friday: http://ietf88streaming.dnsalias.net/ietf/ietf886.m3u Jabber room link: xmpp:aqm@jabber.ietf.org?join -- Wes Eddy MTI Systems From g.white@CableLabs.com Wed Oct 30 13:13:35 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7CE9821E8097 for ; Wed, 30 Oct 2013 13:13:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.235 X-Spam-Level: X-Spam-Status: No, score=-0.235 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_MODEMCABLE=0.768, HOST_EQ_MODEMCABLE=1.368, HTML_MESSAGE=0.001, SARE_SUB_OBFU_Q1=0.227] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vyU8-cDSGMTc for ; Wed, 30 Oct 2013 13:13:31 -0700 (PDT) Received: from ondar.cablelabs.com (ondar.cablelabs.com [192.160.73.61]) by ietfa.amsl.com (Postfix) with ESMTP id 8987721E8063 for ; Wed, 30 Oct 2013 13:13:26 -0700 (PDT) Received: from kyzyl.cablelabs.com (kyzyl [10.253.0.7]) by ondar.cablelabs.com (8.14.7/8.14.7) with ESMTP id r9UKDOjA021076; Wed, 30 Oct 2013 14:13:24 -0600 Received: from exchange.cablelabs.com (10.5.0.19) by kyzyl.cablelabs.com (F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com); Wed, 30 Oct 2013 14:13:24 -0600 (MDT) X-Virus-Status: clean(F-Secure/fsigk_smtp/407/kyzyl.cablelabs.com) Received: from EXCHANGE.cablelabs.com ([fe80::797a:96d1:3c53:18ee]) by EXCHANGE.cablelabs.com ([fe80::797a:96d1:3c53:18ee%11]) with mapi id 14.03.0146.000; Wed, 30 Oct 2013 14:13:24 -0600 From: Greg White To: bloat , "aqm@ietf.org" Thread-Topic: DOCSIS 3.1 support for AQM Thread-Index: AQHO1ax8goSjG7StR0y5LQDUVuujHw== Date: Wed, 30 Oct 2013 20:13:23 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.3.8.130913 x-originating-ip: [10.5.0.27] Content-Type: multipart/alternative; boundary="_000_CE96C48220DEBgwhitecablelabscom_" MIME-Version: 1.0 X-Approved: ondar Subject: [aqm] DOCSIS 3.1 support for AQM X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Oct 2013 20:13:35 -0000 --_000_CE96C48220DEBgwhitecablelabscom_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable FYI, the DOCSIS 3.1 specifications were released yesterday. New in D3.1 i= s mandatory support for AQM (with default =3D on). Cable modems are required to implement a variant of PIE (detailed in Annex = M of MAC and Upper Layer Protocols Interface Specification) as the default AQM= . Modem vendors are free to implement additional algorithms if they so cho= ose. CMTSs are required to implement an AQM meeting certain high-level cri= teria, but the choice of algorithm is left to the implementer. -Greg --_000_CE96C48220DEBgwhitecablelabscom_ Content-Type: text/html; charset="us-ascii" Content-ID: <3300672A0EFF894F92DA8AF406B41647@cablelabs.com> Content-Transfer-Encoding: quoted-printable

FYI, the DOCSIS 3.1 specifications were released yesterday.   New= in D3.1 is mandatory support for AQM (with default =3D on).

Cable modems are required to implement a variant of PIE (detailed in A= nnex M of MAC and Upper Layer Protocols Interface Specificatio= n) as the default AQM.  Modem vendors are free to implement additional algorithms if they so choose.  CMTSs= are required to implement an AQM meeting certain high-level criteria, but = the choice of algorithm is left to the implementer.

-Greg
--_000_CE96C48220DEBgwhitecablelabscom_-- From roland.bless@kit.edu Thu Oct 31 01:25:47 2013 Return-Path: X-Original-To: aqm@ietfa.amsl.com Delivered-To: aqm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E887B21E80C3 for ; Thu, 31 Oct 2013 01:25:46 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.199 X-Spam-Level: X-Spam-Status: No, score=-6.199 tagged_above=-999 required=5 tests=[AWL=-0.177, BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_MED=-4, SARE_SUB_OBFU_Q1=0.227] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HlJffwjn2DjJ for ; Thu, 31 Oct 2013 01:25:39 -0700 (PDT) Received: from iramx2.ira.uni-karlsruhe.de (iramx2.ira.uni-karlsruhe.de [141.3.10.81]) by ietfa.amsl.com (Postfix) with ESMTP id 5481C21E80AE for ; Thu, 31 Oct 2013 01:25:33 -0700 (PDT) Received: from irams1.ira.uni-karlsruhe.de ([141.3.10.5]) by iramx2.ira.uni-karlsruhe.de with esmtps port 25 id 1VbnZ2-0008I6-DE; Thu, 31 Oct 2013 09:25:18 +0100 Received: from i72vorta.tm.uni-karlsruhe.de ([141.3.71.26] helo=vorta.tm.kit.edu) by irams1.ira.uni-karlsruhe.de with esmtp port 25 id 1VbnZF-0005UF-En; Thu, 31 Oct 2013 09:25:25 +0100 Received: from [IPv6:::1] (ip6-localhost [IPv6:::1]) by vorta.tm.kit.edu (Postfix) with ESMTPS id 57035A804E0; Thu, 31 Oct 2013 09:25:25 +0100 (CET) Message-ID: <527213F5.6060406@kit.edu> Date: Thu, 31 Oct 2013 09:25:25 +0100 From: "Bless, Roland (TM)" Organization: Institute of Telematics, Karlsruhe Institute of Technology User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.0 MIME-Version: 1.0 To: Greg White , bloat , "aqm@ietf.org" References: In-Reply-To: X-Enigmail-Version: 1.5.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-ATIS-AV: ClamAV (irams1.ira.uni-karlsruhe.de) X-ATIS-AV: ClamAV (iramx2.ira.uni-karlsruhe.de) X-ATIS-Timestamp: iramx2.ira.uni-karlsruhe.de 1383207918. Subject: Re: [aqm] DOCSIS 3.1 support for AQM X-BeenThere: aqm@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "Discussion list for active queue management and flow isolation." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Oct 2013 08:25:47 -0000 Hi Greg, Am 30.10.2013 21:13, schrieb Greg White: > Cable modems are required to implement a variant of PIE (detailed in > Annex M of MAC and Upper Layer Protocols Interface Specification > ) as > the default AQM. Modem vendors are free to implement additional > algorithms if they so choose. CMTSs are required to implement an AQM > meeting certain high-level criteria, but the choice of algorithm is left > to the implementer. Thanks for the information. I'd be interested in why you have chosen PIE, e.g., instead of sfq-CoDel. Any pointers to evaluation reports/results? Last time I saw a presentation on this it seemed that CoDel was performing quite well. Regards, Roland