Subject: RE: Section 2.4, item (f) of draft-ietf-ipngwg-icmp-v3-04.txt

From: [email-masked]
Date: Thu, 5 Aug 2004 19:17:30 -0500

Alex,

Thanks for pointing this out.  What do you think about the following
as the replacement text?

=============
        The rate-limiting parameters SHOULD be configurable.  In the
        case of a token-bucket implementation, the best defaults depend
        on the computational power of the device and the speed/bandwidth 
        of the interfaces that it has (e.g., a high-end router vs. an 
        embedded host, a 10G ethernet vs. a T1 interface).  For example, 
        in a small/mid -sized device with limited bandwidth interfaces, 
        the possible defaults could be B=10, N=10/s.
==============

Regards
Mukesh

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]On Behalf Of
> ext Alex Conta
> Sent: Thursday, August 05, 2004 11:17 AM
> To: IPv6
> Subject: Section 2.4, item (f) of draft-ietf-ipngwg-icmp-v3-04.txt
> 
> 
> The last paragraph of Section 2.4, item (f) of 
> draft-ietf-ipngwg-icmp-v3-04.txt is pointing to ICMP rate limiting 
> configurability on a per node basis.
> 
> ICMP Rate-limiting configurability makes sense also on a per 
> interface 
> basis.
> 
> In a case of a router having different speed/bandwidth 
> interfaces, for 
> example IPv6 interfaces on fast links, as well as  slower links, the 
> adequate ICMP rate-limiting parameters on a 1G or 10G 
> Ethernet, could be 
> quite different from a T1, or T3 link.
> 
> Regards,
> Alex
> 

--------------------------------------------------------------------
IETF IPv6 working group mailing list
[email-masked]
Administrative Requests: https://www1.ietf.org/mailman/listinfo/ipv6
--------------------------------------------------------------------

---

From: Alex Conta <[email-masked]>
Date: Thu, 05 Aug 2004 23:55:14 -0400

Mukesh,

A first sentence like:

"The rate-limiting parameters SHOULD be configurable per node,
if the node has similar speed/bandwidth interfaces, and/or per 
interface, if the node has disimilar speed/bandwidth interfaces".

or

"The rate-limiting parameters SHOULD be configurable per node,
if the node has similar interfaces, and/or per interface, if the node 
has disimilar speed/bandwidth interfaces".

would be more explicit/descriptive.

The rest is fine.

Thanks,
Alex

---

From: Pekka Savola <[email-masked]>
Date: Fri, 6 Aug 2004 18:50:35 +0300 (EEST)

> interface, if the node has disimilar speed/bandwidth interfaces".

I'm slightly concerned whether we have two implementations of 
interface-specific rate-limiting parameters, required for Draft 
Standard.  Do you know of any?

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: Alex Conta <[email-masked]>
Date: Mon, 09 Aug 2004 13:20:57 -0400

Are there any implementations were ICMP rate limiting is implemented 
separately from other protocol packets traffic management (traffic 
shaping)?

If an IP implementation has support for traffic management, the packet 
schedulers (traffic shaping) associated with interfaces, handle ICMP 
rate limiting per interface (in terms of both configuring and rate 
limiting) similarly to other protocol packets traffic shaping per 
interface.

Regards,
Alex

---

From: Pekka Savola <[email-masked]>
Date: Mon, 9 Aug 2004 21:26:30 +0300 (EEST)

> separately from other protocol packets traffic management (traffic 

Certainly.  For example Linux and BSD only rate-limit ICMP.  Many OSes
in fact don't have any limiting for other kind of traffic :).

---

From: Brian Haley <[email-masked]>
Date: Mon, 09 Aug 2004 14:39:47 -0400

> separately from other protocol packets traffic management (traffic 

HP has at least one UNIX that does ICMPv6 and Mobility Header in common 
code, Linux does ICMPv4/v6 together on a per-route basis.

> limiting) similarly to other protocol packets traffic shaping per 

Not true in the cases I cited.

-Brian

> ------------------------------------------------------------------------
>

---

From: Alex Conta <[email-masked]>
Date: Mon, 09 Aug 2004 17:08:31 -0400

Brian, Peka,

There is no doubt that setting ICMP rate limiting per node in a router 
with both slow and fast interfaces to accomodate one interface may be 
very detrimental to the other: imagine T1 and 1Gbit Ethernet interfaces; 
1% of a T1s 1.5Mbit/sec is 15kbit/sec, which is 0.00000015% of a 
1Gbit/sec, and 1% of a 1Gbit/Sec is 10Mbit/sec, way over 1.5Mbit/sec.

That being said, I am sorry that my message was badly phrased, and so it 
misguided your answers. Nevermind!

Essentially, I was looking for something like the references I am 
mentioning below. These references provide examples of rate limiting 
ICMP per interface in LINUX systems, as well as CISCO routers, 
supposedly in relationship to traffic managament.

I think these references answer Peka's original concern of 2 
implementations that support ICMP rate control per intereface:

***Number 1:
On the LINUX.ORG WEB site - more exactly:
 
http://www.linux.org/docs/ldp/howto/Adv-Routing-HOWTO/lartc.cookbook.icmp-ratelimit.html

refers to the "Linux Advanced Routing & Traffic Control HOWTO Cookbook, 
chapter 15", also available at:

http://www.tldp.org/HOWTO/Adv-Routing-HOWTO/lartc.cookbook.icmp-ratelimit.html

in which it is shown how to limit in LUNIX, ICMP rate on a Ethernet 
interface:

----excerpt
We first set up the prerequisite parts:

  # tc qdisc add dev eth0 root handle 10: cbq bandwidth 10Mbit avpkt 1000
  # tc class add dev eth0 parent 10:0 classid 10:1 cbq bandwidth 10Mbit 
rate \
   10Mbit allot 1514 prio 5 maxburst 20 avpkt 1000

Now we need a filter to assign ICMP traffic to this class:

# tc filter add dev eth0 parent 10:0 protocol ip prio 100 u32 match ip
   protocol 1 0xFF flowid 10:100

----end of excerpt

***Number 2

On the WEB site:

http://www.cisco.com/en/US/products/hw/modules/ps2710/products_data_sheet09186a008015cfee.html

the "Cisco 12000 Series 4-Port Gigabit Ethernet ISE Line Card Features 
and Functions " refers to

"ICMP rate limiting on port and VLAN level".

****Number 3

on

http://www.cisco.com/en/US/tech/tk583/tk385/technologies_white_paper09186a0080174a5b.shtml

there is a reference on how to use CAR to rate limit ICMP packets in the 
following example:

----excerpt
interface xy
  rate-limit output access-group 2020 3000000 512000 786000 conform-action
transmit exceed-action drop

access-list 2020 permit icmp any any echo-reply

------end of excerpt

****Number 4

http://www.zdnet.com.au/news/security/print.htm?TYPE=story&AT=20262795-2000061744t-10000005c

Provides another Cisco IOS example:

----------excerpt
     config t
     Access-list 100 permit icmp any {your network} {your subnet} echo-reply
     Access-list 100 permit icmp any (your Network) (your Subnet) echo
     Interface e1
     Rate-limit input access-group 100 512000 8000 8000 conform action 
transmit exceed action drop

-------end of excerpt

---

From: Pekka Savola <[email-masked]>
Date: Tue, 10 Aug 2004 09:01:14 +0300 (EEST)

> 1Gbit/sec, and 1% of a 1Gbit/Sec is 10Mbit/sec, way over 1.5Mbit/sec.

You may have an assumption that the rate-limiting would have to be a
percentage of the interface speed.  That's (IMHO) a bad strategy,
exactly why you describe: it doesn't handle fast/slow interfaces
appropriately. (However, you could limit the upper bound for token
bucket based on the interface speed, I guess.)

The point we tried to make in the ICMP spec revision is that using a
token-bucket rate-limiter is a good idea, because even if you have a
refill rate of even so low as 10 packets/second (or the like), that
should always be sufficient even on faster links, but not cause
significant problems on slower links.

> supposedly in relationship to traffic managament.

OK.  (I snipped the examples.)  These are the ways to perform manual
rate-limiting (in more general).  For the two platforms I mentioned,
for ICMP only, it's automatically enabled and controllable through
net.ipvX.icmp_ratelimit sysctl's.  For xxxBSD, that's
net.inet.icmp.icmplim.  By default, they start rate-limiting at 100 or
200 pps AFAIR.

---

From: Lowell Gilbert <[email-masked]>
Date: 09 Aug 2004 15:56:34 -0400

> separately from other protocol packets traffic management (traffic

Sure, but I'm not sure that's important.

> limiting) similarly to other protocol packets traffic shaping per

Several systems have something like this, and then *also* have a
separate mechanism for rate-limiting ICMP.  Often, that separate
mechanism on applies to *outgoing* ICMP replies.

The language in question is already more verbose than does any good:
the "and/or" negates any requirements that are intended by the
distinction between having interfaces at similar speeds or not.  I
think the requirement would be the same if it just said the parameters
"should be configurable per node and/or per interface," with perhaps a
*suggestion* recommending per-interface configuration, particularly on
systems that are capable of supporting different interface types or
speeds.  [This is not quite the same as what I would actually have
recommended the document say to begin with; I'm only trying to comment
on the language at the moment.]

Be well.

---

From: Alex Conta <[email-masked]>
Date: Mon, 16 Aug 2004 11:19:14 -0400

> The language in question is already more verbose than does any good:

I don't mind less text.

> distinction between having interfaces at similar speeds or not.

The per node, and per interface mechanisms can coexist.

> speeds.  [...]

My concern is clarity.

"The rate-limiting parameters SHOULD be configurable per node.
and/or per interface if the node has dissimilar speed/bandwidth interfaces.

is not clear enough.

"The rate-limiting parameters SHOULD be configurable per node.
If a node has dissimilar speed/bandwidth interfaces, the rate-limiting 
parameters SHOULD be configurable per interface.

is clearer.

Regards,
Alex

---

From: Fred Templin <[email-masked]>
Date: Mon, 16 Aug 2004 09:35:01 -0700 (PDT)

Alex,

> parameters SHOULD be configurable per interface.

To be even clearer still, perhaps there should be some mention
of what is meant by "interface". In particular, it seems that
rate limiting could be either per-physical interface (like,
one rate limiting parameter per multi-access Ethernet link)
or per-logical interface (like, one rate limiting parameter
per pt-to-pt tunnel).

In the former case, a single rate limiting parameter could
produce too few ICMPs for some neighbors but too many for
others. In the latter case, rate limiting parameters are
per-neighbor and provide the correct levels of ICMPs. But,
the physical/logical interface distinction should be
clearly spelled out.

Thanks - Fred
[email-masked]

---

From: Alex Conta <[email-masked]>
Date: Mon, 16 Aug 2004 13:40:03 -0400

Fred,

Good point regarding ICMP rate-control at logical (or pseudo) interface 
level, a typical example being tunnel interfaces.

I hope Mukesh takes note.

Thanks,
Alex

---

From: Pekka Savola <[email-masked]>
Date: Mon, 16 Aug 2004 21:36:55 +0300 (EEST)

> The per node, and per interface mechanisms can coexist.

Sure.  But why don't we just specify per node mechanisms, and leave it
up to the implementors if they wish to have per interface mechanisms
as well?

> parameters SHOULD be configurable per interface.

I must remain in opposition for this.  Speed/bandwidth based
approaches are not recommended in any case; unless you do them
per-interface, they're almost always totally useless -- which is a
good reason not to use them (as the draft recommends), but use the
token-based limiter instead.

Token bucket-based limiter is just simply much better.  It works well 
with lower speed interfaces (not being too restrictive), with high 
speed interfaces (not allowing too much), and can be implemented on a 
per-node basis.

---

From: Alex Conta <[email-masked]>
Date: Mon, 16 Aug 2004 15:09:37 -0400

Please see in line:

Pekka Savola wrote:

> appropriately. 

This is a misinterpretation. Essentially I am saying that identical 
rate-limiting parameters (N,B in draft terms) on slow/fast interfaces 
would not handle ICMP traffic appropriately.

> (However, you could limit the upper bound for token

Rephrasing your statement using draft parameters terminology, that is, 
N=average rate of transmission, and B=upper bound of transmission,
you're agreeing that Bx for a slow interface "X" should be different 
than Bz for a fast interface "Z".

This is good enough in support of the text suggested for the draft.

> [...]

Regards,
Alex

---

From: Fred Templin <[email-masked]>
Date: Mon, 16 Aug 2004 12:22:13 -0700 (PDT)

> token-based limiter instead.

Implementing per-interface is in fact what Alex is suggesting,
and by: "unless you do them per-interface" above it appears
you agree that this would be useful.

If the rate limiting is done per logical (pt-to-pt) interface,
then adaptation is specific to the bandwidth characteristics of
the encapsulating tunnel endpoint and we can do no better than
this. With any other approach, some traffic sources will not
receive adequate ICMPs while others will be overwhelmed.

Fred
[email-masked]

---

From: Pekka Savola <[email-masked]>
Date: Mon, 16 Aug 2004 22:58:25 +0300 (EEST)

> would not handle ICMP traffic appropriately.

Depends on what you want to achieve with the rate-limiting.

If you want to make sure that ICMP responses don't eat up all of a
very slow link (e.g., 64 kbit/s), a single parameter might not be
sufficient, yes.

But my point is that we shouldn't be overly concerned about this.  
That kind of very slow interfaces will need some kind of queuing, 
rate-limiting, etc. mechanisms to be usable in any case. *)

The main point is to create a reasonable upper bound for the amount of
ICMP packets generated, so that you don't generate an ICMP error out
of potentially every packet.

But please stop for a moment to consider the example default token
bucket values: N=10: that would result in the maximum of 4 kbit/s of
generated ICMP traffic (double this when bursting).  Even if you used
N=100 (which would be sufficient even with 10 Gbit/s interfaces), you
would have the maximum the maximum of 40 Kbit/s of ICMP traffic.

Is this *REALLY* too much message generation?  Something that makes it
worth to *specify* interface-specific values, make implementations
check the interfaces where the packets would be destined to go out to,
implement some logic to use for virtual interfaces, etc.?  Doesn't
look like that to me.  Seems like unnecessary complexity to me.

If you really, really think this is needed, and the WG agrees with 
that, I'd suggest we put interface-specific values as a MAY -- 
definitely not a SHOULD.

> than Bz for a fast interface "Z".

No, I don't really agree with that.  I think B shouldn't need to be 
varied based on the interface speed.  Just picking a sufficiently 
large value, e.g., 10, 20, 50 or 100 should allow sufficient amount of 
ICMP generation on fast interfaces, while still not generating too 
much of it with slow interfaces.

Please remember that to be usable, very slow interfaces need more
generic rate-limiters as well (compare to your reference to particular
implementations e.g., Linux, earlier on this thread).  That's
orthogonal to this limiter.  Hence, we shouldn't need to make a
"perfect" ICMP generation limiter, which would be sufficient on its
own even in the fringe cases, but just a "good enough" ICMP generation
limiter. (See the paragraph above marked with *).

---

From: Fred Templin <[email-masked]>
Date: Mon, 16 Aug 2004 14:21:10 -0700

Pekka,

In the case of logical (pt-to-pt) interfaces, actual per-source traffic
measurements may be readily available. In such cases, why would
an implementation use an aggregate estimator when it can easily
compute actual per-source values for rate limiting??

I vote for the "SHOULD" option for per-interface rate limiting
when per-source traffic measurements are available.

Fred
[email-masked]

---

From: Pekka Savola <[email-masked]>
Date: Tue, 17 Aug 2004 07:41:05 +0300 (EEST)

> when per-source traffic measurements are available.

Are there any implementations doing this or interface-specific
limiting?  None that I know of, but please enlighten us if so.  I
don't think having such in a draft standard is acceptable.

On the other hand, node-based token bucket limiters are commonplace, 
and work without any perceived problems AFAICS even behind slow/fast 
interfaces, behind tunnels, etc. -- why do we need to say more than 
that?

---

From: Fred Templin <[email-masked]>
Date: Tue, 17 Aug 2004 08:50:31 -0700

Pekka,

You are side-stepping my question and twisting it into
a matter of what is currently done vs. what is best.

What is our goal here: to get IPv6 deployed in the right
way, or to preserve the "legacy" deployed IPv6 base?

Fred
[email-masked]

---

From: Alex Conta <[email-masked]>
Date: Tue, 17 Aug 2004 11:55:18 -0400

> Depends on what you want to achieve with the rate-limiting.

I pointed you to the LINUX and CISCO examples. I expect to achieve
nothing different than the LINUX and CISCO implementers/users expect.

The one sentence text suggested for the ICMP specs suggesting a higher 
threshold for a good router ICMP implementation, similar to LINUX or 
CISCO helps the implementers. This means also consistency, since brings 
the router ICMP implementation on the same threshold as the host ICMP 
implementation.

> of potentially every packet.

Rate limiting is well understood. Using the token bucket for rate 
limiting means simply using a better shaper than a timer based, or 
straight bandwidth limiting shaper.

> would have the maximum the maximum of 40 Kbit/s of ICMP traffic.

Saying that one token bucket per node is sufficient, you're either not 
understanding of how token bucket shapers are used, or you're stretching
the characteristics of the token bucket mechanism, making it appear
something which is not.

N = 1000 packet/sec average in mathematical abstraction is similar to
1packet/msec, but in terms of links that have different speed/bandwidth
it is very far from meaning the same thing.

B = 1000 packet/sec maximum burst does not mean the same thing and does 
  not have the same impact relative to a T1, and relative to a 1G interface.

It is one of the reasons traffic management is using multiple token
buckets in a node, in fact multiple token bucket per interface.

> look like that to me.  Seems like unnecessary complexity to me.

The implementation in LINUX and CISCO seem to suggest differently.

On the other hand I am open - in fact I suggested text - to make the 
SHOULD conditional to the existence of per interface traffic shaping.

> that, I'd suggest we put interface-specific values as a MAY -- 

Using MAY does not bring anything and therefore does not make sense. A 
MAY would be useful if there were a list of restrictions, to specify one 
or two restrictions that a router could make exception to.

> definitely not a SHOULD.

SHOULD is not a MUST; it is a suggested feature, not mandated feature. 
If an implementation does not have it, it does not mean it is not compliant.

> varied based on the interface speed.  [...]

You're contradicting yourself from one message to another...

I think I understand:  you are opposed because you are opposed...

Following this dialog, from your starting with a concern of two 
implementations, to moving to something else, every time your concern, 
or problem is answered or responded, makes it quite clear.

Regards,
Alex

---

From: Pekka Savola <[email-masked]>
Date: Tue, 17 Aug 2004 18:56:05 +0300 (EEST)

> way, or to preserve the "legacy" deployed IPv6 base?

No, I'm not sidestepping.  I think I've quite clearly stated that IMHO
not having interface-specific limiters is best for deployment: having
them just adds extra complexity, and seems to encourage to use
bandwidth-based limiters, which are not a good idea.  If you use token
bucket based mechanisms, you don't need to care about interface
speeds.

What's deployed out there is just one argument why we don't need to
specify interface-specific counters at all (or even if we wanted to
point it out, definitely not as a SHOULD).

In the revision, we've definitely already done some changes which have
made some legacy nodes non-compliant, e.g., by recommending against
timer-based approaches (which, unfortunately, have been implemented)  
so that's not a big problem as such.

---

From: Pekka Savola <[email-masked]>
Date: Tue, 17 Aug 2004 19:14:12 +0300 (EEST)

> the router ICMP implementation on the same threshold as the host ICMP 

Those were examples of *generic* rate-limiters.  No-one is doing
rate-limiting of ICMP generation like that (because those would have
to be configured by the users, and those will not just restrict ICMP
generation, but ICMP forwarding etc as well).  The ICMP rate-limiters
are done as part of the ICMP implementation itself.  For example on
Linux, look at something like icmpvX_xrlim_allow in the kernel source
under net/ipv4 or net/ipv6/.  Or if you want a Cisco example how to 
configure this, check out 'ipv6 icmp error-interval' -command.

I think you're confused about what kind of rate-limiting is being
specified in the ICMP spec.

> buckets in a node, in fact multiple token bucket per interface.

We're not discussing rate-limiting of ICMP in general, just discussing 
the number of ICMP messages the node is allowed to generate on its 
own. (The point above!)

A single limiter certainly wouldn't be sufficient if you wanted to 
limit e.g., the ICMP traffic the node is forwarding, but that's 
definitely out of scope of this specification!

> SHOULD conditional to the existence of per interface traffic shaping.

Again, that's a *generic* rate-limiting example.  That's not the point 
and is out of scope for *ICMP generation* rate-limiter.
 
> or two restrictions that a router could make exception to.

MAY makes a note to the implementors to be wary, but does not require 
anything.  This is the maximum we should be doing, and I'd argue we 
don't want even that.

---

From: Alex Conta <[email-masked]>
Date: Tue, 17 Aug 2004 12:14:20 -0400

Pekka Savola wrote:
> limiting?  None that I know of, but please enlighten us if so.  
> 

You've asked this question before, and you were given an answer.

This goes in circles.

Please check the references I pointed you to: LINUX and CISCO 
implementations of interface based ICMP rate control.

> I

It is about time to bring this spec where it should have been earlier.

> interfaces, behind tunnels, etc. --

Nobody said token bucket does not work.

But token buckets are not employed per node in routers, the way you said 
in your previous messages. This is plain misinformation, and I think it 
is really doing no good.

Token buckets are employed per flow and per interface in routers. That 
is the common place.

> why do we need to say more than 
> that?
> 

Because suggesting a token bucket rate control per node for a router
with slow/fast interfaces is plain wrong.

It is time to make the appropriate correction to the spec.

If the specs says something about rate control, which it does, it has to 
be correct. It is correct for hosts, it must be corrected for routers.
--------------------------------------------------------------------
IETF IPv6 working group mailing list
[email-masked]
Administrative Requests: https://www1.ietf.org/mailman/listinfo/ipv6
--------------------------------------------------------------------

---

From: Alex Conta <[email-masked]>
Date: Tue, 17 Aug 2004 13:22:44 -0400

Pekka Savola wrote:
>>I pointed you to the LINUX and CISCO examples. 
> [...]  No-one is doing
> generation, but ICMP forwarding etc as well). [...]
> 
Please read the specs before making these statements.

The text is clear in the ICMP specs: "...limit bandwidth and forwarding 
costs..."

Please read paragraph (f) page 7 of the ICMP specs, which says:

  (f) Finally, in order to limit the bandwidth and forwarding costs
      incurred by sending ICMPv6 error messages, an IPv6 node MUST
      limit the rate of ICMPv6 error messages it sends.

This points me to say that you must be confused, misinformed, or grossly 
misinterpreting.

> the number of ICMP messages the node is allowed to generate on its 

NO. This may be in your imagination.

We're discussing what is in the IPv6 ICMP specification.

> definitely out of scope of this specification!

You really need to read carefully the specification:

Text in paragraph (f) page 8 says clearly:

         A recommended method for implementing the rate-limiting function
         is a token bucket, limiting the average rate of transmission to
         N, where N can either be packets/second or a fraction of the
         attached link's bandwidth, but allowing up to B error messages
         to be transmitted in a burst....

note "transmission", "attached link bandwidth", "transmitted"?

NOTE: this very wording "attached link's bandwidth" suggest the 
connection between rate-control and interface.

alex

---

From: Pekka Savola <[email-masked]>
Date: Tue, 17 Aug 2004 22:18:59 +0300 (EEST)

I will not respond to your other message as you seem to have a
fundamental misunderstanding of the context of the word "send" in the
rate-limiter specification, so any discussion about that would be
useless.

See the clarification below:

> Please read the specs before making these statements.

I would ask you to do the same, and taking the context into account.

> This points me to say that you must be confused, misinformed, or grossly 

If you look at the whole section 2.4, it is blindingly obvious that
the word "send" is used to refer to the node *generating* (i.e.,
originating) an ICMP error message (in response to some other packet
doing something illegal), *NOT* in the more generic meaning which
could include forwarding an ICMP error packet it has received from
some other interface.

Do you agree with this? 

If you don't, ... I wouldn't me amazed as this seems to be your
fundamental misunderstanding.

If you do, and re-read the whole section again keeping that in mind, 
and (f) in particular, you should be able to tease it apart:

 - "in order to limit the bandwidth and forwarding costs *incurred 
by* sending ICMPv6 error messages"

Note the word "incurred".  At least in my book, this does not require
that these forwarding/bandwidth costs happen on the node that is 
generating the ICMP packet.

In other words, the algorithm is specified to reduce the number of
ICMPv6 messages which backbone routers etc. will have to forward over
the Internet, not so such the bandwidth of the generating node (note
that there isn't even a "forwarding cost" of generating an ICMPv6
message, it's generation and sending cost).  [the byproduct, of
course, is that the node originating ICMPv6 errors doesn't waste all
of its resources sending huge amounts of ICMPv6 errors]

 - "an IPv6 node MUST limit the rate of ICMPv6 error messages it 
sends."

Note the word *sends* and its context.  This specifies that ICMPv6
error messages must not be *generated* without limiting.  This
specifies nothing of routers *forwarding* received ICMPv6 messages.

...

Do you agree?

Whether you agree with this or not, I guess we'll need to reword some
of the text to avoid confusing ICMPv6 error *generation* rate-limiters
to *generic* (ICMPv6) rate-limiters (which you keep referring to)  
which are completely out of scope of this specification.

Removing pretty much everything relating to bandwidth could probably
be a good start, and using the word "originate" instead of "send".

---

From: Alex Conta <[email-masked]>
Date: Wed, 18 Aug 2004 04:13:29 -0400

Pekka Savola wrote:

> [...] you seem to have a
> rate-limiter specification [...]
> 

It is amusing for you to say that. As a co-author of the ICMPv6 RFC 
2463, and RFC 1885, work which started in 1994, I have a good insight on 
the meaning of the text, and the meaning of the standard. However I am 
not interested in giving lectures, and am not interested in proving how 
wrong you are in interpreting the ICMP specs. I am rather interested in 
a quick positive outcome and a win-win situation.

Are you ready to be constructive?

If your concern, your vision and your target is "controlling internal 
ICMP message generation" and the mechanism to achieve that is the token 
bucket, I do not understand why does it matter to you the number of 
token buckets, whether it is one, or many?

I hope you understand that with one token bucket per node, you cannot 
control ICMP packet generation in a modern router, so basically, routers
are left out.

Just to make sure we're on the same page:

in a modern router architecture there is distributed packet forwarding, 
i.e. multiple line cards, each line card with its own implementation of 
link, IP, and ICMP processing, as well as traffic management, and 
support for multiple interfaces, that have bandwidth which ranges from 
fractions of 1.5Mbit/sec at one end of the spectrum, and 1Gbit/sec, or 
10Gbit/sec at the other end of the spectrum.

So you understand now that because each line card may have at least one 
IP/ICMP engine, the token-bucket per node cannot apply?

Therefore, the current recommendation in the ICMP specs should be 
upgraded to support rate control per line card. As a line card has 
several interfaces, in general of same type, it is sufficient to specify 
the rate control for one of the interfaces, which brings us to the 
interface based rate control configuration, and the one sentence text 
addition to the ICMP specs that I suggested.

So, in order to support distributed control of ICMP message generation, 
the addition to the spec which I suggested is necessary, since one token 
bucket per node cannot apply.

How is this with you?

---

From: Havard Eidnes <[email-masked]>
Date: Wed, 18 Aug 2004 12:12:58 +0200 (CEST)

Hi,

I think that instead of mandating either a single rate limiter per
router or one per interface for locally originated ICMP traffic, the
specification should be loosened up to allow either.

Multiplying the ICMP rate a node originates by the number of
interfaces should not make much of a difference operationally, and if
a router has many interfaces it also has more paths to spread this
traffic over.  On the other hand, for a strictly software based
router, having to implement a rate limiter per interface is probably
unneeded additional complexity.

Therefore, I think mandating one or the other method would be unwise.

Regards,

- Håvard

---

From: Alex Conta <[email-masked]>
Date: Wed, 18 Aug 2004 10:18:14 -0400

> Therefore, I think mandating one or the other method would be unwise.

SHOULD is not mandating. It is recommending.

MUST, which is mandating, would be too strong, I agree.

Regards,
Alex