Subject: Re: [Idr] packet-rate action for flowspec

From: John Schiel <[email-masked]>
Date: Tue, 1 Dec 2015 08:57:12 -0700

On 11/24/2015 01:15 PM, Aseem Choudhary (asechoud) wrote:
> Hi Wesley,
>
> <snip>
> The maximum value that a 32-bit unsigned integer could hold would
>        limit to specifying under 2.15 Gpps (2.15 billion packets per
>        second).  For large or high-performance networks especially in the
>        future, this may not be sufficient.  The maximum floating point
>        value is much higher (on the order of 10^38) and should be future-
>        Proof.
>
>
> </snip>
>
> For all use-cases, it doesn¹t have to be counted in Billions to call it
> attack.
> A number much lower will qualify as well.

Aseem,

     Is your point that allowing a higher value is not needed? If so, I 
would agree. I see no reason why I would want to allow a rate limit 
larger than that through an interface. The whole idea of limiting the 
rate, to me, is to keep the rate low enough to allow some small amount 
of valid traffic that might get through and at the same time, keep out 
all of the undesirable traffic.

--John

>
> -thanks,
> Aseem
>
>
> On 11/23/15, 6:38 PM, "Wesley Eddy" <[email-masked]> wrote:
>
>> Hello, I'd like to make the working group aware of a very short new draft:
>> https://tools.ietf.org/html/draft-eddy-idr-flowspec-packet-rate-00
>>
>> The idea is to enhance the set of available flowspec actions, by
>> allocating a community value to indicate packets-per-second rate-limits.
>>
>> We had heard from some operators who make use of flowspec that this
>> would be desirable, and had previously included this idea as one part of
>> an earlier draft (draft-eddy-idr-flowspec-exp).  There was a bit of
>> feedback indicating that at least this packet rate limitation may be a
>> reasonable idea to take forward.
>>
>> One important question, not yet addressed in this draft, has been raised
>> by Qiandeng Liang.  In brief, it is how to properly handle the
>> concurrence of both a traffic-rate and a packet-rate action in a
>> flowspec advertisement.  To me, it seems this could safely be
>> implementation dependent, but would like to explore other thoughts the
>> working group may have.
>>
>> Any other feedback is welcome, as well.
>>
>> _______________________________________________
>> Idr mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/idr
> _______________________________________________
> Idr mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/idr

---

From: "Aseem Choudhary (asechoud)" <[email-masked]>
Date: Tue, 1 Dec 2015 16:31:42 +0000

Hi John,

Please see inline.

-thanks,
Aseem

>all of the undesirable traffic.

This is correct. I think, even though defining it as a floating point is
fine,
is not a necessity.
Also, we should describe what is expected behavior when a network device
can count upto a limit ( say 1 million pps).

---

From: John Schiel <[email-masked]>
Date: Wed, 2 Dec 2015 09:47:41 -0700

<snip>

(inline)

Aseem,

     I've looked at some of the supporting RFCs and I agree to your 
point. I think maybe Wes was just trying to stay consistent with the 
traffic-rate definition? I have no strong preference for either 
declaration.

     A traffic rate of zero for the traffic-rate FlowSpec definition in 
RFC5575 means all traffic is dropped. Could we use a high-value setting 
in packet-rate to tell the device processing the FlowSpec rule to take 
the max pps allowed as defined on that device?

--John

<snip>

---

From: Wesley Eddy <[email-masked]>
Date: Wed, 2 Dec 2015 12:18:00 -0500

On 12/2/2015 11:47 AM, John Schiel wrote:
> RFC5575 means all traffic is dropped. Could we use a high-value 
> setting in packet-rate to tell the device processing the FlowSpec rule 
> to take the max pps allowed as defined on that device?
>

You're correct that us initially proposing to use floating-point was 
done only really for consistency with existing use of floating-point in 
traffic-rate per RFC 5575.  I do not know the rationale for why it was 
chosen there, but assume it's because of the wider range of values that 
can be encoded compared to using a simple 32-bit unsigned integer.

To me, this is not a big issue one way or the other.  The range of 
numbers that floating-point can convey is larger, but since generally a 
rate limit won't be very large anyways, this is advantage has dubious 
real value.

I agree with your proposal that a high value (e.g. the maximum that 
whatever the binary representation is can convey) could be used to 
indicate the maximum a device supports, though I do not know if this 
would ever be used in practice.  I do want to avoid designing special 
rules that make this overcomplicated though.  To my knowledge, no 
equivalent for the maximum bits-per-second rate is discussed for the 
traffic-rate action in 5575, and I'm not aware of people that have a 
problem with that ... generally rate limits of any kind that you want to 
put in place would be lower than the maximums that your devices support 
by a fair margin (otherwise there is probably some error!).

---

From: John Schiel <[email-masked]>
Date: Wed, 2 Dec 2015 11:32:35 -0700

<snip>

>>     A traffic rate of zero for the traffic-rate FlowSpec definition 
>> in RFC5575 means all traffic is dropped. Could we use a high-value 
>> setting in packet-rate to tell the device processing the FlowSpec 
>> rule to take the max pps allowed as defined on that device?
>>
>
>
> done only really for consistency with existing use of floating-point 
> in traffic-rate per RFC 5575.  I do not know the rationale for why it 
> was chosen there, but assume it's because of the wider range of values 
> that can be encoded compared to using a simple 32-bit unsigned integer.
>
> numbers that floating-point can convey is larger, but since generally 
> a rate limit won't be very large anyways, this is advantage has 
> dubious real value.
>
>
> problem with that ... generally rate limits of any kind that you want 
> to put in place would be lower than the maximums that your devices 
> support by a fair margin (otherwise there is probably some error!).
Agree, no need for special rules in this draft if they are not already 
there and would also agree to your point about keeping the limits less 
than the maximum rate supported by a device. Some knowledge is required 
to implement the RL so leaving this to letting the device decide what to 
do isn't a good idea.

High-values was just a suggestion to address Aseem's device limit question.

 >> can count upto a limit ( say 1 million pps).

I suggest leaving the max limit for the operator to decide what to do 
with their network devices.

--John

---

From: Jeffrey Haas <[email-masked]>
Date: Wed, 2 Dec 2015 13:51:46 -0500

Wes,

On Wed, Dec 02, 2015 at 12:18:00PM -0500, Wesley Eddy wrote:
> in traffic-rate per RFC 5575.  I do not know the rationale for why
> it was chosen there, but assume it's because of the wider range of
> values that can be encoded compared to using a simple 32-bit
> unsigned integer.

One general headache of 4-byte floating point numbers is that when you start
losing precision, you can have weird discrepancies between what you
configure, vs. what value can be encoded.  This is one of the reasons why
floating point on the wire is annoying.

-- Jeff

---

From: "Acee Lindem (acee)" <[email-masked]>
Date: Wed, 2 Dec 2015 18:56:30 +0000

> On Dec 2, 2015, at 1:51 PM, Jeffrey Haas <[email-masked]> wrote:
> 
> Wes,
> 

I agree.

---

From: Wesley Eddy <[email-masked]>
Date: Wed, 2 Dec 2015 14:25:41 -0500

> floating point on the wire is annoying.

Can anyone who was around at the time help to understand why 
floating-point was specified in RFC 5575 for traffic-rate?

---

From: "Acee Lindem (acee)" <[email-masked]>
Date: Wed, 2 Dec 2015 21:09:19 +0000

Hi Wes, 

> On Dec 2, 2015, at 2:25 PM, Wesley Eddy <[email-masked]> wrote:
> 
> Can anyone who was around at the time help to understand why floating-point was specified in RFC 5575 for traffic-rate?

I wouldn’t blame the RFC 5575 authors. I know IEEE Floating Point has been used for traffic rates since, at least, RFC 2210. 

Thanks,
Acee

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 2 Dec 2015 22:16:46 +0100

Wesley,

The only reason to use floating notation as I recall was to express various
traffic rates with single 4 octets. IEEE.754 was just as good format as any
other with an advantge of being implemented by some vendors already.

Otherwise if we would take say integer we would likely need 4 octets for
bytes/s, 4 octets for kbytes/s, 4 octets for Mbytes/s and 4 octets in
Gbytes/s etc.. And who knows what traffic rates will be tomorrow ....

Best,
r.

---

From: "Aseem Choudhary (asechoud)" <[email-masked]>
Date: Wed, 2 Dec 2015 22:02:03 +0000

IMHO, this is not the case with PPS (packets per sec). Assuming all packets of 64 bytes, this translates to 512 bits per packet. We have 9 bits to save and hence 4-byte integer will be fine without floating-point. It saves much more as we consider higher average packet size. Besides, typically pps will not count till billions of packets, especially for flowspec, as it is counter-productive. I suggest a 4-bytes integer is just fine for pps.

-thanks,
Aseem

From: Robert Raszuk <[email-masked]<mailto:[email-masked]>>
Date: Wednesday, December 2, 2015 at 1:16 PM
To: Wesley Eddy <[email-masked]<mailto:[email-masked]>>
Cc: idr wg <[email-masked]<mailto:[email-masked]>>
Subject: Re: [Idr] packet-rate action for flowspec

Wesley,

The only reason to use floating notation as I recall was to express various traffic rates with single 4 octets. IEEE.754 was just as good format as any other with an advantge of being implemented by some vendors already.

Otherwise if we would take say integer we would likely need 4 octets for bytes/s, 4 octets for kbytes/s, 4 octets for Mbytes/s and 4 octets in Gbytes/s etc.. And who knows what traffic rates will be tomorrow ....

Best,
r.

On Wed, Dec 2, 2015 at 8:25 PM, Wesley Eddy <[email-masked]<mailto:[email-masked]>> wrote:
On 12/2/2015 1:51 PM, Jeffrey Haas wrote:
Wes,

_______________________________________________
Idr mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/idr

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 2 Dec 2015 23:22:04 +0100

Aseem,

If I understand your point you are suggesting to change the unit from bytes
per sec to packets per sec. If so this is IMHO pretty bad idea for the
shipping RFC in number of implementations as the proposed change is not
backwards compatible and will just cause significant deployment mess in
those cases where traffic-rate ext community is actually being used.

However you/WG may suggest and standardize new extended community with pps
unit of measurement if there would be rough consensu to do so though. But
the time for comments on existing specification seems a bit too late.

Cheers,
R.

On Wed, Dec 2, 2015 at 11:02 PM, Aseem Choudhary (asechoud) <
[email-masked]> wrote:

> IMHO, this is not the case with PPS (packets per sec). Assuming all
> packets of 64 bytes, this translates to 512 bits per packet. We have 9 bits
> to save and hence 4-byte integer will be fine without floating-point. It
> saves much more as we consider higher average packet size. Besides,
> typically pps will not count till billions of packets, especially for
> flowspec, as it is counter-productive. I suggest a 4-bytes integer is just
> fine for pps.
>
> -thanks,
> Aseem
>
> From: Robert Raszuk <[email-masked]>
> To: Wesley Eddy <[email-masked]>
> Cc: idr wg <[email-masked]>
> The only reason to use floating notation as I recall was to express
> various traffic rates with single 4 octets. IEEE.754 was just as good
> format as any other with an advantge of being implemented by some vendors
> already.
>

---

From: John Schiel <[email-masked]>
Date: Wed, 2 Dec 2015 16:23:04 -0700

On 12/02/2015 03:22 PM, Robert Raszuk wrote:
> Aseem,
>
> If I understand your point you are suggesting to change the unit from 
> bytes per sec to packets per sec. If so this is IMHO pretty bad idea 
> for the shipping RFC in number of implementations as the proposed 
> change is not backwards compatible and will just cause significant 
> deployment mess in those cases where traffic-rate ext community is 
> actually being used.
>
> However you/WG may suggest and standardize new extended community with 
> pps unit of measurement if there would be rough consensu to do so 
> though. But the time for comments on existing specification seems a 
> bit too late.

Robert,

     Wes has submitted a new draft to handle a new extended community 
for pps, 
https://tools.ietf.org/html/draft-eddy-idr-flowspec-packet-rate-00, thus 
the discussion on the data type.

> Cheers,
> R.
>
>
>
> On Wed, Dec 2, 2015 at 11:02 PM, Aseem Choudhary (asechoud) 
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>     IMHO, this is not the case with PPS (packets per sec). Assuming
>     all packets of 64 bytes, this translates to 512 bits per packet.
>     We have 9 bits to save and hence 4-byte integer will be fine
>     without floating-point. It saves much more as we consider higher
>     average packet size. Besides, typically pps will not count till
>     billions of packets, especially for flowspec, as it is
>     counter-productive. I suggest a 4-bytes integer is just fine for pps.
>

I agree with this statement.

--John

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 3 Dec 2015 00:34:04 +0100

John,

Robert,
>
>     Wes has submitted a new draft to handle a new extended community for
> pps,  https://tools.ietf.org/html/draft-eddy-idr-flowspec-packet-rate-00,
> <https://tools.ietf.org/html/draft-eddy-idr-flowspec-packet-rate-00>thus

​Yes I am aware of that hence my second paragraph.

Aseem however sounded like new proposal defining "packet-rate" should be a
replacement of existing "traffic-rate".

On the other hand if the new packet rate is added as new ext comm type then
Jeff's point stands what actions should be done when both are present in
the update message. The above draft should clearly define it but as far as
I can tell it does not.

R.

​

---

From: John Schiel <[email-masked]>
Date: Wed, 2 Dec 2015 16:50:54 -0700

On 12/02/2015 04:34 PM, Robert Raszuk wrote:
> John,
>
>     Robert,
>
>         Wes has submitted a new draft to handle a new extended
>     community for pps,
>     https://tools.ietf.org/html/draft-eddy-idr-flowspec-packet-rate-00, <https://tools.ietf.org/html/draft-eddy-idr-flowspec-packet-rate-00>thus
> ​ Yes I am aware of that hence my second paragraph.

My apologies.

>
> Aseem however sounded like new proposal defining "packet-rate" should 
> be a replacement of existing "traffic-rate".

Understand.

>
> On the other hand if the new packet rate is added as new ext comm type 
> then Jeff's point stands what actions should be done when both are 
> present in the update message. The above draft should clearly define 
> it but as far as I can tell it does not.
I missed where that was mentioned but regardless, I agree this should be 
defined.

I propose traffic-rate and packet-rate be mutually exclusive. If both 
are allowed at the same time, then you have to be concerned with which 
one takes priority, error handling, resource contention and a whole host 
of other issues I could probably name. Remove all doubt  and don't allow 
both traffic-rate and packet-rate to come across at the same time.

--John

---

From: Wesley Eddy <[email-masked]>
Date: Wed, 2 Dec 2015 19:59:44 -0500

On 12/2/2015 6:50 PM, John Schiel wrote:
> one takes priority, error handling, resource contention and a whole 
> host of other issues I could probably name. Remove all doubt  and 
> don't allow both traffic-rate and packet-rate to come across at the 
> same time.
>

In RFC-speak, we can probably just say that they SHOULD NOT be used 
together due to unclear semantics.

Implementations will need to be robust to receiving both at the same 
time (i.e. not crash, break the session, etc), and they can choose to do 
something reasonable but implementation-specific (e.g. pick one or the 
other).  In this sense, I think it is similar to other strange 
combinations of actions that could be sent.

---

From: "Aseem Choudhary (asechoud)" <[email-masked]>
Date: Thu, 3 Dec 2015 01:47:51 +0000

Hi Robert,

My comments were regarding discussion on the draft (original thread):

-thanks,
Aseem

From: Robert Raszuk <[email-masked]<mailto:[email-masked]>>
Date: Wednesday, December 2, 2015 at 2:22 PM
To: Cisco Employee <[email-masked]<mailto:[email-masked]>>
Cc: Wesley Eddy <[email-masked]<mailto:[email-masked]>>, idr wg <[email-masked]<mailto:[email-masked]>>
Subject: Re: [Idr] packet-rate action for flowspec

Aseem,

If I understand your point you are suggesting to change the unit from bytes per sec to packets per sec. If so this is IMHO pretty bad idea for the shipping RFC in number of implementations as the proposed change is not backwards compatible and will just cause significant deployment mess in those cases where traffic-rate ext community is actually being used.

However you/WG may suggest and standardize new extended community with pps unit of measurement if there would be rough consensu to do so though. But the time for comments on existing specification seems a bit too late.

Cheers,
R.

On Wed, Dec 2, 2015 at 11:02 PM, Aseem Choudhary (asechoud) <[email-masked]<mailto:[email-masked]>> wrote:
IMHO, this is not the case with PPS (packets per sec). Assuming all packets of 64 bytes, this translates to 512 bits per packet. We have 9 bits to save and hence 4-byte integer will be fine without floating-point. It saves much more as we consider higher average packet size. Besides, typically pps will not count till billions of packets, especially for flowspec, as it is counter-productive. I suggest a 4-bytes integer is just fine for pps.

-thanks,
Aseem

Wesley,

Best,
r.

---

From: "Aseem Choudhary (asechoud)" <[email-masked]>
Date: Thu, 3 Dec 2015 01:59:10 +0000

Hi Robert,

Apologies if I sounded like that. Eventhough, it will simplify to have unit, but I agree it is too late.

-thanks,
Aseem

From: Robert Raszuk <[email-masked]<mailto:[email-masked]>>
Date: Wednesday, December 2, 2015 at 3:34 PM
To: John Schiel <[email-masked]<mailto:[email-masked]>>
Cc: idr wg <[email-masked]<mailto:[email-masked]>>
Subject: Re: [Idr] packet-rate action for flowspec

John,

Robert,

    Wes has submitted a new draft to handle a new extended community for pps,  https://tools.ietf.org/html/draft-eddy-idr-flowspec-packet-rate-00, <https://tools.ietf.org/html/draft-eddy-idr-flowspec-packet-rate-00> thus the discussion on the data type.

Aseem however sounded like new proposal defining "packet-rate" should be a replacement of existing "traffic-rate".

On the other hand if the new packet rate is added as new ext comm type then Jeff's point stands what actions should be done when both are present in the update message. The above draft should clearly define it but as far as I can tell it does not.

R.

​

---

From: John Schiel <[email-masked]>
Date: Thu, 3 Dec 2015 09:16:10 -0700

> time (i.e. not crash, break the session, etc), and they can choose to 
> do something reasonable but implementation-specific (e.g. pick one or 
> the other).  In this sense, I think it is similar to other strange 

Fair enough.

Can we make some changes then?

Change:

          +------+--------------------+--------------------------+
          | type | extended community | encoding                 |
          +------+--------------------+--------------------------+
          | TBD  | packet-rate        | 2-byte as#, 4-byte float |
          +------+--------------------+--------------------------+

To:

          +------+--------------------+----------------------------+
          | type | extended community | encoding                   |
          +------+--------------------+----------------------------+
          | TBD  | packet-rate        | 2-byte as#, 4-byte integer |
          +------+--------------------+----------------------------+

And at the bottom of the section 2. Packet Rate Action.

The use of packet-rate filtering SHOULD NOT be used with traffic-rate 
filtering together due to unclear semantics.

--John

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 3 Dec 2015 17:28:53 +0100

Ok - I think this is moving fwd :)

In the spirit of RFC2119 I recommend we express MUST NOT rather then SHOULD
NOT the latter being just a recommendation.

Now with "MUST NOT" IMO we actually should specify the implementation
behaviour not to install entire NLRI as match when both ext communities are
present treating it as sort of soft error. Syslog warning should be logged
of course.

Alternatively with "SHOULD NOT" draft needs to spec out the preference of
which one to use by default when both are present. I think for backwards
compatibility the default should be traffic-rate with a knob to use
packet-rate instead when available.

Best,
r.

On Thu, Dec 3, 2015 at 5:16 PM, John Schiel <[email-masked]> wrote:

>> Implementations will need to be robust to receiving both at the same time
>> (i.e. not crash, break the session, etc), and they can choose to do
>> other).  In this sense, I think it is similar to other strange combinations
>> of actions that could be sent.
>>
>>
> Fair enough.
>

---

From: Jeffrey Haas <[email-masked]>
Date: Fri, 4 Dec 2015 10:58:09 -0500

Robert,

> present treating it as sort of soft error. Syslog warning should be logged

I think the major issue with these sort of things is what happens when
you're on an implementation that doesn't understand some but understands
other forms of these action criteria.  If I understand rate-limit but don't
know pps, I can still do "something useful".

If I understand both, I might be able to do both.

In most cases, I'd expect pps to be set low enough that it'd take effect
before bps.  To that end, I'd be fine with pps taking precedence in
implementations that understand both.

-- Jeff

---

From: John Schiel <[email-masked]>
Date: Wed, 9 Dec 2015 08:39:22 -0700

Sorry, been very busy of late.

I have no real preference on which takes rate limiting action takes 
precedence if both are allowed but I would like to keep the work level 
of this to a minimum so if Robert is correct in his statement that 
traffic-rate take priority for backwards compatibility, then I vote we 
go that direction.

The knob can be tuned in a way that packet-rate action would take effect 
prior to traffic-rate if the levels are set appropriately.

Thoughts?

--John