Subject: [v6ops] draft-gont-v6ops-ipv6-ehs-in-real-world: clarification text

From: Fernando Gont <[email-masked]>
Date: Tue, 14 Apr 2015 05:41:50 -0300

Folks,

At the last v6ops meeting, there seemed to be agreement about including
some text noting that while results indicate that use of IPv6 EHs has
been found to result in packet drops, at least for transit ASes such
behavior is undesirable.

So this is a *first try* (please suggest edits to this text if you
think that's necessary):

"The results presented in this document indicate that in the scenarios
where the corresponding measurements were performed, the use of IPv6
extension headers can lead to packet drops. We note that, in particular,
packet drops occurring at transits Autonomous Systems is undesirable,
and it is expected that this situation will improve over time."

Thanks!

Best regards,
-- 
Fernando Gont
SI6 Networks
e-mail: [email-masked]
PGP Fingerprint: 6666 31C6 D484 63B2 8FB1 E3C4 AE25 0D55 1D4E 7492

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 15 Apr 2015 08:38:23 +1200

On 14/04/2015 20:41, Fernando Gont wrote:
...
> and it is expected that this situation will improve over time."

s/expected/hoped/ perhaps.

   Brian

---

From: Fernando Gont <[email-masked]>
Date: Tue, 14 Apr 2015 17:48:32 -0300

> s/expected/hoped/ perhaps.

Yes -- that's what I really meant (English as 2nd language effect :-) ).

Thanks!

Best regards,

---

From: "Eric Vyncke (evyncke)" <[email-masked]>
Date: Fri, 17 Apr 2015 07:17:31 +0000

Fernando

Your proposal is fine for me, but, I would suggest a slightly stronger
text:
"The results presented in this document indicate that in the scenarios
where the corresponding measurements were performed, the use of IPv6
extension headers can lead to packet drops. We note that
packet drops occurring at transit networks is undesirable
and it is hoped and expected that this situation will improve over time."

Should we say something around the lines of "... Undesirable except when
Those packets cannot be forwarded without impacting the performance and
the health of the network devices" ?

-éric

---

From: Fernando Gont <[email-masked]>
Date: Fri, 17 Apr 2015 15:51:05 -0300

Hi, Eric,

> the health of the network devices" ?

I'm fine with the text you suggest.

The only comment/question is that there are two conflicting povs: frm
the use pov, he'd probably like his packets to go through transparently,
and hence the packet drops are undesirable (no matter what). But from
the network operator pov, the packet drops aren't undesireable (ot one
culd say that they really are.. in the sense that the network operator
would be glad to not need to drop these packets).

Thanks!

Cheers,

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Sat, 18 Apr 2015 01:46:47 +0000

> On Apr 17, 2015, at 3:17 PM, Eric Vyncke (evyncke) <[email-masked]> wrote:
> 

Well, question for you. If we follow RFC 2460, the router in the middle doesn’t know whether the header is there or not. The only systems that should have a performance impact are systems that parse to them or interpret them. I personally would like to believe that we aren’t making *other* systems more complex to save them. If this is being done in a firewall or load balancer, the limitations of the device aren’t primary, the correct operation of the protocol is. Would we be saying that it’s OK for a firewall to drop a packet because it doesn’t feel like dealing with it that day?

What specific devices did you have in mind? What would the argument be?

---

From: "Eric Vyncke (evyncke)" <[email-masked]>
Date: Sat, 18 Apr 2015 05:58:22 +0000

Fred,

The performance impact/network health issues are mainly related to
hop-by-hop which is per RFC 2460 inspected by every router on the path.
This EH should be ignore IMHO by all routers over the Internet (private
networks may still inspect/act on it for RSVP for examine).

If for some reasons, a router in the middle needs access to layer-4
information (IPFIX? DDoS mitigation? ... ?), then the EH chain must be
parsed which can cause a performance impact.

Do not take me wrong: beside ignoring HbH, I really hope that routers on
the paths simply look at layer-3 header and do nothing else. But, for this
I-D, I want to be sure to cover all aspects, hence the modified sentence

-éric

On 18/04/15 03:46, "Fred Baker (fred)" <[email-masked]> wrote:

>Well, question for you. If we follow RFC 2460, the router in the middle
>doesn’t know whether the header is there or not. The only systems that
>should have a performance impact are systems that parse to them or
>interpret them. I personally would like to believe that we aren’t making
>*other* systems more complex to save them. If this is being done in a
>firewall or load balancer, the limitations of the device aren’t primary,
>the correct operation of the protocol is. Would we be saying that it’s OK
>for a firewall to drop a packet because it doesn’t feel like dealing with
>it that day?
>

---

From: Nick Hilliard <[email-masked]>
Date: Sat, 18 Apr 2015 12:23:16 +0100

On 18/04/2015 02:46, Fred Baker (fred) wrote:
> interpret them. 

layer 4 information is routinely parsed to extract n-tuples in order to
create lag / ecmp hashes for load balancing.  I.e. we have a requirement
for routers to interpret ipv6 packet headers on a per-hop basis, and make
consistent routing decisions based on this analysis.

Nick

---

From: Brian E Carpenter <[email-masked]>
Date: Sun, 19 Apr 2015 08:20:44 +1200

> networks may still inspect/act on it for RSVP for examine).

With a slight sigh:

We explicitly covered this in RFC 7045, which updated RFC 2460 in 2013.
http://tools.ietf.org/html/rfc7045#section-2.2

IMHO we should not be messing with that (or other) standards track
statements in this Informational document.

> consistent routing decisions based on this analysis.

Again, we have hashed over this (pun intended) in the past and the conclusions
are in RFC 7045, RFC 6564 and for that matter in RFC 6438 and RFC 7098.

Badsically the design of the extension header mechanism is unfriendly to
middleboxes, which is one reason that I have been trying to make the flow
label useful.

    Brian

---

From: "C. M. Heard" <[email-masked]>
Date: Sat, 18 Apr 2015 18:07:42 -0700 (PDT)

On Fri, 17 Apr 2015, Eric Vyncke (evyncke) wrote:
> I would suggest a slightly stronger text:
> packet drops occurring at transit networks [are] undesirable

I concur with this proposal (modulo correction in square brackets).  
It is consistent with RFC 7045, which reaffirms that intermediate 
nodes SHOULD process extension headers as described in RFC 2460 
(i.e., act upon the options in a Hop-by-Hop Options extension header 
and ignore all other extension headers).

> the health of the network devices" ?

No.  From the standpoint of the user of the general internet, such 
drops by transit providers are never desirable.  They may in some 
cases be a practical necessity (e.g., due to inability to process 
Hop-by-Hop Options at wire speed), but they are still undesirable.

> statements in this Informational document.

Exactly.

//cmh

---

From: Mark ZZZ Smith <[email-masked]>
Date: Sun, 19 Apr 2015 03:50:26 +0000 (UTC)

From: Nick Hilliard <[email-masked]>
 To: [email-masked] 
 Sent: Saturday, 18 April 2015, 21:23
 Subject: Re: [v6ops] draft-gont-v6ops-ipv6-ehs-in-real-world: clarification text

/ That is what the flow label is for, and we should be encouraging vendors of both host OSes and devices which perform LAG/ECMP to be setting/using it for that purpose. Support for the use of it this way is in the current IPv6 node requirements RFC from 2011 (RFC6424).
/ Although it is disabled by default, the Linux kernel has supported automatic generation of flow label values since around August 2014 (sysctl net.ipv6.auto_flowlabels=1 to enable).
/ Regards,mark.

Nick

_______________________________________________
v6ops mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/v6ops

---

From: Nick Hilliard <[email-masked]>
Date: Sun, 19 Apr 2015 12:16:43 +0100

On 19/04/2015 04:50, Mark ZZZ Smith wrote:
> / That is what the flow label is for, and we should be encouraging vendors
> of both host OSes and devices which perform LAG/ECMP to be setting/using it
> for that purpose. Support for the use of it this way is in the current IPv6
> node requirements RFC from 2011 (RFC6424).
> 
> / Although it is disabled by default, the Linux kernel has supported
> automatic generation of flow label values since around August 2014 (sysctl
> net.ipv6.auto_flowlabels=1 to enable).

yes, the flow label will help as one entry in the hash n-tuple but it would
be unwise to depend on this as the only hash entropy source.  L3+L4 header
information is still a better determinant.  It always surprised me that the
flow label has been part of the ipv6 spec since the very earliest
discussions in 1993, but that it wasn't present in the linux kernel until
very recently and even then, only as a non-default option.

Nick

---

From: Mark ZZZ Smith <[email-masked]>
Date: Mon, 20 Apr 2015 11:52:09 +0000 (UTC)

From: Nick Hilliard <[email-masked]>
 To: Mark ZZZ Smith <[email-masked]>; "[email-masked]" <[email-masked]> 
 Sent: Sunday, 19 April 2015, 21:16
 Subject: Re: [v6ops] draft-gont-v6ops-ipv6-ehs-in-real-world: clarification text

yes, the flow label will help as one entry in the hash n-tuple but it would
be unwise to depend on this as the only hash entropy source.
/ The RFC updating the flow label specification says that at a minimum it should be a 3 tuple of SA, DA and uniformly distributed flow label value.

  L3+L4 header
information is still a better determinant.
/ Layer 4 headers aren't good because they may not be directly behind the IPv6 header (as they might have been in IPv4 too because of options) and TCP and UDP aren't the only protocols you might want load distributed.
  It always surprised me that the
flow label has been part of the ipv6 spec since the very earliest
discussions in 1993, but that it wasn't present in the linux kernel until
very recently and even then, only as a non-default option.

/ I think applications being able to set the flow label value if they choose to has been available for a very long time, the thing that is much more recent is automatic label generation.

Nick

---

From: Joe Touch <[email-masked]>
Date: Mon, 20 Apr 2015 08:39:55 -0700

On 4/17/2015 12:17 AM, Eric Vyncke (evyncke) wrote:
...
> the health of the network devices" ?

<sarcasm>

Absolutely - it's entirely unreasonable to expect vendors to support the
protocols they claim, when to do so would impact their profits.

</sarcasm>

Joe

---

From: Joe Touch <[email-masked]>
Date: Mon, 20 Apr 2015 08:43:28 -0700

> If for some reasons, a router in the middle needs access to layer-4
                                              ^^^^^
> parsed which can cause a performance impact.

Let's be clear - "wants" access.

If that's what you want, then that's what you have to pay for (for the
vendor, in complexity to parse the packet to find the L4 header; for the
customer, in price).

But two things ought to be made clear:

	1) this is the router vendor's decision, not a requirement
	of Internet routers

	2) this might not even be worthwhile as we move to more
	secure protocols

Joe

---

From: Gert Doering <[email-masked]>
Date: Mon, 20 Apr 2015 23:21:25 +0200

Hi,

> > parsed which can cause a performance impact.
[..]
> 	1) this is the router vendor's decision, not a requirement

So, please tell me how you build an Internet router that is able to
defend itself against control plane abuse and does not need to look into
L4 to do so?

Building a router that doesn't do ACLs and control-plane rate limiting
is easy, but totally useless in today's Internet...

Gert Doering
        -- NetMaster
-- 
have you enabled IPv6 on something today...?

SpaceNet AG                        Vorstand: Sebastian v. Bomhard
Joseph-Dollinger-Bogen 14          Aufsichtsratsvors.: A. Grundner-Culemann
D-80807 Muenchen                   HRB: 136055 (AG Muenchen)
Tel: +49 (0)89/32356-444           USt-IdNr.: DE813185279

---

From: Joe Touch <[email-masked]>
Date: Mon, 20 Apr 2015 14:28:08 -0700

> defend itself against control plane abuse and does not need to look into

A router can protect its own control plane by looking at the packet
contents, but then it is acting as a host at that point and should be
looking there only for packets addressed to interfaces of that router.
That's not a forwarding function and thus doesn't limit the forwarding
plane.

It is not a requirement that one router protect the control planes of
other routers from abuse. That is a feature - and if you want to sell
that as a feature, your device should support doing so at rate.

Joe

---

From: Mark ZZZ Smith <[email-masked]>
Date: Tue, 21 Apr 2015 01:22:46 +0000 (UTC)

From: Joe Touch <[email-masked]>
 To: Gert Doering <[email-masked]> 
Cc: "[email-masked]" <[email-masked]>; Merike Kaeo <[email-masked]>; "[email-masked]" <[email-masked]>; Fernando Gont <[email-masked]> 
 Sent: Tuesday, 21 April 2015, 7:28
 Subject: Re: [v6ops] draft-gont-v6ops-ipv6-ehs-in-real-world: clarification text

/ I agree with this. It is easy to forget that a "router" is in actually a packet router at the forwarding plane and a host at the control plane.

 

Joe

---

From: Gert Doering <[email-masked]>
Date: Tue, 21 Apr 2015 08:48:11 +0200

Hi,

> That's not a forwarding function and thus doesn't limit the forwarding

Of course, but real world requires that this filter function needs to be
implemented in the forwarding plane, because otherwise packets would just
saturate the link between forwarding and control plane if you would only 
filter on "the other side".

From a purely academic point of view, I totally agree with you.  

Unfortunately, the Internet is not.

Gert Doering
        -- NetMaster

---

From: Mark ZZZ Smith <[email-masked]>
Date: Tue, 21 Apr 2015 08:06:41 +0000 (UTC)

From: Gert Doering <[email-masked]>
 To: Joe Touch <[email-masked]> 
Cc: "[email-masked]" <[email-masked]>; Merike Kaeo <[email-masked]>; "[email-masked]" <[email-masked]>; Fernando Gont <[email-masked]> 
 Sent: Tuesday, 21 April 2015, 16:48
 Subject: Re: [v6ops] draft-gont-v6ops-ipv6-ehs-in-real-world: clarification text
   
Hi,

Of course, but real world requires that this filter function needs to be
implemented in the forwarding plane, because otherwise packets would just
saturate the link between forwarding and control plane if you would only 
filter on "the other side".
/ So it should be easily possible to identify trusted IPv6 source and/or destination addresses for control plane processes/protocols, and drop packets that don't match them those on ingress to that control plane link, filtering them at egress of the forwarding plane. Host firewalling at the ingress of the control plane would then be able to perform further level of filtering if necessary. With the processing power of control plane CPUs these days, they shouldn't have trouble doing that.
/Furthermore, network cards these days perform large receive offload (concatenation of multiple UDP or TCP packets to be able to form a single large one to hand up to the operating system.) Because of their level of UDP or TCP understanding, It may be possible to push TCP or UDP port level filtering down into one of those if the control plane link is actually an ethernet link (or you could put one at the forwarding plane egress of the link to the control plane, and perform TCP/UDP port level filtering there.)
/ Regards,
Mark.

Gert Doering
        -- NetMaster

---

From: Joe Touch <[email-masked]>
Date: Tue, 21 Apr 2015 08:45:31 -0700

> filter on "the other side".

DDOS filtering is a feature, not a requirement.

If you want to offer that as a feature, please stop complaining that
customers should expect that it runs at rate with packets that conform
to existing standards.

> Unfortunately, the Internet is not.

The Internet is also not designed to optimize your profit margins either.

Joe

---

From: Gert Doering <[email-masked]>
Date: Tue, 21 Apr 2015 17:51:03 +0200

Hi,

> DDOS filtering is a feature, not a requirement.

I'm fully at a loss to express my amazement in polite words, so I'm just
*out* of this discussion now.

Gert Doering
        -- NetMaster

---

From: Nick Hilliard <[email-masked]>
Date: Tue, 21 Apr 2015 17:56:45 +0100

> *out* of this discussion now.

Fully agreed on this + that this thread needs to end.  The lack of
operational reality being displayed is pretty severe.

Nick

---

From: Joe Touch <[email-masked]>
Date: Tue, 21 Apr 2015 10:15:31 -0700

> operational reality being displayed is pretty severe.

The Internet is not designed to optimize convenience or profit. The
functions in IPv6 are there for a reason; turning them off to simplify
life for vendors isn't useful to customers.

If you want something that is trivial to parse in hardware, you might
consider offering ATM equipment instead.

Joe

---

From: [email-masked]
Date: Tue, 21 Apr 2015 19:38:00 +0200 (CEST)

> operational reality being displayed is pretty severe.

+1.

Steinar Haug, AS 2116

---

From: Warren Kumari <[email-masked]>
Date: Tue, 21 Apr 2015 14:23:56 -0400

> DDOS filtering is a feature, not a requirement.

<boggle>
Sorry, no....

Now I remember why I got fed up with the previous EH discussions
(draft-taylor-v6ops-fragdrop, draft-bonica-6man-frag-deprecate,
draft-wkumari-6man-long-headers, etc) and stopped following v6ops and
6man.
'parently not much has changed....

W

-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

---

From: Joe Touch <[email-masked]>
Date: Tue, 21 Apr 2015 11:29:00 -0700

...
>> DDOS filtering is a feature, not a requirement.

Then please point us all to the requirements-track RFC that establishes
this as a *requirement* for Internet routers.

Joe

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 21 Apr 2015 14:35:01 -0400

The need to protect the control plane components (routers, controllers, 
whatever you want to call them) from overload is a well-established 
practice in the industry.  It is expected by operators when they buy, 
and delivered by all major vendors in the devices they build.

Given that until recently it has been a behavior inside a device, no, it 
is not documented inan RFC.  That does not mean it is untrue or unrealistic.

Yours,
Joel

---

From: Joe Touch <[email-masked]>
Date: Tue, 21 Apr 2015 11:47:09 -0700

> is not documented inan RFC.  That does not mean it is untrue or

I don't disagree. But it is an E-ticket ride - it is inherently expensive.

I don't object to ops docs that say "this is hard", but I do think it's
not productive to say "and we should make the protocol easier by gutting
the hard parts" unless there is a rationale - one that presumably
doesn't rely only on "because it's hard".

Because that just translates to "because I'm greedy".

Here's an analogy:

	Cars advertise seating capacity.

	It's unrealistic to expect every 4-person car to
	fit four 600-pound people.

	It's equally unrealistic to redefine "person" to
	mean "110-pound model".

There needs to be a balance between what's expensive and inconvenient
vs. neutering a protocol to make it easy to implement.

Protecting the control plane can be done by the receiving host
(destination router). Protecting the network from sending traffic that
is dumped at the destination is certainly a useful capability, but not
to protect the destination router. It protects the network as a whole.

Protecting the network as a whole is useful, and may involve some level
of DPI - or may not. If the traffic is encrypted, then only the
destination router can protect itself anyway. But dropping the packets
because of EH use just makes the filtering router an attacker to the E2E
communication between the routers.

Joe

---

From: Mark ZZZ Smith <[email-masked]>
Date: Wed, 22 Apr 2015 06:20:00 +0000 (UTC)

So I'm going to call the emperor naked.
What I think you and Gert actually want is for EHs to be completely deprecated, so that the TCP and UDP headers are always in the same place in the packet, so that hardware can look at them for the purposes of dropping them in hardware or as inputs into LB. Is that the case?
What I'm curious about then is how do you handle DDoSes that are using IP fragments, where there are no TCP or UDP headers ports to look at?
I'm also curious about how you LB packets (either at layer 2 or layer 3) that don't have TCP or UDP headers, or aren't IP. I've seen LB become completely ineffective at layer 2 because a customer was using MPLS between two routers, so no MAC address variation, and no IP addresses and no TCP or UDP ports to look at.
In fact, given the trouble LAG has caused me in the past 2 years (LAG member links that are not current members are supposed to be considered by the bridge as normal bridge ports, so if you want to avoid loops across the links that are candidate members of the LAG, the IEEE expect you to be running STP of some form across them...), I'm a big fan of the quote on the last slide of this presentation:
"IEEE 802.3ad Link Aggregation(LAG)what it is, and what it is not"
http://www.ieee802.org/3/hssg/public/apr07/frazier_01_0407.pdf

"LAG is good, but it’s not as good as a fatter pipe."

      From: Nick Hilliard <[email-masked]>
 To: [email-masked] 
 Sent: Wednesday, 22 April 2015, 2:56
 Subject: Re: [v6ops] draft-gont-v6ops-ipv6-ehs-in-real-world: clarification text

Nick

---

From: joel jaeggli <[email-masked]>
Date: Tue, 21 Apr 2015 23:32:50 -0700

> So I'm going to call the emperor naked.

not a transit provider so I'm sitting in the edge that said I have some
exposure to this problem space.

hence

https://datatracker.ietf.org/doc/draft-ietf-v6ops-pmtud-ecmp-problem/

> What I think you and Gert actually want is for EHs to be completely
> deprecated, so that the TCP and UDP headers are always in the same place
> in the packet, so that hardware can look at them for the purposes of
> dropping them in hardware or as inputs into LB. Is that the case?
> 
> What I'm curious about then is how do you handle DDoSes that are using
> IP fragments, where there are no TCP or UDP headers ports to look at?

https://datatracker.ietf.org/doc/draft-taylor-v6ops-fragdrop/

> I'm also curious about how you LB packets (either at layer 2 or layer 3)
> that don't have TCP or UDP headers, or aren't IP. I've seen LB become
> completely ineffective at layer 2 because a customer was using MPLS
> between two routers, so no MAC address variation, and no IP addresses
> and no TCP or UDP ports to look at.

as someone who does stateless l3+l4 loadbalancing to servers, if I can't
find the l4 header in an asic based forwarding engine even if a server
can I have problem.

> In fact, given the trouble LAG has caused me in the past 2 years (LAG
> member links that are not current members are supposed to be considered
> by the bridge as normal bridge ports, so if you want to avoid loops
> across the links that are candidate members of the LAG, the IEEE expect
> you to be running STP of some form across them...), I'm a big fan of the
> quote on the last slide of this presentation:
> 
> "IEEE 802.3ad Link Aggregation (LAG) what it is, and what it is not"
> ------------------------------------------------------------------------
> *From:* Nick Hilliard <[email-masked]>
> *To:* [email-masked]
> *Sent:* Wednesday, 22 April 2015, 2:56
> *Subject:* Re: [v6ops] draft-gont-v6ops-ipv6-ehs-in-real-world:
> clarification text
> 
> [email-masked] <mailto:[email-masked]>

---

From: [email-masked]
Date: Wed, 22 Apr 2015 08:40:56 +0200 (CEST)

<flame retardant suit on>

> vs. neutering a protocol to make it easy to implement.

Here we actually agree on the basic principle - but probably not on
where that balance point is.

> (destination router).

This is a fundamental disagreement. Protecting the control plane needs
to be done by *all* intermediate routers in today's somewhat hostile
Internet environment. That also includes some traffic not explicitly
addressed to the routers themselves (for instance traceroute to a
destination reached *through* the routers). Only looking at traffic
addressed to the routers themselves is not sufficient.

> communication between the routers.

We have some services that are available within our AS, and should not
be available outside the AS. One such example is DNS resolvers (there
are others). We have chosen to block such services at the border - by
having the border routers explicitly blocking UDP/TCP port 53 to the
resolver addresses. This obviously means the border routers must be
able to *find* the UDP/TCP header in that part of the IPv6 packet they
are able to inspect at line rate. This is not "DPI" in any sense of
the word - it is very basic filtering.

And here, I assume, is the next disagreement. If the UDP/TCP headers
cannot be found because IPv6 EHs have pushed them beyond what the
routers can inspect at line rate - the packet will be dropped.

If we can get improved filtering capabilities in our routers as part
of regular equipment upgrade cycles - great, everybody is happy. But
we are not likely to buy significantly more expensive routers *just*
to be able to peek further into the packets (number of customers
needing this are *many* orders of magnitude too low - if they exist
at all).

Steinar Haug, AS 2116

---

From: Havard Eidnes <[email-masked]>
Date: Wed, 22 Apr 2015 08:42:56 +0200 (CEST)

> DDOS filtering is a feature, not a requirement.

Reality dictates otherwise.

- Håvard

---

From: Mark ZZZ Smith <[email-masked]>
Date: Wed, 22 Apr 2015 06:46:08 +0000 (UTC)

So I wasn't naively asking those questions, I know what the answers were likely to be.
What I'd really like is for people to actually say what they want, which I think is an Internet with packets that look like
[IPv6 Fixed Hdr][TCP Hdr], total size <= 1280[IPv6 Fixed Hdr][UDP Hdr], total size <= 1280
and that is it.
The consequence would be to deprecate many existing protocols and making them work over fragment unnecessary TCP or UDP.
I'd think OSPF is probably the most important one to start with.

      From: joel jaeggli <[email-masked]>
 To: Mark ZZZ Smith <[email-masked]>; Nick Hilliard <[email-masked]>; "[email-masked]" <[email-masked]> 
 Sent: Wednesday, 22 April 2015, 16:32
 Subject: Re: [v6ops] draft-gont-v6ops-ipv6-ehs-in-real-world: clarification text

hence

---

From: [email-masked]
Date: Wed, 22 Apr 2015 09:12:27 +0200 (CEST)

> The consequence would be to deprecate many existing protocols and making them work over fragment unnecessary TCP or UDP.

I see no reason to deprecate IPv6 EHs. But I need 

 [IPv6 Fixed Hdr] + [IPv6 EHs] + [L4 Hdr] <= hardware inspection limit

in order for the router hardware to be able to filter based on TCP/UDP
headers at line rate. With current equipment, I would expect hardware
inspection limit to be in the range 256 to 1280 bytes.

Steinar Haug, AS 2116

---

From: Jeroen Massar <[email-masked]>
Date: Wed, 22 Apr 2015 09:22:42 +0200

On 2015-04-22 09:12, [email-masked] wrote:
> headers at line rate.

The nasty answer to such a statement is: increase your hardware
inspection limit.

But the easier one, that works today is that very likely your DNS
servers recursing IP space is dedicated.

Hence, any packet headed toward those addresses is "private" and should
not get an answer.

Thus, instead of doing filtering on L4, why not just not route those
packets at all (L3)? Don't even have to firewall it.

My printer for instance has a nice global address, but as the border
router does not forward packets from $internet to $printer-net, a
traceroute or anything else ends up in a simple !N.

No per-port inspection needed, just plain old good routing.

Btw, does your hardware-load-balancer inspect ICMPv6 to do stateless
flow routing for ICMPv6 errors to the correct host? Or are you also just
fumbling with ICMPv6 PTBs like Amazon (+NetFlix) & Google?

Greets,
 Jeroen

---

From: Ole Troan <[email-masked]>
Date: Wed, 22 Apr 2015 09:27:00 +0200

Steinar,

> addressed to the routers themselves is not sufficient.

assuming we’re talking about a transit provider and transit traffic.
to help those of us who are lacking operational clue, could you expand on the cases where this is necessary?

> routers can inspect at line rate - the packet will be dropped.

if you drop packets with DA == your DNS server where you cannot find the L4 information, that seems perfectly fine.
if you drop packets with _any_ DA where you cannot find (or understand) the L4 information then you are offering broken service. again talking about transit here.
to do the former you don’t need to parse deep into the packet, nor deprecate EHs.

> needing this are *many* orders of magnitude too low - if they exist

even if you had routers that could parse to the end of the packet, I don’t quite see how that “helps”. the Internet’s success is to a large extent based on being able to route around broken networks, like this would be.

cheers,
Ole

---

From: [email-masked]
Date: Wed, 22 Apr 2015 09:31:02 +0200 (CEST)

> The nasty answer to such a statement is: increase your hardware

And that will probably happen - as part of regular equipment upgrade
cycles.

> packets at all (L3)? Don't even have to firewall it.

The world is not that simple.

> fumbling with ICMPv6 PTBs like Amazon (+NetFlix) & Google?

Don't have those boxes doing IPv6 in my network.

Steinar Haug, AS 2116

---

From: Jeroen Massar <[email-masked]>
Date: Wed, 22 Apr 2015 09:33:21 +0200

> And that will probably happen - as part of regular equipment upgrade

Changing a protocol that is trying be deployed for 20+ years already is
not going to happen quicker than the above ;)

> The world is not that simple.

Then please define the real problem you are trying to solve.

Your statement was that you have a private DNS server that you do not
want reachable from the outside.

Hence, not routing packets from $outside solves your problem.

Greets,
 Jeroen

---

From: Gert Doering <[email-masked]>
Date: Wed, 22 Apr 2015 10:51:38 +0200

Hi,

On Wed, Apr 22, 2015 at 06:20:00AM +0000, Mark ZZZ Smith wrote:
> deprecated, so that the TCP and UDP headers are always in the same
> place in the packet, so that hardware can look at them for the
> purposes of dropping them in hardware or as inputs into LB. Is that
> the case?

No :-) - Hardware *is* sophisticated today, but only up to a certain limit.

What I think would be implementable with reasonable tradeoffs between
"unsane hardware effort that my end customers are not going to pay" and
"still flexible" is to require that the final EH is contained in the 
first <n> bytes (say, 512) and that the final header must be in the 
first fragment.

Right now, we have "the packet can have any size, and the final header can
be anywhere" - and doing that in hardware is just too expensive.

> What I'm curious about then is how do you handle DDoSes that are
> using IP fragments, where there are no TCP or UDP headers ports to
> look at?

That is actually quite easy: rate-limit fragments against the control plane, 
and drop first fragments if there is no L4 header inside.

(The thing is: not all packets are equal - for some, like ICMP, you want
to permit them towards the control plane up to a certain limit - say, 
10 Mbit/s, sufficient to permit people to do network testing, but not 
to overwhelm your control plane link or CPU.  For others, like BGP from
established peers, you take what they give you, because it speeds up 
convergence.  BGP SYNs, rate-limit.  And so on)

> I'm also curious about how you LB packets (either at layer 2 or
> layer 3) that don't have TCP or UDP headers, or aren't IP. I've
> seen LB become completely ineffective at layer 2 because a customer
> was using MPLS between two routers, so no MAC address variation,
> and no IP addresses and no TCP or UDP ports to look at.

This is actually a hard problem in the generic case.  If the hardware can
find reasonable discriminators, you can hash based on that (like, MAC 
addresses in Ethernet-over-MPLS packets) - if not, fall back to flow label
or "src/dst MAC address", which is usually a spectacularily bad idea 
between routers.

[..]
> "LAG is good, but it???s not as good as a fatter pipe."

True.  But sometimes life is not nice, and you have to live with what
you have - like, you can get 2x 10G wavelength between cities, but not
40G or 100G, or your equipment just cannot handle 40G/100G.

Gert Doering
        -- NetMaster

---

From: Ray Hunter <[email-masked]>
Date: Wed, 22 Apr 2015 12:32:31 +0200

Gert Doering wrote:
> first<n>  bytes (say, 512) and that the final header must be in the
> first fragment.
>
I do think there's a strong need to differentiate between
i)  packets switched by a device (e.g. normal unicast),
ii) packets switched by a device but which require some local action 
(e.g. replying to an ICMP packet with a hop limit expiry, or processing 
hop by hop options which are anyway an allowable exception in RFC7045), and
iii) packets destined for the device itself as an end host (e.g. BGP 
session).

Which to me suggests that the correct fix to this problem is that a 
hardware vendor worried about control plane performance can introduce a 
queueing mechanism between any fast path switch fabric and any slower 
control plane, in a similar way to implementing WRED or other queuing 
mechanisms on outgoing interfaces. Packets containing "excessive EH" 
would then probably be placed in one of the slow/high drop probability 
queues.

So I really do understand the desire to protect any slow path, but I 
don't see this as an IPv6 protocol issue.

IMHO I think this problem was already fixed at the protocol level by 
https://tools.ietf.org/html/rfc7112

---

From: Mark ZZZ Smith <[email-masked]>
Date: Wed, 22 Apr 2015 10:55:26 +0000 (UTC)

From: Gert Doering <[email-masked]>
 To: Mark ZZZ Smith <[email-masked]> 
Cc: Nick Hilliard <[email-masked]>; "[email-masked]" <[email-masked]> 
 Sent: Wednesday, 22 April 2015, 18:51
 Subject: Re: [v6ops] draft-gont-v6ops-ipv6-ehs-in-real-world: clarification text
   
Hi,

/ So would you expect all of your routers to be able to do this level of processing? It seems to me that you'd only need routers that can do that processing facing potential malicious sources e.g. transit, peering, and possibly customer (although customers have a strong disincentive because it is in their interests for the router to stay available). I wouldn't think it would be worth the expense to have this level of hardware filtering available on core routers.

[..]

Gert Doering
        -- NetMaster

---

From: [email-masked]
Date: Wed, 22 Apr 2015 13:25:08 +0200 (CEST)

> So would you expect all of your routers to be able to do this
> level of processing?

Yes.

> It seems to me that you'd only need routers that can do that
> processing facing potential malicious sources e.g. transit, peering,
> and possibly customer (although customers have a strong disincentive
> because it is in their interests for the router to stay
> available).

Customers themselves may have such a disincentive (unless they are
trying to deliberately sabotage the network infrastructure - which has
been known to happen). Customer PCs and other equipment which are
"owned" (i.e. part of a botnet, under control of an external party)
may well have a strong incentive to deliberately create problems.

> I wouldn't think it would be worth the expense to have
> this level of hardware filtering available on core routers.

Reasonable arguments can be made either way (Same level of hardware
filtering for core routers: Potentially more expensive hardware, but
potentially simpler logistics - fewer hardware product variants).

I *might* consider core routers without such hardware filtering *if*
I was running a pure MPLS core, and was reasonably certain customer
traffic wouldn't hit the core routers directly.

Steinar Haug, AS 2116

---

From: Gert Doering <[email-masked]>
Date: Wed, 22 Apr 2015 14:11:03 +0200

Hi,

(re-formatted weird Mark ZZZ quoting style to gain proper attribution)

On Wed, Apr 22, 2015 at 10:55:26AM +0000, Mark ZZZ Smith wrote:
> / So would you expect all of your routers to be able to do this
> level of processing? It seems to me that you'd only need routers
> that can do that processing facing potential malicious sources e.g.
> transit, peering, and possibly customer (although customers have a
> strong disincentive because it is in their interests for the router
> to stay available). I wouldn't think it would be worth the expense
> to have this level of hardware filtering available on core routers.

Unless you define a "core router" to be a pure MPLS P node, which does not 
do *any* IP processing, yes, it will have to do the same thing.

Gert Doering
        -- NetMaster

---

From: Nick Hilliard <[email-masked]>
Date: Wed, 22 Apr 2015 17:55:04 +0100

Mark,

Gert and Steiner have answered many of the issues which you bring up.

> So I'm going to call the emperor naked.

The emperor is not naked.  We live in a real world with real constraints.

> deprecated, so that the TCP and UDP headers are always in the same place in
> the packet, so that hardware can look at them for the purposes of dropping
> them in hardware or as inputs into LB. Is that the case?

Nope, EHs cannot be removed from the spec and no-one is suggesting this.

>  [IPv6 Fixed Hdr] + [IPv6 EHs] + [L4 Hdr] <= hardware inspection limit

I'd suggest that we need a chunk of encapsulation overhead in that
equation.  The hardware inspection limit will vary, but the larger it is,
the more expensive both in terms of financial cost and performance.

> What I'm curious about then is how do you handle DDoSes that are using IP
> fragments, where there are no TCP or UDP headers ports to look at?

s/rtbh or d/rtbh, depending on which looks like it will be more effective.
 But please don't try to imply that because something is broken in ipv4
that we should be ok if it's broken in ipv6 too.  This is a unhelpful fallacy.

> completely ineffective at layer 2 because a customer was using MPLS between
> two routers, so no MAC address variation, and no IP addresses and no TCP or
> UDP ports to look at.

tbh, with great difficulty and expense.  I spend an inordinate amount of
time trying to deal with this particular problem.  And if it's bad for the
L3 ISP stuff I do, it's far worse on an L2 IXP lan.

> member links that are not current members are supposed to be considered by
> the bridge as normal bridge ports, so if you want to avoid loops across the
> links that are candidate members of the LAG, the IEEE expect you to be
> running STP of some form across them...)

yep, bridges require bridging protocols for loop avoidance.

>, I'm a big fan of the quote on the
> last slide of this presentation:
> 

which is fine until the point that a fatter pipe will not handle all the
data you need to transfer.  If I had a choice about using LAGs, I wouldn't
use them.  I don't have that choice.

Nick

---

From: Joe Touch <[email-masked]>
Date: Wed, 22 Apr 2015 10:17:26 -0700

...
> addressed to the routers themselves is not sufficient.

Sure - so here's the solution:

1) declare that your routers won't accept "host" packets (i.e., to their
interface addresses) that use EHs

	if you want to try, extend that to any AS whose routing
	packets you want to DDOS filter (but they need to think this
	is acceptable)

2) if you want to do DDOS filtering, do so on IP addresses for the
specific routers you are trying to protect

	either rate-limit everything to that address, drop
	anything with an EH, rate-limit/drop based on TCP/UDP
	ports, or any combination thereof

I.e., not using EHs is your prerogative, and not forwarding EHs to
others is *their* prerogative, but castrating IPv6 for the entire
Internet is not necessary.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 22 Apr 2015 17:26:43 +0000

Hi, we will want to make use of IPv6 EHs (at least Destination Options
and Fragment Header). Are people trying to get rid of them altogether?

Thanks - Fred
[email-masked]

> -----Original Message-----
> From: v6ops [mailto:[email-masked]] On Behalf Of Joe Touch
> Sent: Wednesday, April 22, 2015 10:17 AM
> To: [email-masked]
> Cc: [email-masked]; [email-masked]; [email-masked];
> [email-masked]

---

From: Gert Doering <[email-masked]>
Date: Wed, 22 Apr 2015 19:54:52 +0200

Hi,

> Internet is not necessary.

I'm not sure what the benfefit is in insisting that IPv6 as currently 
standardized is The Only And Proper Way To Do Networking?  It was designed
20 years ago, and some of the assumptions from back then are turning out
to cause enormous amount of friction today.

"Forwarding packets with EH" is one aspect of this, "RA guard" is 
another one, and "correctly operating firewalls" (be it host-based or
traditional perimeter based) is a third one.

So what's wrong with just fixing the damn protocol, and doing something
productive with our time, like, "play with our kids"?

Gert Doering
        -- NetMaster

---

From: Joe Touch <[email-masked]>
Date: Wed, 22 Apr 2015 11:01:48 -0700

> to cause enormous amount of friction today.

And might not tomorrow.

There are a lot of reasons why EHs are critical - source fragmentation
is one very important one.

> productive with our time, like, "play with our kids"?

Nothing is wrong with fixing something that's broken.

That includes routers that don't support EH, and maybe the long shopping
list of "what if" extensions that we might not need. It also might
include a "jump to the transport header" EH.

But, FWIW, none of these problems are going away even without EHs. Today
it's EHs, tomorrow it'll be tunnel headers (which have the same
problem), and the next it'll be encryption.

It's always useful to explore the entire constellation of what needs to
be done. If it's critical to update IPv6, that ought to be done in
INTAREA, not as a convenience for operators, though.

Joe

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 23 Apr 2015 11:13:37 +1200

On 22/04/2015 22:32, Ray Hunter wrote:

...
> IMHO I think this problem was already fixed at the protocol level by https://tools.ietf.org/html/rfc7112

Exactly, which is one of the many reasons why we need RFC2460bis.

   Brian