Subject: Re: [v6ops] Extension Headers / Impact on Security Devices

From: Enno Rey <[email-masked]>
Date: Thu, 11 Jun 2015 18:58:58 +0200

Eric, All,

On Mon, May 18, 2015 at 06:07:45AM +0000, Eric Vyncke (evyncke) wrote:
> Let me chime in ;-)
> 
> On this topic, I am suffering of schizophrenia with a double personality...
> 
> - security person: I hate when there are too many options and sub-options
> and my usual recommendation is to use white list approach at the
> destination (being plain layer-4 ACL or more content filtering) and at the
> source (let's try to cut down covert channels): block unused ports, block
> unused protocols (remember IP protocol 41), block unused extension headers
> and drop everything which is not a normal IP packet (from address spoofing
> to invalid EH chain).

the problem here is the definition of "normal IP packet" as of RFC2460.
To illustrate this I just quote from today's Cisco advisory (Cisco IOS XR Software Crafted IPv6 Packet Denial of Service Vulnerability) on packets potentially crashing CRS-3 line cards:

"The vulnerability is due to incorrect processing of an IPv6 packet carrying IPv6 extension headers that are valid but unlikely to be seen during normal operation. An attacker could exploit this vulnerability by sending such an IPv6 packet to an affected device that is configured to process IPv6 traffic. An exploit could allow the attacker to cause a reload of the line card, resulting in a DoS condition."

two question come to mind here:

- is a "valid but unlikely" extension header chain "normal"?
- what ("combination of FW & IPS or whatever") would you put in front of a CRS?

my (sad) expectation is that we'll see much more of these (types of) issues in the future. given the current level of freedom that the RFC2460 leaves (see also discussion/picture in http://www.insinuator.net/2015/06/is-ipv6-more-secure-than-ipv4-or-less/) "properly parsing an IPv6 packet, let alone in wire speed" seems a pretty much unsolvable task to me.

That said I still fail to see any useful extension header besides AH&ESP (not even FH) to be transported in the Internet.

best

Enno

 If your firewall cannot implement this policy, then
> it is time to change or to use a combination of FW & IPS or whatever.
> 
> - network person: as I will live with IPv6 for the rest of my life, then I
> want a way to extend it and I need any packet to travel to my source to my
> destination. Any IPv6 packet should be able to traverse the
> Internet/private network as long as its layer-3 header is valid
> (filtering/dropping can be done exceptionally for good operational
> reason). Did we remember the IPv4 teardrop attack? It is blocked by
> default by most routers for years ;-)
> 
> Bugs will plague us for ever... And make our life more complex than the
> above description of course ;-)
> 
> -??ric
> 
> On 17/05/15 20:43, "Silvia Hagen" <[email-masked]> wrote:
> 
> >Hi
> >
> >I keep stumbling about that "recommendational wording" in RFC 2460
> >everytime I teach it.
> >
> >Couldn't we update RFC2460 and make this list a strict order?
> >
> >I would want my firewall to notify me if the EHs in a packet do not
> >follow the list.
> >And limiting the number of possible EHs per packet might be a good idea.
> >
> >Silvia
> >
> >-----Urspr??ngliche Nachricht-----
> >Von: ipv6-wg [mailto:[email-masked]] Im Auftrag von Benedikt
> >Stockebrand
> >Gesendet: Sonntag, 17. Mai 2015 18:39
> >An: [email-masked]
> >Betreff: Re: [ipv6-wg] Extension Headers / Impact on Security Devices
> >
> >Hi Enno and list,
> >
> >Enno Rey <[email-masked]> writes:
> >
> >> hope everybody had a great #RIPE70 meeting. We did!
> >> Many thanks to the organizers and chairs!
> >
> >and thanks to the actual speakers as well the speakers we had to turn
> >down due to time constraints, too:-)
> >
> >> If the chairs consider this appropriate we will happily give a
> >> presentation on this stuff in Bucharest or at another occasion.
> >
> >Sounds good to me!
> >
> >> - looking at the "liberty" RFC2460 provides as for ext_hdrs (wrt to
> >> their number, order[...]
> >
> >Actually, as far as I'm concerned that's the real core of the problem.
> >Or more specifically, the first two lines of RFC 2460, section 4.1:
> >
> >   When more than one extension header is used in the same packet, it is
> >   recommended that those headers appear in the following order:
> >
> >followed on the next page by
> >
> >   Each extension header should occur at most once, except for the
> >   Destination Options header which should occur at most twice (once
> >   before a Routing header and once before the upper-layer header).
> >
> >Note in particular that these are not even RFC 2119 "SHOULD" or
> >"RECOMMENDED" and such.
> >
> >The impact here is actually at least twofold:
> >
> >- It is impossible to implement this as a simple pipeline architecture
> >  in hardware; at least for cases deviating from the "recommendations"
> >  above this effectively becomes either excessively complex to implement
> >  in hardware or an invitation to DoS when implemented in software on an
> >  otherwise hardware router.
> >
> >- As I understand it, at least some of the issues you have found are
> >  effectively based on violating the second paragraph quoted, making it
> >  impossible to come up with a lower bound on how long the header chain
> >  can actually get and therefore leading to the fragmentation related
> >  attacks and similar you have discovered.
> >
> >The original idea was that the extension headers are processed strictly
> >in the order they occur, so one question to ask is if there is any valid
> >reason to violate these "recommendations" for other than malicious
> >purposes.
> >
> >
> >Cheers,
> >
> >    Benedikt
> >
> >-- 
> >Benedikt Stockebrand,                   Stepladder IT Training+Consulting
> >Dipl.-Inform.                           http://www.stepladder-it.com/
> >
> >          Business Grade IPv6 --- Consulting, Training, Projects
> >
> >BIVBlog---Benedikt's IT Video Blog: http://www.stepladder-it.com/bivblog/
> >
> >
> 

-- 
Enno Rey

ERNW GmbH - Carl-Bosch-Str. 4 - 69115 Heidelberg - www.ernw.de
Tel. +49 6221 480390 - Fax 6221 419008 - Cell +49 173 6745902 

Handelsregister Mannheim: HRB 337135
Geschaeftsfuehrer: Enno Rey

=======================================================
Blog: www.insinuator.net || Conference: www.troopers.de
Twitter: @Enno_Insinuator
=======================================================

---

From: "Eric Vyncke (evyncke)" <[email-masked]>
Date: Mon, 15 Jun 2015 22:15:36 +0000

Enno,

As you probably know, Cisco handles PSIRT issues with care, so, I have no
clue on this specific advisory. My own (educated) guess is that the packet
causing a LC reload is fully RFC 2460 compliant but unusual, so, probably
(a guess again) a combination of extension headers triggering a bug.

In this specific case, I would not blame IPv6 or RFC 2460 but rather my
own company (even if the affected versions were quite old as new IOS-XR
versions appear to be immune to this bug). And I appreciate the humor in
your rhetorical question about which FW could protect a CRS... Air gap
probably :-)

More generally, regarding the 'parsing of the header' and performance, my
understanding (and I am NOT a HW engineer) is that there are multiple ways
to parse the header chain:
- purely software in low-end devices, should have a marginal performance
impact
- specific ASIC in middle-end devices (read high end enterprise), probably
some limitations and heavy performance impact (but again within one
organization, so, easier to fix the root cause)
- a lot of specialized network processors in high-end devices (SP gears),
which is a similar case as the 'low-end device', performance impact but
marginal as parsing the chain of headers is not that hard after all _when_
coded correctly

As the extension headers are specific to IPv6, this is an area when we
will all learn (and have learned already) a lot of things...

I respectfully disagree with your last sentence. Of course, destination
AS/host SHOULD only accept useful, known and legit ext headers; but, why
would an ISP filter on ext headers if they do not harm their own infra or
a customer infra? Do you envision an Internet where only TCP/443 and
UDP/43 would be accepted?

Last point, the Internet will have to use IPv6 for 10 or 20 years at
least. If the IETF/ISP block all new _options_ in _existing_ extension
headers, then we will be stuck in 1997 for network innovation. While I do
not like taking security risks, I even fear more the lack of innovation.

Let's talk on Thursday at Silvia's IPv6 Conference in Zurich ;-)

-éric

On 11/06/15 11:58, "Enno Rey" <[email-masked]> wrote:
>To illustrate this I just quote from today's Cisco advisory (Cisco IOS XR
>Software Crafted IPv6 Packet Denial of Service Vulnerability) on packets
>potentially crashing CRS-3 line cards:
>
>"The vulnerability is due to incorrect processing of an IPv6 packet
>carrying IPv6 extension headers that are valid but unlikely to be seen
>during normal operation. An attacker could exploit this vulnerability by
>sending such an IPv6 packet to an affected device that is configured to
>process IPv6 traffic. An exploit could allow the attacker to cause a
>reload of the line card, resulting in a DoS condition."
>
>my (sad) expectation is that we'll see much more of these (types of)
>issues in the future. given the current level of freedom that the RFC2460
>leaves (see also discussion/picture in
>http://www.insinuator.net/2015/06/is-ipv6-more-secure-than-ipv4-or-less/)
>"properly parsing an IPv6 packet, let alone in wire speed" seems a pretty
>much unsolvable task to me.
>
>That said I still fail to see any useful extension header besides AH&ESP
>(not even FH) to be transported in the Internet.
>
>best
>
>Enno
>
>
>
>
>
>
>
>_______________________________________________
>v6ops mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/v6ops

---

From: Ca By <[email-masked]>
Date: Mon, 15 Jun 2015 15:32:11 -0700

> not like taking security risks, I even fear more the lack of innovation.

I keep hearing this and it is very bizarre to hear

With regards to 1997, believing that there will be innovation in the narrow
waist of hour glass (network layer), that is very 1997. The network layer
is stable and boring.... Hence it being narrow.  Innovation is welcome
everywhere except network in the e2e model

> On 11/06/15 11:58, "Enno Rey" <[email-masked] <javascript:;>> wrote:
> >
> >> On 17/05/15 20:43, "Silvia Hagen" <[email-masked] <javascript:;>>
> wrote:
> >>
> >> >Hi
> >> >
> >> >Von: ipv6-wg [mailto:[email-masked] <javascript:;>] Im
> Auftrag von Benedikt
> >> >Stockebrand
> >> >An: [email-masked] <javascript:;>
> >> >Enno Rey <[email-masked] <javascript:;>> writes:
> >> >
> >[email-masked] <javascript:;>
> [email-masked] <javascript:;>

---

From: Mark ZZZ Smith <[email-masked]>
Date: Tue, 16 Jun 2015 05:11:57 +0000 (UTC)

From: Ca By <[email-masked]>
 To: Eric Vyncke (evyncke) <[email-masked]> 
Cc: "[email-masked]" <[email-masked]>; "[email-masked]" <[email-masked]> 
 Sent: Tuesday, 16 June 2015, 8:32
 Subject: Re: [v6ops] Extension Headers / Impact on Security Devices

Enno,

I keep hearing this and it is very bizarre to hear 
With regards to 1997, believing that there will be innovation in the narrow waist of hour glass (network layer), that is very 1997. The network layer is stable and boring.... Hence it being narrow.  Innovation is welcome everywhere except network in the e2e model
/ Absence of evidence is not evidence of absence. 

/ Consider what the network and host situation was in 1997 (wired network access, desktops outselling laptops, dialup Internet for residential, corporates in total probably being the largest spenders on Internet access) to today and the near future (wireless networks, smartphones/tablets being the dominant Internet access device in some markets - and they're multi-homed, M2M/IoT considered to be place where the greatest expansion of Internet connected devices is going to occur in the near future.)
/ IPv6 has had relatively little adoption, and it is only starting to increase quite rapidly in the last few years and that is mainly because of mobile networks. There hasn't and still doesn't seem to be much thinking about how IPv6's differences from IPv4's can be taken advantage of yet - an "IPv4 first" or an "IPv4 == IPv6" mindset seems to be still very common (I think the network virtualisation space is where this is might be most prominent - I think most of the technologies they're inventing require fork lift network upgrades, so the costs of deploying IPv6 at the same time become a lot less significant. So if there is a "cheap" opportunity to deploy IPv6, and IPv6 is going to be the long term network layer 3, why not optimise for it now? That was the thinking behind my "Enhancing Virtual Network Encapsulation with IPv6" draft) 
/ We don't want to disable innovation opportunities that IPv6 may provide before people have arrived at an "IPv6 first" or "IPv4 != IPv6" mindset.  

-éric

---

From: Ca By <[email-masked]>
Date: Tue, 16 Jun 2015 06:08:17 -0700

On Monday, June 15, 2015, Mark ZZZ Smith <[email-masked]> wrote:

>
>   ------------------------------
>  *From:* Ca By <[email-masked]
> <javascript:_e(%7B%7D,'cvml',[email-masked]');>>
> *To:* Eric Vyncke (evyncke) <[email-masked]
> <javascript:_e(%7B%7D,'cvml',[email-masked]');>>
> *Cc:* "[email-masked] <javascript:_e(%7B%7D,'cvml',[email-masked]');>" <
> [email-masked] <javascript:_e(%7B%7D,'cvml',[email-masked]');>>; "
> [email-masked] <javascript:_e(%7B%7D,'cvml',[email-masked]');>" <
> [email-masked] <javascript:_e(%7B%7D,'cvml',[email-masked]');>>
> *Sent:* Tuesday, 16 June 2015, 8:32
> *Subject:* Re: [v6ops] Extension Headers / Impact on Security Devices
>
>
>
> On Monday, June 15, 2015, Eric Vyncke (evyncke) <[email-masked]
> <javascript:_e(%7B%7D,'cvml',[email-masked]');>> wrote:
>
> Enno,
>
> With regards to 1997, believing that there will be innovation in the
> narrow waist of hour glass (network layer), that is very 1997. The network
> layer is stable and boring.... Hence it being narrow.  Innovation is
> welcome everywhere except network in the e2e model
>
> / Consider what the network and host situation was in 1997 (wired network
> access, desktops outselling laptops, dialup Internet for residential,
> corporates in total probably being the largest spenders on Internet access)
> to today and the near future (wireless networks, smartphones/tablets being
> the dominant Internet access device in some markets - and they're
> multi-homed, M2M/IoT considered to be place where the greatest expansion of
> Internet connected devices is going to occur in the near future.)
>
> / IPv6 has had relatively little adoption, and it is only starting to
> increase quite rapidly in the last few years and that is mainly because of
> mobile networks. There hasn't and still doesn't seem to be much thinking
> about how IPv6's differences from IPv4's can be taken advantage of yet - an
> "IPv4 first" or an "IPv4 == IPv6" mindset seems to be still very common (I
> think the network virtualisation space is where this is might be most
> prominent - I think most of the technologies they're inventing require fork
> lift network upgrades, so the costs of deploying IPv6 at the same time
> become a lot less significant. So if there is a "cheap" opportunity to
> deploy IPv6, and IPv6 is going to be the long term network layer 3, why not
> optimise for it now? That was the thinking behind my "Enhancing Virtual
> Network Encapsulation with IPv6" draft)
>
> / We don't want to disable innovation opportunities that IPv6 may provide
> before people have arrived at an "IPv6 first" or "IPv4 != IPv6" mindset.
>
>
>
I have heard those stories, i don't find them compelling.

CB

> [email-masked] <javascript:_e(%7B%7D,'cvml',[email-masked]');>

---

From: Jen Linkova <[email-masked]>
Date: Tue, 16 Jun 2015 21:02:35 +0200

> the problem here is the definition of "normal IP packet" as of RFC2460.

The problem here is what one might mean by "normal" (from Oxford dictionary):
1)  conforming to a standard;
2) usual, typical, or expected;

> - is a "valid but unlikely" extension header chain "normal"?

It is "normal" if you define "normal" as "conforming to a standard",
but it's not "normal" if you define "normal" as "usual/typical".

> - what ("combination of FW & IPS or whatever") would you put in front of a CRS?

Nothing. There is smth to put *on* CRS however - the image which
contains the fix...
I do not think we should blame a protocol for software bugs. I've seen
router crashes caused by ICMP and BGP packets. Shall we go ahead and
start discussing deprecation of the two above mentioned protocols? ;)
Especially taking into account than some people like filtering ICMP on
the edge of their networks? ;)

> my (sad) expectation is that we'll see much more of these (types of) issues in the future. given the current level of freedom that the RFC2460 leaves (see also discussion/picture in http://www.insi…

(shameless plug) a group of enthusiasts have just submitted a new
version of document which discusses exactly this problem:

https://tools.ietf.org/html/draft-wkumari-long-headers-03

Comments are appreciated...

-- 
SY, Jen Linkova aka Furry

---

From: Joe Touch <[email-masked]>
Date: Tue, 16 Jun 2015 13:11:54 -0700

> 2) usual, typical, or expected;

That's the trouble with extensions. You can't expect them until they're
developed AND deployed, so initially they're never "expected" in terms
of actual traffic.

Except that we SHOULD expect *everything* that's in a spec.

Joe

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 17 Jun 2015 13:24:03 +1200

On 17/06/2015 07:02, Jen Linkova wrote:

...

In REQ-2 on HbH headers, you say:

>  The forwarder MUST	
>  process each option as specified in Section 4.2 of [RFC2460].

That aspect of RFC 2460 was fundamentally changed by RFC 7045. And of
course this is the issue addressed by draft-baker-6man-hbh-header-handling.

Personally I still think RFC 7045 is the most realistic on this point,
but Fred would like things to get better ;-).

I'm sure other things in the long-headers draft need revising as a
result of RFC 7045, since its whole topic is the handling of extension
headers ("This document updates RFC 2460 to clarify how intermediate
nodes should deal with such extension headers and with any that are
defined in the future.")

Y'all also need to take account of RFC 7112, which forbids fragmented
header chains.

    Brian

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Wed, 17 Jun 2015 04:45:15 +0000

> On Jun 16, 2015, at 6:24 PM, Brian E Carpenter <[email-masked]> wrote:
> 

And I haven't finished with Dennis Ferguson's comment.

Bottom line, if one accepts the present status quo as the state forever, then we should stop with RFC 7045, and (with Fernando) agree to deprecate all extension headers. I'd like to not do that, and the only way I see to not do that is to not accept the status quo.

---

From: Ca By <[email-masked]>
Date: Tue, 16 Jun 2015 21:52:22 -0700

On Tuesday, June 16, 2015, Fred Baker (fred) <[email-masked]> wrote:

>
> > On Jun 16, 2015, at 6:24 PM, Brian E Carpenter <
> [email-masked] <javascript:;>> wrote:
> >
> Bottom line, if one accepts the present status quo as the state forever,
> then we should stop with RFC 7045, and (with Fernando) agree to deprecate
> all extension headers. I'd like to not do that, and the only way I see to
> not do that is to not accept the status quo.
>

So you build a bridge to nowhere?

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Wed, 17 Jun 2015 07:41:33 +0000

I'm of course missing the preceding email. No problem with the cross-posting, but let's get the question on the table? Is this section 4.1's statement that "it is recommended that those headers appear in the following order"?

I have a hunch that an internet draft requiring headers to be in the listed order would be looked at with approbation. The issue, if any, would likely have regard to repeated Destination Option headers. However

seems to me fairly definitive.

Fernando would like to see a specification of how long the entire list of extension headers may be, in terms of "how large of a hardware buffer has to be implemented in a switch?" That has a couple of issues related to "why does one have to ask the question?" The hardware should definitely be large enough to read in the IPv6 header, the HBH Header, and the Routing Header. After that, the only host that SHOULD need to read it all in is the host itself, and the host can do its processing from RAM. There is one "gotcha" case, which is when someone uses an ACL (disallowing Fragmentation Headers, perhaps, or looking for the TCP port, meaning one has to ask whether a given packet is TCP and what its port number is). The firewall is going to have to chase down the chain of extension headers.

How big is a HBH header? How big is a routing option such as the Segment Routing option? If that's a list of 27 services, each with a 16 byte IPv6 address...

> On May 17, 2015, at 12:18 PM, Enno Rey <[email-masked]> wrote:
> 
> Hi Silvia,
> 
> On Sun, May 17, 2015 at 06:43:11PM +0000, Silvia Hagen wrote:
>> Hi
>> 
>> I keep stumbling about that "recommendational wording" in RFC 2460 everytime I teach it.
>> 
> good luck bringing this suggestion to IETF 6man... ;-)
> 
> 
> 
>> 
>> I would want my firewall to notify me if the EHs in a packet do not follow the list.
> 
> actually when we tested a number of commercial firewalls in late 2013 (results here: https://www.ernw.de/download/TR14_IPv6SecSummit_AAtlasis-RISC-project_results.pdf) it turned out that pretty much all of them follow a (from our perspective "reasonable approach", that is dropping "unusal combinations or number of ext_hdrs" by default. (which, of course, violates RFC 2460. but apparently all major vendors follow a line of reasoning similar to ours).
> so, in short, firewalls "are not affected".
> 
> 
> yes, that _would actually be_ a good idea.
> 
> 
> I'm cross-posting this to v6ops as there's a related discussion going on right now. pls forgive this potential Usenet etiquette violation.
> 
> best
> 
> Enno
> 
> 
> 
> 
>> 
>> Silvia
>> 
>> -----Urspr?ngliche Nachricht-----
>> Von: ipv6-wg [mailto:[email-masked]] Im Auftrag von Benedikt Stockebrand
>> and thanks to the actual speakers as well the speakers we had to turn down due to time constraints, too:-)
>> 
>> Note in particular that these are not even RFC 2119 "SHOULD" or "RECOMMENDED" and such.
>> 
>> The original idea was that the extension headers are processed strictly in the order they occur, so one question to ask is if there is any valid reason to violate these "recommendations" for other than malicious purposes.
>> 
>> 
>> Cheers,
>> 
>>    Benedikt
>> 
>> --

---

From: Enno Rey <[email-masked]>
Date: Wed, 17 Jun 2015 10:14:25 +0200

Fred, All,

> seems to me fairly definitive.

I'm not so sure about this. First probably a capital "SHOULD" would have been more appropriate. Second - and this is our main point/observation - there's too much space of interpretation for actual implementation.
To underline this I will just cite two somewhat random sections from the study we performed on evading security controls (IDPS systems) by means of extension headers, combined with fragmentation (https://www.ernw.de/download/eu-14-Atlasis-Rey-Schaefer-briefings-Evasion-of-HighEnd-IPS-Devices-wp.pdf). One of the devices (all of them latest code incl. some high-end gear) could be evaded by the following:

"Case 1:
- An IPv6 Routing Header Type 0 in the unfragmentable part is used.
- AND a IPv6 Destination Option header is part of the fragmentable piece of the IPv6 datatagram.
- AND The IPv6 Destinations Option header is padded with six (6) octets of bytes (at least).
- AND The IPv6 datagram is fragmented in 2 fragments.
Case 2:
- An IPv6 Hop-by-Hop Option Header and Routing Header Type 0 in the unfragmentable part is
used
- AND a IPv6 Destination Option header is part of the fragmentable piece of the IPv6 datatagram.
- AND The IPv6 datagram is fragmented in 2 fragments.
Case 3:
- An IPv6 Destination Option Header and Routing Header Type 0 in the unfragmentable part is
used.
- AND a IPv6 Destination Option header is part of the fragmentable piece of the IPv6 datatagram.
- AND The IPv6 datagram is fragmented in 2 fragments.
Case 4:
- An IPv6 Hop-by-Hop, Destination Option Header and Routing Header Type 0 in the
unfragmentable part is used
- AND a IPv6 Destination Option header is part of the fragmentable piece of the IPv6 datatagram.
- AND The IPv6 datagram is fragmented in 2 fragments."

Another one by the following:

"a. The unfragmentable part consists of three (3) Destination Option headers
b. The fragmentable part consists of two (2) Destination Option headers plus the layer 4 header.
c. The aforementioned datagram is split in two fragments."

It should be noted that most operating systems happily accept and process (incl., ofc, reassembly) packets crafted in such ways. [see results section in https://www.ernw.de/download/Advanced%20Attack%20Techniques%20against%20IPv6%20Networks-final.pdf].
It should further be noted that some of the techniques we used would even have worked in settings where RFC 7112 would have been strictly applied. 

> Fernando would like to see a specification of how long the entire list of extension headers may be, in terms of "how large of a hardware buffer has to be implemented in a switch?" That has a couple…

Which might be a quite difficult task (not least performance-wise) once this chain, by definition, is abitrarily long.

best

Enno

-- 
Enno Rey

---

From: Jen Linkova <[email-masked]>
Date: Wed, 17 Jun 2015 12:01:33 +0200

> Fernando would like to see a specification of how long the entire list of extension headers may be, in terms of "how large of a hardware buffer has to be implemented in a switch?" That has a couple…

I agree. There is a definitely a big difference in requirements for
"traditional routers" and devices which are going to implement ACLs
(and other types of "multifields classifications" (as QoS based other
data than IP header). IMHO it's reasonable to assume that one might
need different hardware for "just routing" and enhanced QoS/ACL
services (it's a case nowadays anyway). For the latter case the device
has to either, as you said, chaise the whole chain down (which might
be as long as the whole packet) or have a way to deal with packets it
could not parse (applying a specific policy to such packets).

> How big is a HBH header? How big is a routing option such as the Segment Routing option? If that's a list of 27 services, each with a 16 byte IPv6 address...

I'd like to point out that the problem is not specific to IPv6 at all.
How deep is MPLS label stack? Where are TCP flags or port number in
the packet (so I can match 'tcp established' or 'tcp 443')? oops, we
don't know....it depends...so some linecards do not copy enough data,
some (newer ones) do.
We do not know (could not know) exact numbers, the requirements are
changing, so does the hardware we deploy in out network. It's a kind
of race ;) and as long as there is a way to predict/configure the
behavior of the device when it could not fully parse the packet - I'm
fine with it..

---

From: [email-masked]
Date: Wed, 17 Jun 2015 14:02:35 +0200 (CEST)

> > Fernando would like to see a specification of how long the entire list of extension headers may be, in terms of "how large of a hardware buffer has to be implemented in a switch?" That has a coup…

You may call it a firewall. I call it a "border router" - and the
difference between this and a firewall is striking. Most importantly,
my border routers don't keep state for the sessions passing through
them. However, I *do* expect them to handle basic stateless ACLs,
including L4 info (port numbers). For IPv4 and IPv6.

> services (it's a case nowadays anyway).

You may feel it is reasonable. Not everybody agrees. If we compare
with IPv4: All modern routers I know of (including high speed boxes
with multiple 10G and 100G ports) are able to handle stateless ACLs 
based on IPv4 addresses and port numbers. The boxes with multiple
10G and 100G ports process these ACLs at line rate. I don't pay extra
for this functionality - possibly because a box *without* such
functionality would have a limited market.

> For the latter case the device

A small but possibly relevant digression here:

My border routers (which happen to be Juniper routers) are configured
to disallow source routing. This has been the Juniper *default* since
around 2008, see

http://www.juniper.net/techpubs/software/junos/junos85/swconfig85-routing/frameset.html

I expect (but do not know with certainty) that Juniper changed this
source routing default behavior based on customer feedback.

Back to IPv6: I might allow "interesting" IPv6 extension headers within
my own AS - because in such cases I have much more control. There is
no way I'm going to allow IPv6 packets with long chains of "interesting"
IPv6 header chains to pass my border routers. Either they have short
enough header chains that my border routers can inspect the L4 info at
line rate - or they get dropped.

> don't know....it depends...so some linecards do not copy enough data,

The MPLS label stack is normally used with a single provider - thus
the provider is in control.

I agree that the IPv4 packet may have options, making it variable
length. However the length is still limited by the IHL field, which
has a max value of 15 (60 bytes).

Steinar Haug, AS 2116

---

From: Nick Hilliard <[email-masked]>
Date: Wed, 17 Jun 2015 13:19:16 +0100

> don't know....it depends...so some linecards do not copy enough data,

bear in mind that each entry in the mpls label stack will be 4 octets,
which gives.  An EH could be orders of magnitude larger.

Nick

---

From: Jen Linkova <[email-masked]>
Date: Wed, 17 Jun 2015 14:40:36 +0200

> which gives.  An EH could be orders of magnitude larger.

Sorry, I should have made my point clear. What 'm trying to say is that:
1) if the current approach of 'copy X bytes to on-chip memory' there
is always a chance to receive a valid packet which requires more than
X bytes to be copied for correct processing.Therefore I believe we
should discuss how to handle such situation in general instead of
trying to find a magic value of X (I'd suggest 42 then ;)) and limit
all headers length by that number.
2) while just a few years ago you might see linecards which could look
at...how many? 64 bytes? into packets, nowadays they are replaced with
linecards which are able to look much deeper. So while protocol
designers should be aware of such limitation, absolute numbers are
increasing and the situation is improving (btw my measurements do show
that packets with 8 / 16 bytes EHs have much lower drop rate than
packets with 512/1024 bytes EHs, which I dare attribute to the hw
limitations).
3) I assume that vendors are developing hardware and software to meet
customers needs (more or less ;)) - 5 years ago who would need to look
deeper than 5 labels? so some linecards were quite upset to see such
deep label stack and might crash; now I'm told that 'we can inspect 10
labels easily because there are use cases'. Why could not we expect
smth like that happening if/when a use case for EH arises?

---

From: Jen Linkova <[email-masked]>
Date: Wed, 17 Jun 2015 15:22:27 +0200

On Wed, Jun 17, 2015 at 2:02 PM,  <[email-masked]> wrote:
>> IMHO it's reasonable to assume that one might

[skip]

> has a max value of 15 (60 bytes).

I'm glad you mentioned 60 bytes ;) Because there are a lot of
reasonably modern hardware around
which copies 64 bytes on-chip. Which means if you happen such hardware
in your network
and your stateless ACL have 'match tcp flags' rules, you might get
quite unexpected results processing packets with
60 bytes IPv4 header....So, while it might be perfectly fine to have
such cars in the core, I'd expect people not to install then at the
border routers
which are supposed to perform enhanced ACL services. It was my point.

So we all agree that 'variable length is OK as long as our hardware
can look deep enough'? And what people are complaining about is exact
number? Which we do not know yet for IPv6 EHs?

---

From: [email-masked]
Date: Wed, 17 Jun 2015 15:27:50 +0200 (CEST)

> number? Which we do not know yet for IPv6 EHs?

Agreed, variable length *by itself* is not the problem.

I see *large* variable length headers, in combination with complex
parsing rules, as the problem.

Steinar Haug, AS 2116

---

From: Enno Rey <[email-masked]>
Date: Wed, 17 Jun 2015 15:33:28 +0200

Hi,

> parsing rules, as the problem.

(*large* variable headers) exactly, plus fragmentation and "ambiguities" wrt fragmentable vs. unfragmentable part and how headers point to the next one once there's a cut between them due to fragmentation. the, what we consider, "problem space" is much larger, unfortunately.

thanks

Enno

-- 
Enno Rey

---

From: Jen Linkova <[email-masked]>
Date: Wed, 17 Jun 2015 15:41:01 +0200

On Wed, Jun 17, 2015 at 3:33 PM, Enno Rey <[email-masked]> wrote:
> (*large* variable headers) exactly, plus fragmentation

Fragmentation is not specific to IPv6, is it? And, as the fragment
header itself is just 8 bytes (AFAIR, too lazy to check ;)) - I'd not
classify it as *large* variable header.

> and "ambiguities" wrt fragmentable vs. unfragmentable part and how headers point to the next one once there's a cut between them due to fragmentation. the, what we consider, "problem space" is much larger, unfortunately.

Has not RFC7112 addressed this particular problem?

---

From: Enno Rey <[email-masked]>
Date: Wed, 17 Jun 2015 16:00:32 +0200

Hi,

On Wed, Jun 17, 2015 at 03:41:01PM +0200, Jen Linkova wrote:
> Fragmentation is not specific to IPv6, is it?

right
[even though I'm in the camp of "deprecate fragmentation as a whole" ;-), but let's not open this can of worms here].
BUT: fragmentation becomes a (then somewhat IPv6 specific) huge problem space once you have a datagram containing, say, 20 extension headers of partially variable lenght & pretty much arbitrary order, split into, say, 50 fragments. [again, see https://www.ernw.de/download/eu-14-Atlasis-Rey-Schaefer-briefings-Evasion-of-HighEnd-IPS-Devices-wp.pdf].

Such a datagram would be fully valid as of today/RFC2460.

Yes, we're aware of RFC7112. It's just: no OS we know and no devices we're aware of (feel free to provide pointers) implement RFC 7112 as of today. but many attack tools implement the techniques mentioned above. Which is why quite some operators (in particular, but not only) from enterprise and managed service provider/cloud space drop all EHs except, maybe, AH+ESP.

thanks

Enno

 And, as the fragment

-- 
Enno Rey

---

From: Ca By <[email-masked]>
Date: Wed, 17 Jun 2015 08:11:27 -0700

On Tue, Jun 16, 2015 at 9:52 PM, Ca By <[email-masked]> wrote:

>> [email-masked]> wrote:
>> >

Responding to my own comment.

IPv6 is serious business now.  That means it needs to be the narrow waist
of the internet that is small on services and large on stability and
predictability.

Please review
https://www.iab.org/wp-content/IAB-uploads/2011/03/hourglass-london-ietf.pdf

For the folks looking for extension header innovation, would you be willing
to work on IP version X instead of IPv6?  Or perhaps you can use the Class
E IPv4 space for your innovation?

Serious.  IPv6 is not a place for innovation at the Network / Internet
layer.  Attempts to do so achieve the results in
draft-ietf-v6ops-ipv6-ehs-in-real-world-00 also referenced in slide 7

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Wed, 17 Jun 2015 16:46:03 +0000

> On Jun 17, 2015, at 1:14 AM, Enno Rey <[email-masked]> wrote:
> 
> Fred, All,
> 
> I'm not so sure about this. First probably a capital "SHOULD" would have been more appropriate.

RFC 2460 doesn't use the RFC 2119 conventions. Yes, it's hard to remember now, but RFC 2119 conventions weren't always the rule; they derived from similar usage in RFCs 1122/1123 and 1812. History lesson in the presence of adult beverages if that's interesting.

> Second - and this is our main point/observation - there's too much space of interpretation for actual implementation.

Cutting to the chase, on the surface it looks like most of the cases you specified conform to the sequence rule, with the exception of one that has three Destination Option Headers. What I *think* you're pointing out is (from your PDF) that this is part of a sequence of messages, one of which is fragmented, accepted by the intrusion detector, and not accepted by the ultimate host (perhaps a TCP checksum failed?), and as a result bypasses the signature that the intrusion detector is looking for. The issue wasn't that the sequence was incorrect, in most cases; it was that the intrusion detector made a different choice than the end host.

Tell me this. Would you be happier if the fragmentation rule said that the first fragment had to contain the entire IPv6 header, plus the transport layer header (for ACL support)? I think Fernando would support such a statement (I think I have "heard" him make such a statement).

---

From: Jen Linkova <[email-masked]>
Date: Wed, 17 Jun 2015 18:54:59 +0200

On Wed, Jun 17, 2015 at 3:24 AM, Brian E Carpenter
<[email-masked]> wrote:
> That aspect of RFC 2460 was fundamentally changed by RFC 7045.

Oh, very good point. Thanks for pointing this out. Looks like it does
cover our REQ2 (but requires different behavior).

So,  Section 2.1 RFC7045 says about *all* EHs:

"If a forwarding node discards a packet containing a standard IPv6
   extension header, it MUST be the result of a configurable policy and
   not just the result of a failure to recognise such a header. "

and then, in Section 2.1 for HbH:
"The IPv6 Hop-by-Hop Options header SHOULD be processed by
   intermediate forwarding nodes as described in [RFC2460].  However, it
   is to be expected that high-performance routers will either ignore it
   or assign packets containing it to a slow processing path. ".

I read this as 'if a router does not recognize/parse the HbH header,
it MUST not discard it unless there is an explicit policy configured.
It may just ignore it or send for "special processing" via slow path.

So it assumes that it is always safe to ignore HbH header (as if all
options in the header had highest-order two bits set to '00') while
our draft proposed quite different approach ("drop and send ICMP
back").

Ignoring HbH header seems to be reasonable and safe if we are sure
that every single existing or to be developed HbH option can be safely
ignored (or options which could be ignored must be in the very
beginning of the header...).

In the light of RFC7045 it looks to me that one possible approach for
REQ2 might be:
- if a forwarder can not parse the HbH header and there is no
explicitly configurable policy, it SHOULD
either:
   -- if the forwarder can not parse any of options or if all parsed
options have highest-order two bits set to '00') - ignore the header;
   -- if the forwarder was able to parse some of options and at least
one of the options has highest-order two bits set to smth except '00'
- discard the packet and send ICMPv6 message if Section 4.2 of RFC
2460 requires so (sending ICMPv6 MAY be subject to a configurable
policy)
or send it to slow path for full header processing (subject to a
configurable policy).

How does that sound?

> nodes should deal with such extension headers and with any that are

Yes, we'll revise it, thanks for the comment.
> Y'all also need to take account of RFC 7112, which forbids fragmented

We do mention 7112 but I agree, we should explicitly mention that
header chain is limited by a packet size...Will update the doc.

---

From: Enno Rey <[email-masked]>
Date: Wed, 17 Jun 2015 19:43:15 +0200

Fred,

> Cutting to the chase, on the surface it looks like most of the cases you specified conform to the sequence rule, with the exception of one that has three Destination Option Headers. What I *think*…

Let's put it slightly different: the intrusion detector is supposed to detect/block certain application layer attacks. Which it does as long as those come in/pass by without extension headers/fragmentation.
Once the attack packets are crafted in certain ways, the intrusion detector let's them through, the end host happily reassembles and gets hit. The end host does so as the overall datagram does not violate RFC 2460's (dubious) liberty. The detector is "blind" as parsing a single datagram split into an arbitrary number of fragments with an arbitrary number of extension headers in a (mostly) arbitrary order is a, well, complex task.
It's not only intrusion detectors facing this type of difficulty, it's other "black list approach" security controls (like so-called First Hop Security features or Infrastructure ACLs), too. [see, for example, http://www.insinuator.net/2015/01/dhcpv6-guard-do-it-like-ra-guard-evasion/).

> 
> Tell me this. Would you be happier if the fragmentation rule said that the first fragment had to contain the entire IPv6 header, plus the transport layer header (for ACL support)?

'm tempted to reply that this would make the Internet world a safer place (without losing any current functionality) and might speeden up overall IPv6 deployment, as some organizations had one (for some of them: severe) concern less. So it's not about my personal happiness; though the factors mentioned might actually contribute to that as well ;-).

>From that angle RFC 7112 is certainly a step into the right direction. It's just not widely implemented - while IPv6 is "finally here" - and it doesn't "solve" certain cases (see the link above).
That's why I think that a major revision of the whole extension header concept is needed.

Best

Enno

-- 
Enno Rey

---

From: Brian Haberman <[email-masked]>
Date: Wed, 17 Jun 2015 13:48:15 -0400

On 6/17/15 1:43 PM, Enno Rey wrote:

> 
> Let's put it slightly different: the intrusion detector is supposed
> to detect/block certain application layer attacks. Which it does as
> long as those come in/pass by without extension

If the device is trying to inspect application-layer data, it might as
well act like the destination and re-assemble the fragments. Yes, this
is costly processing, but it mitigates this bypass mechanism.

Regards,
Brian

---

From: Joe Touch <[email-masked]>
Date: Wed, 17 Jun 2015 11:02:16 -0700

On 6/17/2015 8:11 AM, Ca By wrote:
> IPv6 is serious business now.  That means it needs to be the narrow
> waist of the internet that is small on services and large on stability
> and predictability.
> 
> Please
> review https://www.iab.org/wp-content/IAB-uploads/2011/03/hourglass-london-ietf.pdf
> 
> For the folks looking for extension header innovation, would you be
> willing to work on IP version X instead of IPv6?  Or perhaps you can use
> the Class E IPv4 space for your innovation?
> 
> layer. ...

Hmm.

The design of the IPv6 header chain system was intended to overcome
these sort of limitations of IPv4, e.g., to understand what to do with
unknown options, to separate HBH vs E2E, etc.

IPv6 *is* the innovation, and because of that - for over 15 years -
"you" (commercial vendors) told users how expensive it would be to
implement and it wasn't ready (because "you" were making higher margins
on IPv4 equipment).

Now that we really need it and thus "you" are making money off it, we
(the IETF innovators) are supposed to go away (i.e., use protocol
extensions that don't impact your profit margins because you don't
support them)?

IMO, IPv6 is an E-ticket ride*. If you want to make money off it, IMO
"you" need to expect to pay the price to keep up.

Joe

*https://en.wikipedia.org/wiki/E_ticket

---

From: Enno Rey <[email-masked]>
Date: Wed, 17 Jun 2015 20:04:09 +0200

> is costly processing, but it mitigates this bypass mechanism.

in theory yes. Given the majority of end host operating systems happily wait up to 60 seconds for "that last [potentially 42th] fragment" of a single datagram coming in, this is probably not an option for all those types of security controls (intrusion detectors, First Hop Security mechanisms, Infrastructure ACLs) expected to work mostly in wire speed. It is hence not an option for a number of networks using such techniques which is why they usually drop all extension headers except for AH, ESP and (in a few cases) FH.
Doing so is a reasonable decision from their side and will not exactly encourage widespread development of new services using extension headers. Which then raises the question: what's the benefit of this thing called extension headers which do not provide much use today and might - given there's a growing number of networks acting as described - not provide much use tomorrow? This thing then seems to add an undesirable layer of complexity.

thanks

Enno

-- 
Enno Rey

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Wed, 17 Jun 2015 18:15:00 +0000

> On Jun 17, 2015, at 11:02 AM, Joe Touch <[email-masked]> wrote:
> 

I don't think that's a fair statement. I'd suggest that in discussing people's motivations, you discuss what you know. I could discuss the issues we have had with product line management, but not in ASCII. That's a topic for the bar.

---

From: Tore Anderson <[email-masked]>
Date: Wed, 17 Jun 2015 20:18:09 +0200

* [email-masked]

> Back to IPv6: I might allow "interesting" IPv6 extension headers
> within my own AS - because in such cases I have much more control.
> There is no way I'm going to allow IPv6 packets with long chains of
> "interesting" IPv6 header chains to pass my border routers. Either
> they have short enough header chains that my border routers can
> inspect the L4 info at line rate - or they get dropped.

Hi Steinar,

I wouldn't react to the above if you were operating an enterprise
network, but considering you're an ISP and transit provider, I find the
above rather surprising (and I do not mean that in a good way).

First, your customers might have a perfectly valid reason to send or
receive IPv6 headers with IPv6 extension header chains you apparantly
will drop at your border. FWIW, if I found out that my upstream
arbitrarily dropped packets because they found them "interesting",
breaking my applications in the process, I would not remain a customer
of theirs for long.

Second, the packets might be encrypted using ESP. In that case, you
have absolutely no way of knowing if the extension header chain is long
enough to be "interesting enough to drop", or if the ESP header is the
only extension header there is ("short enough to forward"). What do you
do then?

Third, your border routers obviously cannot inspect the L4 info in an
ESP-encrypted packet at all, line rate or not. Does that mean you drop
all ESP packets at your AS borders? I really hope not.

Tore

---

From: Brian Haberman <[email-masked]>
Date: Wed, 17 Jun 2015 14:22:33 -0400

Enno,

On 6/17/15 2:04 PM, Enno Rey wrote:
>>> Let's put it slightly different: the intrusion detector is
>>> supposed to detect/block certain application layer attacks. Which
>>> it does as long as those come in/pass by without extension
>> 
>> If the device is trying to inspect application-layer data, it might
>> as well act like the destination and re-assemble the fragments.
>> Yes, this is costly processing, but it mitigates this bypass
>> mechanism.
> 
> in theory yes. Given the majority of end host operating systems
> happily wait up to 60 seconds for "that last [potentially 42th]

Yes, that is a common maximum wait time, but I doubt that it is common.
 Unfortunately, I don't see any peer-reviewed literature right off that
talks about the average wait time to perform fragment re-assembly.

> fragment" of a single datagram coming in, this is probably not an
> option for all those types of security controls (intrusion detectors,
> First Hop Security mechanisms, Infrastructure ACLs) expected to work
> mostly in wire speed. It is hence not an option for a number of
> networks using such techniques which is why they usually drop all
> extension headers except for AH, ESP and (in a few cases) FH. Doing
> so is a reasonable decision from their side and will not exactly
> encourage widespread development of new services using extension
> headers. Which then raises the question: what's the benefit of this
> thing called extension headers which do not provide much use today
> and might - given there's a growing number of networks acting as
> described - not provide much use tomorrow? This thing then seems to
> add an undesirable layer of complexity.

Hmm... The old NFR platform, which I think got purchased by Checkpoint,
performed fragmentation re-assembly prior to doing its analysis.  So, at
least some products close that hole.

Regards,
Brian

---

From: Joe Touch <[email-masked]>
Date: Wed, 17 Jun 2015 11:23:10 -0700

On 6/17/2015 11:15 AM, Fred Baker (fred) wrote:
> I don't think that's a fair statement.

Perhaps not from those on the vendor's front lines trying to get support
for these new protocols into products, but from where the customers sit,
that's what we see.

Recall there was a time when IPv4 didn't run fast enough and there were
proposals to "water it down" so hardware could keep up - or simply skip
IP forwarding altogether (via flow/tag switching).

The point of my comment was to note that we're repeating history here.
We're back where innovation is someone else's problem because hardware
can't keep up.

I have more faith in the hardware designers. *Temporary* edicts such as
"keep the chain short" or sensible caveats such as "keep the headers in
the first fragment" are fine, but I hope we don't gut IPv6 for the sole
reason of "kicking the can down the road".

Joe

---

From: Enno Rey <[email-masked]>
Date: Wed, 17 Jun 2015 20:40:26 +0200

Hi Tore,

On Wed, Jun 17, 2015 at 08:18:09PM +0200, Tore Anderson wrote:
> breaking my applications

that brings us directly to the core of the debate: break "exactly which application?". there's no single application/service using EHs other than AH/ESP and, maybe in a few corner cases, FH today and I doubt we'll see some tomorrow (given developing such a thing is heavily de-incentivized by the growing number of operators mostly dropping EHs).

Taking into account that stateless ACLs of all router vendors we tested (results tb published soon) can be avoided/evaded by adding ~5 extension headers to datagrams I fully understand any operator who does not want SSH on its devices to be reachable from the Internet (over v6 with extension headers) and hence acts in a way similar to the one Steinar described.
I doubt Steinar loses many customers (due to "application breakage") by taking that path. In contrary I expect many of his customers valueing the increased level of device & network availability gained by eliminating an entire class of attacks.

best

Enno

-- 
Enno Rey

---

From: Tore Anderson <[email-masked]>
Date: Wed, 17 Jun 2015 21:12:18 +0200

Hi Enno,

* Enno Rey

> > receive IPv6 headers with IPv6 extension header chains you
> > apparantly will drop at your border. FWIW, if I found out that my
> > upstream arbitrarily dropped packets because they found them
> > "interesting", breaking my applications
> 
> that brings us directly to the core of the debate: break "exactly
> which application?"

Well, ESP at least. And, by extension, any protocol that might be
carried inside ESP, so pretty much all of them.

> Taking into account that stateless ACLs of all router vendors we
> tested (results tb published soon) can be avoided/evaded by adding ~5
> extension headers to datagrams I fully understand any operator who
> does not want SSH on its devices to be reachable from the Internet
> (over v6 with extension headers) and hence acts in a way similar to
> the one Steinar described.

There is a big difference between an operator dropping all packets with
EHs that are destined for *his own devices/routers* (I have no problem
with that - your devices, your rules), and an operator that drops
*transit* traffic destined for his customers because his routers cannot
understand/parse/filter its L4/EH payload.

In my opinion, an ISP/IP transit network shouldn't even attempt to
parse the L4/EH payload in customer traffic (except if the customer
asks for it of course), it should just deliver the packets.

> I doubt Steinar loses many customers (due to "application breakage")
> by taking that path. In contrary I expect many of his customers
> valueing the increased level of device & network availability gained
> by eliminating an entire class of attacks.

The first operator I mentioned above won't lose any customers because
his filtering activities doesn't impact customer traffic. The second
operator would lose my business, at least. And probably others' too, as
business customers might want their site2site IPSEC tunnels to work,
residental customers might want their Xbox One online gaming to work,
and so on.

Tore

---

From: [email-masked]
Date: Wed, 17 Jun 2015 22:26:02 +0200 (CEST)

> understand/parse/filter its L4/EH payload.

I certainly appreciate this distinction. However, problems arise in
practice when customers *ask* for various types of protection - which
are most sensibly implemented on border routers.

Just to be clear - we don't drop IPsec traffic today (neither IPv6
nor IPv4). 

> asks for it of course), it should just deliver the packets.

The problem is - the customer *does* ask for it, in many cases.

> residental customers might want their Xbox One online gaming to work,

I agree - IPSEC tunnels and Xbox One online gaming need to work. That
doesn't mean *all* extension header combinations should be expected
to work - and it appears that they often don't.

I expect we'll see much more of these discussions in the coming years,
as IPv6 traffic grows. And I certainly also expect significant size
IPv6 DDoS attacks - at least some of which will probably be based on
extension header manipulation.

Steinar Haug, AS 2116

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 18 Jun 2015 09:13:35 +1200

Catching up on a few points in this thread that went crazy
while I was sleeping...

On 18/06/2015 02:00, Enno Rey wrote:

...
> Yes, we're aware of RFC7112. It's just: no OS we know and no devices we're aware of (feel free to provide pointers) implement RFC 7112 as of today. 

No, it's too new. But I suggest that it gives you license to drop packets
with fragmented header chains, and tell anyone who complains that they
don't conform to the IPv6 standard.

> but many attack tools implement the techniques mentioned above. Which is why quite some operators (in particular, but not only) from enterprise and managed service provider/cloud space drop all EHs except, maybe, AH+ESP.

Whereas dropping *all* EHs breaks the IPv6 standard.

> E IPv4 space for your innovation?

Now that's a polemic, not an argument. But since you ask: of course not.

> layer.

EHs as an extension mechanism are *not* innovation. They've been in the design
for 20 years. I'm actually with Fred on this: it's time for the hardware
designers to step up. With RFC 7112, we've told them that the maximum packet
size they need to parse is 1280 (after removing tunneling overhead).

   Brian

---

From: Warren Kumari <[email-masked]>
Date: Wed, 17 Jun 2015 19:28:58 -0400

> is costly processing, but it mitigates this bypass mechanism.

It also:
A: assume that all the packets go through this device and
B: that the device has enough state space to store bits until the
fragments have all shown up (a standard attack against reassembling
firewalls / ALGs is to send a fragment with the attack, then a whole
heap-o-fragments that you never intend to complete (to starve buffer
space) and then the final attack fragment.
The FW can choose to A: fail open (allow the oldest uncompleted
fragment when the buffer fills up) or B: fail closed / DoS (drop
oldest fragments when the buffer fills). Neither is good. The attacker
gets to set how long they wait before sending the final bits...

I have some expense reports that I desperately don't want to do, so
I'll procrastinate and try figure out just if this is even build able,
regardless of cost...

Linux has 60s (http://www.cs.fsu.edu/~baker/devices/lxr/http/source/linux/include/net/ipv6.h#L253
), but let's choose 10s for how long we expect to be able to
reassemble for - that's 125GB on a 100Gb interface. Unfortunately we
cannot just use regular DRAM here (buffer memory for 100Gbps needs to
be fast, especially for small packets), so realistically you are
looking at RLDRAM (reduced latency DRAM) - SRAM would be draw even
more power, cost more, lower density, etc and DDR3 doesn't have the
cycle time.

A quick look shows Micron's largest RLDRAM seems to be
MT44K32M36RCT-125E
(http://www.micron.com/~/media/documents/products/data-sheet/dram/1gb_twindie_rldram3.pdf),
a 10ns cycle time 32Meg x 36 wide chip, for 144MB (1152Mbit) per part.
For 125GB I'll need 889 of those.

A Cisco CRS-16 with CRS-X 4-Port 100GE blades gives me 64x 100Gb
ports, so 8TB of fast RAM, or 56,896 ram chips. Calculating current
consumption is tricky, because it depends upon how the memory is being
accessed (which the attacker can also (somewhat) control, but using
Micron's helpful power calculator spreadsheet we get ranges from
2154mw to 4035mw. Assuming there are no reads or writes 99% of the
times, we get a consumption of 2022mw + 151mw, for a total of 2173mw,
or 2.127W per chip. We have almost 57,000 of them, so that's 123KW per
router in memory alone (keep in mind there is also the memory
management, etc) . Currently the max power of a fully loaded CRS-1 16
is ~10KW, so that's a 12X increase just for the RAM.

Tempkin / RAS's NANOG presentation "Help! My Big Expensive Router Is
Really Expensive! "
(https://www.nanog.org/sites/default/files/wednesday.general.temkin.panel.pdf)
and Wobker's 'Power Consumption in  High-End Routing Systems'
(https://www.nanog.org/meetings/nanog54/presentations/Wednesday/Wobker.pdf
) shows the power consumption becomes distinctly linear -- you need to
move so much air to cool many KW that your fan / cooling system starts
to become a significant power draw. At this point you need to start
pumping even more air to cool the cooling, and so it spirals out of
control...
I'm not going to bother trying to calculate that, because, frankly,
I've gotten bored and want my dinner :-)

... and success, it is now too late to do my expense report, that will
be pushed to tomorrow :-)

W

-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 18 Jun 2015 13:31:39 +1200

> our draft proposed quite different approach ("drop and send ICMP

Right. And we can have that discussion, of course, but IMNSHO a firewall
should either let stuff through or drop it for a congigured reason; dropping
it because the product development manager made a lazy choice is not OK.

> beginning of the header...).

Well, we can't know today what sort of crazy option might be invented in
future; so the thinking behind draft-ietf-opsec-ipv6-eh-filtering
seems right to me. Maybe that's a draft you need to be tracking.

To be honest what I was thinking when we added discussion of HbH to
RFC 7045 was not so much fixing the slow routers, but warning the community
that "hop-by-hop" does not mean every hop. Fred would like to fix that
with his 6man draft, and I don't mean to attack that idea.

> explicitly configurable policy, 

AND if the forwarder is trying to be a firewall or otherwise
wants to interfere. If not, just forward the packet, already!

> it SHOULD

As I said: if the forwarder insists on doing something, OK.

But we have to think for a typical use case: does it matter if a
forwarder simply ignores the HbH header? For example, for the
IntServ/RSVP case, it just makes that router transparent to
RSVP. That doesn't break RSVP; discarding the packet does. That
was the thinking behind RFC 7045.

    Brian

---

From: Mark ZZZ Smith <[email-masked]>
Date: Thu, 18 Jun 2015 04:29:21 +0000 (UTC)

Hi,

________________________________
From: Brian Haberman <[email-masked]>
To: [email-masked] 
Sent: Thursday, 18 June 2015, 3:48
Subject: Re: [v6ops] [ipv6-wg] Extension Headers / Impact on Security Devices

/ Not specifically replying to Brian's email, however I think it is a step towards what I'm going to ask.

/ So I think everybody is making valid points, yet the set of points are inconsistent and sometimes in conflict. I think that is a sign to step back and think a bit more about better identifying and describing the problem or problems.

/ I think there are actually 3 security problems people are trying to solve in this discussion:

/ (1) mitigating network capacity DoS attacks on high speed forwarding devices

/ (2) protecting the in-band control plane of forwarding devices

/ (3) performing host/application security via a proxy device in the network ("firewalling, IDS/IPS", i.e., middleboxes)

/ So for each of these different problems, what are the requirements, the constraints and where are they best solved (in the network, in the hosts, or possibly a mix of both)?

/ A few thoughts I have on the above:

/ (1) I think TCP/UDP port level filtering for this purpose is certainly a nice to have but not a necessary to have, because network capacity DoSs will use some other protocol for their DoS if mitigating DoSs using TCP/UDP traffic becomes too effective. Completely dropping DoS traffic identified just by src and/or dst address, or plonking that traffic into a scavenger QoS class for a while if complete dropping is too severe is a general solution that will work regardless of the type of DoS traffic. Using TCP/UDP ports if they're easily available would allow the dropping to be more granular, but is not required to mitigate the network capacity DoS.

/(2) The control plane is really a host (because it is the final destination for the packets sent to the control plane), so it needs "host style" protection, which I think means stateless methods aren't effective enough - for example, I think spoofed src address fragmented OSPF packets could be used to attack the OSPF instance, yet they'd pass the stateless ACLs, which is why we enable IGP/BGP authentication, validate hopcounts etc.. Ultimately, moving the control plane completely out of band (as the telephony people did, and "SDN"/Openflow does) would place the control plane completely out of reach of malicious in-band traffic sources.

/ (3) I think this is always going to be impractical to do well in fast, fixed or limited function hardware, as it is necessary to do just too much host type processing. We scale solutions to problems on the Internet by using "divide-and-conquer" and achieve that by pushing them towards or onto the actual edge i.e., the hosts. So I think (3) is far more possible to achieved by either doing it in lots of lower throughput but higher inspection software based forwarding devices close to the edge, or leaving the hosts to do it themselves (because they know best what is and isn't legitimate traffic for their own OS and applications, and they might also be multi-homed, so have a security perspective that a single device on one of the upstream links can't.), with perhaps close by devices in the network performing a secondary assisting and more coarse security role (e.g., stateless ACLs that match on no more either basic src/dst address or TCP/UDP ports, following an EH chain to find the TCP/UDP port information if necessary, but not doing any further analysis, leaving that to the receiving host)

/ Regards,
Mark.

---

From: [email-masked]
Date: Thu, 18 Jun 2015 08:53:56 +0200 (CEST)

> don't conform to the IPv6 standard.

It may be relevant to ask for RFC 7112 support next time we're doing
an equipment RFQ (in a few years).

> Whereas dropping *all* EHs breaks the IPv6 standard.

Obviously. But until RFC 7112 support is available, I believe we will
see a significant amount of breakage for IPv6 extension headers - and
header chains will be limited to significantly less than 1280 bytes.

> size they need to parse is 1280 (after removing tunneling overhead).

IPv6 extension header processing is sufficiently complex that I'm not
at all sure that "time for the hardware designers to step up" will be
enough. My prediction is that we'll see security alerts from hardware
manufacturers for many years to come, due to the complexity.

Steinar Haug, AS 2116

---

From: Brian Haberman <[email-masked]>
Date: Thu, 18 Jun 2015 07:32:48 -0400

Hi Warren,

> reassemble for - that's 125GB on a 100Gb interface. Unfortunately we

So, just a push back on these two assumptions.  Are they in conflict?
Would you really buffer 10 seconds worth of data on a 100Gbps interface?
 But, that is beside the point...

While the accompanying numbers are interesting (thanks for calculating
them) and scary, they don't really address my point.  These types of
intermediate security devices will always be at a disadvantage since: 1)
They need to inspect all the way to the application (or maybe transport)
layer, 2) at line rate, and 3) protect orders of magnitude more end nodes.

Mark's later e-mail is more articulate in pointing out that there are
three (or maybe more) distinct security issues being discussed here.
These need to be teased apart rather than addressing them partially with
short-term approaches.

Regards,
Brian

---

From: Gert Doering <[email-masked]>
Date: Fri, 19 Jun 2015 00:00:58 +0200

Hi,

> Tell me this. Would you be happier if the fragmentation rule said that the first fragment had to contain the entire IPv6 header, plus the transport layer header (for ACL support)? I think Fernando…

It would certainly make *me* happier...

Gert Doering
        -- NetMaster
-- 
have you enabled IPv6 on something today...?

SpaceNet AG                        Vorstand: Sebastian v. Bomhard
Joseph-Dollinger-Bogen 14          Aufsichtsratsvors.: A. Grundner-Culemann
D-80807 Muenchen                   HRB: 136055 (AG Muenchen)
Tel: +49 (0)89/32356-444           USt-IdNr.: DE813185279

---

From: Joe Touch <[email-masked]>
Date: Thu, 18 Jun 2015 15:32:19 -0700

>> Tell me this. Would you be happier if the fragmentation rule said that the first fragment had to contain the entire IPv6 header, plus the transport layer header (for ACL support)? I think Fernando…

It makes sense to require all of the HBH and routing headers in the 
first fragment.

The rest is impossible to mandate and irrelevant. Anything that looks 
far enough into a packet to need to find the transport header might be 
looking several layers of encapsulation in; that's acting as an 
endpoint, and ought to reassemble the packet at that point.

Joe

---

From: Ole Troan <[email-masked]>
Date: Fri, 19 Jun 2015 09:10:55 +0200

> It would certainly make *me* happier…

done.
RFC7112.

cheers,
Ole

---

From: [email-masked]
Date: Fri, 19 Jun 2015 09:39:11 +0200 (CEST)

> > It would certainly make *me* happier$,1s&(B
> 
> done.
> RFC7112.

As I wrote in another mail,

...
> But until RFC 7112 support is available, I believe we will

And until such support is available, we have to deal with the current
mess. Which may imply more filtering than some people would like.

Steinar Haug, AS 2116

---

From: Ole Troan <[email-masked]>
Date: Fri, 19 Jun 2015 09:56:20 +0200

>>> It would certainly make *me* happier�$,1s&
>> 
>> done.
>> RFC7112.
> 

I don’t think that follows.

cheers,
Ole

---

From: Enno Rey <[email-masked]>
Date: Fri, 19 Jun 2015 10:34:10 +0200

Hi,

On Fri, Jun 19, 2015 at 09:56:20AM +0200, Ole Troan wrote:
> >>> It would certainly make *me* happier???$,1s&
> >> 
> >> done.
> >> RFC7112.
> > 
> I don???t think that follows.

I would second the observation that this (subsequent action) actually happens.
Not least because many consider it a reasonable approach not to process and/or to drop something that induces complexity & insecurity and which at the same time is not needed by any service or application (read: all EHs except ESP and, maybe in some corner cases, AH+FH).

thanks

Enno

-- 
Enno Rey

---

From: Mark ZZZ Smith <[email-masked]>
Date: Fri, 19 Jun 2015 09:13:52 +0000 (UTC)

From: Enno Rey <[email-masked]>
 To: Ole Troan <[email-masked]> 
Cc: [email-masked]; [email-masked] 
 Sent: Friday, 19 June 2015, 18:34
 Subject: Re: [v6ops] [ipv6-wg] Extension Headers / Impact on Security Devices
   
Hi,

/ I think you're only expressing the view of many security engineers, not the many network engineers or the many more users of networks. At an ISP, if you block traffic your customer wants send, all you get is angry customers. They're paying you to deliver any and all of their packets as well as possible, with as minimum restrictions as possible, ideally none, rather than the opposite. This is why I don't think you'll ever get consensus on deprecating EHs, because I think you're only coming the position of trying to solve problem (3) I described in my last email.
/ I'd also observe that deprecating EHs, other than ESP, AH+FH, TCP and UDP, would also prevent them from being used behind the ESP EH - where you won't be able to see what is in them, so you won't be able to care what is in them. You'll also be preventing people from other transport layer protocols, such as SCTP and DCCP, that are feasible to deploy over the IPv6 Internet that couldn't be over the IPv4 network because of IPv4 NAT and other middle boxes.
thanks

Enno

-- 
Enno Rey

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 20 Jun 2015 08:15:08 +1200

> / I think you're only expressing the view of many security engineers, not the many network engineers or the many more users of networks. At an ISP, if you block traffic your customer wants send, al…

I don't think that's what he was saying. I think he was saying "enforce RFC 7112."
Discarding packets whose headers exceed the first fragment strikes me as very
reasonable, even before host stacks are updated to enforce RFC 7112 at the source.

What isn't reasonable is to say "laugh at RFC 7045". Anybody who purports to sell
IPv6 firewalls needs to support RFC 7045 in their next release.

The next order of business in the IETF (6man, not here) is to decide
whether we go further in fixing the standard, which I think you and Fred
are both advocating.

   Brian

---

From: Warren Kumari <[email-masked]>
Date: Fri, 19 Jun 2015 16:19:26 -0400

On Fri, Jun 19, 2015 at 5:13 AM, Mark ZZZ Smith
<[email-masked]> wrote:
>> >>>> Tell me this. Would you be happier if the fragmentation rule said
>> >>>> that the first fragment had to contain the entire IPv6 header, plus the
>> >>>> transport layer header (for ACL support)? I think Fernando would support
>> >>>> such a statement (I think I have "heard" him make such a statement).
>> >>>
> Not least because many consider it a reasonable approach not to process
> and/or to drop something that induces complexity & insecurity and which at
> the same time is not needed by any service or application (read: all EHs
> except ESP and, maybe in some corner cases, AH+FH).
>
> / I think you're only expressing the view of many security engineers, not
> the many network engineers or the many more users of networks. At an ISP, if
> you block traffic your customer wants send, all you get is angry customers.

... and at an ISP if your customer calls you up because they are being
DDoSed out of existence with traffic at some port that they don't use,
and you tell them at that you cannot filter UDP port 80 because, well,
you cannot find it, you also end up with an angry customer.

> They're paying you to deliver any and all of their packets as well as
> possible, with as minimum restrictions as possible, ideally none, rather
> than the opposite.

Perhaps. I figure they are /actually/ paying you to provide them with
good service. Sometimes that includes filtering for them, because you
have bigger pipes than them...

W

> This is why I don't think you'll ever get consensus on
> deprecating EHs, because I think you're only coming the position of trying
> to solve problem (3) I described in my last email.
>
> / I'd also observe that deprecating EHs, other than ESP, AH+FH, TCP and UDP,
> would also prevent them from being used behind the ESP EH - where you won't
> be able to see what is in them, so you won't be able to care what is in
> them. You'll also be preventing people from other transport layer protocols,
> such as SCTP and DCCP, that are feasible to deploy over the IPv6 Internet
> that couldn't be over the IPv4 network because of IPv4 NAT and other middle
> boxes.
>
> thanks
>
> Enno
>
>
>
>
>>
>> cheers,
>> Ole
>
>
>
> --
> Enno Rey
>
> Blog:   || Conference: www.troopers.de

---

From: Ronald Bonica <[email-masked]>
Date: Fri, 19 Jun 2015 21:36:12 +0000

I am aware of one implementation that is almost RFC 7112 compliant. It drops the packet, but fails to send an ICMP message.

                                                                                                                           Ron

> Yes, we're aware of RFC7112. It's just: no OS we know and no devices we're
> aware of (feel free to provide pointers) implement RFC 7112 as of today.

---

From: Mark ZZZ Smith <[email-masked]>
Date: Sat, 20 Jun 2015 03:07:56 +0000 (UTC)

Hi Warren,
      From: Warren Kumari <[email-masked]>
 To: Mark ZZZ Smith <[email-masked]> 
Cc: Enno Rey <[email-masked]>; Ole Troan <[email-masked]>; "[email-masked]" <[email-masked]>; "[email-masked]" <[email-masked]> 
 Sent: Saturday, 20 June 2015, 6:19
 Subject: Re: [v6ops] [ipv6-wg] Extension Headers / Impact on Security Devices
   
<snip>

... and at an ISP if your customer calls you up because they are being
DDoSed out of existence with traffic at some port that they don't use,
and you tell them at that you cannot filter UDP port 80 because, well,
you cannot find it, you also end up with an angry customer.
/ If you can't find it, that may be because it isn't actually there.
/ Here's what I wrote in my earlier email what problem/problems being solved email about network capacity DoS:
/  "(1) I think TCP/UDP port level filtering for this purpose is certainly a nice to have but not a necessary to have, because network capacity DoSs will use some other protocol for their DoS if mitigating DoSs using TCP/UDP traffic becomes too effective. Completely dropping DoS traffic identified just by src and/or dst address, or plonking that traffic into a scavenger QoS class for a while if complete dropping is too severe is a general solution that will work regardless of the type of DoS traffic. Using TCP/UDP ports if they're easily available would allow the dropping to be more granular, but is not required to mitigate the network capacity DoS."

/ although I'd reword that last sentence and say something like "Using TCP/UDP ports if they're easily available would allow the dropping to be more granular, but cannot be relied on to mitigate all network capacity DoSes."
/ If the motivation of the DoS attacker is to exhaust network capacity, then they don't really care what type of traffic it is. Using UDP or TCP is probably convenient, but if that becomes too easy to block, they'll use something else.
/ OTOH, if their motivation is to DoS a particular application, then you're going to have to let some of it through because some of it is legitimate via something like the policed/shaped QoS class, as well as possibly by dropping traffic from a set of sources that appear to be DoS traffic origins rather than legitimate sources.
/ Perhaps wanting the TCP and UDP headers in a known location in the packet so that simple ACLs can filter them is a more specific case of being able to filter packets using a bit mask in an arbitrary specified location within the packet. I'm not aware of any routers where this is an option in ACLs (although it is ringing a very small bell), so perhaps that is what is missing from routers today (I don't know much about OpenFlow, although I'd have expected OF switches to be able to do this. However, quickly downloading and looking at the OF specs, it doesn't seem that specifying an arbitrary bit range and mask to match on is supported). Being able to do that would not only make ACLs more general purpose, but would also allow more accurate filtering on non-TCP/UDP packets too, rather than just using source and/or destination addresses for this traffic. I think that would better alleviate the desire of people to restrict transport layer protocols to only TCP/UDP.
/ A DoS attacker could overcome that by randomising packet contents, however that is more work for them, and I think would classify the DoS attack as a network capacity exhaustion attack, meaning that the methods such as dropping or policing/shaping high volume sources using just IP addresses would be more effective.

Perhaps. I figure they are /actually/ paying you to provide them with
good service. Sometimes that includes filtering for them, because you
have bigger pipes than them...
/ I think sanitising traffic is a different and much harder problem to solve than mitigating a DoS attack. I think mitigating DoSes is enough of a service to provide to customers as part of their Internet access, because going any "deeper" means getting involved in what protocols and therefore what applications the customer is choosing to use and may choose to use in the future. That's a separate service.
/ Regards,/ Mark.

W

---

From: Fernando Gont <[email-masked]>
Date: Fri, 26 Jun 2015 04:34:56 -0300

On 06/17/2015 01:45 AM, Fred Baker (fred) wrote:
>> Personally I still think RFC 7045 is the most realistic on this
>> point, but Fred would like things to get better ;-).
> 
> Bottom line, if one accepts the present status quo as the state
> forever, then we should stop with RFC 7045, and (with Fernando) agree
> to deprecate all extension headers. I'd like to not do that, and the
> only way I see to not do that is to not accept the status quo.

Not sure if that's simply a matter of an error in punctuation (or in my
interpretation)... but for the record, I'm not arguing in favor of
deprecating IPv6 EHs. Actually, I've worked to figured out what's the
status quo, and working on stuff that may help to change that (e.g.,
RFC7112).

Thanks,
-- 
Fernando Gont
SI6 Networks
e-mail: [email-masked]
PGP Fingerprint: 6666 31C6 D484 63B2 8FB1 E3C4 AE25 0D55 1D4E 7492

---

From: Fernando Gont <[email-masked]>
Date: Fri, 26 Jun 2015 04:54:06 -0300

> least some products close that hole.

... for IPv6 traffic?

Cheers,

---

From: Brian Haberman <[email-masked]>
Date: Fri, 26 Jun 2015 08:54:58 -0400

>> least some products close that hole.

I believe so from what I remember (but I can't find documentation right
off).  The NFR was purchased by Checkpoint and became the IPS-1 and the
IPs-1 supports IPv6.

Regards,
Brian