Subject: Re: TSV-ART review of draft-ietf-6man-rfc2460bis-09

From: "C. M. Heard" <[email-masked]>
Date: Tue, 11 Apr 2017 14:31:30 -0700

On Tue, 11 Apr 2017 21:14:06 +0200, Martin Stiemerling wrote:
> Major issues:
>
> - Section 4.8. "Defining New Extension Headers and Options":
>
> It says new hop-by-hop headers must never ever defined. This is problematic,
> as this closing the door forever, even if future instances of the IETF do
> would like to wish to define new hop-by-hop headers. A better way would have
> to say "that new hop-by-hop headers must have IETF consensus".

If a new extension header with hop-by-hop behavior were defined,
it would require a router or other forwarding device to look
beyond the header immediately following the base IPv6 header,
possibly even arbitrarily deep into an IPv6 header chain. That's
not backward compatible with the existing specification, which was
written the way it was to limit the amount of header processing
required in the forwarding path.

Can you provide an example of anything that can be done with a
new hop-by-hop extension header that cannot also be done with
a new hop-by-hop option?

> - Section 4.8. "Defining New Extension Headers and Options":
>
> Also the „not recommended“ to define new extension headers looks strange,
> especially with the phrase "There has to be a very clear justification". The
> term "clear justification" is not an exact engineering specification. Why
> not using "technical protocol specification and real word use case required,
> plus IETF consensus"?

We could perhaps make this clearer by stating that a new extension
header must not be defined unless it can be proven that the desired
functionality cannot reasonably be provided either by a new
destination option or by a new upper layer protocol, but I'm not
convinced that it's that much of an improvement - there's that
pesky word "reasonably," which still implies an exercise of
engineering judgment, which you seem to want to avoid.

For a necessarily imprecise analysis of which of the existing
extension headers meet the "cannot reasonably be provided"
criterion, please see

https://www.ietf.org/mail-archive/web/ipv6/current/msg20215.html

Mike Heard

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 12 Apr 2017 14:47:44 +1200

> new hop-by-hop extension header that cannot also be done with

Also consider that all our experience is that router designers
are mainly allergic to even the existing HbH header. So adding
a new one would really be a lost cause anyway.

> https://www.ietf.org/mail-archive/web/ipv6/current/msg20215.html

And also consider that deploying new extension headers is hard to
impossible anyway, for reasons discussed in RFC 7045. We do need to
set the bar very high. I'd have no objection to adding IETF consensus,
if it isn't already implied.

   Brian

---

From: "Mirja Kuehlewind (IETF)" <[email-masked]>
Date: Wed, 12 Apr 2017 18:41:58 +0200

Hi Mike,

> Am 11.04.2017 um 23:31 schrieb C. M. Heard <[email-masked]>:
> 

Given the current hop-by-hop header is partially not usable, everything…

I thinking would be to basically replace/deprecate the old one and define a new one.

> https://www.ietf.org/mail-archive/web/ipv6/current/msg20215.html

I have to say I find this phrasing also to be useful. I guess the real point is to recommend to rather use a destination option if possible than defining a new extension header.

Mirja

> 
> Mike Heard
> 
> _______________________________________________
> Tsv-art mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tsv-art

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 13 Apr 2017 12:05:00 +1200

> Given the current hop-by-hop header is partially not usable, everything…

But the reason it's not fully supported is intrinsic to the words "hop by hop".
Router designers don't want to spend cycles this way, that's all.
 
> I thinking would be to basically replace/deprecate the old one and define a new one.

That is way out of scope for the promotion to IS (no new features). And as Bob
points out, HbH must come first, so you can't have two...

   Brian
 
> --------------------------------------------------------------------
> IETF IPv6 working group mailing list
> [email-masked]
> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> --------------------------------------------------------------------
>

---

From: "Leddy, John" <[email-masked]>
Date: Thu, 13 Apr 2017 11:52:20 +0000

As more of the service delivery path includes complex virtualized environments with programmable data planes and general software based IPV6 forwarding, the cost/benefit tradeoffs for implementing EH processing changes – even HbH.

If the standard is heavily skewed toward more traditional Vendor based ASIC/NPU forwarding preferences for their implementations, it does future IPV6 development a disservice.

Enabling some of these EH features to be configurable makes sense so current constrained routers won’t be non-compliant with 2460bis and therefore “middle boxes” themselves, without banning/deprecating/withholding and generally shutting down future work in this area.

---

From: [email-masked]
Date: Thu, 13 Apr 2017 17:06:05 +0200

> Enabling some of these EH features to be configurable makes sense so current constrained routers won’t be non-compliant with 2460bis and therefore “middle boxes” themselves, without banning/depreca…

Indeed, and if there was an actual real use for a particular hop by hop option that would obviously be implemented and supported.
today we don't see much use of it, which is also one reason why it is filtered / not well supported.

So I would certainly be supportive of new and useful uses of the hop by hop header. And much more so than workarounds like requiring routers to parse deep into packets. Workarounds which are likely to just trigger the weapons race.

Btw, even for high performance software routers there is a cost to parse deeply into the packet. The memory subsystem is bottleneck, and while processing packet #1 and #2 we prefetch cachelines for packet #3 and #4. If it turns out that when processing packet #1 we need to fetch one or more cache lines we have already lost.

Cheers,
Ole

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 13 Apr 2017 18:14:42 +0200

Hi Ole,

Let's observe that your claim is only partially correct.

Talking to real hardware designers I came to know that it cost nothing to
get into first 150-200 bytes of packet.

Yes aftee that you face cliff effect.

But are we really that concerned about headers beyond that ?

Cheers,
R.

On Apr 13, 2017 5:06 PM, <[email-masked]> wrote:

> > As more of the service delivery path includes complex virtualized
> environments with programmable data planes and general software based IPV6
> forwarding, the cost/benefit tradeoffs for implementing EH processing
> changes – even HbH.
> >
> > If the standard is heavily skewed toward more traditional Vendor based
> ASIC/NPU forwarding preferences for their implementations, it does future
> IPV6 development a disservice.
> >
> > Enabling some of these EH features to be configurable makes sense so
> current constrained routers won’t be non-compliant with 2460bis and
> therefore “middle boxes” themselves, without banning/deprecating/withholding
> and generally shutting down future work in this area.
>
> Indeed, and if there was an actual real use for a particular hop by hop
> option that would obviously be implemented and supported.
> So I would certainly be supportive of new and useful uses of the hop by
> hop header. And much more so than workarounds like requiring routers to
> parse deep into packets. Workarounds which are likely to just trigger the
> weapons race.
>
> Btw, even for high performance software routers there is a cost to parse
> deeply into the packet. The memory subsystem is bottleneck, and while
> processing packet #1 and #2 we prefetch cachelines for packet #3 and #4. If
> it turns out that when processing packet #1 we need to fetch one or more
> cache lines we have already lost.
>
> Cheers,
> Ole
>
>

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 14 Apr 2017 15:22:42 +1200

> get into first 150-200 bytes of packet.

Good. That should make new HbH options viable, as long as we agree that
processing HbH is optional, so that less capable routers can ignore it.
Given that, the current HbH header format is OK, isn't it? Nobody is
proposing to remove the possibility of defining new HbH options within
the HbH header.

> But are we really that concerned about headers beyond that ?

Quoting Steinar Haug a couple of years ago, we need

>>  [IPv6 Fixed Hdr] + [IPv6 EHs] + [L4 Hdr] <= hardware inspection limit
>> 
>> in order for the router hardware to be able to filter based on TCP/UDP
>> headers at line rate.

So, the fixed header is 40 bytes and a TCP header is 20 bytes.
If the hardware limit is 150 bytes as you suggest, that allows for
90 bytes of extension headers. If the limit is 128 bytes, as people
claimed a couple of years ago, that would be 68 bytes of extension
headers.

For most extension headers, that seems like a lot. But routing headers
could be quite long, and shim6 (hardly a success, but it is standardised)
could be even longer in some circumstances. I did calculate two years
ago that a packet with the following could exceed the 68 byte limit:

Hop-by-Hop Options/Router Alert + Fragment + ESP +
Destination Options/Tunnel Encapsulation Limit + SHIM6

Replace SHIM6 with a big Routing Header if you prefer.

Also consider what you might want to do with encapsulated packets
in a router that is also a firewall.

So, falling off the cliff seems like a real possibility.

     Brian

---

From: [email-masked]
Date: Sun, 16 Apr 2017 14:20:18 +0200

Robert,

> Talking to real hardware designers I came to know that it cost nothing to get into first 150-200 bytes of packet.
> 

If it wasn't clear, my point was: Even in _software_ looking deep into the packet has a cost.
Even when packets are DMA'ed directly into L3 cache.

If of course depends. Both in software and in hardware having to do pointer chasing is expensive (which you have to do for extension headers, fixed offset is of course much cheaper).
We have built hardware platforms that can get parse a complete packet at line rate, so there is no fixed maximum parsable header size as such.

I'll see if I can get some real measurements next week for fun. That would be on a modern Intel platform with VPP.

Best regards,
Ole

---

From: "Leddy, John" <[email-masked]>
Date: Sun, 16 Apr 2017 13:04:40 +0000

Ole,

Real data is always helpful, I understand more work requires more cycles, but I want to look at the overall work.

My interest is in looking at where we can reduce the overall amount of work required by redesigning how some of our application platforms work.
Things like state full FW/LB’s, NATs, Application Proxies, Tunnels encap/decaps… are expensive and add a lot of complexity in the design and operation of the systems.  How expensive are these functions in ASICs or software forwarding?

I’ve seen virtualized Encap products (“based on standards”) that have a series of header encap/decap on distributed nodes that are just amazing to consider: 
Ethernet, IP encap, ESP, UDP, MPLS, GRE, VLXLAN, NSH, Ethernet, IP -> in all series of combinations, duplication and ordering.  Throw in a few NATs/LBs/FWs/Proxies along the path for processing.  Forget about trying to figure out how to make PMTUD work in these environments.

We are looking to simplify this by doing “a little more” work at the Network layer where appropriate, not necessarily every node.
Push state to the application/clients for flows/sessions and carry that state in the packets that can be processed in standard ways.
IPV6 gives us the ability to start simplifying just because the address space is larger, but there is much more possible and needed.

I understand many of these functions may be considered “middle box” functions, my goal is to remove as many “middle boxes” as possible in our infrastructure,

John

On 4/16/17, 8:20 AM, "[email-masked]" <[email-masked]> wrote:

    Robert,

    

    Best regards,
    Ole