Subject: Re: I-D Action: draft-voyer-6man-extension-header-insertion-02.txt

From: 神明達哉 <[email-masked]>
Date: Fri, 1 Dec 2017 11:40:09 -0800

At Thu, 30 Nov 2017 17:03:08 -0800,
Tom Herbert <[email-masked]> wrote:

> > What I'm seeing in this thread are rational arguments for and against
> > the proposal. I agree about the logical fallacy in the abstract of the
> > current draft, but that's fixable with a few seconds use of the cursor
> > and Delete key. I'd like to see the rational arguments in the next version
> > of the draft.
> >
> It's not just the abstact that promote the logical fallacy, it occurs
> elsewhere in the draft. For instance:
>
> "Obviously, this FRR service increases the size of the packet during
> its journey within domain D.  This is well-known to operators.
> Well-known mitigation techniques have been deployed for more than 15
> years"
>
> This seems like a pretty useless statement in a protocol
> specification. Either the mitigations should be articulated here or
> the applicable RFCs describing them should be referenced.

+1, or, I would not necessarily say it's a "fallacy" but the current
form of draft is unnecessarily defensive and provocative, while it
doesn't sufficiently provide *technical* answer to questions that was
raised for the proposal, most notably "why not just encapsulation".

I'm happy to help the proposal move forward (although whether I
finally support it depends on technical details), but right now the
tone and technical substance of the draft don't suggest I can
constructively contribute to the discussion.  As Brian put it, I
basically "stopped reading the draft" once I knew it didn't contain
much useful information and didn't even bother to point it out at this
ML.  I suspect there are some others who had the same impression and
took the same (in)action.  Hopefully the next version will be improved
substantially, incorporating the points discussed in this thread, then
I'll be happy to review it more closely.

--
JINMEI, Tatuya

---

From: Mark Smith <[email-masked]>
Date: Sat, 2 Dec 2017 09:07:52 +1100

Hi Brian,

On 1 December 2017 at 11:15, Brian E Carpenter
<[email-masked]> wrote:
> On 01/12/2017 10:27, Mark Smith wrote:
>> On 1 Dec. 2017 6:35 am, "Robert Raszuk" <[email-masked]> wrote:
>>

<snip>

> and Delete key. I'd like to see the rational arguments in the next version

While they may have perhaps been lost in the discussion before, they
were asked for in the previous discussions of the -00 draft.

I asked numerous times (at least 3 from memory), what is wrong with
exclusively using tunnelling for this. I think eventually Robert said
because of the tunnelling overhead.

The trouble with this argument is that this draft doesn't exclusively
propose extension header insertion, it also describes using
tunnelling. The only distinction between which method to use is
whether the packet originates within the SR domain ("Source Domain and
Packet Journey") or originates externally to it ("Transit through a
Source Domain"). So tunnelling overhead is apparently quite acceptable
as long packets happen to come from outside the SR domain. I can't see
anything that explains why packets that originate internally can't or
must not be tunnelled.

So the example SR domains in the draft, which is presumably the same
domain, support both insertion and tunnelling to achieve the same and
single goal of adding SRH information to a packet. Seems unnecessarily
complex to support two methods that achieve the same functional
outcome. More prone to vendor bugs, harder to troubleshoot, more
things to learn and then forget if not used often.

More broadly, I assume the reason to try to use IPv6 for this, rather
than either sticking just with MPLS or inventing something new, is to
leverage IPv6's existence, widely available implementations and pretty
close to current and definite future commoditisation. Leveraging what
already exists for the cost and time benefit may mean making some
compromises to gain from commoditisation.

RFC8200 IPv6 forwarding and IPv6-in-IPv6 tunnelling are commodity,
because they're methods as old as RFCs 1883 and 2473, and are widely
implemented and deployed. Extension Header insertion is not, and
therefore using it is de-commodifying IPv6. The commodity benefits of
IPv6 disappear if fundamental changes are or need to be made to IPv6's
operation. EH insertion is a fundamental change.

I think the objection to tunnelling is really be because the IPv6 SRH
header is too large, because of the use of 128 bit IPv6 addresses as
segment identifiers. "Scrounging" bits somewhere else to try to
minimise overhead is avoiding solving the problem where it is caused.

The TRILL people had this problem, as they considered 7 octet IS-IS
IDs to be too large to include in the TRILL header. They solved it be
using 16 bit "nicknames".

"3.7.  RBridge Nicknames

   Nicknames are 16-bit dynamically assigned quantities that act as
   abbreviations for RBridges' IS-IS IDs to achieve a more compact
   encoding ..."

https://tools.ietf.org/html/rfc6325#section-3

Regards,
Mark.

---

From: Mark Smith <[email-masked]>
Date: Sat, 2 Dec 2017 11:49:57 +1100

On 1 December 2017 at 09:07, Robert Raszuk <[email-masked]> wrote:
> Tom,
>
> The biggest advantage of SRv6 in contrast to SR-MPLS is that the SRH (one or
> more) are not modified through the SR applicability of the packets.
>

I'd have thought the biggest advantage and actual motivation for using
IPv6 for this was to leverage IPv6's near and definite future
commodity. MPLS required forklift upgrades to networks, if you use
IPv6 per RFC8200 (and its predecessors), the majority of existing IPv6
networking devices will work and can continue to be used without
change. and only those that are doing something special need to be
changed.

> So node which understands SRv6 can generate ICMP to both src of the packet
> (poor host) as well as ingress node in SR domain (as it is explicitly listed
> in Ingress Node TLV of first SRH). Node which does not understand SRv6 will
> likely only send it to original src .. but this is a feature not a bug. This
> is how traceroute works.

No, it is not the same as traceroute operation.

Traceroute works on the assumption that the packet headers in the ICMP
response payload match those of the packet that triggered the ICMP
message. That is how the host running traceroute knows which
traceroute process to send the API error message to that then
generates the traceroute hop output.

If the traceroute source host sends IP/UDP or IP/ICMP to trigger the
ICMP messages for tracerouting, yet receives in the ICMP payload
"IP/SRH/UDP" or "IP/SRH/ICMP", and doesn't understand the SRH header
or isn't enabled to understand the SRH header and read past it (which
is probably going to be all end-user hosts), how can the receiving
host know which traceroute process to send the API error to?
Traceroute won't report the hop because it won't receive the sockets
API error message that reports it.

It isn't just traceroute that relies on this, it is all of ICMP. EH
insertion violates that assumption, which is why EH insertion can or
may cause many unexpected ICMP protocol and operational failures.
PMTUD is one of the main examples.

>
> If you do encap how will host now see all the nodes on the way traceroute or
> tracepath ?

They don't see link-layer hops today. An IPv6-in-IPv6 tunnel is using
IPv6 as a link-layer. That's the point of an IPv6-in-IPv6 tunnel - to
make an IPv6 network appear as a single link-layer hop.

> Do you think that hosts admins will be happy to be blinded by
> 6man ?
>

Host admins aren't the actual audience for traceroute, network
operators troubleshooting networks are. We're well familiar with
troubleshooting virtual links that may or may not hide the actual
physical topology and infrastructure, because we configure and operate
it. Ideally host admins should never need to run traceroute because
the network is designed and being operated well enough that paths that
hosts use over it are predictable and testable within the network.

> Btw I am not the one who ever claimed applicability to "controlled networks"
> or "wall gardens" so don't try to use that one with me :)). For me the much
> bigger question is what should go into data plane and what is better to go
> in control plane. We may end up with both and let customers/operators
> choose. But hand waving that doing innovation in data plane is scary is not
> the approach I would endorse.
>

I'll ignore the further logical fallacy appeals. However, I'll address
a point about innovation.

Innovation commonly occurs within existing constraints, to avoid
needing to "boil the ocean". All those innovative electric and self
driving cars and trucks we're hearing about daily? They don't require
that we rip up and replace all the roads, the street signs or traffic
lights, or widen the lanes. They're compatible with our existing road
infrastructure, which is one of the significant values of them as an
innovation. The required costs and resources to adopt them are far
lower because of the compatibility with the existing infrastructure.

If they required changing both the vehicle and the infrastructure they
ride on or in, they wouldn't be called cars, and would cost far more
money to develop and deploy (and would be called "Hyperloop" for
example), and need to carry far greater numbers of people at once to
justify the expense. They wouldn't be individual or small group
transport vehicles, and wouldn't be within financial reach of most
individuals.

Compatibility is one of the 5 factors that influence the adoption of
an innovation. There's a summary of all 5 in this presentation
starting at slide 8 -
http://www.users.on.net/~markachy/The_Rapid_Rise_of_the_MMHH.pdf

> We would be still doing token ring or FDDI ....
>

Another attempt to use emotional appeal rather than technical
argument. Please stop trying to use that method to argue your point,
it doesn't work.

Regards,
Mark.

> Cheers,
> R.
>
>
> On Thu, Nov 30, 2017 at 10:54 PM, Tom Herbert <[email-masked]> wrote:
>>
>> On Thu, Nov 30, 2017 at 1:25 PM, Robert Raszuk <[email-masked]> wrote:
>> > Tom,
>> >
>> > Imagine the case where packets are subject to go via service chains. I
>> > can
>> > steer the packet to the router which has service nodes connected to it
>> > but
>> > original src address may be used to decide which service given packet is
>> > subject to take.
>> >
>> > Sure I can impose an SR function and map it up front such that the
>> > router
>> > above would never need to inspect src address, but why to construct such
>> > architecture which only allows limited set of network programming as
>> > opposed
>> > to offer the choice on how to setup things by network operators ?
>> >
>> Robert,
>>
>> Because the architecture, or at least protocol, is not correct.
>>
>> Suppose a device inserts an EH into a packet that downstream causes
>> the packet to be dropped and generates an ICMP error which goes back
>> to the source. What is a source host supposed do with this? It might
>> be able to determine that it didn't send the EH, and if it's lucky it
>> might even be able to parse the offending EH and maybe even sees the
>> problem that caused the error. But then what? How does the host tell
>> the network to stop mucking with its packets so that they no longer
>> get dropped and the source host doesn't get blamed?
>>
>> ICMP handling is just one of several issues that needs to be
>> addressed. If the only answer is that these aren't concerns because
>> the protocol is only deployed in controlled networks, then I claim
>> that's nothing more than hand waving which is not robust and doesn't
>> scale.
>>
>> Tom
>
>
>
> --------------------------------------------------------------------
> IETF IPv6 working group mailing list
> [email-masked]
> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> --------------------------------------------------------------------
>

---

From: Stefano Salsano <[email-masked]>
Date: Sat, 2 Dec 2017 08:15:04 +0100

Il 2017-12-01 23:07, Mark Smith ha scritto:

<snip>
> anything that explains why packets that originate internally can't or

Hi Mark,

with the first tunneling operation you create a new packet that 
originates in your SR domain.

We definitely want it, so the further SR insertion will only be applied 
to such packets (and hosts outside the domain will not receive 
unexpected ICMP...) and we believe that it is worth paying the tunneling 
overhead here.

We think that for further operations that require adding of SRH 
information to such packets originating in your SR domain, the SR 
insertion is more efficient than tunnelling into a new packet.

This is both from the point of view of the packet size (and increase of 
packet headers size) and from the point of view of the cost of packet 
manipulation operations.

There can be scenarios in which you have more than one such operation to 
be applied to the packet, let us assume two. If you go for tunnelling 
you have the first encapsulation, then two more IPv6-in-IPv6 
encapsulation in the same packet, for a total of 3 encapsulations. If 
you go for tunneling + SR insertion you only have the first 
encapsulation and then "simple" mangling of SRH header.

ciao
Stefano

-- 
*******************************************************************
Stefano Salsano
Professore Associato
Dipartimento Ingegneria Elettronica
Universita' di Roma Tor Vergata
Viale Politecnico, 1 - 00133 Roma - ITALY

http://netgroup.uniroma2.it/Stefano_Salsano/

E-mail  : [email-masked]
Cell.   : +39 320 4307310
Office  : (Tel.) +39 06 72597770 (Fax.) +39 06 72597435
*******************************************************************

---

From: Tom Herbert <[email-masked]>
Date: Sat, 2 Dec 2017 10:05:51 -0800

On Fri, Dec 1, 2017 at 11:15 PM, Stefano Salsano
<[email-masked]> wrote:
> with the first tunneling operation you create a new packet that originates
> in your SR domain.
>
> We definitely want it, so the further SR insertion will only be applied to
> such packets (and hosts outside the domain will not receive unexpected
> ICMP...) and we believe that it is worth paying the tunneling overhead here.
>
Stefano,

The problem is that almost all hosts are outside of the SR domain,
even ones attached to the local network. If hosts were within the SR
domain they could just source packets with the right extension headers
or encapsulation. But that is not common, especially in a public
network. So it seems likely nearly all packets would end up being
encpasulated anyway (i.e. the packet flow in section 3 will be
common).

If packets are tunneled when they enter the domain then that should
provide sufficient isolation so that the encapsulating node can use
whatever extension headers it wants (e.g. tunneling should ensure that
source hosts won't receive unexpected ICMP). But then that's not EH
insertion, that is sourcing a packet with EH in compliance with
RFC8200. The only argument for EH insertion then would be if some
intermediate node in the domain inserted something; this at least only
affects packets sourced within the domain so information is less
likely to leak out of the domain. However, in that case why not do the
segment routing at the ingress node of the domain in conjuction with
the encapsulation and avoid the complexities needed to make EH
insertion work correctly?

> We think that for further operations that require adding of SRH information
> to such packets originating in your SR domain, the SR insertion is more
> efficient than tunnelling into a new packet.
>
Packet manipulation operations where? At an intermediate node
processing is more likely to take a performance hit because of the
presence of extension headers regardles of whether they were inserted
directly or in an encapsulation,

An assumption that more bits in the packet always reduces efficiency
would be incorrect. For instance, we have shown on real hardware that
GRE/UDP/IP is more efficient than GRE/IP. This is because network
devices have optimized for processing UDP/IP as opposed to GRE/IP so
the benefits of additional eight bytes UDP header outweighs the cost.

A performance analysis based on empirical data should be in the draft
if efficiency of packet manipulation is a primary motivation.

> There can be scenarios in which you have more than one such operation to be
> applied to the packet, let us assume two. If you go for tunnelling you have
> the first encapsulation, then two more IPv6-in-IPv6 encapsulation in the
> same packet, for a total of 3 encapsulations. If you go for tunneling + SR
> insertion you only have the first encapsulation and then "simple" mangling
> of SRH header.
>
I understand how nested encapsulation could work to cross multiple
domains, I don't see how this would be done with SR insertion. Maybe
the intent is that packets will have a stack of routing headers?
Please elaborate this use case in the draft.

Tom

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 2 Dec 2017 20:01:25 +0100

​Tom,​

> However, in that case why not do the

​Because in this particular case what is being done is a *local repair*. ​
Local means that packets are redirected at the node which first notices the
failure (usually adjacent to the link or node failure) which is maximum
order of 10s of ms. Yes FIBs/data plane of those nodes are programmed ahead
of failures with backup rewrites too.

There is no time to propagate such information in the control plane to
ingress such that new/adjusted SRH is imposed to achieve comparable repair
results. That is happening in parallel to packets already being under local
protection, but takes 100s of ms or even seconds for all control plane to
converge.

That is what we call fast connectivity restoration vs protocol convergence.
Completely different things.

So yes there are networks which only use control plane convergence
OSPF/ISIS or even BGP and do nothing to protect packets locally when
failure occurs but those are very bad examples of network design or choice
of vendors which can not perform local repairs.

Thx,
R.

---

From: Brian E Carpenter <[email-masked]>
Date: Sun, 3 Dec 2017 08:11:42 +1300

Stefano,
On 02/12/2017 20:15, Stefano Salsano wrote:
...
> packet headers size) and from the point of view of the cost of packet 

This surprises me. I've had discussions with hardware designers (FPGA
in particular, I think) who seriously hate the IPv6 extension header
design on the grounds of the complexity and cost of manipulating the
header chain at line speed. I can imagine that with a specifcally designed
network processor, this might be less of an issue, but is the market
big enough to support such an NP design?

In any case it seems that the main issue is the need for linked-list
operations to handle a header chain, and that is simpler if
the new packet is

<new header><new EH><old header><old EH chain><payload>

compared to

<old header><head of old EH chain><new EH><tail of old EH chain><payload>

since there is no need to walk, break, and re-link the old header chain.
These days, sticking an extra 40 bytes on the front seems unimportant
compared with the linked list operations.

   Brian

---

From: Tom Herbert <[email-masked]>
Date: Sat, 2 Dec 2017 11:23:10 -0800

On Sat, Dec 2, 2017 at 11:01 AM, Robert Raszuk <[email-masked]> wrote:
> Because in this particular case what is being done is a *local repair*.
> So yes there are networks which only use control plane convergence OSPF/ISIS
> or even BGP and do nothing to protect packets locally when failure occurs
> but those are very bad examples of network design or choice of vendors which
> can not perform local repairs.
>
Thanks for the detail Robert. Has there been any thought into using
the flow label to switch the packet over alternate paths? That might
be a zero overhead solution :-)

Tom

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 2 Dec 2017 20:31:54 +0100

Hi Tom,

> ​​
> be a zero overhead solution :-)

The issue is not at the node doing the actual repair switching ... the
issue is at the other nodes which must forward the packet to achieve loop
free native forwarding.

So if we use flow label for forwarding now each IPv6 router needs to do
three lookups ... flow label + SRH + dst address or even 4 if you include
src-dst forwarding.

In many cases of IP networks there is actually no need to apply anything to
the packet. That's how LFA works. Here we are just talking about some
topologies where you must augment given topology with directed forwarding
to avoid loops. That is what TI-LFA delivers.

Cheers,
R.

On Sat, Dec 2, 2017 at 8:23 PM, Tom Herbert <[email-masked]> wrote:

> OSPF/ISIS
> ​​

---

From: Tom Herbert <[email-masked]>
Date: Sat, 2 Dec 2017 11:47:36 -0800

On Sat, Dec 2, 2017 at 11:31 AM, Robert Raszuk <[email-masked]> wrote:
> The issue is not at the node doing the actual repair switching ... the issue
> is at the other nodes which must forward the packet to achieve loop free
> native forwarding.
>
Right, the idea is that at each node the input flow label and dst map
to an output flow label. All the flow label tables are constructed to
provide give loop free forwarding and multipath selection. Routers in
a domain need to participate (so a controlled domain is required), but
this does not change the on-the-wire format of packets. IIRC a single
depth label like this was proposed in some of the encapsulation
protocols for this same purpose.

> three lookups ... flow label + SRH + dst address or even 4 if you include
SRH would be unneeded in this solution. Lookups would just be dst+flow
label. Encapsulation at ingress is probably still needed if nothing
else to be able to restore the flow label at domain egress.

Tom

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 2 Dec 2017 20:53:28 +0100

> provide give loop free forwarding and multipath selection.

​That's essentially SR-MPLS :) ​

SRH would be unneeded in this solution.

​True - but I was just counting in general node's forwarding capabilites​.

​I think the less moving parts we standardize the better :)

Best,
R.​

---

From: Tom Herbert <[email-masked]>
Date: Sat, 2 Dec 2017 12:26:15 -0800

On Sat, Dec 2, 2017 at 11:53 AM, Robert Raszuk <[email-masked]> wrote:
> That's essentially SR-MPLS :)
>
> True - but I was just counting in general node's forwarding capabilites.
>
Do you expect segment routing to be in a general node's forwarding
capabilities? All the past history seems to indicate that vendors
don't want to do extension header processing in any fast path.

> I think the less moving parts we standardize the better :)
>
Or more generally, the less moving parts we use the better :-)

Tom

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 2 Dec 2017 21:35:35 +0100

> capabilities?

​I do. Based on the discussion with vendors or even forwarding ASICs
manufacturers.​

​Very true - I 100% agree. The crux is in finding the bare minimum which
will still
meet good network design ​requirements.

//R.

---

From: Brian E Carpenter <[email-masked]>
Date: Sun, 3 Dec 2017 11:15:36 +1300

> ​That's essentially SR-MPLS :) ​

If you look at the early history of the flow label, which predated MPLS,
this isn't a big surprise. (IPng-ologists also remember CATNIP,
https://tools.ietf.org/html/rfc1707.)

> ​I think the less moving parts we standardize the better :)

It's true, but the flow label is already in every packet, albeit
with different semantics.

   Brian

---

From: Bob Hinden <[email-masked]>
Date: Mon, 4 Dec 2017 13:28:54 -0800

> On Dec 2, 2017, at 2:15 PM, Brian E Carpenter <[email-masked]> wrote:
> 

I will add that swapping flow labels as I think Tom Herbert was proposing has much less side effects than inserting and processing extension headers.

Bob

---

From: "Darren Dukes (ddukes)" <[email-masked]>
Date: Tue, 5 Dec 2017 14:58:55 +0000

Brian, thanks for starting the thread.  There has been a lot of conversation on this topic now for quite some time, and most recently on this thread, with some good explanations… I see the following:
 
1 – RFC8200 has some non-normative text stating non-source nodes do not insert extension headers. Neither RFC2460 nor RFC8200 disallow an intermediate node from inserting SRH.
    - RFC2460 section 4 did not exclude insertion of SRH’s
    - Errata ID: 4657 suggested some text from Fernando disallowing insertion, however the disposition of that Errata was “Held for Document Update”
    - RFC8200 closed Errata 4657 and there is no normative statement disallowing extension header insertion, rather the suggestion.
2 – Others, including yourself, have provided some constructive suggestions/comments that can be expanded upon in upcoming revisions.  We’ll work toward that.

Thanks
  Darren

On 2017-11-27, 5:31 PM, "Brian E Carpenter" <[email-masked]> wrote:

    Hi,
    
    I'm probably repeating myself, but it seems clear that this draft
    in its present form will never get anywhere. Firstly I will try to
    justify this statement, and secondly I'll try to make some
    constructive suggestions.
    
    I. The problems with the current draft:
    
    I.1. No RFC is allowed to have more than five authors without
    an extremely good reason. If six or seven people have contributed 
    roughly equal amounts you might just be able to persuade the
    RFC Editor to list everybody, but 18? Never.
    https://tools.ietf.org/html/rfc7322#section-4.1.1
    
    I.2. More serious is the tone, which starts with the Abstract.
    "The network operator and vendor community has clearly indicated..."
    is, to put it mildly, out of place in an IETF document. We work
    by facts and rough consensus, not by declarations by vested
    interests. I think a lot of people will simply stop reading
    the draft after that sentence.
    
    The tone needs to be rational and factual argument about why
    the proposed mechanism is both desirable and harmless. The
    abstract needs to be a factual summary.
    
    I.3. A detail is that since this draft recommends ignoring
    a rule in RFC 8200, it needs to be aimed at the standards
    track with Updates: 8200.
    
    I.4. Another detail is that in some places the draft seems
    to assume that only segment routing headers will be
    candidates for insertion. That may be true today, but
    probably not for ever.
    
    II. Constructive suggestions.
    
    I understand that the idea is to propose that this rule
    in https://tools.ietf.org/html/rfc8200#section-4 :
    
    "  Extension headers (except for the Hop-by-Hop Options header) are not
       processed, inserted, or deleted by any node along a packet's delivery
       path, until the packet reaches the node (or each of the set of nodes,
       in the case of multicast) identified in the Destination Address field
       of the IPv6 header."
    
    MAY be ignored within an administrative domain for packets that
    terminate within that domain. If that's the intention, I think
    there are several logical steps in the argument that need to
    be laid out. A lot of the ideas are already in the draft; this
    is mainly about making a clear and logical case that the whole
    community can understand.
    
    1. Introduction, outline of use cases [1].
    2. Precise definition of a domain and of its boundary, ingresses and egresses.
    3. Explanation of how the class of packets eligible for header insertion are identified [2].
    4. Explanation of how the well-known issues are avoided (PMTUD, fragmentation, IPSec).
    5. Clear statement of what header manipulations are allowed and which devices may perform them.
    6. Precise statement of which text in RFC 8200 is updated.
    7. Security considerations 
    
    [1] More details on use cases could be an appendix. 
    
    [2] https://tools.ietf.org/html/draft-voyer-6man-extension-header-insertion-02#section-3 
    is IMHO correct as far as transit packets go. But it
    should be presented as a mainstream part of the proposal,
    not as an example.
    
    I hope this is helpful.
       Brian Carpenter

    On 21/11/2017 07:33, [email-masked] wrote:
    > 
    > A New Internet-Draft is available from the on-line Internet-Drafts directories.
    > 
    > 
    >         Title           : Insertion of IPv6 Segment Routing Headers in a Controlled Domain
    >         Authors         : Daniel Voyer
    >                           Daniel Bernier
    >                           John Leddy
    >                           Clarence Filsfils
    >                           Darren Dukes
    >                           Stefano Previdi
    >                           Stefano Salsano
    >                           Antonio Cianfrani
    >                           David Lebrun
    >                           Olivier Bonaventure
    >                           Prem Jonnalagadda
    >                           Milad Sharif
    >                           Hani Elmalky
    >                           Ahmed Abdelsalam
    >                           Robert Raszuk
    >                           Arthi Ayyangar
    >                           Dirk Steinberg
    > 	Filename        : draft-voyer-6man-extension-header-insertion-02.txt
    > 	Pages           : 14
    > 	Date            : 2017-11-20
    > 
    > Abstract:
    >    The network operator and vendor community has clearly indicated that
    >    IPv6 header insertion is useful and required.  This is notably the
    >    case when the entire journey of the packet remains in its source
    >    domain.  In such a context, it does not matter where the extension
    >    header is inserted.  The source domain may decide to place the IPv6
    >    extension header insertion where it suits its best: at the source of
    >    the packet or at any midpoint within the source domain.
    > 
    > 
    > 
    > The IETF datatracker status page for this draft is:
    > https://datatracker.ietf.org/doc/draft-voyer-6man-extension-header-insertion/
    > 
    > There are also htmlized versions available at:
    > https://tools.ietf.org/html/draft-voyer-6man-extension-header-insertion-02
    > https://datatracker.ietf.org/doc/html/draft-voyer-6man-extension-header-insertion-02
    > 
    > A diff from the previous version is available at:
    > https://www.ietf.org/rfcdiff?url2=draft-voyer-6man-extension-header-insertion-02
    > 
    > 
    > Please note that it may take a couple of minutes from the time of submission
    > until the htmlized version and diff are available at tools.ietf.org.
    > 
    > Internet-Drafts are also available by anonymous FTP at:
    > ftp://ftp.ietf.org/internet-drafts/
    > 
    > _______________________________________________
    > I-D-Announce mailing list
    > [email-masked]
    > https://www.ietf.org/mailman/listinfo/i-d-announce
    > Internet-Draft directories: http://www.ietf.org/shadow.html
    > or ftp://ftp.ietf.org/ietf/1shadow-sites.txt
    >

---

From: Fernando Gont <[email-masked]>
Date: Tue, 5 Dec 2017 13:16:22 -0300

>     - RFC8200 closed Errata 4657 and there is no normative statement disallowing extension header insertion, rather the suggestion.

Have you followed the publication process of RFC8200? :-)

EH insertion is forbiden by RFC8200 as much as it could possibly be.

Thanks,
-- 
Fernando Gont
SI6 Networks
e-mail: [email-masked]
PGP Fingerprint: 6666 31C6 D484 63B2 8FB1 E3C4 AE25 0D55 1D4E 7492

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 6 Dec 2017 08:26:42 +1300

> EH insertion is forbiden by RFC8200 as much as it could possibly be.

To be extra clear: RFC8200 does not use the capitalization of normative
keywords as defined in RFC2119. It is written in plain English. The text
"Extension headers (except for the Hop-by-Hop Options header) are not
 processed, inserted, or deleted by any node along a packet's delivery
 path, until the packet reaches the node..."
is just as normative as every other part of this Internet Standard.

That's why any standards-track version of draft-voyer- would need to
formally update RFC8200.

   Brian

---

From: Mark Smith <[email-masked]>
Date: Wed, 6 Dec 2017 06:59:17 +1100

> I will add that swapping flow labels as I think Tom Herbert was proposing has much less side effects than inserting and processing extension headers.

Which would then violate the Flow Label spec unless the Flow Label
value is zero.

I think the less fiddling with packets in flight the better. Change
creates opportunities for failure, so the less change the better.
Encapsulation/decapsulation is much less change than EH insertion or
modification (and restoration if the original value was non-zero) of
the flow label. Change in the network, if necessary, should be
minimised.

The IPv6 SRH EH is obese because of the use of full 128 IPv6 addresses
as segment identifiers. It is not a requirement to use 128 bit values
as segment identifiers as in SR-MPLS they are only 20 bits i.e. the
MPLS label size.

Fix the SRH obesity problem and then there is no need to make a very
fundamental change to IPv6 EH processing, and no need force customers
to forklift upgrade their existing IPv6 networks to adopt SR
(tunnelled SR is an incremental upgrade, as SR implementations can be
plugged in on the side of existing IPv6 forwarding devices, similar to
the way Ipsilon added IP routing capabilities to ATM switches.)

The network wide forklift upgrade required to adopt this SR via EH
insertion will impact its rate of adoption. The cost of the solution
needs to be significantly cheaper than the cost of continuing to deal
with the problem to encourage adoption. Incremental upgrades rather
than forklift upgrades are cheaper in both capex and opex.

Regards,
Mark.

---

From: Tom Herbert <[email-masked]>
Date: Tue, 5 Dec 2017 12:24:40 -0800

> Which would then violate the Flow Label spec unless the Flow Label
Mark,

The exlcusion that the flow label might be change for "compelling
operational reasons" is in practice interpreted as the flow label may
change in flight (i.e. receivers and network nodes cannot assume flow
label doesn't change). So while doing something like this might be
against the spec as written, I have to think this is potentially much
less problematic and less churn than changing RFC8200 to allow
extension header insertion.

Tom

---

From: Mark Smith <[email-masked]>
Date: Thu, 7 Dec 2017 06:52:55 +1100

Hi Tom,

<snip>

> extension header insertion.

During the specification update of the Flow Label to better formalise
it, I suggested Flow Label domains, with the analog being DSCP
domains. That would have facilitated this sort of use. However, that
was rejected - see Appendix A of RFC6436.

I agree using the flow label is less intrusive than EH insertion,
however it is still a fundamental change to IPv6 packet forwarding.
It's already going to be a slow process to have IPv6 forwarding
devices start using the Flow Label as a load balancing input.

The idea to use the Flow Label for forwarding/switching goes back as
far as 2004, and didn't get legs.

https://datatracker.ietf.org/doc/draft-chakravorty-6lsa/

I don't understand why the SRH header is considered set in concrete,
or more specifically how segments are identified in an IPv6 network,
while far more fundamental changes to IPv6 forwarding are considered
fine. I don't think SR has been widely deployed, and probably less so
over IPv6 networks using EH insertion. It's a technology that is only
a 3 or 4 years old (or something like that), compared to IPv6 which is
now more than 20 and easily accessible in as early as 1998 (I was
playing with Linux's implementation then, and Cisco's implementation a
year later). 20 year old IPv6 forwarding implementations and
deployments would be 100% compatible with SR based on tunnelling.

I can think of a number of ways that SR segments can be identified
that would be compatible with IPv6 forwarding that would reduce the
size of the SRH EH

- as suggested before, unique /64 per device, with the /64 carried in
the SRH TLV, reducing the TLV size by 8 octets.

- well known or locally selected single /64 for the SR function, then
unique 64 bit IID per device, with the 64 bit IID carried in the SRH
TLV, reducing the TLV size by 8 octets. This could be automatically
generated via a PRNG. It would be safer to check for collisions,
although probably not necessary.

- well known or locally selected single /64 for the SR function,
segment IDs 20 bits to match SR-MPLS IDs, 20 bits carred in the SRH
TLV, reducing the TLV size by 13 octets.

- well known or locally selected single /64 for the SR function,
segment IDs 32 bits, and matching the OSPF or BGP IDs, which are
represented as "IPv4 addresses", 32 bits carried in the SRH TLV,
reducing the TLV size by 12 octets. In this latter case, the SR-MPLS
IDs could possibly be auto generated by slicing off the lower 20 bits
of the 32 bit IIDs, as people commonly vary the lower octets when they
allocate OSPF or BGP IDs.

Regards,
Mark.

---

From: Tom Herbert <[email-masked]>
Date: Wed, 6 Dec 2017 13:32:37 -0800

Mark,

I don't see how that's going to help much. The authors of this draft
have already stated that in order to doing EH insertion they will need
to encapsulate when packets enter the SR domain, so that's forty-bytes
of overhead right of the bat. And then there's the fixed overhead of
the EH before we finally get to the savings of compressing the EH
data.

But even more important than the size of the headers, is the
processing cost. Processing fixed encapsulation at high speed is
expensive. Processing open ended header chains and TLVs is _really_
expensive to the point that they to date they are undeployable. While
I appreciate the optimism of the authors that the problem of
processing EH TLVs at line rate has been solved, I must admit I am
skeptical. It is hard to ignore a long history which has not been kind
to header chains or TLVs in IP.

The obvious value of using something like the flow label is that it is
a zero overhead solution in the wire protocol. No encpasulation and no
EH. The route lookup for this could be implemented in a TCAM on
src,dst,flow label and the action is just to write a 20 bit value.
ILA, which serves the overlay network use case, is even simpler-- this
is a fixed length lookup on the destination and a write back to the
destination address. Flow label is not used and no change to standard
protocols is required,

Tom

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 7 Dec 2017 11:51:51 +1300

On 07/12/2017 08:52, Mark Smith wrote:
...> I don't understand why the SRH header is considered set in concrete,

The issue is surely whether it is already set in silicon. While
we can argue about anything, there isn't much point in arguing with
hardware.

   Brian

---

From: Mark Smith <[email-masked]>
Date: Thu, 14 Dec 2017 06:38:51 +1100

Hi Tom,

Sorry not to get back to you sooner.

<snip>

> the EH before we finally get to the savings of compressing the EH

So my understanding of the fundamental problem SR is trying to solve
is to remove (MPLS) path state and path monitoring RSVP load from the
control plane. I think this is a good idea, as I think packet networks
scale by pushing complexity out of the network and towards the edge
("dumb network, smart hosts").

The path information is instead shifted to the each packet, carried in
the SRH. While I think using IPv6 addresses to identify hops in an
IPv6 networks is an obvious first choice, it does add considerable
per-packet overhead. 128 bits hop identifiers aren't necessary if
SR-MPLS only needs and uses 20 bits for the same purpose. The schemes
I suggested before would still carrying IPv6 forwarding compatible hop
identifiers while reducing this overhead.

EH insertion to avoid tunnelling overhead as a response to a large SRH
header because of 128 bit hop identifiers is just shifting complexity
that has been moved out of the control plane by SR back into the
network, and into the forwarding plane. I think IPv6 SR looses a lot
of value if it shifts complexity in the network rather than removing
it.

> expensive.

I don't think that's avoidable (although it has occurred to me that
when IPv6 becomes ubiquitous, it might be possible to use it directly
as a the layer 2 on-the-wire frame format).

> Processing open ended header chains and TLVs is _really_
> expensive to the point that they to date they are undeployable.

Agree. This is why the divide-and-conquer approach of pushing
complexity to the edge of the network and into the hosts helps us
build faster and faster networks.

The simpler the processing in the network, the faster it can be done
and it's also less prone to error because there are less things that
can go wrong.

With NAT and middleboxes in IPv4, we've ended up with more complexity
in the network and I've had enough experiences with that that I want
to the simplicity of IPv4 networks before NAT and middleboxes back in
IPv6.

> While

Using the flow label has the advantages you mention, however updating
it is still putting complexity back into the forwarding plane, and is
a fork lift upgrade of the forwarding plane to get it.

The use of the flow label for ECMP also faces this forwarding plane
upgrade problem, although much less so - the upgrade to gain it is
localised to the parts of the network where you need it. You don't
need to universally upgrade all your forwarding plane devices just to
gain flow label ECMP balancing across multiple links between two of
your forwarding plane devices.

Broadly, there seems to be a view that flag day upgrades are
acceptable for new network technologies. That's a far more expensive
and disruptive way to upgrade the capabilities network than the
piecemeal, incremental and localised method that has been used to
build the Internet since the ARPANET flag day. I don't think it should
be preferred, I think it should be the last resort.

> destination address. Flow label is not used and no change to standard

Agree. I think ILA's techniques are much better because they leave the
core forwarding plane alone, with the ILA changes occurring either in
the host or in the first hop network device. The IPv6 forwarding plane
remains conventional, simpler and standard.

Regards,
Mark.

---

From: Tom Herbert <[email-masked]>
Date: Wed, 13 Dec 2017 17:10:56 -0800

> of value if it shifts complexity in the network rather than removing
I suspect that if EH insertion can't be done without encapsulation
then the value proposition for using extension headers here largely
evaporates The overhead of encapsulation is not insignifanct and the
issues of incompatibility of using any EH are still present. An
alternative is to put the routing information as payload of the
encpasulation instead of EH. That is the approach taken by BIER for
instance. For the cost of an eight byte UDP header they get
compatability on any network (and with IPv4 for that matter).

> as a the layer 2 on-the-wire frame format).
Considering that everything is pretty much layer 3 switched point to
point links you could do that. But probably more hassle to change
everything than it's worth.

> remains conventional, simpler and standard.
Less is more!

Tom