Subject: Minor SRH clarifications

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 3 May 2018 23:10:30 -0400

I think all of the following items can be easily addressed.  It may be 
that they already are addressed and I simply missed it.  This email 
covers three loosely related questions:

1 - Internal source to external destination.  I presume it is intended 
to be permitted that an internal packet source may be configured to use 
SRH to control traversal of the segment routing domain even for packets 
that are going to destinations outside the domain.
If the SRH header is put into the base packet by that source, then the 
domain edge can not remove it.  If that were done, the SRH would leak 
out into domains that do not reliably support the flow label 
requirement, and thus can not perform ECMP.
Thus, it seems that there is a requirement that if a source node in an 
SR domain is sending to an eventual destination outside the SR domain, 
the source node needs to use the same encapsulation that an entry edge 
uses, so that the SRH ends at the domain egress and that egress can 
remove the SRH.

2 - Ingress node to internal destination implication on internal 
destination.  If an external packet arrives with a destination that is 
an internal node of the SR domain, the edge node will still need to 
encapsulate the packet in an outer IPv6 header to carry the SRH.
This implies that nodes which may receive packets from outside the SR 
domain need to be prepared to receive IPv6 packets with SRH in an outer 
SRH which they need to strip.  This probably also implies that jsut as 
they need to be prepared to support SRH forwarding, the nodes need to be 
prepared to forward the packet if the inner IPv6 destination is not the 
receiving node.

3) This does lead to the question of whether having a mode without an 
encapsulating header is particularly useful, given that everything has 
to support the encapsulated mode?

Thank you,
Joel

---

From: "Darren Dukes (ddukes)" <[email-masked]>
Date: Mon, 14 May 2018 20:45:13 +0000

Hi Joel, see inline.

> On May 3, 2018, at 11:10 PM, Joel M. Halpern <[email-masked]> wrote:
> 
> I think all of the following items can be easily addressed.  It may be that they already are addressed and I simply missed it.  This email covers three loosely related questions:
> 
> 1 - Internal source to external destination.  I presume it is intended to be permitted that an internal packet source may be configured to use SRH to control traversal of the segment routing domain even for packets that are going to destinations outside the domain.
> If the SRH header is put into the base packet by that source, then the domain edge can not remove it.  If that were done, the SRH would leak out into domains that do not reliably support the flow label requirement, and thus can not perform ECMP.
> Thus, it seems that there is a requirement that if a source node in an SR domain is sending to an eventual destination outside the SR domain, the source node needs to use the same encapsulation that an entry edge uses, so that the SRH ends at the domain egress and that egress can remove the SRH.

I don’t agree with the core of your question:
1 - the flow label is necessary for SR domain ingress routers which perform ip-in-ip encap as per the draft.  Not a source node that is a host within the SR Domain.
2 - If a source decides it is within it’s local policy to send a packet to a destination outside the SR Domain I don’t think this decision should be second-guessed by a border node.

Therefore it is not necessary for the source node (i.e., host) to use the same encap as an SR Domain ingress router, unless that is truly the behavior it requires.

> 
> 2 - Ingress node to internal destination implication on internal destination.  If an external packet arrives with a destination that is an internal node of the SR domain, the edge node will still need to encapsulate the packet in an outer IPv6 header to carry the SRH.

True if the default behavior for every destination within the SR Domain is to encapsulate to the nexthop of that destination.
False if that’s not the default behavior - i.e. external sources are routable from within the SR Domain.

> This implies that nodes which may receive packets from outside the SR domain need to be prepared to receive IPv6 packets with SRH in an outer SRH which they need to strip.  This probably also implies that jsut as they need to be prepared to support SRH forwarding, the nodes need to be prepared to forward the packet if the inner IPv6 destination is not the receiving node.
> 

Yes, in the “true” case above, no in the “False” case above.

> 3) This does lead to the question of whether having a mode without an encapsulating header is particularly useful, given that everything has to support the encapsulated mode?

No, not given the answer to #1 and #2.

Darren

> 
> Thank you,
> Joel
> 
> --------------------------------------------------------------------
> IETF IPv6 working group mailing list
> [email-masked]
> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> --------------------------------------------------------------------

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 14 May 2018 16:51:05 -0400

I am not following your explanations.  Elaborations in line, marked 
<jmh> </jmh>.  Apologies if I am missing the obvious.

Yours,
Joel

> Therefore it is not necessary for the source node (i.e., host) to use the same encap as an SR Domain ingress router, unless that is truly the behavior it requires.

<jmh>This is not about the flow label.  This is about the SRH itself. 
If the source originates its IP packet with an SRH in the regular IPv6 
header that it uses, then the SRH header can not be removed at the 
domain edge.  Intermediate routers can't remove extension headers.  This 
means that the SRH "leaks" rather forcefully out of the domain.  That 
seems a very bad behavior.  </jmh>

> False if that’s not the default behavior - i.e. external sources are routable from within the SR Domain.

<jmh>This si not about default behavior.  The ingress domain edge is not 
permitted, by the IPv6 specs, to add an extension header to the base 
packet.  As such, it has to use the encapsulation mechanism.  So the 
internal host has to be prepared to receive the encapsulated packet.

If you really want to permit teh domain edge to add the SRH into the 
base packet when the destination is within the domain, then
1) You need to sqay that in the spec.  It doesn't say it.
2) You need to call out that this is a change from RFC 8200 behavior.
</jmh>

---

From: "Darren Dukes (ddukes)" <[email-masked]>
Date: Mon, 14 May 2018 21:21:05 +0000

Joel, see <dd> inline

On May 14, 2018, at 4:51 PM, Joel M. Halpern <[email-masked]<mailto:[email-masked]>> wrote:

I am not following your explanations.  Elaborations in line, marked <jmh> </jmh>.  Apologies if I am missing the obvious.

Yours,
Joel

On 5/14/18 4:45 PM, Darren Dukes (ddukes) wrote:
Hi Joel, see inline.
On May 3, 2018, at 11:10 PM, Joel M. Halpern <[email-masked]<mailto:[email-masked]>> wrote:

<jmh>This is not about the flow label.

<dd> see highlighted text above where you mention flow label - thus I answered the flow label question.

 This is about the SRH itself. If the source originates its IP packet with an SRH in the regular IPv6 header that it uses, then the SRH header can not be removed at the domain edge.  Intermediate routers can't remove extension headers.  This means that the SRH "leaks" rather forcefully out of the domain.  That seems a very bad behavior.  </jmh>

<dd>Yes, if a source originates a packet within an SR Domain and that source inserts a segment list to reach a destination outside the domain then, by definition, the source tells the destination what the segment list is.
It may be foolish to do this unintentionally.  Of course in cases where it is not foolish (i.e. internal SIDs are protected at the domain edge) it may be useful.  That is up to the application adding the SRH.

<jmh>This si not about default behavior.  The ingress domain edge is not permitted, by the IPv6 specs, to add an extension header to the base packet.  As such, it has to use the encapsulation mechanism.  So the internal host has to be prepared to receive the encapsulated packet.

If you really want to permit teh domain edge to add the SRH into the base packet when the destination is within the domain, then
1) You need to sqay that in the spec.  It doesn't say it.
2) You need to call out that this is a change from RFC 8200 behavior.
</jmh>

<dd> Joel, this is an IPv6 network, if a node A:: outside the SR domain  sends a packet destined to a node B:: within an SR Domain, the edge can easily permit A:: to send to B:: without encapsulating in outer IP and SRH.  The same is true for the packet from B:: to A::, B:: need not add an SRH.  It’s just IPv6 forwarding.  This was my point with “False” above.

Darren

Thank you,
Joel

--------------------------------------------------------------------
IETF IPv6 working group mailing list
[email-masked]<mailto:[email-masked]>
Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
--------------------------------------------------------------------

---

From: Joel Halpern Direct <[email-masked]>
Date: Mon, 14 May 2018 17:29:17 -0400

Further in line with <jmh2>.
Joel

On 5/14/18 5:21 PM, Darren Dukes (ddukes) wrote:
>>>> I think all of the following items can be easily addressed.  It may 
>>>> be that they already are addressed and I simply missed it.  This 
>>>> email covers three loosely related questions:
>>>>
>>>> 1 - Internal source to external destination.  I presume it is 
>>>> intended to be permitted that an internal packet source may be 
>>>> configured to use SRH to control traversal of the segment routing 
>>>> domain even for packets that are going to destinations outside the 
>>>> domain.
>>>> If the SRH header is put into the base packet by that source, then 
>>>> the domain edge can not remove it.  If that were done, the SRH would 
>>>> leak out into domains *_that do not reliably support the flow label 
>>>> requirement_*, and *_thus can not perform ECMP._*
>>>> Thus, it seems that there is a requirement that if a source node in 
>>>> an SR domain is sending to an eventual destination outside the SR 
>>>> domain, the source node needs to use the same encapsulation that an 
>>>> entry edge uses, so that the SRH ends at the domain egress and that 
>>>> egress can remove the SRH.
>>> 1 - the flow label is necessary for SR domain ingress routers which 
>>> perform ip-in-ip encap as per the draft.  Not a source node that is a 
>>> host within the SR Domain.
>>> 2 - If a source decides it is within it’s local policy to send a 
>>> packet to a destination outside the SR Domain I don’t think this 
>>> decision should be second-guessed by a border node.
>>> Therefore it is not necessary for the source node (i.e., host) to use 
>>> the same encap as an SR Domain ingress router, unless that is truly 
>>> the behavior it requires.
>>

<jmh2> The flow label was an example of problems outside the domain. </jmh2>

> 
>>  This is about the SRH itself. If the source originates its IP packet 
>> with an SRH in the regular IPv6 header that it uses, then the SRH 
>> header can not be removed at the domain edge.  Intermediate routers 
>> can't remove extension headers.  This means that the SRH "leaks" 
>> rather forcefully out of the domain.  That seems a very bad behavior. 
>>  </jmh>
> 
> <dd>Yes, if a source originates a packet within an SR Domain and that 
> source inserts a segment list to reach a destination outside the domain 
> then, by definition, the source tells the destination what the segment 
> list is.
> It may be foolish to do this unintentionally.  Of course in cases where 
> it is not foolish (i.e. internal SIDs are protected at the domain edge) 
> it may be useful.  That is up to the application adding the SRH.
> 

<jmh2> The SRH works specifically because it places requirements on the 
domain to support it.  Not just in specific places, but in supporting 
flow label and permitting packets with extension headers.  Those 
requirements are not met outside of the SRH domain.  As such, sending 
packets outside the SRH domain, has problems.
1) If the receiver does nto know what to do with an SRH, then the sender 
has to know that magically?  And can not use SRH for intra-domain 
steering in that case?  An unfortuante restriction.
2) Since the SRH will leak, there is a significant risk that the packet 
will not make it to the destination.
If the soruce were told to use the encapsulating mode, the edge could 
(and presumably would) remove the outer header, thus removing the SRH 
and solving the problem.
You refer to the domain edge protecting internal SIDs.  Do you expect 
(without documentation) that the edge device will over-write the content 
of the SRH?
</jmh2>

>>>> destination.  If an external packet arrives with a destination that 
>>>> is an internal node of the SR domain, the edge node will still need 
>>>> to encapsulate the packet in an outer IPv6 header to carry the SRH.
>>> True if the default behavior for every destination within the SR 
>>> Domain is to encapsulate to the nexthop of that destination.
>> <jmh>This si not about default behavior.  The ingress domain edge is 
>> not permitted, by the IPv6 specs, to add an extension header to the 
>> base packet.  As such, it has to use the encapsulation mechanism.  So 
>> the internal host has to be prepared to receive the encapsulated packet.
>>
> <dd> Joel, this is an IPv6 network, if a node A:: outside the SR domain 
>   sends a packet destined to a node B:: within an SR Domain, the edge 
> can easily permit A:: to send to B:: without encapsulating in outer IP 
> and SRH.  The same is true for the packet from B:: to A::, B:: need not 
> add an SRH.  It’s just IPv6 forwarding.  This was my point with “False” 
> above.

<jmh2>The case of interest is when node A outside is sending a packet to 
node B inside, and the domain edge wishes to apply traffic steering use 
SRH.  If it uses an encapsulating header (which seems the right way to 
do it) then node B has to be prepared to accept the packet with an extra 
encapsulating header.   It may be that one can argue that it is already 
prepared to do so.  If so, say that.  If not, state the requirement.
</jmh2>

> 
> Darren
> 
>>
>>>> This implies that nodes which may receive packets from outside the 
>>>> SR domain need to be prepared to receive IPv6 packets with SRH in an 
>>>> outer SRH which they need to strip.  This probably also implies that 
>>>> jsut as they need to be prepared to support SRH forwarding, the 
>>>> nodes need to be prepared to forward the packet if the inner IPv6 
>>>> destination is not the receiving node.
>>>>
>>>> 3) This does lead to the question of whether having a mode without 
>>>> an encapsulating header is particularly useful, given that 
>>>> everything has to support the encapsulated mode?
>>>> [email-masked] <mailto:[email-masked]>

---

From: Tom Herbert <[email-masked]>
Date: Tue, 15 May 2018 11:14:24 -0700

On Mon, May 14, 2018 at 2:29 PM, Joel Halpern Direct
<[email-masked]> wrote:
>>> I am not following your explanations.  Elaborations in line, marked <jmh>
>>> </jmh>.  Apologies if I am missing the obvious.
>>>
>>> Yours,
>>> Joel
>>>
>>>>> that they already are addressed and I simply missed it.  This email covers
>>>>> three loosely related questions:
>>>>>
>>>>> to be permitted that an internal packet source may be configured to use SRH
>>>>> to control traversal of the segment routing domain even for packets that are
>>>>> going to destinations outside the domain.
>>>>> domain edge can not remove it.  If that were done, the SRH would leak out
>>>>> into domains *_that do not reliably support the flow label requirement_*,
>>>>> and *_thus can not perform ECMP._*
>>>>> SR domain is sending to an eventual destination outside the SR domain, the
>>>>> source node needs to use the same encapsulation that an entry edge uses, so
>>>>> that the SRH ends at the domain egress and that egress can remove the SRH.
>>>>
>>>> perform ip-in-ip encap as per the draft.  Not a source node that is a host
>>>> within the SR Domain.
>>>> 2 - If a source decides it is within it’s local policy to send a packet
>>>> to a destination outside the SR Domain I don’t think this decision should be
>>>> second-guessed by a border node.
>>>> the same encap as an SR Domain ingress router, unless that is truly the
>>>> behavior it requires.
>>>
>>>
>>> with an SRH in the regular IPv6 header that it uses, then the SRH header can
>>> not be removed at the domain edge.  Intermediate routers can't remove
>>> extension headers.  This means that the SRH "leaks" rather forcefully out of
>>> the domain.  That seems a very bad behavior.  </jmh>
>>
>>
>> then, by definition, the source tells the destination what the segment list
>> is.
>> It may be foolish to do this unintentionally.  Of course in cases where it
>> is not foolish (i.e. internal SIDs are protected at the domain edge) it may
>> be useful.  That is up to the application adding the SRH.
>>
>
> domain to support it.  Not just in specific places, but in supporting flow
> label and permitting packets with extension headers.  Those requirements are
> not met outside of the SRH domain.  As such, sending packets outside the SRH
> domain, has problems.

Joel,

I still don't understand why use of ECMP and flow labels are hard
requirements for SR. If someone were to deploy SR in a network that
doesn't completely support these, does that mean the protocol will
fail some how? Please explain why these stringent requirements are
needed.

Also, if these such requirements are needed then it would probably be
appropriate to require that the flow label is properly set and packet
is ECMP route whenever SR EH is used (cover both encapsulation or non
encapsulation cases).

> 1) If the receiver does nto know what to do with an SRH, then the sender has
> to know that magically?  And can not use SRH for intra-domain steering in
> that case?  An unfortuante restriction.

How is that scenario any different than doing IPIP encapsulation with
SR EH and sending to an endpoint that doesn't know what to do with EH?

> 2) Since the SRH will leak, there is a significant risk that the packet will
> not make it to the destination.

Is that risk any worse than using any other EH in Internet?

> If the soruce were told to use the encapsulating mode, the edge could (and
> presumably would) remove the outer header, thus removing the SRH and solving
> the problem.
> (without documentation) that the edge device will over-write the content of
> the SRH?

IMO, this alludes to the most significant problem of leaking SR, it
exposes internal addresses which is a potential security issue. It's
probably worth mentioning that in the Security Considerations section.

Tom

>>>>> destination.  If an external packet arrives with a destination that is an
>>>>> internal node of the SR domain, the edge node will still need to encapsulate
>>>>> the packet in an outer IPv6 header to carry the SRH.
>>>>
>>>> True if the default behavior for every destination within the SR Domain
>>>> is to encapsulate to the nexthop of that destination.
>>> permitted, by the IPv6 specs, to add an extension header to the base packet.
>>> As such, it has to use the encapsulation mechanism.  So the internal host
>>> has to be prepared to receive the encapsulated packet.
>>>
>>> If you really want to permit teh domain edge to add the SRH into the base
>>> packet when the destination is within the domain, then
>> sends a packet destined to a node B:: within an SR Domain, the edge can
>> easily permit A:: to send to B:: without encapsulating in outer IP and SRH.
>> The same is true for the packet from B:: to A::, B:: need not add an SRH.
>> It’s just IPv6 forwarding.  This was my point with “False” above.
>
>
> node B inside, and the domain edge wishes to apply traffic steering use SRH.
> If it uses an encapsulating header (which seems the right way to do it) then
> node B has to be prepared to accept the packet with an extra encapsulating
> header.   It may be that one can argue that it is already prepared to do so.
> If so, say that.  If not, state the requirement.
> </jmh2>
>
>>
>> Darren
>>
>>>
>>>>> domain need to be prepared to receive IPv6 packets with SRH in an outer SRH
>>>>> which they need to strip.  This probably also implies that jsut as they need
>>>>> to be prepared to support SRH forwarding, the nodes need to be prepared to
>>>>> forward the packet if the inner IPv6 destination is not the receiving node.
>>>>>
>>>>> encapsulating header is particularly useful, given that everything has to
>>>>> support the encapsulated mode?
>>>>

---

From: Joel Halpern Direct <[email-masked]>
Date: Tue, 15 May 2018 14:19:58 -0400

Trimmed, with comments following trimming:

On 5/15/18 2:14 PM, Tom Herbert wrote:
...
> is ECMP route whenever SR EH is used (cover both encapsulation or non

The problem I am concerned about is that if IPv6 packets with SRH are 
sent across the general Internet, under current circumstances many of 
them will be dropped.  Some of them will merely be concentrated on fewer 
links than they should be.

Since we are not the protool police, and most devices out there are not 
currently capable and configured to do flow label ECMP, saying "use flow 
label ECMP" is not vialbe for the public Internet  I am not objecting to 
the requirement that this mechanism be supported inside a domain that 
has chosen to use SRH.  But if SRH leaks, it places this requirement on 
the whole Internet.

My proposal was to simply make sure that SRH does not leak.  Which is 
consistent with the original IESG requirements placed on this work.

Yours,
Joel

---

From: Ole Troan <[email-masked]>
Date: Tue, 15 May 2018 20:55:21 +0200

Joel,

I don’t quite understand why this is relevant specifically to SRH. Don’t your concerns apply equally to any other extension header, IP tunnels, fragmented packets...?

If yes, I suggest this is not within the scope of this draft. 

Cheers,
Ole

> On 15 May 2018, at 20:19, Joel Halpern Direct <[email-masked]> wrote:
> 
>> ....
> The problem I am concerned about is that if IPv6 packets with SRH are sent across the general Internet, under current circumstances many of them will be dropped.  Some of them will merely be concentrated on fewer links than they should be.
> 
> Since we are not the protool police, and most devices out there are not currently capable and configured to do flow label ECMP, saying "use flow label ECMP" is not vialbe for the public Internet  I am not objecting to the requirement that this mechanism be supported inside a domain that has chosen to use SRH.  But if SRH leaks, it places this requirement on the whole Internet.
> 
> My proposal was to simply make sure that SRH does not leak.  Which is consistent with the original IESG requirements placed on this work.
> 
> Yours,
> Joel
>

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 15 May 2018 15:00:52 -0400

This draft specifies two ways to carry the extension header.  One, like 
most tunnels, is easily removed at the domain edge.  The other is not.
I made a request and an observation.

The request was that all packets originating in the domain, but going 
outside the domain, use the easily removed format.

The other was to observe that since this easily removed format is what 
has to be used for packets from outside the domain, we should document 
that internal nodes need to be prepared to receive this format.

This kind of thing is something that all tunnel efforts deal with. 
either before they get to the IESG, or after the IESG pushes back.

Yours,
Joel

---

From: "Darren Dukes (ddukes)" <[email-masked]>
Date: Tue, 15 May 2018 21:12:44 +0000

Splitting this conversation and trimming to second part:
See <dd>

> <jmh2>The case of interest is when node A outside is sending a packet to node B inside, and the domain edge wishes to apply traffic steering use SRH.  If it uses an encapsulating header (which seems the right way to do it) then node B has to be prepared to accept the packet with an extra encapsulating header.

<dd> Yes, SR Domain edge node 1 wants to perform ip-in-ip encap destined to B then B must also be terminating that ip-in-ip payload.  If B can not do this directly then the ip-in-ip encapsulation formed at 1 must terminate at a node close to B that is capable of such decapsulation and forwarding to B. I see nothing new here, is this not obvious Joel?

>   It may be that one can argue that it is already prepared to do so.  If so, say that.  If not, state the requirement.
> </jmh2>

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 15 May 2018 17:25:04 -0400

No, it is not obvious.  Put differently, very little is "obvious" to all 
readers of a document.
In fact, what you describe below is harder than it looks.
If we assume that not all nodes in the domain are required to accept 
IP-in-IP SRH, then I do not know how the edge device will know whether a 
given target can process it.  (Selecting an alternative tunnel 
termination point is a bit tricky, but seems doable, if the ingress can 
know it needs to do so.)

the general rule for RFCs is that if there is behavior that is required, 
state it.  What is obvious to one writer is not obvious to all readers.

Yours,
Joel

...

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 16 May 2018 11:19:06 +1200

> termination point is a bit tricky, but seems doable, if the ingress can 

I've even seen it suggested that *any* IPv6 node that receives an
IPv6-in-IPv6 packet addressed to itself should automatically
decapsulate it and then do whatever it would have done if the
inner packet had arrived by itself. At first sight that sounds OK,
but I suspect that the security area review would be short and
emphatic.

   Brian

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 15 May 2018 19:25:09 -0400

What I would expect for end nodes is that they be required to accept 
IP-in-IP SRH, and then to check that the inner header is also addressed 
to them, dropping the packet otherwise.  Unless the SRH folks want tot 
turn all nodes in the domain into forwarders, which would seem an 
undesirable design choice.

Yours,
Joel