Subject: [sfc] draft-ietf-sfc-nsh-04 with L2 encapsulation

From: Christian Koenning <[email-masked]>
Date: Tue, 17 May 2016 19:35:20 +0000

Good afternoon,

we have been discussing internally the semantics of 
NSH L2 encapsulation as per:

https://tools.ietf.org/html/draft-ietf-sfc-nsh-04

9.1.  GRE + NSH

    L2 Frame:
    +----------+--------------------+--------------------+
    |L2 header | L3 header, proto=47|GRE header,PT=0x894F|
    +----------+--------------------+--------------------+
    ---------------+---------------+
    NSH, NP=0x3    |original frame |
    ---------------+---------------+
    
9.2.  VXLAN-gpe + NSH

    L2 Frame:
    +----------+------------------------+---------------------+
    |L2 header | IP + UDP dst port=4790 |VXLAN-gpe NP=0x4(NSH)|
    +----------+------------------------+---------------------+
    ---------------+---------------+
    NSH,NP=0x3     |original frame |
    ---------------+---------------+

9.3.  Ethernet + NSH
  L2 Frame:
  +-------------------------------+---------------+----------------+
  |Outer Ethernet, ET=0x894F      | NSH, NP = 0x3 | original frame |
  +-------------------------------+---------------+----------------+
  
Also, let us recap that the forwarding behaviour is derived from the
SPI/SI mapping defined by the controlplane.

see p.20 

   +-------------------------------------------------------+
   |  SPI |  SI |  NH                 |   Transport        |
   +-------------------------------------------------------+
   |  10  | 255 |  1.1.1.1            |   VXLAN-gpe        |
   |  10  | 254 |  2.2.2.2            |   nvGRE            |
   |  10  | 251 |  10.1.2.3           |   GRE              |
   |  40  | 251 |  10.1.2.3           |   GRE              |
   |  50  | 200 |  01:23:45:67:89:ab  |   Ethernet         |
   |  15  | 212 |  Null (end of path) |   None             |
   +-------------------------------------------------------+
                     Figure 9: SFF NSH Mapping Example
                     
Now looking at a specific example for forwarding to a VXLAN-GPE
endpoint, is the next-hop defined as the outer layer destination, or is
this the inner layer destination, and some form of ARP or predefined
resolution is assumed ?

Or even to ask more specifically, when forwarding from a SFF to a SF
using VXLAN-GPE NSH with L2 encapsulation, is it expected that the inner
L2 destination address is being changed to the resolved L2 address of
the SF ?

The missing clarity is derived from the definition above, "original
frame." This can either mean encapsulation happens at Classifier Level,
and then the L2 Header will stay untouched. But what is then the purpose
of it ? Or does this mean the inner L2 Addresses need to to rendered hop
by hop as in traditional networks. This would mean we have a possible
unwanted dependency.

The question can be taken out of the transport scope, and simple be
defined as “Does NSH L2 encapsulation care about the inner L2 headers ?”

Please allow us to clarify this.

Many Thanks

Christian Koenning
--
F5 Networks Inc.
[email-masked]

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 17 May 2016 15:48:06 -0400

I am having trouble parsing your question.  I may be misreading inner 
and outer.
The "inner" packet is whatever NSH is carrying.  That is whatever was 
sent into the system.  Individual SF may play with that, but NSH 
forwarding ignores that.

Now, the outer packet depends upon the tunnel type.  If the tunnel is a 
pure Ethernet tunnel, then the NSH table tells you the L2 Address to use 
for the outer packet.
If the tunnel is an L3 encapsulated tunnel (VxLAN, GRE, ...) then the 
NSH information gives you the L3 next hop.  It is the responsibility of 
the outer IP system, not the NSH system, to resolve that IP address to 
an actual Interface and if needed (usually) an L2 MAC address.  NSH and 
SFC don't care how the system does that.  Most commmonly, it is ARP / 
IPv6 ND.  But it can be static configuration or some other form of magic.

Yours,
Joel

On 5/17/16 3:35 PM, Christian Koenning wrote:
> _______________________________________________
> sfc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/sfc
>

---

From: Christian Koenning <[email-masked]>
Date: Tue, 17 May 2016 19:57:53 +0000

Thanks Joel,

Allow me to clarify:

When looking at the inner L2 payload ( outer (NSH) inner L2), 
is the SFF/SF required to change the destination mac address to
what the next-hop might expect ? 

Your answer

>sent into the system.  Individual SF may play with that, but NSH 

Looking at the outer payload, for e.g. a VXLAN-GPE endpoint, you say 
that normal mechanism define the forwarding.

In the specifc example of below
    +-------------------------------------------------------+
    |  SPI |  SI |  NH                 |   Transport        |
    +-------------------------------------------------------+
    |  10  | 255 |  1.1.1.1            |   VXLAN-gpe        |

It means that the VTEP address is 1.1.1.1 ? How about the VNI ?

Christian

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 17 May 2016 16:08:33 -0400

In looking at this question, I sent Christian the following:

Looking at pieces of packets on the wire, for a VXLAN packet carry NSH, 
I will see:

Ehternet Header with Ethernet address this device and that of the next 
IP device towards the IP destination;
IP Header with the IP Source equal to this device and IP Dest equal to 
that of the next SFF, as indicated by the table;
UDP header;
NSH Header;
Original packet, which may be an IP packet or an Ethernet Packet.

If the Original packet is an Ethernet packet, NSH will never change teh 
Ethernet headers in that packet.
Conversely, the first listed Ethernet Header is added and subtracted at 
each hop in accordance with IP forwarding rules.

Based on this, he indicated that his question was indeed about the inner 
packet.  That is left unmodified by any SFF.  Service functions can of 
course do what they like.

Yours,
Joel

---

From: Christian Koenning <[email-masked]>
Date: Wed, 18 May 2016 22:37:06 +0000

Thanks Joel,

this really helps to clarify.

>On 17/05/2016, 21:08, "Joel M. Halpern" <[email-masked]> wrote:

Just to confirm, as all semantics of forwarding within the Service
Function Chaining (SFC) enabled domain are derived from NSH
SPI/SI/Metadata, it also means the inner encapsulation will not need to
support e.g. ARP ?

Or let me ask differently, is ethertype the only reason that the full L2
header is carried through ? If so, why not rewrite the mac addresses to
some reserved IANA range?

https://tools.ietf.org/html/draft-ietf-sfc-nsh-04
9.  NSH Encapsulation Examples

    L2 Frame:
    ---------------+———————+

Many Thanks

Christian Koenning
--
F5 Networks Inc.
[email-masked]

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 18 May 2016 18:43:01 -0400

The carried packet can be either L2 or L3.  The reason is that different 
users may get different services.

For example, if the service chain is being used to deliver a virtualized 
Residential Gateway, it may be processing the Ethernet packets from the 
subscriber.  At some point in the chain, the Ethernet header may (or may 
not) be removed by a service function.  Of course, the subscriber 
service could be an IP service, in which case all that gets delivered 
into the chain is an IP packet.

That is why NSH does not mandate the inner packet type, but rather has a 
type field.

As for how the sender may have derived his Ethernet destination address, 
that depends very much on deployments.  I can think of several different 
ways to do it.  While some of those will require specific service 
functions to make them work, the underlying SFC and NSH mechanisms are 
the same for all of them.

Yours,
Joel

---

From: Ron Parker <[email-masked]>
Date: Thu, 19 May 2016 14:30:36 +0000

I would add that if the "inner" DMAC of a carried L2 frame needed to be changed for some application-specific reason (e.g., some sort of proxy functionality), that would be the responsibility of a service function and not the responsibility of an SFF.

   Ron

-----Original Message-----
From: sfc [mailto:[email-masked]] On Behalf Of Joel M. Halpern
Sent: Wednesday, May 18, 2016 6:43 PM
To: Christian Koenning <[email-masked]>; [email-masked]
Subject: Re: [sfc] draft-ietf-sfc-nsh-04 with L2 encapsulation

The carried packet can be either L2 or L3.  The reason is that different users may get different services.

For example, if the service chain is being used to deliver a virtualized Residential Gateway, it may be processing the Ethernet packets from the subscriber.  At some point in the chain, the Ethernet header may (or may
not) be removed by a service function.  Of course, the subscriber service could be an IP service, in which case all that gets delivered into the chain is an IP packet.

That is why NSH does not mandate the inner packet type, but rather has a type field.

As for how the sender may have derived his Ethernet destination address, that depends very much on deployments.  I can think of several different ways to do it.  While some of those will require specific service functions to make them work, the underlying SFC and NSH mechanisms are the same for all of them.

Yours,
Joel

On 5/18/16 6:37 PM, Christian Koenning wrote:
>> Based on this, he indicated that his question was indeed about the 
>> inner packet.  That is left unmodified by any SFF.  Service functions 
>> can of course do what they like.
>
> SPI/SI/Metadata, it also means the inner encapsulation will not need 
> to support e.g. ARP ?
>
> Or let me ask differently, is ethertype the only reason that the full 
> L2 header is carried through ? If so, why not rewrite the mac 
> addresses to some reserved IANA range?
>
>> Looking at pieces of packets on the wire, for a VXLAN packet carry 
>> NSH, I will see:
>>
>> Ehternet Header with Ethernet address this device and that of the 
>> next IP device towards the IP destination; IP Header with the IP 
>> Source equal to this device and IP Dest equal to that of the next 
>> SFF, as indicated by the table; UDP header; NSH Header; Original 
>> packet, which may be an IP packet or an Ethernet Packet.
>>
>> If the Original packet is an Ethernet packet, NSH will never change 
>> teh Ethernet headers in that packet.
>> Conversely, the first listed Ethernet Header is added and subtracted 
>> at each hop in accordance with IP forwarding rules.
>>
>> Based on this, he indicated that his question was indeed about the 
>> inner packet.  That is left unmodified by any SFF.  Service functions 
>> can of course do what they like.
>>
>> Yours,
>> Joel
>>
>>> When looking at the inner L2 payload ( outer (NSH) inner L2), is the 
>>> SFF/SF required to change the destination mac address to what the 
>>> next-hop might expect ?
>>>
>>> Your answer
>>>> The "inner" packet is whatever NSH is carrying.  That is whatever 
>>>> was sent into the system.  Individual SF may play with that, but 
>>>> NSH forwarding ignores that.
>>>
>>>
>>>> I am having trouble parsing your question.  I may be misreading 
>>>> inner and outer.
>>>> The "inner" packet is whatever NSH is carrying.  That is whatever 
>>>> was sent into the system.  Individual SF may play with that, but 
>>>> NSH forwarding ignores that.
>>>>
>>>> Now, the outer packet depends upon the tunnel type.  If the tunnel 
>>>> is a pure Ethernet tunnel, then the NSH table tells you the L2 
>>>> Address to use for the outer packet.
>>>> If the tunnel is an L3 encapsulated tunnel (VxLAN, GRE, ...) then 
>>>> the NSH information gives you the L3 next hop.  It is the 
>>>> responsibility of the outer IP system, not the NSH system, to 
>>>> resolve that IP address to an actual Interface and if needed 
>>>> (usually) an L2 MAC address.  NSH and SFC don't care how the system 
>>>> does that.  Most commmonly, it is ARP /
>>>>> we have been discussing internally the semantics of NSH L2 
>>>>> encapsulation as per:
>>>>>
>>>>> Also, let us recap that the forwarding behaviour is derived from 
>>>>> the SPI/SI mapping defined by the controlplane.
>>>>>
>>>>> see p.20
>>>>>
>>>>> endpoint, is the next-hop defined as the outer layer destination, 
>>>>> or is this the inner layer destination, and some form of ARP or 
>>>>> predefined resolution is assumed ?
>>>>>
>>>>> Or even to ask more specifically, when forwarding from a SFF to a 
>>>>> SF using VXLAN-GPE NSH with L2 encapsulation, is it expected that 
>>>>> the inner
>>>>> L2 destination address is being changed to the resolved L2 address 
>>>>> of the SF ?
>>>>>
>>>>> The missing clarity is derived from the definition above, 
>>>>> "original frame." This can either mean encapsulation happens at 
>>>>> Classifier Level, and then the L2 Header will stay untouched. But 
>>>>> what is then the purpose of it ? Or does this mean the inner L2 
>>>>> Addresses need to to rendered hop by hop as in traditional 
>>>>> networks. This would mean we have a possible unwanted dependency.
>>>>>
>>>>>
>>>>>
>>>>> The question can be taken out of the transport scope, and simple 
>>>>> be defined as “Does NSH L2 encapsulation care about the inner L2 headers ?”
>>>>>