Subject: [nvo3] Concerns about NVO3 dataplane requirements document

From: Erik Nordmark <[email-masked]>
Date: Tue, 21 Oct 2014 08:37:29 -0700

I expressed this on the phone at the interim meeting and was asked to 
post with a  bit more detail.

According to the call the intended purpose of the requirements document 
is to help the WG choose between different proposed dataplane 
encapsulation protocols. However, I get the impression that 
draft-ietf-nvo3-dataplane-requirements was not written with that purpose 
in mind.

To be clear, the document contains useful background text and various 
discussion on what needs to be done at encapsulation and decapsulation 
points either by an implementation or operationally. However, that does 
not help with the (current) intended purpose.

The actual requirements on encapsulation protocol are few and weak. 
Section 3.3.1 contains a few of them.
But even the requirement on VNID is weak. The document says that there 
MUST be a VNID, but then goes on to

        This field MAY be an
        explicit, unique (to the administrative domain) virtual network
        identifier (VNID) or MAY express the necessary context information
        in other ways (e.g. a locally significant identifier).

While in theory locally significant indentifiers can be made to work, 
they would require an additional control-plane mechanism to handle the 
(dynamic?) mapping between VNID  and the local identifier. Especially 
with BGP EVPN now being handled in the BESS WG, I think we should have a 
requirement for our dataplane that it MUST contain a VNID field.

That section also discusses QoS/CoS. But it's requirement is a MAY for a 
QoS field in the NVO3 overlay header. Either we should require such a 
field or not; otherwise this doesn't help us choose.
(And my personal take is that we can have solutions which map between TS 
QoS/CoS and underlay CoS without also having some QoS field in the NVO3 
header. But my overall point is that the "MAY" isn't a helpful criteria 
to choose between protocols.)

Those are the only "requirements" on the dataplane I've found in the 
document.

There might be other required or desired properties that are not in the 
draft. For instance, one can contemplate a requirement that the 
encapsulation MUST/SHOULD facilitate ECMP in unmodified routers in the 
underlay (e.g., using the common technique of UDP encaps with entropy 
placed in the UDP source port field).

Thus my take is that a document with requirements on the NVO3 dataplane 
encapsulation can be a page or two. Some introduction and background 
followed by a list of required and desired properties of the  
encapsulation format such as:
  - MUST contain an VNID field. This field MUST be large enough to scale 
to 100's of thousands of virtual networks
  - ??? QoS field inside the NVO3 overlay header or not ???
  - MUST/SHOULD facilitate ECMP in unmodified routers in the underlay
(Others participants might have other requirements on the encapsulation 
format. My main message is the focus on the encaps requirements which 
seems to be quite few.)

The implementation and operational requirements in 
draft-ietf-nvo3-dataplane-requirements should IMHO belong in a different 
document. Some might already be covered in the architecture document. 
And others might need to be refined as we specify more details for the 
NVO3 solution.

Hence if we are going to get WG consensus on a dataplane encapsulation 
requirements document I strongly suggest we start small with the above 
(3, give or take) bullets being the actual content.

    Erik

Perfection is achieved, not when there is nothing more to add, but when 
there is nothing left to take away.
    *Antoine de Saint-Exupery
    <http://www.quotationspage.com/quotes/Antoine_de_Saint-Exupery/>*
    /French writer (1900 - 1944)/

---

From: Tom Herbert <[email-masked]>
Date: Tue, 21 Oct 2014 12:33:09 -0700

Hi Eric,

Thank you for sending this. Some comments in line.

On Tue, Oct 21, 2014 at 8:37 AM, Erik Nordmark <[email-masked]> wrote:
>
> I expressed this on the phone at the interim meeting and was asked to post
> with a  bit more detail.
>
> According to the call the intended purpose of the requirements document is
> to help the WG choose between different proposed dataplane encapsulation
> protocols. However, I get the impression that
> draft-ietf-nvo3-dataplane-requirements was not written with that purpose in
> mind.
>
> points either by an implementation or operationally. However, that does not
> help with the (current) intended purpose.
>
> The actual requirements on encapsulation protocol are few and weak. Section
> 3.3.1 contains a few of them.
> But even the requirement on VNID is weak. The document says that there MUST
> be a VNID, but then goes on to
>
>        This field MAY be an
> While in theory locally significant indentifiers can be made to work, they
> would require an additional control-plane mechanism to handle the (dynamic?)
> mapping between VNID  and the local identifier. Especially with BGP EVPN now
> being handled in the BESS WG, I think we should have a requirement for our
> dataplane that it MUST contain a VNID field.
>
> That section also discusses QoS/CoS. But it's requirement is a MAY for a QoS
> field in the NVO3 overlay header. Either we should require such a field or
> not; otherwise this doesn't help us choose.
> header. But my overall point is that the "MAY" isn't a helpful criteria to
> choose between protocols.)
>
>
> underlay (e.g., using the common technique of UDP encaps with entropy placed
> in the UDP source port field).
>
> followed by a list of required and desired properties of the  encapsulation
> format such as:

Agreed, but I think there are a few more probably.

>  - MUST contain an VNID field. This field MUST be large enough to scale to
> 100's of thousands of virtual networks

In even moderate sized deployments hierarchical assignment of VNIDs,
sub-VNs, fragmentation in the space, classed VNIDs might be used--
this probably should be reflected in requirements to allow for that (I
have pointed this out previously).

I believe the bigger item missing around VNID  is security. Isolation
between VNs isolation is a critical requirement, so it follows that
the VNID must be adequately secured to protect against spoofing or
corruption.

>  - ??? QoS field inside the NVO3 overlay header or not ???

I would add that a congestion control mechanism in the encapsulation
layer might also be considered.

>  - MUST/SHOULD facilitate ECMP in unmodified routers in the underlay

Is it reasonable to require UDP based encapsulation to support this,
or at least that any protocol can easily be encapsulated within UDP
(like being done for GRE/UDP)?

> format. My main message is the focus on the encaps requirements which seems
> to be quite few.)

With respect to encapsulation protocol requirements, I would propose
that extensibility (ability to associate meta data with the
encapsulation) is also a requirement.

> document. Some might already be covered in the architecture document. And
> others might need to be refined as we specify more details for the NVO3
> solution.
>

One other thing that should be clarified: The protocol of the
encapsulated packet is described in the req. draft as
"Tenant frame: Ethernet or IP based upon the VNI type".

This is a requirement for a multi-protocol encapsulation which is
good, but seems restrictive in the protocols carried and implies that
protocol is inferred from VNID as opposed the to requiring/allowing
the encapsulation layer to have a protocol field.

Thanks,
Tom

> requirements document I strongly suggest we start small with the above (3,
> give or take) bullets being the actual content.
>
>    Erik
>
>
> there is nothing left to take away. Antoine de Saint-Exupery
> French writer (1900 - 1944)
>
> _______________________________________________
> nvo3 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nvo3
>

---

From: Erik Nordmark <[email-masked]>
Date: Tue, 21 Oct 2014 14:27:44 -0700

> Agreed, but I think there are a few more probably. 

Tom,
I think so too - just need to get the specific requirements written down 
and get some consensus.

> have pointed this out previously).

Are you suggesting that the numeric requirement should be higher? 32 
bit? 64 bit?

I'm trying to tease out some fairly specific requirements that can 
influence the dataplane encapsulation. Could be to have a larger initial 
number of bits, or the ability to expand on the size of the VNID over 
time. Part of this this in with the security question below.
> the VNID must be adequately secured to protect against spoofing or

I understand your concern, but I'm trying to get some actual 
requirements written down.
An example requirement in this space would be that the encapsulation 
format should have the option of providing some keyed hash (or similar) 
which covers at least the VNID to make it harder for an attacker to 
spoof NVO3 packets.

One can get similar security by having a larger VNID space which is 
sparsely populated. A 32 bit VNID plus 64 bit of hash isn't that 
different than a 96 bit VNID unless the hash also covers the inner 
packet (which would be expensive).

In the world of solutions one can envision a variable length VNID which 
can be broken up into flexible parts:
  - ID
  - key ID (to allow for smooth rekeying)
  - keyed hash
But that might be overkill.

Currently I don't quite know how to phrase a requirement without talking 
about potential solutions. We could have a nebulous requirement that the 
dataplane encapsulation format MUST allow the addition of higher 
assurance mechanisms that prevent off-path attackers (those that can not 
intercept NVO3 packets flying by) from injecting NVO3 packets.

> layer might also be considered.

"considered" doesn't sound like a requirement. Did you have something 
specific in mind?

There has been discussion in the IETF about requiring at least some form 
of circuit breaker for tunneling protocols. But the proposals I've seen 
(from Bob Briscoe and folks) seem to need nothing from the tunnel encaps 
protocol (might yse the ECN  bits in the outer IP header). We could list 
some "heads up" non-requirements in the draft such as "The encapsulation 
protocol should not prevent adding circuit breakers for congestion 
control" even though we do not yet know what that would imply.

> (like being done for GRE/UDP)?

A UDP encaps would be the easy way to solve it. But first of all the WG 
needs to decide whether this should be a requirement on the encapsulation.

> encapsulation) is also a requirement.

I guess we would need a bit more detail to get to a requirement which 
helps in the selection.
For instance, is it sufficient to have 2 bits of meta-data? Or do we 
need the ability for a large variable-length list of meta-data (with 
e.g. a userid in the form of a NAI plus a certificate and CRLs ...) I'm 
exaggerating to push on the size question.

Also, some of the meta-data discussion has been about vendor-specific 
meta-data.
If we have a multi-protocol encaps (see below) then vendor-specific 
meta-data can be handled by the vendors getting a code point and doing 
their own header between the NVO3 and the inner Ethernet/IP header. That 
avoids us having a standardization discussion about how much size to 
allocate for some proprietary meta-data that we don't even know about.

Thus if there is meta-data that needs to be standardized, then I think 
we should discuss that specific meta-data and the resulting requirements 
on the dataplane encapsulation.
> the encapsulation layer to have a protocol field.

The WG should entertain whether supporting some protocol type in the 
encapsulation is a requirement or not.
The short list of requirements is silent on that front.

And as you've pointed out, if there is such a requirement, then the next 
question is whether it is an Ethernet type or an IP protocol field (or 
both).

    Erik

---

From: Marc Binderberger <[email-masked]>
Date: Wed, 22 Oct 2014 17:20:33 -0700

Hello Eric and Tom,

from Eric's signature:

   Perfection is achieved, not when there is nothing more to add,
   but when there is nothing left to take away.

I think this is what Eric tries to achieve :-)  Reading the document there is 
some good background/motivation but sometimes many options too - maybe too 
many?

To pick up some of the points:

VNI: we live with "flat" IP addresses and yet they support the rich structure 
in the name space. I don't see why this should be different with overlay 
headers: the control plane (or the configuration) will know about any 
structure and will program the data plane accordingly; VNIs are then just a 
reference numbers (read: flat).

If you plan for something fancy like e.g. copy-VlanID-into-VNI-bits, I would 
still leave this to control-/config plane to negotiate this capability and an 
additional document to describe such features. No need to discuss all 
potential options or ideas now. We just need enough VNI bits.

QoS: I would consider additional QoS bits in the NVO3 overlay header as 
redundant. Either the tenant frame and underlay header have some QoS already, 
then we have the requirement for the data plane to be able to map QoS values 
(probably some small table). Or the tenant-frame has no QoS - well, sounds 
like a fixed mapping then.

Otherwise it's hard to re-use existing underlay forwarding technology (well, 
IPv4/v6).

Security:  could we re-use IPSEC ESP/AH ?  In tunnel mode as we would add 
already an underlay IPv4/IPv6 header?
(I'm no expert in this area but why not re-using other peoples work)

ECMP: with leaf-spine topologies in mind and IP as an underlay I would say 
being able to use already existing IP ECMP methods is a plus to simplify 
deployments. I would make it a requirement.

Meta-Data: I probably missed some discussions (sorry!) but what data would 
this be?
Anyway, it sounds TLV-like and having a variable overhead length may be a 
problem for the overlay MTU. Assuming that this Meta-Data is orthogonal to 
the VNI, would another "MD-ID" field help?  The control-/config plane could 
then map this MD-ID to the Meta-Data and program the data plane accordingly.

I think another point, which was mentioned on the list, is the 
fragmentation/reassembly or MTU problem. For simplicity I would prefer the 
NVO3 header has no support for this. If your tenant frame is IPv4/v6 then 
fragmentation/reassembly should happen on this level. For Ethernet tenant 
frames - no idea but I assume Ethernet networks solve the MTU problem by 
"correct configuration"? So the NVO3 "link" would just be another interface 
with an unusual MTU (?).

The document also mentions the "learning bridge" behaviour. I would have seen 
the details of MAC learning as "control plane" (albeit not necessarily the 
"centralized authority" of the charter).  For the data plane it is a 
requirement to punt packets to the control plane. Well, actually forward the 
packet and punt a copy to control plane. I wonder if we have other 
requirement to trigger such a copy/punt? (e.g. an OAM/alert flag, as 
discussed in VXLAN-gpe)

Regards, Marc

On Tue, 21 Oct 2014 14:27:44 -0700, Erik Nordmark wrote:
> Are you suggesting that the numeric requirement should be higher? 32 bit? 
> 64 bit?
> 
> number of bits, or the ability to expand on the size of the VNID over time. 
> Part of this this in with the security question below.
>> 
> I understand your concern, but I'm trying to get some actual requirements 
> written down.
> An example requirement in this space would be that the encapsulation format 
> should have the option of providing some keyed hash (or similar) which 
> covers at least the VNID to make it harder for an attacker to spoof NVO3 
> packets.
> 
> sparsely populated. A 32 bit VNID plus 64 bit of hash isn't that different 
> than a 96 bit VNID unless the hash also covers the inner packet (which 
> would be expensive).
> 
> In the world of solutions one can envision a variable length VNID which can 
> be broken up into flexible parts:
>  - ID
> dataplane encapsulation format MUST allow the addition of higher assurance 
> mechanisms that prevent off-path attackers (those that can not intercept 
> NVO3 packets flying by) from injecting NVO3 packets.
> 
> There has been discussion in the IETF about requiring at least some form of 
> circuit breaker for tunneling protocols. But the proposals I've seen (from 
> Bob Briscoe and folks) seem to need nothing from the tunnel encaps protocol 
> (might yse the ECN  bits in the outer IP header). We could list some "heads 
> up" non-requirements in the draft such as "The encapsulation protocol 
> should not prevent adding circuit breakers for congestion control" even 
> though we do not yet know what that would imply.
> 
>>> seems
> I guess we would need a bit more detail to get to a requirement which helps 
> in the selection.
> For instance, is it sufficient to have 2 bits of meta-data? Or do we need 
> the ability for a large variable-length list of meta-data (with e.g. a 
> userid in the form of a NAI plus a certificate and CRLs ...) I'm 
> Thus if there is meta-data that needs to be standardized, then I think we 
> should discuss that specific meta-data and the resulting requirements on 
> the dataplane encapsulation.
>>

---

From: Tom Herbert <[email-masked]>
Date: Thu, 23 Oct 2014 08:35:42 -0700

Hi Marc,

Some replies in line...

> potential options or ideas now. We just need enough VNI bits.
Right, the question then is what is minimum number of bits needed.

> (I'm no expert in this area but why not re-using other peoples work)
Interoperability with IPSEC should also be a requirement. Assuming
that an encapsulation header is still needed, there are two possible
use cases: apply ESP/AH on outer packet or apply ESP/AH to inner
packet where the IPSEC header is between network overlay header and
inner IP header, The second is needed if we want to keep the network
virtualization of the packet visible to the network (e.g. firewall by
VNID).

> Meta-Data: I probably missed some discussions (sorry!) but what data would

Security option to validate VNID/encapsulation headers, congestion
control data at encapsulation layer if we develop a method that uses
inband signaling, data performance optimizations (remote checksum
offload and a form of large segment offload have been proposed),
possibly a CRC to cover payload. In GUE the VNID itself is also
optional data.

I believe the forward looking requirement is that we have a protocol
that can adapt to changing conditions and new threats in the DC over
its lifetime. While TCP has proven easy to extend in this environment,
we mostly lack this capability at L3 currently.

> then map this MD-ID to the Meta-Data and program the data plane accordingly.
Yes, every byte that we add to the encapsulation headers effectively
reduces our client's MTU by one byte. We need a very compact and
efficient format for optional data (one reason why GUE does not use
TLVs).

> "correct configuration"? So the NVO3 "link" would just be another interface
This shouldn't be different than any other network tunnels. It is
common practice to set the MTU on a virtual interface to account for
the maximum overhead needed for encapsulation (MTU = link_MTU -
overhead).

---

From: Erik Nordmark <[email-masked]>
Date: Fri, 24 Oct 2014 14:18:41 -0700

> possibly a CRC to cover payload. In GUE the VNID itself is also

Tom,
Those are useful features to consider. However, I wouldn't view them as 
being part of the meta-data discussion. The meta-data discussion was 
triggered by the Geneve draft which proposed ways to encode 
vendor-specific data e.g. for applications like firewalls.

(I guess we need a definition of "meta-data").

    Erik

---

From: Tom Herbert <[email-masked]>
Date: Fri, 24 Oct 2014 14:24:48 -0700

On Fri, Oct 24, 2014 at 2:18 PM, Erik Nordmark <[email-masked]> wrote:

> triggered by the Geneve draft which proposed ways to encode vendor-specific
> data e.g. for applications like firewalls.
>
> Would the term "optional data" be more apropos for what I'm proposing then?

Tom

---

From: Erik Nordmark <[email-masked]>
Date: Fri, 24 Oct 2014 17:44:31 -0700

> reference numbers (read: flat).

I guess we still need to have some idea how many bits would be required 
up front (24, 32, more?) and whether we think this field needs to be 
extensible.

> like a fixed mapping then.

OK
> (I'm no expert in this area but why not re-using other peoples work)

One question is whether the higher assurance is just for the VNI or for 
the whole encapsulated frame. Using something like ESP/AH takes us down 
the path of protecting the whole frame, which might be overkill.
> deployments. I would make it a requirement.
OK

> Meta-Data: I probably missed some discussions (sorry!) but what data would

As I tried to clarify in my response to Tom the meta-data discussion in 
the IETF was mostly about vendor-specific service meta-data, but perhaps 
this term is being used for more general extensibility?

I think there should be ways to add better assurance (checksum, keyed 
hashes) for the NVO3 header. But perhaps that can be in fixed fields in 
a fixed length header.

In terms of the overall architecture there is a desire to carry some 
service meta-data with frames. The sfc WG is thinking about doing that 
using a separate NSH header.

It would be good for the NVO3 WG to have a clear understanding of what 
data needs to be carried with each encapsulated frame. That helps 
determine how flexible and extensible the packet format needs to be.
The experience with extensibility for protocols that are in the 
dataplane (be it IPv4 options, IPv6 extension headers, TRILL options, 
etc) is that they don't tend to get implemented in hardware. And the 
dataplane protocols tend to have a mixture of hardware and software 
implementations - which is different than TCP which is mostly software.
One observation is that we (the IETF + industry) seems to be able to 
redefine fixed-fields (e.g., IPv4 TOS->DSCP+ECN, MPLS labels with new 
semantics like the entropy label) a lot easier than implementing new 
options or extension headers.

> then map this MD-ID to the Meta-Data and program the data plane accordingly.

One would have to require that the underlay MTU exceeds the overlay MTU 
by the maximum encapsulation overhead. Thus a large max size of 
options/extensions has some cost.
> "correct configuration"? So the NVO3 "link" would just be another interface

That seems to be how the hardware encapsulations handle things.
If it was all software on the endpoints then there would be more 
options, but for efficiency we typically want to avoid fragmentation.

> requirement to trigger such a copy/punt? (e.g. an OAM/alert flag, as
While the option of "learning bridge" behavior might be useful, it 
doesn't have anything to do with the dataplane encapsulation format.

Your question about OAM/alert flags is a good one. I think it makes 
sense to define some flags.
Perhaps we also want a "drop packet if you don't know about this flags" 
flag; in many cases the control plane can be used to determine the 
capabilities hence one can avoid sending dataplane packets with some new 
OAM or other feature to endpoints that don't know about it. In such a 
case it is sufficient to have flags that have the "ignore if you don't 
know about it" semantics.

    Erik

---

From: Erik Nordmark <[email-masked]>
Date: Fri, 24 Oct 2014 17:45:26 -0700

On 10/24/14 2:24 PM, Tom Herbert wrote:
> Tom,
>
>     Those are useful features to consider. However, I wouldn't view
>     them as being part of the meta-data discussion. The meta-data
>     discussion was triggered by the Geneve draft which proposed ways
>     to encode vendor-specific data e.g. for applications like firewalls.
>

Or just options/extensions?

Regards,
     Erik

---

From: Jon Hudson <[email-masked]>
Date: Fri, 24 Oct 2014 19:26:44 -0700

Just one comment...

> virtualization of the packet visible to the network (e.g. firewall by

110% agree. More and more conversations with customers are turning to security & encryption concerns.

IPSEC support/interoperability for me is a requirement 

Jon

---

From: Marc Binderberger <[email-masked]>
Date: Sat, 25 Oct 2014 23:28:09 -0700

Hello Erik,

> I guess we still need to have some idea how many bits would be required up 
> front (24, 32, more?) and whether we think this field needs to be 
> extensible.

I would go with a VNI of 24 or 32bit as this seems reasonable. I still have 
to understand why the data plane processing would need to have the concept of 
VNI hierarchy, I would have thought that's an abstraction for the control 
plane (?).

At the end some bit limit must exist or we face problems with (hardware-) 
implementations. We can solve the open question later with an extension 
header - but you mention below this didn't work so well for other protocols - 
or define a next-version NVO3 header when there is need.

For my comment about IPSEC/AH:

> One question is whether the higher assurance is just for the VNI or for the 
> whole encapsulated frame. Using something like ESP/AH takes us down the 
> path of protecting the whole frame, which might be overkill.

agree. My (naive?) idea was to let AH in the NVO3 case only cover the 
underlay IP header, the NVO3 header and maybe the tenant frame header 
(IPv4/v6 or the Ethernet header). Honestly, have to re-read the specs if 
there is a way to chieve this. It would protect all the information needed 
when decapsulating the NVO3 overlay. The tenant frame protection I would 
leave to a separate AH/ESP step before the overlay encap and after decap.

If my idea is bad or impractical then we need to introduce NVO3's own hashing 
procedure.

> As I tried to clarify in my response to Tom the meta-data discussion in the 
> IETF was mostly about vendor-specific service meta-data, but perhaps this 
> term is being used for more general extensibility?

that's how I was thinking about it. This may work as long as the encap/decap 
entity - which can be a NIC, switch/router etc. - does not need to understand 
what the meta-data means and can just add/remove it as opaque data.

> hashes) for the NVO3 header. But perhaps that can be in fixed fields in a 
> fixed length header.

You mean the usual fields of "auth type", "key id" and such, typically part 
of an authentication TLV, should be part of the NVO3 header from the 
beginning? Nothing optional but build-in from the start? 

> using a separate NSH header.

Even if we only cover IPv4, IPv6 and Ethernet as payload we need some 
protocol or next-header field, IMHO. This could cover e.g. NSH as well.

Saying this, one could then also define a next-header value for OAM instead 
of having a OAM (aka RA) flag - and let the hardware trigger on the value 
instead of a flag.
Nevertheless, I think a reserved field for flags is a good idea if we have 
the room in the NVO3 header.

Regards, Marc

> It would be good for the NVO3 WG to have a clear understanding of what data 
> needs to be carried with each encapsulated frame. That helps determine how 
> flexible and extensible the packet format needs to be.

> The experience with extensibility for protocols that are in the dataplane 
> (be it IPv4 options, IPv6 extension headers, TRILL options, etc) is that 
> they don't tend to get implemented in hardware. And the dataplane protocols 
> tend to have a mixture of hardware and software implementations - which is 
> different than TCP which is mostly software.
> One would have to require that the underlay MTU exceeds the overlay MTU by 
> the maximum encapsulation overhead. Thus a large max size of 
> If it was all software on the endpoints then there would be more options, 
> but for efficiency we typically want to avoid fragmentation.
> 
> While the option of "learning bridge" behavior might be useful, it doesn't 
> have anything to do with the dataplane encapsulation format.
> 
> Your question about OAM/alert flags is a good one. I think it makes sense 
> to define some flags.
> OAM or other feature to endpoints that don't know about it. In such a case 
> it is sufficient to have flags that have the "ignore if you don't know 
> about it" semantics.
> 
>    Erik
>

---

From: Tom Herbert <[email-masked]>
Date: Sun, 26 Oct 2014 13:20:20 -0700

On Fri, Oct 24, 2014 at 5:44 PM, Erik Nordmark <[email-masked]> wrote:
> whole encapsulated frame. Using something like ESP/AH takes us down the path
> of protecting the whole frame, which might be overkill.
>>
>>
>>
> In terms of the overall architecture there is a desire to carry some service
> meta-data with frames. The sfc WG is thinking about doing that using a
> separate NSH header.
>

I don't believe this is always true. We have verified that at least
two NICs and one switch chip are capable of parsing any combination of
keyid, sequence number, and checksum fields in GRE for the purposes of
calculating a flow hash on the inner header. In fact, we've been able
to overload the sequence number and checksum fields for our own
options in lieu of HW not supporting a general L3 extensibility
mechanism (like IP options).

> semantics like the entropy label) a lot easier than implementing new options
> or extension headers.
>
> Your question about OAM/alert flags is a good one. I think it makes sense to
> define some flags.
> capabilities hence one can avoid sending dataplane packets with some new OAM
> or other feature to endpoints that don't know about it. In such a case it is
> sufficient to have flags that have the "ignore if you don't know about it"
> semantics.
>
>    Erik
>

---

From: Marc Binderberger <[email-masked]>
Date: Mon, 27 Oct 2014 01:37:39 -0700

Hello Tom et al.,

> calculating a flow hash on the inner header.

Not a contradiction to Erik's statement. He said "tend to" which I think is a 
fair statement. It's not black/white, if your hardware is a fully 
programmable NP then you have a chance for later changes - assuming you still 
have memory/cycles left. If your hardware is an ASIC, typically a pipeline of 
functional blocks, then you may be bound to what was foreseen during the ASIC 
design phase.

> In fact, we've been able

There is a difference between "not supporting" and "not capable to support". 
With NICs you may be actually lucky wrt the hardware used. Nevertheless, I 
think it's not what Erik is highlighting: that the discussion is still a bit 
vague what the "base" functionality should be, what extensions we can already 
foresee, what extensions or details we can postpone even when this means the 
ASIC-based equipment (like a good number of switches on the market) will not 
handle it in hardware (until an ASIC re-spin).

Regards, Marc