Subject: [nvo3] I-D Action: draft-ietf-nvo3-geneve-01.txt

From: [email-masked]
Date: Thu, 14 Jan 2016 12:27:03 -0800

A New Internet-Draft is available from the on-line Internet-Drafts directories.
 This draft is a work item of the Network Virtualization Overlays Working Group of the IETF.

        Title           : Geneve: Generic Network Virtualization Encapsulation
        Authors         : Jesse Gross
                          Ilango Ganga
	Filename        : draft-ietf-nvo3-geneve-01.txt
	Pages           : 26
	Date            : 2016-01-14

Abstract:
   Network virtualization involves the cooperation of devices with a
   wide variety of capabilities such as software and hardware tunnel
   endpoints, transit fabrics, and centralized control clusters.  As a
   result of their role in tying together different elements in the
   system, the requirements on tunnels are influenced by all of these
   components.  Flexibility is therefore the most important aspect of a
   tunnel protocol if it is to keep pace with the evolution of the
   system.  This draft describes Geneve, a protocol designed to
   recognize and accommodate these changing capabilities and needs.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-nvo3-geneve/

There's also a htmlized version available at:
https://tools.ietf.org/html/draft-ietf-nvo3-geneve-01

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-nvo3-geneve-01

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

---

From: Jesse Gross <[email-masked]>
Date: Thu, 14 Jan 2016 20:38:40 +0000

This draft update has a number of editorial changes compared to the previous version:
 * Switching to an editors/contributors model as the author list had grown fairly large.
 * A number of clarifications based on questions or suggestions received from implementors and others. There are no changes to the format, only improvements to wording to make the existing intentions more clear.
 * Initial allocation of a few values in the option class registry for open source projects which are using Geneve already.

On 1/14/16, 12:27 PM, "nvo3 on behalf of [email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]> on behalf of [email-masked]<mailto:[email-masked]>> wrote:

Abstract:

_______________________________________________
nvo3 mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/nvo3

---

From: Tom Herbert <[email-masked]>
Date: Mon, 18 Jan 2016 13:01:08 -0800

>    recognize and accommodate these changing capabilities and needs.
A couple of comments...

The discussion of efficient implementation (section 2.2.1) seems vague
to me. For example, from the draft:

"As new functionality becomes sufficiently well defined to add to
endpoints, supporting options can be designed using ordering
restrictions and other techniques to ease parsing."

What are these ordering restrictions? Does this mean TLVs have (or
might eventually have) some defined ordering as to how they can appear
in the packet? Would this contradict section 4.3: "Option ordering is
not significant".

What are "other techniques to ease parsing"?

Thanks,
Tom

> [email-masked]

---

From: Jesse Gross <[email-masked]>
Date: Wed, 20 Jan 2016 00:13:44 +0000

On 1/18/16, 1:01 PM, "nvo3 on behalf of Tom Herbert" <[email-masked]<mailto:[email-masked]> on behalf of [email-masked]<mailto:[email-masked]>> wrote:

On Thu, Jan 14, 2016 at 12:27 PM,  <[email-masked]<mailto:[email-masked]>> wrote:

Abstract:

A couple of comments...

The draft is purposely trying to be agnostic to different implementations and obviously doesn’t want to mandate any specific techniques. There are several different types of software/devices and different architectures within those classes so it would be difficult to say what the “right” way to do things is today, let alone the future. However, given the number of implementations that either already exist or are emerging, it seems like this is a solvable problem.

Thanks for pointing out the possible inconsistency in the text – I’ll take a look at that.

Jesse

---

From: Tom Herbert <[email-masked]>
Date: Tue, 19 Jan 2016 18:09:44 -0800

On Tue, Jan 19, 2016 at 4:13 PM, Jesse Gross <[email-masked]> wrote:
> On 1/18/16, 1:01 PM, "nvo3 on behalf of Tom Herbert" <[email-masked]
> on behalf of [email-masked]> wrote:
>
> The draft is purposely trying to be agnostic to different implementations
> and obviously doesn’t want to mandate any specific techniques. There are
> several different types of software/devices and different architectures
> within those classes so it would be difficult to say what the “right” way to
> do things is today, let alone the future. However, given the number of
> implementations that either already exist or are emerging, it seems like
> this is a solvable problem.
>
That really doesn't answer my question. Let me rephrase it: "What are
the normative requirements of Geneve with respect to TLV ordering?"

I would extrapolate that they are:

1) Per the protocol, TLV ordering is not relevant. A transmitter MAY
send TLVs in any order. A receiver MUST accept TLVs in any order.
Packets with the same options in a different order MUST be processed
alike.

2) A transmitter SHOULD send TLVs in the prescribed ordering (by
option class and type?). A receiver MAY optimize processing for the
prescribed ordering.

Tom

---

From: Tom Herbert <[email-masked]>
Date: Wed, 20 Jan 2016 10:41:41 -0800

A few more comments:..

General:

The use of the term frame versus packet are inconsistent. For
instance, in section:

"If the receiving endpoint does not recognize this option and this bit
is set then the frame MUST be dropped."

but later

"Packets in which the total length of all options is not equal to the
'Opt Len' in the base header are invalid and MUST be silently dropped
if received by an endpoint."

I think both of these mean the same thing which is to drop the packet.
IMO defining a Geneve frame is unnecessary. Showing outer Ethernet
headers in header format does not add anything useful to the
specification. For that matter, showing inner Ethernet headers is not
necessary either since protocol type could be just as well be other
Ethertypes (IPv4, IPv6, etc.).

>From section 4.2: "some applications may limit the types of options
which are supported or enforce a maximum number or length of options."
This allowance permits applications to set the maximum number of
options to zero, that is to say they don't have to support options at
all. So what is to prevent Geneve TLVs from suffering the same fate of
IP options which were obsoleted by decisions in implementation not to
support them?

>From section 3.3:

"The checksum MAY be set to zero on transmit for packets encapsulated
in both IPv4 and IPv6 [RFC6935]." RFC6935 does not give a blank check
for disabling the UDP checksum is IPv6, there are restrictions and
limitations. Please look at RFC7510  (MPLS/UDP) and GRE/UDP draft to
see how those deal with this.

Congestion considerations should also be taken into account. Please
look at RFC5405 and how were dealing with the issue in GRE/UDP.

>From section 3.4:

"Since these are infrequent control messages, it is RECOMMENDED that
endpoints direct these packets to a high priority control queue (for
example, to direct the packet to a general purpose CPU from a
forwarding ASIC or to separate out control traffic on a NIC)."

I am dubious about this as a recommendation. To do this would require
a lot of complexity in a NIC and is protocol specific. Such a thing
could be done generally by implementing a high priority diff-serv
queue and marking the IP header appropriately. But more than that, a
lot of OAM is about validating a data path or measuring latency-- if
control packets explicitly take a different path then their usefulness
for such things is diminished.

>From section: 3.5.1

"Sending endpoints MUST NOT assume that options will be processed
sequentially by the receiver in the order they were transmitted." I
think this should be worded as "receivers MAY process options in any
order". But, you probably want to also allow that possibility that
processing order might be relevant in cases like when options perform
functions over payload so that there are ordering dependencies. For
example, if one TLV gives CRC over the data and another performs
compression they have to be processed in the opposite order that
sender used to create the packet. Or maybe the intent is that all the
TLVs in a list must be independent?

Are duplicate options allowed in header (two or more TLVs with same
class/type possibly different data)? If so, is there a requirement for
processing order?

"A particular option is specified to have either a fixed length, which
is constant, or a variable length, which may change over time or for
different use cases.  This property is part of the definition of the
option and conveyed by the 'Type'.": Is fixed vs. variable length
indicated by a bit in Type then?

If a receiver receives a fixed length type option but the length field
is not the corresponding value, what is the action the receiver takes?

More generally, I would infer that a critical option that is somehow
malformed (bad length or or other wise) that the packet is to be
dropped similar to dropping when a critical option is unknown. But,
what if an option is not critical but is malformed. Should the packet
be dropped, or should the option be ignored as though it is an unknown
non-critical option?

"For fixed length options, some implementations may choose to ignore
the length field in the option header and instead parse based on the
well known length associated with the type." This seems like a can of
worms to me. It is allowing the length of a TLV to be determined in
two different ways which opens the door to ambiguity when those two
methods are in disagreement. Neither am I sure what the value is in
ignoring the length field, any implementation will already have
implemented the logic to parse variable length options anyway. One
source of truth is always better!

Tom

---

From: Jesse Gross <[email-masked]>
Date: Thu, 28 Jan 2016 21:31:09 +0000

Tom,

Thanks for your comments – answers are below. By the way, I think that your last question in the previous email is related to some of the topics here so I tried to merge them together.

Jesse

On 1/20/16, 10:41 AM, "nvo3 on behalf of Tom Herbert" <[email-masked]<mailto:[email-masked]> on behalf of [email-masked]<mailto:[email-masked]>> wrote:

A few more comments:..

General:

but later

Yes, that is correct, these terms are used interchangeably. I’ve changed the cases that refer to Geneve frames to Geneve packets.

These are commonly used with Geneve so I don’t think it hurts to give concrete references. The draft notes that they are just examples.

The problem with something like IPv4 options is that they are seen as non-critical parts of the protocol and so people choose not to implement them. This is generally true of any component of a protocol that is secondary and is not limited to TLVs. Of course, once it became accepted wisdom that options can’t be used, the problem is compounded.

With Geneve, TLV options are the primary extension mechanism and that flexibility is the reason for the existence of the protocol. Options are not secondary functionality and they are already being used today. Of course Geneve is designed to scale to a wide range of use cases and implementations will have different levels of support but it’s pretty clear that options are useful and important.

I gave some additional information about implementations that I am aware of before:
https://www.ietf.org/mail-archive/web/nvo3/current/msg04970.html

From section 3.3:

I’ve been following the work that has been happening the in tsvwg. Since the considerations are similar, I’ve been waiting for some of that to come a conclusion and hopefully be factored out into a separate document that can be referenced as was discussed in the last couple of meetings.

From section 3.4:

Changing the IP header is generally more likely to impact the underlying forwarding than the ‘O' bit in the Geneve header. As a result, this strikes a balance between marking the IP header and doing nothing at all in relation to the concerns that you mention above. However, all are tools that can be applied in different ways depending on what is actually being measured.

Note that most existing NIC implementations of Geneve have support for this functionality and it does not appear that complexity is a major concern.

From section: 3.5.1

This is a good point and I agree that the current version of the draft is inconsistent.

We definitely want to be able support the situations that you describe where the ordering might be significant. On the other hand, we also want to allow parallel lookup of data from different TLVs, which was the original intention of these statements.

I think the root problem is that ‘processing’ actually consists of at least two steps – parsing and lookup – and these are not necessarily handled in the same way. With TLVs, parsing must respect ordering since the length of previous options dictates the starting point of the next. This means that the receiver is expected to handle any option where ordering has significance in order. Conversely, options which only require a lookup would not have an ordering requirement and can be handled in parallel once extracted.

I replaced the line about not assuming that options will be processed in order with one that says that options MUST NOT affect the parsing or interpretation of any other option. In addition, I changed the requirement in the section on NIC offloads to disallow reordering (I doubt that a NIC would do this anyways). Any further ordering requirements would come from the definition of the option itself.

There are no restrictions on having duplicate TLVs and some options may rely on this, such as an OAM option that is a general framework and is duplicated for each type of information to be collected. I don’t believe there are any additional requirements for ordering beyond the general ones described above.

The definition of an option has some kind of length associated with it – either a particular length or variable in different conditions. TCP, for example, has widely used options of both of these kinds. There is no need to have an explicit bit indicating one vs. the other since the endpoint should either recognize the particular option it is processing or use the length field in the TLV (which is valid in all situations).

The receiver may drop it. I added some text to make this explicit.

For critical options, I would agree that the packet should be dropped. In the case of non-critical options, I think both choices are acceptable. I also added some text for this.

The definitive length of a TLV comes from its specification, which may either be variable or fixed to a single length and anything not conforming to that would be considered an error. Depending on the implementation, assuming a canonical length can reduce either space or lookup requirements so this is one of the “other techniques to ease parsing” that you asked about earlier. It allows simplifying assumptions to be made in some cases while not changing the protocol or diminishing flexibility for others. If it becomes less useful over time then it will just fade away.