Subject: [Fwd: New Version Notification for draft-carpenter-6man-flow-update-02]

From: Brian E Carpenter <[email-masked]>
Date: Wed, 14 Apr 2010 16:48:25 +1200

Hi,

This is completely revised from the proposal we presented
in Anaheim. This version allows locally defined use of
the flow label in a simpler way, as the discussion suggested.
It's still quite a dense read, but we believe that if this was
adopted, it would open the way to actually using the flow label.

   Brian and Sheng

-------- Original Message --------
Subject: New Version Notification for draft-carpenter-6man-flow-update-02
Date: Tue, 13 Apr 2010 21:44:42 -0700 (PDT)
From: IETF I-D Submission Tool <[email-masked]>
To: [email-masked]
CC: [email-masked]

A new version of I-D, draft-carpenter-6man-flow-update-02.txt has been successfully submitted by Brian Carpenter and posted to
the IETF repository.

Filename:	 draft-carpenter-6man-flow-update
Revision:	 02
Title:		 Update to the IPv6 flow label specification
Creation_date:	 2010-04-13
WG ID:		 Independent Submission
Number_of_pages: 10

Abstract:
Various uses proposed for the IPv6 flow label are incompatible with
its existing specification.  This document describes changes to the
specification that permit additional use cases as well as allowing
continued use of the previous specification.

The IETF Secretariat.

---

From: Rémi Després <[email-masked]>
Date: Wed, 14 Apr 2010 18:06:14 +0200

Brian, Sheng,

I carefully read the draft, and read again RFC 3697 and draft-carpenter-6man-flow-update-02.
The draft is IMHO overly hard to understand but, unless I misunderstand its intent (which is quite possible), I appreciate what it proposes.
I do support the intent, but feel uncomfortable with how it is introduced.

In my understanding:
A) The essential changes are:
 A1. Flow Labels MAY be changed within the network, instead of "MUST be delivered unchanged".
 A2. Nodes that set Flow Labels MAY apply local policies, including stateless ones, instead of "SHOULD select new Flow Label values in a well-defined sequence (e.g., sequential or pseudo-random".
B) The essentials of what is kept are:
 B1. Where Flow Labels are set, their values MUST be the same for packets that have the same 5-tuples (or 3-tuples), and are separated by less than 120s. 
 B3. For flow-based routing optimization to be possible, Flow Labels should in general be given different values for different 5-tuples (or 3-tuples) that share the same source-address/destination-address couples. 

If this makes sense, I am convinced that it would be better to propose a clear and simple new RFC, and obsolete RFC 3697.
Trying to complement RFC 3697 with what amounts to a significant change is in my understanding bond to be awkward.

Thoughts?

Kind regards,
RD

Le 14 avr. 2010 à 06:48, Brian E Carpenter a écrit :

> --------------------------------------------------------------------
> IETF IPv6 working group mailing list
> [email-masked]
> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> --------------------------------------------------------------------

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 15 Apr 2010 09:58:53 +1200

Rémi,

Thanks for the analysis. Indeed, the conclusion may be, assuming
the WG is interested at all, that a complete refresh of RFC 3697
is better than publishing a delta.

I agree that it is a bit complicated to explain, although the
underlying concept is quite clear: allow local use of the
flow label, but also enable global use. I also think we were
too careful in RFC 3697 - if we had unambiguously recommended
a pseudo-random label per flow, it might already be used for
load balancing. (By the way, there will soon be an update of
draft-carpenter-flow-ecmp, adding the LAG case and a second
author, which I think will make the load balancing use case
very clear.)

Thanks
   Brian

---

From: Mark Smith <[email-masked]>
Date: Thu, 15 Apr 2010 07:29:36 +0930

Hi Brian and Sheng,

On Wed, 14 Apr 2010 16:48:25 +1200

> adopted, it would open the way to actually using the flow label.

I've had a read through it, although not a comprehensive one, which
I'll endeavour to do in the next day or so.

I'm wondering about this change for a couple of reasons:

--
   2.  If this is done, all packets in a given flow MUST be given the
       same flow label value.  A flow is defined in this case as all
       packets with the same source and destination IPv6 addresses and
       port numbers and the same transport protocol number, i.e., the
       same final Next Header value [RFC2460].  This rule constrains the
       definition of a flow in RFC 3697 for the specific case that a
       router sets the flow label.  However, it does not constrain the
       bits of the flow label in any particular way.
--

Firstly, if the IPv6 packets are fragments, the transport layer header
may not be available. I think that would mean that although these
packets fragments are part of a flow, they wouldn't have their flow
label changed.

Secondly, for IPv6 packets with a number of extension headers before
the transport layer header, I think this rule could impact forwarding
performance. While I'm not sure if it is that practical, however it'd
be good if flow classification could be done using only fixed headers
in the IPv6 packet, or a fixed portion of the fixed header bits.

I suppose partly that comes down to what a 'flow' is. In some contexts,
it is definitely a transport layer connection. In others, e.g. an MPLS
network, I think it could be seen to be all packets that match a
Forwarding Equivalence Class. If it was possible to use a FEC to set
the flow label, once the packet has traversed the MPLS network, and the
MPLS labels are stripped off, the flow label that was set due to the
FEC would be preserved, which might be useful. Is there an opportunity
to make the definition of a flow a bit more general, and then for allow
for the choice of different packet classification methods to be used to
define a flow, based on context e.g. transport layer connection in some
contexts, MPLS FEC, QoS/Diff Serv classifiers etc. in others?

Regards,
Mark.

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 15 Apr 2010 10:13:04 +1200

Hi Mark,

> I'm wondering about this change for a couple of reasons:

Yes, certainly, if the router inserting the flow label is stateless.
I honestly don't see a way round that and it's probably better to
list it as a limitation. It probably doesn't matter - anyone who is
deploying a local scheme for the flow label is presumably able to
also deploy a non-fragmenting network.

> in the IPv6 packet, or a fixed portion of the fixed header bits.

I will send a separate message on that point.

> contexts, MPLS FEC, QoS/Diff Serv classifiers etc. in others?

And that's an even wider question. I'm inclined to duck it, or at
least to assert that it's a much wider question than 6man can tackle.

   Brian

---

From: Shane Amante <[email-masked]>
Date: Wed, 14 Apr 2010 20:10:42 -0600

Brian, Mark,

Brian: FWIW, I like the direction of this version of draft much better than previous versions; however, I agree with Remi that the current version is a bit confusing at the moment and could likely be rewritten to be more simple and just obsolete RFC 3967.  In addition, I'm still a bit unclear and, therefore, uncomfortable on the proposal with respect to flow-labels within an "administratively defined domain".  In particular, if one administrative domain has set their own "locally defined" flow-label, I think the draft could benefit from a stronger emphasis that the flow-label MUST or, at least, SHOULD be reset to 0 upon /leaving/ that domain, otherwise the next domain may (or, will?) misinterpret the meaning of the incoming "locally-defined" flow-label.  The way I interpret the text in Bullet 3 of Section 3 seems written in a way that is specific to packets entering domains that are going to use locally-defined flow-labels, but not how they must treat those same packets as they leave their domain and, most importantly, need to interact with other domains that don't understand or want to use a "standard" definition of a flow-label, i.e.: in Bullets 1, 2 & 4.  Or, perhaps I'm misunderstanding something.

More below.

> also deploy a non-fragmenting network.

Actually, one would hope that implementations are actually "smart" about fragments.  Specifically, one could look for a Fragment Header and, upon noticing it, mask out or exclude Layer-4 transport information, (e.g.: protocol, source and destination port) as input keys for, say, LAG or ECMP.  This means one would only consider the 2-tuple of source and destination IP address, which is better than nothing, but more importantly would still preserve ordering.

> I will send a separate message on that point.

I wholeheartedly agree with Mark's point.  I would add that IPv6 Extension Headers may not impact forwarding performance on PE interfaces, because they are [somewhat] high-touch interfaces that typically must be capable of imposing various 2- or 5-, or 6-tuple[1] classifiers on PE-CE interfaces for applying ACL's (forward/drop), policing/rate-limiting or marking (IPv6 TC field) functions.  *However*, the same is not true of P routers, which generally have several dozen bleeding edge interfaces of 40G and 100G and beyond.  Thus, allowing P routers to more quickly access input keys to a LAG or ECMP hash function is pretty critical.

[1] 6-tuple if you include Traffic Class.

> least to assert that it's a much wider question than 6man can tackle.

Let's not duck ... not yet, anyway.  :-)

With respect to MPLS, there is the following, now expired, draft which might be what you're looking for:
http://tools.ietf.org/html/draft-kompella-mpls-entropy-label-00
... however, there are some challenges with it, namely handling Penultimate Hop Popping (PHP), which isn't discussed in the draft, plus the fact that legacy HW certainly most likely will not support pushing additional labels required to insert an entropy-label.  If you're interested in this draft, I suggest we take if offline and/or to the MPLS list.

That said, I would hope the use of an MPLS entropy label might be avoided for IP over MPLS traffic and, instead, a simpler approach would be to use (even in MPLS networks) just the following as input-keys to LAG or ECMP:
- IPv6 source address;
- IPv6 destination address; and,
- A _proper_ IPv6 flow-label that unambiguously identifies an underlying "flow"
... without needing to seek out Transport layer info.

I believe it would be an implementation detail as to what IPv6 header fields a vendor would allow to be included as input-keys to a hash that would be calculated and put into a IPv6 flow-label.  Although, certainly I would hope we could insist on a minimum of the 2-tuple and "recommend" that they include the more valuable the 5- or 6-tuple as input-keys to the hash algorithm for the flow-label.

-shane

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 15 Apr 2010 15:52:15 +1200

Regards
   Brian Carpenter

On 2010-04-15 14:10, Shane Amante wrote:
> Brian, Mark,
> 
> Brian: FWIW, I like the direction of this version of draft
> much better than previous versions; however, I agree with
> Remi that the current version is a bit confusing at the
> moment and could likely be rewritten to be more simple and
> just obsolete RFC 3967.  In addition, I'm still a bit unclear
> and, therefore, uncomfortable on the proposal with respect to
> flow-labels within an "administratively defined domain".  In
> particular, if one administrative domain has set their own
> "locally defined" flow-label, I think the draft could benefit
> from a stronger emphasis that the flow-label MUST or, at
> least, SHOULD be reset to 0 upon /leaving/ that domain,
> otherwise the next domain may (or, will?) misinterpret the
> meaning of the incoming "locally-defined" flow-label.  The

I'm personally quite attracted by this. It does further damage
to the sacred principle that the flow label is immutable,
but maybe that is the inevitable result anyway.

    Brian

> way I interpret the text in Bullet 3 of Section 3 seems
> written in a way that is specific to packets entering domains
> that are going to use locally-defined flow-labels, but not
> how they must treat those same packets as they leave their
> domain and, most importantly, need to interact with other
> domains that don't understand or want to use a "standard"
> definition of a flow-label, i.e.: in Bullets 1, 2 & 4.  Or,
> perhaps I'm misunderstanding something.
> 
> More below.
> 
>>> On Wed, 14 Apr 2010 16:48:25 +1200 Brian E Carpenter
>>> <[email-masked]> wrote:
>>> 
>>>> Hi,
>>>> 
>>>> This is completely revised from the proposal we
>>>> presented in Anaheim. This version allows locally
>>>> defined use of the flow label in a simpler way, as the
>>>> discussion suggested. It's still quite a dense read,
>>>> but we believe that if this was adopted, it would open
>>>> the way to actually using the flow label.
>>>> 
>>> I've had a read through it, although not a comprehensive
>>> one, which I'll endeavour to do in the next day or so.
>>> 
>>> -- 2.  If this is done, all packets in a given flow MUST
>>> be given the same flow label value.  A flow is defined in
>>> this case as all packets with the same source and
>>> destination IPv6 addresses and port numbers and the same
>>> transport protocol number, i.e., the same final Next
>>> Header value [RFC2460].  This rule constrains the 
>>> definition of a flow in RFC 3697 for the specific case
>>> that a router sets the flow label.  However, it does not
>>> constrain the bits of the flow label in any particular
>>> way. --
>>> 
>>> Firstly, if the IPv6 packets are fragments, the transport
>>> layer header may not be available. I think that would
>>> mean that although these packets fragments are part of a
>>> flow, they wouldn't have their flow label changed.
>> Yes, certainly, if the router inserting the flow label is
>> stateless. I honestly don't see a way round that and it's
>> probably better to list it as a limitation. It probably
>> doesn't matter - anyone who is deploying a local scheme for
>> the flow label is presumably able to also deploy a
>> non-fragmenting network.
> 
> Actually, one would hope that implementations are actually
> "smart" about fragments.  Specifically, one could look for a
> Fragment Header and, upon noticing it, mask out or exclude
> Layer-4 transport information, (e.g.: protocol, source and
> destination port) as input keys for, say, LAG or ECMP.  This
> means one would only consider the 2-tuple of source and
> destination IP address, which is better than nothing, but
> more importantly would still preserve ordering.
> 
> 
>>> Secondly, for IPv6 packets with a number of extension
>>> headers before the transport layer header, I think this
>>> rule could impact forwarding performance. While I'm not
>>> sure if it is that practical, however it'd be good if
>>> flow classification could be done using only fixed
>>> headers in the IPv6 packet, or a fixed portion of the
>>> fixed header bits.
> I wholeheartedly agree with Mark's point.  I would add that
> IPv6 Extension Headers may not impact forwarding performance
> on PE interfaces, because they are [somewhat] high-touch
> interfaces that typically must be capable of imposing various
> 2- or 5-, or 6-tuple[1] classifiers on PE-CE interfaces for
> applying ACL's (forward/drop), policing/rate-limiting or
> marking (IPv6 TC field) functions.  *However*, the same is
> not true of P routers, which generally have several dozen
> bleeding edge interfaces of 40G and 100G and beyond.  Thus,
> allowing P routers to more quickly access input keys to a LAG
> or ECMP hash function is pretty critical.
> 
>>> I suppose partly that comes down to what a 'flow' is. In
>>> some contexts, it is definitely a transport layer
>>> connection. In others, e.g. an MPLS network, I think it
>>> could be seen to be all packets that match a Forwarding
>>> Equivalence Class. If it was possible to use a FEC to set
>>>  the flow label, once the packet has traversed the MPLS
>>> network, and the MPLS labels are stripped off, the flow
>>> label that was set due to the FEC would be preserved,
>>> which might be useful. Is there an opportunity to make
>>> the definition of a flow a bit more general, and then for
>>> allow for the choice of different packet classification
>>> methods to be used to define a flow, based on context
>>> e.g. transport layer connection in some contexts, MPLS
>>> FEC, QoS/Diff Serv classifiers etc. in others?
>> And that's an even wider question. I'm inclined to duck it,
>> or at least to assert that it's a much wider question than
>> 6man can tackle.
> 
> With respect to MPLS, there is the following, now expired,
> draft which might be what you're looking for: 
>  ... however, there are some challenges with it, namely
> handling Penultimate Hop Popping (PHP), which isn't discussed
> in the draft, plus the fact that legacy HW certainly most
> likely will not support pushing additional labels required to
> insert an entropy-label.  If you're interested in this draft,
> I suggest we take if offline and/or to the MPLS list.
> 
> That said, I would hope the use of an MPLS entropy label
> might be avoided for IP over MPLS traffic and, instead, a
> simpler approach would be to use (even in MPLS networks) just
> the following as input-keys to LAG or ECMP: - IPv6 source
> address; - IPv6 destination address; and, - A _proper_ IPv6
> flow-label that unambiguously identifies an underlying "flow"
> I believe it would be an implementation detail as to what
> IPv6 header fields a vendor would allow to be included as
> input-keys to a hash that would be calculated and put into a
> IPv6 flow-label.  Although, certainly I would hope we could
> insist on a minimum of the 2-tuple and "recommend" that they
> include the more valuable the 5- or 6-tuple as input-keys to
> the hash algorithm for the flow-label.
> 
> -shane
> 
> 
> 
> 
> 
>> Brian
>>> Regards, Mark.
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>>> Brian and Sheng
>>>> 
>>>> -------- Original Message -------- Subject: New Version
>>>> Notification for draft-carpenter-6man-flow-update-02 
>>>> Date: Tue, 13 Apr 2010 21:44:42 -0700 (PDT) From: IETF
>>>> I-D Submission Tool <[email-masked]> To:
>>>> [email-masked] CC: [email-masked]
>>>> 
>>>> 
>>>> A new version of I-D,
>>>> draft-carpenter-6man-flow-update-02.txt has been
>>>> successfully submitted by Brian Carpenter and posted to
>>>>  the IETF repository.
>>>> 
>>>> Filename:	 draft-carpenter-6man-flow-update Revision:
>>>> 02 Title:		 Update to the IPv6 flow label specification
>>>>  Creation_date:	 2010-04-13 WG ID:		 Independent
>>>> Submission Number_of_pages: 10
>>>> 
>>>> Abstract: Various uses proposed for the IPv6 flow label
>>>> are incompatible with its existing specification.  This
>>>> document describes changes to the specification that
>>>> permit additional use cases as well as allowing 
>>>>  IETF IPv6 working group mailing list [email-masked] 
>>>> Administrative Requests:
>>>> https://www.ietf.org/mailman/listinfo/ipv6 
>>  IETF IPv6 working group mailing list [email-masked] 
>> Administrative Requests:
>> https://www.ietf.org/mailman/listinfo/ipv6

---

From: Sheng Jiang <[email-masked]>
Date: Thu, 15 Apr 2010 14:35:42 +0800

Hi, Remi,

Thanks for your summury. They are is quite right.

I agree a new and simple document that obsolete RFC 3697 could be bette
approach.

Now, the urgency is to make sure the WG agrees on these proposed changes.
After than, how we move forward could also flow the WG's consensus.

Cheers,

Sheng

> -----Original Message-----
> From: Brian E Carpenter [mailto:[email-masked]] 
> Sent: Thursday, April 15, 2010 5:59 AM
> To: Rémi Després
> Cc: Sheng Jiang; 6man
> Subject: Re: [Fwd: New Version Notification for 
> draft-carpenter-6man-flow-update-02]
> 
> Rémi,
> 
> Thanks for the analysis. Indeed, the conclusion may be, 
> assuming the WG is interested at all, that a complete refresh 
> of RFC 3697 is better than publishing a delta.
> 
> > The draft is IMHO overly hard to understand but, unless I 
> misunderstand its intent (which is quite possible), I 
> appreciate what it proposes.
> >  A2. Nodes that set Flow Labels MAY apply local policies, 
> including stateless ones, instead of "SHOULD select new Flow 
> Label values in a well-defined sequence (e.g., sequential or 
> pseudo-random".
> >  B1. Where Flow Labels are set, their values MUST be the 
> same for packets that have the same 5-tuples (or 3-tuples), 
> and are separated by less than 120s. 
> >  B3. For flow-based routing optimization to be possible, 
> Flow Labels should in general be given different values for 
> different 5-tuples (or 3-tuples) that share the same 
> source-address/destination-address couples. 
> > 
> > If this makes sense, I am convinced that it would be better 
> to propose a clear and simple new RFC, and obsolete RFC 3697.
> > Trying to complement RFC 3697 with what amounts to a 
> significant change is in my understanding bond to be awkward.
> > 
> > Thoughts?
> > 
> > Kind regards,
> > RD
> >    
> > 
> > 
> in Anaheim. 
> >> This version allows locally defined use of the flow label in a 
> >> simpler way, as the discussion suggested.
> >> A new version of I-D, draft-carpenter-6man-flow-update-02.txt has 
> >> been successfully submitted by Brian Carpenter and posted 
> to the IETF repository.
> >>

---

From: Sheng Jiang <[email-masked]>
Date: Thu, 15 Apr 2010 15:06:45 +0800

> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of Shane Amante
> Sent: Thursday, April 15, 2010 10:11 AM
> To: Brian E Carpenter
> Cc: 6man
> Subject: Re: [Fwd: New Version Notification 
> fordraft-carpenter-6man-flow-update-02]
> 
> Brian, Mark,
> 
> how they must treat those same packets as th  ey leave their 

Hi, Shane,

I am fully with you.

Actually, during our updating for 02 version, I talked with Brian, "whether
we need a outbound router to set the router-modified flow label back to
all-zero". By that time, we were considering the backwards compatible with
RFC3976  "delivered the flow label unchanged". Therefore, we cannot change
flow label on the outbound router because we cannot tell whether it is from
hosts or routers.

Now, we are talking about to obsolete 3967 totally, then reset flow label
upon/leaving Flow Label domain would be quite reasonable.

Cheers,

Sheng
 
> >>> This is completely revised from the proposal we presented in 
> >>> Anaheim. This version allows locally defined use of the 
> flow label 
> >>> in a simpler way, as the discussion suggested.
> one, which 
> >> Firstly, if the IPv6 packets are fragments, the transport layer 
> >> header may not be available. I think that would mean that although 
> >> these packets fragments are part of a flow, they wouldn't 
> have their 
> >> flow label changed.
> > 
> > I honestly don't see a way round that and it's probably 
> better to list 
> > it as a limitation. It probably doesn't matter - anyone who is 
> > deploying a local scheme for the flow label is presumably 
> able to also 
> > deploy a non-fragmenting network.
> 
> headers before 
> >> I suppose partly that comes down to what a 'flow' is. In some 
> >> contexts, it is definitely a transport layer connection. 
> In others, 
> >> e.g. an MPLS network, I think it could be seen to be all 
> packets that 
> >> match a Forwarding Equivalence Class. If it was possible 
> to use a FEC 
> >> to set the flow label, once the packet has traversed the MPLS 
> >> network, and the MPLS labels are stripped off, the flow label that 
> >> was set due to the FEC would be preserved, which might be 
> useful. Is 
> >> there an opportunity to make the definition of a flow a bit more 
> >> general, and then for allow for the choice of different packet 
> >> classification methods to be used to define a flow, based 
> on context 
> the following as input-keys to LAG or ECMP:
> - IPv6 source address;
> >>> IETF IPv6 working group mailing list [email-masked] Administrative 
> >>> Requests: https://www.ietf.org/mailman/listinfo/ipv6
> >>>

---

From: Rémi Després <[email-masked]>
Date: Thu, 15 Apr 2010 09:35:30 +0200

> After than, how we move forward could also flow the WG's consensus.

The (rare) expressed opinions are so far favorable to the change but, yes, more time to collect more opinions is appropriate.

Cheers,
RD

---

From: Mark Smith <[email-masked]>
Date: Thu, 15 Apr 2010 19:16:48 +0930

Hi Brian,

On Thu, 15 Apr 2010 10:13:04 +1200

> > Brian E Carpenter <[email-masked]> wrote:

<snip>

> least to assert that it's a much wider question than 6man can tackle.

Are you generally trying to avoid it because it would appear to be an
implied IETF definition of a flow, rather than just the IPv6
definition? Or is it to avoid having to start defining the a possibly
large set of flow definitions?

I think that as the field is an IPv6 one, IPv6 can have a position on
what it considers a flow to be.

I had a bit of a think about a more general definition of what a flow
is and came up with this:

"a set of packets that have a common attribute, or common relationship
with another entity"

with common attribute being things like field (single or multiple
) values, optional fields (e.g. a set of extension headers), same
result of a hash of fields etc.

A common relationship might be the same MPLS FEC, BGP NEXT_HOP, router
incoming interface, destination AS etc.

Looking at the definitions of flows in the IPv6 RFC, and in RFC3697, I
think the above general definition would encompass those as well.

If that was an acceptable general definition, then this draft could
also then define one specific instance of a flow definition, namely the
fields used to identify a transport layer connection, and state that
other instances of applicable flow definitions may be defined in other
RFCs, including non-IPv6 ones.

Regards,
Mark.

---

From: Mark Smith <[email-masked]>
Date: Thu, 15 Apr 2010 20:06:43 +0930

Hi Brian, Shane,

On Thu, 15 Apr 2010 15:52:15 +1200

> but maybe that is the inevitable result anyway.

I think the way the dscp is handled provides a good example. If your
network is QoS enabled, then you reset the dscp value upon ingress
because you can't trust it, otherwise you completely ignore the value of
it as it traverses your network. I think the same model can be applied
to flow labels.

For an ISP, if it considered it's network to be a separate flow domain
from that of it's customers' networks (and my position is that I
would, for both business and residential customers), then every egress
interface towards a customer would have to be resetting the flow label.
ISPs with many PPPoE/PPP customers have as many virtual egress
interfaces as they do customers. I think a lot of those customers
wouldn't implement a flow domain, so having every virtual interface
reset the flow label back to 0 on packet egress would be a relatively
high price for the value it provides to only a small number of
customers.

Regards,
Mark.

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 15 Apr 2010 12:37:02 +0200

On Apr 15, 2010, at 05:52, Brian E Carpenter wrote:

>> the flow-label MUST or, at

It may be useful to discuss this in the context of a more general question:

How does a domain stash temporary information (meaningful only in that domain) into an IPv6 packet, with the intention of removing it again at domain egress?

See also:

  "RPL Option for Carrying RPL Information in Data-Plane Datagrams",
  Jonathan Hui, JP Vasseur, 10-Mar-10, <draft-hui-6man-rpl-option-00.txt>

    The RPL protocol requires data-plane datagrams to carry RPL routing
    information that is processed by RPL routers when forwarding those
    datagrams.  This document describes the RPL option for use within a
    RPL domain.

This work was originally trying to use the flow label in a similar way, and has moved to proposing to insert a hop-hy-hop option.  Note that hop-by-hop options *inserted* on the way destroy AH, so in this case they *must* be removed at egress (including at final delivery; and the fun part about "removing" is then how to reconstitute any padding in the original hbh option, if there was one).

I'm not saying the flow label does not pose an attractive nuisance for addressing some of these problems.
But if it is already taken in a packet (by the sending host or by an encapsulating domain), how does the domain derive the benefit?
We can either stipulate that everyone (end-to-end, encapsulating domain) please use it in a similar way (i.e., 5-tuple), or we can provide for a way to stash it on ingress and reconstruct it on egress (the only way that would be useful for RPL, as its use would not be about the 5-tuple).  None of this makes me particularly happy right now.

(I'm not even saying the flow label work *must* consider this problem.  It just appears it's getting awfully close to it.)

Gruesse, Carsten

---

From: Shane Amante <[email-masked]>
Date: Thu, 15 Apr 2010 08:34:40 -0600

Hi Mark,

> it as it traverses your network. I think the same model can be applied

I believe we're in agreement on the above point.

> high price for the value it provides to only a small number of

I acknowledge the operational burden of resetting the flow-label on egress; however, the same challenge exists on ingress, as well.  IOW, if you don't trust the flow-label setting coming from another "administratively defined domain" (ADD), then you'll always need to have to reset (and/or set) it on ingress before it transits your ADD ... wh/ would translate into roughly the same problem.

Regardless, the way the draft is written now it still seems a bit ambiguous (at best) as to whether the next, downstream "administratively defined domain" (ADD) is supposed to accept "as-is" the incoming, perhaps locally-defined in another ADD, flow-label or not.  Most importantly consideration should be given to what the operational implications will be if that were the case.  IOW, it could result in very poor load-balancing if the incoming locally-defined flow-label does not correspond to, say, a 5-tuple "microflow" and, instead, means something else entirely (e.g.: a "nonce" as per one of the many proposals to recast/re-use/abuse the flow-label).

The reason I offered the proposed text was to make it abundantly clear that there should be no assumptions made about the flow-label once it leaves one "administratively defined domain", since one ADD can't tell if it's locally-defined or not.  In summary, I think the text could stand to be made more clear that a flow-label needs to be reset /either/ on egress from or ingress to an ADD[1].  I think, in practice, you're likely correct that most operators would probably (re)set it only on ingress (similar to IP DSCP or IPv6 TC), since that's a common operation.

Lastly, FWIW, I'd personally rather not carve out a bit in the flow-label (as was discussed in previous revisions of this draft), for "locally-defined" flow-labels since the same operational problem exists.  Only in that case, as a packet enters my ADD I won't be able to do anything with it.  This means I need to have even more complex logic in routers to ignore those flow-labels with that bit set and then attempt to look for Layer-4 Transport header info at every hop for load-balancing calculations.

-shane

[1] I should note that if one or more "consenting" ADD's want to exchange a locally defined flow-label between them, then that's their right, i.e.: your network(s), your rules.  However, I would typically expect this to be an exception, not the rule, given the implications a misunderstanding or misinterpretation a "locally-defined" flow-label might have.

---

From: Rémi Després <[email-masked]>
Date: Thu, 15 Apr 2010 17:38:15 +0200

Le 14 avr. 2010 à 23:59, Mark Smith a écrit :

> may not be available.

> I think that would mean that although these

Strictly speaking, it doesn't prevent routers from changing flow labels, but it does impose that new values depend only on 3-tuples (source address, destination address, last Next Header).

In practice this limits significantly benefits to be expected from allowing routers to modify flow labels.  

A very good point indeed. 

More comments to come in the next answer to Brian.

RD

---

From: Mark Smith <[email-masked]>
Date: Fri, 16 Apr 2010 06:08:57 +0930

On Thu, 15 Apr 2010 08:34:40 -0600

> I acknowledge the operational burden of resetting the flow-label on egress; however, the same challenge exists on ingress, as well.  IOW, if you don't trust the flow-label setting coming from anoth…

You're right, I've been influenced by my past experience. In that
(IPv4) scenario, dscp was set when traffic entered via the ISP transit
links and that was for billing purposes. Traffic coming from customers
had the dscp values left alone.

As long resetting it on egress was an option operation, I think it
would facilitate both points of view.

(That dscp for billing idea was implemented before I started at that
organisation. I didn't mind the idea of labelling packets to categorise
them, however I didn't like using the dscp, as that has a specific QoS
meaning. The flow label field seems to me to be a bit more general
purpose, so I'd think the flow label would be the right field to use
in that scenario)

Regards,
Mark.

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 16 Apr 2010 11:50:28 +1200

Hi Mark,

> large set of flow definitions?

Actually, because I have enough email to deal with right now anyway :-)

Your proposed definition looks reasonable, but the IPFIX people have
a rather different definition:

"A flow is defined as a set of IP packets passing an observation point
 in the network during a certain time interval.  All packets belonging
 to a particular flow have a set of common properties....
 A flow is considered to be expired if no packet of this flow has been
 observed for a given timeout interval" [RFC3917]

The crucial difference is the tiemout. In the observational world
of IPFIX, a flow is deemed to end when there is a certain period of
silence. So a stop-start flow meeting your definition might be
broken up into multiple flows by IPFIX. That's where things get vague.

Regards
   Brian

---

From: Mark Smith <[email-masked]>
Date: Mon, 19 Apr 2010 07:11:25 +0930

Hi Brian,

On Fri, 16 Apr 2010 11:50:28 +1200

> broken up into multiple flows by IPFIX. That's where things get vague.

My understanding is that IPFIX is generally an IETF standardised
version of Cisco's Netflow. In Netflow, timeouts are needed for a
few reasons:

o A timeout is used to expire flows for protocols that aren't
connection oriented  (e.g. UDP) or the implementation doesn't know
are connection oriented because the implementation doesn't understand
the protocol. When the timeout occurs, a netflow record is emitted
towards the netflow collector, and the router clears the netflow cache
entry for the flow.

o Long lived TCP connections may go for long enough that people want
to know partial flow information about them. This timeout is an upper
bound on how long a flow can continue before a netflow record is
emitted. The TCP flow stays active in the flow cache, so there will be
at least another netflow record at the end of the TCP connection or if
the flow cache becomes full and the flow is punted from it.

So those timeouts are more to do with emission of flow records, and
flow cache maintenance, rather than being a more general definition of
what a flow is.

Regards,
Mark.

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 21 Apr 2010 16:33:03 +1200

Mark,

I was hoping someone else might chime in, but since they didn't, let
me extract from your messages:

> So those [IPFIX] timeouts are more to do with emission of flow records, and
> what a flow is

Certainly. But supposing someone was led to build a stateful flow marker.
(That's on the assumption that we change the rules to allow routers to
overwrite the flow label.) A stateful marker would have exactly the same
problem with UDP flows as an IPFIX meter. By contrast, a source host
never has this problem, because it knows by construction when a flow
ends (chances are, flow == socket).

    Brian

---

From: Philip Homburg <[email-masked]>
Date: Wed, 21 Apr 2010 09:11:38 +0200

In your letter dated Wed, 21 Apr 2010 16:33:03 +1200 you wrote:
>By contrast, a source host

I'm wondering about the 'never' part. What if an application uses sendto()
to communicate over UDP with multiple peers, and some of those peers happen
to be located on a single remote host. 

How is the kernel supposed to keep track of those flows? Or does the
application have to maintain flow-ids?

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 22 Apr 2010 09:12:45 +1200

> application have to maintain flow-ids?

The latter, I think, in such a complex case. RFC 3697 says:

  "To enable applications and transport protocols to define what packets
   constitute a flow, the source node MUST provide means for the
   applications and transport protocols to specify the Flow Label values
   to be used with their flows."

That was because we intentionally did not define exactly what a flow is;
the IPv6 WG didn't want to do so at that time.

    Brian