Subject: [Fwd: I-D Action:draft-carpenter-6man-flow-update-00.txt]

From: Brian E Carpenter <[email-masked]>
Date: Thu, 18 Feb 2010 16:34:05 +1300

Hi,

This may seem a bit unexpected, but after working on draft-carpenter-flow-ecmp
(just updated) and working with my student Qinwen Hu on some aspects
of the flow label, it seemed like time for another look at the flow
label standard, and Sheng Jiang was having similar thoughts.

We'd like to discuss this in Anaheim if possible.

    Brian

-------- Original Message --------
Subject: I-D Action:draft-carpenter-6man-flow-update-00.txt
Date: Wed, 17 Feb 2010 18:15:02 -0800 (PST)
From: [email-masked]
Reply-To: [email-masked]
To: [email-masked]

A New Internet-Draft is available from the on-line Internet-Drafts directories.

	Title           : Update to the IPv6 flow label specification
	Author(s)       : B. Carpenter, S. Jiang
	Filename        : draft-carpenter-6man-flow-update-00.txt
	Pages           : 9
	Date            : 2010-02-17

Various uses proposed for the IPv6 flow label are incompatible with
its existing specification.  This document describes changes to the
specification that permit additional use cases as well as allowing
continued use of the previous specification.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-carpenter-6man-flow-update-00.txt

---

From: Vishwas Manral <[email-masked]>
Date: Fri, 19 Feb 2010 21:03:32 -0800

Hi Brian,

There was a discussion we had regarding the Flow label about 4 years back.
It may be of use for your draft.

Here is the link:
http://www.ops.ietf.org/lists/v6ops/v6ops.2006/threads.html#00010

Thanks,
Vishwas

On Wed, Feb 17, 2010 at 7:34 PM, Brian E Carpenter <
[email-masked]> wrote:

> --------------------------------------------------------------------
> IETF IPv6 working group mailing list
> [email-masked]
> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> --------------------------------------------------------------------
>

---

From: Mark Smith <[email-masked]>
Date: Sat, 20 Feb 2010 23:28:50 +1030

Hi Brian,

On Thu, 18 Feb 2010 16:34:05 +1300

> We'd like to discuss this in Anaheim if possible.

I've had a read through this draft and support what it is
proposing.

In regarding the following processing rules:

   o  Considering packets outbound from the Flow Label Domain, if MSB =
      0, a boundary router MUST NOT change the flow label.  If MSB = 1,
      it MUST set all 20 bits of the flow label to zero, so that the
      locally defined behaviour is not exported from the domain.
   o  Considering packets inbound to the Flow Label Domain, if MSB = 0,
      a boundary router MUST NOT change the flow label.  If an inbound
      packet has MSB = 1, it has originated from a source not following
      the current specification.  This is considered to be an extremely
      unlikely case, and the boundary router MUST set all 20 bits of the
      flow label to zero, as the choice least likely to cause unwanted
      behaviour.  (Note that this means the rules for inbound and
      outbound packets at the boundary router are identical.)

one thought I had would be that there could be a use case where e.g.
when a packet leaves a flow domain and has it's MSB=1, and enters
another flow domain e.g. crossing the boundary between enterprise
network and an ISP, the flow label could be changed to a MSB=1 value
of local significance to the second flow domain. This would be somewhat
similar to how ISPs can provide specific BGP community values for it's
customers to set to influence routing within the ISP's AS. If there is
no specified MSB=1 value for the second flow domain, then the flow label
must be set to all zeros.

Regards,
Mark.

---

From: Brian E Carpenter <[email-masked]>
Date: Sun, 21 Feb 2010 08:30:14 +1300

> must be set to all zeros.

Yes, that's worth thinking about. It would be unfortunate if the second
domain is forbidden from using a local scheme if the first domain
already did so.

   Brian

---

From: Sheng Jiang <[email-masked]>
Date: Mon, 22 Feb 2010 11:15:52 +0800

> > This may seem a bit unexpected, but after working on 
> > draft-carpenter-flow-ecmp (just updated) and working with 
> my student 
> > Qinwen Hu on some aspects of the flow label, it seemed like 
> time for 
> > another look at the flow label standard, and Sheng Jiang 
> was having similar thoughts.
> > 
> I've had a read through this draft and support what it is proposing.
> 
> when a packet leaves a flow domain and has it's MSB=1, and 
> enters another flow domain e.g. crossing the boundary between 
> enterprise network and an ISP, the flow label could be 
> changed to a MSB=1 value of local significance to the second 
> flow domain. This would be somewhat similar to how ISPs can 
> provide specific BGP community values for it's customers to 
> set to influence routing within the ISP's AS. If there is no 
> specified MSB=1 value for the second flow domain, then the 
> flow label must be set to all zeros.

Yes, the second flow domain should be also able to use the local defined
behaviors. In my opinion, when the packet leaves a flow domain, the egress
router does not sure whether there will be another flow domain or not (the
second flow domain may be appear to be several hops away). So, it should
leave the MSB=1 to keep the possibility. The rest 19 bits may be reset to
all 0 depends or untouched depending on whether the egress router wants to
prevent the exporting of local defined behaviours. Actually, I don't think
it is necessary to do so. The leaking of locally defined behaviours is not
harmful.

Cheers,

Sheng

---

From: Mark Smith <[email-masked]>
Date: Mon, 22 Feb 2010 19:52:00 +1030

Hi Sheng,

On Mon, 22 Feb 2010 11:15:52 +0800
Sheng Jiang <[email-masked]> wrote:

> second flow domain may be appear to be several hops away).

I was thinking more of the situation where the egress router does know
that the adjacent domain is a flow domain, and therefore is configured
to change the flow value to another MSB=1 value or leave it alone if
the original MSB=1 value matches the configured outgoing MSB=1 value.
If there is no new configured outbound MSB=1 value, then I think it is
better to set the flow value to MSB=0/all bits zero, indicating that
the packet has left the prior flow domain, and that the egress router
does not have any knowledge of any subsequent flow domains or their
flow domain values.

> So, it should

I think leaking them could be, if there are equivalent flow values in
the subsequent flow domain which have different meanings.

Regards,
Mark.

---

From: Rémi Després <[email-masked]>
Date: Mon, 22 Feb 2010 13:03:54 +0100

> continued use of the previous specification.

+1 for this proposal.

In fact I made a very similar proposal in May 2008(ref www.ietf.org/mail-archive/web/behave/current/msg03773.html).

Now, if some new specific behavior is retained for flow-label values that don't have to be preserved e2e, like that proposed by Mark Smith , I would suggest that, to leave freedom for the future, this would again be limited to a subset of flow label values (e.g. first two bits = 11).

Regards,
RD

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 23 Feb 2010 09:41:24 +1300

> it is necessary to do so. The leaking of locally defined behaviours is not

After some thought, I believe that is correct. (I don't know if we'll have
time to update the draft before the cutoff, since I have some travel coming
up.)

    Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 23 Feb 2010 09:46:50 +1300

> the subsequent flow domain which have different meanings.

Yes, so you send them on with MSB = 1 (local usage allowed)
but with the other bits zero (local usage not set).

    Brian

---

From: Mark Smith <[email-masked]>
Date: Tue, 23 Feb 2010 07:52:22 +1030

Hi Brian,

On Tue, 23 Feb 2010 09:46:50 +1300

> but with the other bits zero (local usage not set).

I'll re-read the draft again tonight to check my thinking, however this
is how I've been seeing the possible values

o  0x00000 - unknown/undefined and of no flow domain significance,
covering the common case today

0  0x00001 through 0x7ffff - end-to-end significance, so routers should
not change the value, regardless of whether flow domains exist or not

0  0x80000 through 0xfffff - local flow domain significance, can be
changed by a flow domain egress or ingres router. If the router does
not know of a new next (egress) or current flow domain (ingres) value,
it resets the flow value to 0x00000. Otherwise, the flow value is set
to an appropriate new or current flow domain's MSB=1 value. I think
this later option of modifying the value from a previous MSB=1 value to
a new one is the only difference between what I've been thinking and
what is in the draft.

Regards,
Mark.

---

From: Sheng Jiang <[email-masked]>
Date: Tue, 23 Feb 2010 14:30:45 +0800

> -----Original Message-----
> From: Mark Smith 
> [mailto:[email-masked]] 
> Sent: Monday, February 22, 2010 5:22 PM
> To: Sheng Jiang
> Cc: 'Brian E Carpenter'; '6man'
> Subject: Re: [Fwd: I-D Action:draft-carpenter-6man-flow-update-00.txt]
> 
> Hi Sheng,
> 
> 
> > >    o  Considering packets inbound to the Flow Label 
> Domain, if MSB = 
> > > 0,
> > >       unlikely case, and the boundary router MUST set all 
> 20 bits of 
> > > the
> > > of local significance to the second flow domain. This would be 
> > > somewhat similar to how ISPs can provide specific BGP community 
> > > values for it's customers to set to influence routing within the 
> > > ISP's AS. If there is no specified MSB=1 value for the 
> second flow 
> > > domain, then the flow label must be set to all zeros.
> > 
> > Yes, the second flow domain should be also able to use the local 
> > defined behaviors. In my opinion, when the packet leaves a flow 
> > domain, the egress router does not sure whether there will 
> be another 
> > flow domain or not (the second flow domain may be appear to 
> be several hops away).
> 
> I was thinking more of the situation where the egress router 
> does know that the adjacent domain is a flow domain, and 
> therefore is configured to change the flow value to another 
> MSB=1 value or leave it alone if the original MSB=1 value 
> matches the configured outgoing MSB=1 value.

Right.

> If there is no new configured outbound MSB=1 value, then I 
> think it is better to set the flow value to MSB=0/all bits 
> zero, indicating that the packet has left the prior flow 
> domain, and that the egress router does not have any 
> knowledge of any subsequent flow domains or their flow domain values.

Not sure here. All 0 bits mean killing the possibility the second flow
domain uses local defined behavior. It seems not right.
 
> > So, it should
> > leave the MSB=1 to keep the possibility. The rest 19 bits 
> may be reset 
> > to all 0 depends or untouched depending on whether the 
> egress router 
> > wants to prevent the exporting of local defined behaviours. 
> Actually, 
> > I don't think it is necessary to do so. The leaking of 
> locally defined 
> > behaviours is not harmful.
> > 
> 
> I think leaking them could be, if there are equivalent flow 
> values in the subsequent flow domain which have different meanings.

All right.  The leaking of locally defined behaviors may cause confusing
meanings. Then, the egress router should prevent the exporting of local
defined behaviours.

Cheers,

Sheng

> > > > Various uses proposed for the IPv6 flow label are incompatible 
> > > > with its existing specification.  This document 
> describes changes 
> > > > to the specification that permit additional use cases 
> as well as 
> > > > allowing continued use of the previous specification.
> > > > 
> http://www.ietf.org/internet-drafts/draft-carpenter-6man-flow-update
> > > -0
> > > > 0.txt
> > > > 
> > > > 
> ------------------------------------------------------------------
> > > > -- IETF IPv6 working group mailing list [email-masked] 
> > > > Administrative Requests: 
> > > > https://www.ietf.org/mailman/listinfo/ipv6
> > > > 
> > > IETF IPv6 working group mailing list [email-masked] Administrative 
> > > Requests: https://www.ietf.org/mailman/listinfo/ipv6
> > >

---

From: Sheng Jiang <[email-masked]>
Date: Tue, 23 Feb 2010 14:57:30 +0800

> Sent: Tuesday, February 23, 2010 5:22 AM
> To: Brian E Carpenter
> Cc: Sheng Jiang; '6man'
> > >>>    o  Considering packets outbound from the Flow Label 
> Domain, if 
> > >>> MSB =
> Domain, if MSB 
> > >>> = 0,
> > >>>       unlikely case, and the boundary router MUST set 
> all 20 bits 
> > >>> of the
> > >>> network and an ISP, the flow label could be changed to a MSB=1 
> > >>> value of local significance to the second flow domain. 
> This would 
> > >>> be somewhat similar to how ISPs can provide specific 
> BGP community 
> > >> domain, the egress router does not sure whether there will be 
> > >> another flow domain or not (the second flow domain may 
> be appear to be several hops away).
> > > 
> > > I was thinking more of the situation where the egress router does 
> > > know that the adjacent domain is a flow domain, and therefore is 
> > > configured to change the flow value to another MSB=1 
> value or leave 
> > > it alone if the original MSB=1 value matches the 
> configured outgoing MSB=1 value.
> > > If there is no new configured outbound MSB=1 value, then 
> I think it 
> > > is better to set the flow value to MSB=0/all bits zero, 
> indicating 
> > > that the packet has left the prior flow domain, and that 
> the egress 
> > > router does not have any knowledge of any subsequent flow 
> domains or 
> > > their flow domain values.
> > > 
> > >> So, it should
> > >> leave the MSB=1 to keep the possibility. The rest 19 bits may be 
> > >> reset to all 0 depends or untouched depending on whether 
> the egress 
> > >> router wants to prevent the exporting of local defined 
> behaviours. 
> > >> Actually, I don't think it is necessary to do so. The leaking of 
> > >> locally defined behaviours is not harmful.
> > >>
> > > 
> > > I think leaking them could be, if there are equivalent 
> flow values 
> > > in the subsequent flow domain which have different meanings.
> > 
> but with 
> > the other bits zero (local usage not set).
> > 
> 
> I'll re-read the draft again tonight to check my thinking, 
> however this is how I've been seeing the possible values

The possible values is one thing the others does not sure and want to
discuss in 6man mail list.

In the current 00 version, MSB=0 is to obey the rules of [RFC3697]. It means
the in-path routers cannot do anything in the current all zero situation,
which has overwhelming proportion now. It means when hosts do not use the
flow label, the other can NOT use it too! That's actually, in my opinion
what we should change.

In we can agreed all zero should be possible for using local defined
behaviors, there are two solutions:

A: MSB 0, the remaining 19 bits is allowed for Flow Label Domain usage.

B: Like Mark proposed, make all 0 outstanding value

Discussion are appreciated.

Regards,

Sheng
 
> o  0x00000 - unknown/undefined and of no flow domain 
> significance, covering the common case today
> 
> 0  0x00001 through 0x7ffff - end-to-end significance, so 
> routers should not change the value, regardless of whether 
> flow domains exist or not
> 
> 0  0x80000 through 0xfffff - local flow domain significance, 
> can be changed by a flow domain egress or ingres router. If 
> the router does not know of a new next (egress) or current 
> flow domain (ingres) value, it resets the flow value to 
> 0x00000. Otherwise, the flow value is set to an appropriate 
> new or current flow domain's MSB=1 value. I think this later 
> option of modifying the value from a previous MSB=1 value to 
> a new one is the only difference between what I've been 
> thinking and what is in the draft.
> 
> Regards,
> Mark.

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 24 Feb 2010 10:02:00 +1300

See below...

> Discussion are appreciated.

I think we can make this OK by the following type of rule (I really
haven't worked out all the possible cases carefully, however).

  If a packet arrives with all 0, then local use is allowed but
  if the packet leaves the local domain the label MUST be set
  back to all 0.

That actually requires that the local use method MUST include a flag
bit meaning "this was originally all 0", but that seems easy enough.
Then the current default behaviour is perfectly preserved when viewed
outside the local domain.

      Brian

---

From: Rémi Després <[email-masked]>
Date: Wed, 24 Feb 2010 14:39:29 +0100

> outside the local domain.

What makes the most sense to me is:
- first bit =0  ==> Must be preserved end to end.
- If there is some local use to traverse a domain, the value at entrance has to be restored at exit.

Now, if the entrance value is all 0, this fact can be coded in the local-use format, so that it is easy to restore the entrance value at exit. 
On the other hand, if the entrance value is E2E and non 0, some means to keep this value in the modified packet has to be found. How to do it has to be taken care of when designing each particular local use. 

Reasonable?

RD

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 25 Feb 2010 09:10:06 +1300

> On the other hand, if the entrance value is E2E and non 0, some means to keep this value in the modified packet has to be found. How to do it has to be taken care of when designing each particular…

100%, in my humble opinion. So there are actually three cases when
a packet arrives in a local-use domain (either from a local host or
from a border router):

Flow label is zero: local use allowed (with MSB=1) but label must be
set to zero on exit from the domain.

Value between 1 and 0x7FFFF: local use not allowed, label must not be changed

MSB=1: local use allowed.

     Brian

---

From: Sheng Jiang <[email-masked]>
Date: Thu, 25 Feb 2010 07:15:46 +0800

> From: Brian E Carpenter [mailto:[email-masked]] 
> Sent: Thursday, February 25, 2010 4:10 AM
> To: Rémi Després
> Cc: Sheng Jiang; '6man'
> >>  If a packet arrives with all 0, then local use is allowed but  if 
> >> the packet leaves the local domain the label MUST be set  
> back to all 
> >> 0.
> >>
> > - If there is some local use to traverse a domain, the 
> value at entrance has to be restored at exit.
> > 
> > Now, if the entrance value is all 0, this fact can be coded 
> in the local-use format, so that it is easy to restore the 
> entrance value at exit. 
> > On the other hand, if the entrance value is E2E and non 0, 
> some means to keep this value in the modified packet has to 
> be found. How to do it has to be taken care of when designing 
> each particular local use. 
> > 
> > Reasonable?
> 
> 100%, in my humble opinion. So there are actually three cases 
> when a packet arrives in a local-use domain (either from a 
> local host or from a border router):
> 
> Flow label is zero: local use allowed (with MSB=1) but label 
> must be set to zero on exit from the domain.
> 

This is petty good. It solves our original worry: local use is not allowed
when flow label is all zero. I guess we can update the draft with this
design now.

Cheers,

Sheng

---

From: Rémi Després <[email-masked]>
Date: Thu, 25 Feb 2010 08:48:58 +0100

Brian, Sheng,

The proposal seems nicely stabilized as regards the MSB=0 case.
IMHO though, more than proposed so far could be done concerning the MSB=1 case.

In sec 3, instead of "If the MSB of the flow label is 1, the remaining 19 bits MAY obey a locally defined set of rules and those bits MAY be changed en route", we could have:
"If the MSB of the flow label is 1, the remaining 19 bits MAY obey a different defined set of rules. Each such set will have to be identified by a specific value of the 3 bits that follow the MSB."

We therefore:
- avoid to exclude some new rule sets that would apply end to end;
- prevent that some new rule set would take for itself all possible values having MSB = 1.

Sorry to have this a little late, but better late than never, right? 

RD

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 26 Feb 2010 09:03:48 +1300

> "If the MSB of the flow label is 1, the remaining 19 bits MAY obey a different defined set of rules. Each such set will have to be identified by a specific value of the 3 bits that follow the MSB."

Hmm. I am not anxious to invent a central registration scheme that would mean the user
only has 16 bits left to play with. Some of the schemes I have seen (references
in the draft) use most of the 20 bits in ingenious ways.

     Brian

---

From: Sheng Jiang <[email-masked]>
Date: Fri, 26 Feb 2010 10:40:35 +0800

> Sent: Friday, February 26, 2010 4:04 AM
> To: Rémi Després
> Cc: Sheng Jiang; '6man'
> > In sec 3, instead of "If the MSB of the flow label is 1, 
> the remaining 19 bits MAY obey a locally defined set of rules 
> and those bits MAY be changed en route", we could have:
> > "If the MSB of the flow label is 1, the remaining 19 bits 
> MAY obey a different defined set of rules. Each such set will 
> have to be identified by a specific value of the 3 bits that 
> follow the MSB."
> 
> 
> Hmm. I am not anxious to invent a central registration scheme 
> that would mean the user only has 16 bits left to play with. 
> Some of the schemes I have seen (references in the draft) use 
> most of the 20 bits in ingenious ways.
> 
>      Brian
> 
> > We therefore:
> >>> Subject: Re: [Fwd: I-D 
> >>> Action:draft-carpenter-6man-flow-update-00.txt]
> >>>
> >>> in the local-use format, so that it is easy to restore 
> the entrance 
> >>> value at exit.
> >>> some means to keep this value in the modified packet has to be 
> >>> found. How to do it has to be taken care of when designing each 
> >>> particular local use.
> >>>> Reasonable?
> >>> 100%, in my humble opinion. So there are actually three 
> cases when a 
> >>> packet arrives in a local-use domain (either from a local host or 
> >>> from a border router):
> >>>
> >>> Value between 1 and 0x7FFFF: local use not allowed, label 
> must not 
> >>> be changed
> >>>
> >> This is petty good. It solves our original worry: local use is not 
> >> allowed when flow label is all zero. I guess we can update 
> the draft 
> >> with this design now.
> >>
> >> Cheers,
> >>
> >> Sheng
> >>
> > 
> > 
>

---

From: Sheng Jiang <[email-masked]>
Date: Fri, 26 Feb 2010 10:49:18 +0800

> most of the 20 bits in ingenious ways.

I guess, this is a little bit out of the scope. I prefer to discuss how to
use the remaining 19 bits with specific use cases. For me, until we have
specific use cases, we cannot decide whether identify bits for subset of
local defined behaviors are needed and how many bits may be needed. I also
agree with Brian, 19 bits is already very limited, give out any bit is
luxurious.

Sheng