Subject: draft-fioccola-v6ops-ipv6-alt-mark-00 and RFC7837

From: Fioccola Giuseppe <[email-masked]>
Date: Thu, 22 Mar 2018 10:33:36 +0000

Hi All,
After the discussion about the bits from the flow label field, the authors would consider a new alternative.
RFC 7837 specifies an IPv6 destination option capable of carrying Congestion Exposure markings. We could redefine this RFC, that is experimental, and use 2 bits from the Reserved field to encode the alternate marking method for an HBH option.
We would like to ask your opinion.

Thanks,

Giuseppe

Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle persone indicate. La diffusione, copia o qualsiasi altra azione derivante dalla conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbiate ricevuto questo documento per errore siete cortesemente pregati di darne immediata comunicazione al mittente e di provvedere alla sua distruzione, Grazie. 

This e-mail and any attachments is confidential and may contain privileged information intended for the addressee(s) only. Dissemination, copying, printing or use by anybody else is unauthorised. If you are not the intended recipient, please delete this message and any attachments and advise the sender by return e-mail, Thanks. 

Rispetta l'ambiente. Non stampare questa mail se non è necessario.

---

From: Fred Baker <[email-masked]>
Date: Tue, 27 Mar 2018 18:17:08 +0100

A characteristic of the Destination Options field is that it is interpreted by a system designated by the destination address. In a unicast scenario, there is obviously only one of those; in an anycast scenario, it might apply to a variety of systems, each with different routing. Is that a characteristic of your use case? I understood you to be measuring to a system that the traffic passed through, but not identifying the system in the destination address.

> On Mar 22, 2018, at 10:33 AM, Fioccola Giuseppe <[email-masked]> wrote:
> 
> Hi All,
> --------------------------------------------------------------------
> IETF IPv6 working group mailing list
> [email-masked]
> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> --------------------------------------------------------------------

---

From: Fioccola Giuseppe <[email-masked]>
Date: Wed, 28 Mar 2018 08:26:48 +0000

Hi Fred,
You are right! It would be best to allow the performance measurement both end to end and hop by hop. 
Probably we have to extend the Destination Options to Hop-by-Hop Options field.
Thanks,

Giuseppe,

---

From: "Van De Velde, Gunter (Nokia - BE/Antwerp)" <[email-masked]>
Date: Wed, 28 Mar 2018 08:38:03 +0000

May i assume the following assumption for the intended measurement use-case:
   On the router there should be no difference regarding forwarding behavior  
   when a packet is either 'marked' or 'not market'? It is assumed that the router does 
   never fallback to slow path when a marking is present.
 
This brings me to the following: Does placement of ALTERNATE MARKING in a new IPv6 HbH extension header not punt those packets to CPU for local processing?

G/

-----Original Message-----
From: Fioccola Giuseppe [mailto:[email-masked]] 
Sent: Wednesday, March 28, 2018 10:27
To: Fred Baker <[email-masked]>
Cc: [email-masked]; [email-masked]
Subject: R: draft-fioccola-v6ops-ipv6-alt-mark-00 and RFC7837

Hi Fred,

Giuseppe,

---

From: Tim Chown <[email-masked]>
Date: Wed, 28 Mar 2018 10:24:17 +0000

> On 28 Mar 2018, at 09:38, Van De Velde, Gunter (Nokia - BE/Antwerp) <[email-masked]> wrote:
> 

I think that is covered by Section 5 of RFC 7837, which you might view as a document describing the lesser of many evils... HBH is explicitly not used for that reason.

Tim

---

From: "Van De Velde, Gunter (Nokia - BE/Antwerp)" <[email-masked]>
Date: Wed, 28 Mar 2018 10:44:01 +0000

I'm not sure it is the lesser of all evils. It does break some fundamental understanding of what a router should look at in a IPv6 header.

So basically what we are saying:

"Lets stuff this marking info in a destination ext header (Conex header), but because transit router using fast-path are blind for destination headers, lets respin the required HW to look at destination extension headers nevertheless and break fundamental IPv6 rules on header processing"?

So, how is this different from:

"Lets stuff this marking info in the flow label of the IPv6 tunnel encapsulation (for example in Mobile backhaul), and change on the relevant routers the flow label entropy hashing for ECMP?" 

The above does not need a respin of HW at all, there is no need to change the IPv6 fundamental assumptions, and it is backward compatible with anything that is out there in the market.... to me that sounds as lesser evil as the Conex implications.

G/

-----Original Message-----
From: Tim Chown [mailto:[email-masked]] 
Sent: Wednesday, March 28, 2018 12:24
To: Van De Velde, Gunter (Nokia - BE/Antwerp) <[email-masked]>
Cc: Fioccola Giuseppe <[email-masked]>; Fred Baker <[email-masked]>; [email-masked]; [email-masked]
Subject: Re: draft-fioccola-v6ops-ipv6-alt-mark-00 and RFC7837

Tim

---

From: Tim Chown <[email-masked]>
Date: Wed, 28 Mar 2018 10:49:33 +0000

> On 28 Mar 2018, at 11:44, Van De Velde, Gunter (Nokia - BE/Antwerp) <[email-masked]> wrote:
> 

Yes, perhaps that was a bad phrase.  But RFC 7837 has trodden this path before, for similar reasons to the alt-mark draft.

> The above does not need a respin of HW at all, there is no need to change the IPv6 fundamental assumptions, and it is backward compatible with anything that is out there in the market.... to me tha…

Well, we've had at least two shifts in Flow Label spec; the original 24 bit label was reduced to 20 bits (perhaps some bits should have been reserved then, you might say), and later RFC 6437 said you could change the Flow Label in flight if it's currently set to 0 (but of course it's now quite commonly set, compared to 10 years ago).  Is it too late for another shift?

There's no easy answer here, is there?

Tim

---

From: "Van De Velde, Gunter (Nokia - BE/Antwerp)" <[email-masked]>
Date: Wed, 28 Mar 2018 11:42:49 +0000

>Well, we've had at least two shifts in Flow Label spec; the original 24 bit label was reduced to 20 bits (perhaps some bits should
>have been reserved then, you might say), and later RFC 6437 said you could change the Flow Label in flight if it's currently set to 0
>(but of course it's now quite commonly set, compared to 10 years ago).  Is it too late for another shift?

>There's no easy answer here, is there?

GV> In this case there is no real shift, because flow label is NOT changed for payloads on the fly by any router and neither on any transit router. The ingress router, which is the router that adds the IPv6 tunnel encapsulation, originates a flow label accordingly when pushing the IPv6 tunnel encapsulation header. And even more, this flow-label remains untouched, until the packet reaches the tunnel end-point and the tunnel encapsulation is removed, and the original payload with original headers (IPv4, IPv6 or mickey-mouse protocol) is delivered.

GV> This particular technology finds its use-case in strictly controlled operator networks. The operator use-case is intended to transparently transport payload between ingress and egress network edges. Typically tunnel encapsulations are pushed upon the payload to manage the original non-augmented payload; to steer the payload (Traffic engineering); to provide services (L2/3VPNs, QoS, etc..) and to provide SLA metrics. These pushed payload management encapsulations are imposed at ingress edge, and are removed again at network egress edge... The complete environment for the use-case is 100% under control of the operator, and the operator is in full awareness of the compromise/trade-off between using marking in flow label or not using marking semantics. This same operator can ask for example his equipment vendors to update the Network Operating SW for the equipment to provide enhanced ECMP based hashing, which seems an easier request as spinning new HW to achieve the same goal. 

G/ 

-----Original Message-----
From: Tim Chown [mailto:[email-masked]] 
Sent: Wednesday, March 28, 2018 12:50
To: Van De Velde, Gunter (Nokia - BE/Antwerp) <[email-masked]>
Cc: Fioccola Giuseppe <[email-masked]>; Fred Baker <[email-masked]>; [email-masked]; [email-masked]
Subject: Re: draft-fioccola-v6ops-ipv6-alt-mark-00 and RFC7837

Tim

> Cc: Fioccola Giuseppe <[email-masked]>; Fred Baker 
> <[email-masked]>; 
> [email-masked]; [email-masked]
>>  On the router there should be no difference regarding forwarding 
>> behavior  when a packet is either 'marked' or 'not market'? It is 
>> assumed that the router does  never fallback to slow path when a marking is present.
>> 
>>> IETF IPv6 working group mailing list [email-masked] Administrative 
>>> Requests: https://www.ietf.org/mailman/listinfo/ipv6

---

From: Tom Herbert <[email-masked]>
Date: Wed, 28 Mar 2018 08:52:28 -0700

On Wed, Mar 28, 2018 at 4:42 AM, Van De Velde, Gunter (Nokia -
> GV> In this case there is no real shift, because flow label is NOT changed for payloads on the fly by any router and neither on any transit router. The ingress router, which is the router that adds…
I don't understand this part of the argument. If encapsulation is
already being done, then why not put the two bits inside the
encapsulation? For instance, as I mentioned in intarea WG, we could
easily allocate two flag bits in GUE header for this purpose. I think
VXLAN and some other encapsulations also allow this.

> GV> This particular technology finds its use-case in strictly controlled operator networks. The operator use-case is intended to transparently transport payload between ingress and egress network e…
I think you're oversimplifying this. This might be plausible in small
networks of a couple thousand hosts, but in a large sprawling network
that would have a mix of different technologies, management methods,
and hardware from different vendors-- I don't think this approach
scales. Having worked in such environments, I've never seen even one
instance where we just called up "our vendor" and asked then for
protocol change and everything just worked seamlessly. Introducing a
change like this could be a major effort.

Tom

---

From: Ole Troan <[email-masked]>
Date: Wed, 28 Mar 2018 18:35:17 +0200

Indeed, if you encapsulate you can use as many bits you like. 

Cheers 
Ole

> On 28 Mar 2018, at 17:52, Tom Herbert <[email-masked]> wrote:
>

---

From: "Van De Velde, Gunter (Nokia - BE/Antwerp)" <[email-masked]>
Date: Wed, 28 Mar 2018 16:55:11 +0000

I assume that GUA intends refer to Global Unicast Address, correct?

Next if i understand correctly you suggest for example to use the least two signifiant bits for context semantics for the marking?

That could work at first glance to provide marking context...

A follow-up thought is that we should consider that all protocols providing (overlay) services which run across tunnel encaps need some augmentation (for example all that wonderfull BGP based overlay technology). Or that a tunnel tail-end needs to terminate two addresses on the same tunnel instance. Sounds not as such a simple proposal either... not sure this type of solution is easier for an operator to manage compared to living with the artifact of the outer header flow-label proposal.

G/

Get Outlook for iOS<https://aka.ms/o0ukef>
________________________________
From: Ole Troan <[email-masked]>
Sent: Wednesday, March 28, 2018 6:35:17 PM
To: Tom Herbert
Cc: Van De Velde, Gunter (Nokia - BE/Antwerp); [email-masked]; [email-masked]
Subject: Re: draft-fioccola-v6ops-ipv6-alt-mark-00 and RFC7837

Cheers
Ole

---

From: Tom Herbert <[email-masked]>
Date: Wed, 28 Mar 2018 11:53:14 -0700

On Wed, Mar 28, 2018 at 9:55 AM, Van De Velde, Gunter (Nokia -
> I assume that GUA intends refer to Global Unicast Address, correct?
I think you may have misread that. The reference was to GUE which is
an encapsulation protocol.

> Next if i understand correctly you suggest for example to use the least two
> signifiant bits for context semantics for the marking?
>
That would have the same problem the flow label solution has wrt ECMP.
Every device that use IP addresses in an ECMP hash would need to be
updated to mask the two bits in the address.

> A follow-up thought is that we should consider that all protocols providing
> (overlay) services which run across tunnel encaps need some augmentation
> (for example all that wonderfull BGP based overlay technology). Or that a
> tunnel tail-end needs to terminate two addresses on the same tunnel
> instance. Sounds not as such a simple proposal either... not sure this type
> of solution is easier for an operator to manage compared to living with the
> artifact of the outer header flow-label proposal.
>
If it's part of the encapsulation then it justs becomes an attribute
of encapsulation and tunnels that already needs to be managed. If the
flow label semantics are modified, then the operator has to consider
the impact and management of every intermediate device that might be
affected regarless of whether it's a tunnel endpoint.

Tom

> Get Outlook for iOS
> Cc: Van De Velde, Gunter (Nokia - BE/Antwerp);
>>>>> "Lets stuff this marking info in the flow label of the IPv6 tunnel
>>>>> encapsulation (for example in Mobile backhaul), and change on the relevant
>>>>> routers the flow label entropy hashing for ECMP?"
>>>>>
>>>>> The above does not need a respin of HW at all, there is no need to
>>>>> change the IPv6 fundamental assumptions, and it is backward compatible with
>>>>> anything that is out there in the market.... to me that sounds as lesser
>>>>> evil as the Conex implications.
>>>
>>>> Well, we've had at least two shifts in Flow Label spec; the original 24
>>>> bit label was reduced to 20 bits (perhaps some bits should
>>>> have been reserved then, you might say), and later RFC 6437 said you
>>>> could change the Flow Label in flight if it's currently set to 0
>>> GV> In this case there is no real shift, because flow label is NOT
>>> changed for payloads on the fly by any router and neither on any transit
>>> router. The ingress router, which is the router that adds the IPv6 tunnel
>>> encapsulation, originates a flow label accordingly when pushing the IPv6
>>> tunnel encapsulation header. And even more, this flow-label remains
>>> untouched, until the packet reaches the tunnel end-point and the tunnel
>>> encapsulation is removed, and the original payload with original headers
>>> (IPv4, IPv6 or mickey-mouse protocol) is delivered.
>>>
>>> GV> This particular technology finds its use-case in strictly controlled
>>> operator networks. The operator use-case is intended to transparently
>>> transport payload between ingress and egress network edges. Typically tunnel
>>> encapsulations are pushed upon the payload to manage the original
>>> non-augmented payload; to steer the payload (Traffic engineering); to
>>> provide services (L2/3VPNs, QoS, etc..) and to provide SLA metrics. These
>>> pushed payload management encapsulations are imposed at ingress edge, and
>>> are removed again at network egress edge... The complete environment for the
>>> use-case is 100% under control of the operator, and the operator is in full
>>> awareness of the compromise/trade-off between using marking in flow label or
>>> not using marking semantics. This same operator can ask for example his
>>> equipment vendors to update the Network Operating SW for the equipment to
>>> provide enhanced ECMP based hashing, which seems an easier request as
>>> spinning new HW to achieve the same goal.
>>>
>>> <[email-masked]>; [email-masked];
>>> [email-masked]
>>>> I'm not sure it is the lesser of all evils. It does break some
>>>> fundamental understanding of what a router should look at in a IPv6 header.
>>>
>>> Yes, perhaps that was a bad phrase.  But RFC 7837 has trodden this path
>>> before, for similar reasons to the alt-mark draft.
>>>
>>>> "Lets stuff this marking info in a destination ext header (Conex
>>>> header), but because transit router using fast-path are blind for
>>>> destination headers, lets respin the required HW to look at destination
>>>> extension headers nevertheless and break fundamental IPv6 rules on header
>>>> processing"?
>>>>
>>>> "Lets stuff this marking info in the flow label of the IPv6 tunnel
>>>> encapsulation (for example in Mobile backhaul), and change on the relevant
>>>> routers the flow label entropy hashing for ECMP?"
>>>>
>>>> The above does not need a respin of HW at all, there is no need to
>>>> change the IPv6 fundamental assumptions, and it is backward compatible with
>>>> anything that is out there in the market.... to me that sounds as lesser
>>>> evil as the Conex implications.
>>>
>>> Well, we've had at least two shifts in Flow Label spec; the original 24
>>> bit label was reduced to 20 bits (perhaps some bits should have been
>>> reserved then, you might say), and later RFC 6437 said you could change the
>>> Flow Label in flight if it's currently set to 0 (but of course it's now
>>> quite commonly set, compared to 10 years ago).  Is it too late for another
>>> shift?
>>>
>>>>> This brings me to the following: Does placement of ALTERNATE MARKING in
>>>>> a new IPv6 HbH extension header not punt those packets to CPU for local
>>>>> processing?
>>>>
>>>> I think that is covered by Section 5 of RFC 7837, which you might view
>>>> as a document describing the lesser of many evils... HBH is explicitly not
>>>> used for that reason.
>>>>
>>>> Tim
>>>>
>>>>> G/
>>>>>
>>>>> A characteristic of the Destination Options field is that it is
>>>>> interpreted by a system designated by the destination address. In a unicast
>>>>> scenario, there is obviously only one of those; in an anycast scenario, it
>>>>> might apply to a variety of systems, each with different routing. Is that a
>>>>> characteristic of your use case? I understood you to be measuring to a
>>>>> system that the traffic passed through, but not identifying the system in
>>>>> the destination address.
>>>>>
>>>>>> On Mar 22, 2018, at 10:33 AM, Fioccola Giuseppe
>>>>>> <[email-masked]> wrote:
>>>>>>
>>>>>> Hi All,
>>>>>> After the discussion about the bits from the flow label field, the
>>>>>> authors would consider a new alternative.
>>>>>> RFC 7837 specifies an IPv6 destination option capable of carrying
>>>>>> Congestion Exposure markings. We could redefine this RFC, that is
>>>>>> experimental, and use 2 bits from the Reserved field to encode the alternate
>>>>>> marking method for an HBH option.
>>>>>> Questo messaggio e i suoi allegati sono indirizzati esclusivamente
>>>>>> alle persone indicate. La diffusione, copia o qualsiasi altra azione
>>>>>> derivante dalla conoscenza di queste informazioni sono rigorosamente
>>>>>> vietate. Qualora abbiate ricevuto questo documento per errore siete
>>>>>> cortesemente pregati di darne immediata comunicazione al mittente e di
>>>>>> provvedere alla sua distruzione, Grazie.
>>>>>>
>>>>>> This e-mail and any attachments is confidential and may contain
>>>>>> privileged information intended for the addressee(s) only. Dissemination,
>>>>>> copying, printing or use by anybody else is unauthorised. If you are not the
>>>>>> intended recipient, please delete this message and any attachments and
>>>>>> advise the sender by return e-mail, Thanks.
>>>>>>

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 29 Mar 2018 08:38:18 +1300

Hi Tim,
On 28/03/2018 23:49, Tim Chown wrote:
...
> Well, we've had at least two shifts in Flow Label spec; the original 24 bit label was reduced to 20 bits (perhaps some bits should have been reserved then, you might say), and later RFC 6437 said y…

The issue is that 20 bits only gives you a space of a million possible pseudo-random values.
That isn't many, when you consider packet flows at Internet scale. The collision risk
using cheap hash algorithms is non-negligible.
https://researchspace.auckland.ac.nz/bitstream/handle/2292/13240/flowhashRep.pdf?sequence=2
That sort of collision rate is acceptable for ECMP and other load balancing usage, but
not for much else.

So personally I believe that *for Internet wide use* we should not revisit this.

    Brian