Subject: [nvo3] OAM: 96 bits of customer data?

From: Erik Nordmark <[email-masked]>
Date: Fri, 22 May 2015 08:31:50 -0700

Deepak,

Didn't have time to go into this on the call, but the draft and your 
slides refer to "96 bits of customer data"
I understand why that was needed for TRILL - basically the entropy for 
ECMP/LAG is calculated at each hop in TRILL so it needs to look at the 
inner Ethernet, IP, and TCP/UDP headers.

But for NVO3 the thinking is to use a UDP header where the source UDP 
port is set (in the ingress NVE) to some hash of those inner addresses 
and ports.
If that is the source of entropy, why do we need to also carry 96 bits 
of the inner packet in the OAM frames?

Regards,
     Erik

---

From: Erik Nordmark <[email-masked]>
Date: Fri, 22 May 2015 08:32:50 -0700

Deepak,

Regards,
     Erik

---

From: "Deepak Kumar (dekumar)" <[email-masked]>
Date: Fri, 22 May 2015 20:28:40 +0000

Hi Erik,

Agreed we don't ned 96 bytes of customer data in all scenario for intra-DC
but it will required in few scenarios and need to be use judiciously by
user.

Network Diagram (Tried my best I will do better diagram in the draft)

        /  S1 \       / S2  \           (L3 VTEP)
       /       \     /       \      (full mesh in east/west, x1<->s1,s2,
x2<->s1,s2, ...)
      /         \   /         \
     X1         X2       X3       X4 ...
     /  \     /  |
    /    \   /   |           full mesh in east/west l1<->x1, x2, x3, x4,
l2 <-> x1,x2,x3,x4, l3 ..)
   /      \ /    | 
  L1            L2       L3        L4 Š       (L2 VTEP)

   |            |
  H1            H2 ŠŠ

Now H1 is host, L1 is Switch, X1 is switch in underlay, S1 is switch with
l3 gateway functionality.

All switches are connected in East West fully connected with multiple way
ecmp.

Customer want to test L1 - Lx scenario providing Host information (H1, and
Hx)
For example: H1 and Hx are in inter subnet and across mobility domain or
any scenario where traffic will reach till L3 VTEP.
L3 VTEP will do de-encap of outer header, vxlan header and do again
re-encap of packet towards Lx.
Now to forward packet toward Lx Inner payload should be same as customer
information so right hashing is choosen as it's done for real data packet.

Thanks,
Deepak

---

From: Qin Wu <[email-masked]>
Date: Mon, 25 May 2015 05:43:06 +0000

Agree with Deepak.

-Qin
-----邮件原件-----
发件人: nvo3 [mailto:[email-masked]] 代表 Deepak Kumar (dekumar)
发送时间: 2015年5月23日 4:29
收件人: Erik Nordmark
抄送: [email-masked]
主题: Re: [nvo3] OAM: 96 bits of customer data?

Hi Erik,

Agreed we don't ned 96 bytes of customer data in all scenario for intra-DC but it will required in few scenarios and need to be use judiciously by user.

        /  S1 \       / S2  \           (L3 VTEP)

   |            |
  H1            H2 ŠŠ

All switches are connected in East West fully connected with multiple way ecmp.

Customer want to test L1 - Lx scenario providing Host information (H1, and
Hx)
For example: H1 and Hx are in inter subnet and across mobility domain or any scenario where traffic will reach till L3 VTEP.
L3 VTEP will do de-encap of outer header, vxlan header and do again re-encap of packet towards Lx.
Now to forward packet toward Lx Inner payload should be same as customer information so right hashing is choosen as it's done for real data packet.

Thanks,
Deepak

>of the inner packet in the OAM frames?

_______________________________________________
nvo3 mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/nvo3

---

From: Haoweiguo <[email-masked]>
Date: Tue, 26 May 2015 00:44:08 +0000

Yes, Deepak's explanation is reasonable, for inter-subnet ping/trace, layer 3 gateway needs to know the original customer payload head for re-encaping to dest.

Thanks,
weiguo
________________________________________
From: Qin Wu [[email-masked]]
Sent: Monday, May 25, 2015 13:43
To: Deepak Kumar (dekumar); Erik Nordmark
Cc: [email-masked]
Subject: Re: [nvo3] OAM: 96 bits of customer data?

Agree with Deepak.

-Qin
-----邮件原件-----
发件人: nvo3 [mailto:[email-masked]] 代表 Deepak Kumar (dekumar)
发送时间: 2015年5月23日 4:29
收件人: Erik Nordmark
抄送: [email-masked]
主题: Re: [nvo3] OAM: 96 bits of customer data?

Hi Erik,

        /  S1 \       / S2  \           (L3 VTEP)

   |            |
  H1            H2 ŠŠ

Thanks,
Deepak

---

From: Erik Nordmark <[email-masked]>
Date: Wed, 27 May 2015 10:35:23 -0700

Deepak,

As I understand your example below it is about testing inter-subnet 
traffic i.e. a case where you have overlay routers which forward packets 
based on the original/inner IP header.

That seems like a case were one could be using existing IP tools (ping, 
traceroute) in the overlay, and perhaps extending the information 
returned in ICMP errors following the approach used for MPLS (RFC4884 
and RFC4950). Perhaps we should consider those RFCs more in NVO3 since 
it means less need for new tools and new training - many ping and 
traceroute implementations already print the extensions.

Do you see a need for the 96 bits of customer data when there is no NVO3 
overlay routers in the path?

Thanks,
    Erik

---

From: "Darrel Lewis (darlewis)" <[email-masked]>
Date: Thu, 28 May 2015 18:50:56 +0000

On May 27, 2015, at 10:35 AM, Erik Nordmark <[email-masked]> wrote:

> As I understand your example below it is about testing inter-subnet traffic i.e. a case where you have overlay routers which forward packets based on the original/inner IP header.
> 
> That seems like a case were one could be using existing IP tools (ping, traceroute) in the overlay, and perhaps extending the information returned in ICMP errors following the approach used for MPLS (RFC4884 and RFC4950). Perhaps we should consider those RFCs more in NVO3 since it means less need for new tools and new training - many ping and traceroute implementations already print the extensions.

This approach seems reasonable to me.  Its certainly what we’ve attempted to do with the LISP overlay.  I think the problem space is complex enough without trying to bold the ocean.  The good thing about this incremental approach is that if operational experience proofs a need, it can be addressed then.

-=Darrel

> 
> Do you see a need for the 96 bits of customer data when there is no NVO3 overlay routers in the path?
> 
> Thanks,
>   Erik
> 
>> 
>>

---

From: "Deepak Kumar (dekumar)" <[email-masked]>
Date: Thu, 28 May 2015 19:36:16 +0000

Hi,

Inline +++DK:

>>traceroute implementations already print the extensions.

+++DK: Even in case of intra-subnet traffic we will go till Spine if
customer creates their data center in multiple pods or domains.

Eg:-
Each PODs has 100 Leafs pair, and connectivity between POD for even
briding traffic route is only at L3 Spine.

Within POD path taken will not go till S1,s2, but for inter-POD it will
reach S1.

Also Traceroute or TTL expiry doesn't test the exact datapath in
forwarding in spine, as real traffic will cover as ASIC treat outer layer
TTL expiry different than looking at inner header, de-encap, encap
changing vni and forwarding towards the leafs.

Thanks,
Deepak

>
>This approach seems reasonable to me.  Its certainly what we've attempted
>to do with the LISP overlay.  I think the problem space is complex enough
>without trying to bold the ocean.  The good thing about this incremental
>approach is that if operational experience proofs a need, it can be
>addressed then.
>
>-=Darrel
>
>>