Subject: Re: [sfc] FW: New Version Notification for draft-ao-sfc-oam-path-consistency-03.txt

From: "Kent Leung (kleung)" <[email-masked]>
Date: Thu, 8 Nov 2018 08:36:36 +0000

Hi Ting. I’m not convinced with the following: ‘SFF will distribute the SFC traffic flow to a given SF according to the Flow ID”. It’s possible that SFF forwards based on the Flow ID TLV. But in reality, a forwarding plane digging deep into the TLV header to find the field and distributed packets is bad for performance. In general, packets are forwarded based on L2-4 header. In stateless case, it’s some hash function (consistent hash) used for distribution. So, SFF may distributed SFC traffic based on Flow ID but COAM Reply message should not assume that’s always the case, IMHO.

I think the logic should be SFF determines the SF Identifier carried in the COAM Reply message the same way that SFF would have forwarded the packet to the SF. That may be based on L2 MAC, L3-4 header, etc. (whatever the load balancing operation is for data forwarding). This method would ensure the right SF Identifier is in the COAM Reply message.

Kent

From: sfc <[email-masked]> On Behalf Of [email-masked]
Sent: Monday, October 22, 2018 1:50 AM
To: [email-masked]
Subject: [sfc] FW: New Version Notification for draft-ao-sfc-oam-path-consistency-03.txt

Hi all，

We have updated the draft about the SFC OAM message to verify the consistency of the SFC path. This new version adds the specification about the path consistency check in load balance case. Please help rewiew and share your comments.

Any suggestions or questions are welcome.

Best Regards.

Ting Ao

原始邮件
发件人：[email-masked]<mailto:[email-masked]> <[email-masked]<mailto:[email-masked]>>
收件人：Gregory Mirsky <[email-masked]<mailto:[email-masked]>>;敖婷00071246;Zhonghua Chen <[email-masked]<mailto:[email-masked]>>;Greg Mirsky <[email-masked]<mailto:[email-masked]>>;
日 期 ：2018年10月19日 15:08
主 题 ：New Version Notification for draft-ao-sfc-oam-path-consistency-03.txt

A new version of I-D, draft-ao-sfc-oam-path-consistency-03.txt
has been successfully submitted by Ting Ao and posted to the
IETF repository.

Name:        draft-ao-sfc-oam-path-consistency
Revision:    03
Title:        SFC OAM for path consistency
Document date:    2018-10-16
Group:        Individual Submission
Pages:        9
URL:            https://www.ietf.org/internet-drafts/draft-ao-sfc-oam-path-consistency-03.txt
Status:         https://datatracker.ietf.org/doc/draft-ao-sfc-oam-path-consistency/
Htmlized:       https://tools.ietf.org/html/draft-ao-sfc-oam-path-consistency-03
Htmlized:       https://datatracker.ietf.org/doc/html/draft-ao-sfc-oam-path-consistency
Diff:           https://www.ietf.org/rfcdiff?url2=draft-ao-sfc-oam-path-consistency-03

Abstract:
   Service Function Chain (SFC) defines an ordered set of service
   functions (SFs) to be applied to packets and/or frames and/or flows
   selected as a result of classification.  SFC Operation,
   Administration and Maintenance can monitor the continuity of the SFC,
   i.e., that all elements of the SFC are reachable to each other in the
   downstream direction.  But SFC OAM must support verification that the
   order of traversing these SFs corresponds to the state defined by the
   SFC control plane or orchestrator, the metric referred in this
   document as the path consistency of the SFC.  This document defines a
   new SFC OAM method to support SFC consistency, i.e. verification that
   all elements of the given SFC are being traversed in the expected
   order.

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 8 Nov 2018 16:23:59 +0700

(speaking only as a participant.)

Two aspects:
First, since these are active OAM packets, without thte proposed TLV 
there is no L3 / L4 header to use for the ECMP when that applies.
Second, since this is an OAM packet anyway, I had assumed that 
slow-pathing the packet was an acceptable and intended tradeoff.  Maybe 
I misread the document in this regard.

Yours,
Joel

On 11/8/18 3:36 PM, Kent Leung (kleung) wrote:
> Hi Ting. I’m not convinced with the following: ‘SFF will distribute the 
> SFC traffic flow to a given SF according to the Flow ID”. It’s possible 
> that SFF forwards based on the Flow ID TLV. But in reality, a forwarding 
> plane digging deep into the TLV header to find the field and distributed 
> packets is bad for performance. In general, packets are forwarded based 
> on L2-4 header. In stateless case, it’s some hash function (consistent 
> hash) used for distribution. So, SFF may distributed SFC traffic based 
> on Flow ID but COAM Reply message should not assume that’s always the 
> case, IMHO.
> 
> I think the logic should be SFF determines the SF Identifier carried in 
> the COAM Reply message the same way that SFF would have forwarded the 
> packet to the SF. That may be based on L2 MAC, L3-4 header, etc. 
> (whatever the load balancing operation is for data forwarding). This 
> method would ensure the right SF Identifier is in the COAM Reply message.
> 
> Kent
> 
> *From:*sfc <[email-masked]> *On Behalf Of [email-masked]
> *Sent:* Monday, October 22, 2018 1:50 AM
> *To:* [email-masked]
> *Subject:* [sfc] FW: New Version Notification for 
> draft-ao-sfc-oam-path-consistency-03.txt
> 
> Hi all，
> 
> We have updated the draft about the SFC OAM message to verify the 
> consistency of the SFC path. This new version adds the specification 
> about the path consistency check in load balance case. Please help 
> rewiew and share your comments.
> 
> *发件人**：[email-masked] <mailto:[email-masked]> 
> <[email-masked] <mailto:[email-masked]>>
> 
> *收件人：*Gregory Mirsky <[email-masked] 
> <mailto:[email-masked]>>;敖婷00071246;Zhonghua Chen 
> <[email-masked] <mailto:[email-masked]>>;Greg Mirsky 
> <[email-masked] <mailto:[email-masked]>>;
> 
> *日 **期 **：*2018年10月19日 15:08
> 
> *主 **题 **：New Version Notification for 
> draft-ao-sfc-oam-path-consistency-03.txt*
> 
> 
> URL: 
> https://www.ietf.org/internet-drafts/draft-ao-sfc-oam-path-consistency-03.txt
> Htmlized: 
> https://datatracker.ietf.org/doc/html/draft-ao-sfc-oam-path-consistency
> _______________________________________________
> sfc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/sfc
>

---

From: "Kent Leung (kleung)" <[email-masked]>
Date: Thu, 8 Nov 2018 10:15:14 +0000

Hi Joel. OK, the COAM Request message don't carry the L3/L4 header of the flow. It's true that COAM messages don't need to be high performance. But I’m talking about the flow traffic traversing the SFF. SFF has to load balance the flow traffic to SF. The distribution logic is not likely to be based on NSH Flow ID TLV (which requires deep parsing on per packet basis). SFF needs to respond with the SF ID that it would have load balanced the flow traffic. 

So, assuming SFF is load balancing based on L2-L4 header, then COAM Request message should carry a new TLV that can carry the load balancing parameters. My last paragraph should still apply. Note Flow ID is only 4 bytes, not big enough to carry packet header.

> I think the logic should be SFF determines the SF Identifier carried 
> in the COAM Reply message the same way that SFF would have forwarded 
> the packet to the SF. That may be based on L2 MAC, L3-4 header, etc.

Kent

-----Original Message-----
From: Joel M. Halpern <[email-masked]> 
Sent: Thursday, November 8, 2018 1:24 AM
To: Kent Leung (kleung) <[email-masked]>; [email-masked]; [email-masked]
Subject: Re: [sfc] FW: New Version Notification for draft-ao-sfc-oam-path-consistency-03.txt

Two aspects:
First, since these are active OAM packets, without thte proposed TLV there is no L3 / L4 header to use for the ECMP when that applies.
Second, since this is an OAM packet anyway, I had assumed that slow-pathing the packet was an acceptable and intended tradeoff.  Maybe I misread the document in this regard.

Yours,
Joel

On 11/8/18 3:36 PM, Kent Leung (kleung) wrote:
> Hi Ting. I’m not convinced with the following: ‘SFF will distribute 
> the SFC traffic flow to a given SF according to the Flow ID”. It’s 
> possible that SFF forwards based on the Flow ID TLV. But in reality, a 
> forwarding plane digging deep into the TLV header to find the field 
> and distributed packets is bad for performance. In general, packets 
> are forwarded based on L2-4 header. In stateless case, it’s some hash 
> function (consistent
> I think the logic should be SFF determines the SF Identifier carried 
> in the COAM Reply message the same way that SFF would have forwarded 
> the packet to the SF. That may be based on L2 MAC, L3-4 header, etc.
> has been successfully submitted by Ting Ao and posted to the IETF 
> repository.
> 
> Title:        SFC OAM for path consistency Document date:    
> 2018-10-16
> Status: 
> https://datatracker.ietf.org/doc/draft-ao-sfc-oam-path-consistency/
> Htmlized: 
> https://tools.ietf.org/html/draft-ao-sfc-oam-path-consistency-03
> Htmlized: 
> Diff: 
> https://www.ietf.org/rfcdiff?url2=draft-ao-sfc-oam-path-consistency-03
> 
> Abstract:
> Please note that it may take a couple of minutes from the time of 
> submission until the htmlized version and diff are available at tools.ietf.org.
> 
> The IETF Secretariat
> 
> 
>

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 8 Nov 2018 17:23:53 +0700

I think from this that you are suggesting that the TLV needs to carry 
the full information we expect load placement decisions to use (without 
DPI, but with L4 header checks).  That sounds reasonable.  Since NSH 
TLVs are whatever size we say they are, it also seems achievable.

Yours,
Joel

---

From: <[email-masked]>
Date: Fri, 9 Nov 2018 09:41:14 +0800 (CST)

Hi Kent and Joel,

I really appreciate your comments and have this discussion. So I wonder should we specify the Flow ID TLV content in the draft? In my understanding that it's related to implementation. We only need to tell SFF when you get COAM message, you should send a certain SF information based on Flow ID. As regards what's the value in the Flow ID, that's depend on the policy. Maybe I miss something important:)

BR.

Ting

原始邮件

发件人：JoelM.Halpern <[email-masked]>
收件人：Kent Leung (kleung) <[email-masked]>;敖婷00071246;[email-masked] <[email-masked]>;
日 期 ：2018年11月08日 18:26
主 题 ：Re: [sfc] FW: New Version Notification fordraft-ao-sfc-oam-path-consistency-03.txt

Yours,
Joel

---

From: "jmh.direct" <[email-masked]>
Date: Fri, 09 Nov 2018 09:04:01 -0500

The complexity is that the creator of the TLV  has to provide wnough information tbat all the implementations along the path can do the right thibg.  Which, as far as I know means we need at least the IP source and dest, the protocol, the port pair, and probably the flow label.
Yours,Joel

Sent via the Samsung Galaxy S® 6, an AT&T 4G LTE smartphone
-------- Original message --------From: [email-masked] Date: 11/8/18  20:41  (GMT-05:00) To: [email-masked] Cc: [email-masked], [email-masked] Subject: Re: Re: [sfc] FW: New Version Notification fordraft-ao-sfc-oam-path-consistency-03.txt 
Hi Kent and Joel,I really appreciate your comments and have this discussion. So I wonder should we specify the Flow ID TLV content in the draft? In my understanding that it's related to implementation. We only need to tell SFF when you get COAM message, you should send a certain SF information based on Flow ID. As regards what's the value in the Flow ID, that's depend on the policy. Maybe I miss something important:)
BR.Ting

原始邮件发件人：JoelM.Halpern <[email-masked]>收件人：Kent Leung (kleung) <[email-masked]>;敖婷00071246;[email-masked] <[email-masked]>;日 期 ：2018年11月08日 18:26主 题 ：Re: [sfc] FW: New Version Notification fordraft-ao-sfc-oam-path-consistency-03.txtI think from this that you are suggesting that the TLV needs to carry 
the full information we expect load placement decisions to use (without 
DPI, but with L4 header checks).  That sounds reasonable.  Since NSH 
TLVs are whatever size we say they are, it also seems achievable.

Yours,
Joel

---

From: Greg Mirsky <[email-masked]>
Date: Sun, 11 Nov 2018 21:57:09 +0700

Dear All,
I agree with Kent that using Flow ID TLV in the fast path will likely be
expensive. On the other hand, load balancing may use information other than
L3/L4, e.g., the label stack. We may not know the proprietary methods of
load balancing similarly as we don't know how particular vendor selects the
path in ECMP environment. Perhaps we can suggest how to make the use of the
Flow ID TLV less costly. I think that if the Flow ID TLV is mandatory and
always MUST be the first in MD Type 2 is used, then effectively, it as part
of the fixed header. The Flow ID value MUST be initialized to 0 and MAY be
set to the non-zero value that represents a flow. The method to generate
the value is outside the scope of the specification.
What do you think?

Regards,
Greg

On Fri, Nov 9, 2018 at 9:04 PM jmh.direct <[email-masked]>
wrote:

> The complexity is that the creator of the TLV  has to provide wnough
> information tbat all the implementations along the path can do the right
> thibg.  Which, as far as I know means we need at least the IP source and
> dest, the protocol, the port pair, and probably the flow label.
>
> Yours,
> Joel
>
>
>
> -------- Original message --------
> From: [email-masked]
> Date: 11/8/18 20:41 (GMT-05:00)
> To: [email-masked]
> Cc: [email-masked], [email-masked]
> Subject: Re: Re: [sfc] FW: New Version Notification
> fordraft-ao-sfc-oam-path-consistency-03.txt
>
> Hi Kent and Joel,
>
> I really appreciate your comments and have this discussion. So I wonder
> should we specify the Flow ID TLV content in the draft? In my understanding
> that it's related to implementation. We only need to tell SFF when you get
> COAM message, you should send a certain SF information based on Flow ID. As
> regards what's the value in the Flow ID, that's depend on the policy. Maybe
> I miss something important:)
>
>
> BR.
>
> Ting
>
>
>
> 原始邮件
> *发件人：*JoelM.Halpern <[email-masked]>
> *收件人：*Kent Leung (kleung) <[email-masked]>;敖婷00071246;[email-masked] <
> [email-masked]>;
> *日 期 ：*2018年11月08日 18:26
> *主 题 ：**Re: [sfc] FW: New Version Notification
> fordraft-ao-sfc-oam-path-consistency-03.txt*

---

From: Joel Halpern Direct <[email-masked]>
Date: Sun, 11 Nov 2018 11:47:20 -0500

(Speakkng individually)

I originally ended the note with this thought, but then realized it 
probably should be the top line.  Note that there are algorithms for 
flow placement which are stable over the lifetime of flows, but unstable 
over longer time periods.  I know of no way to make a later probe packet 
match the path of the original data packet in such cases.  We may be 
trying to solve a problem that is not even well-defined.

Within the blow placement handling, I think there are two separate 
issues I think.

One is the need for the flow information for active OAM messaging.  When 
the OAM is intended to follow the same path as some specific flow, that 
seems important.  Including it as a TLV seems fine, since we do not 
expect OAM messages to use the same processing logic as a data path.

There is a problem case that immediately crops up.  What happens with 
service functions (such as NAT of many varieties) that change the 
information used for most load balancing.  Those service functions do 
not process OAM, much less understand and modify this TLV.  So the 
information becomes incorrect.

The second question is supporting SFFs that make load balancing 
decisions for data packets.  The current approach is that it is an 
implementation matter so long as it complies with being flow stable.  We 
could mandate an entropy TLV that is to be used instead of the packet 
information, with the requirement that the entropy be set stably for all 
packets in the same flow.  This would keep it small.
It would however restrict the load balancing choices the implementations 
make.  And if we wanted it to work for OAM, we would have to place some 
careful restrictions on it.

I see at least three additional problems with that.  First, making a TLV 
mandatory for a use case that only some deployments need strikes me as 
an unfortunate choice.  Second, I have never seen just one "must be 
first".  Which means that making any thing "always first" is very risky.
Third, I am personally uncomfortable making such a change to our agreed 
and published RFC (8300) for something that may be useful.

While there are other ways to carry the information in active OAM< it is 
qutie difficult to allow the desired implementation flexibility while 
still getting OAM to match packet flow load balancing.

Yours,
Joel

On 11/11/18 9:57 AM, Greg Mirsky wrote:
> expensive. On the other hand, load balancing may use information other 
> than L3/L4, e.g., the label stack. We may not know the proprietary 
> methods of load balancing similarly as we don't know how particular 
> vendor selects the path in ECMP environment. Perhaps we can suggest how 
> to make the use of the Flow ID TLV less costly. I think that if the Flow 
> ID TLV is mandatory and always MUST be the first in MD Type 2 is used, 
> then effectively, it as part of the fixed header. The Flow ID value MUST 
> be initialized to 0 and MAY be set to the non-zero value that represents 
> a flow. The method to generate the value is outside the scope of the 
> specification.
> What do you think?
> 
> Regards,
> Greg
> 
> On Fri, Nov 9, 2018 at 9:04 PM jmh.direct <[email-masked] 
> <mailto:[email-masked]>> wrote:
> 
>     information tbat all the implementations along the path can do the
>     right thibg.  Which, as far as I know means we need at least the IP
>     source and dest, the protocol, the port pair, and probably the flow
>     label.
> 
>     Yours,
>     Joel
> 
> 
> 
>     From: [email-masked] <mailto:[email-masked]>
>     To: [email-masked] <mailto:[email-masked]>
>     Cc: [email-masked] <mailto:[email-masked]>, [email-masked]
>     <mailto:[email-masked]>
>     I really appreciate your comments and have this discussion. So I
>     wonder should we specify the Flow ID TLV content in the draft? In my
>     understanding that it's related to implementation. We only need to
>     tell SFF when you get COAM message, you should send a certain SF
>     information based on Flow ID. As regards what's the value in the
>     Flow ID, that's depend on the policy. Maybe I miss something important:)
> 
> 
>     BR.
> 
>     Ting
> 
> 
> 
>     原始邮件
>     *发件人：*JoelM.Halpern <[email-masked]
>     <mailto:[email-masked]>>
>     *收件人：*Kent Leung (kleung) <[email-masked]
>     <mailto:[email-masked]>>;敖婷00071246;[email-masked]
>     <mailto:[email-masked]> <[email-masked] <mailto:[email-masked]>>;
>     *日 期 ：*2018年11月08日 18:26
>      > From: Joel M. Halpern <[email-masked]
>     <mailto:[email-masked]>>
>      > To: Kent Leung (kleung) <[email-masked]
>     <mailto:[email-masked]>>; [email-masked]
>     <mailto:[email-masked]>; [email-masked] <mailto:[email-masked]>
>      >> *From:*sfc <[email-masked]
>     <mailto:[email-masked]>> *On Behalf Of [email-masked]
>     <mailto:[email-masked]>
>      >> *To:* [email-masked] <mailto:[email-masked]>
>      >> *发件人**：[email-masked]
>     <mailto:[email-masked]> <mailto:[email-masked]
>     <mailto:[email-masked]>>
>      >> <[email-masked]
>     <mailto:[email-masked]> <mailto:[email-masked]
>     <mailto:[email-masked]>>>
>      >>
>     <mailto:[email-masked]>
>      >> <mailto:[email-masked] <mailto:[email-masked]>>>;
>     敖婷00071246;Zhonghua Chen
>      >> <[email-masked]
>     <mailto:[email-masked]> <mailto:[email-masked]
>     <mailto:[email-masked]>>>;Greg Mirsky
>      >> <[email-masked]
>     <mailto:[email-masked]> <mailto:[email-masked]
>     <mailto:[email-masked]>>>;
>      >>
>      >> submission until the htmlized version and diff are available at
>     tools.ietf.org <http://tools.ietf.org>.
>      >>
>      >> The IETF Secretariat
>      >>
>      >>
>      >>
>      >> [email-masked] <mailto:[email-masked]>
>     [email-masked] <mailto:[email-masked]>