Subject: [OPSAWG] Call for Adoption "draft-song-opsawg-ifit-framework"

From: Haoyu Song <[email-masked]>
Date: Mon, 21 Oct 2019 18:34:12 +0000

Dear OPSAWG chairs,

The following draft has been extensively discussed and gone through six revisions. Network operators confirmed it is useful.
We believe the draft is mature enough to be adopted by the WG therefore we request the chairs to initiate the adoption call for this draft.
Thank you very much for the consideration!

https://datatracker.ietf.org/doc/draft-song-opsawg-ifit-framework/

Best regards,
Haoyu

---

From: "qinfengwei" <[email-masked]>
Date: Tue, 22 Oct 2019 14:43:03 +0800

Dear OPSAWG chairs,

         I support the adoption poll as a co-author.

         I have read this doc and the updates are great. I think it is
useful in the existing network operation and  maintenance and it has been
ready for adoption as a wg doc.

Thanks,

Fengwei Qin

发件人: OPSAWG [mailto:[email-masked]] 代表 Haoyu Song
发送时间: 2019年10月22日 02:34
收件人: [email-masked]; [email-masked]
主题: [OPSAWG] Call for Adoption "draft-song-opsawg-ifit-framework"

Dear OPSAWG chairs,

The following draft has been extensively discussed and gone through six
revisions. Network operators confirmed it is useful.

We believe the draft is mature enough to be adopted by the WG therefore we
request the chairs to initiate the adoption call for this draft.

Best regards,

Haoyu

---

From: 신종윤님 <[email-masked]>
Date: Tue, 22 Oct 2019 20:07:55 +0000

Dear WG,

I think that the draft is ready for WG adoption and would like to request the chairs to start the adoption call.
I believe that this documents give us the framework to help to apply data plane on-path telemetry solution across the various network environment (SR-MPLS, SRv6, VXLAN, etc.), which is one of the most important features in our next SDN platform for optimal and efficient network operation.

Best regards,
Jongyoon

From: Haoyu Song [mailto:[email-masked]]
Sent: Tuesday, October 22, 2019 2:34 AM
To: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: Call for Adoption "draft-song-opsawg-ifit-framework"

Dear OPSAWG chairs,

Best regards,
Haoyu

---

From: "Joe Clarke (jclarke)" <[email-masked]>
Date: Tue, 22 Oct 2019 22:47:50 +0000

The comments below are from me as an individual contributor.

I have read the latest revision of this document.  I still do not have a clear idea of what it is solving, TBH.  It doesn’t define a new protocol, yet it makes claims about an architecture that implies a protocol between devices, a controller, and applications (see the figure in section 2).  In Section 3.2, iFIT is referenced as having an ability to cache or send accumulated data.  I don’t see how a framework can do this.  Nor do I see how a framework can dynamically load new data probes as mentioned in Section 3.3.  If this were a controller with an application architecture and specifications for component interoperability, perhaps, but I do not see that in this document.

In Section4, the document mentions a closed-loop for iFIT applications whereby applications can manage iFIT closed loops on top of a controller.  But again, I don’t see how.  Do the applications make API calls?  What calls do they make?  What makes it an “iFIT closed loop”?

Ultimately, the summary says that iFIT combines algorithmic and architectural schemes into the framework, but I don’t see where that is done in a specific, implementable way (e.g., in Section 3.1.2 you begin to describe how you can adaptively sample packets, but you talk about abstract signals to/from the controller).  Nor do I see how one would implement iFIT.  When I read the iFIT draft, I feel like I’m missing a normative chunk that explains how the various pieces of this framework are to interact in a well-specified manner.

It seems to me that perhaps a more useful document is one that focus on the implementation of PBT and/or IOAM, operational experiences, best practices, etc.

Joe

On Oct 21, 2019, at 14:34, Haoyu Song <[email-masked]<mailto:[email-masked]>> wrote:

Dear OPSAWG chairs,

Best regards,
Haoyu

---

From: Haoyu Song <[email-masked]>
Date: Wed, 23 Oct 2019 09:42:39 +0000

Hi Joe,

Thank you for the detailed comments!

Let me try to explain the purpose of this draft better. Given the recent on-path data plane telemetry techniques and standard works, in this draft we discuss the deployment challenges and potential opportunities for applications. There is no such a document in IETF AFAIK and we feel it’s needed (also confirmed by some network operators who are interested in such techniques)

Most related standard proposals so far only defines the data plane protocol and lack considerations for a complete solution. To this end, we discuss various points that a solution should pay attention to and how these can be composed to support applications. Along with the discussion, we provide some examples and use cases to trigger new ideas.

We deliberately make iFIT an open framework and avoid introducing any new protocol and enforcing any specific approaches, because otherwise we are in danger to put unnecessary constraints on implementation approaches and hurt the possibility of innovation. While we mean to keep this document informational, we may consider to add more discussions on reference designs, operational experiences,  and best practices as you suggested.

Some points you raised below also deserves more detailed explanation, such as how to make an iFIT closed loop and how architecture and algorithm components can be composed to form such a loop. Perhaps a complete example can help to explain that. I’ll consider all this in future revisions.

In a sense, this document indeed aims to discuss the implementation, operational experiences, and best practices  of PBT, IOAM, and other similar techniques. We hope this document will trigger new drafts on management plane/control plane and innovative solutions.

Best regards,
Haoyu

From: Joe Clarke (jclarke) <[email-masked]>
Sent: Tuesday, October 22, 2019 3:48 PM
To: Haoyu Song <[email-masked]>
Cc: [email-masked]; [email-masked]
Subject: Re: Call for Adoption "draft-song-opsawg-ifit-framework"

Joe

Dear OPSAWG chairs,

https://datatracker.ietf.org/doc/draft-song-opsawg-ifit-framework/<https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fdraft-song-opsawg-ifit-framework%2F&data=02%7C01%7Chaoyu.song%40futurewei.com%7Cda0fc97e527c49ed97ff08d75741e18d%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637073812783772433&sdata=rDJ4qcmsAD2IqXhKMGOh%2BEFnxFFdsxHHXGANBzcv0uI%3D&reserved=0>

Best regards,
Haoyu

---

From: "Frank Brockners (fbrockne)" <[email-masked]>
Date: Thu, 24 Oct 2019 16:41:46 +0000

Just took a read through the document as well – and I can echo Joe’s comments.
The scope of the document is not clear, nor does one understand what problem iFIT would address and solve.
That said, the document seems to have a very specific implementation in mind, as it refers to specific things such “iFIT Applications”, “iFIT Nodes”, etc. – but none of things are defined in the document.

Cheers, Frank

From: OPSAWG <[email-masked]> On Behalf Of Haoyu Song
Sent: Mittwoch, 23. Oktober 2019 11:43
To: Joe Clarke (jclarke) <[email-masked]>
Cc: [email-masked]; [email-masked]
Subject: Re: [OPSAWG] Call for Adoption "draft-song-opsawg-ifit-framework"

Hi Joe,

Best regards,
Haoyu

From: Joe Clarke (jclarke) <[email-masked]<mailto:[email-masked]>>
Sent: Tuesday, October 22, 2019 3:48 PM
To: Haoyu Song <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: Re: Call for Adoption "draft-song-opsawg-ifit-framework"

Joe

Dear OPSAWG chairs,

Best regards,
Haoyu

---

From: "Joe Clarke (jclarke)" <[email-masked]>
Date: Thu, 24 Oct 2019 22:21:22 +0000

On Oct 23, 2019, at 05:42, Haoyu Song <[email-masked]<mailto:[email-masked]>> wrote:

Hi Joe,

Tianran and I emailed on the chairs list, and I agree a deployment guide would be beneficial that shares practical experiences, operator considerations, and best practices.  However, the iFIT document does not read that way.  It reads more like a very specific architecture that is not fully fleshed out.  It leaves one (well, it leaves me) guessing as to what iFIT really is and how can I really be compliant to it.  What do I _really implement_ in this?

I haven’t participated in IPPM, so I can’t comment.  But in general, yes, having at least deployment considerations for things like IOAM and PBT would benefit operators.

Your comment about applications is one area where I find iFIT nebulous, though.  I don’t get a clear picture from the draft as to what an iFIT application is, and thus if I read this as a guide to help me implement, say, IOAM end-to-end, I’m still left wondering about specific things I may want to do in terms of where to export, what type of IOAM tracing to use, and what pitfalls to be aware of.

That said, you do cover things like a need for sampling as well as impact to devices.  Those are helpful but incomplete.  I’d love to get a more detailed understanding as to the impact I could expect from using IOAM and operating on packets versus the overhead incurred by something like PBT which still entails on-box processing.

It is this latter thing which I think is needed more.

Ultimately, I think what I’m saying is that it sounds like today we need a guide on practical deployment considerations around various OAM solutions, but we don’t yet need a noun to call such a thing.  I’d like to see such a document as I think it would be very useful outside opsawg and very useful to operators globally.

Joe

---

From: "qinfengwei" <[email-masked]>
Date: Fri, 25 Oct 2019 13:34:46 +0800

Hi Frank,

Speaking as the co-author, I think the abstraction described well the scope:

   In-situ Flow Information Telemetry (iFIT) provides a method for

   efficiently applying underlying on-path flow telemetry techniques,

   applicable across various network environments.

   This document outlines the iFIT framework, which enumerates several

   critical functional components and describes how these components are

   assembled together to achieve a complete and closed-loop working

   solution for on-path flow telemetry.

We, China Mobile, have deployed one IFIT data plane with many vendor supports.

https://datatracker.ietf.org/doc/draft-cheng-mpls-inband-pm-encapsulation/

We compared and chose the suitable data plane. We considered the forwarding performance. 

We also considered how to reduce the data export to the controller. And so on.

Data plane is not the only thing as in a complete solution.

We want to see this be generalized in a framework to achieve a complete and closed-loop working solution for on-path flow telemetry.

The document is not complete yet, some parts like section 3.6 and 4 are newly added into the draft according to the comments from the mailing list.

Your contributions are welcome.

Thanks,

Fengwei Qin

发件人: OPSAWG [mailto:[email-masked]] 代表 Frank Brockners (fbrockne)
发送时间: 2019年10月25日 00:42
收件人: Haoyu Song; Joe Clarke (jclarke)
抄送: [email-masked]; [email-masked]
主题: Re: [OPSAWG] Call for Adoption "draft-song-opsawg-ifit-framework"

Cheers, Frank

Hi Joe,

Best regards,

Haoyu

Joe

On Oct 21, 2019, at 14:34, Haoyu Song <[email-masked]> wrote:

Dear OPSAWG chairs,

 <https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fdraft-song-opsawg-ifit-framework%2F&data=02%7C01%7Chaoyu.song%40futurewei.com%7Cda0fc97e527c49ed97ff08d75741e18d%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637073812783772433&sdata=rDJ4qcmsAD2IqXhKMGOh%2BEFnxFFdsxHHXGANBzcv0uI%3D&reserved=0> https://datatracker.ietf.org/doc/draft-song-opsawg-ifit-framework/

Best regards,

Haoyu

---

From: "Frank Brockners (fbrockne)" <[email-masked]>
Date: Fri, 25 Oct 2019 14:02:58 +0000

Hi Fengwei Qin,

thanks a lot for the additional context. You confirm that “iFIT nodes” etc. are indeed new functions with data plane and control plane functionality that refer to a specific deployment that you’ve done. BTW/ - as a side note: It was surprising to me that despite your deployment seems to be based on draft-cheng-mpls-inband-pm-encapsulation – the draft-song-opsawg-ifit-framework does not even mention it.
We’re all aware that there are several proprietary solutions that resemble IOAM.  We started the IOAM effort several years ago in IETF to jointly arrive at a standard that allows for multivendor interoperable solutions.  Focusing on yet another variant of IOAM isn’t really moving the industry forward.

Rather than trying to write a generic document which seems to try to provide an overview of “what is out there” it would be much more helpful, if you could provide a detailed description of the issues and experiences (more specific and detailed than what you have in section 1 of draft-song-opsawg-ifit-framework) that you encountered in your deployment. Based on that understanding, we could jointly explore how to evolve the existing standardization effort on IOAM.

Thanks, Frank

From: qinfengwei <[email-masked]>
Sent: Freitag, 25. Oktober 2019 07:35
To: Frank Brockners (fbrockne) <[email-masked]>; 'Haoyu Song' <[email-masked]>; Joe Clarke (jclarke) <[email-masked]>
Cc: [email-masked]; [email-masked]
Subject: 答复: [OPSAWG] Call for Adoption "draft-song-opsawg-ifit-framework"

Hi Frank,

Thanks,
Fengwei Qin

发件人: OPSAWG [mailto:[email-masked]] 代表 Frank Brockners (fbrockne)
发送时间: 2019年10月25日 00:42
收件人: Haoyu Song; Joe Clarke (jclarke)
抄送: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
主题: Re: [OPSAWG] Call for Adoption "draft-song-opsawg-ifit-framework"

Cheers, Frank

From: OPSAWG <[email-masked]<mailto:[email-masked]>> On Behalf Of Haoyu Song
Sent: Mittwoch, 23. Oktober 2019 11:43
To: Joe Clarke (jclarke) <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: Re: [OPSAWG] Call for Adoption "draft-song-opsawg-ifit-framework"

Hi Joe,

Best regards,
Haoyu

Joe

Dear OPSAWG chairs,

Best regards,
Haoyu

---

From: Haoyu Song <[email-masked]>
Date: Fri, 25 Oct 2019 18:13:38 +0000

Hi Frank,

IOAM is one of the techniques covered by iFIT, although an important one. In the draft, we include many others (we may omit some but we’ll include them once we are aware of them), the reason for that is also discussed in the draft: we don’t expect one technique can meet all the data plane telemetry requirements. In the future, it’s very likely we’ll end up with multiple options, each with its own applicability and advantages.

Also, this draft doesn’t meant to summarize such techniques. Rather, it provides a solution framework which may apply one or more such underlying techniques. While we have listed all the issues and challenges we know so far, we do want to add more operational experiences as the draft progresses. To this end, we request collaborations and contributions from all the operators and vendors. We believe this will benefit the entire community.

Best,
Haoyu

From: Frank Brockners (fbrockne) <[email-masked]>
Sent: Friday, October 25, 2019 7:03 AM
To: qinfengwei <[email-masked]>; Haoyu Song <[email-masked]>; Joe Clarke (jclarke) <[email-masked]>
Cc: [email-masked]; [email-masked]
Subject: RE: [OPSAWG] Call for Adoption "draft-song-opsawg-ifit-framework"

Hi Fengwei Qin,

Thanks, Frank

From: qinfengwei <[email-masked]<mailto:[email-masked]>>
Sent: Freitag, 25. Oktober 2019 07:35
To: Frank Brockners (fbrockne) <[email-masked]<mailto:[email-masked]>>; 'Haoyu Song' <[email-masked]<mailto:[email-masked]>>; Joe Clarke (jclarke) <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: 答复: [OPSAWG] Call for Adoption "draft-song-opsawg-ifit-framework"

Hi Frank,

We, China Mobile, have deployed one IFIT data plane with many vendor supports.
https://datatracker.ietf.org/doc/draft-cheng-mpls-inband-pm-encapsulation/<https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fdraft-cheng-mpls-inband-pm-encapsulation%2F&data=02%7C01%7Chaoyu.song%40futurewei.com%7C1138aa61827241949d1a08d75954129d%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637076089965871536&sdata=IrxMLx0vMPvvXoOuRTdly2C0V%2FndlZDzzM8YP%2FKQFeE%3D&reserved=0>

Thanks,
Fengwei Qin

Cheers, Frank

Hi Joe,

Best regards,
Haoyu

Joe

Dear OPSAWG chairs,

https://datatracker.ietf.org/doc/draft-song-opsawg-ifit-framework/<https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fdraft-song-opsawg-ifit-framework%2F&data=02%7C01%7Chaoyu.song%40futurewei.com%7C1138aa61827241949d1a08d75954129d%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637076089965881506&sdata=dk6jjUHDXzRWMdXp3dPUn1U4DvTHY5I%2Bdj3YzwZwYF8%3D&reserved=0>

Best regards,
Haoyu

---

From: Haoyu Song <[email-masked]>
Date: Fri, 25 Oct 2019 18:25:19 +0000

Hi Frank,

The document specifically lists some issues and challenges when deploying the data plane on path telemetry techniques. Do you have any question on them and do you think anything is missing?
Also, I don’t agree that we have a very specific implementation in mind. Actually, we just discuss a very loose framework about it but we think a reasonable application would somehow follow such an architecture from high level. Do you find any unrealistic part or too specific part about this framework? Do think any specific topic should also be discussed in this draft?
We appreciate any feedbacks at this level of details which will help us substantially.
Thanks for pointing out some terms that we haven’t clearly defined. We’ll fix that in the next revision.

Best,
Haoyu

From: Frank Brockners (fbrockne) <[email-masked]>
Sent: Thursday, October 24, 2019 9:42 AM
To: Haoyu Song <[email-masked]>; Joe Clarke (jclarke) <[email-masked]>
Cc: [email-masked]; [email-masked]
Subject: RE: Call for Adoption "draft-song-opsawg-ifit-framework"

Cheers, Frank

Hi Joe,

Best regards,
Haoyu

Joe

Dear OPSAWG chairs,

https://datatracker.ietf.org/doc/draft-song-opsawg-ifit-framework/<https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fdraft-song-opsawg-ifit-framework%2F&data=02%7C01%7Chaoyu.song%40futurewei.com%7Cecdbd9f104324ec1b15708d758a11229%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637075321128476200&sdata=VCAh2aHIscuFb9Jxsg5UF53zAs7OUZxXuS5I6bvzKzg%3D&reserved=0>

Best regards,
Haoyu

---

From: "Frank Brockners (fbrockne)" <[email-masked]>
Date: Sun, 27 Oct 2019 11:20:09 +0000

Hi Haoyu,

Thanks. Per what I said below, neither do I understand what the document tries to solve nor do I understand the scope of the document.
The document introduces specific functions like iFIT nodes and iFIT applications but does not specify those. Fengwei Qin described that those indeed refer to a specific proprietary solution (i.e. draft-cheng-mpls-inband-pm-encapsulation): Is it this one https://www.huawei.com/en/press-events/news/2019/6/first-ifit-pilot-5g-transport-network-beijing-unicom-huawei?

Thanks, Frank

From: Haoyu Song <[email-masked]>
Sent: Freitag, 25. Oktober 2019 20:25
To: Frank Brockners (fbrockne) <[email-masked]>; Joe Clarke (jclarke) <[email-masked]>
Cc: [email-masked]; [email-masked]
Subject: RE: Call for Adoption "draft-song-opsawg-ifit-framework"

Hi Frank,

Best,
Haoyu

From: Frank Brockners (fbrockne) <[email-masked]<mailto:[email-masked]>>
Sent: Thursday, October 24, 2019 9:42 AM
To: Haoyu Song <[email-masked]<mailto:[email-masked]>>; Joe Clarke (jclarke) <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: RE: Call for Adoption "draft-song-opsawg-ifit-framework"

Cheers, Frank

Hi Joe,

Best regards,
Haoyu

Joe

Dear OPSAWG chairs,

Best regards,
Haoyu