Subject: [Nfvrg] Call for adoption of draft-natarajan-nfvrg-containers-for-nfv

From: "Diego R. Lopez" <[email-masked]>
Date: Wed, 4 Jan 2017 00:14:10 +0000

Hi,

This first message of the new year is to launch a two-week adoption call for draft-natarajan-nfvrg-containers-for-nfv. Ramki and I believe the document is mature enough to consider its adoption, once it has evolved from an analysis of container technology into a more comprehensive discussion of lightweight technologies in NFV.

Please indicate in your comments “support” or “no support” and discuss how this draft will contribute to the goals of NFVRG.

The current draft is available at:

https://datatracker.ietf.org/doc/draft-natarajan-nfvrg-containers-for-nfv/

Be goode,

--
"Esta vez no fallaremos, Doctor Infierno"

Dr Diego R. Lopez
Telefonica I+D
http://people.tid.es/diego.lopez/

e-mail: [email-masked]
Tel:    +34 913 129 041
Mobile: +34 682 051 091
----------------------------------

---

From: Azhar Sayeed <[email-masked]>
Date: Fri, 6 Jan 2017 11:10:57 -0500

Hi Diego and Authors..

Can you clarify if any data path acceleration techniques were used to measure throughput between guest and host OS. If not what is the usefulness of that metric - If the idea is to show raw comparisons then fine - if the idea is to show how bad the VMs are when compared to Unikernel and containers then you have achieved it it well..

The main issues with Unikernels or containers for NFV are not discussed in depth - Issues such as single threading support, IP address assignment and container networking need further exploration and study. Need at least statements in the document that those are for further study.

So perhaps I am missing the point of adoption of this draft - may be the objectives can be clarified.

Regards,
Azhar

> On Jan 3, 2017, at 7:14 PM, Diego R. Lopez <[email-masked]> wrote:
> 
> Hi,
> 
> _______________________________________________
> Nfvrg mailing list
> [email-masked]
> https://www.irtf.org/mailman/listinfo/nfvrg

---

From: "ram krishnan" <[email-masked]>
Date: Fri, 6 Jan 2017 12:47:23 -0800

Hi Azhar,

Thanks for the comments and the interest. The rationale behind adoption is we see strong community interest in the topic and have reasonable content in the document. We still have more steps like last call before the RFC publication and are expecting good community contribution to the document prior to that.

For the performance comparisons, we didn't use HW acceleration techniques since they are application and deployment specific; for example, a small CPE in an enterprise branch may never use any hardware acceleration because of the low throughput requirements. Any specific suggestions including references in this area are most welcome.

Container networking is definitely an interesting topic. We will certainly capture the challenges in a mixed Container/OpenStack environment, how efforts like Kuryr are attempting to address these and how SR-IOV plays out in this scenario. Any other suggestions are welcome.

Can you please elaborate more on the single threading support? 

Thanks,

Ramki

-----Original Message-----
From: Nfvrg [mailto:[email-masked]] On Behalf Of Azhar Sayeed
Sent: Friday, January 6, 2017 8:11 AM
To: Diego R. Lopez <[email-masked]>
Cc: [email-masked]
Subject: Re: [Nfvrg] Call for adoption of draft-natarajan-nfvrg-containers-for-nfv

Hi Diego and Authors..

Regards,

Azhar

> On Jan 3, 2017, at 7:14 PM, Diego R. Lopez < <mailto:[email-masked]> [email-masked]> wrote:

>  <https://datatracker.ietf.org/doc/draft-natarajan-nfvrg-containers-for-nfv/> https://datatracker.ietf.org/doc/draft-natarajan-nfvrg-containers-for-nfv/

>  <http://people.tid.es/diego.lopez/> http://people.tid.es/diego.lopez/

> e-mail:  <mailto:[email-masked]> [email-masked]

>  <mailto:[email-masked]> [email-masked]

>  <https://www.irtf.org/mailman/listinfo/nfvrg> https://www.irtf.org/mailman/listinfo/nfvrg

Nfvrg mailing list

 <mailto:[email-masked]> [email-masked]

 <https://www.irtf.org/mailman/listinfo/nfvrg> https://www.irtf.org/mailman/listinfo/nfvrg

---

From: "ram krishnan" <[email-masked]>
Date: Fri, 6 Jan 2017 13:23:47 -0800

Hi Roberto,

Valid point. Do you have any specific examples in mind?

Thanks,

Ramki

From: Roberto Riggio [mailto:[email-masked]] 
Sent: Friday, January 6, 2017 12:59 PM
To: ram krishnan <[email-masked]>
Cc: Azhar Sayeed <[email-masked]>; Diego R. Lopez <[email-masked]>; [email-masked]
Subject: Re: [Nfvrg] Call for adoption of draft-natarajan-nfvrg-containers-for-nfv

Hi,

I was wondering if the lightweight NFV performance comparison should also be extended

to the rest of the stack. For example openstack + opnfv could be very heavyweight while 

other management platforms for containers (if they exists) could be executed on low 

power platforms (which could make sense in some deployments).

R.

On Fri, Jan 6, 2017 at 9:47 PM, ram krishnan <[email-masked] <mailto:[email-masked]> > wrote:

Hi Azhar,

Thanks,

Ramki

-----Original Message-----
From: Nfvrg [mailto:[email-masked] <mailto:[email-masked]> ] On Behalf Of Azhar Sayeed
Sent: Friday, January 6, 2017 8:11 AM
To: Diego R. Lopez <[email-masked] <mailto:[email-masked]> >
Cc: [email-masked] <mailto:[email-masked]> 
Subject: Re: [Nfvrg] Call for adoption of draft-natarajan-nfvrg-containers-for-nfv

Hi Diego and Authors..

Regards,

Azhar

> Tel:    +34 913 129 041 <tel:+34%20913%2012%2090%2041> 

> Mobile: +34 682 051 091 <tel:+34%20682%2005%2010%2091> 

Nfvrg mailing list

_______________________________________________
Nfvrg mailing list
[email-masked] <mailto:[email-masked]> 
https://www.irtf.org/mailman/listinfo/nfvrg

-- 

--------------------------------------------------------

Roberto Riggio, Ph.D.

CREATE-NET

Chief Scientist 

Future Networks (FuN)

Via alla Cascata 56/D - 38123 Povo Trento (Italy)

e-mail: [email-masked] <mailto:[email-masked]>  <- NEW EMAIL ADDRESS

office: (+39) 0461 31 24 81

Fax: (+39) 0461 42 11 57

mobile: (+39) 338 72 93 203

skype: hamvil

homepage: http://www.robertoriggio.net/

--------------------------------------------------------

---

From: Roberto Riggio <[email-masked]>
Date: Fri, 6 Jan 2017 21:58:49 +0100

Hi,

I was wondering if the lightweight NFV performance comparison should also
be extended
to the rest of the stack. For example openstack + opnfv could be very
heavyweight while
other management platforms for containers (if they exists) could be
executed on low
power platforms (which could make sense in some deployments).

R.

On Fri, Jan 6, 2017 at 9:47 PM, ram krishnan <[email-masked]> wrote:

> Hi Azhar,
>
>
>
> Thanks for the comments and the interest. The rationale behind adoption is
> we see strong community interest in the topic and have reasonable content
> in the document. We still have more steps like last call before the RFC
> publication and are expecting good community contribution to the document
> prior to that.
>
>
>
> For the performance comparisons, we didn't use HW acceleration techniques
> since they are application and deployment specific; for example, a small
> CPE in an enterprise branch may never use any hardware acceleration because
> of the low throughput requirements. Any specific suggestions including
> references in this area are most welcome.
>
>
>
> Container networking is definitely an interesting topic. We will certainly
> capture the challenges in a mixed Container/OpenStack environment, how
> efforts like Kuryr are attempting to address these and how SR-IOV plays out
> in this scenario. Any other suggestions are welcome.
>
>
>
> Can you clarify if any data path acceleration techniques were used to
> measure throughput between guest and host OS. If not what is the usefulness
> of that metric - If the idea is to show raw comparisons then fine - if the
> idea is to show how bad the VMs are when compared to Unikernel and
> containers then you have achieved it it well..
>
>
>
> The main issues with Unikernels or containers for NFV are not discussed in
> depth - Issues such as single threading support, IP address assignment and
> container networking need further exploration and study. Need at least
> statements in the document that those are for further study.
>
>
>
> > This first message of the new year is to launch a two-week adoption call
> for draft-natarajan-nfvrg-containers-for-nfv. Ramki and I believe the
> document is mature enough to consider its adoption, once it has evolved
> from an analysis of container technology into a more comprehensive
> discussion of lightweight technologies in NFV.
>
> >
>
> > Please indicate in your comments “support” or “no support” and discuss
> how this draft will contribute to the goals of NFVRG.
>
> >
>
> > Tel:    +34 913 129 041 <+34%20913%2012%2090%2041>
>
> > Mobile: +34 682 051 091 <+34%20682%2005%2010%2091>
>

e-mail: [email-masked] <- NEW EMAIL ADDRESS

---

From: Roberto Riggio <[email-masked]>
Date: Sat, 7 Jan 2017 11:52:11 +0100

I was thinking about frameworks like dockyard. There are many other tools
(more or less complex)
to manage containers without going to full blown VIMs like openstack.
However what is missing is
a Lightweight MANO to replace platforms like OPNFV etc. il the low-end side
of the spectrum. We
are actually working on this but it is still very far from a public release.

R.

On Fri, Jan 6, 2017 at 10:23 PM, ram krishnan <[email-masked]> wrote:

> *From:* Roberto Riggio [mailto:[email-masked]]
> *Sent:* Friday, January 6, 2017 12:59 PM
> *To:* ram krishnan <[email-masked]>
> *Cc:* Azhar Sayeed <[email-masked]>; Diego R. Lopez <
> [email-masked]>; [email-masked]
>
> *Subject:* Re: [Nfvrg] Call for adoption of draft-natarajan-nfvrg-
> containers-for-nfv
>
>
>
> Hi,
>
>
>

---

From: "Diego R. Lopez" <[email-masked]>
Date: Mon, 9 Jan 2017 08:33:56 +0000

Hi Roberto,

I would be very much interested in knowing more about this approach. At the last OpenStack Summit in Barcelona I had the opportunity to discuss the approach of a lightweight VIM (or even a general lightweight cloud orchestration) for use cases like the one that you mention. We believe the current OpenVIM inside OpenMANO is a clear example of it.

Be goode,

On 7 Jan 2017, at 11:52 , Roberto Riggio <[email-masked]<mailto:[email-masked]>> wrote:

I was thinking about frameworks like dockyard. There are many other tools (more or less complex)
to manage containers without going to full blown VIMs like openstack. However what is missing is
a Lightweight MANO to replace platforms like OPNFV etc. il the low-end side of the spectrum. We
are actually working on this but it is still very far from a public release.

R.

On Fri, Jan 6, 2017 at 10:23 PM, ram krishnan <[email-masked]<mailto:[email-masked]>> wrote:
Hi Roberto,

Thanks,
Ramki

From: Roberto Riggio [mailto:[email-masked]<mailto:[email-masked]>]
Sent: Friday, January 6, 2017 12:59 PM
To: ram krishnan <[email-masked]<mailto:[email-masked]>>
Cc: Azhar Sayeed <[email-masked]<mailto:[email-masked]>>; Diego R. Lopez <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>

Hi,

R.

On Fri, Jan 6, 2017 at 9:47 PM, ram krishnan <[email-masked]<mailto:[email-masked]>> wrote:

Hi Azhar,

Thanks,

Ramki

-----Original Message-----
From: Nfvrg [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Azhar Sayeed
Sent: Friday, January 6, 2017 8:11 AM
To: Diego R. Lopez <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>
Subject: Re: [Nfvrg] Call for adoption of draft-natarajan-nfvrg-containers-for-nfv

Hi Diego and Authors..

Regards,

Azhar

> On Jan 3, 2017, at 7:14 PM, Diego R. Lopez <[email-masked]<mailto:[email-masked]>> wrote:

> e-mail: [email-masked]<mailto:[email-masked]>

> Tel:    +34 913 129 041<tel:+34%20913%2012%2090%2041>

> Mobile: +34 682 051 091<tel:+34%20682%2005%2010%2091>

> [email-masked]<mailto:[email-masked]>

Nfvrg mailing list

[email-masked]<mailto:[email-masked]>

_______________________________________________
Nfvrg mailing list
[email-masked]<mailto:[email-masked]>
https://www.irtf.org/mailman/listinfo/nfvrg

e-mail: [email-masked]<mailto:[email-masked]> <- NEW EMAIL ADDRESS

e-mail: [email-masked]<mailto:[email-masked]> <- NEW EMAIL ADDRESS

Dr Diego R. Lopez
Telefonica I+D

---

From: <[email-masked]>
Date: Mon, 9 Jan 2017 09:16:51 +0000

I think it would be interesting to compare the contents of this document with the contents of the ETSI one to understand how they relate to each other and add a clause to highlight/discuss major differences, if any...
http://www.etsi.org/deliver/etsi_gs/NFV-EVE/001_099/004/01.01.01_60/gs_nfv-eve004v010101p.pdf
Bruno

-----Message d'origine-----
De : Nfvrg [mailto:[email-masked]] De la part de Diego R. Lopez
Envoyé : mercredi 4 janvier 2017 01:14
À : [email-masked]
Objet : [Nfvrg] Call for adoption of draft-natarajan-nfvrg-containers-for-nfv

Hi,

Be goode,

Dr Diego R. Lopez
Telefonica I+D

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: "Diego R. Lopez" <[email-masked]>
Date: Mon, 9 Jan 2017 09:32:12 +0000

Hi Bruno,

I agree putting the document in the framework of EVE004 would be interesting. The draft is focused on performance aspects, as a detailed analysis of what is mentioned in Section 6.2 of EVE004.

Be goode,

On 9 Jan 2017, at 10:16 , [email-masked]<mailto:[email-masked]> <[email-masked]<mailto:[email-masked]>> wrote:

-----Message d'origine-----
De : Nfvrg [mailto:[email-masked]] De la part de Diego R. Lopez
Envoyé : mercredi 4 janvier 2017 01:14
À : [email-masked]
Objet : [Nfvrg] Call for adoption of draft-natarajan-nfvrg-containers-for-nfv

Hi,

Be goode,

Dr Diego R. Lopez
Telefonica I+D

Dr Diego R. Lopez
Telefonica I+D

---

From: Kostas Pentikousis <[email-masked]>
Date: Mon, 9 Jan 2017 13:19:28 +0100 (CET)

Hi Diego, all,

| The current draft is available at:
| 
| https://datatracker.ietf.org/doc/draft-natarajan-nfvrg-containers-for-nfv/

It seems that the draft expired at the time of this review. I had a cursory look and I have a few comments on the expired version -03.

First, it would be good to be more explicit about the scope, goal and applicability of the draft. 

Second, the draft does not cover salient research papers in this very active area. In fact, the first reference is to a wikipedia article; only one peer-reviewed paper is cited; most references are web pages. This should be significantly improved; see also RFC 5743, sec. 2.1.

Up to section 4, we only read text with high-level challenges (e.g. "When a resource component is compromised, quarantine the compromised entity but ensure service continuity for other resources") or requirements (e.g. "a management solution ... Is secure"). The conclusion explicitly says that the draft "presented the challenges when building an NFV platform", thus I'm not sure why the draft title includes the word "analysis".

Section 4 reports on a particular set of results. Was there any consultation with bmwg, for instance? I think I mentioned it in the last meeting in Berlin that some more information regarding the experiments is needed. It's not clear how many samples do you have, or what do the reported numbers represent (first run, mean, median, ...), what is the impact of the platform and the respective components, or what new insights we get from this particular set of experiments compared to other work. In general, I think sec. 4 is more suitable for a peer-reviewed workshop paper to be cited by the draft.

Finally, the conclusion is not particularly enlightening: "We conclude that choosing a solution is nuanced, and depends on how much value different NFV operators place on criteria such as strong isolation, performance and compatibility with applications and management frameworks." I would appreciate something more specific.

A few nits as well: On the header (p. 2 ff.), the draft is dated "September 2015", not as on the first page (July 8, 2016). Indentation on 3rd level headings looks strange, ditto for bullet style (cf. lists in sec. 3.1.1 vs. 3.2.1, and the remainder of the document). I'm not sure why there's a need for a single 3rd level section in both cases; ditto for 3.3.1 and 3.4.1.

To sum up, I think more work is needed prior to RG adoption.

Best regards,

Kostas

---

From: Ram Krishnan <[email-masked]>
Date: Mon, 9 Jan 2017 07:02:25 -0800

Hi Roberto,

I echo Diego's thoughts. Would certainly be interested in knowing more.

​Thanks,
Ramki​

On Mon, Jan 9, 2017 at 12:33 AM, Diego R. Lopez <
[email-masked]> wrote:

> Hi Roberto,
>
> I would be very much interested in knowing more about this approach. At
> the last OpenStack Summit in Barcelona I had the opportunity to discuss the
> approach of a lightweight VIM (or even a general lightweight cloud
> orchestration) for use cases like the one that you mention. We believe the
> current OpenVIM inside OpenMANO is a clear example of it.
>
> Be goode,
>
> On 7 Jan 2017, at 11:52 , Roberto Riggio <[email-masked]> wrote:
>
>> *Subject:* Re: [Nfvrg] Call for adoption of
>> draft-natarajan-nfvrg-containers-for-nfv
>>
>>
>>
>> Hi,
>>
>>
>>
>> Thanks for the comments and the interest. The rationale behind adoption
>> is we see strong community interest in the topic and have reasonable
>> content in the document. We still have more steps like last call before the
>> RFC publication and are expecting good community contribution to the
>> document prior to that.
>>
>>
>>
>> Container networking is definitely an interesting topic. We will
>> certainly capture the challenges in a mixed Container/OpenStack
>> environment, how efforts like Kuryr are attempting to address these and how
>> SR-IOV plays out in this scenario. Any other suggestions are welcome.
>>
>>
>>
>> The main issues with Unikernels or containers for NFV are not discussed
>> in depth - Issues such as single threading support, IP address assignment
>> and container networking need further exploration and study. Need at least
>> > This first message of the new year is to launch a two-week adoption
>> call for draft-natarajan-nfvrg-containers-for-nfv. Ramki and I believe
>> the document is mature enough to consider its adoption, once it has evolved

-- 
Thanks,
Ramki

---

From: Roberto Riggio <[email-masked]>
Date: Mon, 9 Jan 2017 16:24:32 +0100

Hi,

actually we are bumping into this aspect continuously while working on NFV
and
in particular on MEC scenario. Essentially, echoing what Feynman said in
relation
to nanotechnology, it seems that there "is plenty of space at the bottom"
of the
NFV domain.

In particular we had several doubts regarding whatever it makes sense to use
an openstack+an ETSI mano framework to orchestrate edge workloads where
you have low footprint whiteboxes. Also in some cases the VNF that you end
up
orchestrating and pretty different from your regular VNF. E.g. they could
be part
of a portion of a RAT stack running as a VNF (see the small cell forum work
on
functional split).

What we are doing is to develop a VIM+MANO framework that could orchestrate
containers and clickOS instances. At the moment we have only basic
orchestration
for clickOS based VNFs (with support for stateful migration). Significant
work is
to be done in order to address also the most popular container
technologies. The
click OS orchestrator is already available on github.

Given the scope of this document what is really interesting to me is to
understand
what would be the KPIs to be considered in order to motivate such a
lightMANO
(virtualization/orchestration overhead? baseline resources requirement just
to bootstrap
the system? ...)

R.

On Mon, Jan 9, 2017 at 4:02 PM, Ram Krishnan <[email-masked]> wrote:

>>> For the performance comparisons, we didn't use HW acceleration
>>> techniques since they are application and deployment specific; for example,
>>> a small CPE in an enterprise branch may never use any hardware acceleration
>>> because of the low throughput requirements. Any specific suggestions
>>> including references in this area are most welcome.
>>>
>>>
>>>

---

From: "Diego R. Lopez" <[email-masked]>
Date: Mon, 30 Jan 2017 19:39:00 +0000

Hi,

After checking the expressions on the draft and the comments received, we think we cannot declare consensus on adoption of this draft yet.

We’d like to request the authors to address the comments received so far, and come back with a new version as soon as they see fit, as this work seems to us of high interest to the community.

Be goode,

On 4 Jan 2017, at 01:14 , DIEGO LOPEZ GARCIA <[email-masked]<mailto:[email-masked]>> wrote:

Hi,

Be goode,

Dr Diego R. Lopez
Telefonica I+D

Dr Diego R. Lopez
Telefonica I+D