Subject: Re: [AAA-DOCTORS] [OPS-DIR] FW: Evaluation: draft-thomson-beep-async-02.txt to Proposed Standard

From: "Ersue, Mehmet (NSN - DE/Munich)" <[email-masked]>
Date: Wed, 6 May 2009 19:18:53 +0200

Dear Martin,

> Sent: Tuesday, May 05, 2009 1:50 AM
> To: Romascanu, Dan (Dan)
> Cc: Alexey Melnikov; Ersue, Mehmet (NSN - DE/Munich)
> Subject: RE: [OPS-DIR] FW: Evaluation: 
> draft-thomson-beep-async-02.txt toProposed Standard
> 
> Thanks for forwarding this on Dan.
> 
> I'll deal with the smaller comments below, but there are two 
> concerns that I'd like to address up front.
> 
> ==Use Case
> 
> The draft is quite clear on the use case - large volumes of 
> requests that take a significant amount of time.  I don't 
> think that the draft could be any clearer without numbers.
> 
> If numbers are necessary for justification, imagine 1000 
> requests per second with a 50+ second processing time.  We've 
> looked at the same problem for HTTP and it starts pushing the 
> limits of what a single machine can handle.  Even if the 
> machine could handle the actual processing load, 50,000 
> connections stresses most TCP stacks.
> 
> A single BEEP connection might handle this, but no BEEP stack 
> is required to support 50,000 channels (the limit is 257), 
> which is what would be required.  Mehmet asserts that BEEP 
> concurrency is adequate, but this is demonstrably not true in 
> this case.

I don't claim that this functionality is not useful. What I mean 
is that the existing features might be already sufficient to solve 
the same problem space and a new feature might not be necessary 
as an "additional mandatory standard". 

I would like to have the explanation and reasoning why this is a 
required feature and why the existing features are not able to solve 
the problem "as text in the document", IOW text proving that it is 
a necessity and not nice to have.

I don't have anything against this feature if it gets published as an
Informational RFC. 

However, the document is meant to become a BEEP-bis standard
and I don't agree to publish it as a mandatory standard, without 
the decision of a WG to extend an existing standard, without the 
thorough discussion and development of a WG and without the 
consensus reached in an IETF WG maillist, which is IMO absolutely 
necessary if we want to extend an existing "IETF standard" with 
an additional "mandatory standard feature".

Threads in the BEEP maillist show that some of the implementers 
of the BEEP standard think this feature is not needed. At least I saw 
long mails from Sam Roberts and David Kramer, which you obviously 
couldn't convince for the importance of your draft.
PS: AFAIU there have been 4 persons commenting on your draft on 
the BEEP maillist on  www.beepcore.org (except Rainer Gerhards 
who had a different concern). 50% of them were in favor and 50% against.

I definitely don't see the necessary consensus in this maillist for 
the publication of an BEEP-bis standard. 

> ==Configurability
> 
> Mehmet requests that the document define (?) a configuration 
> parameter to limit the accumulation of state on the client 
> peer.  While this is not explicit in the document, I don't 
> believe that it is necessary to be any more explicit than the 
> security section is:
> 
>       ...; the number of outstanding requests can be
>    restricted to protect against excessive accumulation of state.
> 
> Because this aspect is internal to the client peer, BEEP 
> stacks are able to address this how they choose.

I believe the short text on "state information" in the security 
considerations section is insufficient for the necessary scalability 
discussion in a standard-track document because of the reasons 
I indicated.

See more below . . .

Mehmet

> More inline below...
> 
> > -----Original Message-----
> > From: Romascanu, Dan (Dan) [mailto:[email-masked]]
> > Sent: Monday, 4 May 2009 9:11 PM
> > To: Thomson, Martin
> > Cc: Alexey Melnikov; Ersue, Mehmet (NSN - DE/Munich)
> > Subject: FW: [OPS-DIR] FW: Evaluation: 
> draft-thomson-beep-async-02.txt
> > toProposed Standard
> > 
> >  Martin,
> > 
> > Please find below the OPS-DIR review of draft-thomson-beep-async-
> > 02.txt.
> > 
> > 
> > Regards,
> > 
> > Dan
> > 
> > -----Original Message-----
> > From: Ersue, Mehmet (NSN - DE/Munich) [mailto:[email-masked]]
> > Sent: Sunday, May 03, 2009 10:47 PM
> > To: Romascanu, Dan (Dan)
> > Cc: [email-masked]; [email-masked]; [email-masked]
> > Subject: RE: [OPS-DIR] FW: Evaluation: 
> draft-thomson-beep-async-02.txt
> > toProposed Standard
> > 
> > 
> > Hi Dan,
> > 
> > I reviewed the document draft-thomson-beep-async-02.txt in 
> general and
> > for its operational impact.
> > 
> > Summary:
> > The Blocks Extensible Exchange Protocol (BEEP) provides a protocol
> > framework for the development of application protocols.  
> This document
> > describes an BEEP feature that enables asynchrony for individual
> > channels.
> > 
> > 
> > Review summary:
> > 
> > The draft is an extension feature to the BEEP (Blocks Extensible
> > Exchange Protocol) protocol.
> > BEEP provides already asnychrony based on pipelining (intra-channel)
> > and
> > parallelism (inter-channel). This documents provides a feature as an
> > extension to BEEP, that enables the creation of an asynchronous
> > channel,
> > which is not limited to pipelining. I.e. a serving peer can process
> > requests and respond in any order.
> > 
> > The document aims to become a proposed standard RFC but there is no
> > discussion on why an asynchronous channel is really needed 
> additional
> > to
> > the already existing asynchrony mechanisms in the original BEEP
> > protocol, IOW why are the existing mechanisms are not 
> sufficient. The
> > documents claims the necessity but there are also no examples of
> > applications which would need the described mechanism.
> > 
> > Although the document introduces additional state information which
> > needs to be stored and managed it does not discuss scalability
> > sufficiently. The document also does not introduce a configurable
> > parameter for the 'maximum count of allowed parallel asynchronous
> > channels'.
> > Also the document misses to disscuss what happens when a 
> response does
> > not follow a request, i.e. is missing the exception handling
> > thereafter.
> > 
> > Configurability in general has not been discussed in the document.
> > Performance considerations have not been discussed in the document.
> > 
> > The reviewer couldn't find the mail archive at the URL given in the
> > concluded BEEP WG charter page. The current maillist seems 
> to be linked
> > on www.beepcore.org and the "asynchronous BEEP" thread is under:
> > 
> http://groups.google.com/group/beepwg/browse_thread/thread/db0
> 255c634c7
> > e
> > b0c/982e385e39fef215?lnk=gst&q=+Asynchronous+BEEP#982e385e39fef215
> > 
> > The document shepherding info states that are a few seconders on the
> > maillist. The reviewer of the document found a lot of mails 
> questioning
> > the sense of such an extension to the BEEP protocol (see URL above,
> > e.g.
> > "Anyways, reading through the draft, I think one reason I 
> am resistant
> > to this feature is that I do not fully understand when it 
> would better
> > than the alternatives.").
> 
> These concerns were expressed by one or two on the mailing 
> list.  The use case was not the problem, only the manner in 
> which it was solved.
> 
> The objectors held that the problem should be solved by the 
> application protocol, not BEEP.  By adding an identifier to 
> application-layer messages, requests and responses could be 
> sent in MSG frames.  
> 
> Aside from doubling the number of messages, this places a 
> burden on the application protocol.  Consider RFC 4227 
> (BEEP+SOAP) and whether it would be appropriate to modify the 
> payload to support a particular protocol binding.
> 
> > Review Questions:
> > 
> > Is the document readable?
> > 
> > 	Yes.
> > 
> > Does it contain nits?
> > 
> >       Yes (see below).
> > 
> >   Miscellaneous warnings:
> > 
> > 
> --------------------------------------------------------------
> ---------
> > -
> > ----
> > 
> >   == The document doesn't use any RFC 2119 keywords, yet 
> seems to have
> > RFC
> >      2119 boilerplate text.
> 
> This is consistent with the source material (RFC 3080).  It's 
> a small thing, but I see no strong reason to make a change.

I don't agree. Standard documents should use the RFC 2119 
KEYWORDS in a specific way and should highlight the 
KEYWORDS for clarity.

> >   == The document seems to lack a disclaimer for 
> pre-RFC5378 work, but
> > was
> >      first submitted before 10 November 2008.  Should you add the
> > disclaimer?
> >      (See the Legal Provisions document at
> >      http://trustee.ietf.org/license-info for more information.).
> 
> The disclaimer is not necessary.  I am able to answer for all 
> of the contributors.

I'm not really sure whether the disclaimer is not necessary. The 
BEEP standard is used as a basic input for the document. Actually 
this is going to be a BEEP-bis document.

> > Is the document class appropriate?
> > 
> >       No. I would have nothing against such an extension if 
> it would be
> > Informational. I do not think this is a really necessary and
> > long-awaited standard feature.
> 
> The comments above should address this and the consequent concerns.
>  
> > Is the problem well stated?
> > 
> >       The mechanism has been described to some extent but exception
> > handling and scalability discussion is not sufficient.
> > 
> > Is the problem really a problem?
> > 
> >       The reviewer is not sure whether this feature is 
> really needed as
> > a standard. The necessity of the feature has been also questioned on
> > the
> > maillist.
> > 
> > Does the document consider existing solutions?
> > 
> >      Obviously, there are a few products using BEEP
> > (http://www.beepcore.org/products.html). The reviewer would 
> like to see
> > some use cases asking for the mechanism and requirements from BEEP
> > users.
> > 
> > Does the solution break existing technology?
> > 
> >      No.
> > 
> > Does the solution preclude future activity?
> > 
> >      No.
> > 
> > Is the solution sufficiently configurable?
> > 
> >     Configuration parameters are not discussed in the document.
> >     Additional parameters supporting scalability would be 
> interesting
> > to
> > have.
> >
> > Can performance be measured?  How?
> > 
> >     Performance considerations have not been discussed in 
> the document.
> > 
> > Does the solution scale well?
> > 
> >     A configurable parameter for the 'maximum count of 
> allowed parallel
> > asynchronous channels' would support scalability.
> > 
> > Cheers,
> > Mehmet
> > 
> 
> --------------------------------------------------------------
> ----------------------------------
> This message is for the designated recipient only and may
> contain privileged, proprietary, or otherwise private information.  
> If you have received it in error, please notify the sender
> immediately and delete the original.  Any unauthorized use of
> this email is prohibited.
> --------------------------------------------------------------
> ----------------------------------
> [mf2]
>

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 06 May 2009 18:55:17 +0100

>Dear Martin,
Dear Mehmet,

>a necessity and not nice to have.
Noted. I will add an RFC Editor note for this, or the author will revise 
the document to add such text.

>and I don't agree to publish it as a mandatory standard,
>
This feature is not mandatory - it is negotiated by the client. So I 
don't understand why you keep calling this BEEP-bis and "mandatory 
standard".

>without 
 [snip]

-- 
IETF Application Area Director, <http://www.ietf.org/IESGmems.html>
Internet Messaging Team Lead, <http://www.isode.com>
JID: same as my email address

---

From: Lisa Dusseault <[email-masked]>
Date: Wed, 6 May 2009 11:56:37 -0700

I agree with Mehmet on the general perceptions here.  When we publish
an Internet Standard that extends another Internet Standard, we are
perceived as saying that the later RFC is the one people should
implement.  The negotiation can be for backwards compatibility, rather
than for long-term co-existence with original-BEEP implementations.

I also agree that extending an Internet Standard on the Standards
Track should demonstrate a higher level of consensus.

Lisa

On Wed, May 6, 2009 at 10:55 AM, Alexey Melnikov
<[email-masked]> wrote:
>>> Subject: RE: [OPS-DIR] FW: Evaluation: draft-thomson-beep-async-02.txt
>>> toProposed Standard
>>>
>>> I'll deal with the smaller comments below, but there are two concerns
>>> that I'd like to address up front.
>>>
>>> ==Use Case
>>>
>>> The draft is quite clear on the use case - large volumes of requests that
>>> take a significant amount of time.  I don't think that the draft could be
>>> any clearer without numbers.
>>>
>>> If numbers are necessary for justification, imagine 1000 requests per
>>> second with a 50+ second processing time.  We've looked at the same problem
>>> for HTTP and it starts pushing the limits of what a single machine can
>>> handle.  Even if the machine could handle the actual processing load, 50,000
>>> A single BEEP connection might handle this, but no BEEP stack is required
>>> to support 50,000 channels (the limit is 257), which is what would be
>>> required.  Mehmet asserts that BEEP concurrency is adequate, but this is
>>> demonstrably not true in this case.
>>>
>>
>> I don't claim that this functionality is not useful. What I mean is that
>> the existing features might be already sufficient to solve the same problem
>> space and a new feature might not be necessary as an "additional mandatory
>> standard".
>> I would like to have the explanation and reasoning why this is a required
>> feature and why the existing features are not able to solve the problem "as
>> text in the document", IOW text proving that it is a necessity and not nice
>> to have.
>>
>
> Noted. I will add an RFC Editor note for this, or the author will revise the
> document to add such text.
>
> This feature is not mandatory - it is negotiated by the client. So I don't
> understand why you keep calling this BEEP-bis and "mandatory standard".
>
>> without the decision of a WG to extend an existing standard, without the
>> thorough discussion and development of a WG and without the consensus
>> reached in an IETF WG maillist, which is IMO absolutely necessary if we want
>> to extend an existing "IETF standard" with an additional "mandatory standard
>> feature".
>>
>
> [snip]
>
> --