Subject: Protocols, extensions, compatibility

From: "Adrien W. de Croy" <[email-masked]>
Date: Fri, 30 Mar 2012 23:10:43 +0000

Hi all.
  
There have been some questions raised about whether the best way 
forward is a new protocol, or patches to existing protocols etc.
  
I'm sure many of you are familiar with IMAP, and on the imap-ext 
mailing list recently there has also been discussion about exactly this.
  
IMAP like HTTP was conceived a long time ago.  At that time, people 
hadn't thought about all the things we wanted mail or web to do.
  
So as time went by, the desires for features left an ever-widening gap 
between what customers wanted developers to create to solve the issues 
they faced, and what protocols would support.
  
The classic response to this has been the definition of optional 
extensions to the core protocol, or frankly hacks.
  
This has happened in both HTTP and IMAP.
  
The situation as it stands with IMAP, is that any IMAP client 
implementor frankly has a number of nightmares on his/her hands.
  
Because the extensions were by and large optional, and not independent 
of each other (frequently RFCs for IMAP extensions alter other 
extension specs for the case where both exist), a client has to deal 
with an ever-increasing interdependency issue.  Clients need to be able 
to do the same task in a multitude of different ways to cope with 
different server option support.
  
This results in ever-increasing complexity, increased risk of 
implementation errors due to interdependencies, increased time to 
develop features, and generally reduced performance.  Development 
gridlock.
  
The same thankfully cannot be said for HTTP at the moment, although 
there are a few areas where dealing with optional support makes life 
difficult.
  
So, where am I going with this?
  
Eventually a requirement may come along which simply doesn't fit with 
the current protocol.  That's when it becomes interesting to consider 
starting afresh.
  
We may consider we aren't at this point with HTTP.  Personally, I think 
things like long-polling are an indication that HTTP isn't doing what 
we want, and currently doesn't support the semantics we want.

Adrien

---

From: Mike Belshe <[email-masked]>
Date: Sat, 31 Mar 2012 02:03:25 +0200

On Sat, Mar 31, 2012 at 1:10 AM, Adrien W. de Croy <[email-masked]> wrote:

>  Hi all.
>
> There have been some questions raised about whether the best way forward
> is a new protocol, or patches to existing protocols etc.
>
> I'm sure many of you are familiar with IMAP, and on the imap-ext mailing
> list recently there has also been discussion about exactly this.
>
> IMAP like HTTP was conceived a long time ago.  At that time, people hadn't
> thought about all the things we wanted mail or web to do.
>
> between what customers wanted developers to create to solve the issues they
> faced, and what protocols would support.
>
> The situation as it stands with IMAP, is that any IMAP client implementor
> frankly has a number of nightmares on his/her hands.
>
> Because the extensions were by and large optional, and not independent of
> each other (frequently RFCs for IMAP extensions alter other extension specs
> for the case where both exist), a client has to deal with
> an ever-increasing interdependency issue.  Clients need to be able to do
> the same task in a multitude of different ways to cope with different
> server option support.
>
> implementation errors due to interdependencies, increased time to develop
> features, and generally reduced performance.  Development gridlock.
>
> The same thankfully cannot be said for HTTP at the moment, although there
> are a few areas where dealing with optional support makes life difficult.
>
> Eventually a requirement may come along which simply doesn't fit with the
> current protocol.  That's when it becomes interesting to consider starting
> afresh.
>
> things like long-polling are an indication that HTTP isn't doing what we
> want, and currently doesn't support the semantics we want.
>

Patrick McManus did a nice presentation and discusses how 'HTTP has hit a
wall'.

http://www.ducksong.com/codebits11/

He makes some similar points, but discussing mostly at the connection mgmt
& protocol level.

Video is here:  https://codebits.eu/intra/s/session/220

Mike

---

From: Peter Lepeska <[email-masked]>
Date: Sat, 31 Mar 2012 12:44:45 -0400

Watching this video, Patrick McManus makes a great point about increasing
connections increases the likelihood that SYNs and SYN-ACKs will get
dropped. That's a very convincing argument for me AGAINST my suggestion to
simply increase the per host limit.

Thanks for sending the link,

Peter

On Fri, Mar 30, 2012 at 8:03 PM, Mike Belshe <[email-masked]> wrote:

>
>
> On Sat, Mar 31, 2012 at 1:10 AM, Adrien W. de Croy <[email-masked]>wrote:
>
>>  Hi all.
>>

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 31 Mar 2012 19:01:44 +0200

> Thanks for sending the link,

I agree with him on this too, and for having run tests with very large
per-host limits one year ago, I can say that these losses *do* happen.
What's worse is that they generally don't happen below a certain limit,
and that they happen by batches when you go beyond that limit. Of course,
this limit depends on the network between the two TCP endpoints and its
load and there is no real way to guess it.

Willy