Subject: RE: CONNECT message including tunneled data

From: Henrik Nordström <[email-masked]>
Date: Fri, 01 Feb 2008 15:17:25 +0100

fre 2008-02-01 klockan 16:35 +1300 skrev Adrien de Croy:
>  
> > POST :-)
> 
> back and forth and back and forth and back and forth etc etc with a
> single command and response?

Yes, if the intermediaries support forwarding of the response while
still sending the request.

If not then two requests is used, one POST and one GET, and application
logics to bind the two connections together at the application.

> I would have thought it would be easier to use CONNECT :)

True. But the split POST/GET approach do provide some transport benefits
which may be useful..

> > OH, I'm not advocating changing CONNECT itself.  The method name is
> > hard-coded into every proxy; the semantics cannot be changed.  Any new
> > strategy would need to use a new method name, at least.
> 
> OK, which begs the obvious question - does this belong in HTTP?  We
> could re-implement SOCKS5 over HTTP, but why would we?

Agreed. SOCKS5(+) is already there, and  with support in very many
clients and covering the area of generic proxying pretty well.

> Sounds like you're proposing a fairly radical departure to the current
> target goal of HTTP.  Whilst I'd love to see HTTP move to a more
> state-driven multi-transactional multiplexed command protocol, it would
> no longer be or bear any resemblance to HTTP.  You'd need to assign a
> new port number for it :)

Or run it over SCTP.

> Some of these discussions should be kept in mind when looking to design
> HTTP 2.0

Agreed.

Regards
Henrik

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 1 Feb 2008 18:17:06 +0000

Adrien de Croy wrote:
> > > What other HTTP method allows you to send any amount of data back
> > > and forth not delineated into HTTP messages?
> > 
> > POST :-)
> 

Yes, when supported by intermediaries (i.e. often no).

> > However, since overlapping request/response doesn't work with many
> > agents, it is more common to issue two POSTS (or 1 POST, 1 GET) on
> > separate connections, one for the upstream data, and one for the
> > downstream data.  People do actually use this method already.
> > 
> 

It's not possible to use CONNECT, if you want to establish a two way
message stream with some remote server.  The intermediaries will
intercept CONNECT, but they will forward POST (or POST+GET pair).
They get converted into application streams at the server, dependent
on particular URLs.

> Whilst I'd love to see HTTP move to a more state-driven
> multi-transactional multiplexed command protocol,

A nice turn of phrase.  While we're at it, let's add lease-based
tree-structured caching, delta transmission, and peer to peer
distributed hashing :-) No, just half-joking...

> it would

As long as the new port number is 80, for deployment reasons :-)

> > Only when the proxy is near the client.  
> 
> Sure - that's what I meant by normally.

I'm not sure if that's the normal use of
two-way-streaming-over-port-80-started-with-HTTP.  It's obviously the
normal use of CONNECT.

> > Well, HTTP proxies are often available when nothing else is.
> 
> Agreed, which places HTTP in a fairly unique position.

Going further, I notice that CONNECT is sometimes not available, so a
really robust tunneller must use the POST+GET method.

> Actually it's very common for CONNECT to be used for HTTPS.  Many people
> prefer to not provide NAT to all users.  

These days, why wouldn't they just NAT only port 443?  Using a CONNECT
proxy seems to add nothing by way of security compared with
port-restricted NAT, and just means server load and bigger latency.

On the face of it, I suspect CONNECT is still used for port 443 only
because it dates from a time when port-restricted NAT (or any NAT)
wasn't really an option.

-- Jamie

---

From: Adrien de Croy <[email-masked]>
Date: Sat, 02 Feb 2008 09:43:40 +1300

> Yes, when supported by intermediaries (i.e. often no).
Right - because everything needs for starters to be HTTP/1.1 compliant, 
else you can't send chunked requests

> They get converted into application streams at the server, dependent
lost me there - you mean then establishing a 2-way stream with some 
remote HTTP process?

> distributed hashing :-) No, just half-joking...
don't hold back! :)

> As long as the new port number is 80, for deployment reasons :-)
can you imagine trying to support both protocols over the same port?

> port-restricted NAT, and just means server load and bigger latency.
there are some things you can do intercepting port 443 sequenced data 
(i.e. up the TCP stack).  Such as checking higher level protocol (i.e. 
make sure it's SSL/TLS), checking certificates, and certificate 
revokation lists.  These aren't easy to do in NAT.

> because it dates from a time when port-restricted NAT (or any NAT)
There are still plenty of security reasons why many people prefer not to 
offer NAT on their networks.

There are still plenty of people that think NAT is an abomination since 
it "breaks" TCP.  Most customers don't care about that sort of thing 
though, and for them it comes down to security/control issues.

I would have thought in the end, if someone is designing a new client 
server protocol, a good option would actually be to just use HTTP 
directly.  Most media players for instance support an HTTP mode.

although sometimes it goes a bit far, like UPnP - Soap over XML over 
HTTP over... wait for it.... multicast UDP...

-- 
Adrien de Croy - WinGate Proxy Server - http://www.wingate.com