Subject: Re: Adjusting our spec names

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sun, 01 Apr 2012 08:06:17 +0000

In message <CAP+FsNdU7g6u_taJ94jwBkkQUCEKq=[email-masked]>
, Roberto Peon writes:

>I can believe there is a different world ahead w.r.t. bandwidth, but seeing
>massive RTT decreases would be surprising.

Yes, what I meant was that the bandwidth/RTT ratio will change
for the worse.

>> None of the proposals we have seen so far is anywhere near being
>> feasible in such a context.
>
>What is your basis for this statement? After all, your argument about
>technology improvements in the future cuts both ways:

My basis for this statement is a lot of experience in moving and
processing data at high speed, and the lack of improvements in
computers.

If you look at the technological advances on the computer in the
last ten years, there almost hasn't been any.

We can add more transistors to a chip, but we can't make them run
faster.

What we have seen and what we will see is wider and wider NUMA
running at the same basic 3-4 GHz clock.

There are no solid-state physics advances that indicates we will
see anything else the next 20 years.

In fact, we might not even see more and more transistors, as "extreme
ultraviolet litography" appear to be a lot harder to tame in a
semi-fab than anybody expected.

You are right that we see, and will see a lot of offloading, I'm
working with an ethernet controller now that has close to 1000 pages
of datasheet, and for many features of the chip the description is
at best summary.

But offloading only gets you so far, because setting things
up takes time.  That's why you don't see very much offloading in
practice.

To a first approximation, processing a transaction can be done in:

	time = a + b * len

and it is the 'a' constant that gets you in the offloading case.

We saw this very clearly with crypto-offloading, for instance HiFns
chips, where the cost of the setup was so high that in many cases
it was faster to just do the crypto on the CPU instead.

Right now everybody is scrambling to show that they can handle
full packet load at 10Gb/s and 40Gb/s.  At 10Gb/s that's 14 million
packets per second, and it's a serious challenge for most CPUs to
do non-symbolic work to those packets.

Doing a single SSL stream at 1Tb/s should not really be a problem
in hardware.

Doing 10 million 10kB HTTP transactions in a second is going to be
a heck of a challenge, if nothing else for the memory footprint.

>As for your first point... are you suggesting that we develop something
>other than HTTP/2.0?

No, I'm suggesting that we develop a HTTP/2.0 worthy of the '2' part
of the name :-)

>I'd suggest that we should design so that this is possible, but... I think
>that (charter aside) things which are fundamentally different from what
>HTTP does today would be biting off more than we can chew right now :/

Not really.

If you look at what HTTP does it's actually very primitive, you can
send messages consisting of:
	{
	verb (GET)
	noun (URI)
	routing info (Host:, session cookies)
	metadata (other headers)
	(body)
	}

and get replies on almost the same form:
	{
	result (200)
	routing info (session cookies)
	metadata
	(body)
	}

The performance bottleneck is not going to be the endpoints, but rather
the points along the path where traffic from many sessions
is aggregated:

	The telcos outgoing mobile->HTTP proxy
	The load-balancer of $bigsite

Both of those are essentially HTTP routers and they don't care about
the metadata and the body, they only care about some of the "envelope"
information, so they can find out where to send the stuff.

Burying the envelope information in gzip or SSL would make make it
patently impossible to route 1Tb/s HTTP traffic, but a well thought
out design in HTTP/2.0 will make it possible to do both it in
hardware or software.

There is a lot of focus on the list right now with respect to
compression and encoding to save bytes, but some of those saved
saved bytes should be spent again, to make the protocol faster to
process at the choke points in the network.

One of the really big problems with HTTP/1.1 in that respect, is the
lack of a session-concept, which people hack around with cookies.

HTTP/2.0 should have real sessions, so that HTTP-routers don't have
to inspect and mangle cookies.

We should absolutely _also_ make the protocol faster at the endpoints,
but the focus there is of a different kind, (Do we really need Date:
at all ?) and there are other factors than speed in play (security,
anti-DoS etc.)

And we should also keep the HTTP/1.1 -> HTTP/2.0 issue in mind, but
I don't think it is very important, because nobody is going to be
able to drop HTTP/1.1 service for the next 10-15 years anyway, so
if our HTTP/2.0 can wrap up a HTTP/1.1 message and pass it through,
we're practically home.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: Willy Tarreau <[email-masked]>
Date: Sun, 1 Apr 2012 11:00:52 +0200

> Yes, what I meant was that the bandwidth/RTT ratio will change

Yes the BDP is going to increase as bandwidth grows faster than RTT shrinks,
with a lower bound to RTT.

> see anything else the next 20 years.

I agree. I predict that we'll see "smarter" processors, but not much
faster ones. Smarter is still something to be defined :-)

> up takes time.  That's why you don't see very much offloading in

I was saying to Mike on thursday that for the last few years I've been
expecting that we'd see crypto offloaded into NICs at the datagram level.
That would be very cheap and would not bring processing overhead like
streaming crypto does, probably making datagrams more appealing in the
future.

> a heck of a challenge, if nothing else for the memory footprint.

Hence the need to reduce the per-transaction processing cost.

(...)
> to inspect and mangle cookies.

I'd love to see this but this will never totally alleviate the need for
inspecting cookies since in practice we see several levels of sessions
in application infrastructures. I'm personally used to commonly see at
least two layers.

That said I'm working with Brian Carpenter and Sheng Jiang on a way to
use IPv6 flow label for this, but the more I think about it, the more
I think it would be useful to have a TCP extension to transport such
application session information (that's out of the scope of this WG).

> if our HTTP/2.0 can wrap up a HTTP/1.1 message and pass it through,

In fact what we precisely need is to be able to :
  - convert a 2.0 request into a 1.1 request
  - convert a 1.1 response into a 2.0 response
  - let 1.1 request pass unmolested

The need to convert 1.1 requests to 2.0 is much less important in my
opinion because most browsers will switch 2.0 on by default.

Regards,
Willy

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sun, 01 Apr 2012 09:06:48 +0000

In message <[email-masked]>, Willy Tarreau writes:

>[...]

>application session information (that's out of the scope of this WG).

I'm not sure I see how that would work, if a proxy of some kind
connects to a server, there will be multiple sessions in the same
TCP connection.

---

From: Willy Tarreau <[email-masked]>
Date: Sun, 1 Apr 2012 11:28:13 +0200

> connects to a server, there will be multiple sessions in the same

This is a good point, though if the info is sent along with the request
it's not necessarily a problem. Sort of an out-of-band session info. I'm
not saying this is trivial (otherwise we'd already have it), but it's
worth experimenting to avoid the current mess of deciphering SSL and
parsing L7 just to find the server.

Willy

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sun, 01 Apr 2012 09:32:58 +0000

In message <[email-masked]>, Willy Tarreau writes:

>[...] but it's

I think we should just accept that HTTP messages in practice will
be routed, and provide sensible information for routing decisions
in the "envelope".

---

From: Roberto Peon <[email-masked]>
Date: Mon, 2 Apr 2012 00:08:34 +0200

> see anything else the next 20 years.

Ah. My belief is that it is manufacturing methods that must change to
enable the next speed increases, not the transistor properties themselves
as cooling and size seem to be the big limiters. One could go 'clockless'
as well. Building a successful e to o substrate on a chip would yield big
changes. We could build 'up' not just out.

Personally, I wouldn't put money on a bet that chips won't become faster. I
agree, however, that chips that look strongly like the ones we have today
may not, and at some point in time speed-of-light will be the bottleneck,
but we're not there or close to it yet.

> it was faster to just do the crypto on the CPU instead.

Not having played with Hi/FNs stuff, I don't know where the problems lay,
but it sounds like there was an architectural screwup if that was the case.
Can you speak as to why the setup was so expensive? That just seems wrong
and bizarre.

>From what I've seen, the CPUs aren't the bottleneck, it is the architecture
of the IO subsystems of our OSs and often a lack of multiqueue on the NICs.

> Doing a single SSL stream at 1Tb/s should not really be a problem

Agreed.

We're not that far off from being able to do it in software either. On a 12
core 3 Ghz system (which are available these days), you'd have to do ~28
bits/cycle on each core. Bus speeds of current architectures would likely
be the holdup.

> No, I'm suggesting that we develop a HTTP/2.0 worthy of the '2' part

heh

> >HTTP does today would be biting off more than we can chew right now :/

To be clear, my opinion is that we can lay the groundwork on which a change
of semantics is built (e.g. from request->response to more arbitrary
message<->message or message<->stream mappings), but that attempting to
define those semantics here and now will make it less likely for the entire
effort to succeed.

I also think that, if we could succeed at defining additional semantics as
seen from the endpoints, that it would be very valuable and useful... so
our disagreement is just in timing.

>        The load-balancer of $bigsite

imho, from the point of view of the client, the load-balancer of $big-site
or any other reverse-proxy is likely the endpoint as viewed by the client.
I do believe in making these as fast as possible without sacrificing what
the user and the content provider wants to do with the applications to
whatever the forward proxies and traffic mungers want to do.

> lack of a session-concept, which people hack around with cookies.

 I prefer to think of it as a shared context because session has so many
other interpretations at this point, but yes, I agree. Willy's proposal has
parts/ideas which address that and which (as Willy knows) I think are
helpful and valuable.

> to inspect and mangle cookies.

Defining sessions may be ok, so long as discrimination against various
endpoints can be accomplished by intermediaries only when the user has
given them permission to do so. Given the often non-aligned interests of
any non reverse-proxy intermediary with the user, I think this is necessary
for the long-term health of the internet.

Perhaps the wg (i.e. we) should do an analysis of motivation and incentives
for forward/reverse proxy and other intermediaries? Understanding the game
theory would be helpful for everyone and would inform decisions that are
more likely to benefit everyone in the long-term.

It doesn't matter if my site and LB can do 1Tb/s if AT&T decides to
throttle my site because I'm not double-paying or otherwise bribing them to
let my packets through in something like a protection racket. To be clear
AT&T is proposing something like this (with much different wording, of
course) for mobile right now. I'd rather not rely upon the regulatory
environment for a good outcome on stuff like this...

> if our HTTP/2.0 can wrap up a HTTP/1.1 message and pass it through,

I think it is important for the transition period, where people deploy
reverse proxies or loadbalancers which speak HTTP/2.0 to the client over
the large-RTT net segment and then speak HTTP/1.1 to their backends until
they eventually upgrade.

-=R

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sun, 01 Apr 2012 23:55:59 +0000

In message <[email-masked]>

>Not having played with Hi/FNs stuff, I don't know where the problems lay,

No screw-up, the HiFn crew were pretty smart people.

Its simply that the distance from a userland application that needs
crypto, down to the kernel and out to a PCI device takes a devastating
amount of time, relative to the actual crypto operation.

Footnote:

	VIA went and did what should be done: they added AES as a
	CPU instruction, exploiting politics the same way ZyXel did
	in the 1990ies, when they added 3DES to their modems.

	Rumours has it that Intel and AMD have CPU hardware have
	supported crypto-instructions for several years, but that
	the necessary microcode-update file is "Teserved for certain
	customers only."

What Intel now does with offloading into the ethernet-chips is
similar to what IBM did with the 3745 Mainframe Frontend back in
the 1980ies.  The words "desparate" and "futile" were used a lot
about it in the early 1990ies.

>> full packet load at 10Gb/s and 40Gb/s.
>

Well, as I hinted above, the evolution of the IBM mainframe is very
enlightening here, we see the exact same order of developments in
the PC platform these days:

	More complex and specialized instructions are added
	More parallel execution units rather than faster EUs
	More and larger caches to cover speed disparity EU<-->RAM
	Virtualization to get better utilization of numerous EUs
	EUs move further and further away from I/O units
	I/O Units gain more and more processing power to compensate.

I think this may be a law of nature of sorts, a curse which tangle
up any successful machine architecture.  If you look at the Thumb2
instruction set, ARM seems to be headed down the same deroute.

So no, I don't put much trust in processing offload: It's complicated
to implement both in hardware and software , it has high latency
an the performance gain is only very slight.

But it will probably be done anyway.

The most we can do, is make the protocol sensible for it.

>define those semantics here and now will make it less likely for the entire

I think the prudent thing would be to design the new
serializations/transports for "message<->message", to be future
compatible, and use them only for the subset functionality of
"request->response" for now.

>or any other reverse-proxy is likely the endpoint as viewed by the client.

Yes, that's what makes it the bottleneck, but in protocol terms the
endpoint is where a request gets replied to with a response.

>given them permission to do so.

I have read draft studies that showed that you could tell when
people where typing their password over a SSH connection, based on
the timing of the packets.  The timing pattern eliminated about 50%
of the entropy from the password typed, due to the constraints in
the QWERTY layout.

Privacy is a _lot_ harder than most people think.

I think we need to have a realistic view of what level of privacy
can be guaranteed, and I would like to see us move to a model
more like email, where you have a plain-text envelope used for
routing and the (optionally) privacy protected content.

I would argue, that for HTTP the request envelope consists of:
	request (GET/POST...)
	URI (maybe without the query part ?)
	Host:
	Session identifier (today a cookie)
	Length of transmitted body, if any.
For reponses the envelope is only
	status code (200,503,...)
	session identifier
	Length of transmitted body, if any.

This should make the "HTTP-router" people happy and shield them from
semantic interpretations.

>more likely to benefit everyone in the long-term.

I second that.  It's quite clear that there are some divisions now
as evident in the "Mandatory TLS" vs. "You got to be kidding!"
factions.  Settling those "ideological" differences before we dive
into bits and bytes can and will save us time and conflicts down
the road.

I think at the end of the day, its rather simple though:

1) Clients should know if and when they have end to end privacy.

2) Clients should know when they have positive authentication
   of the server. (Can the client trust the server)

3) Servers should know when they have positive authentication
   of the client. (Can the server trust the client)

4) Servers should have access to means of client identification.
   (eg: For load-balancing, so all requests from the client end up
   on the same server)

5) Proxies should be able to enforce content-inspection, and
   possibly content-modification, but users should be made clearly
   aware that this happens (ref: 1)  In this case 2 & 3 applies
   to the client-proxy relationship.

6) Seen from the server-side, proxies come in "trustworthy" (server-side
   reverse proxy/http-router), "suspect" (3rd party CDN) and
   "untrustworthy" (forward/client side), with respect to their
   willingness to follow instructions from the server.
   (For instance about object validity lifetime or cacheability.)

>throttle my site [...]

The extended OSI model indicates that this is a layer 8 (politics)
problem.

Trying to solve it a lower layers doesn't work and should
not be attempted.

>the large-RTT net segment and then speak HTTP/1.1 to their backends until

That's one of the primary reasons why I want to keep HTTP/2.0
request->response for now.

It's important to realize that the 1.1 <-> 2.0 conversion does not
have to be 100% perfect in all aspects, it just has to be semantically
perfect, which is a much lower bar to clear.

---

From: Nicolas Mailhot <[email-masked]>
Date: Fri, 6 Apr 2012 13:05:57 +0000 (UTC)

>    to the client-proxy relationship.

Sometimes proxy needs are even more basic: corp may decide its users have no
need of facebook at work, so the proxy will only want to block facebook acesses
(unless the user works at marketing and proves it by authenticating on the proxy
with a marketing ID)