Subject: Re: [v6ops] Happy eyeballs update, draft-ietf-v6ops-happy-eyeballs-02

From: Ray Hunter <[email-masked]>
Date: Sat, 28 May 2011 17:21:29 +0200

>
> Subject:
> Re: [v6ops] Happy eyeballs update, draft-ietf-v6ops-happy-eyeballs-02
> From:
> Ted Lemon <[email-masked]>
> Date:
> Fri, 27 May 2011 16:02:07 -0400
>
> To:
> Philip Homburg <[email-masked]>
> CC:
> [email-masked]
>
> Content-Transfer-Encoding:
> quoted-printable
> Precedence:
> list
> MIME-Version:
> 1.0 (Apple Message framework v1227)
> References:
> <[email-masked]> 
> <[email-masked]> 
> <[email-masked]> 
> <[email-masked]> 
> <BANLkTi=RK_Cxcu4O5Y11bb=[email-masked]> 
> <[email-masked]> 
> <[email-masked]> 
> <[email-masked]> 
> <[email-masked]> 
> <[email-masked]> 
> <[email-masked]> 
> <[email-masked]> 
> <[email-masked]> 
> <[email-masked]> 
> <[email-masked]>
> In-Reply-To:
> <[email-masked]>
> Message-ID:
> <[email-masked]>
> Content-Type:
> text/plain; charset="iso-8859-1"
> Message:
> 2
>
>
> Le May 27, 2011 à 3:56 PM, Philip Homburg a écrit :
>    
>> >  I'm not a human-computer-interface expert, so I can only guess. There is
>> >  difference between users noticing a difference, and users complaining.
>>      
>
> If you can only guess, doesn't that mean that you don't know?   If you don't know the answer to a question, how does it help anyone for you to guess?
>
> (BTW, you definitely will notice a 50ms increase in latency if it happens for every packet in a lockstep protocol.   But we're talking about TCP here, and TCP is a sliding-window protocol.)
>
>
>    
I have worked with people who measure user satisfaction. It can be a 
tricky business, with people fooling themselves unless you go for an ABX 
type test. One second can be significant on web page load times, 
depending in what the user is doing at that time.

In enterprise networks, length is much more important than width IMVHO. 
People seem to be prepared to pay a lot of money for latency reduction. 
Witness the success of WAN optimization. QoS is generally the number one 
unsolved challenge I come across. Not getting the large bandwidths at 
the right price.

TCP may well have a sliding window, but what about the applications 
running over it?

Telnet character echo for example. Or multiple serial SQL queries at 
start up?

There are enough badly written apps out there that ping pong at higher 
levels.
Users of those apps will definitely notice a change from 10mS to 50mS. 
(100 packets per second versus 20)

And what about TCP slow start? That also multiplies up the effect of 
latency for short lived sessions.

We could possibly avoid this discussion by simply reversing the default 
to be "choose the fastest address family" as Philip originally proposed, 
and then adding some mechanism for the ISP's that really need to force 
use of IPv6 by handicapping IPv4 if they so wish (and at their business 
risk). They could possibly even do that with a network latency simulator 
unit on their IPv4 interconnect ;) I don't believe in handicapping the 
majority for the needs of the few.

Talking of usability experts, before we conclude in the WG that the 
whole World will be either happy or unhappy with happy eyeballs 02 and 
the proposed hard-coded default settings, have there been any 
quantitative double-blind tests done on the effects on user satisfaction 
with real web pages with the two alternative settings (handicap IPv4 
with IH = 100mS versus no preference of Address Family = IH of 0mS)?

Would seem an easy enough test to set up, and worthwhile doing given 
that we all seem to think that the draft is generally a great idea, 
whilst the current discussion seems mainly to center around the choice 
of the default value for IH IMVHO.

Although the next follow up question might be, "where is the acceptable 
tolerance level if an IH>0 is a requirement from the WG for the hard 
coded default"?

Not so easy to answer, but perhaps another pointer that IH=0 might be 
the correct choice for default, so setting IH>0 is at the risk of 
whoever wants to do that.

best regards,
RayH

---

From: Daniel Roesen <[email-masked]>
Date: Sat, 28 May 2011 17:31:39 +0200

On Sat, May 28, 2011 at 05:21:29PM +0200, Ray Hunter wrote:
> TCP may well have a sliding window, but what about the applications running 
> over it?
>
> Telnet character echo for example. Or multiple serial SQL queries at start 
> up?
>
> Users of those apps will definitely notice a change from 10mS to 50mS. (100 
> packets per second versus 20)

Indeed. I went thru months long escalation when a customer upgraded from
a 4ms RTT direct 2mbps line between two of their sites to a 8mbps 12ms RTT
MPLS L3VPN connection. For their application, it made the difference
between "somewhat sluggish" to "absolutely unusable, we have to wait up
to 15 minutes for results".

Turned out their custom application frontend was using SQL to speak to
the backend at the other site, and hundreds of ping-pong round trips
happening, easily seen and visualized with a TCP analyzer. It was easy to
calculate that it was not the technology swap from direct E1 to multiple
bundled SDSL lines, but the increased RTT in combination with a lousy
written application expecting LAN-type latency.

And that was just 4ms to 12ms.

Best regards,
Daniel

-- 
CLUE-RIPE -- Jabber: [email-masked] -- dr@IRCnet -- PGP: 0xA85C8AA0

---

From: Ted Lemon <[email-masked]>
Date: Sat, 28 May 2011 11:50:11 -0400

Le May 28, 2011 à 11:21 AM, Ray Hunter a écrit :
> I have worked with people who measure user satisfaction. It can be a tricky business, with people fooling themselves unless you go for an ABX type test. One second can be significant on web page load times, depending in what the user is doing at that time.

Absolutely.   A second is a long time.

> In enterprise networks, length is much more important than width IMVHO. People seem to be prepared to pay a lot of money for latency reduction. Witness the success of WAN optimization. QoS is generally the number one unsolved challenge I come across. Not getting the large bandwidths at the right price.

I have a friend who used to work at a company that did high-speed arbitrage over networks.   They cared about latency.   They *really* cared about latency.  I certainly would not claim that latency is never important, nor even that 50ms of latency is such a small amount that it is never important.

> Telnet character echo for example. Or multiple serial SQL queries at start up?

For the former, 50ms isn't detectable.   It's too short.   Latency certainly matters with telnet, but 50ms isn't the benchmark.   For sequential SQL queries, you're talking about a custom app.   The implementor would have to choose to implement happy eyeballs--it's not like it just happens.

> There are enough badly written apps out there that ping pong at higher levels.
> Users of those apps will definitely notice a change from 10mS to 50mS. (100 packets per second versus 20)

Again, why do you expect that the implementors of these apps would prioritize implementing Happy Eyeballs over fixing their lockstep bugs?   These are probably legacy apps that work "well enough" and that nobody wants to mess with.   They aren't going to be doing Happy Eyeballs.   Feel free to contradict me with real examples.

> And what about TCP slow start? That also multiplies up the effect of latency for short lived sessions.

Sure, but only for short-lived sessions that are in that >0<=100 latency gap.   If the latency variation is higher than that, the IPv6 connection won't win the race, and subsequent attempts will have a reduced preference for IPv6, so that the lag will be eliminated quickly.   And it's not very much lag to begin with.

> We could possibly avoid this discussion by simply reversing the default to be "choose the fastest address family" as Philip originally proposed, and then adding some mechanism for the ISP's that really need to force use of IPv6 by handicapping IPv4 if they so wish (and at their business risk). They could possibly even do that with a network latency simulator unit on their IPv4 interconnect ;) I don't believe in handicapping the majority for the needs of the few.

But we don't know which address family is fastest.  Furthermore, you are misstating the problem.   The problem is not that we are always going to wait 100ms before trying IPv6.   The problem is that we are going to give IPv6 a 100ms head start when IPv6 is available *and* the host we are trying to connect to supports it.   Sometimes this will be the right choice: IPv6 connectivity will be at parity with IPv4, or IPv6 will be faster than IPv4, or IPv4 to that host might not even work.   Sometimes it will be the wrong choice: IPv4 would have been faster, and we wasted 100ms.

And a key factor here is that there are things we can do to tune the network to avoid making the wrong choice.   It's not the case that a happy-eyeballs-implementing app will necessarily choose wrong in these cases, because there are things that the network manager can do to prevent it.   You would like there to be more things the network manager can do.

> Talking of usability experts, before we conclude in the WG that the whole World will be either happy or unhappy with happy eyeballs 02 and the proposed hard-coded default settings, have there been any quantitative double-blind tests done on the effects on user satisfaction with real web pages with the two alternative settings (handicap IPv4 with IH = 100mS versus no preference of Address Family = IH of 0mS)?

How would you set this up?   I guess you could set up a pessimal configuration, and see what happens.   But then you'd only be testing how this plays in the worst case.   The Chrome implementation is working fine for me on a v4-only network and a dual-stack network with comparable latency, but that doesn't really tell us anything.

---

From: Ray Hunter <[email-masked]>
Date: Sat, 28 May 2011 19:04:00 +0200

Ted Lemon wrote:
> How would you set this up? I guess you could set up a pessimal 
> configuration, and see what happens. But then you'd only be testing 
> how this plays in the worst case. The Chrome implementation is working 
> fine for me on a v4-only network and a dual-stack network with 
> comparable latency, but that doesn't really tell us anything.
>    
You'd be better of asking a real expert to design the test, but I would 
imagine something similar to other ABX double blind tests: you  
automatically compile multiple copies of the application statically 
linked to a lib with IH either set to 0 or 100 on a semi-random basis 
calling them build 1 .. n. You'd store the compilation settings of IH 
used for each build in a file that you only examine after conducting the 
trials. You have a network nightmare box that automatically switches in 
say 90mS of additional latency semi-randomly for the IPv6 traffic every 
X seconds (again logging the actual settings automatically and warning 
you when a new setting had been applied, but without revealing what the 
setting actually was) You sit behind your computer and use the various 
builds to load standard pages that won't cache, noting whether you 
thought there was latency present or not, and at the end whether you 
thought the build was IH=0 or IH=100. After sufficient trials, you open 
the build and latency log files and look at the correlation for how many 
times you guessed correctly. Repeat with other values of IH and or 
latency and or test subjects until bored/ result is clear whether 
additional latency is detectable or not and whether the IH compilation 
setting had an impact on if the test subject could detect it.

---

From: Gert Doering <[email-masked]>
Date: Sat, 28 May 2011 22:03:59 +0200

Hi,

> (100 packets per second versus 20)

Don't mix session startup delay with in-session roundtrip delay.

Way different things.  You're concerned about in-session RTT, while
happy eyeballs will add initial session startup delay.

> latency for short lived sessions.

Completely irrelevant for the discussion at hand.  TCP is not modified,
and no extra delay is added to individual TCP packets.

Gert Doering
        -- NetMaster
-- 
did you enable IPv6 on something today...?

SpaceNet AG                        Vorstand: Sebastian v. Bomhard
Joseph-Dollinger-Bogen 14          Aufsichtsratsvors.: A. Grundner-Culemann
D-80807 Muenchen                   HRB: 136055 (AG Muenchen)
Tel: +49 (89) 32356-444            USt-IdNr.: DE813185279

---

From: Ray Hunter <[email-masked]>
Date: Sun, 29 May 2011 06:41:40 +0200

> and no extra delay is added to individual TCP packets.
Depends on the statistics for RTT for IPv4 v IPv6. If they are semi 
random distributions around an underlying equivalent latency then it 
will make little difference and IPv6 will be preferred but not cause any 
"harm." If the RTT difference has a static offset due to structural 
differences in path, then the initial offset will remain valid for all 
packets, because IPv6 got that initial head start but was still chosen.

Different boxes and services could cause very different physical / 
switching paths through the enterprise network

Possible issues in the enterprise that are probably not often an issue 
in the Internet: Network provider P may not be at all IPv6 capable, so 
you tunnel or temporarily contract another provider for IPv6 for that 
portion of the network (tunneling usually means losing DSCP QoS), WAN 
optimization vendor R is not yet IPv6 capable, Firewall vendor J 
implements hardware acceleration for IPv4 but not IPv6, Load balancer 
vendor X fast switches IPv4 but process switches IPv6, policy based 
routing may send some IPv4 via Internet offload to reduce costs but IPv6 
runs direct via MPLS (no IPv6 NAT means that the return path is 
difficult to engineer for policy based routing if traffic ends up at the 
same subnet), so some of these cases are where IPv4 should actually be 
preferred even though it is apparently slower for the SYN ACK race.

I agree it's a dilemma to choose the right value for the default, hence 
my preference is IH=0 or as close to 0 as possible that it is not 
detectable by users.

Regards,
RayH