Subject: Re: [v6ops] Happy-eyeballs - P-value & behavior

From: Andrew Yourtchenko <[email-masked]>
Date: Tue, 1 Mar 2011 15:02:47 +0100 (CET)

Hi Javier,

On Mon, 28 Feb 2011, Javier Ubillos wrote:

> On Fri, 2011-02-25 at 18:35 +0100, Andrew Yourtchenko wrote:
> *cut*
>>
>>> Also, if both connections fail, I suggest we reset P to 0.
>>
>> Seems like it won't be a big deal - but do you have more details on the
>> motivation behind ? If both connections failed, that would say either about the
>> very sketchy connectivity of the target, or possibly that the source has lost
>> their connectivity on that interface.
>>
>> If the latter, then I think resetting would make sense under the assumption that
>> we have a different attachment later. If that happens more often than either the
>> target total unreachability or the 'spurious connectivity losses' (like, bad
>> connectivity on wireless).
>
> Perhaps that's a more sensible approach.
> If the hosts IPs change, reset P (we *might* have changed networks).

Thinking of this more - if we converge aggressively and quickly enough, 
this might even be a smaller optimization. All will depend on how much code it 
is vs. how much is at stake to just let it "fix itself" vs. "restart".

>
>>
>>>
>>> Multiplying with -1, consider the scenario:
>>> P is high, favouring IPv6 and delaying IPv4 connections.
>>> If IPv4 still wins, it means that suddenly is significantly better than
>>> IPv6 (it won dispite IPv6's headstart).
>>> Shouldn't we multiply P by -1?
>>> Does it matter if we start "flapping" between IPv4/6?
>>> If it does matter, e.g. we want SSL, wouldn't a slow-flapping
>>> ("normal"-P management) be just as bad?
>>
>> I think this would optimize the case where we switched a link - but deoptimize
>> the case where we had a one-time blip in the connectivity ?
>
> After some closer consideration.. Yes, you'r right.
> I commented on that in the mail to/from Simon Perreault.
> So, as mentioned above, perhaps a better method to achieve what I'm
> trying to get at is to detect local address changes.
>
>>>
>>>
>>> * Why should we keep a P-value at all?
>>>
>>> After a mail-exchange with Dan Wing, my conclusion is basically, to
>>> avoid unnecessary SYN-spamming. Which makes sense to me, SYNs don't have
>>> a back-off mechanism as an "ordinary" TCP flow has. However, using a
>>
>> I think not only SYN-spamming - but also accept()-spamming on the server side I think.
>>
>> Arguably from the capacity planning a host should be able to support 2x
>> accept()-load - but if this is normal scenario, that seems like a bit of a
>> waste.
>>
>> (This is a theoretical view 'in case everyone implemented this', obviously do
>> not apply in the beginning)
>>
>
> Correct me if I'm mistaken, but...
> If the responder dosen't allocate the TCB until it recieves the ACK
> (third handshake packet), then the implementation shouldn't consume much
> resources until that point (protecting against SYN-floods).
> And, the initiator, shouldn't be sending two ACKs, but rather one ACK
> and one RST (as it has chosen a winning socket).

You mean the syncookies, right ? I thought by default they are inactive - only 
kicking in when the host has a lot of incomplete connections ? (I would need to 
verify about the second accept - I do not remember how the server-side select 
loop behaves if the connection-in-progress gets terminated).

>
>>> P-value, where P is a delay to be introduced before trying the "other"
>>> AF seems sub-optimal to me.
>>> Perhaps some other congestion-control for SYNs is more appropriate?
>>> A trivial algorithm _could_ be:
>>>
>>>  | Always send a SYN for both IPv4 & IPv6 (be aggressive)
>>>  | If some SYN does _not_ receive a SYN-ACK begin
>>> SYN-congestion-control
>>>  --> Select the best AF (v4 or v6) according to some criteria
>>>    | During time T, only use the best AF
>>>    |* When time T expires, test sending a SYN on the other AF, and see
>>> if it gets through (multiple tests?)
>>>    | If the congestion is gone go to start (i.e resume an aggressive
>>> approach)
>>>    | Else - reset T and resume the timid approach.
>>>
>>> The reason I'm suggesting a "default" aggressive approach, that I don't
>>> think that SYN packets make out a noticeable large fraction of the total
>>> amount of traffic. I've started some captures to compare our offices
>>> ratio of tcp/tcp-syn packets (overall and for www traffic).
>>
>> While we're thinking radically different ideas: how about a "zebra retransmit".
>>
>> Start with your "favourite" AF, and if you need to retransmit - then send the
>> SYN for a different AF. This way the amount of SYNs will always be exactly the
>> same. If more than 50% of the time over a certain period you fallback - change
>> your favourite protocol.
>>
>> This idea would get nuked by the purists, I suspect :-)
>
> Isn't the core problem that we don't want to start with AF_INET6 (our
> favourite protocol) and have to wait for it to time-out before we switch
> over to AF_INET?
> Isn't it that which is what's causing service-providers to start use
> whitelisting for their AAAA records?

Sorry if I was unclear. The current problem looks roughly like this:

<pre>

|- 0sec        SYN(AF_INET6)
|
|- 2sec        SYN(AF_INET6)
|
|- 4sec        SYN(AF_INET6)
|
|- 8sec        SYN(AF_INET6)
|
...  lunch time ...
|
|- [a lot] sec SYN(AF_INET) 
|
v
time

</pre>

What I suggested above is either:

<pre>
|- 0sec        SYN(AF_INET6)
|
|- 2sec        SYN(AF_INET)
|
|- 4sec        SYN(AF_INET6)
|
|- 8sec        SYN(AF_INET)
|
... no lunch! ...
|
|- [a lot] sec: does not matter. We tested both AFs quickly.
|
v
</pre>

or:

<pre>
|- 0sec        SYN(AF_INET)
|
|- 2sec        SYN(AF_INET6)
|
|- 4sec        SYN(AF_INET) 
|
|- 8sec        SYN(AF_INET6)
|
... no lunch! ...
|
|- [a lot] sec: does not matter. We tested both AFs quickly.
|
v
</pre>

So rate-wise we would send exactly the same # of packets. But it's very impure 
approach since the timings would depend on the connections from each other.

>
>>>
>>>
>>> * P-scope
>>> Currently we're choosing the easy-way out, and keeping a system-wide P.
>>> However, I'm assuming that a per-destination P is more relevant. Perhaps
>>> even destination+service. I think destination+service is overkill. We
>>> _want_ to re-use P and not have to re-init/test too many times.
>>> We'll continue with a system wide P for now, but we'll likely go the
>>> per-destination P, hopefully with some nice aggregation feature too.
>>
>> How is the PMTUD info stored ? I think that the connectivity
>> characteristics may be related to the PMTUD. P might then sit nicely
>> in one of the address families alongside with MSS ?
>
> I'm not sure I understand what you're getting at.
> The name-based socket will keep a readable P value. We could allow an
> interface which could access AF_INET6 data (such as PMTUD).
> In the happy-eyeballs case, we could also make an interface to get the
> MSS and have it point to the active socket (IPv4/v6).

it was just a thought on a possible place to store the aggregated P values, 
together with whereever the cached MSS values are stored (if.)
sorry I should have been clearer in the wording.

cheers,
andrew