Subject: Re: TSV-DIR Review of draft-ietf-shim6-protocol-09.txt

From: Iljitsch van Beijnum <[email-masked]>
Date: Thu, 22 Nov 2007 17:59:45 +0100

On 22 nov 2007, at 17:17, Bernard Aboba wrote:

> However, this relationship is not explored.  The TCP keepalive  
> interval
> is generally kept quite large, partly out of a desire not to tear down
> idle TCP connections due to a transient failure.  The SHIM6 keepalive
> interval during idle is not defined in the Failure Detection document,
> but my impression was that it could be much shorter and this would
> seem to collide with the philosophy of TCP keepalives.

Shim6 REAP keepalives aren't sent when a shim6 context is idle.

The REAP protocol assumes that traffic must always be bidirectional,  
so when there has been outgoing traffic but no incoming traffic, there  
must be a failure. Keepalives exist to accommodate the cases where  
there is legitimately only incoming traffic but no return traffic.  
When traffic is flowing in both directions or when there is no  
traffic, there is no need to send keeplives.

> The SCTP algorithms make extensive use
> of transport layer information such as retransmission counts, which
> the SHIM6 Failure Detection document seems to assume will be  
> unavailable.

Right. Shim6 must work for all kinds of communication. However, it  
would be good to make use of transport protocol knowledge when  
available. You feel there are missed opportunities in this area?

> In general, it would not be desirable for SHIM6 to initiate the re- 
> homing
> of a TCP connection due to a transient failure.  Link layer "down"
> indications or resulting address deprecations are examples of this.

The trouble is, how do you know a problem is transient?

About address deprecation: I do seem to remember a discussion where  
the conclusion was that deprecation is no reason to stop using an  
address just because it's deprecated. Telling the other end that an  
address should no longer be used when it's deprecated would have that  
effect, so if the proto document mandates that, that could be  
problematic.

(One scenario is a router that no longer sends RAs but still continues  
to route, it would be possible to use the addresses after they've  
become deprecated until they become invalid in this case.)

> 6.  Interactions of SHIM6 with congestion control.  Section 4.3 of the
> Failure Detection document talks about exploration timeout values.
> Exploration can be kicked off if no inbound traffic is
> received within Send Timeout (default = 10 seconds).

> The first observation is that the Send Timeout should probably depend
> on the RTO estimate, as it does in SCTP.  Otherwise we could have a
> network with a high RTO and SHIM6 exploration could commence after  
> RTO is
> backed off only a few times.  This would be undesirable from a  
> congestion
> control point of view.

We need the timeout to be somewhat long to accommodate the case where  
a host receives a packet, then does processing and finally sends an  
answer. However, it also needs to be fairly short so that we have time  
to repair a failure before the user, application or transport protocol  
give up. I don't think alignment with the transport's retransmission  
timeout makes sense here.

> The suggested value of the Initial Probe Timeout (500ms)
> is less than RTOmin and 4 probes can be sent before initiating
> exponential backoff.  This seems like it could violate "conservation
> of packets".  Why doesn't exponential backoff begin immediately?

Then you'd either have to send the first few probes in quick  
succession without leaving a reasonable amount of time for responses  
to come back, or it would take very long for the first 5 or so probes  
to go out. 500 ms is still relatively aggressive as it's well below  
the maximum observed RTTs on the internet.

Iljitsch

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Fri, 23 Nov 2007 12:46:57 +0100

[redirecting to the shim6 list]

> That makes sense.  It would be helpful for the protocol document
> to include this explanation in the keepalive section

You're right, this is not made explicit in the document.

I'll leave it up to others to decide if another iteration is  
warranted, but if so, i think we should add this.

> (and also for
> the Failure Detection specification to remove the reference to
> TCP Keepalives, since they're unrelated).

You're right.

> Yes.  If the transport layer can make the information available, then
> it seems to me that Failure Detection could be improved, providing
> for TCP a better approximation of the functionality in SCTP.

Which REAP values do you think could be changed based on information  
available to transport protocols?

>>> homing of a TCP connection due to a transient failure.  Link layer  
>>> "down"

> You don't.  That's why "link down" indications are best ignored by
> both the Internet and Transport layers.

A multihoming solution ignoring outages kind of defeats the  
purpose.  :-)

Note that we explicitly say that this applies to local knowledge that  
an address doesn't work, although then the issue is confused by  
mentioning deprecation, which leaves the address involved still  
reachable.

The issue is also different from regular operation, because if I pull  
out my ethernet cable when I'm not running shim6, the only two options  
are keeping the session open until it times out or giving up  
immediately. The latter is suboptimal because I could put the cable  
back in before the session times out. But with shim6, it's possible to  
rehome the connection to a different interface so it keeps running  
whether or not the cable is plugged in again.

>> About address deprecation: I do seem to remember a discussion  
>> where  the
>> conclusion was that deprecation is no reason to stop using an   
>> address
>> just because it's deprecated. Telling the other end that an  address
>> should no longer be used when it's deprecated would have that   
>> effect, so
>> if the proto document mandates that, that could be  problematic.

> It is suggested not mandated.  However, it's hard to see a  
> circumstance
> in which this would be helpful (and it will often hurt), so I'd  
> prefer to
> see the suggestion removed.

Marcelo, Erik?

>>> network with a high RTO and SHIM6 exploration could commence  
>>> after  RTO
>>> is backed off only a few times.  This would be undesirable from a
>>> congestion control point of view.

>> We need the timeout to be somewhat long to accommodate the case  
>> where  a
>> host receives a packet, then does processing and finally sends an  
>> answer.
>> However, it also needs to be fairly short so that we have time  to  
>> repair
>> a failure before the user, application or transport protocol  give  
>> up.  I
>> don't think alignment with the transport's retransmission  timeout  
>> makes
>> sense here.

> The RTO represents the best estimate of the maximum time that can
> expire until an ACK is expected.   So while I'd agree that failover  
> should
> occur prior to transport connection teardown, it is not desirable  
> for this
> to occur before a minimum number of RTOs has expired.  The time that
> this takes will depend on the RTO.  For example, if the goal is to  
> re-home
> after 3 timeouts, using an RTOmin of 1 second, three timeouts will  
> take
> 7 seconds.   However, where the RTO is much larger,  10 seconds might
> correspond to fewer timeouts (maybe only 2).

Ok, the text about how the retransmission timeout is computed in RFC  
793 is somewhat complex, so rather than risking getting it wrong, let  
me ask you: what are values one would expect to see for the RTO in a  
number of common deployments, such as a low RTT / high bandwidth LAN,  
a long fat pipe, and a low bandwidth / high delay mobile link?

We do not mandate an interval for generating keepalives except a  
maximum of 8 seconds, but I would expect implementers to go for 3  
seconds based on the 10 second timeout. The other end can't then  
dynamically reduce the timeout, at most it can increase it.

It would be possible to adjust the keepalive interval based on RTT  
estimates, though.

>> succession
>> without leaving a reasonable amount of time for responses  to come  
>> back,
>> or it would take very long for the first 5 or so probes  to go out.  
>> 500 ms
>> is still relatively aggressive as it's well below  the maximum  
>> observed
>> RTTs on the internet.

> The issue is kicking off SHIM6 exploration simultaneously with
> transport layer congestive backoff.  While SHIM6 exploration is  
> designed
> to find alternate paths, the paths could still share a bottleneck.    
> So
> while transport layer congestive backoff  is attempting to let packets
> drain from the network, SHIM6 will be injecting more packets.  In  
> these
> situations, aggressively sending Probes will not improve the  
> likelihood
> that they will get through.

Hm, I don't see that happening very easily. If there is a routine  
level of congestion, you would still expect to see at least one packet  
coming in every 10 seconds. So it should be rare for path exploration  
to be triggered by congestion. If the congestion is bad enough that  
this does happen, then I'd consider that a feature, as shim6 will then  
most likely select another path, avoiding the congested part of the  
network.

> With respect to 500ms being "well below the maximum observed RTT on  
> the
> Internet", I'd observe that RTOmin is set at 1 second.  So my
> recommendation would be to set the minimum Initial Probe Timeout to
> RTOmin, and allow upwards adjustment based on the RTO estimate, if
> available.

In a transport protocol, you're sending every packet to the same  
destination over (presumably) the same path, so there is no value in  
sending message 2 if there's still a chance that there will be an  
answer for message 1. In REAP, ideally, we'd want to probe all paths  
immediately and then use the one for which we get the first answer.  
That would be too aggressive. 500 ms is short enough to allow an  
answer to come back in 95% or so of all cases, but it also has the  
property that if the first probed path has a rather long delay (700 ms  
mobile link) while the second probe has a much better delay (150 ms  
wired) then the second probe will see a reply faster than the first  
one so we select the second path, which is probably the right choice  
in this case.

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Sat, 24 Nov 2007 13:31:37 +0100

On 23 nov 2007, at 15:05, Bernard Aboba wrote:

> My concern is about wireless media which can experience large  
> variations
> in signal/noise ratio, in the process generating transient "link down"
> indications.   This could cause those connections to migrate to other
> media/interfaces.

Wouldn't that be something that should be fixed in the driver for that  
interface? Declaring a link to be down has significant implications on  
many systems, this shouldn't be done at the drop of a hat for links  
where this determination isn't easily made. Having drivers declare  
links down too soon and then having the next layer ignore that is not  
a good solution, especially because there are also link layers which  
can determine their up/down status much more accurately.

> If the host has implemented the strong host model, then
> when the transient "link down" is resolved, the connection won't  
> resume
> using the prior outbound interface.  This could lead to applications
> experiencing sub-optimal conditions  long-term  based on a transient
> event.

Hm, I must say that I don't know off the top of my head if shim6 will  
automatically rehome to the primary address pair after some time. I'll  
have to reread the specifications. Or does anyone else remember this?

> There are a few approaches that come to mind:

> a. Continue to make decisions based on timers, perhaps using the "link
> down" indication as a hint to lower the timer values (e.g. requiring
> only two retransmissions instead of three)

I'm not a fan of timer-based decision making if it can be avoided  
because it's extra work and you pretty much always wait too long or  
not long enough.

> 2. Suggest the weak host model to be used along with SHIM6, so that if
> the "link down" proves to be transient, the connection will migrate  
> back
> to its former outgoing interface.

That would be good, yes.

>> estimates,
>> though.

> If the information is available, this might be the best approach.

But is it worth the trouble? The timeout will remain the same (10  
seconds unless something else is established during the shim context  
setup) so the only difference is that if the RTT is 10 ms you could  
choose to send a keepalive after 9980 ms but if it's 1500 ms you send  
the keepalive after 6000 ms. Implementers will probably just use 3  
seconds so 3 keepalives are seen before a timeout or 4 seconds so 2  
are seen before a timeout.