Subject: Re: [IPsec] Secure Crash Discovery for IKEv2

From: Yoav Nir <[email-masked]>
Date: Fri, 19 Sep 2008 07:31:26 +0100

Hi Stephen.

On Sep 19, 2008, at 4:52 AM, Stephen Bevan wrote:

> The drafts for QCD and SIR both assume that the crash, reboot and
> subsequent INVALID_SPI will be generated less time than DPD can detect
> failure.  Both point at RFC 4306 section 2 suggests DPD should run for
> several minutes to indicate the results that can be expected from
> using DPD.  I did not see either mention what typical/expected
> crash/reboot times are.

Boot times can be very short. See below.

DPD is not always appropriate, because of issues at the client side.  
DPD requires sending occasional keep-alive packets, which consumes  
bandwidth and battery on small devices, and in other cases keeps a PPP  
connection (PPTP, L2TP-over-DSL) alive, which costs the user money and  
reduces battery life.

> The commodity PC hardware I'm typing this message on will spend at
> about 30 seconds in the BIOS before it even starts booting the kernel
> and it takes a minute or more before the IKE daemon is up and ready
> to send out an INVALID_SPI.  Thus with either protocol the best I
> could hope for on this hardware would be 1 minute of traffic loss.
> Two ways of bringing down that time are :-
>
>  1. Get hardware the boots an order of magnitude faster.
>
>  2. Use an IKE daemon that allows me to configure the DPD settings to
>     whatever I deem correct for my toplogy.
>
> I'd dearly love 1 but where do I get it and how much more does it
> cost?  Thus I'm left looking at 2 which is a quality of implementation
> issue that requires no protocol changes.

VPN gateways that run on specialty hardware rather than commodity PCs  
or servers boot up and are ready for IKE connections in less than 20  
seconds. They are very common, probably more so than "software-based"  
VPN gateways. But that doesn't matter. As a vendor, I can tell you  
that nearly all our customers use some kind of hot standby gateway.  
Unless all IKE and IPsec SAs between the active and stand-by member  
are synchronized, the stand-by will take over within a second, using  
the same identifiers, but with no state. This is equivalent to a  
single gateway rebooting instantly.

> There may be other scenarios where the protocols do better than DPD
> with configurable timeouts in which case I suggest the introductory
> sections of the drafts be updated to make the case for those scenarios
> _much_ more strongly in order to clearly indicate why DPD is not
> sufficient and a new protocol is required.

Thank you. In the next update, I will emphasize the hot stand-by case.

_______________________________________________
IPsec mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ipsec

---

From: Tero Kivinen <[email-masked]>
Date: Fri, 19 Sep 2008 16:05:48 +0300

> connection (PPTP, L2TP-over-DSL) alive, which costs the user money and  

As I explained in my previous email DPD can also be done in more
efficient way, i.e. by detecting the one way communication.

Our implementation does it so that when it detecs it has been sending
out packets for more than 10 seconds without receiving any single
packet back, it will start DPD process. If the DPD gets reply back
immediately (as it normally does as other end is still up, but traffic
is one way for some other reason than reboot) this faster one-way dpd
trigger is disabled until traffic changes back to two-way.

In normal case this does not require any extra bandwidth or battery
power, as normally traffic is two-way (i.e. TCP acks etc coming back).

In case the traffic changes to one-way outgoing traffic, without any
incoming packets then we do one extra DPD very quickly after traffic
changes to one-way, and after that we do not do any extra DPDs.

Keeping PPP connections etc alive does not matter as by definition
there is traffic going out all the time (one one-way packet is not yet
enough to trigger the one-way traffic trigger).

> seconds.

Quite a many of the ADLS modems require more than 20 seconds to just
negotiate the ADLS link up (I witnessed this for one modem when I was
trying to "guess" correct parameters for it to replace my broken ADLS
modem, and I needed to reboot it several dozen times, and even when
the actual boot time of the device was around 10-20 seconds, the ADLS
link took at least same amount of time to establish). 

> They are very common, probably more so than "software-based"  

It is much better to keep the IKE SAs synced between the gateways,
thus there is no need for secure crash recovery. The hot standby can
just send IKE SA delete notifications for old IPsec SAs the other end
had and create new IPsec SAs, or it can even have synced state for the
IPsec SAs too (but that is much more expensive because of the replay
protection counters).

I do not think this hot standby case is that relevant for the secure
crash recovery, as those methods used there are much better anyways.
-- 
[email-masked]
_______________________________________________
IPsec mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ipsec

---

From: Yoav Nir <[email-masked]>
Date: Fri, 19 Sep 2008 17:53:36 +0100

> trigger is disabled until traffic changes back to two-way.

That's a good way, but then if your email client polls the IMAP server  
every 5 minutes, you'll only get a burst of activity every 5 minutes.  
Even a gateway running on a commodity PC has time to reboot in 5  
minutes, and DPD will take more time than necessary (30 seconds in  
your case IIRC)

> crash recovery, as those methods used there are much better anyways.

We actually sync IKE SAs between gateways, although this is getting  
more difficult, because IKEv2 requires the message counters to be  
synched. My co-authors tell me that very often, their gateways are  
deployed with no back-channel sync. We are hearing more people who  
want a physical separation between the gateways (like putting them in  
different parts of town) and in that case synching becomes much more  
difficult.

---

From: Scott C Moonen <[email-masked]>
Date: Fri, 19 Sep 2008 14:56:27 -0400

Hi Tero, you comment:

>>      "Our implementation does it so that when it detecs it has been 
sending out packets . . . without receiving any single packet back, it 
will start DPD process."

This made me go back to RFC 4306 to make sure I was understanding it 
right.  It seems there is a little bit of a lack of clarity in the RFC. 
This sentence suggests the approach you have taken; i.e., to do a liveness 
test only when there is unreciprocated outbound traffic on the SAs:

>>      "If there has only been outgoing traffic on all of the SAs 
associated with an IKE_SA, it is essential to confirm liveness of the 
other endpoint to avoid black holes."

But the sentences immediately following this in the RFC suggest that a 
liveness test should be done even when there is dead air, i.e., without 
regard to whether we have been sending traffic.

>>      "If no cryptographically protected messages have been received on 
an IKE_SA or any of its CHILD_SAs recently, the system needs to perform a 
liveness check in order to prevent sending messages to a dead peer. 
Receipt of a fresh cryptographically protected message on an IKE_SA or any 
of its CHILD_SAs ensures liveness of the IKE_SA and all of its CHILD_SAs."

Is it the intent of the RFC to do liveness checks only when there is 
unreciprocated outbound traffic?  Should the RFC allow or forbid liveness 
checks more broadly in the "dead air" case where there is no traffic 
flowing in either direction?  There may be some value in proactively 
discovering the peer's deadness in the dead-air case, but Yoav's concerns 
about overutilized connections also have merit.  If the RFC takes an 
agnostic approach to the dead-air case, we may still want to recommend 
that if an implementation allows for dead-air liveness checks, it should 
be configurable on a per-PAD-entry basis.  Or perhaps the RFC could 
recommend a more conservative approach to the dead-air case, such as 
backing off the interval between dead-air liveness checks with each 
successful iteration.

Scott Moonen ([email-masked])
z/OS Communications Server TCP/IP Development
http://scott.andstuff.org/
http://www.linkedin.com/in/smoonen

From:
Tero Kivinen <[email-masked]>
To:
Yoav Nir <[email-masked]>
Cc:
[email-masked], Stephen Bevan <[email-masked]>
Date:
09/19/2008 09:07 AM
Subject:
Re: [IPsec] Secure Crash Discovery for IKEv2

---

From: Tero Kivinen <[email-masked]>
Date: Mon, 22 Sep 2008 16:45:54 +0300

> minutes, and DPD will take more time than necessary (30 seconds in  

If you are only polling email every 5 minutes, it does not matter if
one of them is delayed by 30-60 seconds more, because the other has
been rebooted. It is not like the user would really notice whether the
announcement of new email came exactly 5 minutes later or 6 minutes
later... If the other end is up and running it will send invalid spi
notifications which can be used by the client to reduce the timeout
used for the DPD (i.e. instead of retransmitting dozen times during 5
minutes when no reply is received, you only retransmit half a dozen
times during 60 seconds).

Usually things which happens once every 5 minutes or so, are not
really time critical.

> synched.

Yes. Fortunately the IKEv2 traffic is not high volume traffic, thus
that does not require too much bandwidth. 

> My co-authors tell me that very often, their gateways are  

Putting them on different parts of the town requires quite a lot from
the other infrastructure, as it requires them to still keep the same
IP-address, and use the routing protocols or similar to reroute
packets from one site to another when the first site's VPN gateway
goes down. I think this is again different situation than secure crash
recovery (i.e. this is failover case, where the solutions are usually
bit different).
-- 
[email-masked]
_______________________________________________
IPsec mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ipsec

---

From: Tero Kivinen <[email-masked]>
Date: Mon, 22 Sep 2008 16:58:14 +0300

> regard to whether we have been sending traffic.

Yes. We do both. 

> unreciprocated outbound traffic?

No. The RFC leaves that completely to the implementations, but gives
two recommendations when it would be useful to do, i.e. when the
traffic has been unidirectional or when there has not been any traffic
for long time. 

> Should the RFC allow or forbid liveness checks more broadly in the
> "dead air" case where there is no traffic flowing in either
> direction? There may be some value in proactively discovering the
> peer's deadness in the dead-air case, but Yoav's concerns about
> overutilized connections also have merit.

All those depends so much about the scenario and situation where IKEv2
is used, thus those are better to leave in the implementations to take
care of (or to the configuration of the implementation).

In some scenarios it might be better to disable DPD completely. In
some cases you want to do it very quickly and often. In mobile device
cases you most likely want to do it quite quickly when the traffic
changes to have only outbound traffic, but after successful DPD you
should not another until you have new evidence that situation has
changed, just to save battery life and link resources. 

> If the RFC takes an agnostic approach to the dead-air case, we may
> still want to recommend that if an implementation allows for
> dead-air liveness checks, it should be configurable on a
> per-PAD-entry basis.

I do not think the RFC needs to mandate anything like that. Some
implementations are not really meant for generic use, thus they can
have very limited configurability, but their implementations might
have lots of hard coded support for the scenario they are used in (for
example they might disable DPDs completely if they know that the phone
is out of coverage from the network, as doing DPD in that situation
would simply waste resources by removing the SAs even when the other
end might still be up and working after the network coverage is
regained etc). 

> Or perhaps the RFC could recommend a more conservative approach to
> the dead-air case, such as backing off the interval between dead-air
> liveness checks with each successful iteration.

Backing off is good option in certain scenarios. It might not be good
option in some other situations. I think the RFC should leave this
kind of things to the implementations. The RFC should document the
protocol so that both ends can implement it correctly and two
implementations can talk to each other. It should not try to mandate
things how implementations should use those protocols. It can give
hints what kind of implementations would be useful in what kind of
scenarios, but even those might be better to leave to those documents
describing the scenarios (i.e. some 3GPP document might document what
kind of DPD behavior works best when using IKEv2 to protect something
there, as they know which behavior might be best in that scenario).
-- 
[email-masked]
_______________________________________________
IPsec mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ipsec

---

From: "Stephen Bevan" <[email-masked]>
Date: Thu, 18 Sep 2008 20:52:22 -0700

Yoav Nir writes:
 > I sent out this email about a month ago, and the response was, well,
 > underwhelming.
 > [snip]
 > So we're asking the participants of this WG again for their opinion
 > about the two (well, three if you include birth certificates) options
 > for solving the de-synchronization problem.

---

From: "Stephen Bevan" <[email-masked]>
Date: Fri, 19 Sep 2008 12:40:53 -0700

On Fri, Sep 19, 2008 at 11:56 AM, Scott C Moonen <[email-masked]> wrote:
> checks more broadly in the "dead air" case where there is no traffic flowing
> in either direction?  There may be some value in proactively discovering the
> peer's deadness in the dead-air case, but Yoav's concerns about overutilized
> connections also have merit.

If a user wants to detect of failure by sending a DPD probe every X
seconds (regardless of traffic or not) and failing over to a different
gateway if there is no response after Y tries then the user should be
able to set X and Y to whatever values they are comfortable with.  If
the RFC is going to attempt to place restrictions on that capability
for compliant systems then it needs to offer an alternative to the
same problem.  The Secure Crash Detection is not such an alternative
since it cannot help in the situation where the problem is the failure
of an intermediate node or a NIC such that the ESP never makes it to
the  peer IPsec host.
_______________________________________________
IPsec mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ipsec

---

From: "Stephen Bevan" <[email-masked]>
Date: Fri, 19 Sep 2008 13:04:48 -0700

On Fri, Sep 19, 2008 at 9:53 AM, Yoav Nir <[email-masked]> wrote:
> That's a good way, but then if your email client polls the IMAP server every
> 5 minutes, you'll only get a burst of activity every 5 minutes. Even a
> gateway running on a commodity PC has time to reboot in 5 minutes, and DPD
> will take more time than necessary (30 seconds in your case IIRC)

If the poll is only being done every 5 minutes then why would the user
care that the failover took 30 seconds or even a minute?  That is not
to say a user might not care but any argument in this direction needs
to be clear about what the user's expectations are for their topology.

> We actually sync IKE SAs between gateways, although this is getting more
> difficult, because IKEv2 requires the message counters to be synched. My
> co-authors tell me that very often, their gateways are deployed with no
> back-channel sync. We are hearing more people who want a physical separation
> between the gateways (like putting them in different parts of town) and in
> that case synching becomes much more difficult.

If the thrust of the draft is about simplifying failover then I think
it needs to explicitly address the pros and cons of synching IKE&IPsec
SAs and critically whether this analysis is altered if one assumes
that the IPsec host is also a satefull firewall.  If it is a firewall
I would expect it to be syncing regardless of IKE/IPsec and thus it is
not a matter of sync or no synch, but rather of what and how much is
being synched.   I realize that stateful firewalling is outside the
scope of IKE/IPsec and so in abstract is irrelevant.  However, for
those vendors that do combine IKE/IPsec/firewall then the usefulness
of this protocol is going to be judged in this context.
_______________________________________________
IPsec mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ipsec

---

From: "Stephen Bevan" <[email-masked]>
Date: Sun, 21 Sep 2008 13:45:56 -0700

> that case synching becomes much more difficult.

I didn't address the last sentence in my previous reply and it is an
interesting case which I'd like to understand exactly what this
scenario is and how the drafts address it since it is not mentioned in
the drafts or Wiki.  The following are some questions that I think
either/both should address to clarify this scenario :-

Are the physical separate gateways protecting the same shared network
(say via a private connection) or are they protecting  physically
independent networks (but with the same IPs)?  If the former, then at
the expense of additional configuration on the client all the gateways
could be up simultaneously providing load-sharing and obviating the
need for either protocol (unless each physically separate gateway in
reality consists of one or more gateways which are monitoring each
other).  If the gateways are protecting different physically separate
networks are they still monitoring each other such that the backup can
assume the IP of the primary on failure?  If so, why is synching
difficult if monitoring is being done?  If there is no monitoring, by
what mechanism or mechanisms does traffic from the client directed to
the primary IP get delivered to the backup when the primary fails such
that either protocol can be used and what is the expected time for the
takeover to occur?
_______________________________________________
IPsec mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ipsec