Subject: [L2tpext] draft-galtzur-l2tpext-gr-01.txt and draft-ietf-l2tpext-failover-02.txt

From: "W. Mark Townsley" <[email-masked]>
Date: Tue, 27 Jan 2004 11:26:11 +0100

I would like to try and offer a level set here for continuing the discussion.

I see that these two drafts are trying to solve similar problems, though perhaps 
optimized differently. Fundamentally, both intend to address the restablishment 
of control connection and session state for L2TP during a failure scenario. That 
may mean failover from an active to standby RP, or the restart of an RP while 
continuing to forward packets.

For the most part, either mechanism could be made to work for L2TPv3 or L2TPv2. 
I think there are obvious benefits if we can have a single mechanism for both.

The only fundamental difference I see between the two protocols themselves which 
might affect the design of the associated restart method is that in L2TPv3 the 
Control Connection ID is *not* carried in the L2TP Data Message Header, and in 
L2TPv2 the analogous Tunnel ID *is* carried in the L2TP Data Message Header. 
Thus, v3 has the advantage over v2 in that it can restart with a different 
Control Connection ID without affecting the forwarding plane. This difference is 
tangible, but would hopefully not prohibit convergence on a common mechanism.

I believe the more fundamental reason we see two divergent solutions is that 
each is being targeted to different deployment environments.

draft-galtzur-l2tpext-gr-01.txt is targeted at the current L2TPv3 space, while 
draft-ietf-l2tpext-failover-02.txt is targeted at the more mature RFC2661 L2TPv2 
space. As such,

draft-galtzur-l2tpext-gr-01.txt is optimized for an environment which is:

- Provisioned in a relatively static manner
- Has a fairly small number of sessions per LCCE
- Largely addresses restart of a single RP while continuing to forward traffic 
on a separate dataplane

draft-ietf-l2tpext-failover-02.txt is optimized for an environment which is:

- Provisioned in a largely dynamic manner
- Has a large number of sessions per LCCE
- Addresses restart of an RP, or failover to a secondary RP or node, where some 
amount of session and control connection state may be checkpointed (e.g., beyond 
that necessary for forwarding alone), and where traffic flow may be interrupted 
during failover.

One might argue that, over time, the deployment environment of L2TPv3 will look 
more and more like that of L2TPv2. e.g., higher density, more dynamic 
provisioning via auto-discovery or NMS systems, etc. Thus, some of the scaling 
benefits of draft-ietf-l2tpext-failover-02.txt could become more relevant as well.

That said, I don't believe the draft-ietf-l2tpext-failover-02.txt has all the 
kinks worked out in it. draft-galtzur-l2tpext-gr-01.txt has the advantage of 
being quite simple, is relatively complete, and requires very little (if any) 
state checkpointing beyond that which could be gleaned from the still-active 
forwarding plane which is assumed to be present. It's perfectly adequate for 
cases where you aren't concerned about how fast the sessions recover (and if you 
have a forwarding plane that is still passing packets for fairly static 
sessions, why should you care?).

Authors, please let me know if I have not articulated the situation accurately.

Next step is to try and come up with a single method (or very close to a single 
method) that operates with both versions of L2TP.

Thanks,

- Mark

_______________________________________________
L2tpext mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/l2tpext

---

From: Vipin Jain <[email-masked]>
Date: Wed, 28 Jan 2004 02:11:33 -0800 (PST)

My comments:

I think both drafts are complete (I am attaching
draft-ietf-l2tpext-failover-03.txt for everyone's review).

However, the main trade-off seems to be the simplicity vs. scalability, as
identified as goals in in Appendix A of draft-ietf-l2tpext-failover-03.txt.

Just wondering: Why the mechanism described in
draft-ietf-l2tpext-failover-03.txt would not work (and scale) for L2TPv3 as
well?

thanks,
-- vipin

> method) that operates with both versions of L2TP.

__________________________________
Do you Yahoo!?
Yahoo! SiteBuilder - Free web site building tool. Try it!
http://webhosting.yahoo.com/ps/sb/

Network Working Group                                         Vipin Jain
Internet-Draft                                       Riverstone Networks
Expires July 2004                                                 Editor

                                                            January 2004

                Fail Over extensions for L2TP "failover"

Status of this Memo

   This document is an Internet-Draft and is subject to all provisions
   of Section 10 of RFC2026.

   This document is an Internet-Draft and is subject to all provisions
   of Section 10 of RFC2026 except that the right to produce derivative
   works is not granted.

   This document is an Internet-Draft and is NOT offered in accordance
   with Section 10 of RFC2026, and the author does not provide the IETF
   with any rights other than to publish as an Internet-Draft

   The distribution of this memo is unlimited. Please send comments to
   the L2TP mailing list ([email-masked]).

Copyright Notice

   Copyright (C) The Internet Society (2002).  All Rights Reserved.

Abstract

   L2TP is a connection-oriented protocol that has shared state between
   active endpoints. Some of this shared state is vital for operation
   but may be rather volatile in nature, such as packet sequence numbers
   used on the L2TP Control Connection. When failure of one side of a
   control connection occurs, a new control connection is created and
   associated with the old connection by exchanging information about
   the old connection. Such a mechanism is not intended as a replacement
   for an active fail over with some mirrored connection states, but as
   an aid just for those parameters that are particularly difficult to
   have immediately available. Protocol extensions to L2TP defined in
   this document are intended to facilitate state recovery, providing
   additional resiliency in an L2TP network and improving a remote
   system's layer 2 connectivity.

Jain                        expires July 2004                   [Page 1]

INTERNET DRAFT                  FAILOVER                     August 2003

Contributors

   Following is the complete list of contributors to this document.

   Paul Howard
   Juniper Networks
   Mark Townsley
   Cisco Systems
   Vipin Jain
   Riverstone Networks
   Sam Henderson
   Cisco Systems
   Ly Loi
   Tahoe Networks
   Leo Huber
   Extreme Networks
   Keyur Parikh
   Sentito Networks

Table of Contents

   Status of this Memo ..........................................    1
   1.0 Introduction .............................................    2
   2.0 Protocol Operation .......................................    3
   2.1 Pre Failover Operation ...................................    4
   2.2 Failover Recovery Process ................................    5
   2.2.1 Recovery Tunnel Establishment ..........................    6
   2.2.2 Control and Data Channel Reset .........................    6
   2.2.3 Begin Session Synchronization Timer ....................    7
   2.3 Failover Recovery Process ................................    8
   2.4 Post Failover Cleanup ....................................    9
   2.5 Procedure to Reset Only Data Channel .....................   10
   3.0 IANA Considerations ......................................   11
   4.0 Security Considerations ..................................   11
   5.0 Acknowledgements .........................................   11
   6.0 Author Addresses .........................................   11
   9.0 References................................................   13

   Appendix A ..................................................    14
   Appendix B ..................................................    15

Terminology

   Endpoint: An L2TP control connection endpoint, either LAC or LNS.

      Active Endpoint: An endpoint that is currently providing service.

      Backup Endpoint: A redundant endpoint standing by for the active endpoint.

Jain                        expires July 2004                   [Page 2]

INTERNET DRAFT                  FAILOVER                     August 2003

      Recovered Tunnel: An old tunnel that has been recovered using the
      mechanism described in this document.

      Recovery Tunnel: A new tunnel that is established only to recover an
      old tunnel.

      Failover: The action of a Backup Endpoint taking over the service of an
      Active Endpoint. This could be due to administrative action or failure
      of the Active Endpoint.

1.0 Introduction

   The goal of this draft is to aid the overall resiliency of an L2TP
   endpoint by introducing extensions to RFC 2661 [L2TP] that will
   minimize the recovery time of the L2TP layer after a failover, while
   minimizing the impact on its performance. Therefore it is assumed that
   the endpoint's overall architecture is also supportive in the
   resiliency effort.

   To ensure proper operation of a L2TP endpoint after a failover, the
   associated information of the tunnels and sessions between them must be
   correct and consistent. This includes both the configured and dynamic
   information. The configured information is assumed to be correct and
   consistent after a failover, otherwise the tunnels and sessions would
   not have been setup in the first place. The dynamic information, which
   is also referred to as stateful information, changes with the
   processing of tunnel's control and data packets. Currently, the only
   such information that is essential to the tunnel's operation is its
   sequence numbers. For the tunnel control channel, the inconsistencies
   in its sequence numbers can result in the termination of the entire
   tunnel. For tunnel sessions, the inconsistency in its sequence, when
   used, can cause significant data loss thus giving perception of "service
   loss" to the end user.

   Thus, an optimal resilient architecture that aims to minimize  "service
   loss" after a failover must make provision for the tunnel's essential
   stateful information - i.e. its sequence numbers. Currently, there are
   two options available: the first option is to ensure that the backup
   endpoint is in complete sync with the active with respect to the
   control and data sessions sequence numbers. The other option is to
   simply re-establish all the tunnels and its sessions after a failover.
   The drawback of the first option is that it adds significant
   performance and complexity impact to the endpoint's architecture,
   especially as tunnel and session aggregation increases. The drawback of
   the second option is that it increases the "service loss"  time,
   especially as the architecture scales.

Jain                        expires July 2004                   [Page 3]

INTERNET DRAFT                  FAILOVER                     August 2003

   To alleviate the above-mentioned drawbacks of the current options, this
   draft introduces a mechanism to bring the dynamic stateful information
   of a tunnel to correct and consistent state after a failure. Proposed
   mechanism, currently, defines the recovery of tunnels and sessions that
   were in established state prior to the failure.

2.0 Protocol Operation

   The control channel protocol recovery consists of four phases - pre failover,
   failover recovery, session state synchronization, and post failover cleanup.

   Pre failover operation allows an endpoint to specify its failover
   capabilities and timer values, attributes that are used when failover
   occurs.

   Failover recovery is started at the failed endpoint when it initiates
   a new L2TP tunnel (called recovery tunnel), for every old tunnel that
   needs recovery. The recovery tunnel serves four purposes: 1) It provides
   a means of authentication and a three-way handshake to ensure both ends
   agree on the failover for a given tunnel. 2) It identifies the old
   tunnel that needs recovery.  3) It identifies if the failed endpoint
   would like to recover control and/or data path. 4) It identifies
   the Ns and Nr values to be used in the recovered tunnel on both ends.
   Upon establishing the recovery tunnel, two endpoints reset their control
   and data channel, after which the new tunnel could be torn down.
   The sessions that were in established state resume traffic and new
   sessions could be established on the recovered tunnel.

   Session state synchronization process gets rid of the sessions
   that resulted into inconsistent states due to the failure on one of
   the endpoints. Two endpoints utilize FSQ/FSR messages over the
   recovered tunnel to agree upon the state of various sessions within
   the tunnel.

   Post failover cleanup concludes the failover process by doing three
   actions: 1) Cleanup the sessions that were not in established state.
   2) Teardown various tunnels established for the purpose of recovery.
   3) Stop accepting FSQ and FSR messages.

   Data channel failure without control channel failure could avoid
   recovery tunnel establishment, and indicate the reset by simply sending
   Data Channel Failure message over the existing control channel.

   2.1 Pre Failover Operation

      An endpoint that supports the failover protocol defined in this
      document MUST include Failover Capability AVP in SCCRQ or SCCRP
      during tunnel establishment.

Jain                        expires July 2004                   [Page 4]

INTERNET DRAFT                  FAILOVER                     August 2003

      Failover Capability AVP

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |M|H| rsvd  |      Length       |           Vendor Id [IETF]    |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |         Attribute Type [TBD]  |         Reserved          |D|C|
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |              Recovery Time (in milliseconds)                  |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

      The AVP MAY be hidden (the H-bit set to 0 or 1). The AVP is not
      mandatory (the M-bit MUST be set to 0).

      The D bit, when set indicates that an endpoint is capable of
      supporting its peer's data channel failure.  The C bit, when set
      indicates that an endpoint is capable of supporting its peer's
      control channel failure.

      Recovery Time is the time in milliseconds an endpoint asks its
      peer to wait before assuming the failure recovery to fail.  This
      time includes the control channel timeout period as described in
      [L2TP]. A value of zero indicates that the endpoint is not capable
      of failing over the control channel. For example, and endpoint MAY
      set it to zero if it can not preserve the state sessions within
      the tunnel.

   2.2 Failover Recovery Procedure

      Failover recovery procedure consists of three steps: 1) Recovery
      tunnel establishment 2) Control and/or data channel reset 3) Begin
      Session Synchronization timer.

      2.2.1 Recovery tunnel establishment

         Failed endpoint establishes a new tunnel, called recovery
         tunnel, for every old tunnel it wishes to recover. The purpose
         of the recovery tunnel is solely to recover the corresponding
         old tunnel. An endpoint MUST not send any control message on
         this tunnel except mentioned in this document. To indicate
         failure on its end, the recovery tunnel MUST include Tunnel
         Recovery AVP in its SCCRQ message.

Jain                        expires July 2004                   [Page 5]

INTERNET DRAFT                  FAILOVER                     August 2003

         Tunnel Recovery AVP

          0                   1                   2                   3
         0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
         |M|H| rsvd  |      Length       |           Vendor Id [IETF]    |
         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
         |         Attribute Type [TBD]  |          Reserved         |D|C|
         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
         |       Recover Tunnel Id       |    Recover Remote Tunnel Id   |
         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
         |           Session Sync Wait Time (in milliseconds)            |
         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

         This AVP MAY be hidden (the H-bit set to 0 or 1). The AVP is
         mandatory (the M-bit is set to 1).

         The D bit is set when a failed endpoint would like to recover
         the data channel for sessions within the tunnel. Where the C
         bit is set when the failed endpoint would like to recover the
         control channel.

         Recover Tunnel Id encodes the tunnel id that is subjected to
         recovery.  Similarly, Recover Remote Tunnel Id encodes the
         remote tunnel id corresponding to the old tunnel.

         Session Sync Wait Time is the time in milliseconds an endpoint
         asks its peer to wait before starting post failure operation
         related cleanup. This value is derived from the time an
         endpoint would take to clean up session inconsistencies with
         respect to resources required. A zero value, though not
         recommended, indicates that peer could start the post failure
         cleanup immediately after the tunnel is recovered.

         Upon getting an SCCRQ with Tunnel Recovery AVP the non failed
         endpoint validates Recover Tunnel Id values and responds with
         SCCRP. It MUST terminate the tunnel if Recover Tunnel Id or
         Remote Recover Tunnel Id is unknown.

         If non failed endpoint accepts the SCCRQ, it MAY include
         Suggested Control Channel Sequence AVP in the SCCRP.

Jain                        expires July 2004                   [Page 6]

INTERNET DRAFT                  FAILOVER                     August 2003

         Suggested Control Sequence AVP

          0                   1                   2                   3
         0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
         |M|H| rsvd  |      Length       |           Vendor Id [IETF]    |
         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
         |         Attribute Type [TBD]  |            Reserved           |
         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
         |        Suggested Ns           |         Suggested Nr          |
         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

         This AVP MAY be hidden (the H-bit set to 0 or 1). The AVP is
         not mandatory (the M-bit is set to 0).

         This is an optional AVP, suggesting the Ns and Nr values to be
         used by the failed endpoint. If this AVP is  present in an
         SCCRP message, the failed endpoint MUST set the Ns and Nr
         values of the recovered tunnel to the suggested values. When
         this AVP is not sent in SCCRP or not present in an incoming
         SCCRP, the Ns and Nr values for the recovery tunnel are set to
         zero. It is RECOMMENDED that the non failed endpoint suggests
         the Ns and Nr values to help avoid the interference in
         recovered tunnel's control channel with old control packets.

         To authenticate the peer during tunnel recovery, an endpoint
         MUST follow the procedure described in [L2TP] section 5.1.1
         using the same secret used to authenticate the old tunnel.  Not
         being able to authenticate could be a reason to terminate then
         new tunnel. For any reason if failed endpoint could not
         establish the new tunnel could then it MUST assume that
         recovery has failed and cleanup the tunnel and sessions within.

         Any control packet received on the tunnel that has not been
         recovered (i.e. till control channel has not been reset) is
         siliently discarded.

         The procedure described in this document does not specifically
         consider the scenario of both endpoints failing simultaneously.
         In such situations, an endpoint MAY assume that the recovery
         has failed.

      2.2.2 Control and Data Channel Reset

         Failed endpoint in Tunnel Recovery AVP (SCCRQ) indicates if it
         would like to reset control channel, and optionally data
         channel along with.  Data channel reset without control channel

Jain                        expires July 2004                   [Page 7]

INTERNET DRAFT                  FAILOVER                     August 2003

         reset uses the mechanism described in section 3.0 of this
         document.

         Control channel reset means flushing the transmit and receive
         windows, and resetting the control channel sequence numbers
         (i.e. Ns and/or Nr values). The control channel on failed
         endpoint is reset upon getting a valid SCCRP, whereas control
         path on non failed endpoint is reset upon getting a valid
         SCCCN. If an endpoint does not receive Suggested sequence
         number AVP in SCCRP then it MUST reset Ns and Nr values to
         zero. Similarly if non failed endpoint opts not to send
         suggested sequence number AVP then it MUST reset Ns and Nr
         values to zero.

         Data channel reset requires no action if data channel doesn't
         use sequence numbers. Whereas if data channel were using
         sequence numbers then the sequence numbers are reset as
         follows: - Failed endpoint MUST stop receiving and transmitting
         data packets on all sessions, before sending SCCRQ on the
         recovery tunnel.  - Failed endpoint resets Ns to zero. It also
         sets Nr from the Ns received in the first data packet after
         sending SCCCN on the recovery tunnel.  - After resetting Ns and
         Nr values, failed endpoint can start transmitting and receiving
         the data.  - Non failed endpoint reset the Nr to zero upon
         receipt of a valid SCCN.  It doesn't reset the Ns value.

         After resetting the control channel, it MUST allow sending and
         receiving various control messages including new session
         establishments.

      2.2.3 Begin Session Synchronization Timer

         Once the tunnel recovery is done i.e. control/data channel
         is(are) reset, each endpoint starts Session Synchronization
         Timer, value of which is indicated by the peer in Tunnel
         Recovery AVP. When this time expires it starts the Post
         Failover process described in section 2.4

   2.3 Session State Synchronization

      During failure it is likely to have two endpoints disagree on the
      existence/state of a session. Therefore it becomes very important
      for the two endpoints to agree upon the sessions that they are
      going to preserve in a tunnel. If failover happens while a session
      is being established or being torn down, it is possible that one
      of the endpoints consider the session in the established state
      where as its peer consider the same session to be down or non

Jain                        expires July 2004                   [Page 8]

INTERNET DRAFT                  FAILOVER                     August 2003

      existent.  Two such situations occur when an endpoint fails after
      sending:
      - A CDN message that never made to the peer.
      - An ICCN message that never made to to the peer.

      Session State inconsistency applies only for control channel
      failure not data channel failure.

      Following mechanism is proposed to identify and clear the sessions
      that exists on an endpoint but non existent on its peer.

      - After the recovery tunnel is established, the sessions that were
      not in established state are silently cleared i.e. without sending
      a CDN message to the peer.

      - Both endpoints SHOULD identify the sessions that might have been
      in inconsistent states, perhaps based on data channel inactivity.

      - After identifying the suspect sessions, an endpoint requests its
      peer the state of these sessions by sending a new message, message
      type [TBD], called as Failover Session Query (FSQ) message.  This
      message MUST include at least one Failover Session State AVPs.  An
      endpoint may send another FSQ messages on the recovered tunnel
      before getting a response for the first FSQ.  Failover Session
      State AVP is described as follows:

      While the set of established sessions is synchronized among two
      endpoints, it is possible to get an ICRQ for a session an endpoint
      beleive is already in established state. It MUST respond it with a
      CDN and clear the session on its end.

      Failover Session State AVP

      0                   1                   2                   3
      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |M|H| rsvd  |      Length       |           Vendor Id [IETF]    |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |      Attribute Type [TBD]     |            Session Id         |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |        Remote Session Id      |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

      This AVP MAY be hidden (the H-bit set to 0 or 1). The AVP is
      mandatory (the M-bit is set to 1).

      Session Id identifies the local session id sender had assigned,
      for which it would like to query the state on its peer.  Remote
      Session Id is the remote session id for the same session.

Jain                        expires July 2004                   [Page 9]

INTERNET DRAFT                  FAILOVER                     August 2003

      - If an endpoint receives an FSQ message, it responds with
      Failover Session Response (FSR) message, message type [TBD], that
      has one Failover Session State AVP for each requested AVP in FSQ.
      For each Failover Session State AVP received in FSQ message, it
      MUST find out if the Remote Session Id is a valid existing local
      session id and that it is paired with the Session Id specified in
      the message.  If it discovers that the session is non-existing it
      MUST respond back with a Failover Session State AVP in FSR message
      with Session Id field set to zero, and Remote Session Id value set
      to the received value of Session Id in the corresponding AVP in
      the FSQ message.

      - When an endpoint receives an FSR message, it MUST use the Remote
      Session Id field to identify the local session and silently
      (without sending a CDN) clean the session if Session Id is set to
      zero. Otherwise it considers the session to be in established
      state and recovered.

      - FSQ and FSR messages SHOULD not be entertained after Recovery
      time has elapsed.

      - FSQ and FSR Messages MAY include Challenge and Challenge
      Response AVPs as defined by section 5.1.1 [L2TP] to authenticate
      the peer.

   2.4 Post Failover Cleanup

      This process is triggered when Recovery time has elapsed since the
      control and data channel were reset. Cleanup process does
      following:
      - Teardown recovery tunnels if they are still established.
      - Stop receiving FSQ and FSR messages from the peer.
      - Cleanup any remaining stale sessions. Sessions that are
      responded by the peer in FSR message to be up, are not considered
      stale even when data channel inactivity is observed.
      - Other resources acquired to start failover process.

   3.0 Data Channel Only Failure

      Mechanism described in section 2.0 MUST not be used to reset the
      data channel failure alone. If an endpoint receives an SCCRQ with
      Tunnel Recovery AVP that doesn't have 'C' bit set then it MUST
      terminate the tunnel.

Jain                        expires July 2004                  [Page 10]

INTERNET DRAFT                  FAILOVER                     August 2003

      Data failure channel reset is not needed when sequence numbers are
      not used. Two endpoints reset the data channel failure by sending
      a new message, Data Channel Failure (DCF) message over the
      existing control channel.

4. IANA Considerations

   This document requires four new "AVP Attributes" and three new
   messages to be assigned through IETF Consensus [RFC2434] as indicated
   in Section 10.1 of [RFC2661]. These are:

         Failover Capability AVP (section 2.1)

         Tunnel Recovery AVP (section 2.2.1)

         Suggested Control Sequence AVP (section 2.2.1)

         Failover Session State AVP (section 2.3)

         Failover Session Query Message (FSQ) (section 2.3)

         Failover Session Response Message (FSR) (section 2.3)

         Data Channel Failure Message (section 3.0)

5. Security Considerations

   The failover mechanism described here leaves a some room (1 in 2^32)
   for an intruder to discover the old tunnel id of an existing tunnel
   by trying out various possibilities in Recovery Tunnel Id and
   Recovery Remote Tunnel Id AVP.

   It also introduces an opportunity for an intruder to spoof the
   FSQ/FSR messages and know the active sessions.

6. Acknowledgements

   Leo Huber of Extreme Networks provided valuable suggestions to help
   define the failover concept.  Ly Loi reviewd the draft and provided
   suggestions on improving it.

8. Author Information

      Vipin Jain
      Riverstone Networks
      5200 Great America Parkway
      Santa Clara, CA 95054
      Phone: +1 408.655.9115
      Email: [email-masked]

Jain                        expires July 2004                  [Page 11]

INTERNET DRAFT                  FAILOVER                     August 2003

      Paul W. Howard
      Juniper Networks
      10 Technology Park Drive
      Westford, MA 01886
      Email: [email-masked]

      Keyur Parikh
      Sentito Networks
      238 Littleton Road Suite 101
      Westford, MA 01886
      Phone: +1 978.399.0223
      Email: [email-masked]

      W. Mark Townsley
      Cisco Systems
      7025 Kit Creek Road
      PO Box 14987
      Research Triangle Park, NC 27709
      Email: [email-masked]

8. References

   [L2TP] Townsley, et. al., "Layer Two Tunneling Protocol L2TP", RFC2661

Appendix A

This section describes some design considerations that came up during
discussions when developing the proposal:

   A.1  Backward compatibility and extensibility

      -  The mechanism should be backwards compatible; i.e. it should
      not redefine existing behavior of [L2TP] compliant systems.

      - The protocol should allow a peer to detect failover capabilities
      in advance, for it to fall back to other failover mechanisms
      should peer does not support proposed failover protocol.

      - The protocol should allow future extensions to fail-over
      mechanism at ease.

Jain                        expires July 2004                  [Page 12]

INTERNET DRAFT                  FAILOVER                     August 2003

   A.2  Less failover recovery time

   The mechanism should have least possible time to recover from
   failover (target of 3-5 seconds for 30k tunnels). Specifically it
   should take following into consideration:

      - Faster recovery: by utilizing less number of messages exchanged
      to recover from failover

      - CPU intensiveness: less cpu intensive a proposal is, better are
      the changes of faster recovery

      - Parallel establishment of various tunnels: by keeping different
      tunnel reestablishments independent of one another.

   A.3  Less Payload data loss

   The mechanism should have least possible impact on data flows for
   sessions with sequencing enabled.

   A.4  Minimum interference with pre-failure control traffic

   The mechanism should define a way of clearly distinguishing the
   messages that were sent before failover from that which are sent
   after.  Specifically, it should define a mechanism that avoid
   confusion between sequence numbers that were used before and after if
   the same Tunnel Id is used.

   A.5  Simplicity

   Simpler the protocol is, better are the changes of being adopted by
   everybody. Following would help achieve this:

      - Use of existing AVPs, messages and packet formats.

      - Avoid introducing special considerations and mechanisms a new
      implementation would have to deal with.

      - Simpler post fail-over synchronization mechanism.

   A.6  Security

   The mechanism should provide a mechanism to authenticate peers when
   resynchronization is happening after a failover.

Jain                        expires July 2004                  [Page 13]

INTERNET DRAFT                  FAILOVER                     August 2003

   A.7 Scalability

   It is very important for a proposed protocol to work well for a
   scalable deployment. This includes dealing with all design
   considerations discussed above for scalable deployments, having
   thousands of tunnels or sessions or mix of the two.

   A target of 30,000 tunnels carrying 150,000 to 200,000 sessions from
   300 peers was considered during the design.

Appendix B

   Figure below outlines the the failover protocol operation for an
   example tunnel. The failover protocol does not preclude an endpoint
   from recovering multiple tunnels in parallel. It also allows an
   endpoint from sending multiple FSQs to recover quickly.

      Pre Failover Exchange (section 2.1):

      Endpoint                                             Peer
                   (assigned tid = x, failover capable)
      SCCRQ       -------------------------------------->  validate SCCRQ

                   (assigned tid = y, failover capable)
      validate    <--------------------------------------  send SCCRP
      SCCRP, etc.

      .... <after tunnel gets created, sessions are established> ....

      < This Node fails >

      Failed endpoint establishes recovery tunnel (section 2.2.1).
      Initiate recovery tunnel establishment for the old tunnel 'x':

      Failed Endpoint                                      Peer

                (assigned tid = z, Recovery AVP)
      SCCRQ     ----------------------------------->  Detects failover
              (recover tid = x, recover remote tid = y)  validate SCCRQ

               (assigned tid = w, opt. Suggested Seqence AVP)
      validate <-----------------------------------   send SCCRP
      SCCRP    (recover tid = y, recover remote tid = x)
      reset Ns/Nr

      SCCN      ----------------------------------->  validate and reset
                                                      the old tunnel.

Jain                        expires July 2004                  [Page 14]

INTERNET DRAFT                  FAILOVER                     August 2003

      Start session synchronization timer (section 2.2.3)

      Session states are synchronized both endpoints may send FSQs and
      cleanup stale sessions (section 2.3)

                 (FSS AVP for sessions s1, s2, s3..)
      send FSQ  -------------------------------------> compute the state
                                                       of sessions in FSQ

                 (FSS AVP for sessions s1, s2, s3...)
      deletes  <-------------------------------------- send FSR
      stale sessions, if any

                 (FSS AVP for sessions s7, s8, s9...)
      compute  <-------------------------------------- send FSQ
      the sate of
      sessions in FSQ

                 (FSS AVP for sessions s7, s8, s9...)
      send FSR --------------------------------------> delete stale sessions,
                                                       if any

      Session Sync Time expires, Start Post Failover process (section 2.4)

      Cleanup and terminate tid = 'z'
      StopCCN  --------------------------------------> Cleanup 'w'

      Stop receiving and sending FSQs and FSRs.
      Cleanup sessions that are still stale within the tunnel.

Jain                        expires July 2004                  [Page 15]

---

From: Vipin Jain <[email-masked]>
Date: Wed, 28 Jan 2004 02:11:33 -0800 (PST)

My comments:

thanks,
-- vipin

                                                            January 2004

Status of this Memo

Copyright Notice

Abstract

Contributors

   Paul Howard
   Juniper Networks
   Mark Townsley
   Cisco Systems
   Vipin Jain
   Riverstone Networks
   Sam Henderson
   Cisco Systems
   Ly Loi
   Tahoe Networks
   Leo Huber
   Extreme Networks
   Keyur Parikh
   Sentito Networks

Table of Contents

Terminology

1.0 Introduction

2.0 Protocol Operation

      Failover Capability AVP

       0                   1                   2                   3

         Tunnel Recovery AVP

          0                   1                   2                   3

          0                   1                   2                   3

      0                   1                   2                   3

4. IANA Considerations

6. Acknowledgements

8. Author Information

      Vipin Jain
      Riverstone Networks

      Paul W. Howard
      Juniper Networks
      10 Technology Park Drive
      Westford, MA 01886

      Keyur Parikh
      Sentito Networks

      W. Mark Townsley
      Cisco Systems
      7025 Kit Creek Road
      PO Box 14987

8. References

Appendix A

   A.5  Simplicity

   A.6  Security

   A.7 Scalability

Appendix B

      Endpoint                                             Peer

      < This Node fails >

      Failed Endpoint                                      Peer