Subject: Hitless Restart 02 : 2-way, BDR, grace policy

From: Erblichs <[email-masked]>
Date: Sat, 11 May 2002 16:00:19 -0700

Hitless Restart -02 by J. Moy : Feb02 to Aug02

        Pre-note: I have NOT previously done this level of
        restart with a grace-LSA and am not currently working
        on this type of functionality..

        My assumptions are that I understand this draft and
        their are a few things that may be a bit clearer...
        Sorry ahead of time...

        1) Overview 1.:
           Should their by an explicit statement that 2-way
           DRothers link partners, not not play a role in
           hitless restart?

        2) Overview 1.:
           If you are a DRother, what does this really gain
           you?

        3) Is restart time started when the first grace-LSA
           is sent out OR when the last grace-LSA is acknowledged?

        4) 2) Op of restart router (3) If the restarting ...
           Couldn't you use nvram to store that you were the
           DR? You then wouldn't have to wait 1/2 on average
           of the hello interval.. Complication, what happens
           with demand circuits and suppressed hellos?

        5) Same section as #4.
           How do you preserve the fact that you were the BDR
           in this Draft? I actually don't really see any mention
           of the BDR...
                -- My suggestion in #4 would work as well for a
                   BDR or a DRother..

        #4 and #5) For consistency sake, these items should then
          be checked with hellos, etc..

        6) 2.1) (2) What happens if the nvram cypto seq # somehow
        is no-longer correct after the restart? I assume this
        can be changed.

        7)  2.1) A grace-LSA is originated ..
           Am I correct in assuming that the below question would be
        implicitly covered as a topolgy change and proceed to normal
        restart?

        What happens if a BDR comes up during restart (before all
        grace-LSA are acknowledged?

        8) 2.1) A grace-LSA is originated ..
        A) Conflict with 3.1 (4)b. How do you resolve a grace-LSAs that
        were xmit'ed and acked by one or more neighbors and one
        or more grace-LSAs that are in conflict with exceeding the
        local policy of time T.
        B) How do you detect that you are violating time T?
        C) Can you resend a grace-LSA with a different grace-period?
           What should a router do, if two grace-LSAs were sent
           with the same grace-periods? With different grace periods?
        D) What happens if you violate your grace-LSA value?
        E) What happens if the grace-period is exceeding long and
           router X was the DR?
        F) When do you stop re-xmiting grace-LSAs and proceed to
           normal restart? Can you spend you entire grace-period
           doing re-xmits?

        9) 2.2 (2) add a d) router that would be a helper candidate
        came up after restart was started and all grace-LSAs were
        acknowledged.

        10) Why not save xmit OSPF hellos per interface in nvram,
         so after a restart prior restart hellos can be reloaded
         and sent?

        11) #4) "or they do not recieved the grace-LSA"
                I don't understand this phrase..
            Are you talking about not reciving a ack from a router
            Y candidate?

        Thanks,
                Mitchell Erblich

---

From: Erblichs <[email-masked]>
Date: Tue, 21 May 2002 08:18:44 -0700

Group,

        I got a couple of private replies and wanted
        to clarify a few items.

        IF the system memory could be checkpointed/saved
        AND later resumed upon restart,

        then the main issues become,

           * what is the minimum of the OSPF interfaces
        DeadInterval? This is the value that the system must
        be re-booted so none its neighbors don't declare it as
        dead and free its associated resources.

            * can you save the memory in a format that
        upon resumption you could restore it in newer
        data structures? i.e. a new release.

           * how do you save the hw/sw registers, that upon
        bootup, the system behaves properly?

        Etc..

        Then grace-lsa's would not be necessary, no knowledge
        of the neigbors that you are about to perform this
        procedure would be necessary, and it would be
        backward compatible with neighboring routers.

        This work was done at Terayon a couple of years
        ago and I called it "Fast Re-start". This general
        feature was also done at Sun Microsystems and they
        call it checkpoint/resume.

        Mitchell Erblich
        ======================
        PS:  #8 from below would partially be satisfied if
        MaxAge flushing was explicitly specified for grace-LSAs..

Erblichs wrote:
>            Should their be an explicit statement that 2-way

---

From: Acee Lindem <[email-masked]>
Date: Tue, 21 May 2002 13:49:42 -0400

>         dead and free its associated resources.

Mitchell,

I guess if you go by RFC 2338, the answer is 2 (at least with our implementation
I assure that the dead interval is greater than the hello interval).
Presumably you could extend the dead interval to match the grace interval
for planned restarts. For unplanned restarts, you don't have the luxury of sending
the grace LSAs before you go down so the point it moot.

>         call it checkpoint/resume.

This is definitely a potential solution. However, if you checkpointed
your Link State database, neighbor state, and some flooding state (to
assure it is reliable) you wouldn't need any cooperation with other
routers or modifications to the base OSPF protocol. The advantage of
learning this state from the other routers in the network is that it
is much lower overhead.

--
Acee

---

From: Amit Srivastava <[email-masked]>
Date: Tue, 21 May 2002 21:41:21 -0700

Hi All,
          I would like to know the Applications which use Opaque LSA and if there is any good material on net about the
          Opaque LSA
regards
Amit

---------------------------------
Do You Yahoo!?
LAUNCH - Your Yahoo! Music Experience

---

From: Manav Bhatia <[email-masked]>
Date: Wed, 22 May 2002 10:22:15 +0530

Amit,
Apart from extensive applicability in TE, opaque LSAs are used in other
places...

Router Capability Exchange:
draft-galand-an-routing-00.txt

IPv6 in IPv4 AS deployments
draft-many-ngtrans-connect-ipv6-igp-00.txt

BGP MPLS VPNs/TE
draft-jamieson-mpls-vpn-00.txt
draft-abarbanel-idr-bgp4-te-01.txt

If you are just looking for an e.g. for an application of opaque LSA's you
can have a look at
the ospf-hitless-restart by John which use opaque LSA's to inform other
ospf routers of the
intention to do a hitless-restart (use by OSPF).

Regards,
Manav

----- Original Message -----
From: "Amit Srivastava" <[email-masked]>
To: <[email-masked]>
Sent: Wednesday, May 22, 2002 10:11 AM
Subject: Opaque LSA

|
|  Hi All,
|           I would like to know the Applications which use Opaque LSA and
if there is any good material on net about the
|           Opaque LSA
| regards
| Amit
|
|
|
| ---------------------------------
| Do You Yahoo!?
| LAUNCH - Your Yahoo! Music Experience

---

From: Acee Lindem <[email-masked]>
Date: Wed, 22 May 2002 01:54:01 -0400

> draft-galand-an-routing-00.txt

Manav,

I haven't seen this one - can you send me a pointer?

Amit,

Note that John's GPL OSPF implementation is available at http://www.ospf.org/

Thanks,
--
Acee

---

From: Manav Bhatia <[email-masked]>
Date: Wed, 22 May 2002 11:36:24 +0530

Acee,
draft-galand-an-routing-00.txt documents a way to include information on
active routers capabilities in the routing protocols.

The active routers receive code to be executed with the payload of the IP
packets ("active packets") as input. This draft uses Opaque LSAs for
distribution since the information contained in Opaque LSAs may be used
directly by OSPF or indirectly by some application wishing to distribute
information throughout the OSPF domain.

Pl. visit
http://alternic.net/drafts/drafts-g-h/draft-galand-an-routing-00.html for
more details.

Regards,
Manav

----- Original Message -----
From: "Acee Lindem" <[email-masked]>
To: <[email-masked]>
Sent: Wednesday, May 22, 2002 11:24 AM
Subject: Re: Opaque LSA

| Manav Bhatia wrote:
| > Amit,
| > Apart from extensive applicability in TE, opaque LSAs are used in other
| > places...
| >
| > Router Capability Exchange:
| > draft-galand-an-routing-00.txt
|
| Manav,
|
| I haven't seen this one - can you send me a pointer?
|
| >
| > IPv6 in IPv4 AS deployments
| > draft-many-ngtrans-connect-ipv6-igp-00.txt
| >
| > BGP MPLS VPNs/TE
| > draft-jamieson-mpls-vpn-00.txt
| > draft-abarbanel-idr-bgp4-te-01.txt
| >
| > If you are just looking for an e.g. for an application of opaque LSA's
you
| > can have a look at
| > the ospf-hitless-restart by John which use opaque LSA's to inform other
| > ospf routers of the
| > intention to do a hitless-restart (use by OSPF).
|
| Amit,
|
| Note that John's GPL OSPF implementation is available at
http://www.ospf.org/
|
| >
| > Regards,
| > Manav
| >
| > ----- Original Message -----
| > From: "Amit Srivastava" <[email-masked]>
| > To: <[email-masked]>
| > Sent: Wednesday, May 22, 2002 10:11 AM
| > Subject: Opaque LSA
| >
| >
| > |
| > |  Hi All,
| > |           I would like to know the Applications which use Opaque LSA
and
| > if there is any good material on net about the
| > |           Opaque LSA
| > | regards
| > | Amit
| > |
| > |
| > |
| > | ---------------------------------
| > | Do You Yahoo!?
| > | LAUNCH - Your Yahoo! Music Experience
| >
|
| Thanks,
| --
| Acee

---

From: Erblichs <[email-masked]>
Date: Wed, 22 May 2002 09:17:15 -0700

> learning this state from the other routers in the network is that it

I am not sure how you are catagorizing "overhead".

* Assuming like quantities of information, accessing data from physical
 memory is always faster than acessing it from a link partner. Their
 should be a minimalistic amount of work to verify correctness when
 restoring data from memory.

* Their is no unbounded acknowledgement time quantum required, like
their is for reliable grace-LSAs (2.1 : "should retransmit the
grace-LSAs until they are acknowledged"). In my assumption, this
could lead to a flushing of grace-LSAs and then the resending of
the grace-LSA with a different grace-period because one possible
"helper" had a late response. Worst case scenario..

* Grace-LSAs are opaque and opaque LSAs are "optional", not part
of the v2 OSPF spec.

* Their could be a signficant amount of work/overhead to flood
these grace-LSAs.. And then maybe reflood due to flushing or
just with a newer sequence..

* Just because grace-LSAs are ack'ed, their is no requirement
that they are guaranteed to be helpers (2.1 : " mere reception
of grace-LSAs does not imply acceptance of helper responsibilities")

Thus, I do not know why this draft does not have some form of
"grace-LSA" that DOES imply acceptance by setting some form
of acceptance flag/bit.

> for planned restarts. For unplanned restarts, you don't have the luxury
With a variation of CPR, it is possible to have various checkpoints
of the system (say once every 1 minute) for unplanned fast-restarts.

So Checkpoint/Resume (CPR) tends to be a bit complicated,
but allows the router to have a shorter down-time in any circumstance.

Mitchell Erblich
----------------------

---

From: Acee Lindem <[email-masked]>
Date: Wed, 22 May 2002 12:29:01 -0400

>  restoring data from memory.

Agreed. I was talking about the steady state overhead of checkpointing
your link state database, neighbor state, and flooding state to non-volatile
memory or a standby processor. Depending on your SW/HW architecture this may
be a perfectly valid option. My point is that this type of scheme could be
completely transparent and not require any IETF involvement.

> "helper" had a late response. Worst case scenario..

Maybe I'm misunderstanding you. Anyway, once an LSA is on a neighbor
retransmission queue I don't re-originate it each time it is
retransmitted. You are right that different routers will have different
grace termination times if one does not receive the initial grace LSA.
However, we don't live in a perfect world.

> * Grace-LSAs are opaque and opaque LSAs are "optional", not part

This is a problem. There are also major vendors that claim to
support opaque LSAs but do *not* support the type 9 (link-local LSAs). My
opinion is that they should have spoken up back when RFC 2370 was being
debated.

> "grace-LSA" that DOES imply acceptance by setting some form

If a router receives an LSA that "match" its pre-restart router LSA
it will detect that a router doesn't support graceful restart and
terminate graceful restart. This will happen quite quickly if your
SPF timers are set low and its been implemented correctly.

> but allows the router to have a shorter down-time in any circumstance.

Agreed.

--
Acee