Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
From: Russ White <[email-masked]>
To: Randy Bush <[email-masked]>
Cc: Alex Zinin <[email-masked]>, John Engelhart <[email-masked]>, Vijay Gill <[email-masked]>, [email-masked]
Date: Thu, 29 Mar 2001 06:12:34 -0500 (EST)

And I, in turn, apologize, if I've been a bit rough.... This has
been a bit of a rough thread, and I'm not normally so rough in
email (as someone else has already pointed out to me).

:-)

Russ

On Thu, 29 Mar 2001, Russ White wrote:

> 
> Apology accepted. Not that I was upset (I have much thicker skin
> than that).
> 
> :-)
> 
> Russ
> 
> On Thu, 29 Mar 2001, Randy Bush wrote:
> 
> > russ,
> > 
> > my apologies for using a hammer when words failed.  my frustration at my
> > poor ability to communicate is not helped by raising my voice.  sorry.
> > 
> > randy
> > 
> 
> _____________________________
> [email-masked] <>< Grace Alone
> 
> 

_____________________________
[email-masked] <>< Grace Alone

Date: Thu, 29 Mar 2001 06:05:22 -0500 (EST)
From: Russ White <[email-masked]>
Reply-To: Russ White <[email-masked]>
To: Randy Bush <[email-masked]>
Cc: Alex Zinin <[email-masked]>, John Engelhart <[email-masked]>, Vijay Gill <[email-masked]>, [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

Apology accepted. Not that I was upset (I have much thicker skin
than that).

:-)

Russ

On Thu, 29 Mar 2001, Randy Bush wrote:

> russ,
> 
> my apologies for using a hammer when words failed.  my frustration at my
> poor ability to communicate is not helped by raising my voice.  sorry.
> 
> randy
> 

_____________________________
[email-masked] <>< Grace Alone

From: Randy Bush <[email-masked]>
To: Russ White <[email-masked]>
Cc: Alex Zinin <[email-masked]>, John Engelhart <[email-masked]>, Vijay Gill <[email-masked]>, [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Date: Thu, 29 Mar 2001 03:22:40 -0500
Sender: [email-masked]

russ,

my apologies for using a hammer when words failed.  my frustration at my
poor ability to communicate is not helped by raising my voice.  sorry.

randy

Date: Wed, 28 Mar 2001 19:53:09 -0800
From: Alex Zinin <[email-masked]>
Reply-To: Alex Zinin <[email-masked]>
Organization: Cisco Systems
To: Randy Bush <[email-masked]>
Cc: Russ White <[email-masked]>, John Engelhart <[email-masked]>, Vijay Gill <[email-masked]>, [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

Randy:

> stop telling me what you think i should want.

Never did. The point is: you don't want it---don't use it,
but there are people who do.

> keep it simple.  make it more reliable.  make it perform well.

Constantly working on this.

Thanks!

Alex.

To: Russ White <[email-masked]>
Cc: Alex Zinin <[email-masked]>, John Engelhart <[email-masked]>, Vijay Gill <[email-masked]>, [email-masked], [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt 
Date: Wed, 28 Mar 2001 19:55:29 -0800
From: Enke Chen <[email-masked]>
Sender: [email-masked]

Russ,

> Date: Wed, 28 Mar 2001 22:01:38 -0500 (EST)
> From: Russ White <[email-masked]>
> Reply-To: Russ White <[email-masked]>
> To: Alex Zinin <[email-masked]>
> Cc: John Engelhart <[email-masked]>, Vijay Gill <[email-masked]>,
> 	[email-masked], [email-masked]
> Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
> In-Reply-To: <[email-masked]>
> Message-ID: <[email-masked]>
> 
> 
> My sentiments exactly. Doing graceful manual restart will cover
> 99.9% of the requirements for handling unexpected events, and it
> doesn't make sense to leave out that .01% just because we don't
> think we will ever need/want it.

I can not agree with your statement of 99.9% vs .01%. I believe
that the handling of unexpected software crashes is much more
important than described (if not more important than planned events).

I spent quite a few years on network engineering and operations.
Based on my experience, customers usually can handle planned events
much better as customers are notified in advance.  Unexpected traffic
disruptions, however, can result in much severer impact and
dissatisfaction from customers.

It seems to me that whatever we can do to minimize the traffic impact
by planned events and unplanned software crashes (as they do occur,
unfortunately) will be helpful. I believe that idea of the proposed
graceful/hitless restart is a right step toward the goal.

-- Enke

> 
> Both drafts will handle either--we should drop the entire 'crash
> vs maual restart' issue and look at the drafts as they stand, to
> see which parts of which we like, and which provides better
> state/better capabilities into the future, from a technical point
> of view.
> 
> :-)
> 
> Russ
> 
> On Wed, 28 Mar 2001, Alex Zinin wrote:
> 
> > 
> > John:
> > 
> > > My $0.02.
> > 
> > > Everyone is trying to solve the corner case of "What do you do when the
> > > router has an exception case (crash, bug, hardware failure, whatever)
> > > with hitless restart?"
> > 
> > Unfortunately, this is the impression people get when reading this
> > thread. Even though the discussion was mostly about the unplanned
> > restart, it does not mean that the solution itself is directly
> > targeted to it.
> > 
> > > My answer- Don't.  Build it in, but don't force it.
> > 
> > Exactly the point. The two extremes we should try to avoid
> > are 1) Just don't care about unplanned and only keep planned
> > in mind, and 2) Solve unplanned and force people to use it.
> > 
> > My point is: 1) the solution must provide equally good functionality
> > for both planned and unplanned restart cases, and
> > 2) the implementations must allow the administrator to disable
> > support for unplanned restart if he/she believes this is necessary
> > at the moment.
> > 
> > I guess I don't have to say that it would be hardly acceptable for
> > me if solution provided questionable support for unplanned restart.
> > 
> > Thanks,
> > 
> > Alex.
> > 
> > 
> > 
> 
> _____________________________
> [email-masked] <>< Grace Alone
> 
> 

Date: Wed, 28 Mar 2001 22:53:11 -0500 (EST)
From: Russ White <[email-masked]>
Reply-To: Russ White <[email-masked]>
To: Randy Bush <[email-masked]>
Cc: Alex Zinin <[email-masked]>, John Engelhart <[email-masked]>, Vijay Gill <[email-masked]>, [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

Randy:

I understand your points. You are opposed to both drafts,
then? You don't want graceful restart under any conditions?

Russ

On Wed, 28 Mar 2001, Randy Bush wrote:

> > My sentiments exactly. Doing graceful manual restart will cover 99.9% of
> > the requirements for handling unexpected events
> 
> 99% of the unexpected events are crashes of complex and hence unreliable
> boxes with too many bells and whistles and not enough hard tested solid code
> that supports all the cards and runs at line rate.
> 
> this conversation is a wonderful vingette, "we know what's best for the
> customer" while the customers are desperately trying to find ear-wax removal
> tools to send vendors for christmas, birthdays, or any and every occasion.
> 
> stop telling me what you think i should want.  observe that i have already
> followed tli's advice and voted with my dollars.  and my and others' votes
> seem to have been noted on bottom lines.  i still, despite much evidence to
> the contrary, hope it will percolate from the bottom line to the product
> line.
> 
> keep it simple.  make it more reliable.  make it perform well.  please!  i
> want to buy it and like it.
> 
> randy
> 

_____________________________
[email-masked] <>< Grace Alone

From: Randy Bush <[email-masked]>
To: Russ White <[email-masked]>
Cc: Alex Zinin <[email-masked]>, John Engelhart <[email-masked]>, Vijay Gill <[email-masked]>, [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Date: Wed, 28 Mar 2001 22:30:00 -0500
Sender: [email-masked]

> My sentiments exactly. Doing graceful manual restart will cover 99.9% of
> the requirements for handling unexpected events

99% of the unexpected events are crashes of complex and hence unreliable
boxes with too many bells and whistles and not enough hard tested solid code
that supports all the cards and runs at line rate.

this conversation is a wonderful vingette, "we know what's best for the
customer" while the customers are desperately trying to find ear-wax removal
tools to send vendors for christmas, birthdays, or any and every occasion.

stop telling me what you think i should want.  observe that i have already
followed tli's advice and voted with my dollars.  and my and others' votes
seem to have been noted on bottom lines.  i still, despite much evidence to
the contrary, hope it will percolate from the bottom line to the product
line.

keep it simple.  make it more reliable.  make it perform well.  please!  i
want to buy it and like it.

randy

Date: Wed, 28 Mar 2001 22:01:38 -0500 (EST)
From: Russ White <[email-masked]>
Reply-To: Russ White <[email-masked]>
To: Alex Zinin <[email-masked]>
Cc: John Engelhart <[email-masked]>, Vijay Gill <[email-masked]>, [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

My sentiments exactly. Doing graceful manual restart will cover
99.9% of the requirements for handling unexpected events, and it
doesn't make sense to leave out that .01% just because we don't
think we will ever need/want it.

Both drafts will handle either--we should drop the entire 'crash
vs maual restart' issue and look at the drafts as they stand, to
see which parts of which we like, and which provides better
state/better capabilities into the future, from a technical point
of view.

:-)

Russ

On Wed, 28 Mar 2001, Alex Zinin wrote:

> 
> John:
> 
> > My $0.02.
> 
> > Everyone is trying to solve the corner case of "What do you do when the
> > router has an exception case (crash, bug, hardware failure, whatever)
> > with hitless restart?"
> 
> Unfortunately, this is the impression people get when reading this
> thread. Even though the discussion was mostly about the unplanned
> restart, it does not mean that the solution itself is directly
> targeted to it.
> 
> > My answer- Don't.  Build it in, but don't force it.
> 
> Exactly the point. The two extremes we should try to avoid
> are 1) Just don't care about unplanned and only keep planned
> in mind, and 2) Solve unplanned and force people to use it.
> 
> My point is: 1) the solution must provide equally good functionality
> for both planned and unplanned restart cases, and
> 2) the implementations must allow the administrator to disable
> support for unplanned restart if he/she believes this is necessary
> at the moment.
> 
> I guess I don't have to say that it would be hardly acceptable for
> me if solution provided questionable support for unplanned restart.
> 
> Thanks,
> 
> Alex.
> 
> 
> 

_____________________________
[email-masked] <>< Grace Alone

Date: Wed, 28 Mar 2001 10:07:04 -0800
From: Alex Zinin <[email-masked]>
Reply-To: Alex Zinin <[email-masked]>
Organization: Cisco Systems
To: John Engelhart <[email-masked]>
Cc: Vijay Gill <[email-masked]>, <[email-masked]>, <[email-masked]>
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

John:

> My $0.02.

> Everyone is trying to solve the corner case of "What do you do when the
> router has an exception case (crash, bug, hardware failure, whatever)
> with hitless restart?"

Unfortunately, this is the impression people get when reading this
thread. Even though the discussion was mostly about the unplanned
restart, it does not mean that the solution itself is directly
targeted to it.

> My answer- Don't.  Build it in, but don't force it.

Exactly the point. The two extremes we should try to avoid
are 1) Just don't care about unplanned and only keep planned
in mind, and 2) Solve unplanned and force people to use it.

My point is: 1) the solution must provide equally good functionality
for both planned and unplanned restart cases, and
2) the implementations must allow the administrator to disable
support for unplanned restart if he/she believes this is necessary
at the moment.

I guess I don't have to say that it would be hardly acceptable for
me if solution provided questionable support for unplanned restart.

Thanks,

Alex.

From: Randy Bush <[email-masked]>
To: Ben Black <[email-masked]>
Cc: Srihari Ramachandra <[email-masked]>, Vijay Gill <[email-masked]>, Alex Zinin <[email-masked]>, Yakov Rekhter <[email-masked]>, [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Date: Tue, 27 Mar 2001 23:35:20 -0500
Sender: [email-masked]

> This has got to be one of the most absurd conversations I've seen on a 
> technical list in some time (note, I don't read NANOG).  Why are we
> talking about routers GUESSING at why they crashed?  Falling back to
> old images and HOPING they work?  How many iterations of this process
> before someone figures out the box isn't coming back and they just
> wasted 20 minutes waiting for the magical hitless restart to work?

talking to jay today, i realized:

i can not remember seeing an igp crash without it being just a part of a
whole router crash.  but if the vendors add a bunch of useless bells and
whistles like this, it should gunk up the igps enough so we will have igp
crashes.  then it will have justified itself.  full employment for vendor
embellishers!

randy

Date: Tue, 27 Mar 2001 19:53:49 -0600
From: [email-masked]
To: [email-masked]
Cc: Russ White <[email-masked]>, Avi Freedman <[email-masked]>, Vijay Gill <[email-masked]>, [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

> > But why _preclude_ it? If you don't trust it, turn it off. If you
> > do, turn it on. I can't understand why you would want to prevent
> > someone else from using it just because you don't trust it.
> 
> Because it presumes that router software vendors IMPLEMENT the *dont do it*
> knob correctly, and I am tired of presuming things like that. 
> Does "no service lmi" ring a bell?

this is a low blow :P

I have routers that crash when a 32 bit tcp session timer
rolls over.  that doesn't mean I'm going to stop using telnet/ssh 
on the boxes and disable bgp.

> 
> Alex
> 
> 
> > 
> > Russ
> > 
> > On Tue, 27 Mar 2001, Avi Freedman wrote:
> > 
> > > 
> > > I think the thing that may be missing is that hardware failures are
> > > an order of magnitude less of a problem than software failures.  If
> > > the goal is to deal with hardware failures, some of the solutions
> > > under discussion sound interesting - but to deal with software crashes,
> > > I agree, it's scary to even think that someone might implement such
> > > things.
> > > 
> > > Avi
> > > 
> > > > On Mon, 26 Mar 2001, Russ White wrote:
> > > > 
> > > > > I would suggest that we not preclude restarts due to crashes when
> > > > > deciding the best way to do this. Painting ourselves into a
> > > > > corner based on the _current_ network operations and hardware
> > > > > capabilities is akin to giving yourself a 32 bit address space
> > > > > for the entire world....
> > > > 
> > > > Russ, just because it can be done technically doesn't make it a good idea.
> > > > Random crashes are _bad_, more so when they are unexplained and no matter
> > > > how good we can make a technical solution, it doesn't mean that the
> > > > underlying problem it is trying to solve is the right problem. Maybe you
> > > > are addressing the symptoms, instead of the cause.
> > > > 
> > > > /vijay
> > > > 
> > > > 
> > > > 
> > > > 
> > > 
> > 
> > _____________________________
> > [email-masked] <>< Grace Alone
> > 
> > 
> > 
> > 
> 

-- 
Edward Crabbe
=============
Sr. Network Architect  
Backbone Group
Exodus Communications
-------------
v0x: 408 346 1544

Date: Tue, 27 Mar 2001 19:48:35 -0600
From: [email-masked]
To: Ben Black <[email-masked]>
Cc: Srihari Ramachandra <[email-masked]>, Vijay Gill <[email-masked]>, Alex Zinin <[email-masked]>, Yakov Rekhter <[email-masked]>, [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

> So far the vendor arguments here have been:
> 
> 1) You might want it later, so let's require it now.
> 
> 	This is obviously dubious since the customers have clearly stated
>         the feature they actually want and why.

'uhm, we want the boxes to stop crashing.'

I agree, that doesn't mean that I don't want to see interesting 
features deprecated because the box still won't stay up for more
than six months.

> 
> 2) If we add enough knobs, you can configure this to work, maybe.
> 
> 	See my first paragraph.

what are you talking about dude?

> These are not compelling.  If there really is a compelling argument
> for investing all the time and valuable developer resources on designing
> and implementing a feature people have not only not requested, but
> specifically stated they do not want, then let's see them.  In the mean
> time, can we focus on the requirements that have actually been 
> articulated by the people who BUY and USE the routers?
> 
> Hitless restart on demand, nothing else.

I can agree with that.

> 
> 
> Ben
> 
> On Tue, Mar 27, 2001 at 09:18:03AM -0800, Srihari Ramachandra wrote:
> > 
> > 
> > > Case B is even more interesting. The software crashed. Ok, so now I'm
> > > supposed to believe that crashing software can be fixed by adding MORE
> > > software and protocol extentsions, even though the original software
> > > didn't work well enough not to crash in the first place? 
> > 
> > If there is an option of falling back to a previous version of the
> > software (just after a crash), don't you think one could use the restart
> > functionality to achieve minimal impact.
> > 
> > srihari...
> > 
> > 
> > 

-- 
Edward Crabbe
=============
Sr. Network Architect  
Backbone Group
Exodus Communications
-------------
v0x: 408 346 1544

Date: Tue, 27 Mar 2001 20:55:01 -0500 (EST)
From: Russ White <[email-masked]>
Reply-To: Russ White <[email-masked]>
To: Danny McPherson <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt,  draft-ietf-idr-restart-00.txt
Sender: [email-masked]

>   o "Graceful Restart Mechanism for BGP" 
>     (draft-ietf-idr-restart-00.txt), which seems to have found 
>     it's way onto the WG item page and needs the underlying IGP 
>     support before it's interesting.  Of course, the last IDR
>     charter milestone entry is Jun 97.  "Charter Bashing" was on
>     the Minneapolis agenda, though I'm I missed the meeting, the
>     minutes haven't been posted yet, and I'm not sure what the 
>     outcome was.

As I recall, there was no competing entry in the fray there, and
the general concensus was that it was interesting enough work to
go forward. It's known by the chairs that this draft isn't useful
unless the igp drafts go through.

>   o "Restart signaling for ISIS" (draft-shand-ISIS-restart-00.txt),
>     which discusses restart and resync for IS-IS.  
> 
>     The IS-IS WG charter is quite dated as well...

There wasn't an isis meeting at minneapolis, so this wasn't
discussed, nor was the charter bashed. I did discuss this with
several people off line, and the interest seems to be there from
what I recall.

> I'll not argue the merits of the two proposals (type 9's
> versus new bits in existing packets, resync, nsync, fairy
> dust, etc..), as I think they've pretty much been beat to
> death.

Actually, I think very little of the technical merit of either
draft has been discussed. Out of the entire thread that I've seen
thus far (and participated in), only about 1 message out of five
is technical. The rest seems to be worries about router crash
reasons, etc.--it's not focused on what a good protocol mechanism
would look like.

> Certainly, requirements documents would help to scope this
> discussion, as would up to date charters and milestones.  
> After all, I'd hope an associated requirements document would
> specify whether the additional functionality of Alex's
> proposal is interesting to the community.

Agreed. A requirements doc would go far towards bringing out what
is really wanted here, and then we could pull pieces from each
draft as needed to fill those requirements.

> IMO, if we're planning to implement any of this, we might as
> well go ahead and provide a full set of capabilities (not
> that any one of them MUST be enabled in a network, mind you),
> as well as a common framework and architecture for
> NSF/headless routing.

Agreed.

Russ

_____________________________
[email-masked] <>< Grace Alone

Date: Tue, 27 Mar 2001 20:44:59 -0500 (EST)
From: Russ White <[email-masked]>
Reply-To: Russ White <[email-masked]>
To: [email-masked]
Cc: Alex Zinin <[email-masked]>, Ben Black <[email-masked]>, Srihari Ramachandra <[email-masked]>, Vijay Gill <[email-masked]>, Yakov Rekhter <[email-masked]>, [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

> >  We do have a requirement now. It has already been
> >  mentioned that not all customers are subscribed to the list,
> >  also it is not a prerequisite for our customers to be
> >  IETF-active or willing to speak up in order to provide us
> >  with their requirements.
> 
> Since this had already been asked but never answered - Who
> are those customers that are asking for it? Lets be very
> specific? Are they also the same people who are claiming that
> 7513 can have 250 BGP sessions taking peering and a few full
> views of the internet?

How exactly does denigrating unknown parties help this at
all? Somehow I thought this was supposed to be a technical
discussion?

> Will it disable the mechanism or the manifestations of the
> mechanism? Would disabling of the mechanism on a single
> device affect how the other devices behave?

It _already_ disables the mechanisms _and_ the code path in the
implementation we have been testing in the lab.

Russ
_____________________________
[email-masked] <>< Grace Alone

To: [email-masked], [email-masked]
Cc: [email-masked], [email-masked], [email-masked], [email-masked], [email-masked], [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt,draft-ietf-idr-restart-00.txt
From: Yasuhiro Ohara <[email-masked]>
Date: Wed, 28 Mar 2001 10:37:52 +0900
Lines: 16
Sender: [email-masked]

ruwhite> There are some platforms on which it just won't ever be
ruwhite> implemented in hardware. Smaller edge boxes that you'll never be
ruwhite> willing to pay the price of having two processors with the other
ruwhite> necessary components in them.

Is that really running OSPF or BGP ? :-)
But..

ruwhite>  It's just not cost effective to use
ruwhite> the same solution in every place--we need a spectrum of solutions
ruwhite> available.

I am agree with this. More functionality more better.

    yasu

Date: Tue, 27 Mar 2001 20:27:35 -0500 (EST)
From: Russ White <[email-masked]>
Reply-To: Russ White <[email-masked]>
To: Yasuhiro Ohara <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked], [email-masked], [email-masked], [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt,draft-ietf-idr-restart-00.txt
Sender: [email-masked]

There are some platforms on which it just won't ever be
implemented in hardware. Smaller edge boxes that you'll never be
willing to pay the price of having two processors with the other
necessary components in them. It's just not cost effective to use
the same solution in every place--we need a spectrum of solutions
available.

Protocol level graceful/hitless restart should one of the items
available in that spectrum.

:-)

Russ

On Wed, 28 Mar 2001, Yasuhiro Ohara wrote:

> steve.buchko> That is another option.  There is a whole spectrum of
> steve.buchko> redundancy models one could implement, ranging from
> steve.buchko> "none at all" to processors running in total
> steve.buchko> lock-step execution (personally, I don't believe in
> steve.buchko> lock-step, since it only protects against hardware
> steve.buchko> failures).  But the more state you need to
> steve.buchko> propogate between processors, the harder it is to
> steve.buchko> implement.  I think hitless/graceful restart is at a
> steve.buchko> very nice point on the spectrum, striking a
> steve.buchko> good balance between reliability, and implementation
> steve.buchko> complexity .... it allows for graceful continuation of
> steve.buchko> forwarding while minimizing the amount of
> steve.buchko> state information that needs to be propogated between
> steve.buchko> the processors.
> 
> So you are saying that "It is harder to implement by hardware. Let's
> implement it by software", isn't it ? ;p)
> 
> Yes, I agree with you. It will be harder to implement by hardware
> than by software. But someday, we must implement it by hardware,
> right?
> 
> 	yasu
> 

_____________________________
[email-masked] <>< Grace Alone

To: [email-masked]
Cc: [email-masked], [email-masked], [email-masked], [email-masked], [email-masked], [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt,draft-ietf-idr-restart-00.txt
From: Yasuhiro Ohara <[email-masked]>
Date: Wed, 28 Mar 2001 09:07:16 +0900
Lines: 24
Sender: [email-masked]

steve.buchko> That is another option.  There is a whole spectrum of
steve.buchko> redundancy models one could implement, ranging from
steve.buchko> "none at all" to processors running in total
steve.buchko> lock-step execution (personally, I don't believe in
steve.buchko> lock-step, since it only protects against hardware
steve.buchko> failures).  But the more state you need to
steve.buchko> propogate between processors, the harder it is to
steve.buchko> implement.  I think hitless/graceful restart is at a
steve.buchko> very nice point on the spectrum, striking a
steve.buchko> good balance between reliability, and implementation
steve.buchko> complexity .... it allows for graceful continuation of
steve.buchko> forwarding while minimizing the amount of
steve.buchko> state information that needs to be propogated between
steve.buchko> the processors.

So you are saying that "It is harder to implement by hardware. Let's
implement it by software", isn't it ? ;p)

Yes, I agree with you. It will be harder to implement by hardware
than by software. But someday, we must implement it by hardware,
right?

	yasu

Date: Tue, 27 Mar 2001 18:31:15 -0500
From: "Steve Buchko" <[email-masked]>
Organization: Alcatel CID
To: Yasuhiro Ohara <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked], [email-masked], [email-masked], [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt,draft-ietf-idr-restart-00.txt
Sender: [email-masked]

Yasuhiro Ohara wrote:
> What is the case of hardware failure not caused by a bug ?
> I can't guess...(and guessing that the bug we calls...)

Hardware (RAM, processors, any of the tens to hundreds of components on a
card) can fail for all sorts of reasons.  It could be defective components
provided by the chip supplier, bad solder joints on the board, ESD damage by
a careless installer, etc. etc.  That is why hardware redundancy is so
important in a truely fault-tolerant system.  The cool thing is, a
multi-processor redundancy model can protect against both hardware and
software failures.

> If the backup'ed module can help so fast, why the hitless/graceful
> restarts can be made inside the router itself ?

That is another option.  There is a whole spectrum of redundancy models one
could implement, ranging from "none at all" to processors running in total
lock-step execution (personally, I don't believe in lock-step, since it only
protects against hardware failures).  But the more state you need to
propogate between processors, the harder it is to implement.  I think
hitless/graceful restart is at a very nice point on the spectrum, striking a
good balance between reliability, and implementation complexity .... it
allows for graceful continuation of forwarding while minimizing the amount of
state information that needs to be propogated between the processors.

Cheers,
steve

To: [email-masked]
Cc: [email-masked], [email-masked], [email-masked], [email-masked], [email-masked], [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
From: Yasuhiro Ohara <[email-masked]>
Date: Wed, 28 Mar 2001 08:07:27 +0900
Lines: 34
Sender: [email-masked]

steve.buchko> It was pointed out during the OSPF meeting last week
steve.buchko> that a processor might fail through no fault of routing.
steve.buchko> The restart could be a result of a hardware failure on
steve.buchko> the control module, or it could be caused by a bug in a
steve.buchko> software process totally unrelated to routing.  In such
steve.buchko> scenarios, it would be beneficial to be able to maintain
steve.buchko> forwarding context - avoiding huge churn in the IGP
steve.buchko> along with the subsequent churn in BGP - especially if
steve.buchko> you are dealing with a hardware architecture that can
steve.buchko> bring a routing processor back
steve.buchko> online in milliseconds.  

What is the case of hardware failure not caused by a bug ?
I can't guess...(and guessing that the bug we calls...)

steve.buchko> For example, there is a huge installed base of products
steve.buchko> out there that implement an active-standby redundancy
steve.buchko> model.  They work extremely well, getting networks close
steve.buchko> to, if not actually achieving, the much-coveted
steve.buchko> five-9's reliability (and yes, they manage to do so in
steve.buchko> the presence of hundreds, even thousands of state
steve.buchko> changes per second).  Unfortunately, these
steve.buchko> architectures don't work so well with protocols like
steve.buchko> OSPF, that insist on completely flushing their state
steve.buchko> upon detecting a reset of the communications
steve.buchko> link with their peer.

If the backup'ed module can help so fast, why the hitless/graceful
restarts can be made inside the router itself ?

I still don't see the reason of doing hitless/graceful restart by
protocol level...

    yasu

To: BGP mailing list <[email-masked]>
From: Danny McPherson <[email-masked]>
Reply-To: [email-masked]
Subject: Re: Confed question (RFC 3065) 
Date: Tue, 27 Mar 2001 15:48:00 -0700
Sender: [email-masked]

Technically, I guess it's still correct as "confederation
identifier", as defined in RFC 1965 [and intuitively], should
represent the "confederation of ASs".

Of course, I did intend to change it in order to make things more 
clear in the update, though it's now apparent I missed it.  

If there's ever another update (which seems unlikely at this point)
it will be clarified.

-danny

> Am I missing something obvious here?  From RFC 3065:
> 
> b) When a given BGP speaker advertises the route to a BGP speaker
>       located in a neighboring autonomous system that is a member of the
> 
>       local autonomous system confederation, then the advertising
>       speaker shall update the AS_PATH attribute as follows:
> 
>       1) ....
> 
>       2) if the first path segment of the AS_PATH is not of type
>          AS_CONFED_SEQUENCE the local system shall prepend a new path
>          segment of type AS_CONFED_SEQUENCE to the AS_PATH, including
>          its own confederation identifier in that segment.
> 
> 
> Should the confederation identifier in the above para be interpreted as
> AS number?
> 
> thanks
> 
> VJ
> 
> 
> 
> 

To: [email-masked], [email-masked], [email-masked]
From: Danny McPherson <[email-masked]>
Reply-To: [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt,  draft-ietf-idr-restart-00.txt
Date: Tue, 27 Mar 2001 15:35:23 -0700
Sender: [email-masked]

[Folks, please note that I'm trying to discuss the restart 
issue in general, and as such, have cross posted this message
to three lists.  Replies with comments specific to one of the
three should be addressed solely to that list.]

As I see this work happening in OSPF, IS-IS & IDR, I think 
it's pertinent to take a step back for a moment.  So, what 
work has been [individually] submitted to the IETF at this
point regarding restart mechanisms:

IDR:

  o "Graceful Restart Mechanism for BGP" 
    (draft-ietf-idr-restart-00.txt), which seems to have found 
    it's way onto the WG item page and needs the underlying IGP 
    support before it's interesting.  Of course, the last IDR
    charter milestone entry is Jun 97.  "Charter Bashing" was on
    the Minneapolis agenda, though I'm I missed the meeting, the
    minutes haven't been posted yet, and I'm not sure what the 
    outcome was.

    I think an overhaul of the IDR charter is necessary.

IS-IS:

  o "Restart signaling for ISIS" (draft-shand-ISIS-restart-00.txt),
    which discusses restart and resync for IS-IS.  

    The IS-IS WG charter is quite dated as well...

OSPF:

  o "OSPF Restart Signaling" (draft-ietf-ospf-restart-01.txt), 
    which was the first of the OSPF proposals and triggered 
    [unfortunately] competing proposals to provide this 
    functionality.  This work falls into the "We'll accept 
    anything as a WG item" category".

  o "Hitless OSPF Restart" (draft-ietf-ospf-hitless-restart-00.txt),
    which is John's proposal and falls into the "we'll accept 
    anything as a WG item" category.  I guess this as well is 
    one of the benefits of having a charter with goals and 
    milestones as recent as Jun 98.  

  o I think an overhaul of the OSPF charter is necessary (have I
    missed something in this area?).

I'll not argue the merits of the two proposals (type 9's versus new 
bits in existing packets, resync, nsync, fairy dust, etc..), as I 
think they've pretty much been beat to death.  However, regardless of 
what some of the author(s) have said, I feel they could be merged into 
a single document and consensus reach in the WG based on the technical 
merits of each.  If a consensus can't be reached, I'd rather not see 
either progress as standards track documents (or informational, for 
that matter), and hope the IESG feels this way as well.

On a general note...  Having just become a chair and put through the 
WG Charter ringer myself, I find it surprising the IESG is being as 
loose with these WG as seems apparent.  Not speaking to restart work 
directly, but perhaps tightening things up a bit would eliminate some 
of the cruft (i.e., charters, goals & milestones and the like)?  

Certainly, requirements documents would help to scope this discussion, 
as would up to date charters and milestones.  After all, I'd hope an 
associated requirements document would specify whether the additional 
functionality of Alex's proposal is interesting to the community.  

IMO, if we're planning to implement any of this, we might as well go 
ahead and provide a full set of capabilities (not that any one of them 
MUST be enabled in a network, mind you), as well as a common framework
and architecture for NSF/headless routing.

Obviously, based on the number of drafts and amount of discussion, 
there is clearly interest in this type of functionality.  Of course
(qua ex-operator), I also tend to agree when Vijay, Ben, et al. that 
adding a bunch of code to get around code instabilities is, well, 
questionable at best.  I'm not a big fan of the recent influx of folks 
with SOS (Shiny Object Syndrome, -cjw), though this work would seem to
have merit.

To change gears for just a second (and try to qualify my "qua operator"
comments):

I recall being in a meeting once with a vendor (when I was an operator) 
and the vendor was proposing these sexy ways to use MPLS [the acronym of 
the decade!] to provide restoration capabilities to get around link 
failures in the network core.  So, after the 500 or so slides they used 
to explain how simple it actually was, I shared one slide with them.  My 
one [boring] slide simply showed [both quantitatively and qualitatively]
that router failures actually resulted in nearly an order of magnitude 
more downtime in my [real] network than link failures, and that I'd prefer 
they fix the former before addressing the latter.   Needless to say...

Anyways, to post my opinions:

  o I think the work has merit but needs to be coordinated between 
    the various WGs, and that a single solution for each protocol
    [all addressing a common architecture/framework/set of requirements] 
    needs to developed.

  o As Tony alludes to, I personally prefer truly fault tolerant systems
    that employ stateful failover to a standby control card, as NO 
    protocol changes are required.  Of course, I'm still interested in 
    accommodating my neighbors :-)

Finally, regarding the "Vote with your dollars" comment, remember that 
problems in neighboring [and distant] networks often causes you money
and heartache as well!

-danny

Date: Tue, 27 Mar 2001 17:34:51 -0500
From: "Steve Buchko" <[email-masked]>
Organization: Alcatel CID
To: [email-masked]
Cc: Alex Zinin <[email-masked]>, Russ White <[email-masked]>, Avi Freedman <[email-masked]>, Vijay Gill <[email-masked]>, [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

I believe that Alex & Alex both have the same objective - making the routers
more reliable and more resilient.

However, I don't believe that one solution precludes the other.  If routing
software is crashing all the time, or even occasionally, of course it needs
to be fixed!

But, we should also keep in mind that unexpected resets will occur for all
sorts of reasons.  It would be nice if routers and networks could handle such
failures more gracefully.   

It was pointed out during the OSPF meeting last week that a processor might
fail through no fault of routing.  The restart could be a result of a
hardware failure on the control module, or it could be caused by a bug in a
software process totally unrelated to routing.  In such scenarios, it would
be beneficial to be able to maintain forwarding context - avoiding huge churn
in the IGP along with the subsequent churn in BGP - especially if you are
dealing with a hardware architecture that can bring a routing processor back
online in milliseconds.  

For example, there is a huge installed base of products out there that
implement an active-standby redundancy model.  They work extremely well,
getting networks close to, if not actually achieving, the much-coveted
five-9's reliability (and yes, they manage to do so in the presence of
hundreds, even thousands of state changes per second).  Unfortunately, these
architectures don't work so well with protocols like OSPF, that insist on
completely flushing their state upon detecting a reset of the communications
link with their peer.

The hitless restart proposals give vendors additional flexibility in the
redundancy and fault-tolerance models that they wish to support.  I do not
believe we should deliberately restrict the potential applications in which
these mechanisms can be used.  It would be terribly short-sighted of us to
design hitless restart protocols in such a way as to preclude unplanned restarts.

steve

[email-masked] wrote:
> 
> >
> > >> But why _preclude_ it? If you don't trust it, turn it off. If you
> > >> do, turn it on. I can't understand why you would want to prevent
> > >> someone else from using it just because you don't trust it.
> >
> > > Because it presumes that router software vendors IMPLEMENT the *dont do it*
> > > knob correctly, and I am tired of presuming things like that.
> > > Does "no service lmi" ring a bell?
> >
> > Technical discussion here presume that people have enough expertise
> > to code and test. I'm sure anyone can find quite a few bugs in any
> > vendor's SW, however, the topic of this thread is an OSPF mechanism,
> > so I'd stick with it.
> 
> So how about making OSPF not to blow up once in a while spewing state across
> the board as a pre-requisite for vendors being able to introduce new
> features into OSPF? Especially, since people who are involved into these
> discussion have enough expertise to code and test.
> 
> ALex

Date: Tue, 27 Mar 2001 16:25:51 -0500 (EST)
From: <[email-masked]>
To: Alex Zinin <[email-masked]>
Cc: Russ White <[email-masked]>, Avi Freedman <[email-masked]>, Vijay Gill <[email-masked]>, [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

> 
> >> But why _preclude_ it? If you don't trust it, turn it off. If you
> >> do, turn it on. I can't understand why you would want to prevent
> >> someone else from using it just because you don't trust it.
> 
> > Because it presumes that router software vendors IMPLEMENT the *dont do it*
> > knob correctly, and I am tired of presuming things like that. 
> > Does "no service lmi" ring a bell?
> 
> Technical discussion here presume that people have enough expertise
> to code and test. I'm sure anyone can find quite a few bugs in any
> vendor's SW, however, the topic of this thread is an OSPF mechanism,
> so I'd stick with it.

So how about making OSPF not to blow up once in a while spewing state across
the board as a pre-requisite for vendors being able to introduce new
features into OSPF? Especially, since people who are involved into these
discussion have enough expertise to code and test.

ALex

Date: Tue, 27 Mar 2001 16:19:46 -0500 (EST)
From: <[email-masked]>
To: Alex Zinin <[email-masked]>
Cc: Ben Black <[email-masked]>, Srihari Ramachandra <[email-masked]>, Vijay Gill <[email-masked]>, Yakov Rekhter <[email-masked]>, [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

> 
> Ben:
> 
>  I don't think you summarize correctly.
> 
>  We do have a requirement now. It has already been
>  mentioned that not all customers are subscribed to the list,
>  also it is not a prerequisite for our customers to be
>  IETF-active or willing to speak up in order to provide us
>  with their requirements.

Since this had already been asked but never answered - 
Who are those customers that are asking for it? Lets be very specific? Are
they also the same people who are claiming that 7513 can have 250 BGP
sessions taking peering and a few full views of the internet?
 
>  Knobs will be there anyway. They are necessary to control
>  the mechanism.

Will there be a magic knob that would be an equivalent of the path

#ifndef __THROW_CRAP_OUT__
do_hitless_stuff()
#endif

>  The one we were talking about was to allow the admin to disable the
>  mechanism in certain situations if [s]he believe it might be dangerous in
>  that particular one.

Will it disable the mechanism or the manifestations of the mechanism?
Would disabling of the mechanism on a single device affect how the other
devices behave?

>  I think it is fair enough.

Alex

Date: Tue, 27 Mar 2001 12:42:20 -0800
From: Alex Zinin <[email-masked]>
Reply-To: Alex Zinin <[email-masked]>
Organization: Cisco Systems
To: Ben Black <[email-masked]>
Cc: Srihari Ramachandra <[email-masked]>, Vijay Gill <[email-masked]>, Yakov Rekhter <[email-masked]>, [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

Ben:

 I don't think you summarize correctly.

 We do have a requirement now. It has already been
 mentioned that not all customers are subscribed to the list,
 also it is not a prerequisite for our customers to be
 IETF-active or willing to speak up in order to provide us
 with their requirements.

 Knobs will be there anyway. They are necessary to control
 the mechanism. The one we were talking about was to allow
 the admin to disable the mechanism in certain situations
 if [s]he believe it might be dangerous in that particular
 one.

 I think it is fair enough.

Alex.

> So far the vendor arguments here have been:

> 1) You might want it later, so let's require it now.

>         This is obviously dubious since the customers have clearly stated
>         the feature they actually want and why.

> 2) If we add enough knobs, you can configure this to work, maybe.

>         See my first paragraph.

> These are not compelling.  If there really is a compelling argument
> for investing all the time and valuable developer resources on designing
> and implementing a feature people have not only not requested, but
> specifically stated they do not want, then let's see them.  In the mean
> time, can we focus on the requirements that have actually been 
> articulated by the people who BUY and USE the routers?

> Hitless restart on demand, nothing else.

> Ben

> On Tue, Mar 27, 2001 at 09:18:03AM -0800, Srihari Ramachandra wrote:
>> 
>> 
>> > Case B is even more interesting. The software crashed. Ok, so now I'm
>> > supposed to believe that crashing software can be fixed by adding MORE
>> > software and protocol extentsions, even though the original software
>> > didn't work well enough not to crash in the first place? 
>> 
>> If there is an option of falling back to a previous version of the
>> software (just after a crash), don't you think one could use the restart
>> functionality to achieve minimal impact.
>> 
>> srihari...
>> 
>> 
>> 

Date: Tue, 27 Mar 2001 12:02:30 -0800
From: Ben Black <[email-masked]>
To: Srihari Ramachandra <[email-masked]>
Cc: Vijay Gill <[email-masked]>, Alex Zinin <[email-masked]>, Yakov Rekhter <[email-masked]>, [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

This has got to be one of the most absurd conversations I've seen on a 
technical list in some time (note, I don't read NANOG).  Why are we
talking about routers GUESSING at why they crashed?  Falling back to
old images and HOPING they work?  How many iterations of this process
before someone figures out the box isn't coming back and they just
wasted 20 minutes waiting for the magical hitless restart to work?

So far the vendor arguments here have been:

1) You might want it later, so let's require it now.

	This is obviously dubious since the customers have clearly stated
        the feature they actually want and why.

2) If we add enough knobs, you can configure this to work, maybe.

	See my first paragraph.

These are not compelling.  If there really is a compelling argument
for investing all the time and valuable developer resources on designing
and implementing a feature people have not only not requested, but
specifically stated they do not want, then let's see them.  In the mean
time, can we focus on the requirements that have actually been 
articulated by the people who BUY and USE the routers?

Hitless restart on demand, nothing else.

Ben

On Tue, Mar 27, 2001 at 09:18:03AM -0800, Srihari Ramachandra wrote:
> 
> 
> > Case B is even more interesting. The software crashed. Ok, so now I'm
> > supposed to believe that crashing software can be fixed by adding MORE
> > software and protocol extentsions, even though the original software
> > didn't work well enough not to crash in the first place? 
> 
> If there is an option of falling back to a previous version of the
> software (just after a crash), don't you think one could use the restart
> functionality to achieve minimal impact.
> 
> srihari...
> 
> 
> 

Date: Tue, 27 Mar 2001 10:05:16 -0800
From: Alex Zinin <[email-masked]>
Reply-To: Alex Zinin <[email-masked]>
Organization: Cisco Systems
To: <[email-masked]>
Cc: Russ White <[email-masked]>, Avi Freedman <[email-masked]>, Vijay Gill <[email-masked]>, [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

alex:

>> But why _preclude_ it? If you don't trust it, turn it off. If you
>> do, turn it on. I can't understand why you would want to prevent
>> someone else from using it just because you don't trust it.

> Because it presumes that router software vendors IMPLEMENT the *dont do it*
> knob correctly, and I am tired of presuming things like that. 
> Does "no service lmi" ring a bell?

Technical discussion here presume that people have enough expertise
to code and test. I'm sure anyone can find quite a few bugs in any
vendor's SW, however, the topic of this thread is an OSPF mechanism,
so I'd stick with it.

Alex.

Date: Tue, 27 Mar 2001 09:58:58 -0800
From: Alex Zinin <[email-masked]>
Reply-To: Alex Zinin <[email-masked]>
Organization: Cisco Systems
To: Curtis Villamizar <[email-masked]>
Cc: [email-masked], Russ White <[email-masked]>, Ben Black <[email-masked]>, Vijay Gill <[email-masked]>, Yakov Rekhter <[email-masked]>, [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

Curtis:

Thanks a lot for your answers. See my comments below, pls.

>>  Thanks a lot for your thoughts. Couple of questions for you.
>> 
>>  1) Considering the fact that we introduced the possibility to
>>     abort the restart procedure on neighboring router when the
>>     network topology changes, and aside from the discussion on
>>     whether unplanned restart should be covered or not,
>>     could you please articulate your preference in favor of
>>     John's proposal vs ours?

> John's proposal relies on the type 9 LSAs.  The rest of the protocol
> remains unchanged except in the modified semantics during the grace
> period on both the router coming up and the helper.  Your proposal
> modifies existing database description and LSAs to include new bits.
> Adding the new type 9 LSAs and some new semantics seems to me to be a
> simpler approach and less likely to cause bugs in the base
> functionality.

Let me first clarify the details of our proposal and then comment
on your comparison.

Our proposal does not change any LSAs. What it does is introduces
additional flags in Hello and DBD packets. Processing of the flag
in Hellos is extremely simple---when set, start the timer and
ignore the absence of your router ID in the packet. Processing of
the flag in DBD is no worse either---if set, it is non-disruptive
LSDB resync, so set the flag, but proceed as in standard spec.

The main difference between two proposals (in addition to what Russ
already said) is as follows:

 1) John's proposal asks the neighbors to "close their eyes for
    a while and ignore" whatever happens to the router. Ours allows
    the neighbor to closely follow the process and abort it when
    the neighbor is suspected to be insane. I.e., in our proposal
    we're always sure about what the router _is_ doing, while in
    John's the neighbors believe the info provided by the previous
    incarnation of the process and ignore whatever happens to
    the new one (even if the adjacency is bouncing).

 2) John's proposal affects Hello packet processing and LSA
    origination with the contents of link-local LSAs. This reverse
    dependency between Hellos and link-local-LSAs shoots us in the foot
    in the unplanned restart case (yeah, we're not discussing
    it ;) when the restarting router just can't guarantee reliable
    delivery of the LSA before it sends the first Hello.
    In our proposal, all flags and hence all corresponding changes
    in processing are introduced exactly in the required context,
    i.e., processing of Hellos is changed by look at a flag in
    the Hello packet, processing of DBDs is changed by looking at
    a flag in the DBD packets.

 3) As far as implementation is concerned, if working code is
    still an argument here ;), it speaks for itself.

>>  2) Concerning your comment:
>> 
>> > The issue with John's draft is that you have to send a grace LSA after
>> > crashing.  This would not be difficult for the backup route processor
>> > to do and it would not be impossible to do on the way down on a single
>> > processor, assuming some limited sanity remained that would normally
>> > dump core, etc.  A question is whether we want to make this more like
>>  
>>    How can you guarantee that the neighbors do receive the Grace
>>    LSA before the first Hello from the restarting OSPF process and that
>>    they receive it _before_ the Hello?

> John's proposal exchanges the grace period (actually gives the expire
> time) in a type 9 LSA.  Yours does this in the Hello.  No big
> difference.  A potential modification to what John proposed would be
> to allow a type 9 LSA that gives the grace time after detecting the
> router is down.  This would be similar to what you have in the hello.

Not really, actually.
When the restarting router sends its first Hello, it needs to be sure
that the neighbors are in the helper mode. We do this by marking the
Hello packets themselves, so when (or if) the neighbor receives the
Hello, it is guaranteed to see the flag and act accordingly.
In John's proposal, if the router sends type-9 LSAs after the restart,
it cannot guarantee they are delivered, because it does not even
know who the neighbors are and whom to expect the Acks from, so,
as you will find in one of John's e-mails, the router is supposed
to blindly shoot one or several copies of the link-state update
packet containing the grace LSA hoping that the neighbors will receive
it and if they do, process it before the first Hello.

> Alternately, a router that retains enough sanity to function could
> detect that its control process is dead and send grace LSAs to each
> neighbor.

I think I cannot agree that this is safe assumption. In most
situations this would require changes in the code that you don't
want to make... I'd prefer to let the process crash if it wants
to crash :)

Thank you!

Alex.

>   It can then begin the process of dumping core on the old
> route control process.  Depending on the OS capability and available
> memory or availability of another route processor it could start the
> new route control process (or enable one that is preloaded on another
> processor) while dumping the old one.  If the router drops to a boot
> PROM with minimal capability, then the other method, exchanging a
> grace interval well in advance of a failure, would have to be used.

>>  Thanks again,
>> 
>> -- 
>> Alex Zinin

> Curtis

From: Tony Li <[email-masked]>
Date: Tue, 27 Mar 2001 10:02:21 -0800 (PST)
To: Vijay Gill <[email-masked]>
Cc: Alex Zinin <[email-masked]>, <[email-masked]>, <[email-masked]>, [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

 | Then we have the vendors who seem to be solving a problem the operators
 | seem not to be too interested in finding answers for.

Many operators think that a 99.999% availability level would be the holy
grail of all features.  It would allow operators to stop deploying two
separate systems to achieve redundancy.

Not all operators are on these lists.  

Tony

Date: Tue, 27 Mar 2001 12:53:00 -0500 (EST)
From: Russ White <[email-masked]>
Reply-To: Russ White <[email-masked]>
To: Curtis Villamizar <[email-masked]>
Cc: Ben Black <[email-masked]>, Vijay Gill <[email-masked]>, Yakov Rekhter <[email-masked]>, [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

> -- In Alex/Abhay/et al's draft, the resync is done with
> normal dbd's/lsa's with extra bits set, while in John's draft
> resync is done with Type 9's.

Oops! :-( I just reread John's draft, and see that he is using
normal lsa's to resync. The only technical difference at this
point remains the signalling.

Should we use type 9's to signal a restart, or define a new
signalling method in the hello's that can be used for this and
anything that comes up in the future?

Russ

_____________________________
[email-masked] <>< Grace Alone

Date: Tue, 27 Mar 2001 09:18:03 -0800 (PST)
From: Srihari Ramachandra <[email-masked]>
To: Vijay Gill <[email-masked]>
Cc: Alex Zinin <[email-masked]>, Yakov Rekhter <[email-masked]>, [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

> Case B is even more interesting. The software crashed. Ok, so now I'm
> supposed to believe that crashing software can be fixed by adding MORE
> software and protocol extentsions, even though the original software
> didn't work well enough not to crash in the first place? 

If there is an option of falling back to a previous version of the
software (just after a crash), don't you think one could use the restart
functionality to achieve minimal impact.

srihari...

Date: Tue, 27 Mar 2001 09:37:03 -0500 (EST)
From: <[email-masked]>
To: Russ White <[email-masked]>
Cc: Avi Freedman <[email-masked]>, Vijay Gill <[email-masked]>, [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

> But why _preclude_ it? If you don't trust it, turn it off. If you
> do, turn it on. I can't understand why you would want to prevent
> someone else from using it just because you don't trust it.

Because it presumes that router software vendors IMPLEMENT the *dont do it*
knob correctly, and I am tired of presuming things like that. 
Does "no service lmi" ring a bell?

Alex

> 
> Russ
> 
> On Tue, 27 Mar 2001, Avi Freedman wrote:
> 
> > 
> > I think the thing that may be missing is that hardware failures are
> > an order of magnitude less of a problem than software failures.  If
> > the goal is to deal with hardware failures, some of the solutions
> > under discussion sound interesting - but to deal with software crashes,
> > I agree, it's scary to even think that someone might implement such
> > things.
> > 
> > Avi
> > 
> > > On Mon, 26 Mar 2001, Russ White wrote:
> > > 
> > > > I would suggest that we not preclude restarts due to crashes when
> > > > deciding the best way to do this. Painting ourselves into a
> > > > corner based on the _current_ network operations and hardware
> > > > capabilities is akin to giving yourself a 32 bit address space
> > > > for the entire world....
> > > 
> > > Russ, just because it can be done technically doesn't make it a good idea.
> > > Random crashes are _bad_, more so when they are unexplained and no matter
> > > how good we can make a technical solution, it doesn't mean that the
> > > underlying problem it is trying to solve is the right problem. Maybe you
> > > are addressing the symptoms, instead of the cause.
> > > 
> > > /vijay
> > > 
> > > 
> > > 
> > > 
> > 
> 
> _____________________________
> [email-masked] <>< Grace Alone
> 
> 
> 
> 

Date: Tue, 27 Mar 2001 09:07:31 -0500 (EST)
From: Russ White <[email-masked]>
Reply-To: Russ White <[email-masked]>
To: Curtis Villamizar <[email-masked]>
Cc: Alex Zinin <[email-masked]>, [email-masked], Ben Black <[email-masked]>, Vijay Gill <[email-masked]>, Yakov Rekhter <[email-masked]>, [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

> >  1) Considering the fact that we introduced the possibility to
> >     abort the restart procedure on neighboring router when the
> >     network topology changes, and aside from the discussion on
> >     whether unplanned restart should be covered or not,
> >     could you please articulate your preference in favor of
> >     John's proposal vs ours?
> 
> John's proposal relies on the type 9 LSAs.  The rest of the
> protocol remains unchanged except in the modified semantics
> during the grace period on both the router coming up and the
> helper.  Your proposal modifies existing database description
> and LSAs to include new bits. Adding the new type 9 LSAs and
> some new semantics seems to me to be a simpler approach and
> less likely to cause bugs in the base functionality.

I see it just the opposite.... With type 9's, you have to watch
for the type 9, figure out what it is (it could be any sort of
type 9 going into the future), pull the right pieces out, and
touch the database. You almost end up with a seperate piece of
code, independant of the normal lsa processing stuff, touching
the database.

If you are using normal lsa's, with bits set, you have a simple
'if' statement in the code. If this bit is set, only accept the
lsa/dbd if I'm in graceful restart. It's probably some largish
number of lines of code to handle the type 9's (along with the
issues of multiple places to enter the database) vs 2/3 lines of
code to use the existing packet formats and a single bit.

> >  2) Concerning your comment:
> > 
> > > The issue with John's draft is that you have to send a grace LSA after
> > > crashing.  This would not be difficult for the backup route processor
> > > to do and it would not be impossible to do on the way down on a single
> > > processor, assuming some limited sanity remained that would normally
> > > dump core, etc.  A question is whether we want to make this more like
> >  
> >    How can you guarantee that the neighbors do receive the Grace
> >    LSA before the first Hello from the restarting OSPF process and that
> >    they receive it _before_ the Hello?
> 
> John's proposal exchanges the grace period (actually gives
> the expire time) in a type 9 LSA.  Yours does this in the
> Hello.  No big difference.  A potential modification to what
> John proposed would be to allow a type 9 LSA that gives the
> grace time after detecting the router is down.  This would be
> similar to what you have in the hello.

The big difference here is whether we want to extend the hello so
we can handle other sorts of signals in the future, or just start
using type 9's for everything. I'm not certain I like type 9's
for singalling between routers, especially when they could be
used to actually carry some pertinent routing information in the
future.

It would just make the code that handles type 9's a mess. It
seems cleaner to me to create new signalling capabilities in the
protocol if we need them.

> Alternately, a router that retains enough sanity to function
> could detect that its control process is dead and send grace
> LSAs to each neighbor.  It can then begin the process of
> dumping core on the old route control process.  Depending on
> the OS capability and available memory or availability of
> another route processor it could start the new route control
> process (or enable one that is preloaded on another
> processor) while dumping the old one.  If the router drops to
> a boot PROM with minimal capability, then the other method,
> exchanging a grace interval well in advance of a failure,
> would have to be used.

I wouldn't want to signal the grace period until the event
occurs. If you pre-signal, then you end up waiting for the other
router to recover. If you signal when it occurs, you are
explicitely saying "I'm up and here, but I need some time to get
my act together."

Russ

_____________________________
[email-masked] <>< Grace Alone

To: Alex Zinin <[email-masked]>
Cc: Curtis Villamizar <[email-masked]>, Russ White <[email-masked]>, [email-masked], Ben Black <[email-masked]>, Vijay Gill <[email-masked]>, Yakov Rekhter <[email-masked]>, [email-masked], [email-masked]
Reply-To: [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt 
Date: Tue, 27 Mar 2001 08:51:22 -0500
From: Curtis Villamizar <[email-masked]>
Sender: [email-masked]

In message <[email-masked]>, Alex Zinin writes:
> 
> Curtis:
> 
>  Thanks a lot for your thoughts. Couple of questions for you.
> 
>  1) Considering the fact that we introduced the possibility to
>     abort the restart procedure on neighboring router when the
>     network topology changes, and aside from the discussion on
>     whether unplanned restart should be covered or not,
>     could you please articulate your preference in favor of
>     John's proposal vs ours?

John's proposal relies on the type 9 LSAs.  The rest of the protocol
remains unchanged except in the modified semantics during the grace
period on both the router coming up and the helper.  Your proposal
modifies existing database description and LSAs to include new bits.
Adding the new type 9 LSAs and some new semantics seems to me to be a
simpler approach and less likely to cause bugs in the base
functionality.

>  2) Concerning your comment:
> 
> > The issue with John's draft is that you have to send a grace LSA after
> > crashing.  This would not be difficult for the backup route processor
> > to do and it would not be impossible to do on the way down on a single
> > processor, assuming some limited sanity remained that would normally
> > dump core, etc.  A question is whether we want to make this more like
>  
>    How can you guarantee that the neighbors do receive the Grace
>    LSA before the first Hello from the restarting OSPF process and that
>    they receive it _before_ the Hello?

John's proposal exchanges the grace period (actually gives the expire
time) in a type 9 LSA.  Yours does this in the Hello.  No big
difference.  A potential modification to what John proposed would be
to allow a type 9 LSA that gives the grace time after detecting the
router is down.  This would be similar to what you have in the hello.

Alternately, a router that retains enough sanity to function could
detect that its control process is dead and send grace LSAs to each
neighbor.  It can then begin the process of dumping core on the old
route control process.  Depending on the OS capability and available
memory or availability of another route processor it could start the
new route control process (or enable one that is preloaded on another
processor) while dumping the old one.  If the router drops to a boot
PROM with minimal capability, then the other method, exchanging a
grace interval well in advance of a failure, would have to be used.

>  Thanks again,
> 
> -- 
> Alex Zinin

Curtis

From: Randy Bush <[email-masked]>
To: Vijay Gill <[email-masked]>
Cc: [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Date: Tue, 27 Mar 2001 04:25:40 -0800
Sender: [email-masked]

> Fix broken stuff first. A sentiment anyone who has had the (dis)pleasure
> of fixing and tracking protocol implementation induced loops will be more
> than willing to echo.

echo

i can not help but wonder how much unreliability is caused by complexity.

randy

Date: Tue, 27 Mar 2001 06:29:05 -0500 (EST)
From: Russ White <[email-masked]>
Reply-To: Russ White <[email-masked]>
To: Curtis Villamizar <[email-masked]>
Cc: Ben Black <[email-masked]>, Vijay Gill <[email-masked]>, Yakov Rekhter <[email-masked]>, [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

> John has come up with a good feature.  It would be good to
> allow the neighbor routers to declare the non-responding
> router inconsistent, but it would also be good to acknowlege
> what reality exist today and be able to turn this feature
> off.  It would also be good to be able to recover from a
> crash or disable that feature and just recover from
> controlled restart depending on whether slosh is more
> disruptive than inconsistency during reload time.
> 
> This means that neither proposal is entirely right for all
> situations. Features of the two need to be combined.  IMHO it
> is easier/better to use John's proposal as a basis and add
> (or complete) the procedures for recovery on crash and offer
> the ability to disable the feature where neighbors route
> around if anything in the IGP changes.
> 
> The issue with John's draft is that you have to send a grace
> LSA after crashing.  This would not be difficult for the
> backup route processor to do and it would not be impossible
> to do on the way down on a single processor, assuming some
> limited sanity remained that would normally dump core, etc.  
> A question is whether we want to make this more like the IDR
> work where there is a grace period determined at open and
> applied from the time the helper side detects failure.

The primary differences between the drafts are:

-- In Alex/Abhay/et al's draft, the grace signalling is done with
additional information in the hello packets, while in John's it
is in a type 9 LSA outside the hello process.

-- In Alex/Abhay/et al's draft, the resync is done with normal
dbd's/lsa's with extra bits set, while in John's draft resync is
done with Type 9's.

Other than this, they are _technically_ substantially the
same. They will both handle manual restarts, and they will both
handle crashes. The primary difference is in presentation--John
has presented his as being primarily for manual restarts, while
Alex/Abhay/et al. have presented theirs for both.

So, getting _beyond_ the presentation, I'd like to think about
the technical issues with how the signalling is done and how the
resync is done.

Of course, I prefer signalling in the hello's, and using the
native packet formats for resync, rather than hiding everything
in type 9's. It's easier to code for, and easier ot understand on
the wire. 

Further using type 9's to do everything sets a precedent that I'm
not entirely comfortable with.

:-)

Russ

_____________________________
[email-masked] <>< Grace Alone

Date: Tue, 27 Mar 2001 06:18:22 -0500 (EST)
From: Russ White <[email-masked]>
Reply-To: Russ White <[email-masked]>
To: Vijay Gill <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

I'd like to see 100% crash-proog software. But the reality is,
even if you had that, random events still occur. They always
will. We have the capability of allowing the protocol to
participate in recovering from these random events, rather than
just dying.

Why not, at the least, build the spec so it can deal with a ot of
different situations, rather than just one?

This capability may be used in a multithreaded system to fall
back to another process. It may be used on a multi-processor box
to fall back to another processor. It might be used in a myriad
of ways.

It may even be used to manually restart a router.

Throwing out the ability to respond to unexpected network events
in this idea throwing out the baby with the bathwater, all
because you think the bathwater is currently too dirty. When you
clean the bathwater up, you might wish you had the baby back--and
they you might find yourself in a position where you can't go
back without a lot of changes in what you have.

Russ

On Mon, 26 Mar 2001, Vijay Gill wrote:

> On Mon, 26 Mar 2001, Russ White wrote:
> 
> > I would suggest that we not preclude restarts due to crashes when
> > deciding the best way to do this. Painting ourselves into a
> > corner based on the _current_ network operations and hardware
> > capabilities is akin to giving yourself a 32 bit address space
> > for the entire world....
> 
> Russ, just because it can be done technically doesn't make it a good idea.
> Random crashes are _bad_, more so when they are unexplained and no matter
> how good we can make a technical solution, it doesn't mean that the
> underlying problem it is trying to solve is the right problem. Maybe you
> are addressing the symptoms, instead of the cause.
> 
> /vijay
> 
> 
> 
> 

_____________________________
[email-masked] <>< Grace Alone

Date: Tue, 27 Mar 2001 06:13:20 -0500 (EST)
From: Russ White <[email-masked]>
Reply-To: Russ White <[email-masked]>
To: Avi Freedman <[email-masked]>
Cc: Vijay Gill <[email-masked]>, [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

But why _preclude_ it? If you don't trust it, turn it off. If you
do, turn it on. I can't understand why you would want to prevent
someone else from using it just because you don't trust it.

Russ

On Tue, 27 Mar 2001, Avi Freedman wrote:

> 
> I think the thing that may be missing is that hardware failures are
> an order of magnitude less of a problem than software failures.  If
> the goal is to deal with hardware failures, some of the solutions
> under discussion sound interesting - but to deal with software crashes,
> I agree, it's scary to even think that someone might implement such
> things.
> 
> Avi
> 
> > On Mon, 26 Mar 2001, Russ White wrote:
> > 
> > > I would suggest that we not preclude restarts due to crashes when
> > > deciding the best way to do this. Painting ourselves into a
> > > corner based on the _current_ network operations and hardware
> > > capabilities is akin to giving yourself a 32 bit address space
> > > for the entire world....
> > 
> > Russ, just because it can be done technically doesn't make it a good idea.
> > Random crashes are _bad_, more so when they are unexplained and no matter
> > how good we can make a technical solution, it doesn't mean that the
> > underlying problem it is trying to solve is the right problem. Maybe you
> > are addressing the symptoms, instead of the cause.
> > 
> > /vijay
> > 
> > 
> > 
> > 
> 

_____________________________
[email-masked] <>< Grace Alone

Date: Mon, 26 Mar 2001 23:50:53 -0800
From: Alex Zinin <[email-masked]>
Reply-To: Alex Zinin <[email-masked]>
Organization: Cisco Systems
To: Vijay Gill <[email-masked]>
Cc: <[email-masked]>, <[email-masked]>
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

Vijay:

>> after a crash. Seems to be a reasonable middle ground to me.
>> Or you explicitly do not want _anybody_ to _ever_ implement
>> the unplanned case? Seems to be not realistic to me.

> I guess this is ok, I'm going to go with what Dr. Li said and vote with my
> dollars. There seems to be no value in dragging this on here any more, as
> I am sure we are not going to converge.

I think it is important to converge on respecting each other's
opinions and making sure the solution is flexible enough to
match them. I feel we have this type of agreement. Let's see
what other folks say tomorrow :)

> Thanks for the good points bought to light (by you, curtis and russ w. on
> the pro side of the argument).

My pleasure!
Thanks a bunch!

Alex.

Date: Tue, 27 Mar 2001 02:41:58 -0500
From: Vijay Gill <[email-masked]>
To: Alex Zinin <[email-masked]>
Cc: <[email-masked]>, <[email-masked]>
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

On Mon, 26 Mar 2001, Alex Zinin wrote:

> I don't mind anyone speaking up, of course, but I believe
> the reasons for the restart are pretty obvious and what we're arguing
> about is pretty much whether the unplanned restart case should be
> covered.

So far, I've counted a bunch of operational folks saying this isn't
something high on their priority list: Avi, Alex Y, Daniel Golding, Ben
and myself, along with Vince et al (in the ospf wg meeting)

Then we have the vendors who seem to be solving a problem the operators
seem not to be too interested in finding answers for.

> after a crash. Seems to be a reasonable middle ground to me.
> Or you explicitly do not want _anybody_ to _ever_ implement
> the unplanned case? Seems to be not realistic to me.

I guess this is ok, I'm going to go with what Dr. Li said and vote with my
dollars. There seems to be no value in dragging this on here any more, as
I am sure we are not going to converge.

Thanks for the good points bought to light (by you, curtis and russ w. on
the pro side of the argument).

/vijay

Date: Mon, 26 Mar 2001 23:29:29 -0800
From: Alex Zinin <[email-masked]>
Reply-To: Alex Zinin <[email-masked]>
Organization: Cisco Systems
To: Vijay Gill <[email-masked]>
Cc: <[email-masked]>, <[email-masked]>
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

Vijay:

>> And how do you think forwarding state preservation would contribute
>> to the problem here?

>  am saying that perhaps the synching between your newly initialized
> databases and the forwarding state may not be as easy as it appears on
> paper. I've lost track of how many times I've had to add a null route and
> then delete it before the forwarding state got synched with the routing
> state and that was without any mystery crash to scramble the brains of the
> router. I am not saying this cannot be done properly, I'm saying that
> given what I've seen so far, I'm not holding my breath.

Point taken.

>> There are multiple requirements for the graceful/hitless restart.

> Lets hear the people who want this speak up and articulate some of the
> reasons.

I don't mind anyone speaking up, of course, but I believe
the reasons for the restart are pretty obvious and what we're arguing
about is pretty much whether the unplanned restart case should be
covered.

Your opinion is it should not. My opinion is it should and hence
I wouldn't like the solution to exclude it, but I am not going to
try to convince you to agree with me, instead I would prefer that
implementations include the option of not restarting gracefully
after a crash. Seems to be a reasonable middle ground to me.
Or you explicitly do not want _anybody_ to _ever_ implement
the unplanned case? Seems to be not realistic to me.

Alex.

Date: Tue, 27 Mar 2001 01:58:35 -0500
From: Vijay Gill <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>, <[email-masked]>
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

On Tue, 27 Mar 2001, Curtis Villamizar wrote:

>
> The forwarding state can be read when the route processor comes up or
> takes over.  Forwarding state which matches the routes that are
> learned, is preserved.  State which doesn't match is replaced or
> specially aged out in the case of prefixes that went away completely
> or were completely bogus (if bad forwarding data had existed).

> We are not talking about preserving corrupt BGP Adj-In-RIB or OSPF LSA
> across reboots and sending it back out (afaik, at least not yet).
> The forwarding data is kept in place until it can be reconstructed by
> relearning everything and deriving new forwarding data from the
> information that was relearned.

Curtis,

  consider a corrupt protocol packet, but before it crashes the software,
you update the Forwarding state. Up comes the router, gracefully, and the
sycn/cleaner doesn't catch the fact that you've injected corrupt
forwarding plane entries. Wirespeed loops across my expensive wan
infrastructure and an operational debugging nightmare. Networks barely
work today, and all I ask is: Please stop helping me.

Fix broken stuff first. A sentiment anyone who has had the (dis)pleasure
of fixing and tracking protocol implementation induced loops will be more
than willing to echo.

/vijay

Date: Mon, 26 Mar 2001 22:51:24 -0800
From: Alex Zinin <[email-masked]>
Reply-To: Alex Zinin <[email-masked]>
Organization: Cisco Systems
To: Avi Freedman <[email-masked]>
Cc: [email-masked], [email-masked] (Vijay Gill), [email-masked] (Yakov Rekhter), [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

Avi:

 As Curtis already explained, OSPF data is NOT preserved, but
 recollected from the neighbors. That's why we go through the
 LSDB resync process.

Alex.

> Some people would rather have any effort available expended towards
> good software (or a real OS, if that crutch is needed).  I can see
> bringing back BGP quickly, but the disasters possible with crunched
> OSPF data are soooo much worse.  80k routes causing cyclic crashing
> and propagation, anyone?

> Avi

>> 
>> alex:
>> 
>>  I hope it is clear by now that nobody is trying to solve OSPF
>>  bugs leading to crashes with additional mechanisms in the protocol.
>> 
>>  However, realistically speaking, SW will always have bugs, and
>>  the question is how one would like to react on a crash. If you
>>  want the process to remain down---no mechanisms are required.
>>  If some people want to try to bring it back up there definitely
>>  are people who want to do it without resetting the forwarding
>>  state.
>> 
>> Alex.
>> 
>> 
>> 
>> >> Or this way: would you not like your router to at least try to
>> >> bring the process that has just crashed back up with (of course,
>> >> with an oscillation prevention mechanism)?
>> 
>> > How about starting by not having the process/router crash to begin with?
>> > That would solve 99% of the problems that you are trying to address.
>> 
>> >> > Case B is even more interesting. The software crashed. Ok, so now I'm
>> >> > supposed to believe that crashing software can be fixed by adding MORE
>> >> > software and protocol extentsions, even though the original software
>> >> > didn't work well enough not to crash in the first place?  Essentially my
>> >> > take on this is: I couldn't write robust software, so let me make the
>> >> > system more robust by adding more software!
>> >> 
>> >> Following your logic here we would probably have to remove
>> >> protected memory support from all CPUs and OSes and have all programmers
>> >> fix all their bugs in all their programs... Oh, dear...
>> 
>> > Page fault caused by de-referrencing a virtual address while running in a
>> > state that can be pre-empted is very very very different from a page fault 
>> > caused by a process that does not pay attention to the heap boundaries...
>> 
>> 
>> > Alex
>> 
>> 
>> 

Date: Tue, 27 Mar 2001 01:52:35 -0500
From: Vijay Gill <[email-masked]>
To: Alex Zinin <[email-masked]>
Cc: <[email-masked]>, <[email-masked]>
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

On Mon, 26 Mar 2001, Alex Zinin wrote:

> And how do you think forwarding state preservation would contribute
> to the problem here?

 am saying that perhaps the synching between your newly initialized
databases and the forwarding state may not be as easy as it appears on
paper. I've lost track of how many times I've had to add a null route and
then delete it before the forwarding state got synched with the routing
state and that was without any mystery crash to scramble the brains of the
router. I am not saying this cannot be done properly, I'm saying that
given what I've seen so far, I'm not holding my breath.

> There are multiple requirements for the graceful/hitless restart.

Lets hear the people who want this speak up and articulate some of the
reasons.

/vijay

Date: Mon, 26 Mar 2001 22:35:21 -0800
From: Alex Zinin <[email-masked]>
Reply-To: Alex Zinin <[email-masked]>
Organization: Cisco Systems
To: Curtis Villamizar <[email-masked]>
Cc: Russ White <[email-masked]>, [email-masked], Ben Black <[email-masked]>, Vijay Gill <[email-masked]>, Yakov Rekhter <[email-masked]>, [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

Curtis:

 Thanks a lot for your thoughts. Couple of questions for you.

 1) Considering the fact that we introduced the possibility to
    abort the restart procedure on neighboring router when the
    network topology changes, and aside from the discussion on
    whether unplanned restart should be covered or not,
    could you please articulate your preference in favor of
    John's proposal vs ours?

 2) Concerning your comment:

> The issue with John's draft is that you have to send a grace LSA after
> crashing.  This would not be difficult for the backup route processor
> to do and it would not be impossible to do on the way down on a single
> processor, assuming some limited sanity remained that would normally
> dump core, etc.  A question is whether we want to make this more like
 
   How can you guarantee that the neighbors do receive the Grace
   LSA before the first Hello from the restarting OSPF process and that
   they receive it _before_ the Hello?

 Thanks again,

-- 
Alex Zinin

Monday, March 26, 2001, 7:42:15 PM, Curtis Villamizar wrote:

> In message <[email-masked]>, Ru
> ss White writes:
>> 
>> So, what you are saying is that because of the current state of
>> software, you'd rather limit what you might be able to do in the
>> future? Having the ability to respond to crashes doesn't preclude
>> the ability to deal with a graceful restart due to manual
>> intervention. In fact, done right, the protocol changes are the
>> same for both.
>> 
>> I'm certain just about _any_ vendor wil be able to figure out how
>> to allow configuration for crash recovery vs manual restart. If
>> you don't want crash recovery because you don't think you can
>> trust the software implementation, turn it off.
>> 
>> Precluding dealing with crashes makes no sense to me. Or is this
>> just going to be yet another forum to fuss about the current
>> software available or the various vendor's abililties to write
>> software?
>> 
>> Russ
>> 
>> On Mon, 26 Mar 2001, Ben Black wrote:
>> 
>> > I don't believe the issue is hardware capability.  As is being made quite
>> > plain, these protocols can be extended to include the ability for hitless
>> > restart.  The open question in the minds of many operators is "given the
>> > quality of software we have come to expect from most vendors, why would
>> > we want them to attempt to implement a feature with such horrendous
>> > failure modes?".  Or words to that effect.
>> > 
>> > 
>> > Ben

> Guys,

> I'm with a vendor now so I'm supposed to try to tell you how reliable
> router software is becoming and how extremely rare crashes will be.
> If I were in sales I'd try to do that with a completely straight face.

> I've been on the side of the fence that Vijay and Ben are on and I
> remember way back then holding a different opinion.  My thoughts were
> that router vendors could barely keep up with the protocol features
> without introducing new ways for the routers to crash on a regular
> basis.  The last thing I wanted was for them to further destabilize
> their code base by trying to add redundancy features.  My feeling was
> that adding more complexity to the problem that was being solved would
> just make the code all the more unstable.

> The activity in this area seems to oscillate between that opinion and
> the other extreme where carrier people are chanting the 5-9s mantra
> (usually people with little real world experience and ignoring the
> 5-9s jokes about requiring my circuits or home service or whatever to
> stay up for the next 2,000 years to get 5-9s of reliability after the
> last big outage, not realizing that the greater community regards the
> whole 5-9s thing in the telephone industry as a big joke and just
> marketing hype from the telco community).

> Now I'm a router vendor guy and we too are doing the FT thing.  It
> seems that although the activity oscillates, there is increasing
> credibility that somebody might actually get this working.  It is
> pointless to argue further whether it should be done or whether the
> idea should be buried indefinitely.

> A real issue is whether the forwarding table should be suspect enough
> to be disposed of if the router crashes.  One argument is that routing
> and forwarding are decoupled so forwarding should be OK.  The counter
> argument is that routers aren't supposed to just fall over and die
> doing nothing (discounting sun spot activity and the claims by some
> that it impacts parity memory in routers without ECC) and if the
> router fell over due to change in routing it didn't fully process that
> change and is not consisitent with other routers in the domain.

> In either case there is an impact.  Route slosh causes transient
> inconsistency in the IGP and even more so due to BGP if there are 10s
> of thousands of routes to change as a consequence of changing the IGP.
> Even if the forwarding table were inconsistent, the impact of that
> slosh can be greater than the time to reload (except in the case where
> routers take many minutes to reload) and is certainly much greater
> than the time it takes to switch to a warm backup route processor.

> In short, John's claim that routers should dispose of the forwarding
> table has little real credibility among those that understand the
> whole problem as the problem now stands with OSPF and BGP in big
> providers.

> This problem is however changing.  Providers are using MPLS in their
> core and advertising the MPLS LSPs at a fixed metric to the non-MPLS
> routers beyond the core.  When a router in the core is impacted, when
> routing around it, the LSP reroutes and BGP prefixes in the non-MPLS
> routers never change.  Those BGP routes at the ingress of the MPLS
> tunnel need not change if forwarding is done in a two stage lookup and
> a small table entry accomplishes the reroute.

> Purely internal MPLS LSR may not even run BGP at all.  A larger issue
> may be how to deal with an MPLS LSR acting as an ingress going down
> since it won't initiate reroute if topology changes elsewhere.  For
> interior LSRs the issue is that they cannot participate in the
> signaling for a reroute initiated by an ingress.  Here is where the
> feature John described of taking the router out of use when any change
> occurs may be needed.

> When an ingress MPLS LSR crashes, if it is taken out of service BGP
> routes in the non-MPLS routers that were pointing toward it will have
> to be changed to point to another router.  Because chances of loops
> are very small in this particular case, keeping the forwarding intact
> in the down router and also routing away from it may be the best
> tactic if the LSR does not have a backup route processor.  If it does
> and the switch to the backup is fast, then it may be better not to try
> to route around it.

> In cases where there is no BGP impact, slosh impact is very small.  If
> a router's forwarding is inconsistent there may be less impact making
> the IGP change and causing some MPLS tunnels to reroute.  This makes
> John right for those cases.

> So John is both wrong in some situations to propose taking a router
> out of service on crash and when anything changes and right in others,
> or maybe too forward thinking (or maybe too much of a purist but the
> reason he's right in some situations doesn't matter).

> John has come up with a good feature.  It would be good to allow the
> neighbor routers to declare the non-responding router inconsistent,
> but it would also be good to acknowlege what reality exist today and
> be able to turn this feature off.  It would also be good to be able to
> recover from a crash or disable that feature and just recover from
> controlled restart depending on whether slosh is more disruptive than
> inconsistency during reload time.

> This means that neither proposal is entirely right for all situations.
> Features of the two need to be combined.  IMHO it is easier/better to
> use John's proposal as a basis and add (or complete) the procedures
> for recovery on crash and offer the ability to disable the feature
> where neighbors route around if anything in the IGP changes.

> The issue with John's draft is that you have to send a grace LSA after
> crashing.  This would not be difficult for the backup route processor
> to do and it would not be impossible to do on the way down on a single
> processor, assuming some limited sanity remained that would normally
> dump core, etc.  A question is whether we want to make this more like
> the IDR work where there is a grace period determined at open and
> applied from the time the helper side detects failure.

> Curtis

Date: Mon, 26 Mar 2001 22:30:05 -0800
From: Alex Zinin <[email-masked]>
Reply-To: Alex Zinin <[email-masked]>
Organization: Cisco Systems
To: Curtis Villamizar <[email-masked]>
Cc: Vijay Gill <[email-masked]>, [email-masked], [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

>> I would like to not crash in the first place, but if it does crash, at
>> least start from a known, fresh state so that you don't propogate corrupt
>> state around the network, causing the rest of the boxes to crash as well.
>> Again a purely hypothetical scenario since that would never every happen,
>> especially not on/around nov 6-7, 1998, though that date does seem to
>> stick in my memory for some unknown reason.

> So far we are talking about saving forwarding state here.

> The forwarding state can be read when the route processor comes up or
> takes over.  Forwarding state which matches the routes that are
> learned, is preserved.  State which doesn't match is replaced or
> specially aged out in the case of prefixes that went away completely
> or were completely bogus (if bad forwarding data had existed).

> We are not talking about preserving corrupt BGP Adj-In-RIB or OSPF LSA
> across reboots and sending it back out (afaik, at least not yet).  The
> forwarding data is kept in place until it can be reconstructed by
> relearning everything and deriving new forwarding data from the
> information that was relearned.

Yes, this is what we have in mind.
Thanks, Curtis.

Alex.

Date: Mon, 26 Mar 2001 22:28:21 -0800
From: Alex Zinin <[email-masked]>
Reply-To: Alex Zinin <[email-masked]>
Organization: Cisco Systems
To: [email-masked]
Cc: Vijay Gill <[email-masked]>, Yakov Rekhter <[email-masked]>, [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

alex:

 I hope it is clear by now that nobody is trying to solve OSPF
 bugs leading to crashes with additional mechanisms in the protocol.

 However, realistically speaking, SW will always have bugs, and
 the question is how one would like to react on a crash. If you
 want the process to remain down---no mechanisms are required.
 If some people want to try to bring it back up there definitely
 are people who want to do it without resetting the forwarding
 state.

Alex.

>> Or this way: would you not like your router to at least try to
>> bring the process that has just crashed back up with (of course,
>> with an oscillation prevention mechanism)?

> How about starting by not having the process/router crash to begin with?
> That would solve 99% of the problems that you are trying to address.

>> > Case B is even more interesting. The software crashed. Ok, so now I'm
>> > supposed to believe that crashing software can be fixed by adding MORE
>> > software and protocol extentsions, even though the original software
>> > didn't work well enough not to crash in the first place?  Essentially my
>> > take on this is: I couldn't write robust software, so let me make the
>> > system more robust by adding more software!
>> 
>> Following your logic here we would probably have to remove
>> protected memory support from all CPUs and OSes and have all programmers
>> fix all their bugs in all their programs... Oh, dear...

> Page fault caused by de-referrencing a virtual address while running in a
> state that can be pre-empted is very very very different from a page fault 
> caused by a process that does not pay attention to the heap boundaries...

> Alex

Date: Mon, 26 Mar 2001 22:04:59 -0800
From: Alex Zinin <[email-masked]>
Reply-To: Alex Zinin <[email-masked]>
Organization: Cisco Systems
To: Vijay Gill <[email-masked]>
Cc: <[email-masked]>, <[email-masked]>
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

Vijay:

>> Or this way: would you not like your router to at least try to
>> bring the process that has just crashed back up with (of course,
>> with an oscillation prevention mechanism)?

> I would like to not crash in the first place, but if it does crash, at
> least start from a known, fresh state so that you don't propogate corrupt
> state around the network, causing the rest of the boxes to crash as well.

And how do you think forwarding state preservation would contribute
to the problem here?

> Cross talk seems to be a problem here, so let me ask a simple base
> question:

> Have we considered why people appear to be asking for hitless restart?

> Maybe it is because the boxes keep on crashing? This is a wild assumption
> on my part of course, but it may be useful as a base reason.

> Ok, so people are asking for hitless restart because boxes keep crashing.
> How about we fix the "crashing" part and we'll get to hitless restart and
> the requirements thereof later, instead of wildly code yet more cruft?

There are multiple requirements for the graceful/hitless restart.
Unplanned restart is definitely not the only one. It is still there,
however. Again, the fact that a process died and had to be restarted
does not mean it will keep on dying every minute and one necessarily
wants to stop forwarding.

So, the points are:

 1) Yes, we do see a requirement for unplanned restart

 2) No, people are not asking for graceful restart only because
    of crashes (and no, we're not so stupid to fix bugs by just
    restarting the process)

 3) For those who do not need the unplanned restart covered,
    a knob will be introduced to control the behavior.

Alex.

To: Vijay Gill <[email-masked]>
Cc: Alex Zinin <[email-masked]>, [email-masked], [email-masked]
Reply-To: [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt 
Date: Tue, 27 Mar 2001 00:50:19 -0500
From: Curtis Villamizar <[email-masked]>
Sender: [email-masked]

In message <[email-masked]>, 
Vijay Gill writes:
> On Mon, 26 Mar 2001, Alex Zinin wrote:
> 
> > Let me ask you a question: when you see a process/router crash
> > after it has been running for a while, what is the first thing
> > you're doing---decode the crashinfo and gdb the coredump or bring
> > the router back up?
> 
> Normally we don't get a choice since we are more conversant that we like
> with the crashinfo/dump process and how well that works.
> 
> > Or this way: would you not like your router to at least try to
> > bring the process that has just crashed back up with (of course,
> > with an oscillation prevention mechanism)?
> 
> I would like to not crash in the first place, but if it does crash, at
> least start from a known, fresh state so that you don't propogate corrupt
> state around the network, causing the rest of the boxes to crash as well.
> Again a purely hypothetical scenario since that would never every happen,
> especially not on/around nov 6-7, 1998, though that date does seem to
> stick in my memory for some unknown reason.

So far we are talking about saving forwarding state here.

The forwarding state can be read when the route processor comes up or
takes over.  Forwarding state which matches the routes that are
learned, is preserved.  State which doesn't match is replaced or
specially aged out in the case of prefixes that went away completely
or were completely bogus (if bad forwarding data had existed).

We are not talking about preserving corrupt BGP Adj-In-RIB or OSPF LSA
across reboots and sending it back out (afaik, at least not yet).  The
forwarding data is kept in place until it can be reconstructed by
relearning everything and deriving new forwarding data from the
information that was relearned.

Curtis

Date: Tue, 27 Mar 2001 00:35:33 -0500 (EST)
From: <[email-masked]>
To: Alex Zinin <[email-masked]>
Cc: Vijay Gill <[email-masked]>, Yakov Rekhter <[email-masked]>, [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

> Let me ask you a question: when you see a process/router crash
> after it has been running for a while, what is the first thing
> you're doing---decode the crashinfo and gdb the coredump or bring
> the router back up?
> 
> Or this way: would you not like your router to at least try to
> bring the process that has just crashed back up with (of course,
> with an oscillation prevention mechanism)?

How about starting by not having the process/router crash to begin with?
That would solve 99% of the problems that you are trying to address.

> > Case B is even more interesting. The software crashed. Ok, so now I'm
> > supposed to believe that crashing software can be fixed by adding MORE
> > software and protocol extentsions, even though the original software
> > didn't work well enough not to crash in the first place?  Essentially my
> > take on this is: I couldn't write robust software, so let me make the
> > system more robust by adding more software!
> 
> Following your logic here we would probably have to remove
> protected memory support from all CPUs and OSes and have all programmers
> fix all their bugs in all their programs... Oh, dear...

Page fault caused by de-referrencing a virtual address while running in a
state that can be pre-empted is very very very different from a page fault 
caused by a process that does not pay attention to the heap boundaries...

Alex

Date: Tue, 27 Mar 2001 00:31:10 -0500
From: Vijay Gill <[email-masked]>
To: Alex Zinin <[email-masked]>
Cc: <[email-masked]>, <[email-masked]>
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

On Mon, 26 Mar 2001, Alex Zinin wrote:

> Let me ask you a question: when you see a process/router crash
> after it has been running for a while, what is the first thing
> you're doing---decode the crashinfo and gdb the coredump or bring
> the router back up?

Normally we don't get a choice since we are more conversant that we like
with the crashinfo/dump process and how well that works.

> Or this way: would you not like your router to at least try to
> bring the process that has just crashed back up with (of course,
> with an oscillation prevention mechanism)?

I would like to not crash in the first place, but if it does crash, at
least start from a known, fresh state so that you don't propogate corrupt
state around the network, causing the rest of the boxes to crash as well.
Again a purely hypothetical scenario since that would never every happen,
especially not on/around nov 6-7, 1998, though that date does seem to
stick in my memory for some unknown reason.

> > didn't work well enough not to crash in the first place?  Essentially my
> > take on this is: I couldn't write robust software, so let me make the
> > system more robust by adding more software!
>
> Following your logic here we would probably have to remove
> protected memory support from all CPUs and OSes and have all programmers
> fix all their bugs in all their programs... Oh, dear...

Cross talk seems to be a problem here, so let me ask a simple base
question:

Have we considered why people appear to be asking for hitless restart?

Maybe it is because the boxes keep on crashing? This is a wild assumption
on my part of course, but it may be useful as a base reason.

Ok, so people are asking for hitless restart because boxes keep crashing.
How about we fix the "crashing" part and we'll get to hitless restart and
the requirements thereof later, instead of wildly code yet more cruft?

thanks

/vijay

Date: Mon, 26 Mar 2001 21:18:29 -0800
From: Alex Zinin <[email-masked]>
Reply-To: Alex Zinin <[email-masked]>
Organization: Cisco Systems
To: Vijay Gill <[email-masked]>
Cc: Yakov Rekhter <[email-masked]>, <[email-masked]>, <[email-masked]>
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

Vijay:

> Ok, so the question I have now is: Why is the OSPF process crashing?

> a) bad hardware

> b) bad software

> Case A is interesting if we consider the subtle cases, such as ... oh I
> don't know, how about a Process Memory Parity Error, not that it happens
> in a real network or anything. Now essentially we have some scrambled
> state somewhere in the box, we don't know what it is, all we know is that
> it caused our software to crash.  Let us gracefully restart the process.
> Never mind that we don't know what really happened, what state any of the
> databases are in, and we can now assume we gracefully rebuild all of them?
> Right.

Let me ask you a question: when you see a process/router crash
after it has been running for a while, what is the first thing
you're doing---decode the crashinfo and gdb the coredump or bring
the router back up?

Or this way: would you not like your router to at least try to
bring the process that has just crashed back up with (of course,
with an oscillation prevention mechanism)?

> Case B is even more interesting. The software crashed. Ok, so now I'm
> supposed to believe that crashing software can be fixed by adding MORE
> software and protocol extentsions, even though the original software
> didn't work well enough not to crash in the first place?  Essentially my
> take on this is: I couldn't write robust software, so let me make the
> system more robust by adding more software!

Following your logic here we would probably have to remove
protected memory support from all CPUs and OSes and have all programmers
fix all their bugs in all their programs... Oh, dear...

> The idea here being that perhaps the assumptions underlying "graceful"
> restart could stand to be be examined a bit more carefully. What exactly
> are the cases you are solving for?

Those including the unplanned restart case.

Thanks,

Alex.

Date: Mon, 26 Mar 2001 23:49:59 -0500
From: Vijay Gill <[email-masked]>
To: Russ White <[email-masked]>
Cc: <[email-masked]>, <[email-masked]>
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

On Mon, 26 Mar 2001, Russ White wrote:

> I would suggest that we not preclude restarts due to crashes when
> deciding the best way to do this. Painting ourselves into a
> corner based on the _current_ network operations and hardware
> capabilities is akin to giving yourself a 32 bit address space
> for the entire world....

Russ, just because it can be done technically doesn't make it a good idea.
Random crashes are _bad_, more so when they are unexplained and no matter
how good we can make a technical solution, it doesn't mean that the
underlying problem it is trying to solve is the right problem. Maybe you
are addressing the symptoms, instead of the cause.

/vijay

Date: Mon, 26 Mar 2001 23:45:25 -0500
From: Vijay Gill <[email-masked]>
To: Alex Zinin <[email-masked]>
Cc: Yakov Rekhter <[email-masked]>, <[email-masked]>, <[email-masked]>
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

On Mon, 26 Mar 2001, Alex Zinin wrote:

>
> 1. I think for the sakes of the discussion on graceful
>    OSPF restart, it would be very useful to get rid
>    of the assumption that a crash happens to the whole
>    control card. Assuming that a crash in the OSPF process
>    does not need to crash the whole card, it becomes
>    apparent that a solution for unplanned OSPF restart
>    is practically important.

Ok, so the question I have now is: Why is the OSPF process crashing?

a) bad hardware

b) bad software

Case A is interesting if we consider the subtle cases, such as ... oh I
don't know, how about a Process Memory Parity Error, not that it happens
in a real network or anything. Now essentially we have some scrambled
state somewhere in the box, we don't know what it is, all we know is that
it caused our software to crash.  Let us gracefully restart the process.
Never mind that we don't know what really happened, what state any of the
databases are in, and we can now assume we gracefully rebuild all of them?
Right.

Case B is even more interesting. The software crashed. Ok, so now I'm
supposed to believe that crashing software can be fixed by adding MORE
software and protocol extentsions, even though the original software
didn't work well enough not to crash in the first place?  Essentially my
take on this is: I couldn't write robust software, so let me make the
system more robust by adding more software!

Also known as the: My head hurts, so let me hit my foot with a hammer,
that way my head doesn't appear to hurt as much by comparison.

> 2. Even if unplanned OSPF restart is not considered to
>    be necessary by a set of people now, this does not mean
>    that no networks need it now or in future, and definitely
>    this does not mean that it is ok for the solution itself
>    to have a flaw in this particular scenario. This will just
>    come back to us in a while.

The idea here being that perhaps the assumptions underlying "graceful"
restart could stand to be be examined a bit more carefully. What exactly
are the cases you are solving for?

>    with it because of no guarantee a graceful restart will actually
>    happen.

/vijay

To: Russ White <[email-masked]>
Cc: Ben Black <[email-masked]>, Vijay Gill <[email-masked]>, Yakov Rekhter <[email-masked]>, [email-masked], [email-masked]
Reply-To: [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt 
Date: Mon, 26 Mar 2001 22:42:15 -0500
From: Curtis Villamizar <[email-masked]>
Sender: [email-masked]

In message <[email-masked]>, Ru
ss White writes:
> 
> So, what you are saying is that because of the current state of
> software, you'd rather limit what you might be able to do in the
> future? Having the ability to respond to crashes doesn't preclude
> the ability to deal with a graceful restart due to manual
> intervention. In fact, done right, the protocol changes are the
> same for both.
> 
> I'm certain just about _any_ vendor wil be able to figure out how
> to allow configuration for crash recovery vs manual restart. If
> you don't want crash recovery because you don't think you can
> trust the software implementation, turn it off.
> 
> Precluding dealing with crashes makes no sense to me. Or is this
> just going to be yet another forum to fuss about the current
> software available or the various vendor's abililties to write
> software?
> 
> Russ
> 
> On Mon, 26 Mar 2001, Ben Black wrote:
> 
> > I don't believe the issue is hardware capability.  As is being made quite
> > plain, these protocols can be extended to include the ability for hitless
> > restart.  The open question in the minds of many operators is "given the
> > quality of software we have come to expect from most vendors, why would
> > we want them to attempt to implement a feature with such horrendous
> > failure modes?".  Or words to that effect.
> > 
> > 
> > Ben

Guys,

I'm with a vendor now so I'm supposed to try to tell you how reliable
router software is becoming and how extremely rare crashes will be.
If I were in sales I'd try to do that with a completely straight face.

I've been on the side of the fence that Vijay and Ben are on and I
remember way back then holding a different opinion.  My thoughts were
that router vendors could barely keep up with the protocol features
without introducing new ways for the routers to crash on a regular
basis.  The last thing I wanted was for them to further destabilize
their code base by trying to add redundancy features.  My feeling was
that adding more complexity to the problem that was being solved would
just make the code all the more unstable.

The activity in this area seems to oscillate between that opinion and
the other extreme where carrier people are chanting the 5-9s mantra
(usually people with little real world experience and ignoring the
5-9s jokes about requiring my circuits or home service or whatever to
stay up for the next 2,000 years to get 5-9s of reliability after the
last big outage, not realizing that the greater community regards the
whole 5-9s thing in the telephone industry as a big joke and just
marketing hype from the telco community).

Now I'm a router vendor guy and we too are doing the FT thing.  It
seems that although the activity oscillates, there is increasing
credibility that somebody might actually get this working.  It is
pointless to argue further whether it should be done or whether the
idea should be buried indefinitely.

A real issue is whether the forwarding table should be suspect enough
to be disposed of if the router crashes.  One argument is that routing
and forwarding are decoupled so forwarding should be OK.  The counter
argument is that routers aren't supposed to just fall over and die
doing nothing (discounting sun spot activity and the claims by some
that it impacts parity memory in routers without ECC) and if the
router fell over due to change in routing it didn't fully process that
change and is not consisitent with other routers in the domain.

In either case there is an impact.  Route slosh causes transient
inconsistency in the IGP and even more so due to BGP if there are 10s
of thousands of routes to change as a consequence of changing the IGP.
Even if the forwarding table were inconsistent, the impact of that
slosh can be greater than the time to reload (except in the case where
routers take many minutes to reload) and is certainly much greater
than the time it takes to switch to a warm backup route processor.

In short, John's claim that routers should dispose of the forwarding
table has little real credibility among those that understand the
whole problem as the problem now stands with OSPF and BGP in big
providers.

This problem is however changing.  Providers are using MPLS in their
core and advertising the MPLS LSPs at a fixed metric to the non-MPLS
routers beyond the core.  When a router in the core is impacted, when
routing around it, the LSP reroutes and BGP prefixes in the non-MPLS
routers never change.  Those BGP routes at the ingress of the MPLS
tunnel need not change if forwarding is done in a two stage lookup and
a small table entry accomplishes the reroute.

Purely internal MPLS LSR may not even run BGP at all.  A larger issue
may be how to deal with an MPLS LSR acting as an ingress going down
since it won't initiate reroute if topology changes elsewhere.  For
interior LSRs the issue is that they cannot participate in the
signaling for a reroute initiated by an ingress.  Here is where the
feature John described of taking the router out of use when any change
occurs may be needed.

When an ingress MPLS LSR crashes, if it is taken out of service BGP
routes in the non-MPLS routers that were pointing toward it will have
to be changed to point to another router.  Because chances of loops
are very small in this particular case, keeping the forwarding intact
in the down router and also routing away from it may be the best
tactic if the LSR does not have a backup route processor.  If it does
and the switch to the backup is fast, then it may be better not to try
to route around it.

In cases where there is no BGP impact, slosh impact is very small.  If
a router's forwarding is inconsistent there may be less impact making
the IGP change and causing some MPLS tunnels to reroute.  This makes
John right for those cases.

So John is both wrong in some situations to propose taking a router
out of service on crash and when anything changes and right in others,
or maybe too forward thinking (or maybe too much of a purist but the
reason he's right in some situations doesn't matter).

John has come up with a good feature.  It would be good to allow the
neighbor routers to declare the non-responding router inconsistent,
but it would also be good to acknowlege what reality exist today and
be able to turn this feature off.  It would also be good to be able to
recover from a crash or disable that feature and just recover from
controlled restart depending on whether slosh is more disruptive than
inconsistency during reload time.

This means that neither proposal is entirely right for all situations.
Features of the two need to be combined.  IMHO it is easier/better to
use John's proposal as a basis and add (or complete) the procedures
for recovery on crash and offer the ability to disable the feature
where neighbors route around if anything in the IGP changes.

The issue with John's draft is that you have to send a grace LSA after
crashing.  This would not be difficult for the backup route processor
to do and it would not be impossible to do on the way down on a single
processor, assuming some limited sanity remained that would normally
dump core, etc.  A question is whether we want to make this more like
the IDR work where there is a grace period determined at open and
applied from the time the helper side detects failure.

Curtis

Date: Mon, 26 Mar 2001 19:29:02 -0500 (EST)
From: Russ White <[email-masked]>
Reply-To: Russ White <[email-masked]>
To: Daniel Golding <[email-masked]>
Cc: Ben Black <[email-masked]>, Vijay Gill <[email-masked]>, Yakov Rekhter <[email-masked]>, [email-masked], [email-masked]
Subject: RE: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

> I must agree. Consider that the last refuge from routing
> software instability is the cold restart. In many cases it
> must be turned to when all efforts to restart routing
> processes have failed (on those routers even capable of
> independently restarting routing processes). We will be
> essentially "moving the bar" from having to do a cold restart
> to having to complete a "factory presets" reload procedure in
> the worst cases of routing process instability. One of the
> features - if you will - of current routing software is that
> the state is not persistent across device restarts.

Again, I don't see how precluding the ability to restart for
unexpected events will help the implementation of the ability to
do a graceful manual restart. 

These are our choices at the moment--the reason this thread
started:

-- expected events only should be handled
-- the protocol should be designed so both expected and
   unexpected events can be handled

> As an operator, I would much rather see some sort of true,
> stateful, failover between primary and backup routing
> processors, for use either in a hardware failure, or for
> on-line upgrades.

Okay, so....

-- You would rather have a secondary processor in the box that
came up when the primary failed. 

-- I would argue that you would rather get a new database, rather
than using the old one that the primary processor just failed on
(I can see saving configuration and temporary forwarding
information, but not the ospf database).

-- Precluding unexpected events from this spec will preclude the
ability to fail over to a secondary processor and grab a fresh
database.

You've argued against what you say you are in favor of.

:-)

Russ
_____________________________
[email-masked] <>< Grace Alone

Date: Mon, 26 Mar 2001 18:25:03 -0500 (EST)
From: Russ White <[email-masked]>
Reply-To: Russ White <[email-masked]>
To: Ben Black <[email-masked]>
Cc: Vijay Gill <[email-masked]>, Yakov Rekhter <[email-masked]>, [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

> I'm not clear on how choosing not to pursue a feature now
> precludes its pursuit in the future.  If a genuine demand for
> the feature you describe emerges, then it can certainly be
> designed and implemented. Until that time, I suggest we all
> remember the mantra of IETF50:

> What are the requirements?

vs.

> I don't believe the issue is hardware capability.  As is
> being made quite plain, these protocols can be extended to
> include the ability for hitless restart.  The open question
> in the minds of many operators is "given the quality of
> software we have come to expect from most vendors, why would
> we want them to attempt to implement a feature with such
> horrendous failure modes?".  Or words to that effect.

Are you saying there are no requirements, or that you don't trust
the vendors to implement this? Or are you saying that because you
don't trust vendors to implement this, there are no requirements?

I'm trying to get back to a technical reason why we should
preclude handling unexpected events. I don't see that changing
the protocol to handle one is all that different than changing
the protocol to handle both. Is there some _technical_ reason why
handling both is so much different than just handling one?

Russ
_____________________________
[email-masked] <>< Grace Alone

Date: Mon, 26 Mar 2001 15:17:01 -0800
From: Alex Zinin <[email-masked]>
Reply-To: Alex Zinin <[email-masked]>
Organization: Cisco Systems
To: Ben Black <[email-masked]>
Cc: Russ White <[email-masked]>, Vijay Gill <[email-masked]>, Yakov Rekhter <[email-masked]>, [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

Ben:

I know it's obvious :), but I'd like to say I
completely agree with the points Russ is making.

> I'm not clear on how choosing not to pursue a feature now precludes
> its pursuit in the future.

This does not preclude it at all, of course.
However, it does affect the process of evaluation of the proposals
on the table, as well as what it takes to make the mechanism
actually work for unplanned restart.

> If a genuine demand for the feature you
> describe emerges, then it can certainly be designed and implemented.

These is not a question of another feature per se. It is a question
about the set of scenarios the mechanism should cover, and we
already need to cover the unplanned restart case today.

Alex.

> Until that time, I suggest we all remember the mantra of IETF50:

> What are the requirements?

From: Tony Li <[email-masked]>
Date: Mon, 26 Mar 2001 15:16:33 -0800 (PST)
To: Vijay Gill <[email-masked]>
Cc: Yakov Rekhter <[email-masked]>, <[email-masked]>, <[email-masked]>
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

Vijay,

Some thoughts to consider:

Whether you want it or not, someone will implement it.

Someone will eventually make it work.

And someday, someone will even deploy it in production.

The question then is will it have any merit.  If it has no merit, then you
should let your competition deploy it.  If it does have merit, then you
want to be the one who deploys it.  Vote with your dollars, please.

Tony

Vijay Gill writes:
 | On Mon, 26 Mar 2001, Yakov Rekhter wrote:
 | 
 | > Folks,
 | >
 | > Correct me if I am wrong, but draft-ietf-ospf-hitless-restart-00.txt
 | > doesn't cover the case when a router crashes and restarts.
 | 
 | If a router crashes by some unknown error, then it should start from a
 | clean slate IMO. I already have enough trouble trying to get routers to
 | stay up and adding more software to fix the broken software doesn't seem
 | like the optimal solution to me.
 | 
 | > However, unless OSPF would be able to support graceful restart after
 | > crash, having graceful restart capability in BGP may not be that
 | > useful when BGP is used in conjunction with OSPF.
 | 
 | The only truely graceful restart case I'm interested in is migration from
 | control plane to backup control plane, updating software and then
 | migrating back.
 | 
 | In actual operational networks, while doing maint windows at 3 am,
 | graceful restart is usually the least of our concerns. Something that
 | ranks way way higher is:
 | 
 |    will this router come back up?
 | 
 | Shortly followed by:
 | 
 |    Please call remote hands to power cycle, since the router is now wedged
 |    hard, and the out of band console break isn't working.
 | 
 | 
 | Into this mess, throwing yet more software just seems like a bad idea.
 | 
 | But then again, I actual do maint windows once in a while, so my
 | perspective may be different than that of the Networks of Powerpoint (tm,
 | sm, pat pending) crowd.
 | 
 | /vijay
 | 

Date: Mon, 26 Mar 2001 14:50:29 -0800
From: Alex Zinin <[email-masked]>
Reply-To: Alex Zinin <[email-masked]>
Organization: Cisco Systems
To: Yakov Rekhter <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

Reading the replies from Vijay and John, I believe
several comments need to be made:

1. I think for the sakes of the discussion on graceful
   OSPF restart, it would be very useful to get rid
   of the assumption that a crash happens to the whole
   control card. Assuming that a crash in the OSPF process
   does not need to crash the whole card, it becomes
   apparent that a solution for unplanned OSPF restart
   is practically important.

2. Even if unplanned OSPF restart is not considered to
   be necessary by a set of people now, this does not mean
   that no networks need it now or in future, and definitely
   this does not mean that it is ok for the solution itself
   to have a flaw in this particular scenario. This will just
   come back to us in a while.

3. I also share a feeling that draft-ietf-ospf-hitless-restart-00.txt
   does not address the unplanned OSPF restart scenario
   very well because two major reasons:

     o After a restart Grace-LSAs are sent unreliably, hence,
       if even one neighbor does not receive it (the link-state
       update packet is dropped somewhere), the graceful
       restart process is cancelled.

     o Since link-state update packets carrying the Grace-LSAs
       and Hello packets are processed in different contexts,
       there's no practical guarantee that even if the link-state
       Update is received it is processed before the first
       hello from the restarting router.

   I believe that these two points make the mechanism described
   in *-ospf-hitless-restart-* non-deterministic in case of
   an unplanned restart.

   While working on our solution (draft-ietf-ospf-restart-01.txt +
   draft-ietf-ospf-oob-resync-01.txt), we considered a mechanism
   essentially similar to John's proposal and decided not to go
   with it because of no guarantee a graceful restart will actually
   happen.

-- 
Alex Zinin

Monday, March 26, 2001, 12:13:41 PM, Yakov Rekhter wrote:

> Folks,

> Correct me if I am wrong, but draft-ietf-ospf-hitless-restart-00.txt
> doesn't cover the case when a router crashes and restarts.

> At the same time, the IDR WG is working on a proposal that *does*
> cover the case when a router crashes and restarts (see
> draft-ietf-idr-restart-00.txt).

> However, unless OSPF would be able to support graceful restart
> after crash, having graceful restart capability in BGP may not be
> that useful when BGP is used in conjunction with OSPF.

> In view of the above, I wonder whether there is a need for more
> consistency in the area of hitless restart between the IDR and the
> OSPF WGs.

> Yakov.

Date: Mon, 26 Mar 2001 14:44:09 -0800
From: Ben Black <[email-masked]>
To: Russ White <[email-masked]>
Cc: Vijay Gill <[email-masked]>, Yakov Rekhter <[email-masked]>, [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

On Mon, Mar 26, 2001 at 05:44:35PM -0500, Russ White wrote:
> 
> So, what you are saying is that because of the current state of
> software, you'd rather limit what you might be able to do in the
> future? Having the ability to respond to crashes doesn't preclude
> the ability to deal with a graceful restart due to manual
> intervention. In fact, done right, the protocol changes are the
> same for both.
> 

I'm not clear on how choosing not to pursue a feature now precludes
its pursuit in the future.  If a genuine demand for the feature you
describe emerges, then it can certainly be designed and implemented.
Until that time, I suggest we all remember the mantra of IETF50:

What are the requirements?

Ben

Date: Mon, 26 Mar 2001 17:44:35 -0500 (EST)
From: Russ White <[email-masked]>
Reply-To: Russ White <[email-masked]>
To: Ben Black <[email-masked]>
Cc: Vijay Gill <[email-masked]>, Yakov Rekhter <[email-masked]>, [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

So, what you are saying is that because of the current state of
software, you'd rather limit what you might be able to do in the
future? Having the ability to respond to crashes doesn't preclude
the ability to deal with a graceful restart due to manual
intervention. In fact, done right, the protocol changes are the
same for both.

I'm certain just about _any_ vendor wil be able to figure out how
to allow configuration for crash recovery vs manual restart. If
you don't want crash recovery because you don't think you can
trust the software implementation, turn it off.

Precluding dealing with crashes makes no sense to me. Or is this
just going to be yet another forum to fuss about the current
software available or the various vendor's abililties to write
software?

Russ

On Mon, 26 Mar 2001, Ben Black wrote:

> I don't believe the issue is hardware capability.  As is being made quite
> plain, these protocols can be extended to include the ability for hitless
> restart.  The open question in the minds of many operators is "given the
> quality of software we have come to expect from most vendors, why would
> we want them to attempt to implement a feature with such horrendous
> failure modes?".  Or words to that effect.
> 
> 
> Ben
> 
> On Mon, Mar 26, 2001 at 05:22:45PM -0500, Russ White wrote:
> > 
> > I would suggest that we not preclude restarts due to crashes when
> > deciding the best way to do this. Painting ourselves into a
> > corner based on the _current_ network operations and hardware
> > capabilities is akin to giving yourself a 32 bit address space
> > for the entire world....
> > 
> > Oh, we already did that, didn't we? :-)
> > 
> > Russ
> > 
> > On Mon, 26 Mar 2001, Vijay Gill wrote:
> > 
> > > On Mon, 26 Mar 2001, Yakov Rekhter wrote:
> > > 
> > > > Folks,
> > > >
> > > > Correct me if I am wrong, but draft-ietf-ospf-hitless-restart-00.txt
> > > > doesn't cover the case when a router crashes and restarts.
> > > 
> > > If a router crashes by some unknown error, then it should start from a
> > > clean slate IMO. I already have enough trouble trying to get routers to
> > > stay up and adding more software to fix the broken software doesn't seem
> > > like the optimal solution to me.
> > > 
> > > > However, unless OSPF would be able to support graceful restart after
> > > > crash, having graceful restart capability in BGP may not be that
> > > > useful when BGP is used in conjunction with OSPF.
> > > 
> > > The only truely graceful restart case I'm interested in is migration from
> > > control plane to backup control plane, updating software and then
> > > migrating back.
> > > 
> > > In actual operational networks, while doing maint windows at 3 am,
> > > graceful restart is usually the least of our concerns. Something that
> > > ranks way way higher is:
> > > 
> > >    will this router come back up?
> > > 
> > > Shortly followed by:
> > > 
> > >    Please call remote hands to power cycle, since the router is now wedged
> > >    hard, and the out of band console break isn't working.
> > > 
> > > 
> > > Into this mess, throwing yet more software just seems like a bad idea.
> > > 
> > > But then again, I actual do maint windows once in a while, so my
> > > perspective may be different than that of the Networks of Powerpoint (tm,
> > > sm, pat pending) crowd.
> > > 
> > > /vijay
> > > 
> > > 
> > 
> > _____________________________
> > [email-masked] <>< Grace Alone
> > 
> 

_____________________________
[email-masked] <>< Grace Alone

Date: Mon, 26 Mar 2001 14:30:58 -0800
From: Ben Black <[email-masked]>
To: Russ White <[email-masked]>
Cc: Vijay Gill <[email-masked]>, Yakov Rekhter <[email-masked]>, [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

I don't believe the issue is hardware capability.  As is being made quite
plain, these protocols can be extended to include the ability for hitless
restart.  The open question in the minds of many operators is "given the
quality of software we have come to expect from most vendors, why would
we want them to attempt to implement a feature with such horrendous
failure modes?".  Or words to that effect.

Ben

On Mon, Mar 26, 2001 at 05:22:45PM -0500, Russ White wrote:
> 
> I would suggest that we not preclude restarts due to crashes when
> deciding the best way to do this. Painting ourselves into a
> corner based on the _current_ network operations and hardware
> capabilities is akin to giving yourself a 32 bit address space
> for the entire world....
> 
> Oh, we already did that, didn't we? :-)
> 
> Russ
> 
> On Mon, 26 Mar 2001, Vijay Gill wrote:
> 
> > On Mon, 26 Mar 2001, Yakov Rekhter wrote:
> > 
> > > Folks,
> > >
> > > Correct me if I am wrong, but draft-ietf-ospf-hitless-restart-00.txt
> > > doesn't cover the case when a router crashes and restarts.
> > 
> > If a router crashes by some unknown error, then it should start from a
> > clean slate IMO. I already have enough trouble trying to get routers to
> > stay up and adding more software to fix the broken software doesn't seem
> > like the optimal solution to me.
> > 
> > > However, unless OSPF would be able to support graceful restart after
> > > crash, having graceful restart capability in BGP may not be that
> > > useful when BGP is used in conjunction with OSPF.
> > 
> > The only truely graceful restart case I'm interested in is migration from
> > control plane to backup control plane, updating software and then
> > migrating back.
> > 
> > In actual operational networks, while doing maint windows at 3 am,
> > graceful restart is usually the least of our concerns. Something that
> > ranks way way higher is:
> > 
> >    will this router come back up?
> > 
> > Shortly followed by:
> > 
> >    Please call remote hands to power cycle, since the router is now wedged
> >    hard, and the out of band console break isn't working.
> > 
> > 
> > Into this mess, throwing yet more software just seems like a bad idea.
> > 
> > But then again, I actual do maint windows once in a while, so my
> > perspective may be different than that of the Networks of Powerpoint (tm,
> > sm, pat pending) crowd.
> > 
> > /vijay
> > 
> > 
> 
> _____________________________
> [email-masked] <>< Grace Alone
> 

Date: Mon, 26 Mar 2001 17:22:45 -0500 (EST)
From: Russ White <[email-masked]>
Reply-To: Russ White <[email-masked]>
To: Vijay Gill <[email-masked]>
Cc: Yakov Rekhter <[email-masked]>, [email-masked], [email-masked]
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

I would suggest that we not preclude restarts due to crashes when
deciding the best way to do this. Painting ourselves into a
corner based on the _current_ network operations and hardware
capabilities is akin to giving yourself a 32 bit address space
for the entire world....

Oh, we already did that, didn't we? :-)

Russ

On Mon, 26 Mar 2001, Vijay Gill wrote:

> On Mon, 26 Mar 2001, Yakov Rekhter wrote:
> 
> > Folks,
> >
> > Correct me if I am wrong, but draft-ietf-ospf-hitless-restart-00.txt
> > doesn't cover the case when a router crashes and restarts.
> 
> If a router crashes by some unknown error, then it should start from a
> clean slate IMO. I already have enough trouble trying to get routers to
> stay up and adding more software to fix the broken software doesn't seem
> like the optimal solution to me.
> 
> > However, unless OSPF would be able to support graceful restart after
> > crash, having graceful restart capability in BGP may not be that
> > useful when BGP is used in conjunction with OSPF.
> 
> The only truely graceful restart case I'm interested in is migration from
> control plane to backup control plane, updating software and then
> migrating back.
> 
> In actual operational networks, while doing maint windows at 3 am,
> graceful restart is usually the least of our concerns. Something that
> ranks way way higher is:
> 
>    will this router come back up?
> 
> Shortly followed by:
> 
>    Please call remote hands to power cycle, since the router is now wedged
>    hard, and the out of band console break isn't working.
> 
> 
> Into this mess, throwing yet more software just seems like a bad idea.
> 
> But then again, I actual do maint windows once in a while, so my
> perspective may be different than that of the Networks of Powerpoint (tm,
> sm, pat pending) crowd.
> 
> /vijay
> 
> 

_____________________________
[email-masked] <>< Grace Alone

Date: Mon, 26 Mar 2001 15:21:01 -0500
From: Vijay Gill <[email-masked]>
To: Yakov Rekhter <[email-masked]>
Cc: <[email-masked]>, <[email-masked]>
Subject: Re: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Sender: [email-masked]

On Mon, 26 Mar 2001, Yakov Rekhter wrote:

> Folks,
>
> Correct me if I am wrong, but draft-ietf-ospf-hitless-restart-00.txt
> doesn't cover the case when a router crashes and restarts.

If a router crashes by some unknown error, then it should start from a
clean slate IMO. I already have enough trouble trying to get routers to
stay up and adding more software to fix the broken software doesn't seem
like the optimal solution to me.

> However, unless OSPF would be able to support graceful restart after
> crash, having graceful restart capability in BGP may not be that
> useful when BGP is used in conjunction with OSPF.

The only truely graceful restart case I'm interested in is migration from
control plane to backup control plane, updating software and then
migrating back.

In actual operational networks, while doing maint windows at 3 am,
graceful restart is usually the least of our concerns. Something that
ranks way way higher is:

   will this router come back up?

Shortly followed by:

   Please call remote hands to power cycle, since the router is now wedged
   hard, and the out of band console break isn't working.

Into this mess, throwing yet more software just seems like a bad idea.

But then again, I actual do maint windows once in a while, so my
perspective may be different than that of the Networks of Powerpoint (tm,
sm, pat pending) crowd.

/vijay

To: [email-masked], [email-masked]
Cc: [email-masked]
Subject: draft-ietf-ospf-hitless-restart-00.txt, draft-ietf-idr-restart-00.txt
Date: Mon, 26 Mar 2001 12:13:41 -0800
From: Yakov Rekhter <[email-masked]>
Sender: [email-masked]

Folks,

Correct me if I am wrong, but draft-ietf-ospf-hitless-restart-00.txt
doesn't cover the case when a router crashes and restarts.

At the same time, the IDR WG is working on a proposal that *does*
cover the case when a router crashes and restarts (see
draft-ietf-idr-restart-00.txt).

However, unless OSPF would be able to support graceful restart
after crash, having graceful restart capability in BGP may not be
that useful when BGP is used in conjunction with OSPF.

In view of the above, I wonder whether there is a need for more
consistency in the area of hitless restart between the IDR and the
OSPF WGs.

Yakov.

To: Randy Bush <[email-masked]>
Cc: Enke Chen <[email-masked]>, Susan Hares <[email-masked]>, [email-masked]
Reply-To: [email-masked]
Subject: Re: Fwd: For Alain Durand 
Date: Mon, 26 Mar 2001 14:44:19 -0500
From: Curtis Villamizar <[email-masked]>
Sender: [email-masked]

In message <[email-masked]>, Randy Bush writes:
> >> why is it necessary to expand the size of the router id?  does it need to
> >> be an ip address, or is it just an id?
> > It is really just an id. Some ISPs simply assign the id's sequentially like
> > 1, 2, 3 ... on their networks. This id itself does not need to be globally
> > unique. What needs to be unique is the pair <asn, router-id>. This part may
> > not be clear in the spec..  How about we do this:
> >      o clarify this uniqueness.
> >      o add some text in the collision resolution portion to favor the one
> >        with large AS number when the router-id's are equal.
> > I do not see any need to expand this field. It seems to be wastful and
> > introduces unnecessary complexity.
> 
> way cool,  thanks.
> 
> randy

The router has to have an address which is reachable regardless as to
which interfaces are down (as long as some internal interfaces are up)
for the purpose of having a reliable address for IBGP.  By convention,
this has been the loopback devices which also by convention has been
used as the router-id.  While the router-id could be defined to be an
arbitrary integer it hasn't been used that way.

This is more of an IPv6 issue than a four byte AS issue.  Since it is
a convention used in IPv4, we could drop the convention for IPv6.  It
would be a lot easier than supporting an IPv6 router-id.

Curtis

From: Randy Bush <[email-masked]>
To: Enke Chen <[email-masked]>
Cc: Susan Hares <[email-masked]>, [email-masked]
Subject: Re: Fwd: For Alain Durand 
Date: Sat, 24 Mar 2001 16:49:03 -0800
Sender: [email-masked]

>> why is it necessary to expand the size of the router id?  does it need to
>> be an ip address, or is it just an id?
> It is really just an id. Some ISPs simply assign the id's sequentially like
> 1, 2, 3 ... on their networks. This id itself does not need to be globally
> unique. What needs to be unique is the pair <asn, router-id>. This part may
> not be clear in the spec..  How about we do this:
>      o clarify this uniqueness.
>      o add some text in the collision resolution portion to favor the one
>        with large AS number when the router-id's are equal.
> I do not see any need to expand this field. It seems to be wastful and
> introduces unnecessary complexity.

way cool,  thanks.

randy

To: Randy Bush <[email-masked]>
Cc: Susan Hares <[email-masked]>, [email-masked], [email-masked]
Subject: Re: Fwd: For Alain Durand 
Date: Sat, 24 Mar 2001 15:23:58 -0800
From: Enke Chen <[email-masked]>
Sender: [email-masked]

Randy,

> From: Randy Bush <[email-masked]>
> MIME-Version: 1.0
> Content-Type: text/plain; charset=us-ascii
> Content-Transfer-Encoding: 7bit
> To: Susan Hares <[email-masked]>
> Cc: [email-masked]
> Subject: Re: Fwd:  For Alain Durand
> References: <[email-masked]>
> Message-Id: <[email-masked]>
> Date: Thu, 22 Mar 2001 09:49:55 -0600
> Sender: [email-masked]
> Precedence: bulk
> 
> why is it necessary to expand the size of the router id?  does it need to be
> an ip address, or is it just an id?

It is really just an id. Some ISPs simply assign the id's sequentially like
1, 2, 3 ... on their networks. This id itself does not need to be globally
unique. What needs to be unique is the pair <asn, router-id>. This part may
not be clear in the spec..  How about we do this:

     o clarify this uniqueness.
     o add some text in the collision resolution portion to favor the one
       with large AS number when the router-id's are equal.

I do not see any need to expand this field. It seems to be wastful and
introduces unnecessary complexity.

-- Enke

From: "proto man" <[email-masked]>
To: [email-masked]
Subject: AGGREGATION and RFC 1997
Date: Sat, 24 Mar 2001 06:35:57 -0000
Sender: [email-masked]

Hi,
  RFC 1997 does not specify whether duplicates need to be eliminated
  when aggregating communities...
  Are duplicate communties not expected?
Thanks,
-Pm
_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com

From: "proto man" <[email-masked]>
To: [email-masked]
Subject: Re: Confed question (RFC 3065)
Date: Sat, 24 Mar 2001 01:24:44 -0000
Sender: [email-masked]

Some more questions :

1) How one deals with AGGREGATOR-AS in a confederation environment.
(Do we always put confederation-id that would be loss of information
if we are sending the aggregate to a confederation peer )

2) Does any one enforce that paths that are being aggregated have same
MED,NEXTHOP.... ( major vendors don't seem to do ). Is it because this
does not cause any protocol problems...?

Thanks for any reply, any reply at all ;)
-pm

>From: "Vaibhavkumar J. Shah" <[email-masked]>
>Reply-To: [email-masked]
>To: BGP mailing list <[email-masked]>
>Subject: Confed question (RFC 3065)
>Date: Fri, 23 Mar 2001 16:37:30 -0500
>
>Am I missing something obvious here?  From RFC 3065:
>
>b) When a given BGP speaker advertises the route to a BGP speaker
>       located in a neighboring autonomous system that is a member of the
>
>       local autonomous system confederation, then the advertising
>       speaker shall update the AS_PATH attribute as follows:
>
>       1) ....
>
>       2) if the first path segment of the AS_PATH is not of type
>          AS_CONFED_SEQUENCE the local system shall prepend a new path
>          segment of type AS_CONFED_SEQUENCE to the AS_PATH, including
>          its own confederation identifier in that segment.
>
>
>Should the confederation identifier in the above para be interpreted as
>AS number?
>
>thanks
>
>VJ
>
>
>
>
>

_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com

From: "proto man" <[email-masked]>
To: [email-masked], [email-masked]
Subject: Re: Confed question (RFC 3065)
Date: Sat, 24 Mar 2001 01:07:49 -0000
Sender: [email-masked]

Hi,

  I see a relevant mail archive dated Feb 1999 :

=>In message <[email-masked]>, Rohit Dube 
write
>s:
=>> On Thu, 11 Feb 1999 09:49:24 PST [email-masked] writes:
=>> =>Hi,
=>> =>   AS_PATH modification rules about propagating a route to a
=>> =>CONFEDERATION-EBGP peer goes :
=>> =>
=>> =>   "2) if the first path segment of the AS_PATH is not of type
=>> =>AS_CONFED_SEQUENCE the local system shall prepend a new path segment 
of
=>> =>type AS_CONFED_SEQUENCE to the AS_PATH, including its own
=>> =>confederation identifier in that segment."
=>> =>^^^^^^^^^^^^^^^^^^^^^^^^
=>> =>
=>> =>   Question : Should this not be "its own AS number" instead.
=>>
=>> Yup you are right.
=>>
=>> The RFC may need an update on this as also on the segment type defines.
=>> AS_CONFED_SEQUENCE is defined as 4 whereas most deployed implementations
=>> that I am aware of actually use a type of 3.
=>>
=>> pst, care to update??
=>>
=>> --rohit.
=>

Hope this helps.
-pm

>From: "Vaibhavkumar J. Shah" <[email-masked]>
>Reply-To: [email-masked]
>To: BGP mailing list <[email-masked]>
>Subject: Confed question (RFC 3065)
>Date: Fri, 23 Mar 2001 16:37:30 -0500
>
>Am I missing something obvious here?  From RFC 3065:
>
>b) When a given BGP speaker advertises the route to a BGP speaker
>       located in a neighboring autonomous system that is a member of the
>
>       local autonomous system confederation, then the advertising
>       speaker shall update the AS_PATH attribute as follows:
>
>       1) ....
>
>       2) if the first path segment of the AS_PATH is not of type
>          AS_CONFED_SEQUENCE the local system shall prepend a new path
>          segment of type AS_CONFED_SEQUENCE to the AS_PATH, including
>          its own confederation identifier in that segment.
>
>
>Should the confederation identifier in the above para be interpreted as
>AS number?
>
>thanks
>
>VJ
>
>
>
>
>

_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com

Date: Fri, 23 Mar 2001 16:37:30 -0500
From: "Vaibhavkumar J. Shah" <[email-masked]>
Reply-To: [email-masked]
Organization: Nortel Networks
To: BGP mailing list <[email-masked]>
Subject: Confed question (RFC 3065)
Sender: [email-masked]

Am I missing something obvious here?  From RFC 3065:

b) When a given BGP speaker advertises the route to a BGP speaker
      located in a neighboring autonomous system that is a member of the

      local autonomous system confederation, then the advertising
      speaker shall update the AS_PATH attribute as follows:

      1) ....

      2) if the first path segment of the AS_PATH is not of type
         AS_CONFED_SEQUENCE the local system shall prepend a new path
         segment of type AS_CONFED_SEQUENCE to the AS_PATH, including
         its own confederation identifier in that segment.

Should the confederation identifier in the above para be interpreted as
AS number?

thanks

VJ

To: Ben Black <[email-masked]>
Cc: [email-masked]
Subject: Re: NEW_AS_SET? 
Date: Thu, 22 Mar 2001 21:58:27 -0800
From: Enke Chen <[email-masked]>
Sender: [email-masked]

Ben,
See the following text from the draft:

   To preserve AS path information with 4-octets AS numbers across OLD
   BGP speakers, this document defines a new AS path attribute, called
   NEW_AS_PATH.  This attribute is optional transitive and contains the
   AS path encoded with 4-octets AS numbers. The NEW_AS_PATH attribute
   has the same semantics as the AS_PATH attribute, except that it is
   optional transitive, and it carries 4-octets AS numbers.  The Type
   Code for this attribute is <TBD>

   To prevent the possible propagation of confederation path segments
   outside of a confederation, the path segment types AS_CONFED_SEQUENCE
   and AS_CONFED_SET [RFC3065] are declared invalid for the NEW_AS_PATH
   attribute.

Please note that "The NEW_AS_PATH attribute has the same semantics as the
AS_PATH attribute, except ...". So both AS_SET and AS_SEQUENECE are
valid for the NEW_AS_PATH attribute.

-- Enke

> Date: Thu, 22 Mar 2001 21:42:14 -0800
> From: Ben Black <[email-masked]>
> To: [email-masked]
> Subject: NEW_AS_SET?
> Message-ID: <[email-masked]>
> Mime-Version: 1.0
> Content-Type: text/plain; charset=us-ascii
> Content-Disposition: inline
> User-Agent: Mutt/1.2.5i
> Sender: [email-masked]
> Precedence: bulk
> 
> draft-ietf-idr-as4bytes-01.txt does not include a specification for
> a 4-byte-compatible AS_SET attribute.  is this intentional or an
> oversight?  if an oversight, i'm assuming the language would be
> straightforward for the authors to add.
> 
> 
> ben
> 
> 

Date: Thu, 22 Mar 2001 21:42:14 -0800
From: Ben Black <[email-masked]>
To: [email-masked]
Subject: NEW_AS_SET?
Sender: [email-masked]

draft-ietf-idr-as4bytes-01.txt does not include a specification for
a 4-byte-compatible AS_SET attribute.  is this intentional or an
oversight?  if an oversight, i'm assuming the language would be
straightforward for the authors to add.

ben

Date: Thu, 22 Mar 2001 11:46:08 -0500
From: Jeffrey Haas <[email-masked]>
To: Randy Bush <[email-masked]>
Cc: Susan Hares <[email-masked]>, [email-masked]
Subject: Re: Fwd:  For Alain Durand
Sender: [email-masked]

On Thu, Mar 22, 2001 at 09:49:55AM -0600, Randy Bush wrote:
> why is it necessary to expand the size of the router id?  does it need to be
> an ip address, or is it just an id?

The current specification lists it as:
: A given BGP speaker sets the value of its BGP Identifier to an IP
: address assigned to that BGP speaker.  The value of the BGP Identifier
: is determined on startup and is the same for every local interface
: and every BGP peer.

Outside of an AS, this value is often exposed via the aggregator
attribute (which also carries a 2-byte AS).

This value is used in other extensions to the protocol (such as route
reflection) but the semantics can be completely local for those.

> randy

-- 
Jeff Haas 
NextHop Technologies

From: Randy Bush <[email-masked]>
To: Susan Hares <[email-masked]>
Cc: [email-masked]
Subject: Re: Fwd:  For Alain Durand
Date: Thu, 22 Mar 2001 09:49:55 -0600
Sender: [email-masked]

why is it necessary to expand the size of the router id?  does it need to be
an ip address, or is it just an id?

randy

Date: Thu, 22 Mar 2001 10:44:36 -0500
To: [email-masked]
From: Susan Hares <[email-masked]>
Subject: Fwd:  For Alain Durand
Sender: [email-masked]

>
>The draft draft-dupont-durand-idr-ipv6-bgp-routerid-00.txt was discussed in
>the meeting this afternoon.
>
>We would like to bring the following discussion to the mailing list.
>
>1) We would like to make clear that this proposal is for short term
>     operation of IPv6 only networks. It does not pretend to address
>     long term issues. It is a candidate for BCP, meaning it can be
>     redefined anytime.
>
>2) We would like to know if the suggested format is OK.
>     We already agreed on changing the prefix 0xF to 0xE
>     as suggested by a comment received by the authors.
>     Specifically,  the issue is: is  4096 large enough TODAY
>     to accommodate all the IPv6 only BGP speakers
>     which need a unique router-ID within an AS ?
>
>3) We did not address the case of 4-byte AS number because
>     we think that this is part of a long term solution and we would like
>     to suggest IDR to study 128 bit Router-ID solution along the side of
>     4-BYTE AS number.
>
>         - Alain & Francis

To: [email-masked], [email-masked]
Subject: Re: I speak too quickly, so here it is in writing.
Cc: [email-masked]
Date: Thu, 22 Mar 2001 06:46:23 +0100 (CET)
From: [email-masked] (Sean Doran)
Sender: [email-masked]

| Although these are rather fine examples of using communities to shape
| your traffic, the coarse-grained control of "add <n> to my aspath
| length" isn't there.  Any particular reason?

Never been asked for it by a customer.

Become one, or bribe him, and BC or a colleague will surely
give you such a community to play with to your heart's content. :-)

I will read your draft in spare moments in the next few days.

	Sean.

Date: Wed, 21 Mar 2001 23:31:15 -0500
From: Jeffrey Haas <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re: I speak too quickly, so here it is in writing.
Sender: [email-masked]

On Wed, Mar 21, 2001 at 11:37:00PM +0100, [email-masked] wrote:
> I refer everyone to the following command line:
> 
> whois -h whois.ripe.net AS1755

Thanks for the suggestion.

> As Rudiger notes, there is more to it than supporting simple
> prepend, as one engages in increasingly significant multihoming,

Although these are rather fine examples of using communities to shape
your traffic, the coarse-grained control of "add <n> to my aspath
length" isn't there.  Any particular reason?

> I do not think that there are any situations which really call
> for a protocol change to BGP4, especially one that is dangerous
> if there is any forgetful-filtering.

If you have the impression that the suggested ASE draft was
"dangerous" to anyone but the customer in a misconfiguration,
you may wish to review 

http://www.nexthop.com/standards/drafts/draft-jhaas-ase-01.txt

And it is only "dangerous" in the sense of installing routes to
yourself.  This already can happen as a result of other misconfigurations
and is no more dangerous than installing an aggregate containing
your address space.

As mentioned, this didn't make it out before the deadline and
fixed the issue of requiring filtering.

> Anyone who wishes to turn this and similar examples into a BCP
> is more than welcome to poll me for help.

I'll probably take you up on this.

> 	Sean.

-- 
Jeff Haas 
NextHop Technologies

From: [email-masked]
To: [email-masked]
Subject: I speak too quickly, so here it is in writing.
Date: Wed, 21 Mar 2001 23:37:00 +0100 (CET)
Sender: [email-masked]

I refer everyone to the following command line:

whois -h whois.ripe.net AS1755

WARNING the output is LONG

Scan down through the evidence of significant multihoming to
the Community support definitions section, or more precisely,
to "Communities to control traffic"

An RBS is a customer router.  ("regional boundary system" for history wonks).

As Rudiger notes, there is more to it than supporting simple
prepend, as one engages in increasingly significant multihoming,
however it is not intractable, and does not impose much workload
on networks which have significant multihomig to begin with
(and it's a one-shot thing anyway to set up some boilerplate).

I do not think that there are any situations which really call
for a protocol change to BGP4, especially one that is dangerous
if there is any forgetful-filtering.

Anyone who wishes to turn this and similar examples into a BCP
is more than welcome to poll me for help.

	Sean.
- --
Sean Doran <[email-masked]>

Date: Wed, 21 Mar 2001 17:32:12 -0500 (EST)
From: abha ahuja <[email-masked]>
To: [email-masked]
Subject: pointer to convergence papers
Sender: [email-masked]

With all of this convergence discussion, it reminded me that I don't know
if anyone ever posted the links to the papers that Craig and I did
regarding convergence...

I've linked them to http://www.merit.edu/~ahuja.

Please let us know if you have any questions.

=abha ;)

++++++++++++++++++
abha ahuja
<[email-masked]>

Date: Wed, 21 Mar 2001 11:35:51 -0500
To: [email-masked]
From: Susan Hares <[email-masked]>
Subject: Charter for IDR
Sender: [email-masked]

--=====================_691584==_.ALT
Content-Type: text/plain; charset="us-ascii"; format=flowed

Inter-Domain Routing (idr) Charter
Inter-Domain Routing (idr)
Last Modified: 9-March-01
Chair(s):
mailto:[email-masked]
mailto:[email-masked]
Routing Area Director(s):
mailto:[email-masked]
mailto:[email-masked]
Routing Area Advisor:
mailto:[email-masked]
Mailing Lists:
General Discussion:[email-masked]
To Subscribe: [email-masked]
Archive: ftp://ftp.merit.edu/mail.archives/idr

Description of Working Group:
The main list for this working group is [email-masked]
- List: [email-masked]
- To Subscribe: [email-masked]
- Archive: merit.edu:/pub/archive/idr

The Inter-Domain Routing Working Group is chartered to standardize and promote
the Border Gateway Protocol Version 4 (BGP-4) as scalable inter-autonomous
system routing protocols capable of supporting policy based routing for TCP/IP
internets. The objective is to promote the use of BGP-4 to support IP 
version 4
and IP version 6. The working group will continue to work on the scaling of
BGP.

Goals of this working group:

1) BGP-4 (standards track)
This document contains the specification of the BGP protocol that enables 
it to
be used as a protocol for exchange of ``inter-autonomous system information''
among routers to support forwarding of IPv4 packets across multiple autonomous
systems. The standardization report for BGP-4 will be completed along with
the BGP-4 MIB.
2) BGP-4 MIB (standards track)
The BGP-4 document contains the MIB definitions for standard BGP-4. This MIB
document predates many current "MIB" formatting specifications, and does not
contain many optional BGP functions. This MIB will be revised for the current
format. After the revised MIB has been completed, the BGP-4 MIB
standardization report will be completed.
3) Review and Evaluate Revisions of Existing RFC on AS Confederations,
Route
Refresh, and Extended Communities.
In addition to reviewing the revisions for AS Confederations and Route
Refresh, the working group will consider the extended communities draft.
4) BGP 4 MIBs for AS Confederations, Route Reflection, Communities and
Extended Communities AS confederations, Route Reflection, Communities and
Extended Communities are optional RFCs for BGP in use in the Internet.
MIB documents for each of these optional RFCs will be completed.
5) MP BGP - Review and evaluate for current RFC 2858, RFC 2547 (MPBGP for
IPv6), and creation of MPBGP MIBs These documents contain information
about passing information for Address Families IPv6 and IPv4 with or
without a SubAddress. MIB Documents for the MPBGP will created for the
MPBGP.
6) Evaluate Cooperative Route Filtering Capability for BGP-4
7) Scaling Issues in BGP
We will discuss BGP Scaling issues such as 4 Byte AS Path, BGP table size
increase, and BGP Convergence issues.

Goals and Milestones:
Mar '01  Submit new MIB documents for Optional Capabilities,
                 Route Reflection, Communities, Confederations to the IESG.
                 Review revisions to AS Confederation, BGP Outbound 
Filtering and BGP
                 traffic Engineering. Extended Communities, Graceful 
Restart, and Global
                 Communities for submission to IESG. Submit Drafts to the IESG.

Apr '01 Complete revisions of BGP-4 MIB documents for BGP-4 and submit
                    them to the IESG.

                   Working Group Last call on the additional MIBs for 
optional Capabilities,
                    Route Reflection, Communities, Confederations and 
submit to IESG.

May '01 Prepare standardization reports for BGP-4 MIB and BGP-4 for
                 consideration as an Internet Standard. Send to working 
group for review.
                 Prepare MPBGP MIB documents for MPBGP for Multicast and 
MPBGP for IP v6.
                 Send to working group for review.

Jul '01         Submit standardization report for MP-BGP for Multicast and 
IP v6
                 as a Proposed Internet Standard to the IESG.
Aug '01 Review standardization report for BGP-4 and BGP MIB, MP-BGP for
                 Multicast and IP v6 as Proposed Internet standard.
                 Review the MPBGP MIBs for IP v4 and IP v6 and submit to 
the IESG.

Internet-Drafts:

A Border Gateway Protocol 4 (BGP-4) (135269 bytes)
Definitions of Managed Objects for the Fourth Version of Border Gateway 
Protocol
(BGP-4) (62696 bytes)
Cooperative Route Filtering Capability for BGP-4 (21791 bytes)
Graceful Restart Mechanism for BGP (18824 bytes)
BGP support for four-octet AS number space (14777 bytes)

Request For Comments:

A Border Gateway Protocol 3 (BGP-3) (RFC 1267) (80724 bytes)
Definitions o Managed Objects for the Border Gateway Protocol (Version 3) (RFC
1269) (25717 bytes)
Experience with the BGP Protocol (RFC 1266) (21938 bytes)
BGP Protocol Analysis (RFC 1265) (20728 bytes)
Default Route Advertisement In BGP2 And BGP3 Versions Of The Border Gateway
Protocol (RFC 1397) (4124 bytes)
BGP OSPF Interaction (RFC 1403) (36174 bytes)
Definitions of Managed Objects for the Fourth Version of the Border Gateway
Protocol (BGP-4) using SMIv2 (RFC 1657) (45505 bytes)
BGP4/IDRP for IP---OSPF Interaction (RFC 1745) (43675 bytes)
BGP-4 Protocol Analysis (RFC 1774) (23823 bytes)
Experience with the BGP-4 protocol (RFC 1773) (19936 bytes)
A Border Gateway Protocol 4 (BGP-4) (RFC 1771) (131903 bytes)
A BGP/IDRP Route Server alternative to a full mesh routing (RFC 1863) (37426
bytes)
Guidelines for creation, selection, and registration of an Autonomous System
(AS) (RFC 1930) (22073 bytes)
BGP Route Reflection An alternative to full mesh IBGP (RFC 1966) (14320 bytes)
An Application of the BGP Community Attribute in Multi-home Routing (RFC 1998)
(16953 bytes)
BGP Communities Attribute (RFC 1997) (8275 bytes)
Using a Dedicated AS for Sites Homed to a Single Provider (RFC 2270) (12063
bytes)
Protection of BGP Sessions via the TCP MD5 Signature Option (RFC 2385) (12315
bytes)
BGP Route Flap Damping (RFC 2439) (86376 bytes)
A Framework for Inter-Domain Route Aggregation (RFC 2519) (25394 bytes)
Use of BGP-4 Multiprotocol Extensions for IPv6 Inter-Domain Routing (RFC 2545)
(10209 bytes)
BGP Route Reflection An alternative to full mesh IBGP (RFC 2796) (20174 bytes)
Capabilities Advertisement with BGP-4 (RFC 2842) (8366 bytes)
Multiprotocol Extensions for BGP-4 (RFC 2858) (23305 bytes)
Route Refresh Capability for BGP-4 (RFC 2918) (7354 bytes)
Autonomous System Confederations for BGP (RFC 3065) (20529 bytes)

--=====================_691584==_.ALT
Content-Type: text/html; charset="us-ascii"

<html>
Inter-Domain Routing (idr) Charter<br>
Inter-Domain Routing (idr)<br>
Last Modified: 9-March-01<br>
Chair(s): <br>
<font color="#0000FF"><u><a href="mailto:[email-masked]" eudora="autourl">mailto:skh@nexthop.</a><a href="mailto:[email-masked]" eudora="autourl">com</a></u></font>
<br>
<font color="#0000FF"><u><a href="mailto:[email-masked]" eudora="autourl">mailto:yakov@cisco.</a><a href="mailto:[email-masked]" eudora="autourl">com<br>
</a></u></font>Routing Area Director(s): <br>
<font color="#0000FF"><u><a href="mailto:[email-masked]" eudora="autourl">mailto:oran@cisco.</a><a href="mailto:[email-masked]" eudora="autourl">com</a></u></font> <br>
<font color="#0000FF"><u><a href="mailto:[email-masked]" eudora="autourl">mailto:rcoltun@redback.</a><a href="mailto:[email-masked]" eudora="autourl">com<br>
</a></u></font>Routing Area Advisor: <br>
<font color="#0000FF"><u><a href="mailto:[email-masked]" eudora="autourl">mailto:rcoltun@redback.</a><a href="mailto:[email-masked]" eudora="autourl">com<br>
</a></u></font>Mailing Lists: <br>
General Discussion:[email-masked] <br>
To Subscribe: [email-masked] <br>
Archive: <a href="ftp://ftp.merit.edu/mail.archives/idr" eudora="autourl"><font color="#0000FF"><u>ftp://ftp.merit.edu/mail.archives/</a><a href="ftp://ftp.merit.edu/mail.archives/idr" eudora="autourl">idr<br>
<br>
</a></u></font>Description of Working Group: <br>
The main list for this working group is [email-masked] <br>
- List: [email-masked] <br>
- To Subscribe: [email-masked] <br>
- Archive: merit.edu:/pub/archive/idr<br>
<br>
<br>
The Inter-Domain Routing Working Group is chartered to standardize and promote <br>
the Border Gateway Protocol Version 4 (BGP-4) as scalable inter-autonomous <br>
system routing protocols capable of supporting policy based routing for TCP/IP <br>
internets. The objective is to promote the use of BGP-4 to support IP version 4 <br>
and IP version 6. The working group will continue to work on the scaling of <br>
BGP.<br>
<br>
Goals of this working group:<br>
<br>
1) BGP-4 (standards track) <br>
This document contains the specification of the BGP protocol that enables it to <br>
be used as a protocol for exchange of ``inter-autonomous system information'' <br>
among routers to support forwarding of IPv4 packets across multiple autonomous <br>
systems. The standardization report for BGP-4 will be completed along with <br>
the BGP-4 MIB.<br>
2) BGP-4 MIB (standards track) <br>
The BGP-4 document contains the MIB definitions for standard BGP-4. This MIB <br>
document predates many current &quot;MIB&quot; formatting specifications, and does not <br>
contain many optional BGP functions. This MIB will be revised for the current <br>
format. After the revised MIB has been completed, the BGP-4 MIB <br>
standardization report will be completed.<br>
3) Review and Evaluate Revisions of Existing RFC on AS Confederations, <br>
Route <br>
Refresh, and Extended Communities. <br>
In addition to reviewing the revisions for AS Confederations and Route <br>
Refresh, the working group will consider the extended communities draft.<br>
4) BGP 4 MIBs for AS Confederations, Route Reflection, Communities and <br>
Extended Communities AS confederations, Route Reflection, Communities and <br>
Extended Communities are optional RFCs for BGP in use in the Internet. <br>
MIB documents for each of these optional RFCs will be completed.<br>
5) MP BGP - Review and evaluate for current RFC 2858, RFC 2547 (MPBGP for <br>
IPv6), and creation of MPBGP MIBs These documents contain information <br>
about passing information for Address Families IPv6 and IPv4 with or <br>
without a SubAddress. MIB Documents for the MPBGP will created for the <br>
MPBGP.<br>
6) Evaluate Cooperative Route Filtering Capability for BGP-4<br>
7) Scaling Issues in BGP <br>
We will discuss BGP Scaling issues such as 4 Byte AS Path, BGP table size <br>
increase, and BGP Convergence issues.<br>
<br>
<br>
Goals and Milestones: <br>
Mar '01&nbsp; Submit new MIB documents for Optional Capabilities, <br>
<x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</x-tab><x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</x-tab>Route Reflection, Communities, Confederations to the IESG.<br>
<x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</x-tab><x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</x-tab>Review revisions to AS Confederation, BGP Outbound Filtering and BGP <br>
<x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</x-tab><x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</x-tab>traffic Engineering. Extended Communities, Graceful Restart, and Global <br>
<x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</x-tab><x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</x-tab>Communities for submission to IESG. Submit Drafts to the IESG.<br>
<br>
Apr '01 Complete revisions of BGP-4 MIB documents for BGP-4 and submit <br>
<x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; them to the IESG.<br>
<br>
<x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Working Group Last call on the additional MIBs for optional Capabilities, <br>
<x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Route Reflection, Communities, Confederations and submit to IESG.<br>
<br>
May '01 Prepare standardization reports for BGP-4 MIB and BGP-4 for <br>
<x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</x-tab><x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</x-tab>consideration as an Internet Standard. Send to working group for review. <br>
<x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</x-tab><x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</x-tab>Prepare MPBGP MIB documents for MPBGP for Multicast and MPBGP for IP v6. <br>
<x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</x-tab><x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</x-tab>Send to working group for review.<br>
<br>
Jul '01 <x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</x-tab>Submit standardization report for MP-BGP for Multicast and IP v6 <br>
<x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</x-tab><x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</x-tab>as a Proposed Internet Standard to the IESG.<br>
Aug '01 Review standardization report for BGP-4 and BGP MIB, MP-BGP for <br>
<x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</x-tab><x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</x-tab>Multicast and IP v6 as Proposed Internet standard. <br>
<x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</x-tab><x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</x-tab>Review the MPBGP MIBs for IP v4 and IP v6 and submit to the IESG.<br>
<br>
Internet-Drafts: <br>
<br>
A Border Gateway Protocol 4 (BGP-4) (135269 bytes) <br>
Definitions of Managed Objects for the Fourth Version of Border Gateway Protocol <br>
(BGP-4) (62696 bytes) <br>
Cooperative Route Filtering Capability for BGP-4 (21791 bytes) <br>
Graceful Restart Mechanism for BGP (18824 bytes) <br>
BGP support for four-octet AS number space (14777 bytes) <br>
<br>
<br>
Request For Comments:<br>
<br>
A Border Gateway Protocol 3 (BGP-3) (RFC 1267) (80724 bytes) <br>
Definitions o Managed Objects for the Border Gateway Protocol (Version 3) (RFC <br>
1269) (25717 bytes) <br>
Experience with the BGP Protocol (RFC 1266) (21938 bytes) <br>
BGP Protocol Analysis (RFC 1265) (20728 bytes) <br>
Default Route Advertisement In BGP2 And BGP3 Versions Of The Border Gateway <br>
Protocol (RFC 1397) (4124 bytes) <br>
BGP OSPF Interaction (RFC 1403) (36174 bytes) <br>
Definitions of Managed Objects for the Fourth Version of the Border Gateway <br>
Protocol (BGP-4) using SMIv2 (RFC 1657) (45505 bytes) <br>
BGP4/IDRP for IP---OSPF Interaction (RFC 1745) (43675 bytes) <br>
BGP-4 Protocol Analysis (RFC 1774) (23823 bytes) <br>
Experience with the BGP-4 protocol (RFC 1773) (19936 bytes) <br>
A Border Gateway Protocol 4 (BGP-4) (RFC 1771) (131903 bytes) <br>
A BGP/IDRP Route Server alternative to a full mesh routing (RFC 1863) (37426 <br>
bytes) <br>
Guidelines for creation, selection, and registration of an Autonomous System <br>
(AS) (RFC 1930) (22073 bytes) <br>
BGP Route Reflection An alternative to full mesh IBGP (RFC 1966) (14320 bytes) <br>
An Application of the BGP Community Attribute in Multi-home Routing (RFC 1998) <br>
(16953 bytes) <br>
BGP Communities Attribute (RFC 1997) (8275 bytes) <br>
Using a Dedicated AS for Sites Homed to a Single Provider (RFC 2270) (12063 <br>
bytes) <br>
Protection of BGP Sessions via the TCP MD5 Signature Option (RFC 2385) (12315 <br>
bytes) <br>
BGP Route Flap Damping (RFC 2439) (86376 bytes) <br>
A Framework for Inter-Domain Route Aggregation (RFC 2519) (25394 bytes) <br>
Use of BGP-4 Multiprotocol Extensions for IPv6 Inter-Domain Routing (RFC 2545) <br>
(10209 bytes) <br>
BGP Route Reflection An alternative to full mesh IBGP (RFC 2796) (20174 bytes) <br>
Capabilities Advertisement with BGP-4 (RFC 2842) (8366 bytes) <br>
Multiprotocol Extensions for BGP-4 (RFC 2858) (23305 bytes) <br>
Route Refresh Capability for BGP-4 (RFC 2918) (7354 bytes) <br>
Autonomous System Confederations for BGP (RFC 3065) (20529 bytes)<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
</html>

--=====================_691584==_.ALT--

From: "proto man" <[email-masked]>
To: [email-masked]
Subject: BGP basic questions
Date: Wed, 21 Mar 2001 10:53:52 -0000
Sender: [email-masked]

Hi,

Sorry for the repeat e-mail :

Just reposting forwarded message (Thanks Susan) under new heading to hope 
for some reply.

I have some basic implementation questions in  BGP :

1) How should a peer react if it receives an attribute type code of 0 or
255 Should it bother checking the attribute flags for transitive/mandatory
nature or send notification? If it is to send notification what is the
error code, subcode should be?

2) Let's say there are two IBGP paths p1 (nexthop N1), p2 (nextop N2) to a
prefix 'p'. Both N1 and N2 are reachable through OSPF. N1 has a
lower cost than N2. All other parameters being equal I guess p1 wins over
p2. Now let's say N2 suddenly is reachable now through a RIP route,
I am guessing the routes should be resorted without comparing NEXTHOP's
IGP cost. This would be the same case if an EBGP directly connected route
should come in.

So should an implementation keep track of this comparison status while
sorting the routes and recalculating whenever the comparison criteria
breaks down?

3) Does the ASPATH aggregation algorithm specified in the bgp draft
automatically extend to confederation paths?

4) Is an ASPATH segment with 0 as the number of AS valid?
If not should the error code be MALFORM_ASPATH....?

5) Is this interpretation of minRouteAdvertisementInterval correct
behaviour ?

  IBGP routes advertise ASAP with a delay for packing
  EBGP routes advertise with a bounded minimum delay of
minRouteAdvertisementInterval.
  withdraws - send ASAP.

Or does it depend on the type of the peer we are sending routes to...
for e.g 30 second bounded minimum delay for EBGP peers and
    5 second bounded minimum delay for IBGP peers ( atleast this is
what i see as default in cisco - probably not a question for this mailing
list...)

6) What should be the default action if we receive originator ID,
  CLUSTER-LIST from  an EBGP peer?

7) Route damping draft advises route-damping be applied only to EBGP routes.
  What about confederations? Can route-damping be applied to
confederation EBGP
  routes?

8) Is there any requirement that confederation sub-ases have private AS
numbers?

Thanks,

9) 6.3 Connection collision states :

          If the value of the local BGP Identifier is less than the
     remote one, the local system closes BGP connection that already
     exists (the one that is already in the OpenConfirm state), and
     accepts BGP connection initiated by the remote system.

                -- why cannot be the connection that already exists be
the one initiated by
     the remote speaker?

     3. Otherwise, the local system closes newly created BGP connection
     (the one associated with the newly received OPEN message), and
     continues to use the existing one (the one that is already in the
     OpenConfirm state).

                -- why should the existing connection not be the one
initiated by the remote
      speaker? ( and the newly created connection be the one initiated by
the local speaker )

Thanks,
-pm

_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com

From: "proto man" <[email-masked]>
To: [email-masked]
Subject: BGP implementation question
Date: Wed, 21 Mar 2001 09:11:49 -0000
Sender: [email-masked]

Hi,

  These are implementation specific questions :
  If this is not appropriate for this list please let me know.

1) If a neighbor's outbound/inbound policy changes in terms of a aspath
  regular expression change, ideally only those routes with the
  affected aspaths should be  re-evaluated.
(same for a community regular expression policy).

  I am unable to figure out a database that would selectively
  associate routes that are affected by such user defined regular
  expressions.

  Does any vendor do this?

  I understand if the policy change is in terms of a prefix change
  one could use the normal routing database (patricia/dpt) to associate
  affected routes...

  Does cisco's clear ip bgp peer soft-out actually manages this..
  Or for that matter junipers'.. ( again apologize for vendor specific
  questions)

2) Is there a general guideline for trade-offs between
   route-packing and time taken for update...
   For e.g with two routes with source attributes A1,A2 could
   conceivably go to same outbound attribute A3 ( negating a pointer
   comparison between A1,A2 for route-packing )....

Thanks,
-pm
_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com

Date: Tue, 20 Mar 2001 15:47:28 -0500
To: [email-masked]
From: Susan Hares <[email-masked]>
Subject: Fwd: Re: IDR agenda for next week
Cc: [email-masked]
Sender: [email-masked]

Forwarding this information for the priya.

Sue

>Hi,
>Sorry to bother you.
>
>I have been unsuccessfully trying to post the following question to
>[email-masked] for few weeks now. (even tried [email-masked])
>I am subscribed to the idr list.
>I did not get any reply from the administrator too.
>
>Can you please forward these questions?
>
>Thanks a lot
>-priya moniker
>
>
>
>Hi,
>
>I apologize if there are multiple versions of this question...
>I have been unsuccessful in sending this to the mailing list and hence
>the different distribution lists...
>
>I have some basic implementation questions in  BGP :
>
>1) How should a peer react if it receives an attribute type code of 0 or
>255 Should it bother checking the attribute flags for transitive/mandatory
>nature or send notification? If it is to send notification what is the
>error code, subcode should be?
>
>2) Let's say there are two IBGP paths p1 (nexthop N1), p2 (nextop N2) to a
>prefix 'p'. Both N1 and N2 are reachable through OSPF. N1 has a
>lower cost than N2. All other parameters being equal I guess p1 wins over
>p2. Now let's say N2 suddenly is reachable now through a RIP route,
>I am guessing the routes should be resorted without comparing NEXTHOP's
>IGP cost. This would be the same case if an EBGP directly connected route
>should come in.
>
>So should an implementation keep track of this comparison status while
>sorting the routes and recalculating whenever the comparison criteria
>breaks down?
>
>3) Does the ASPATH aggregation algorithm specified in the bgp draft
>automatically extend to confederation paths?
>
>4) Is an ASPATH segment with 0 as the number of AS valid?
>  If not should the error code be MALFORM_ASPATH....?
>
>
>5) Is this interpretation of minRouteAdvertisementInterval correct
>behaviour ?
>
>   IBGP routes advertise ASAP with a delay for packing
>   EBGP routes advertise with a bounded minimum delay of
>minRouteAdvertisementInterval.
>   withdraws - send ASAP.
>
>  Or does it depend on the type of the peer we are sending routes to...
>for e.g 30 second bounded minimum delay for EBGP peers and
>     5 second bounded minimum delay for IBGP peers ( atleast this is
>what i see as default in cisco - probably not a question for this mailing
>list...)
>
>6) What should be the default action if we receive originator ID,
>   CLUSTER-LIST from  an EBGP peer?
>
>
>7) Route damping draft advises route-damping be applied only to EBGP routes.
>   What about confederations? Can route-damping be applied to 
> confederation EBGP
>   routes?
>
>8) Is there any requirement that confederation sub-ases have private AS 
>numbers?
>
>Thanks,
>
>
>
>9) 6.3 Connection collision states :
>
>           If the value of the local BGP Identifier is less than the
>      remote one, the local system closes BGP connection that already
>      exists (the one that is already in the OpenConfirm state), and
>      accepts BGP connection initiated by the remote system.
>
>                 -- why cannot be the connection that already exists be 
> the one initiated by
>      the remote speaker?
>
>      3. Otherwise, the local system closes newly created BGP connection
>      (the one associated with the newly received OPEN message), and
>      continues to use the existing one (the one that is already in the
>      OpenConfirm state).
>
>                 -- why should the existing connection not be the one 
> initiated by the remote
>       speaker? ( and the newly created connection be the one initiated by 
> the local speaker )
>
>
>Thanks,
>-pm
>
>
>
>_________________________________________________________________
>Get your FREE download of MSN Explorer at http://explorer.msn.com
>

Date: Thu, 15 Mar 2001 09:18:50 -0500
To: [email-masked]
From: Susan Hares <[email-masked]>
Subject: IDR agenda for next week
Cc: yakov Rekhter <[email-masked]>, [email-masked], Keyur Patel <[email-masked]>, [email-masked], [email-masked], Rob Coltun <[email-masked]>
Sender: [email-masked]

IDR Meeting

Date: 3/21/01
Time: 15:30pm - 17:30pm

1) BGP Scaling  [40 minutes]
        Geoff Houston's presentation of BGP scaling from NANOG  [15 minutes]
        Abha Ahuja's presentation from NANOG  [10 minutes]
        4 Byte AS Implementation Experience and Deployment [5 minutes]

  2) BGP Convergence  [15 minutes]
        BGP Convergence Benchmarks [ Sue Hares, Howard Berkowitz]  [15 minutes]

  3) BGP, VPNs and Scaling  [20 minutes]
  	Scaling update from PPVPN and TWEG [10 minutes]
         ORF for AS path  [Keyur Patel and Sue Hares]  [10 minutes]

  4)  BGP MIBs and standardization   [10 minutes]

        1) BGP-4 MIB revision,  Route Reflect, Communities, AS Confederations
             [Jeff Haas]
        2) BGP-opt mib and upcoming  MPBGP MIBs  [Jeff Haas]
        3) Standardization Reports  [Jeff Haas]
        4) Historical MPBGP [Dave Meyers]

5) Discussion of Protocol Work for BGP Scaling [20 minutes]

        Next Generation BGP Requirements from IRTF [10 minutes]
        Discussion of BGP Requirements for Next Generation [5 minutes]

6) BGP Charter Bashing  [10 minutes]

Date: Tue, 13 Mar 2001 23:01:02 -0500
From: Jeffrey Haas <[email-masked]>
To: Cliff Sojourner <[email-masked]>
Cc: [email-masked], [email-masked], Alfred Nothaft <[email-masked]>
Subject: Re: BGP4-OPT-PARAMETERS-MIB comments
Sender: [email-masked]

On Tue, Mar 13, 2001 at 04:51:29PM -0800, Cliff Sojourner wrote:
> Hi folks,

Hi Cliff,

> no offense -seriously- but this is the absolute worst MIB I have seen in
> a real long time.  is this an April Fools' joke?

Not intentionally.  I was trying to get the MIB in for the intitial
submission before the deadline.  The MIB went out with several
errors.  Unfortunately, the MIB errors weren't caught because
the UCD-snmp package didn't catch them.  We're now using the 
SMIlint package which does much better.
(Poor excuse, but its the only one I can offer. :-|)

Updates which include many fixes to the BGP mib and the bgp-opt
mib are available at 

http://www.nexthop.com/standards/wip_abstracts.shtml

Specifically:
http://www.nexthop.com/standards/drafts/draft-ietf-idr-bgp4-mib-07.txt
http://www.nexthop.com/standards/drafts/draft-jhaas-bgp4-mib-opt-01.txt

> should be BGP4-MIB.

Its bgp-mib.

[several SMI bugs elided...]

> at a semantic level, specify how the rows of bgpAuthenticationTable are
> created.  it appears there are automatically two (sent, received) for
> _every_ peer?
> 
> instead of using -1 for bgpAuthenticationCode to represent "no
> authentication", consider the following:
> 
> - add a RowStatus object to control creating entries for a peer
> - add Administrative and Operational Status for controlling whether
> authentication is enabled or disabled for that peer.

Thanks for the suggestions. 

> if bgpProtocolExtensions really is an OID registration point, then it
> should be OBJECT-IDENTITY, not OBJECT-TYPE.  anyways I can not
> understand the DESCRIPTION to see exactly what this object value should
> be.

This is a "mount point" for Capability Negotiated protocol extension
MIBs.  It has been suggested in private e-mail that the "enterprise
specific" portion of this "mount point" really doesn't belong in the
BGP MIB.  It probably will be removed.

> Cliff

P.S. If I was going for an April Fool's MIB, I probably would have
done a MIB for the engineers lunching location election protocol. :-)

-- 
Jeff Haas 
NextHop Technologies

Date: Fri, 09 Mar 2001 20:38:06 -0500
To: [email-masked]
From: Susan Hares <[email-masked]>
Subject: Agenda topics for IETF
Cc: [email-masked], [email-masked]
Sender: [email-masked]

Please send any agenda topics to [email-masked] and [email-masked].
Here's a preview of some of the Agenda topics:

1) BGP Scaling issues
	Normal BGP , BGP for VPNs , and Traffic Engineering and BGP

2) BGP Convergence
	BGP convergence benchmark draft, and BGP MED churn type II

3) BGP MIBs and BGP standardization of BGP-4 (draft 12), MPBGP multicast 
and IP v6

We have had 2 BGP and Traffic Engineering drafts.   Is there any interest in
these drafts?

Sue Hares

From: "Dmitri Krioukov" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>, "John G. Scudder" <[email-masked]>, <[email-masked]>
Subject: RE: BGP Route Oscillation
Date: Fri, 2 Mar 2001 17:13:25 -0500
Importance: Normal
Sender: [email-masked]

Tim,

I'm afraid I do not follow your proposed terminology.
>From the implementation perspective, there are input
policies, selection process, installed routes and
output policies. Where exactly in this picture do
your "apparent" and "actual" policies fit?

Another question would be why you decided to ignore
the MED attribute in you paper? Just for simplification
reasons or because it doesn't really fit into your policy-
only picture?

In other words, I think that if we understand the term
"policy" traditionally, then I'd agree that the MED-related
oscillations represent a "different animal" since MED comparison
here belongs more to the selection process than to the policies.
On the other hand, I do agree with you that if we consider BGP
operation as a whole (including all four elements listed at the
very beginning -- call the result "actual" policies or not),
then all currently known oscillations are caused by
inconsistencies on the global level.
--
dima.

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]On Behalf Of tim
> griffin
> Sent: Friday, February 16, 2001 10:41 AM
> To: John G. Scudder
> Cc: [email-masked]; [email-masked]
> Subject: Re: BGP Route Oscillation
>
>
> John,
>
> Thanks for your comments.
>
> >Many of the papers you cite talk about discovering incommensurable
> >policies, either statically or through protocol extensions.  Since
> >the MED oscillation problem isn't a result of configured policy, I
> >don't see how such research can directly apply.
> >
> >In the MED oscillation case, you can get the oscillation even if
> >there is no configured policy at all.  Thus I think it's a different
> >animal than the oscillation cases which have been covered in the
> >literature to date.
>
> Let's call the policies we type into a configuration file "apparent"
> routing policies, and call the dynamic ranking of routes that gets
> realized on a router the "actual" routing policies.  The actual
> policies on a router result from a (sometimes complex) interaction
> of its apparent policies, the BGP route selection process, and the
> actual policies of neighboring routers.  I think most of the papers
> on BGP oscillation are really about incommensurable *actual* policies,
> and so they do apply.  In the MED oscillation examples routers
> end up with actual policies that together have no solution.
>
> Actual policies are in principle close to the function discussed in
> draft-ietf-idr-bgp4-12.txt, Section 9.1:
>
>    The selection process is formalized by defining a function that takes
>    the attribute of a given route as an argument and returns a non-
>    negative integer denoting the degree of preference for the route.
>    The function that calculates the degree of preference for a given
>    route shall not use as its inputs any of the following: the existence
>    of other routes, the non-existence of other routes, or the path
>    attributes of other routes. Route selection then consists of
>    individual application of the degree of preference function to each
>    feasible route, followed by the choice of the one with the highest
>    degree of preference.
>
> Perhaps MED is bad because it can make the translation from
> apparent to actual policies very hard to understand. MED is special
> in that it is not "compositional" like other steps in the
> route selection process.  If we ignore MED, then we can think of
> ranking each route independently of other routes just as implied
> by the description of the ranking function in Section 9.1 quoted above.
> That is, we can think of the selection process as imposing some sort
> of simple lexical order on routes. But MED breaks this nice model,
> since its rule is conditional -- compare MEDs only when the next hop
> ASNs are the same.  Now actual policies are no longer simple and intuitive
> rankings of paths.  A route's rank can now depend on the existence,
> or non-existence of other routes.  But isn't this the kind of ranking
> that is explicitly prohibited in Section 9.1? If so, why is it allowed
> in the Phase 2 tie breaking?  I have always assumed that prohibition
> of "existential criteria" was there so that human beings could understand
> (and debug) the actual routing policies that result from their
> apparent ones.
>
> A simple fix might be to eliminate the MED step entirely, and
> add the MED value to the igp cost of a route.  I realize this will
> cause other problems.  However, it is easy to understand and may
> cover most of the current uses of MED just fine. It would
> require more cooperation between those sending and those listening
> to MEDs, which might not be a bad thing.
>
> > [...] The difference is that it is at least
> >possible to fix the policies so as to eliminate the oscillation --
> >i.e., you can view the incommensurable policies as a case of
> >misconfiguration.
>
> I'm not sure that incommensurable policies will *always* be the result
> of misconfiguration.  That is, I can't rule out a scenario where
>
>    1) everyone involved is doing something locally sane,
>    2) the interaction of the local policies result in protocol oscillation
>    3) no one is willing to budge (change their policies). Every AS
>       involved is convinced that the others are at fault.
>
> Cheers,
> Tim
>

Date: Thu, 01 Mar 2001 16:20:26 -0500
From: "Sirish Bajpai" <[email-masked]>
Organization: Alcatel CID
To: [email-masked]
Subject: BGP routes vs. MBGP routes
Sender: [email-masked]

Hi All,

I have a few questions on co-existence of conventional BGP-4
and MBGP :

1) How should the traditional BGP routes be treated by a MBGP
router :
  a) For unicast only OR
  b) For unicast & multicast both.

Specifically can PIM use conventioanl BGP routes for its RPF checks.

(a) side of the arguement is that BGP-4 traditionally caters to unicast
traffic only. Multicast routes are carried in MP extensions with SAFI
2 or 3. This case renders SAFI 1 (NLRI for unicast) somewhat redundant.

(b) side argues that conventional BGP routes are "general purpose"
routes and thus can be used by anything. If a peer wants a route to be
used for unicast only, then it should send it in MP attribute with SAFI
1. Same for multicast routes with SAFI 2. Again this case renders SAFI 3
(NLRI for unicast & multicast) redundant.

Are there any specified guidelines favoring (a) or (b) above.

2) If (a) is true, does it mean I have to run all conventional BGP
updates through both unicast and multicast policies.

3) What does Cisco do. I read somewhere that they do (a) above.

My thanks and appericiation for any inputs on this.

Sirish Bajpai
Alcatel USA