Subject: RE: [Isis-wg] [IS-IS WG] Subsecond Timers

From: Jeff Parker <[email-masked]>
Date: Tue, 19 Dec 2000 14:15:42 -0500

My understanding of the two proposals has them
in strict opposition: the convergence draft 
tries to minimize the impact of a crash by 
reducing the time needed to route around any 
problem, and the Restart proposal tries to 
work fast enough to avoid getting caught.

Of course there are other uses for this
functionality, but if you know you want to
bring one side down for an upgrade, you can 
simply configure the box to start sending 
hellos with a longer Holding Timer.  
In IS-IS each side gets to dictate their own 
criteria.  I think that if you try this with 
OSPF, the other side will break the association.  

My point on the converge draft was simply this:
there is only one aspect of the draft that
describes on-the-wire behavior, and that would
have to do with the meaning of the Holding Time.
One side could send "1" if we trusted both sides to
be able to respond to hellos in a timely fashion,
but we cannot go below that with today's draft.
If the authors think 1 second is fast enough, we
could simply change the granularity of the resend
in the MIB, and keep the protocol behavior unchanged.  

- jeff parker

> OSPF restart (which I guess can be extended for ISIS) and 
> fast convergence seems to have conflicting goal (by way of
> affecting routerDeadInterval, one desiring longer other
> desiring shorter), don't you think?
> 
> This needs to be tackled in one or the other RFC too.
> 
> -----Original Message-----
> From: Jeff Parker [mailto:[email-masked]]
> Sent: Monday, December 18, 2000 8:38 PM
> To: [email-masked]
> Subject: [Isis-wg] [IS-IS WG] Subsecond Timers
> 
> 
> The "Sub Second convergence" draft
> 
> 	draft-alaettinoglu-ISIS-convergence-00.ps
> 
> has a number of useful suggestions about
> implementation.  While most of these ideas
> can be implemented today, the one change that
> would be needed to include all recommendations
> would be a new way to encode the hold timer in 
> hello messages.  Do the authors plan to put that 
> forward in a later version of the document?  
> 
> - jeff parker
> _______________________________________________
> Isis-wg mailing list  -  [email-masked]
> http://external.juniper.net/mailman/listinfo/isis-wg
> 
_______________________________________________
Isis-wg mailing list  -  [email-masked]
http://external.juniper.net/mailman/listinfo/isis-wg

---

From: Tony Przygienda <[email-masked]>
Date: Tue, 19 Dec 2000 20:57:46 +0100 (MET)

> in the MIB, and keep the protocol behavior unchanged.  

ISIS restart draft doesn't even exist yet and the 
fast convergence draft is _not_ a workgroup item 
and had generated some contingency  in terms of 
viability, 
therefore I would prefer not to see work going on based on 
extrapolation of things to come but rather have the
group focused working on items within our scope.
That does not mean thta I discourage the discussions
happening, but I'd prefer not to see changes 
creeping into existing drafts before the facts.

	-- tony

---

From: Cengiz Alaettinoglu <[email-masked]>
Date: Fri, 22 Dec 2000 10:12:00 -0800 (PST)

> creeping into existing drafts before the facts.

Tony,

A large group of people who spoke at the microphone expressed the
need/interest for this. The topic is of increasing interest.

It's difficult to get credible experience with the real effects of
using sub-second timers if they can't be specified.  It is all right
for some vendors not to be able to implement this today because they
lack the horsepower (which becomes more and more available
everyday). But, some vendors who want to differentiate their products
will want to implement this sooner. And everyone will get there
eventually. It is better to have a standard way of specifying
this. (We are not suggesting everyone to switch using subsecond timers
immediately nor instead of a level-2 notification scheme. But the WG
should not stop people from being able to do so).

The subsecond convergence is covered by the WG charter. Hence, this
work can be a WG item if the chairs/WG agree.  I dont mind doing the
work with other volunteers to write what needs to be added to the wire
protocol into a short draft.

Happy holidays,

Cengiz

-- 
Cengiz Alaettinoglu           Packet Design Inc.

---

From: Tony Przygienda <[email-masked]>
Date: Fri, 22 Dec 2000 22:23:56 +0100 (MET)

> need/interest for this. The topic is of increasing interest.

whether the discussion on the mike expressed need/interest 
or only mild interest/irritation (Vijay) is bound to interpretation 
by parties tinted by the agenda
they pursuit so I don't buy this argument. There was no
motion made and consensus built and implying any such thing 
on the list here should be done only carefully. 

> everyday). 

Lack of horsepower is not the problem with sub-second hellos in terms
of _real_ implementation of such protocols,
this is just something 
you extrapolated from your black-box observatioons. There were people 
(including me) doing that sub-second stuff on different IGPs
in proprietary fashion around for quite a long time and 
the effects were to put it mildly "surprising". Otherwise, 
believe me, it would have been widely deployed already. 

> But, some vendors who want to differentiate their products
> The subsecond convergence is covered by the WG charter. 

There is a lot of things that can be interpreted as "being in the charter"
but won't make sense.  Are you making a formal motion here to make the 
"subsecond-timers" being a workgroup item ?  And if so, it would help
your cause immensly to have a couple of ISPs deploying the protocol 
today on a large scale saying "yes, go for it" on this list and 
moreover, clearly stating what is the problem this sub-second 
hellos they hope will solve for them ?  And then, of course 
workgroup items need a draft so we know what we talk about in 
terms of bits & bytes first. 

> Hence, this

IMHO, that only makes sense if you remain completely backwards-compatible with the
existing implementations. If you intend to change the hello interval fields or
their semantics
in the existing hello-packets than I would say, you rather sit on the sidelines
until a new version of IS-IS happens that will not be compatible with the 
deployed one or convince some preople to do it under the wraps as proprietary
thing or move the work into IRTF 
since I would be most reluctant to open the "new ISIS-version" can of worms
right now.

thta's all modulo other Tony's opinion of course since I didn't have any
chance to syn up with him on that topic yet ...

	-- tony

---

From: Vijay Gill <[email-masked]>
Date: Fri, 22 Dec 2000 17:46:10 -0500

> need/interest for this. The topic is of increasing interest.

Having worked with very large scale networks, the trendline based on
emprical experience, network meltdowns, and implementation failures has
always been to turn up the timers such as hellos and the hello-mults.

Emperical observations from working on ISP type networks indicate that
setting the isis hello-multiplier to 12 or thereabouts seem to result in
more stable networks.

> using sub-second timers if they can't be specified.  It is all right

But first, instead of setting the multipliers and timers to milliseconds,
getting the large isps to set them to 3 down from 12 might be a good
interim step. Large networks prefer stability before speed of convergence
in general, and a reduction in a factor of 4 should give us good feedback.

> lack the horsepower (which becomes more and more available everyday).
> But, some vendors who want to differentiate their products will want
> to implement this sooner. And everyone will get there eventually. It

Considering that vendors can barely implement the spec as it stands. The
number of people I'd trust to implement isis, (outside of Dr. Li, John
Moy, Dr. Przygienda, henk, dkatz and bcole) I can count on one hand), I'll
not be holding my breath anytime soon that these vendors will actually see
use in operational networks of any size.

/vijay

---

From: Henk Smit <[email-masked]>
Date: Sat, 23 Dec 2000 01:35:02 +0100 (MET)

> setting the isis hello-multiplier to 12 or thereabouts seem to result in

  Most of this "empirical experience" is based on very old networks
 built out of routers with 20 MHz processors (cisco 7000s come to mind).
 And in those days the queuing strategies on linecards, and between
 linecards and control planes was all less elaborate.

  FYI, when I worked at cisco, I have worked with customers that used
 holdtimes of only several seconds, but those were not ISPs.

> in general, and a reduction in a factor of 4 should give us good feedback.

  Note that the lowest possible value of the ISIS holdtimer is 1 second.
 With the current protocol one could let a router send, say, 5 hellos in
 1 second. That would be a considerable improvement over today's networks.

 (For those that are interested, over a year ago I implemented a feature
  in cisco IOS that would allow you to configure a holdtime of just one
  second. This feature is available in 12.1 for sure. Ask you cisco
  support about the details. Maybe Stefano has ported it to 12.0S).

  I agree it would be very interesting if some networks (ISPs or
 other networks) would get more familiar with holdtimes of 1 second
 before we define a new TLV. On the other hand, issuing an experimental
 TLV would not hurt much.

> use in operational networks of any size.

  Thank you very much. But you are exaggerating. I don't think low
 holdtimers are the tricky thing to implement fast convergence. The
 tricky part is to react quickly to events in your network, without
 ever risking a meltdown. There is more to it than just faster hellos.

Tony Prz wrote:

> IMHO, that only makes sense if you remain completely
> backwards-compatible with the existing implementations. If you intend
> to change the hello interval fields or their semantics in the existing
> hello-packets than I would say, you rather sit on the sidelines until
> a new version of IS-IS happens that will not be compatible with the
> deployed one or convince some preople to do it under the wraps as
> proprietary thing or move the work into IRTF since I would be most
> reluctant to open the "new ISIS-version" can of worms right now.

  Tony, maybe you should do a little technical thinking before we
 start the politics. ;-) Doing sub-second holdtimers in a backward
 compatible manner is *very* simple. All we need is a new TLV to be
 included in the IIHs.

  Don't forget this is not OSPF. You configure the holdtimer on the
 router that is sending the IIHs. Suppose you configure your router
 to send an IIH every 20 ms, and you want the holdtime to be 100 ms.
 So what your router must do is:
 1) send hello every 20 ms
 2) set the old-fashioned advertised holdtimer to 1 second
 3) set the advertised holdtimer in the new TLV to 100 milliseconds.

  If the neighbor supports the new TLV, it will see both the 1 second
 old-fashioned holdtimer and the new-fashioned 100 millisecond holdtimer.
 Of course it will prefer the new holdtimer. So after the neighbor missed
 5 hellos from your router, it will declare the adjacency down. Exactly
 what your wanted.
  If the neighbor does not support the new TLV, it will use a holdtime
 of 1 second. So only after it missed 50 hellos it will declare the
 adjecency down. Not very quick, but still it would detect the down
 event. And the simplicity is that there is no negotiation involved
 at all. You include the TLV or you don't. You understand it when
 you receive it, or you don't. The protocol does not break.

    Hope this helps,

          Henk.

---

From: Tony Przygienda <[email-masked]>
Date: Sat, 23 Dec 2000 01:28:34 +0100 (MET)

>  compatible manner is *very* simple. All we need is a new TLV to be

Read once more through my mail: I did _not_ say that it could _not_ be done ;-)
Thanks for your proposal, I assume you'll go ahead and help with the draft
then ?

	-- tony

---

From: Henk Smit <[email-masked]>
Date: Sat, 23 Dec 2000 01:48:17 +0100 (MET)

> Thanks for your proposal, I assume you'll go ahead and help with the draft

  I'd be happy to leave that for the people who are gonna implement this.
 At the moment I am spending my time doing other things than IS-IS.
 It is not gonna be more than 1 or 2 pages.
 And I think I have already seen people offering to write a draft.

       Henk.

---

From: Vijay Gill <[email-masked]>
Date: Sat, 23 Dec 2000 01:26:46 -0500

>  linecards and control planes was all less elaborate.

I am not going to sit and explain to my management why we made the front
page of the WSJ. If I do have to do it, I better have a very good story,
and given our past experiences, I'm going to err on the side of caution,
till at least someone else of large size has done this and I trust the
people running that network.

>  ever risking a meltdown. There is more to it than just faster hellos.

Given the quality of implementations, I think we will just have to agree
to disagree ;) (this isn't a slam on any implementation, it is just that
operational experience has shown that there are some very subtle and
strange interactions in large networks that can cause meltdowns due to
combinations of bugs that individually are just annoying.) We are paranoid
and crazy for good reason ;)

/vijay

---

From: Tony Li <[email-masked]>
Date: Fri, 22 Dec 2000 22:31:55 -0800 (PST)

| thta's all modulo other Tony's opinion of course since I didn't have any
 | chance to syn up with him on that topic yet ...

I'll take that as an invitation to opine:

<WG co-chair hat>
The proposal that has been put forth is a reasonable WG work item.  There
is clearly an audience and sufficient technical opinion.  Therefore, it
seems appropriate for the WG to entertain the work.
</WG co-chair hat>

<geek hat>
However, I can't believe that this is the best way of solving this problem.
The issue that I have with this approach is that it implies that the
responsiveness of our systems is now going to drop to sub-second levels.
While I can believe that there are real-time systems that can be built with
the required responsiveness, not all of them were originally designed this
way and the resulting modifications might be more than challenging.
Comparatively, the protocol changes are trivial.  

Are there other things that we can do?  Certainly.  Make better use of
SONET.  Make more use of Ethernet link failures.  

Somehow, just making everything run faster seems like the brute force
solution and puts stability at risk.
</geek hat>

Tony

---

From: Mike Truskowski <[email-masked]>
Date: Sat, 23 Dec 2000 8:06:00 PST

I agree... while you can make the protocols run faster
you put a burden on the cpu and bandwidth which 
generates and transport the intelligence...

I do not understand the referal to better use of sonet
and/or ethernet link information...  I understand the
local significance but not the remote side....  Maybe
I took this wrong.

Mike

---

From: "Chris Whyte" <[email-masked]>
Date: Tue, 26 Dec 2000 15:19:55 -0800

> people running that network.

And one could argue that some ISPs are making, or will make, the front page
of the WSJ because of the additional complexity they're putting in their
network just to get around the fact that we can't do sub-second, or just
plain faster, igp restoration.

My experience says there's a lot of interest in achieving this goal without
adding another protocol, technology or whatever to the equation. I would
really love to see the WG look long and hard at changing the protocol, if
necessary, in order to achieve it.

I guess I'm just not convinced that any one specific group has looked at the
routing  protocols as much as they've looked at other (newer) protocols or
technologies which attempt to achieve this goal. I think it would be nice to
be able to possibly optimize something that people already know, love and
understand than to start somewhat fresh with a new approach that will just
add more complexity since the routing protocol certainly isn't going away.

> and crazy for good reason ;)

But I'm really curious, how much experimentation has been done in this area
in the last year or so? I really don't know but I suspect it's been minimal.
I understand the paranoia but I also believe that some, or much, of this
paranoia can be attributed to historical issues that don't necessarily apply
to more recent router implementations. And I'd hate to see it continue to
gate any possible progress in this area since we have much better separation
between data and control planes, faster CPUs and, overall, a much better
understanding of some of the fundamental implementation and network design
issues that can be done in order to avoid meltdown these days.

Thanks,

Chris

---

From: Randy Bush <[email-masked]>
Date: Tue, 26 Dec 2000 15:51:10 -0800

> plain faster, igp restoration.

yes, but it would be hard to support.  e.g. there is a reason for danny's
recent draft-mcpherson-isis-transient-00.txt.

but we foolish operators do appreciate your concern for us.

> necessary, in order to achieve it.

once again, do you really think that this should be a high priority effort
given the other problems we face?

randy

---

From: Dave Katz <[email-masked]>
Date: Tue, 26 Dec 2000 23:25:27 -0800 (PST)

Simple issue.  This is not really a protocol issue, but a fundamental
implementation issue.  At least at this time, the dominant
implementations are done in run-to-completion schedulers, and as such
are a train wreck in the making.

The protocol can do one second hold times now, and a trivial extension
could choose something even shorter, but until we all reimplement our
routers with true real-time o/s's this will never work (for large
values of "work," as in "you bet your business on it.")

Then if some of us fix it, we will get interesting partial collapses
instead (I've alread seen this multiple times involving another familiar
link-state IGP.)

My mantra for years has been, "the implementations have a long way to go
before we hit the fundamental limitations of the protocols" and I still
believe this to be true in multiple dimensions.

   Reply-To: "Chris Whyte" <[email-masked]>
   From: "Chris Whyte" <[email-masked]>
   Cc: "Cengiz Alaettinoglu" <[email-masked]>,
	   "Tony Przygienda" <[email-masked]>, "Jeff Parker" <[email-masked]>,
	   <[email-masked]>, <[email-masked]>
   References: <[email-masked]>
   Organization: cisco Systems, Inc.
   MIME-Version: 1.0
   Content-Type: text/plain;
	   charset="iso-8859-1"
   Content-Transfer-Encoding: 7bit
   X-Priority: 3
   X-MSMail-Priority: Normal
   X-Mailer: Microsoft Outlook Express 5.00.2919.6600
   X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6600
   Sender: [email-masked]
   Errors-To: [email-masked]
   X-BeenThere: [email-masked]
   X-Mailman-Version: 2.0rc1
   Precedence: bulk
   List-Help: <mailto:[email-masked]?subject=help>
   List-Post: <mailto:[email-masked]>
   List-Subscribe: <http://external.juniper.net/mailman/listinfo/isis-wg>,
	   <mailto:[email-masked]?subject=subscribe>
   List-Id: IETF IS-IS working group <isis-wg.external.juniper.net>
   List-Unsubscribe: <http://external.juniper.net/mailman/listinfo/isis-wg>,
	   <mailto:[email-masked]?subject=unsubscribe>
   List-Archive: <http://external.juniper.net/pipermail/isis-wg/>
   Date: Tue, 26 Dec 2000 15:19:55 -0800

   Thanks,

   Chris

---

From: James Carlson <[email-masked]>
Date: Wed, 27 Dec 2000 12:03:38 -0500 (EST)

> local significance but not the remote side....  Maybe

For SONET, you could reasonably send RDI-P upstream if something
untoward happens to your local routing process or your ability to
forward received packets.  On detecting a positive loss of service by
reception of RDI-P at the other end, you could remove the link from
your database and start negotiation over from scratch when the defect
disappears.

On point-to-point Ethernet, you could probably do something similar by
taking the link out of sync intentionally for the duration of your
local outage.  Assuming the peer has been properly implemented to
monitor link status, this should give rapid and accurate indication of
remote failure.

For multi-access Ethernet, there's obviously a problem.  However,
since there are few 10BASE5 systems left, the worst problem can be
ignored.  For the remainder, it'd be helpful to get the bridge
("switch") vendors to support notification of link loss in a manner
that'd work well with routing protocol implementations.

I can see how detecting failure at the routing protocol level might
not be avoidable in some situations, but I think fast-hello is a
band-aid at best, and a likely source of instability and scalability
trouble.  The detection and propagation of failure information isn't
just a routing protocol issue; it's a system issue.

I don't think that this should be carried to the other extreme either
(lengthening the hello interval).  The hello exchange provides a
reliable upper bound on the delay of upper level failure detection in
case the other failure notification mechanisms themselves fail.  It's
important, but it shouldn't be the only means of detecting problems.

(For what it's worth, similar issues occur with many other protocols,
such as RIP.  Most of these protocols have a "goodbye" mechanism [such
as sending out infinite hop count routes] that can be used to speed up
failure detection.)

-- 
James Carlson, Internet Engineering       <[email-masked]>
SUN Microsystems / 1 Network Drive         71.234W   Vox +1 781 442 2084
MS UBUR02-212 / Burlington MA 01803-2757   42.497N   Fax +1 781 442 1677
Second Edition now available - http://people.ne.mediaone.net/carlson/ppp
_______________________________________________
Isis-wg mailing list  -  [email-masked]
http://external.juniper.net/mailman/listinfo/isis-wg

---

From: Dave Katz <[email-masked]>
Date: Wed, 27 Dec 2000 10:47:00 -0800 (PST)

what do you think of van's point that waiting for three hello timeouts,
   as opposed to noticing the physical (or framing) link going down, may
   be contributing the largest part of convergence delay?  and, if you
   believe it, how might making such a change affect stability?

   randy

Well, if there is physical evidence that the link has gone down, every
implementation that I know of will respond to it.  The hello timeout
is definitely the slowest part of the *IGP* convergence (assuming that
there is no low-level signalling to glean), but in the kinds of
networks I'm familiar with (big, expensive, and full of ~100K BGP
routes) the problem is much broader.  If you have to change the next
hops on tens of thousands of routes, this can take a sizable chunk of
time (no, I don't have any hard numbers).

The IGP implementations that I'm familiar with (one of which I haven't
seen the innards of in almost four years, so I can't vouch for its
current state) are pretty heavily biased toward stability at the
expense of convergence, due to "interesting" field experience.  We
haven't seen an IGP meltdown (well, at least an ISIS meltdown) in a
major network in quite awhile, knock on wood.  Given this, perhaps its
not a bad time to open up the covers again and think about convergence
improvements *very* carefully (I really don't want to see a New York
Times article with the headline, "MegaNet down for 19 hours;
programmer sought") but part of this careful analysis has to be
looking at what the *real* issues are, and where the target-rich
environments lie.

For example, how often do links fail, in real life, in such a way that
nobody tells the protocol and it has to time out?  (And how many of
those failures-to-communicate are due to bugs?)  What are the factors
that contribute to global convergence (as opposed to a localized
picture of IGP convergence?)  Like all cases of premature
optimization, it's at best a waste of time (and at worst
destabilizing) to start playing with things without really knowing
what's going on.

All of this can be done outside the protocol spec, which is why I
don't feel that there's any real pressure in tinkering with the
protocol at this time.

If it turns out that the holdtime delay is truly important, and
reducing it by an order of magnitude will truly reap more than
negligible benefits (I'm skeptical) then we've got some interesting
software architectural work ahead of us to pull this off in a way that
won't endanger the stability of the infrastructure.  It's a *lot* more
work than just twisting timers.  (For fun, go set the hello interval
to 1 and the hold time to 3 and then take a major transient, say, in
BGP, and then see how things hold up...)

--Dave

---

From: "Chris Whyte" <[email-masked]>
Date: Wed, 27 Dec 2000 11:32:50 -0800

Ok. I have two questions/comments then...

1. So are you saying that the suggestions made in
draft-alaettinoglu-isis-convergence-00.txt, which do involve a change to
is-is, aren't worth pursuing because of the reason(s) below?

2. Forgive my ignorance but it's not clear to me why we shouldn't continue
to pursue changes to spec (eg shorter hold times), which ISPs appear to be
interested in, given that there are at least some vendors working on true
real-time o/s router implementations.

Don't we always want to be in the situation where "the implementations have
a long way to go before we hit the fundamental limitations of the
protocols"?

Thanks,

Chris

>    References:
<[email-masked]>
>    List-Unsubscribe:
<http://external.juniper.net/mailman/listinfo/isis-wg>,

---

From: Tony Przygienda <[email-masked]>
Date: Wed, 27 Dec 2000 11:57:50 -0800

> are a train wreck in the making.

yes and no, if you go away from the run-to-completion scheduler and _even_
if you implement the whole thing on a real-time scheduler, let's say understanding
priority and relative & absolute slip, you face another monster, namely locking of
which a protocol like ISIS will have plenty, even in the hello processing/generation ;-)

It is a perversity well-known in the database world (and IGPs are nothing else but
distributed, loosely synchronized, real-time databases) that once you go into
pre-emption, not your procedural/task design determines runtime behavior
but primarily your locking disciplines/granularity/frequence/priority inheritance.
Therefore  you design high-performance databases these days starting from
the locking disciplines & granularity and  not from task/operating system side. At least
last time I checked (and I admit it's a while) ;-)

So, my strong feeling is that once you go into this realm you end up being
very disappointed with what you can achieve by just adding preemptive, real-time tasks
unless you start from designing your database specifically for the performance
profile you want to achieve. Nobody tackled that one yet as far as I know in the
IP world.

Again, to prevent bashing, I'm _not_ saying it can_not_ be done, I'm just pointing out
another piece of the dragon you get out the cave when you grab the beatiful
end of the tail sticking out  ;-)

> believe this to be true in multiple dimensions.

yes ;-) ...

    thanks

    --- tony

---

From: Tony Przygienda <[email-masked]>
Date: Wed, 27 Dec 2000 12:00:20 -0800

yes and no, if you go away from the run-to-completion scheduler and _even_
if you implement the whole thing on a real-time scheduler, let's say understanding
priority and relative & absolute slip, you face another monster, namely locking ;-)

It is a perversity well-known in the database world (and IGPs are nothing else
but distributed, loosely synchronized, real-time databases) that once you go into
pre-emption, not your procedural/task design determines runtime behavior
but primarily your locking disciplines/granularity/frequence/priority inheritance.
Therefore  you design high-performance databases these days starting from
the locking disciplines & granularity and  not from task/operating system side. At least
last time I checked (and I admit it's a while) ;-)

So, my strong feeling is that once you go into this realm you end up being
very disappointed with what you can achieve unless you start to design your
protocol starting from the locking behavior and as far I know nobody tried that
in the IP world yet.

yes ;-) ...

    --- tony

---

From: Randy Bush <[email-masked]>
Date: Wed, 27 Dec 2000 06:59:12 -0800

dave,

randy

---

From: Rohit Dube <[email-masked]>
Date: Wed, 27 Dec 2000 16:24:15 -0500

On Wed, 27 Dec 2000 12:00:20 -0800 Tony Przygienda writes:
=>
=>Dave Katz wrote:
=>
=>> Simple issue.  This is not really a protocol issue, but a fundamental
=>> implementation issue.  At least at this time, the dominant
=>> implementations are done in run-to-completion schedulers, and as such
=>> are a train wreck in the making.
=>
=>yes and no, if you go away from the run-to-completion scheduler and _even_
=>if you implement the whole thing on a real-time scheduler, let's say understa
>nding
=>priority and relative & absolute slip, you face another monster, namely locki
>ng ;-)
=>
=>It is a perversity well-known in the database world (and IGPs are nothing els
>e
=>but distributed, loosely synchronized, real-time databases) that once you go 
>into
=>pre-emption, not your procedural/task design determines runtime behavior
=>but primarily your locking disciplines/granularity/frequence/priority inherit
>ance.
=>Therefore  you design high-performance databases these days starting from
=>the locking disciplines & granularity and  not from task/operating system sid
>e. At least
=>last time I checked (and I admit it's a while) ;-)

And don't forget you access methods - concurrent B+-tree access
and the like. (And those who use lower fan-out balanced trees may want to
check out some work by Sabine Hanke et al - 
http://www.imada.ou.dk/~kslarsen/RelBal/)

From my own experience doing similar work both with Databases and 
IGPs/Route-table managers, optimizing performance with locking can get
fairly hairy and depends not just on granualrity and locking disciplines
but also on workloads (number of readers, writers) and lock-timeout
algorithms in addition to the points that you mention.

FWIW,
--rohit.

---

From: Dave Katz <[email-masked]>
Date: Wed, 27 Dec 2000 15:29:49 -0800 (PST)

I haven't had time to read this in detail, though I did have a private
discussion with Van & Co. about the contents, and a brief reading
seemed to line up with the discussion.

There are certainly things in these implementations that could be done
better, and some of them would be quite easy (though others may not
be).  The reservation I have with this work is that it does not
characterize the potential gains in real world environments, and we
should all be loathe to open the covers on our protocol
implementations without understanding what the real gains will be.  I
just don't like making front page news, and as a matter of principle I
don't like to do something this risky without a clear idea of why.
The stakes are just too high.

It's worth noting that, no matter how hard they hammered on the
implementations, they couldn't make them fall over.  There is a reason
for this.

I'm not opposed to the suggestions made, either in implementation
issues or in protocol extensions.  It's a very useful thing for
someone to have taken a look at this stuff and to have made a reasoned
and intelligent analysis.  My only point is that, from a pragmatic
standpoint with deployed products and operational networks, we have
lots of room for improvement in other areas before doing invasive
surgery on the IGPs.  Convergence is a much broader issue, with many
interesting (and unexpected) interactions, than simply finishing an
SPF quickly.

I have no objection to pursuing protocol extensions, particularly
backward compatible ones, if people want to do this.  I don't think,
however, that ISPs are interested in protocol changes; rather, they
are interested in improving overall convergence times while
maintaining the stability of their networks.  Those that are convinced
that speeding up hello times or SPF implementations will make their
networks much better have been sold a bill of goods--it's just not
that simple.

A vendor that can field a very large network that will remain stable
while maintaining one second hold times will have done very impressive
work deserving of high profit margins, without having touched the
protocol.  It should be interesting to see.  Having done that, said
vendor should be well positioned to explore the subsecond realm.

Absolutely not.  Where we want to be is in the position where the networks
have a long way to go before they hit the fundamental limitations of the
protocol implementations.  Only then will the fundamental limitations of
the protocol prove really interesting.
_______________________________________________
Isis-wg mailing list  -  [email-masked]
http://external.juniper.net/mailman/listinfo/isis-wg

---

From: Dave Katz <[email-masked]>
Date: Wed, 27 Dec 2000 15:37:17 -0800 (PST)

X-JNPR-Received-From: outside

No argument here.  I would, in fact, like all of my competitors to port all
of their code to a preemptive o/s immediately.  ;-)  I've had to talk
people out of this idea many times over the years--you trade an obvious
problem for a whole bunch of really subtle ones.

Even naive protocol implementations are difficult.  Scalable, stable ones
are very difficult (look at how many are out there.)  Scalable, stable,
fast ones are even harder still.

--Dave

---

From: Randy Bush <[email-masked]>
Date: Wed, 27 Dec 2000 15:54:14 -0800

> implementation that I know of will respond to it.

i would have thought so.  but van seems to think otherwise, and claims
measurement to boot.

> seen the innards of in almost four years, so I can't vouch for its current
> state) are pretty heavily biased toward stability at the expense of
> convergence, due to "interesting" field experience.

i assure you that some of us deeply appreciate this spoilsport approach,
boring though others may think it.

randy

---

From: Randy Bush <[email-masked]>
Date: Wed, 27 Dec 2000 16:24:26 -0800

> however, that ISPs are interested in protocol changes

wrongo!  to paraphrase mo, i want lots of them made available.  so my
competitors can try them.

randy

---

From: Tony Przygienda <[email-masked]>
Date: Wed, 27 Dec 2000 15:36:35 -0800

> competitors can try them.

carefull Randy, some people on this list MAY take you literally ;-)

    -- tony

---

From: Randy Bush <[email-masked]>
Date: Wed, 27 Dec 2000 17:51:46 -0800

> carefull Randy, some people on this list MAY take you literally ;-)

but we just *love* to read about our competitors's meltdowns in the wsj.
after all, what's the fun in running a boring network when you can look
oh so modern and clever in front of your drinking friends at nanog?

randy

---

From: "Chris Whyte" <[email-masked]>
Date: Wed, 27 Dec 2000 21:37:55 -0800

> oh so modern and clever in front of your drinking friends at nanog?

Well hell, I love boring and simplistic but maybe someone can explain why
changes to the protocol to achieve something like faster restoration is more
operationally complex than a large mesh of TE and bypass tunnels (ie FRR),
which also requires changes to the protocol. And certainly running a network
that uses the latter approach is no walk in the park.

I realize I'm mixing a bit of operational complexity concern with protocol
implementation issues but is it simply that the risk of meltdown is much
greater? And if so, are there not things we can do like using an spf backoff
algorithm in the face of instability to prevent meltdown (very similar to
Henk's old work)?

And if this is not interesting to others, or is beyond the scope of the
mailing list, please respond to me offlist. I would be more than fascinated
to hear people's opinions/answers.

Thanks,

Chris

---

From: Tony Li <[email-masked]>
Date: Thu, 28 Dec 2000 00:05:36 -0800 (PST)

Chris,

 | Well hell, I love boring and simplistic but maybe someone can explain why
 | changes to the protocol to achieve something like faster restoration is more
 | operationally complex than a large mesh of TE and bypass tunnels (ie FRR),
 | which also requires changes to the protocol. And certainly running a network
 | that uses the latter approach is no walk in the park.

One last time: it's not the changes to the protocol.  It's the changes to
the rest of the system.  For the routing protocol to detect a loss of
adjacency at the rates that are proposed, the routing protocol process must
run at much higher rates to generate the hello packets, receive them, and
time out failed adjacencies.

Given the current run-to-completion operating system schedulers that we all
seem to use, you basically have two choices: either try to run to
completion with MUCH tighter timers throughout the ENTIRE operating system
or convert to a pre-emptive scheduling mechanism.  Both are fraught with
risk.

Whereas if you take a look at a TE approach, you find that the underlying
protocol has NOT changed in any substantive way.  It's being (ab)used as a
mechanism for transporting unnatural information, but the dynamics of the
protocol itself has not changed one whit.  It's absolutely true that
there's more complexity in the TE code.  But when (not if) it breaks, all
it does is to cause localized failures in the TE subsystem.  Basic unicast
routing should be very much unaffected.

 | I realize I'm mixing a bit of operational complexity concern with protocol
 | implementation issues but is it simply that the risk of meltdown is much
 | greater? And if so, are there not things we can do like using an spf backoff
 | algorithm in the face of instability to prevent meltdown (very similar to
 | Henk's old work)?

Again, it's not the speed of SPFs that matter.  It's that you now have to
be able to respond to a timer expiration within 10ms or so.  That's about 
three orders of magnitude faster than today.

Tony

---

From: "Chris Whyte" <[email-masked]>
Date: Fri, 29 Dec 2000 15:13:28 -0800

> time out failed adjacencies.

Yes, I understood that. Sorry, I'm quite stubburn. I sometimes ask people to
repeat things more than once. :-)

> or convert to a pre-emptive scheduling mechanism.  Both are fraught with

Ok. I think that one has sunk in.

> routing should be very much unaffected.

So everything above clearly helps me understand the protocol differences and
how they impact the (rest of the) system but it what about the operational
differences? My experience says there are also lots of big problems in
networks due to operational complexity. And, as you admitted, since there's
more complexity in the TE code then this clearly should increase operational
challenges.

Yes, I know... this is something that I'm sure you all would prefer to
discuss somewhere else.

My only goal (other than learning from this thread) is that I would like to
see if there are any changes to the protocol that can be made in order to
achieve convergence in the neighborhood of hundreds of milliseconds (I'm not
necessarily stuck on some number like 50ms) yet not substantially increase
operational complexity (or at least what I consider to be operational
complexity :-)).

And as someone else mentioned to me recently; convergence goals of
milliseconds really requires better abstraction boundaries along with some
richer metrics (eg 1-prop delay + 2-some link quality indicator to weight
1). I believe the former requires a slight fundamental change in how we
design networks today (maybe not) and the latter has challenges in how we
can make a multi-metric truly useful.

Anyway, you get my point.

> three orders of magnitude faster than today.

I never said 10ms. I'd be very happy with 3 x 200ms (like I said above). :-)

But hey, if you think I'm just wasting your time (and my time), I'll shut
up.

Thanks,

Chris

---

From: RJ Atkinson <[email-masked]>
Date: Sun, 31 Dec 2000 12:27:34 -0500

At 18:19 26/12/00, Chris Whyte wrote:

>And one could argue that some ISPs are making, or will make, 
>the front page of the WSJ because of the additional complexity 
>they're putting in their network just to get around the fact 
>that we can't do sub-second, or just plain faster, igp restoration.

        One could argue anything I suppose; the particular example
above would be a false argument as it happens.

>But I'm really curious, how much experimentation has been done 
>in this area in the last year or so? I really don't know 
>but I suspect it's been minimal.

        Hence the desire to do some *science* with some
*controlled experiments* on any proposal in this area
BEFORE the IETF *standardises* any changes to a protocol
that works.

        On this list, so far, folks are NOT saying "never change 
the protocol".  Folks ARE saying, please let the interested folks 
run some scientific private experiments, document the changes 
made, and their results, and let the proposal have some good
peer review -- BEFORE we *standardise* a change to the protocol.

        None of these items require ANY IETF action be taken *yet*.
Folks like Van have, for many years, done the science and run 
appropriate experiments *before* coming to an IETF WG to
propose changes to standards.

        If folks have documented results from some small scale
or large scale experiments, please post a pointer to the 
documentation on the change made, the experimental setup, 
and the results.

        If folks have small scale results that are positive and
want to try a large scale experiment, one approach would be to
describe the change and the experiment in an EXPERIMENTAL RFC.
Then, AFTER there is a high level of confidence in the proposal,
based on such experiments, one could move the potential spec
change into a non-experimental RFC.
        
Ran
[email-masked]

---

From: RJ Atkinson <[email-masked]>
Date: Sun, 31 Dec 2000 12:38:38 -0500

At 18:13 29/12/00, someone wrote:
>My only goal (other than learning from this thread) is that 
>I would like to see if there are any changes to the protocol 
>that can be made in order to achieve convergence in the 
>neighborhood of hundreds of milliseconds (I'm not necessarily 
>stuck on some number like 50ms) yet not substantially increase
>operational complexity (or at least what I consider to be 
>operational complexity :-)).

        It is fascinating to me that there is still interest in 
"changes to the protocol" after several folks have suggested that 
convergence time improvements by tuning one's implementation
(without a protocol change) are more likely to be a ripe
opportunity.

        My guess would be that whoever had good/valid/stable ideas 
about tuning without a protocol change and made them work
would sell a lot more product than one who wanted/needed 
to change the protocol.

Ran