Subject: Re: [Idr] Adoption of draft-uttaro-idr-bgp-persistence-01 as IDR WG document

From: <[email-masked]>
Date: Wed, 6 Jun 2012 09:18:07 +0000

Hi,

Support.

Regards,
Bruno

>From: John Scudder >Sent: Thursday, May 31, 2012 7:10 PM
>
>Folks,
>
>We have received a request from the authors to adopt draft-uttaro-idr-bgp-
>persistence-01 as an IDR WG document.  Please send your comments to the list.
>The deadline for comments is June 15, 2012 at noon EDT.
>
>Thanks,
>
>--John
>_______________________________________________
>Idr mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/idr

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Wed, 6 Jun 2012 10:30:31 +0000

+1

Jim Uttaro

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Wednesday, June 06, 2012 5:18 AM
To: John Scudder; [email-masked] List
Subject: Re: [Idr] Adoption of draft-uttaro-idr-bgp-persistence-01 as IDR WG document

Hi,

Support.

Regards,
Bruno

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 06 Jun 2012 12:06:36 -0400

The question for adopting this seems to be whether the folks who have to 
turn it on can tell when it is safe to do so, and when it isn't.
The L2VPN example in the document is an example of why it is worth 
considering.  The L3VPN example seems to show a dangerous use case.

Yours,
Joel M. Halpern

On 6/6/2012 5:18 AM, [email-masked] wrote:
> Hi,
>
> Support.
>
> Regards,
> Bruno
>
>> From: John Scudder>Sent: Thursday, May 31, 2012 7:10 PM
>>
>> Folks,
>>

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 06 Jun 2012 09:41:31 -0700

Hi Joel,

Great comment. In fact some of the co-authors of this document target to 
use it in also in the SAFI 1/1 or 2/1 even so the draft softly target a 
bit different application space:

    "This document addresses
    new services whose requirements for persistence diverge from the
    Internet routing point of view."

With this in mind how about either explicitly enumerating those 
AFI/SAFIs which can use such knob or specify those AFI/SAFIs which MUST 
NOT support it ?

Of course this is assuming we will go with accepting this as WG doc in 
the first place :)

Best regards,
R.

>> Ce message et ses pieces jointes peuvent contenir des informations
>> confidentielles ou privilegiees et ne doivent donc
>> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez
>> recu ce message par erreur, veuillez le signaler
>> a l'expediteur et le detruire ainsi que les pieces jointes. Les
>> messages electroniques etant susceptibles d'alteration,
>> This message and its attachments may contain confidential or
>> privileged information that may be protected by law;
>> As emails may be altered, France Telecom - Orange is not liable for
>> messages that have been modified, changed or falsified.
>> Thank you.
>>

---

From: Randy Bush <[email-masked]>
Date: Wed, 06 Jun 2012 10:04:09 -0700

> The question for adopting this seems to be whether the folks who have
> to turn it on can tell when it is safe to do so, and when it isn't.

seems to me that this is a discussion for *after* it is adopted by the
wg.  adoption is not a process of making it perfect.  it is a decision
of relevance.

randy

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Wed, 6 Jun 2012 17:12:55 +0000

Joel,

	I think it is a matter of the level "incorrectness" we are willing to accept in the event of a catastrophic failure.. In L3VPN there is certainly the possibility that the PE will miss a routing change that would change the VPNs path selection and subsequent topology.. That being said we would rather have some "incorrectness" instead of a catastrophic outage for customers L3VPN... 

There are other applications i.e using labeled BGP to provide NH reachability which are extremely static and basically never change that are excellent candidates.. 

In the draft L2 and L3 have been enumerated, I actually have a deliverable to write up the 3107 application ( Not done yet )... That being said I do not anticipate that we will characterize every application of BGP Persistence..

Jim Uttaro

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Joel M. Halpern
Sent: Wednesday, June 06, 2012 12:07 PM
To: [email-masked]
Cc: [email-masked] List
Subject: Re: [Idr] Adoption of draft-uttaro-idr-bgp-persistence-01 as IDR WG document

Yours,
Joel M. Halpern

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Wed, 6 Jun 2012 17:14:59 +0000

Robert 

	We can add addl use cases.. I have this on my to do list.. I believe the 3107 use case is high on the list, BGP signaled multicast may be another, BRAS etc... We could have a long list here and I do not want to be prescriptive so I will add the 3107 use case

Jim Uttaro

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Robert Raszuk
Sent: Wednesday, June 06, 2012 12:42 PM
To: Joel M. Halpern
Cc: [email-masked] List
Subject: Re: [Idr] Adoption of draft-uttaro-idr-bgp-persistence-01 as IDR WG document

Hi Joel,

    "This document addresses

Best regards,
R.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 06 Jun 2012 14:10:12 -0400

There do seem to be significant cases where this is useful.
What I am asking is that we highlight very clearly the risks, and try to 
be clearer about how to tell when it is a good choice.  This is very 
different from the classical IDR policy of just providing knobs, and 
leaving the rest up to the vendors to guess / provide to customers / get 
wrong.

Yours,
Joel

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 06 Jun 2012 16:49:26 -0700

Jim,

Adding more use cases is clearly good thing.

But I was in particular asking for a stronger text explicitly forbidding 
implementation to make IPv4 or IPv6 routes as persistent.

For 1/4, 1/128 or any other "application" SAFIs I think that at the end 
you may do what you like :)

Today Internet churn is already not low ... this draft if used 
incorrectly for Internet can cause additional re-advertisements based on 
local flaps (where GR you are keep comparing it with would not).

And just as a reference point on old NPE400 I peered with the AS290 
yesterday here in Tokyo interop I get 60%-70% CPU use all the time

CE_c72_Internet#sh proc cpu
CPU utilization for five seconds: 99%/23%; one minute: 67%; five 
minutes: 83%
....
  103    26649044    209857     126987 62.32% 41.94% 56.76%   0 BGP Router

So while any knob may look nice on the paper it does have an impact 
Internet wide and that's main reason I am a bit skeptical on adding more 
to SAFI 1/1 and 2/1.

In general I think we are really missing the indirection in signalling 
SAFI wide parameters in BGP. Perhaps if WG thinks this would be good 
idea we could propose a simple new draft for defining a new signalling 
SAFI for all other SAFI wide events.

That way in the event of use of persistent communities or shutdown on 
existing SAFIs single message could be send not re-advertisements of 
millions of routes. And that significance will only grow when you 
attempt to do path validation or other fancy processing inbound of each 
advertisement.

I know you can apply persistence to subset of prefixes however I doubt 
that this would be practical deployment case. Clearly for shutdown it 
does not make much sense to only signal subset of marked routes in a 
given SAFI that are going down soon.

Best regards,
R.

> Robert
>
> We can add addl use cases.. I have this on my to do list.. I believe
> the 3107 use case is high on the list, BGP signaled multicast may be
> another, BRAS etc... We could have a long list here and I do not want
> to be prescriptive so I will add the 3107 use case
>
> Jim Uttaro
>
> -----Original Message----- From: [email-masked]
> [mailto:[email-masked]] On Behalf Of Robert Raszuk Sent:
> Wednesday, June 06, 2012 12:42 PM To: Joel M. Halpern Cc:
> [email-masked] List Subject: Re: [Idr] Adoption of
> draft-uttaro-idr-bgp-persistence-01 as IDR WG document
>
> Hi Joel,
>
> Great comment. In fact some of the co-authors of this document target
> to use it in also in the SAFI 1/1 or 2/1 even so the draft softly
> target a bit different application space:
>
> "This document addresses new services whose requirements for
> persistence diverge from the Internet routing point of view."
>
> AFI/SAFIs which can use such knob or specify those AFI/SAFIs which
> MUST NOT support it ?
>
> Of course this is assuming we will go with accepting this as WG doc
> in the first place :)
>
> Best regards, R.
>
>
>> The question for adopting this seems to be whether the folks who
>> have to turn it on can tell when it is safe to do so, and when it
>> isn't. The L2VPN example in the document is an example of why it is
>> worth considering.  The L3VPN example seems to show a dangerous use
>> case.
>>
>> Yours, Joel M. Halpern
>>
>>> Regards, Bruno
>>>> We have received a request from the authors to adopt
>>>> draft-uttaro-idr-bgp- persistence-01 as an IDR WG document.
>>>> Please send your comments to the list. The deadline for
>>>> comments is June 15, 2012 at noon EDT.
>>>>
>>>> Thanks,
>>>>
>>>> --John _______________________________________________ Idr
>>>> mailing list [email-masked]
>>> confidentielles ou privilegiees et ne doivent donc pas etre
>>> diffuses, exploites ou copies sans autorisation. Si vous avez
>>> recu ce message par erreur, veuillez le signaler a l'expediteur
>>> et le detruire ainsi que les pieces jointes. Les messages
>>> electroniques etant susceptibles d'alteration, France Telecom -
>>> Orange decline toute responsabilite si ce message a ete altere,
>>> deforme ou falsifie. Merci.
>>>
>>> privileged information that may be protected by law; they should
>>> not be distributed, used or copied without authorisation. If you
>>> have received this email in error, please notify the sender and
>>> delete this message and its attachments. As emails may be
>>> altered, France Telecom - Orange is not liable for messages that
>>> have been modified, changed or falsified. Thank you.
>>>
>>> _______________________________________________ Idr mailing list
>>> [email-masked] https://www.ietf.org/mailman/listinfo/idr
>>>
>> _______________________________________________ Idr mailing list
>> [email-masked] https://www.ietf.org/mailman/listinfo/idr
>>
>>
>
>
> _______________________________________________ Idr mailing list
> [email-masked] https://www.ietf.org/mailman/listinfo/idr
>
>

---

From: <[email-masked]>
Date: Thu, 7 Jun 2012 08:46:09 +0200

Support

>draft-uttaro-idr-bgp-

________________________________________________________________________
_________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations
confidentielles ou privilegiees et ne doivent donc pas etre diffuses,
exploites ou copies sans autorisation. Si vous avez recu ce message par
erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les
pieces jointes. Les messages electroniques etant susceptibles
d'alteration, France Telecom - Orange decline toute responsabilite si ce
message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged
information that may be protected by law; they should not be
distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and
delete this message and its attachments.
As emails may be altered, France Telecom - Orange is not liable for
messages that have been modified, changed or falsified.
Thank you.

---

From: Pradosh Mohapatra <[email-masked]>
Date: Thu, 7 Jun 2012 00:25:12 -0700

Hi Robert,

> But I was in particular asking for a stronger text explicitly forbidding implementation to make IPv4 or IPv6 routes as persistent.

I prefer to have a section that articulates the risks and provides design guidelines, as per Joel's suggestion. That would be a better technical specification than making a table of AFI/SAFI applicability.

- Pradosh

---

From: Rob Shakir <[email-masked]>
Date: Thu, 7 Jun 2012 11:06:14 +0100

On 6 Jun 2012, at 19:10, Joel M. Halpern wrote:

> What I am asking is that we highlight very clearly the risks, and try to be clearer about how to tell when it is a good choice.  This is very different from the classical IDR policy of just providing knobs, and leaving the rest up to the vendors to guess / provide to customers / get wrong.

Hi Joel, All,

As per Jim's previous comment, it is a case of balancing of the level of incorrectness that is acceptable in a particular network deployment. Having put together the L3VPN use case example, I did try to highlight what the potential deployment concerns may be. I will look to expand Section 5 of the draft to be much clearer on risks and the way to choose whether a deployment of this functionality is "a good choice" as per your recommendation.

On the wider point, (and at the risk of sounding like a broken record) my view is that we need to balance absolute protocol correctness against the commercial and technical constraints within which protocols are deployed. Sometimes this means that there are knobs introduced that are potentially harmful, however, this shouldn't mean we don't have the knobs, just that they should be clearly labelled as to their potential issues, so that an operator can decide whether their particular deployment requires the functionality. This does not necessarily mean cataloguing all the possible ways a new mechanism could be deployed, and dictating which are acceptable (particularly as what is acceptable in some networks may not be in others), but rather (as per Joel's  suggestion) providing the necessary pointers as to how to make such a decision.

(I also support this document's adoption as a working group draft, and encourage further discussion as to additions that should be made in the next revision).

Cheers,
r.

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 07 Jun 2012 06:57:15 -0700

Hi Pradosh,

I am therefor awaiting a new section in the draft that quantitatively 
articulates the risk and Internet wide behaviour including BGP wave 
effect caused by re-advertisement of entire IPv4/IPv6 table upon the bgp 
session reset.

    As the STALE routes are not dynamically updated anymore, it's
    desirable that they be only used in last resort.  Hence when
    comparing paths for a prefix, a non STALE path should be preferred
    over a STALE path.

Btw enabling the above quoted text still suffers from the identical loop 
in the partially upgraded AS as shutdown draft. Shutdown solved this by 
enforcing no change to best path itself other then via local pref. The 
draft at this point only describes use of local pref in the case of all 
paths being STALE or says that ASBR if receiving paths over EBGP as 
stale "MAY" lower local pref. This is not correct.

However I also have a suggestion to a bit mitigate the effect wave 
effect of re-advertisement. As of now the draft reads:

    Assuming a session failure has occurred, a BGP persistent router
    SHOULD retain BGP routes unless they carry the DO_NOT_PERSIST
    community and propagate paths to downstream speakers that indicate
    that a given path is now stale.

How about modification to the above text with the following:

    Assuming a session failure has occurred, a BGP persistent router
    SHOULD retain BGP routes unless they carry the DO_NOT_PERSIST
    community and only after GR timer expiry propagate paths to
    downstream speakers that indicate that a given path is now stale.

The above change still considers the case where basic GR is deployed and 
allows to avoid the BGP churn when BGP session flaps. If it does not 
come back you can start your re-advertisement mark stating that route is 
stale.

As in all cases you are assuming that next hop is still reachable why 
the document does not recommend to upon the above BGP session failure to 
automatically install/activate in FIB IP or in the case of intra-domain 
MPLS or IP encapsulation of subject traffic to a BGP next hop ? That way 
you are actually assuring uninterrupted data plane delivery while local 
control plane recovers. Of course this would only apply to those BGP 
speakers which are in data plane.

Rgs,
R.

>> But I was in particular asking for a stronger text explicitly
>> forbidding implementation to make IPv4 or IPv6 routes as
>> persistent.
>
>
> I prefer to have a section that articulates the risks and provides
> design guidelines, as per Joel's suggestion. That would be a better
> technical specification than making a table of AFI/SAFI
> applicability.
>
> - Pradosh
>
>
>

---

From: "David Smith (djsmith)" <[email-masked]>
Date: Thu, 7 Jun 2012 09:05:47 -0500

support

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
John Scudder
Sent: Thursday, May 31, 2012 1:10 PM
To: [email-masked] List
Subject: [Idr] Adoption of draft-uttaro-idr-bgp-persistence-01 as IDR
WGdocument

Folks,

We have received a request from the authors to adopt
draft-uttaro-idr-bgp-persistence-01 as an IDR WG document.  Please send
your comments to the list.  The deadline for comments is June 15, 2012
at noon EDT.

Thanks,

--John

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 07 Jun 2012 07:06:00 -0700

Hi Rob,

> Sometimes this means that there are knobs introduced that are potentially harmful, however, this shouldn't mean we don't have the knobs, just that they should be clearly labelled as to their potential issues, so that an operator can decide whether their particular deployment requires the functionality.

The observation I would like to make to the above quote is that while I 
think we can go wild with zoo of knobs for local AFI/SAFIs (example 
L2VPN or L3VPN) we should be very careful to provide knobs destabilizing 
the Internet.

And the main issue is that the ISP configuring such knob will not suffer 
but the entire internet will. The ISP in most cases careless about 
remote folks router's CPU when he decides to re-advertise with STALE, 
his peers will not delete this community and upon his link flap he will 
start to bump the version numbers of all BGP speakers within and 
including his upstream reach. I bet in most cases he will not even 
realize that ... most vendors manuals or release notes provide text what 
the knob does without specifying a thing about the side effects.

Rgs,
R.

---

From: <[email-masked]>
Date: Thu, 7 Jun 2012 14:08:25 +0000

Hi Robert, 

>From: Pradosh Mohapatra >Sent: Thursday, June 07, 2012 9:25 AM
>
>Hi Robert,
>
>
>I prefer to have a section that articulates the risks and provides design
>guidelines, as per Joel's suggestion. That would be a better technical
>specification than making a table of AFI/SAFI applicability.

+1

Note also that there is no strict mapping between a service and an AFI/SAFI.
E.g. to come back to the Internet service you picked:
- one network could carry Internet routes inside a L3 VPN. 
- some IP unicast routes may never be advertised in the Internet routing system. 

Bruno

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 07 Jun 2012 07:12:57 -0700

Hi Bruno,

> - one network could carry Internet routes inside a L3 VPN.

That's exactly why I am not talking services, but AFI/SAFI limit. You 
may do what you like within 1/128. But please do not break 1/1 as I may 
transit via ft/orange.

> - some IP unicast routes may never be advertised in the Internet routing system.

not sure how this is relevant to the topic.

Best,
R.

---

From: Pierre Francois <[email-masked]>
Date: Thu, 07 Jun 2012 16:35:42 +0200

Hello,

Support.

Pierre.

On 5/31/12 7:10 PM, John Scudder wrote:
> Folks,
>
> We have received a request from the authors to adopt draft-uttaro-idr-bgp-persistence-01 as an IDR WG document.  Please send your comments to the list.  The deadline for comments is June 15, 2012 at noon EDT.
>
> Thanks,
>
> --John

---

From: Jakob Heitz <[email-masked]>
Date: Thu, 7 Jun 2012 11:45:53 -0400

>    comparing paths for a prefix, a non STALE path should be preferred

should a path with a shorter prefix also be preferred?
Say the stale path is 10.0.0.0/24 and 10.0.0.0/16 exists and is not stale.

---

From: Rob Shakir <[email-masked]>
Date: Thu, 7 Jun 2012 18:02:32 +0100

On 7 Jun 2012, at 15:06, Robert Raszuk wrote:

> The observation I would like to make to the above quote is that while I think we can go wild with zoo of knobs for local AFI/SAFIs (example L2VPN or L3VPN) we should be very careful to provide knobs destabilizing the Internet.

Apologies, I should have clarified this in my earlier mail, as such, the intention of (and discussions within) this mechanism is not to apply to all AFIs - quoting from the abstract:

"This document addresses new services whose requirements for persistence diverge from the Internet routing point of view."

The fact that (a proportion of) L[23]VPN deployments can be considered in isolation (i.e., for a particular SP's deployments) makes them more immediate targets for deployment of persistence; there may be deployments of IPv4 Unicast that exist in the same isolation (i.e., are explicitly NOT the Internet). For this reason, I do not think that explicitly forbidding a particular AFI/SAFI is required, but that does not mean the document should not provide strong wording that guides against deployment for Internet routing applications.

Cheers,
r.

---

From: <[email-masked]>
Date: Thu, 7 Jun 2012 20:38:05 +0000

Hi Robert,

>From: Robert Raszuk [mailto:[email-masked]]
>Sent: Thursday, June 07, 2012 4:13 PM
>
>Hi Bruno,
>

You said that in your opinion, persistence should not be used for Internet inter-domain routing because this would create churn in the Internet. 
Then you deduct that therefore, persistence must not be specified for AFI/SAFI 1/1.
In short your deduction is: no persistence for internet --> no persistence for AFI 1/1

Now what I'm saying is:

1) given that some AS may carry Internet (routing & traffic) within a L3 VPN (1/128), then according to your deduction, persistence should also not be used for 1/128 (because your Internet traffic may transit across an L3 VPN AS using persistence).

Hence limiting the applicability of persistence for 1/1 would not be enough to meet your goal.

>not sure how this is relevant to the topic.

2) Disallowing persistence for 1/1 would not allow using persistence for IPv4 unicast routes which are private only and never advertised in the Internet.

Hence "no persistence for internet --> no persistence for AFI 1/1 1" would be too much.

Conclusion [(1) + (2)]: I don't think limiting the applicability of persistence on a per AFI/SAFI basis is the best way to achieve your goal.

Best,
Bruno

---

From: <[email-masked]>
Date: Thu, 7 Jun 2012 20:50:45 +0000

Robert,

>From: Robert Raszuk
>Sent: Thursday, June 07, 2012 1:49 AM
>
>Jim,
>

You are assuming that persistence is meant to replace GR.
But this is _not_ the case.
Cf:
- Slide 4 of latest IDR meeting http://tools.ietf.org/agenda/83/slides/slides-83-idr-7.pdf
- draft section 7 http://tools.ietf.org/html/draft-uttaro-idr-bgp-persistence-01#section-7

In short: persistence and GR address largely independent use cases and can be enabled independently.

Hence in your case, if GR is used (to locally recover from a short duration failure in the control plane only), the addition of persistence does not change this (the same failure will also be locally concealed by GR).

Best regards,
Bruno

>_______________________________________________________________________________
>__________________________________________
>>>>
>>>>
>>>>
>>>>

---

From: Russ White <[email-masked]>
Date: Thu, 07 Jun 2012 19:43:57 -0400

> I prefer to have a section that articulates the risks and provides design guidelines, as per Joel's suggestion. That would be a better technical specification than making a table of AFI/SAFI applic…

We should make certain these get included in the final doc before it
passes to last call.

Russ

-- 
<><
[email-masked]
[email-masked]

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 08 Jun 2012 04:29:06 -0700

Bruno,

I think you have misunderstood my comment.

I am not assuming persistence to replace GR.

I am observing that re-advertsing all routes with STALE community when 
GR is enabled _immediately_ when the EBGP session goes down is a bad idea.

I am recommending to give GR and BGP session a chance to restart per GR 
spec. Then after the GR timer expires your persistence state machine 
could kick in.

Example:

t0 - session goes down
t1 - you start the best path run to look for alternative valid path
t2 - you start re-advertsing full table again (either new best path or 
for single best paths you attache STALE community)
t3 - session comes up

Note that session may be reestablish much sooner then you manage to 
propagate full table to all your peers.

The way the draft is written you have just caused a completely 
unnecessary churn. Note also that the next hop is still valid and 
reachable for all the paths.

So I would in fact not treat both as completely independent. I can see 
how they could be complementary and work together in concert.

Best,
R.

---

From: <[email-masked]>
Date: Fri, 8 Jun 2012 14:12:20 +0000

Robert,

>spec. Then after the GR timer expires your persistence state machine

I agree with your recommendation. Actually that's what the current version of the draft proposes:

- Cf §7. "Interactions between GR and Persistence"   http://tools.ietf.org/html/draft-uttaro-idr-bgp-persistence-01#section-7
- Or cf slide 5 of the slides presented in Paris which says:
"	- Persistence and GR can be enabled independently
	- If both are enabled on a BGP session, the principle is to start first with Graceful Restart
		a. If GR recovers --> GR ends, back to normal BGP --> Persistence never used
		b. else (GR fails) --> Persistence starts"

>how they could be complementary and work together in concert.

Great. I agree with you: that's roughly the title of slide 4: "Persistence & Graceful Restart: complementary use cases."

Best regards,
Bruno
>> - Slide 4 of latest IDR meeting
>http://tools.ietf.org/agenda/83/slides/slides-83-idr-7.pdf
>> Hence in your case, if GR is used (to locally recover from a short duration
>failure in the control plane only), the addition of persistence does not change
>this (the same failure will also be locally concealed by GR).
>>
>> Best regards,
>> Bruno
>>

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 08 Jun 2012 07:35:10 -0700

Hi Bruno,

> version of the draft proposes:

Actually the current version of the draft says much earlier:

"4.  Operation

4.1.  BGP session failure

    Assuming a session failure has occurred, a BGP persistent router
    SHOULD retain BGP routes unless they carry the DO_NOT_PERSIST
    community and propagate paths to downstream speakers that indicate
    that a given path is now stale."

That is wrong to say the above as below sections contradict with it.

> Great. I agree with you: that's roughly the title of slide 4:
> "Persistence & Graceful Restart: complementary use cases."

I am glad we are converging ;)

How about _mandating_ GR to be used with persistence ? At least for some 
global AFI/SAFIs ?

If so that may to some extend address my concern with unnecessary BGP 
churn while in the same time allow you to accomplish 100% of your goal?

-----

However for the record I am still not supportive of this idea in 
general. I clearly see that this idea is the tool to increase wrong 
designs of BGP networks. It's in fact a patch to a very bad network 
design. We should highly avoid such patches.

In network designs where you have native path redundancy as well as you 
do not relay on single vendor/source BGP implementation for any BGP 
control plane element this idea does not bring any value.

Yes I am aware that popular vendors do not support seamless multi-bgp 
implementation control planes today. But this can be/is being easily 
addressed.

Best regards,
R.

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Fri, 8 Jun 2012 14:42:41 +0000

Comments In-Line..

Jim Uttaro

-----Original Message-----
From: Robert Raszuk [mailto:[email-masked]] 
Sent: Friday, June 08, 2012 10:35 AM
To: [email-masked]
Cc: [email-masked] List; Joel M. Halpern; UTTARO, JAMES
Subject: Re: [Idr] Adoption of draft-uttaro-idr-bgp-persistence-01 as IDR WG document

Hi Bruno,

"4.  Operation

4.1.  BGP session failure

That is wrong to say the above as below sections contradict with it.
[Jim U>] Why is that.. If the two are used in conjunction thaen after GR behavior is complete and Persistence is activated then this could be a behavior based upon what the SP wants to do in terms of their unique requirements..

How about _mandating_ GR to be used with persistence ? At least for some 
global AFI/SAFIs ?
[Jim U>] Is that like a papal mandate ;) I think that folks are capable of understanding the behavior as much as any other technology..

-----

In network designs where you have native path redundancy as well as you 
do not relay on single vendor/source BGP implementation for any BGP 
control plane element this idea does not bring any value.
[Jim U>] The notion of BGP persistence as you point out would not be needed if after the things you mention ( Plus others ) would have been sufficient safeguarded our network from this type of failure mode.

Best regards,
R.

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 08 Jun 2012 08:08:39 -0700

Hi Jim,

> [Jim U>] Why is that.. If the two are used in conjunction thaen after
> GR behavior is complete and Persistence is activated then this could
> be a behavior based upon what the SP wants to do in terms of their
> unique requirements..

The above quote does not mention GR.

> In network designs where you have native path redundancy as well as
> you do not relay on single vendor/source BGP implementation for any
> BGP control plane element this idea does not bring any value. [Jim
> U>] The notion of BGP persistence as you point out would not be
> needed if after the things you mention ( Plus others ) would have
> been sufficient safeguarded our network from this type of failure
> mode.

Exactly.

IMHO we should shoot for right design not for patching badly made holes. 
Keep on mind that when we die (sooner or later) new folks will see 
persistent draft as an IDR/IETF recommendation.

I really do not see this an advantage of any sort.

Also Jakob's question still stays open .. Should STALE /16 be more 
preferred then STALE /24 which is covered by /16 ? While I have no idea 
how is he going to run best path across different prefixes it would be 
great to have community agreement on this question.

In general I am not sure if you realize, but this is difficult space you 
are entering from different perspective as the draft does not limit 
itself to only control plane BGP speaker action. In data plane some BGP 
implementations advertise only active (RIB/FIB) inserted routes while 
some do not care. Especially for the latter case where RIB has a route 
via different protocol and next hop self is set withdrawing it from BGP 
is not the best idea when original session the prefix was learned goes 
down. But I am not sure how much of real practical cases you would plan 
to include in the draft.

Thx,
R.

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Fri, 8 Jun 2012 15:28:17 +0000

Comments In-Line 

Jim Uttaro

-----Original Message-----
From: Robert Raszuk [mailto:[email-masked]] 
Sent: Friday, June 08, 2012 11:09 AM
To: UTTARO, JAMES
Cc: [email-masked]; [email-masked] List; Joel M. Halpern
Subject: Re: [Idr] Adoption of draft-uttaro-idr-bgp-persistence-01 as IDR WG document

Hi Jim,

Exactly.

IMHO we should shoot for right design not for patching badly made holes.
[Jim U>] Yes Robert.. I endeavor to create $hitty designs so that the network can fail and then I can spend my time explaining why we need some superfluous solutions to problems we created ourselves.. I mean as you point out the obvious design safeguards should be in place and gee whiz us dopey SP designers are just too plain stupid to figure out the basics of a robust scalable design.. What insight.. Hold on I am going to make some new holes in the network, going to get my shovel right now ;)

I really do not see this an advantage of any sort.
[Jim U>] Yes I think I have gotten that point

In general I am not sure if you realize, but this is difficult space you 
are entering from different perspective as the draft does not limit 
itself to only control plane BGP speaker action. 
[Jim U>] I think you have to realize that BGP is being used in lots of new applications in 2k+
In data plane some BGP 
implementations advertise only active (RIB/FIB) inserted routes while 
some do not care. Especially for the latter case where RIB has a route 
via different protocol and next hop self is set withdrawing it from BGP 
is not the best idea when original session the prefix was learned goes 
down. But I am not sure how much of real practical cases you would plan 
to include in the draft.

Thx,
R.

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 08 Jun 2012 08:40:07 -0700

Jim,

> whiz us dopey SP designers are just too plain stupid to figure out
> the basics of a robust scalable design.. What insight..

My comment is not about that "SP designers are just too plain stupid".

My comment is about that industry for it's commercial reasons does not 
offer today a hybrid BGP control plane solutions. Even when some vendors 
have multiple independent BGP implementation today and could offer it 
with minimal effort they are not. That one is just an amazing miss.

So you still want to patch holes rather then avoid them as you go by 
what is shipping rather then what should be shipping. I disagree with 
such approach.

Best,
R.

---

From: "John G. Scudder" <[email-masked]>
Date: Wed, 13 Jun 2012 15:24:44 -0400

Folks,

Although there has been a lively discussion about the merits of the draft, I have seen only four replies from people other than co-authors that clearly stated a preference for adoption. If you have not yet stated a preference, and have one, I encourage you to do so.

The comment period will end on Friday.

--John

P.S.: If you replied but changed the subject line it is possible I didn't include your message in my tally.

---

From: Randy Bush <[email-masked]>
Date: Thu, 14 Jun 2012 06:06:24 +0900

i guess i support adoption, but wonder if there is a walter scott idr
sub-wg

  o fix the grammar

  o 4.1.1 needs a capitalized MUST and has yucchy semantic issues,
    e.g. it mandates propagating a stale path whose pref has been
    lowered when that may not have been or may no longer be the best
    path.  it mandates generating a withdraw when the route may not have
    been propagated, ...

  o 4.1.2 needs to make clear that all routers should use the same
    value for knocking down local-pref

  o 8 there are other attacks.  e.g. a wire grab no longer needs to keep
    ip state up at all.  i am sure i could think of more if i had
    coffee.

randy

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 13 Jun 2012 16:25:59 -0700

Hi John,

#1 question which Jakob stated is still not answered. Should STALE path 
be less or more preferred then non stale less specific one. The question 
makes a lot of sense and if the answer is yes then I am afraid BGP 
implementations will become really complicated.

There is one way to avoid such complication by adding a trick to 
simple-va which would conditionally suppress stale path regardless of 
next hope if less specific covering it is present without STALE flag. 
This could be much simpler then messing with best path to compare 
different nets. That would also mean that persistence MUST be always 
used with simple-va.

#2 In all cases where persistence draft may apply (VPLS) you may as well 
configure the service without BGP. We are just propagating static config 
in BGP here essentially.

#3 For most other cases right network design removes the need for 
persistence draft. For transient BGP session flaps basic GR with perhaps 
longer timer works fine to maintain data plane connectivity.

#4 The proposal does not mandate new capability in BGP. In networks with 
mixed persistence support and no support especially on EBGP links this 
seems extremely dangerous feature. Draft discusses the sender cases 
where bgp speaker is persistence capable. But the draft neglects to 
mention the handling on the receiver where the receiver is not capable 
of recognizing the STALE community and still selects the STALE path as 
best even if there is non STALE path present for the given net. As 
community is an opaque value for it the re-advertisement of STALE will 
happen and good valid path even if longer will be not promoted as best.

As mentioned in the other comment the way the draft is worded today has 
a potential to introduce significant churn in the Internet when any 
single BGP speaker experiences BGP session flap and starts the 
re-advertise full table with and without STALE color. To mitigate that 
persistence was suggested to use _only_ with GR (and do nothing before 
the GR timer expires .. Bruno agreed with that - but the draft does not 
mandate GR which IMHO it should).

To clearly conclude looking at cost/benefit equation I am not supporting 
adoption of this proposal the way the draft is fundamentally written 
today. It is not that we could work on bits and pieces after adoption. 
It is that there are fundamental issues with it today.

Also I am not agreeing in principle with basic pre-assumption for this 
work Jim mentioned in the reply that flaky and "maybe reachable" route 
is better then no route. I prefer the latter. The reason being is that 
if we declare no reachability in BGP other (upper/application) layers 
could converge around it. If we "pretend" in BGP the reachability is 
there while it is not, application few hops away has no way of knowing it.

Best regards,
R.

> Robert,
>
> The general tone of your comments leads me to think you may be
> against WG adoption of the document, but I haven't seen any clear
> statement from you so thus far I am not counting you either way. If
> you want to clarify a position feel free to either ucast me or reply
> to the list.
>
> By the way, thank you for your contribution to the discussion.
>
> --John
>

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 13 Jun 2012 17:05:14 -0700

> there while it is not, application few hops away has no way of knowing it.

Let me expand a bit on this point as I think this is important.

Imagine an ASBR with BGP connection to peering AS as well as with 
wireless connection as backup to the other AS. The protocol running over 
wireless connection is ROLL.

By default ROLL will have higher admin distance (will be less preferred) 
when competing against the same prefix at the RIB insertion.

If we do not remove the flaky and "perhaps reachable" prefix from BGP 
the other backup path has no chance to be installed in RIB.

That effectively means that we have broken any backup/redundancy options 
which may be coming to a router outside of BGP.

/* The same story also applies to ALTO or similar tools where sources 
for information may be coming from other then BGP protocols. */

Best regards,
R.

---

From: Enke Chen <[email-masked]>
Date: Wed, 13 Jun 2012 17:32:16 -0700

Hi, folks:

As I stated on this mailing list before, I believe that most of the key 
requirements specified in the "persistence draft" can be easily 
addressed by a couple knobs with GR:

    o introduce the concept of the "minimal stale timer" in GR:  This 
will allow the stale routes to be kept for a much longer time if so 
configured locally.

    o use a stale path first, and then switch to an alternate path after 
a certain amount of time: This would be locally configurable.

They can be easily covered by adding some text in the GR re-spin which 
is in progress.  It seems logical also for GR to cover these points.  
Then GR would not be limited to just "shorter timers".

As the concept of GR is well established, and it already provides 
"persistence", I do not see much value for IDR to have another spec that 
conceptually and operationally overlaps with it.

-- Enke

---

From: "John G. Scudder" <[email-masked]>
Date: Wed, 13 Jun 2012 20:48:47 -0400

Robert,

To this point:

On Jun 13, 2012, at 7:25 PM, Robert Raszuk wrote:

> In networks with 
> seems extremely dangerous feature.

Your concern for "mixed persistence support" would actually seem to support the opposite point of view, that this *should* be standardized. As you know, one key reason to standardize things is to facilitate deployment of such features in mixed networks.

I expect this will not be persuasive to you given your other objections, but I thought I should point it out.

Regards,

--John

---

From: John Leslie <[email-masked]>
Date: Wed, 13 Jun 2012 20:52:55 -0400

John G. Scudder <[email-masked]> wrote:
> 
> Although there has been a lively discussion about the merits of the draft,
> I have seen only four replies from people other than co-authors that
> clearly stated a preference for adoption. If you have not yet stated a
> preference, and have one, I encourage you to do so.

   Despite the impressive author list, I'd really rather not adopt it.

   Most of my particular dislikes _could_ be addressed: but this sort
of thing should be between consenting adults, with warning labels; and
I don't see a practical way to ensure reasonably quick propagation of
unreachability.

   In no way do I mean to say that an individual AS can't continue to
_use_ stale routes -- but we don't need an RFC for them to do so. I'm
very nervous about _advertising_ stale routes; and I fear that with
idr-bgp-persistence, we'd have legacy routers advertising them as if
they were good.

   (You asked...)

--
John Leslie <[email-masked]>

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 13 Jun 2012 19:52:50 -0700

Hi John,

Actually not. Standardization does not equal that a feature will be 
enabled or router image automatically upgraded after RFC is issued :).

As long as EBGP speaker I do not control will not support this extension 
I will still signal a STALE path with the community which he will 
consider as good as not STALE one.

To your point the case today between any two adjacent ASes (and within 
the AS) you can achieve persistence by just agreeing on some community 
value internally.

Best regards,
R.

---

From: Enke Chen <[email-masked]>
Date: Wed, 13 Jun 2012 20:09:05 -0700

Just try to point out the obvious:  in general a new global community is 
difficult to roll out.  The lack of scoping creates many operationally 
issues such as who can set it, who can override it, and where to accept 
and where to filter.

It can be a burden for all involved, e.g., it has to be filtered out 
everywhere in a network if one does not want to use it.

-- Enke

On 6/13/12 7:52 PM, Robert Raszuk wrote:
> As long as EBGP speaker I do not control will not support this 
> extension I will still signal a STALE path with the community which he 
> will consider as good as not STALE one.
>
>> Your concern for "mixed persistence support" would actually seem to 
>> support the opposite point of view, that this *should* be 
>> standardized. As you know, one key reason to standardize things is to 
>> facilitate deployment of such features in mixed networks.
>>
>> I expect this will not be persuasive to you given your other 
>> objections, but I thought I should point it out.
>>
>> Regards,
>>
>> --John

---

From: Jie Dong <[email-masked]>
Date: Thu, 14 Jun 2012 08:09:31 +0000

> attempt to do path validation or other fancy processing inbound of each

Agreed that using a single message instead of re-advertising millions of routes in such cases would be better. 

Best regards,
Jie

> Sent: Thursday, June 07, 2012 7:49 AM
> To: UTTARO, JAMES
> Cc: [email-masked] List; Joel M. Halpern
> ________________________________________________________________
> _________________________________________________________
> >>>
> >>>
> >>>
> >>>

---

From: <[email-masked]>
Date: Thu, 14 Jun 2012 10:26:46 +0000

Hi Jakob,

>From: Jakob Heitz [mailto:[email-masked]] >Sent: Thursday, June 07, 2012 5:46 PM
>

You are raising a good point.
While preferring non stale more specific prefix over stale prefix could probably make sense, IMHO I would not push persistence in this direction:
- too much change in BGP
- to some extend, the point of more specific prefixes being preferred in the forwarding plane independently of the BGP preferences is a different subject. E.g. draft-francois-limited-scope-specifics-01 is another aspect.

Regards,
Bruno

---

From: <[email-masked]>
Date: Thu, 14 Jun 2012 10:32:09 +0000

Robert,

>From: Robert Raszuk >Sent: Thursday, June 14, 2012 1:26 AM
>
>Hi John,
>
>#1 question which Jakob stated is still not answered. 

Done.

>Should STALE path

I'm missing your point. In this cases, are you calling for persistence or are you calling for not using BGP?

>longer timer works fine to maintain data plane connectivity.

GR hides the issue.
Persistence let's application know about the issue while allowing application to still use the route.
IMO, proving information and choice to applications seems a least an option to consider.

>happen and good valid path even if longer will be not promoted as best.

Within the AS, I don't recall that the draft "neglects" anything.
Between ASes, the issues is indeed more complexes and probably still need to be worked on. Yet, some simple solutions addressing the deployment uses cases seem a priori doable (e.g. withdraw stale path on eBGP except on specifically agreed BGP sessions). Possibly, the advertisement of a BGP capability could possibly be useful over eBGP sessions.

>re-advertise full table with and without STALE color.

No. That's not an accurate summary of your other comment. You were comparing GR versus persistence. I replied that persistence is not intended to replace GR. Hence you should compare GR vs GR + persistence. Then you replied that "that may to some extend address my concern with unnecessary BGP churn". IMHO this is a understatement, but I called it quit.

> To mitigate that

No, I did not agreed with that.
Cf the above for a summary of what I said.
So far, I don't see a reason to mandate GR when persistence is used.
Also, your point seems rather to mandate GR in all cases, independently of persistence. This point is out of scope of the persistence draft. Write another draft if you want to mandate GR.

>It is that there are fundamental issues with it today.

You would need to detail the "fundamental issues" that you are seeing. If these are the points 1 to 4 in this email, I have not being convinced.

>there while it is not, application few hops away has no way of knowing it.

Again, that's exactly why persistence is richer than the "just use GR" proposal that you are pushing as an alternate. 
I'm not sure what kind of "applications" you are referring to. If they don't have access to BGP routes, they probably don't care about BGP routes. If they do, persistence does provide the information regarding the states of those routes. Contrary to GR, it does not "pretend" that the BGP reachability has not changed.

Best regards,
Bruno

---

From: <[email-masked]>
Date: Thu, 14 Jun 2012 10:32:20 +0000

Robert,

>From: Robert Raszuk >Sent: Thursday, June 14, 2012 2:05 AM
>

That's exactly why the persistence approach is richer than the GR approach that you are pushing instead. Indeed, the key difference is that persistence do advertise the error condition and does not "pretend" nothing happened (like GR does).

>when competing against the same prefix at the RIB insertion.

IMHO the interaction between BGP and ROLL is probably not the key concern for the community (or the key reason the disregard the persistence proposal)
As you mentioned, the point you raised is the default configuration. Persistence could probably address this by changing the admin distance of the routes marked as stale. OTOH, I'm a bit curious on how your preferred alternative (GR) would handle your imaginary case.

Best regards,
Bruno

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Thu, 14 Jun 2012 12:59:11 +0000

Ah the morning mail has arrived ;) Comments In-Line..

Jim Uttaro

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Robert Raszuk
Sent: Wednesday, June 13, 2012 7:26 PM
To: John G. Scudder
Cc: [email-masked] List
Subject: Re: [Idr] Adoption of draft-uttaro-idr-bgp-persistence-01 as IDR WG document

Hi John,

[Jim U>] I do not see this in scope of BGP Persistence.. This was never the intent and I see no reason to change that for the applications of interest

#2 In all cases where persistence draft may apply (VPLS) you may as well 
configure the service without BGP. We are just propagating static config 
in BGP here essentially.
[Jim U>] BGP is a powerful tool for discovery and signaling..I guess we could also build GRE tunnels to create L3 VPNs too.. But these approaches are simply not practical..

#3 For most other cases right network design removes the need for 
persistence draft. For transient BGP session flaps basic GR with perhaps 
longer timer works fine to maintain data plane connectivity.
[Jim U>] I think I have addressed the Persistence vs GR ad nauseaum.. So I guess here goes high level yet again...
GR is done at a per session lacks path granularity
GR does not survive through failure modes that I require i.e Timer Expiry, Malformed Update etc...
GR is better suited where the session is assumed to be coming back in a short timeframe
GR semantic ties the session to the forwarding as is appropriate
GR does not survive through multiple restarts
GR upon coming back floods all of the paths relearned.. I think I saw a draft to "fix" this but have not read it yet..It is good that the GR authors are coming around to the "make before break" approach as detailed in the BGP Persistence draft as it makes sense for the GR application also.
GR assumes that paths that are "suspect" are still considered best, there is no informing the topology that paths over this session should be de-preffed.. Essentially GR is deciding for the SP. I want the control.

Do we really need to have this conversation again, and again, and again.. The religious aspect is becoming annoying.. 

[Jim U>] Addressed in Paris. 

[Jim U>] Internet is not the intended use case. 

[Jim U>] Incorrect.. I never said the path was flaky, only the mechanism by which it was learned may be suspect.. I think folks need to get a perspective on other applications that use BGP..

Best regards,
R.

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Thu, 14 Jun 2012 13:02:08 +0000

Enke,

Here is the text from my note to Robert.. 

Did not get a chance to look back at the other e-mails on this so this list may be incomplete..

If we are going to re-spin GR to become Persistence then maybe we should simply add the subset of functionality provided by GR to the Persistence draft..

Jim Uttaro

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Enke Chen
Sent: Wednesday, June 13, 2012 8:32 PM
To: John G. Scudder
Cc: [email-masked] List
Subject: Re: [Idr] Adoption of draft-uttaro-idr-bgp-persistence-01 as IDR WG document

Hi, folks:

-- Enke

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 14 Jun 2012 06:07:43 -0700

Bruno,

 > While preferring non stale more specific prefix over stale prefix
 > could probably

You have completely reversed the question. More specific will be 
preferred today by default no need to discuss it.

The question was about preferring less specific non stale vs more 
specific stale.

I gave example how it could be solved with simple-va easy extension.

Best,
R.

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Thu, 14 Jun 2012 13:10:27 +0000

Enke,

	One other note here..  

	The notion of "can easily be addressed by a couple of knobs with GR:" is exactly the problem with using GR.. Should I have to go ahead and modify the behavior on thousands of endpoints by changing different knobs, oh crap I better make sure they all realize the same behavior.. This is impractical from a consistent network architecture/design viewpoint.. 

Jim Uttaro

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Enke Chen
Sent: Wednesday, June 13, 2012 8:32 PM
To: John G. Scudder
Cc: [email-masked] List
Subject: Re: [Idr] Adoption of draft-uttaro-idr-bgp-persistence-01 as IDR WG document

Hi, folks:

-- Enke

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Thu, 14 Jun 2012 13:14:02 +0000

Without divulging too much internal AT&T plans, we are embarking on rolling out a number of communities to accomplish our network design goals. These communities have recently been defined and are the process of standardization. I think we can figure out the operational end.. What we really need is for folks to understand the reqs we have for BGP today in the post internet BGP signaled era.. 

Jim Uttaro

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Enke Chen
Sent: Wednesday, June 13, 2012 11:09 PM
To: [email-masked]
Cc: [email-masked] List
Subject: Re: [Idr] Adoption of draft-uttaro-idr-bgp-persistence-01 as IDR WG document

-- Enke

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 14 Jun 2012 06:20:10 -0700

Jim,

> [Jim U>] BGP is a powerful tool for discovery and signaling..I guess
> we could also build GRE tunnels to create L3 VPNs too.. But these
> approaches are simply not practical..

For your reference BGP next hop is used today to automatically 
encapsulate L3VPN packets from src to remote PE.

mGRE is a very practical approach .. much more practical and much more 
scalable then running LDP or RSVP-TE for providing service transport 
encapsulation in number of cases .. from large service provider WANs 
with 10,000s PEs to modern data centers with 100,000s+ hypervisors.

I am amazed by your lack of practical information.

Best,
R.

---

From: Randy Bush <[email-masked]>
Date: Thu, 14 Jun 2012 22:59:57 +0900

> The notion of "can easily be addressed by a couple of knobs with GR:"
> is exactly the problem with using GR.. Should I have to go ahead and
> modify the behavior on thousands of endpoints by changing different
> knobs, oh crap I better make sure they all realize the same
> behavior.. This is impractical from a consistent network
> architecture/design viewpoint..

won't all bgp speakers in your network will have to be configured to
deal with persistence?  same kinky deal.  death by complexity.

randy

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Thu, 14 Jun 2012 14:09:59 +0000

Yes.. But there is no variation in the config it is applied the same way on all supported endpoints regardless of vendor, OS release, platform.. Knobs by various vendors on different releases generally mean inconsistency.. I do think that some AFI/SAFI should be configured with Persistence as a default but that is not in scope of this discussion...

Jim Uttaro

-----Original Message-----
From: Randy Bush [mailto:[email-masked]] 
Sent: Thursday, June 14, 2012 10:00 AM
To: UTTARO, JAMES
Cc: 'Enke Chen'; John G. Scudder; [email-masked] List
Subject: Re: [Idr] Adoption of draft-uttaro-idr-bgp-persistence-01 as IDR WG document

randy

---

From: Randy Bush <[email-masked]>
Date: Thu, 14 Jun 2012 23:13:54 +0900

> Yes.. But there is no variation in the config it is applied the same
> way on all supported endpoints regardless of vendor, OS release,
> platform.. Knobs by various vendors on different releases generally
> mean inconsistency.

for those who still configure networks at the keyboard while talking of
a post-bgp internet.  :)

randy

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 14 Jun 2012 07:16:52 -0700

> I do think that some AFI/SAFI should be configured with Persistence as a default

Let me point our that you are apparently confusing NETCONF with BGP.

The former is persistent by design the latter is not (and it is a 
feature for the latter not a bug).

Best,
R.

---

From: <[email-masked]>
Date: Thu, 14 Jun 2012 14:59:26 +0000

Robert,

>You have completely reversed the question. 

You are right. My mistake in rephrasing. Please read :s/more/less
(i.e.: While preferring non stale less specific prefix over stale prefix could probably make sense...)

Sorry for the spam.
However, hopefully, there should not be typo in other part of the email.

>More specific will be
>> While preferring non stale more specific prefix over stale prefix could
>probably make sense, IMHO I would not push persistence in this direction:
>> - too much change in BGP
>> - to some extend, the point of more specific prefixes being preferred in the
>forwarding plane independently of the BGP preferences is a different subject.
>E.g. draft-francois-limited-scope-specifics-01 is another aspect.
>>
>> Regards,
>> Bruno
>>
>>
>> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu
>ce message par erreur, veuillez le signaler
>> a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
>electroniques etant susceptibles d'alteration,
>information that may be protected by law;
>> As emails may be altered, France Telecom - Orange is not liable for messages
>that have been modified, changed or falsified.
>> Thank you.
>>
>>
>>
>

---

From: Enke Chen <[email-masked]>
Date: Thu, 14 Jun 2012 08:54:35 -0700

Jim,

It may be true that "you can figure out the operational end",  but can 
the whole world figure out?  With a global community, the whole world 
will need to deal with it.   I do not think that is necessary, nor fair.

-- Enke

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Thu, 14 Jun 2012 18:23:43 +0000

Enke,

	Not sure I understand. Referring to the slides Bruno presented is your concern that the STALE CV is used to provide the semantic across AS boundaries?? For the iBGP case the use of Cost Community, Local Pref have been identified as mechanisms to de-pref the STALE path. Is this what you are speaking too?.. Across AS boundaries the STALE CD would only need to be interpreted once and the desired iBGP mechanism set.. Pls Clarify..

The technology is necessary and we need to be fair to our customers who expect that a failure in our control plane that simply advertises paths should not tear down their VPLS,VPN,3107 MultiCast etc...network.

Jim Uttaro

-----Original Message-----
From: Enke Chen [mailto:[email-masked]] 
Sent: Thursday, June 14, 2012 11:55 AM
To: UTTARO, JAMES
Cc: [email-masked]; [email-masked] List; Enke Chen
Subject: Re: [Idr] Adoption of draft-uttaro-idr-bgp-persistence-01 as IDR WG document

Jim,

-- Enke

---

From: Enke Chen <[email-masked]>
Date: Thu, 14 Jun 2012 11:36:13 -0700

Jim,

What happens if an EBGP peer does not recognize this global STALE 
community?  Isn't true that It may end up using the path that has been 
stale for days even in the presence of an alternate path.

-- Enke

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Thu, 14 Jun 2012 19:05:18 +0000

Enke,

	You are essentially echoing my concern for using GR and how it does not inform an iBGP topology about path learned over a GR Active session.. I can support this if GR is used as originally intended as a short duration hit with control plane expected back quickly, but too be clear if there was an alternate path within the iBGP topology it would not be used while GR is active and not informing that paths learned there are suspect within or between AS domains true??. We cannot have it both ways.

To address your concern directly. In the models I am considering we control or have cooperative arrangements with other partners that would ensure that the behavior you describe would not occur. For a generic eBGP peer with a customer the first thing that must happen is that the de-preffed path is the "only" path. For a VPN customer the majority of Hub sites i.e DCs are multi-homed, in the internet model I am not that close too but I believe we have multiple peering points, that being said we must clearly identify the use case and the risk/benefit analysis.

Jim Uttaro

-----Original Message-----
From: Enke Chen [mailto:[email-masked]] 
Sent: Thursday, June 14, 2012 2:36 PM
To: UTTARO, JAMES
Cc: [email-masked]; [email-masked] List; Enke Chen
Subject: Re: [Idr] Adoption of draft-uttaro-idr-bgp-persistence-01 as IDR WG document

Jim,

-- Enke

---

From: Jakob Heitz <[email-masked]>
Date: Thu, 14 Jun 2012 16:08:39 -0400

I asked whether it was necessary, not whether it was too hard.

There is (in my mind) an essential difference
between a stale path and a low preference path:

A low preference path exists.
A stale path may not exist.

In GR, we consider a short term existence doubt acceptable.
You are proposing a long term existence doubt.

This is the possibility of long term continuous
traffic drop, not just traffic taking a longer path.

On Thursday, June 14, 2012 3:27 AM, [email-masked] <mailto:[email-masked]> wrote:

>>>    comparing paths for a prefix, a non STALE path should be
>>>    preferred over a STALE path.
>> 
> could probably make sense, IMHO I would not push persistence in this
> direction:  
> - too much change in BGP
> - to some extend, the point of more specific prefixes being preferred
> in the forwarding plane independently of the BGP preferences is a
> different subject. E.g. draft-francois-limited-scope-specifics-01 is
> another aspect.   
> 
> Regards,
> Bruno
> 
-- 
Jakob Heitz.

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Thu, 14 Jun 2012 20:19:56 +0000

Jakob,

Comments In-Line..

Jim Uttaro

-----Original Message-----
From: Jakob Heitz [mailto:[email-masked]] 
Sent: Thursday, June 14, 2012 4:09 PM
To: [email-masked]; [email-masked]
Cc: Pradosh Mohapatra; UTTARO, JAMES; [email-masked] List; Joel M. Halpern
Subject: RE: [Idr] Adoption of draft-uttaro-idr-bgp-persistence-01 as IDR WG document

A low preference path exists.
[Jim U>] Agreed
A stale path may not exist.
[Jim U>] Based on the service this is certainly true. Thus the reasoning is that Persistence is applicable to use cases where the "risk" of inconsistency is out-weighed by a total outage.. as an ex, a 3107 BGP solution that creates an IGP is one which is very stable and should never change unless a NH is added/deleted, a BGP Signaled VPLS is another ex.. Let me be very clear we are not advocating that Persistence be used in an environment where there is a lot of normal churn that changes paths and associated NHs..

In GR, we consider a short term existence doubt acceptable.
[Jim U>] Agreed. That is exactly the philosophical reason that changing knobs and timers does not turn it into Persistence.. The fundamental assumption is that it is a short duration hit. To increase the timers turns it into a medium/long existence and is not useful if there are other alternative paths available and GR is not capable of informing the topology of suspect paths..
You are proposing a long term existence doubt.
[Jim U>] Yes.

This is the possibility of long term continuous
traffic drop, not just traffic taking a longer path.
[Jim U>] First select appropriate AF/SAFI, service.. Then make a determination of catastrophic vs incremental outage...

-- 
Jakob Heitz.

---

From: Randy Bush <[email-masked]>
Date: Fri, 15 Jun 2012 08:15:33 +0900

> The technology is necessary and we need to be fair to our customers
> who expect that a failure in our control plane that simply advertises
> paths should not tear down their VPLS,VPN,3107 MultiCast
> etc...network.

so you are expecting catastrophic failures of your control plane and yet
you expect this ever more complex mechanism to bail you out.  and you
require that your ebgp peers play along?

randy

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 14 Jun 2012 17:31:06 -0700

> GR is done at a per session lacks path granularity

That's another characteristic of the proposal which requires serious 
consideration. In my previous 4 points I have focused on effects of 
re-advertisement of paths marked as STALE.

But the proposal introduces upon BGP session failure per path decision 
to keep and use the path (+re-advertise as STALE) or not.

That effectively means that all BGP implementation optimizations which 
went into the shipping code basis to delete as soon as possible _all_ 
paths which came from particular BGP session upon it's failure are now 
to be modified to consider only those which carry DO_NOT_PERSIST community.

Having GR per session and Persistance per path is also an interesting 
complication. The current draft seems to be saying (unlike some authors) 
that it is ok to keep GR in place and during GR timer do not resignal 
them as STALE. However we have heard already that it is wrong to keep 
the path in GR mode for the GR timer without telling about it to customers.

Quote:

"GR assumes that paths that are "suspect" are still considered best, 
there is no informing the topology that paths over this session should 
be de-preffed.. Essentially GR is deciding for the SP. I want the control."

I am quite not clear what the behaviour of GR+Persistance is really 
expected to be.

While all can be developed it does cost cpu, memory and stability.

Regards,
R.