Subject: grow: anycast ops draft

From: Joe Abley <[email-masked]>
Date: Mon, 1 Nov 2004 15:03:28 -0500

Kurtis Lindqvist and I squeezed this draft through before the cut-off:

   http://www.ietf.org/internet-drafts/draft-kurtis-anycast-bcp-00.txt

> Abstract
>
>    As the Internet has grown, many services with high availability
>    requirements have emerged.  The requirements of these services have
>    increased the demands on the reliability of the infrastructure on
>    which those services rely.
>
>    Many techniques have been employed to increase the availability of
>    services deployed on the Internet.  This document presents
>    operational experience of wide-scale service distribution using
>    anycast, and proposes a series of recommendations for others using
>    this approach.

Anycast distribution of services (the practice of injecting 
reachability information for an address into a routing system in 
multiple places, from autonomous nodes) is not at all a new idea; 
however, there are an increasing number of people using anycast 
approaches to distribute high-profile services, and few published 
guidelines for doing so. The aim of this document is to provide some 
guidelines.

There are many, many people with experience deploying anycast services 
whose thoughts and opinions are not in that -00 draft. Hopefully some 
of them are on this list, and they can help us make the draft better.

We have a brief slot in the grow meeting next Tuesday to introduce this 
draft, and we plan to propose that it becomes a working group document. 
If people on this list have a chance to review the draft before then, 
that would be great.

Joe

_________________________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/grow.html
web archive:        http://darkwing.uoregon.edu/~llynch/grow/

---

From: Pekka Savola <[email-masked]>
Date: Mon, 8 Nov 2004 14:23:06 +0200 (EET)

>  http://www.ietf.org/internet-drafts/draft-kurtis-anycast-bcp-00.txt

I see Geoff already responded with a lot of good comments.  Some 
comments from me..

Overall I thought this was a very good and useful document, definitely
something we should be producing.  A loud 'yes' for the adoption for this as
WG item!

Obviously, as with all -00 I-Ds, there's a lot which can be improved. 
Below.

substantial
-----------

    Where a routing advertisement from a node corresponds to a single
    Service Address, this coupling might be such that availability of the
    service triggers the route advertisement, and non-availability of the
    service triggers a route withdrawal.  This can be achieved using
    routing protocol implementations on the same servers which provide
    the service being distributed.

==> The last sentence is a required but not sufficient condition.  The draft
should also expand the text (at least a paragraph) to say how the routing
protocol (or some other means) monitors the "health" of the service.

That is, just because the node participates in the routing protocol,
it does not offer a guarantee of the health of the service, just about
the health of the node.  And this is vital with anycast, as we don't want
the requests to be blackholed if the service does not work, but the nodes
are still operational.

    Where multiple Service Addresses are covered by the same covering
    route, there is no longer a tight coupling between the advertisement
    of that route and the individual services associated with the covered
    host routes.  The resulting impact on signaling availability of
    individual services is discussed in Section 4.4.1.

==> the draft goes on at significant length to describe hacks to work around
this problem, to be able to deploy multiple services at the same anycast
prefix.

The draft needs to expicitly state, in a separate paragraph, a simpler
alternative: "just don't do that".  It's sufficient for DNS at least.  Get a
different prefix for every service.  Assuming that something around
/24 is sufficient, there are still plenty of those.

That's much simpler operationally than running tunneling across different
nodes.

    For services which are distributed across the global Internet using
    BGP, equal-cost paths are normally not a consideration: BGP's exit
    selection algorithm usually selects a single, consistent exit for a
    single destination regardless of whether multiple candidate paths
    exist.  Implementations of BGP exist that support multi-path exit
    selection, however, and corner cases where dual selected exits route
    to different nodes are possible.  Analysis of the likely incidence of
    such corner cases for particular distributions of Anycast Nodes are
    recommended for services which involve multi-packet transactions.

==> I fear that the BGP ecmp has to become more prominent here.  AFAICS,
it's implemented and used especially for stub ASs which, for some reason,
want to provide this kind of more balanced load balancing.

4.4.5  Reverse Path Forwarding Checks

    Reverse Path Forwarding (RPF) checks, first described in [8], are
    commonly deployed as part of ingress interface packet filters on
    routers in the global Internet in order to deny packets whose source
    addresses are spoofed (see also [10]).  Deployed implementations of
    RPF make available two modes of operation: a loose mode, and a strict
    mode.

    Strict-mode RPF checks can cause non-spoofed packets to be denied
    when they originate from multi-homed site, since selected paths might
    legitimately not correspond with the ingress interface of non-spoofed
    packets from the multi-homed site.  A collection of anycast nodes
    deployed across the Internet is largely indistinguishable from a
    distributed, multi-homed site to the routing system, and hence this
    risk also exists for anycast nodes, even if individual nodes are not
    multi-homed.

    Care should be taken to ensure that strict-mode RPF is not enabled in
    peer networks connecting to anycast nodes.

==> I think there are a lot of problems with this text, and you may not have
thought it through.  A few comments:

  - the first and second paragraph should be referencing RFC3704, which
is discussing *exactly* these problems, and also introduces different
flavours of doing RPF.

  - the second and third paragraphes are not really accurate.  It depends on
how the preferences are selected.  If you hook up an anycast node to its
first-hop ISP, almost always the ISP will consider that exit preferable, so
strict RPF works.  (Where it may not work is further upstream, but there
strict RPF-like techniques are not applied in any case..)
Even if strict RPF did not work, you could still use
feasible path RPF as described in RFC3704.

The way I see it, the biggest issue with RPF-like techniques is when people
don't employ them, and use static access lists instead.  As anycast
addresses might not conform to the local topology, the guy/gal who has
written the ACLs might have forgotten the anycast prefix(es) completely.

    As is described in  having the Anycast Node avoid black-holing
    traffic in the event of a failure on the software or subsystem
    providing the service should be avoided.  As described, this can be
    done with withdrawing the announcement of the prefix corresponding to
    the service address, or the covering route.  However, the nodes could
    also try and handle the failure in a number of ways.  This can be
    with as also previously described tunneling to other instances of the
    Anycasted service, and using a IGP over the tunnels, route incoming
    client queries to the other destination.  The Anycasted node could
    also contain separate systems for trying to restart the service in
    question, and if successful again re-announce the service prefix.

==> this seems like a major operational pain in the ass, and a way to add
mysterious errors in the process.

Consider for example the effect of tunneling on MTU, as discussed in
draft-savola-mtufrag-tunneling-02.txt.  If the anycast is used in such a
manner (non-DNS/UDP) that the MTU is maxed out, the tunneling likely
will cause problems in some cases.  Does the tunnel implementation do
fragmenting -- if so, even with DF bit set for v4, and what about v6? If
not, and just sends ICMP errors back, the service just mysteriously fails
(under these rare self-healing conditions, making it very difficult to
debug) for those poor bastards who are inappropriately filtering the ICMP
packet too big messages.

All in all, I'd think tunneling is a bad option here and we should work
around with that by recommending appropriate allocation policies.  It's
still a good idea to discuss the problems of tunneling though -- I'd suggest
an appendix for that, if it's too prominent in the spec.

6.  Security Considerations

==> this section probably needs to discuss, with a short paragraph, issues
relating to availability.  A badly implemented anycast solution has a good
chance to decrease the availability rather than increase it.
As any anycast node is capable of blackholing its
users, it's often also good sense to provide multiple independent
addresses/services to achieve the same service.  E.g., there should be
a couple of addresses in DNS for service X, which would be anycasted by
differerent sets of nodes.

semi-editorial
--------------
    The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
    "SHOULD", "SHOULD NOT", "RECOMMENDED",  "MAY", and "OPTIONAL" in this
    document are to be interpreted as described in RFC 2119.

==> if you use these keywords (and I'm not sure whether it's a good idea
because this is an operational document, not a document describing how
to make interoperable implementations, you must also add normative
reference to it.

    Anycast Node: an internally-connected collection of hosts and routers
       which together provide service for an anycast service address.

==> the way you say this is confusing '.. hosts and routers', and can be
read two ways:
  1) "one instance providing a particular anycast service", or
  2) "an anycast service, including all the nodes that implement it"
I think you mean the latter, but it requires rewording to make that clear. 
I suggest you don't use the plural tense.

==> please also add linebreaks between different definitions ?

    4.  Triangulation of traffic sources, in the case of attack (or
        query) traffic which incorporates spoofed source addresses;

==> this is too brief, it's not clear what you mean here; please expand.

    Some services have very short transaction times, and may even be
    carried out using a single packet request and a single packet reply
    in some cases (the DNS is an example of this).

==> s/DNS/DNS using UDP/

    Some anycast deployments have very predictable routing systems, which
    can remain stable for long periods of time (e.g.  anycast within an
    IGP, where node selection changes only occur as a response to node
    failures).  Other deployments have far less predictable
    characteristics (e.g.  a densely-deployed array of nodes across the
    global Internet).

==> you make this sound as if IGPs would have intrinsic characteristics to
make them typically stable compared to BGP.  I'd consider adding:
s/IGP/well-managed IGP, as described in Section 4.4.3,/

    When a service is anycast within an IGP the routing system is
    typically under the control of the same organization who is providing
    the service, and hence the relationship between service transaction
    characteristics and network stability are likely to be relatively
    well-understood.  This technique is consequently applicable to a
    larger number of applications than Internet-wide anycast service
    distribution (see Section 4.1).

==> as above, to provide truth in advertising, this section needs to discuss
or provide pointer to 4.4.3 and some other appropriate sections on why IGPs
may not be better than inter-domain unless they have been designed properly.

   As is described in  having the Anycast Node avoid black-holing

==> missing reference here..

7.  Protocol Considerations

    This document does not impose any protocol considerations.

==> is this section needed?

9  References

==> split to informative/normative.

editorial
---------
Lindqvist & Abley        Expires April 22, 2005                 [Page 1]
Internet-Draft                  Anycast                     October 2004

==> the abbrev should be something a bit more descriptive than plain
"Anycast" ?

  Since it is usually a requirement that a single
    client-server interaction is carried out between a client the same
    server node for the duration of the transaction, it follows that the
    routing system's node selection decision ought to be stable for an
    order of magnitude longer than the expected transaction time, if the
    service is to be provided reliably.

==> s/client the same/client and the same/ ?

    o  A root server service might be distributed throughout the Internet
       with nodes located in regions with poor external connectivity, to
       ensure that the DNS functions adequately within the region during
       times of external network failure.

==> s/root/root DNS/

   well-documented examples of import polices which filter on RIR

==> spell out RIR ?

    AS:es at multiple locations and could therefor propagate the same

==> s/AS:/AS/
==> s/therefor/therefore/

    When Anycast services are deployed across networks operated by
    others, their reachability is dependent on routing polices and
    topology changes (planned and unplanned) which are unpredictable and
    sometimes difficult to identify.

==> only if there were routing polices, but no.. s/police/policies/ :)

    Codependencies are avoided by making each node as autonomous and

==> s/Code/Co-de/ (easier to read because doesn't start as "code") ?

    facilities on the Internet such as RIPE's Routing
    Information Service [14] and the University of
           Oregon Route
    Views Project [15].

==> some screwed up formatting here..

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings
_________________________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/grow.html
web archive:        http://darkwing.uoregon.edu/~llynch/grow/

---

From: Kurt Erik Lindqvist <[email-masked]>
Date: Wed, 17 Nov 2004 00:02:20 +0100

Pekka,

some comments and sorry for the late reply.

> something we should be producing.  A loud 'yes' for the adoption for 

I saw Dave already send that out.

> Obviously, as with all -00 I-Ds, there's a lot which can be improved. 

I am sure! :-)

> substantial

Geoff had the same comment and I sent some suggestion of new text to 
him, please let me know if you think that is sufficient or if you think 
we should add more text.

> That's much simpler operationally than running tunneling across 

I think it is fair that we should point out the risks with the approach 
and the added complexity. I will try and write something up the pros 
and cons with tunneling.

> want to provide this kind of more balanced load balancing.

I am not really sure how you feel ecmp should be emphasized even more? 
It's a full chapter on it's own....

> is discussing *exactly* these problems, and also introduces different

Ok, good point.

> feasible path RPF as described in RFC3704.

The case I had in mind I described in the mail to Geoff. Does that make 
the case clearer? I agree we should update the text to make it more 
clear though.

> written the ACLs might have forgotten the anycast prefix(es) 

I am assuming that if you connect an anycast service, you will also 
accept traffic to those prefixes. If you point gun to foot and pull 
trigger, you will shoot yourself in the foot...

> debug) for those poor bastards who are inappropriately filtering the 

I think adding a tunneling consideration section might be a good idea 
though. Will do.

> differerent sets of nodes.

I see your point, but is that really something for a security 
considerations section?

> semi-editorial

I removed it.

> I think you mean the latter, but it requires rewording to make that 
> clear. I suggest you don't use the plural tense.

I think I will leave this to the native english speaker of the two 
authors :-)

> ==> please also add linebreaks between different definitions ?

I'll see if I can get xml2rfc to do this :-)

> ==> this is too brief, it's not clear what you mean here; please 

Ok, good point.

>    in some cases (the DNS is an example of this).

Ok.

> s/IGP/well-managed IGP, as described in Section 4.4.3,/

ok.

> may not be better than inter-domain unless they have been designed 

Ok.

> ==> missing reference here..

Fixed.

> ==> is this section needed?

Uhm, I thought so.

> ==> split to informative/normative.

Well, removing the normative language (actually only the definitions) 
should have resolved this.

> ==> the abbrev should be something a bit more descriptive than plain

Ok.

> ==> s/client the same/client and the same/ ?

ok.

>       times of external network failure.
Ok.

>   well-documented examples of import polices which filter on RIR

Done.

> ==> s/therefor/therefore/

ok.

> ==> only if there were routing polices, but no.. s/police/policies/ :)

Bummer! :-)

> ==> s/Code/Co-de/ (easier to read because doesn't start as "code") ?

ok.

> ==> some screwed up formatting here..

Fixed.

Thanks!

- - kurtis -

---

From: Danny McPherson <[email-masked]>
Date: Thu, 18 Nov 2004 09:50:10 -0700

I'd like to see this becoming a WG document.  I agree with most
of Geoff & Pekka's comments, here are a few additional/overlapping
ones as well:

3.2 Item 4, I like the new text proposed by Geoff.

3.2 You may also want to mention something here regarding
benefits regarding provisioning considerations in that all
servers may maintain a single visible address, and
therefore no changes are required to add new servers.

In section 4.1 you might wish to complete this statement:

    Some services have very short transaction times, and may even be
    carried out using a single packet request and a single packet reply
    in some cases (the DNS is an example of this).  Other services
    involve far longer-lived transactions (e.g.  bulk file downloads and
    audio-visual media streaming).

By saying "and are therefore not likely candidates for anycasting".
Though I guess you do say this implicitly in the proceeding
text.

In section 4.2 you discuss using this for FTP.  Is that really
a good idea?  I think there should perhaps be some set of
recommendations, and short-lived connectionless protocols
should be the focus.

This draft seems to imply that for locally-scoped anycast
an IGP is employed.  This may or may not be the case, as BGP
is just as likely to be employed for this purpose.

Section 4.4.1 perhaps you're strecthing it a bit with
the tunneling "recommendations"?  I suggest you remove this
altogether.

Secton 4.4.2 "throughout the network" should be "throughout
the Internet".  You should be able to does this confidently
throughout your own network, one would hope.

Also, I'm not sure we want folks advertising lots of more
specifics globally if they really don't need to, so I'd
rather see some reflection of that in this text, and
agree with others that have commented this scares me:

    In such cases it is necessary to propagate a route which covers the
    Service Address, and which has a sufficiently short prefix that it
    will not be caught by commonly-deployed import policies.  In many
    cases this will be a 24-bit prefix, but there are other
    well-documented examples of import polices which filter on RIR
    allocation boundaries, and hence some experimentation may be prudent.

Is this really the case:

    Common implementations of flap dampening penalizes oscillating
    advertisements based on the observed AS_PATH, and not on the NLRI.
    For this reason, network instability which leads to route flapping
    from a single anycast node ought not to cause advertisements from
    other nodes (which have different AS_PATH attributes) to be dampened.

One of the issues with flap dampening is that much of the attribute
state (e.g., AS_PATH) for a given prefix isn't maintained, and
therefore, all paths are penalized, not just the mis-behaving
paths.  You talk about this in following text somewhat, but this
seems to confuse the issue.

Regarding this text:

Perhaps it would be more appropriate to say that consideration
should be given to the effects of strict-mode RPF if deploying
anycast nodes, not that strict-mode should not be used.

Regarding the COMMUNITY recommendations defined in 4.4.6, it's often
a good idea to employ more explicit policies that require presence
of an explicit advertise-this-prefix community value, rather than
advertising based on implicit policy -- it if one of the defined
communities is not there.

Section 4.5 should perhaps mention something regarding the ill-
effects on caching this may have, esp., e.g., with DNS
resolvers.

It might be useful to mention some services that are deployed
quite widely today employing anycast.  E.g., DNS (you obviously
covered), NTP, RADIUS, MSDP (might want a reference here for
Anycast RP), etc..

Also, a word about this not being IPv6 anycast is liklely
warranted here as well.

---

From: Joe Abley <[email-masked]>
Date: Thu, 18 Nov 2004 19:50:28 -0500

Hi Danny,

> therefore no changes are required to add new servers.

That's a useful point, thanks.

> recommendations, and short-lived connectionless protocols

I think it depends on how the anycast nodes are deployed, and what the 
objectives of the service distribution are. Here's an example.

Citylink is the operator of two exchange points in New Zealand (the APE 
and the WIX). They offer near-free hosting for people serving local 
content (or local mirrors of content; there's a debian mirror there 
too); servers are numbered out of 202.7.4.0/22. 202.7.4.0/22 is 
advertised to peers over the APE and WIX, and is not advertised for 
transit by Citylink in New Zealand.

202.7.4.0/22 is also advertised for transit in California from a single 
host, which has dummy ftp and HTTP servers running that provide 
messages which say "Sorry, you need to peer with us in New Zealand to 
access this content." (In fact, the HTTP servers return redirects to 
URLs not hosted on 202.7.4.0 addresses which explain the details, but 
the general principle holds).

The addresses within 202.7.4.0/22 are service addresses; each of those 
services is effectively anycast. It's stable, and works very nicely.

What I tried to do in this section (it's my fault, not Kurtis' :-) was 
describe as many possible issues as I could think of and recommend that 
people do analysis on their own to confirm whether their own 
application/protocol/whatever is suitable for anycast deployment. I 
think it is presumptuous to try to lay down the law about which 
protocols are suitable for anycast deployments and which are not.

However, if there are strong opinions to the contrary, I would 
certainly do my best to accommodate them.

> is just as likely to be employed for this purpose.

I didn't mean to imply that; indeed, in most IGPs it's not possible to 
deploy a local node, since all the participants in the IGP necessarily 
see all the routing information, and not just part of it. Which part of 
the document gave you that impression?

> the tunneling "recommendations"?  I suggest you remove this

I have a bunch of new text to describe the risks and benefits of the 
tunnel approach. There are at least two high-profile anycast 
deployments in the network today which use this approach, and given 
that both of them have had a great deal of success with them, it seems 
wrong not to describe them.

I hope that the number of warning bells and red triangles I dangle 
around the text will make it clear that careful consideration is 
required before that path is followed.

> throughout your own network, one would hope.

Thanks.

> agree with others that have commented this scares me:

I agree, and I inserted a whole new section on this, this afternoon.

> seems to confuse the issue.

People I've talked to at cisco and Juniper tell me that as-paths are 
penalised, not prefixes. I've not seen the source code, though, and I 
haven't carried out any experiments which would allow me to tell. It 
would be great if you could expand on your paragraph, above.

> anycast nodes, not that strict-mode should not be used.

Yes, I added a bunch of extra text following Pekka's comments on this.

> communities is not there.

With F, we set NO_EXPORT on 192.5.5.0/24 (the route which covers 
f.root-servers.net). There are some peers who have legitimate reasons 
to want to propagate the route beyond their AS (to their customers). 
When we are asked to remove NO_EXPORT towards their routers, we instead 
insist that they strip it on their side; this ensures that when the 
session is reconfigured on their router five RIFs down the track by an 
engineer who has no idea what F-root is, the default is that 
propagation is restricted, not that it is advertised to peers and 
transit providers.

Is that an example of the kind of thing you had in mind?

Thanks!

> covered), NTP, RADIUS, MSDP (might want a reference here for

Thanks, I will. A reference to RFC 3068 might not go amiss either, now 
that I think of it.

> Also, a word about this not being IPv6 anycast is liklely

I have a section sitting empty right now, pending text for that.

There are v6 anycast deployments in production today which use 
unicast-scope v6 prefixes to anycast services in precisely the same way 
that v4 anycast is done (e.g. we at ISC advertise 2001:500::/48 in many 
places that we advertise 192.5.5.0/24; 2001:500::1035 is F's v6 
address). So the general approach is practical with v6.

I'm weak on the differences between this and "IPv6 anycast". Is the 
difference simply the addresses used, or are there other things to 
consider?

Joe

---

From: Pekka Savola <[email-masked]>
Date: Fri, 19 Nov 2004 13:12:09 +0200 (EET)

> some comments and sorry for the late reply.

No problem, I reciprocated :).  I haven't commented on the issues we 
seemed to be agreeing about.

> I saw Dave already send that out.

In case it wasn't clear, I was also opting for adopting this document 
now :).

> him, please let me know if you think that is sufficient or if you think

I checked the text and it seems OK.

>> ==> I fear that the BGP ecmp has to become more prominent here. 
>> AFAICS, it's implemented and used especially for stub ASs which, 
>> for some reason, want to provide this kind of more balanced load 
>> balancing.
>

I meant that the issues with ECMP should be emphasized more, in 
particular with edge networks, which try to distribute the load as 
evenly as possible (e.g., over two T1s or T3s or whatever).

Just as a suggestion, maybe one approach here might be rewording this 
a bit like:

    exist.

    However, implementations of BGP exist that support multi-path exit
    selection, and many end-sites are using these techniques to try to
    ensure uniform outbound and inbound traffic balancing.  Analysis of
    the likely incidence of
    such corner cases for particular distributions of Anycast Nodes are
    recommended for services which involve multi-packet transactions.

>>  - the second and third paragraphes are not really accurate.  It 
>> depends on how the preferences are selected.  If you hook up an 
>> anycast node to its first-hop ISP, almost always the ISP will 
>> consider that exit preferable, so strict RPF works.  (Where it may 
>> not work is further upstream, but there strict RPF-like techniques 
>> are not applied in any case..) Even if strict RPF did not work, you 
>> could still use feasible path RPF as described in RFC3704.
>

I checked the text, and I'm not still 100% sure of this case.  Are you 
describing something like:

                            [anycast node]
                                 |
                            [customer ISP/site]
                    peer         | 1)        peer
    [anycast node] ------- [ transit ISP ] ------- [peer ISP] --- [anycast node]
                     2)          | 4)         3)
                             (upstream)

Where have you envisaged that strict uRPF would be used?  1), 2), and 
3)?  Or just at 1)?

I will note that Feasible Path RPF (or its administrative trick 
equivalents) should work in all cases 1)-3): then each border router 
would see the link as preferable, but that particular preference might 
not necessarily be propagated to other routers w/ iBGP.

> trigger, you will shoot yourself in the foot...

Agree, of course.  From above, I'm not sure that strict uRPF is 
typically a problem especially if you are able to do some tricks with 
it (like assign a weight for the prefix in the ingress policy, or use 
feasible uRPF).. but rather those folks who don't have any 
'automation' at all.

>> ==> this section probably needs to discuss, with a short paragraph, 
>> issues relating to availability.  A badly implemented anycast 
>> solution has a good chance to decrease the availability rather than 
>> increase it. As any anycast node is capable of blackholing its 
>> addresses/services to achieve the same service.  E.g., there should 
>> be a couple of addresses in DNS for service X, which would be 
>> anycasted by differerent sets of nodes.
>

Availability is one important aspect of security, so if this 
discussion doesn't go here, it should go into a separate subsection 
and be referred to from here.

---

From: Danny McPherson <[email-masked]>
Date: Sun, 21 Nov 2004 09:07:43 -0700

On Nov 18, 2004, at 5:50 PM, Joe Abley wrote:

> I think it is presumptuous to try to lay down the law about which 

OK, I understand this.  You may want to say something to that
effect explicitly in the document.

> see all the routing information, and not just part of it. Which part 
> of the document gave you that impression?

The entire document, in general.  it may have just been my
reading, but..

> would be great if you could expand on your paragraph, above.

Have a look at the recent discussion on "Future of
route flap dampening spec" on the IDR mailing list for
some discussion of this.  I don't believe any of the
widely-deployed implementations use the AS_PATH as part
of the damping key.  There are arguments both for
and against doing this.  This applies to other attributes
as well, as you might imagine.

> When we are asked to remove NO_EXPORT towards their routers, we 
> instead insist that they strip it on their side; this ensures that 
> when the session is reconfigured on their router five RIFs down the 
> track by an engineer who has no idea what F-root is, the default is 
> that propagation is restricted, not that it is advertised to peers and 
> transit providers.
>

I was more concerned with the fact that if some event occurs that
results in a more-specific route not being tagged with the appropriate
community (e.g., NO_EXPORT) that it's NOT advertised implicitly.  For
example, assume you set the NO_EXPORT community by applying a
route-map to a redistribution filter and reference a route filter/prefix
list to match and set on.  Now, assume you're updating the
prefix list when the redistribution process runs and one or more
of the prefixes hasn't been added to the rewritten list yet.  When
the more-specific is redistributed into BGP it doesn't match the
NO_EXPORT clause of the route-map and is advertised without the
community, and subsequently leaked by your peering routers to
other autonomous systems - then withdrawn in some subsequent
iteration of the redistribution process per the policy list now
contains the prefix in question.

Or consider a more simple example: One of the iBGP routers
within a domain is configured not to propagate BGP communities.

In either of these examples, had you instead written a policy
at the edge of the network that said community XYZ MUST be
present for the route to be announced, they wouldn't have been
leaked inadvertently.  I.e., explicit presence is required.

Make sense?

> unicast-scope v6 prefixes to anycast services in precisely the same 
> way that v4 anycast is done (e.g. we at ISC advertise 2001:500::/48 in 
> many places that we advertise 192.5.5.0/24; 2001:500::1035 is F's v6 

There's a good bit of discussion of this in RFC 2373 and some of
the related subsequent documents.  I can't really think of anything that
would be unique to IPv4 beyond the "Required Anycast Addresses" stuff,
so I'll defer on this one :-)

-danny