Subject: Re: [dnsop] Re: Root Anycast (fwd)

From: Iljitsch van Beijnum <[email-masked]>
Date: Fri, 1 Oct 2004 10:49:29 +0200

On 1-okt-04, at 2:48, Dean Anderson wrote:

>> Note though that it's *very* hard to create a setup where packets are
>> delivered to different multicast instances, as it's hard to imagine 
>> how
>> any real-world anycast setup could match the criteria in

> Its quite easy for anycast: (real names used, but not real 
> relationships)

>                 Av8
>               /   \
>           sprint   att
>               \   /
>               F-root

> If Av8 turns on PPLB, traffic to F-root will go through both sprint and
> att on a per-packet basis.

No, that won't happen because in order for BGP to install multiple 
paths in the routing table, the following conditions must be met:

"To be candidates for multipath, paths to the same destination need to 
have the following characteristics equal to the best path's 
characteristics:

[...]

• 	One of the following:
	◦ 	Neighboring AS or sub-AS (before the  eiBGP  Multipath feature was 
added)
	◦ 	AS-PATH (after the  eiBGP  Multipath feature was  added)"

(From the URL I cited in my previous message.)

In this case either of the sub-conditions won't be met.

Per packet load balancing is inappropriate in cases where the entire 
path is different, as the limitations of the inferior path are imposed 
on the full set. BGP has enough knobs to distribute the load without 
this anyway. FIB-based load balancing is only appropriate when a small 
part of a single path is populated with more than one physical link. 
And in most of those cases, load balancing mechanisms that make sure 
one flow runs over one link are more desirable. Only in cases where 
this leads to a significantly unbalanced traffic distribution, per 
packet load balancing is a good idea. (But even then most people don't 
enable it as they are (too, IMO) afraid of packet reordering.)

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: Dean Anderson <[email-masked]>
Date: Fri, 1 Oct 2004 15:02:20 -0400 (EDT)

> paths in the routing table, the following conditions must be met:

If you use bgp multipath only, that is true. If you turn on PPLB, as 
described in my URL, then that isn't true.

-- 
Av8 Internet   Prepared to pay a premium for better service?
www.av8.net         faster, more reliable, better service
617 344 9000

---

From: "Joe Shen" <[email-masked]>
Date: Sat, 2 Oct 2004 19:45:45 +0800

Is there situation that multiple root servers installed behine multiple routers within one AS?
e.g. 
                                                               / ------ Router 1 ---- Root Server-F-1
       AV8-----  ASBR-------Router-P  /
                                                             \
                                                               \  ----- Router 2 ---- Root Server - F-2

If router-P enables PPLB, would there be some problem with TCP based DNS requests? 

Joe

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Iljitsch van Beijnum
Sent: Friday, October 01, 2004 4:49 PM
To: Dean Anderson
Cc: [email-masked]; [email-masked]
Subject: Re: [dnsop] Re: Root Anycast (fwd)

On 1-okt-04, at 2:48, Dean Anderson wrote:

>> how any real-world anycast setup could match the criteria in

> If Av8 turns on PPLB, traffic to F-root will go through both sprint 
> and att on a per-packet basis.

[...]

• 	One of the following:

.
dnsop resources:_____________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/dnsop.html
mhonarc archive: http://darkwing.uoregon.edu/~llynch/dnsop/index.html

---

From: "Joe Shen" <[email-masked]>
Date: Sat, 2 Oct 2004 19:48:40 +0800

>described in my URL, then that isn't true.

Yup,  I think  PPLB  proper only in situation that multiple physical
links exist between two routers. 
And, PPLB is not good for ISPs who tries balancing traffic between
peers.

Joe

.

---

From: Paul Vixie <[email-masked]>
Date: Sat, 02 Oct 2004 16:25:36 +0000

> multiple routers within one AS?

yes.  that situation exists inside cogent, with c-root.

> If router-P enables PPLB, would there be some problem with TCP based

your diagram didn't make sense to me so i'll answer without reference to
"router-p".  if cogent's backbone engineering staff enabled PPLB on the
wrong set of output interfaces, all kinds of things would break, including
tcp sessions to c-root.  fortunately, their backbone engineers are smart
enough to know how to use (or not use) the tools available to them.

the rootops were pretty careful when we turned on anycast.  presumably
the other anycast services around the net, like woody's and rodney's, were
also deployed very carefully.  careful as in getting multiple experts in
a room to argue out the fine points.  careful as in monitoring the results
and making sure there weren't any unreported (or reported) failures.

anycast has worked very well.  both inter-AS and intra-AS.  the fact that
a not-clueful-enough engineer *could* build a non-working topology using
anycast and PPLB as ingredients, does not mean that anycast or PPLB are
bad.  it means you have to be clueful-enough before you use either tool.
(and remember kids, all power tools can kill.)

one hopes that an actual policy maker would find an actual expert for advice.
such an expert would be expected to have read

http://www.cisco.com/univercd/cc/td/doc/product/software/ios120/120newft/120limit/120s/120s21/pplb.htm

where it says

      Restrictions

      Out-of-Sequence Packets

      Using per-packet load balancing to share the traffic load across
      available paths to a given destination can cause out-of-sequence
      packets in a particular data flow. This can result in
      unsatisfactory data transmission for video and voice streaming.

and they would know that PPLB is basically a link bundling technology used
when all members of the PPLB group start and end in the same router-pair;
in other words it could mostly turn a pair of OC3c's into an "OC6" but it
would be unsafe in any broader context, even when anycast is not in use.

now, could y'all please stop feeding the trolls?

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Mon, 4 Oct 2004 23:01:41 +0200

On 2-okt-04, at 18:25, Paul Vixie wrote:

> nycast has worked very well.  both inter-AS and intra-AS.  the fact 
> that

It's not as simple as that. It's possible for bad things to happen if:

1. some DNS server is anycast (TLD servers are worse than roots because 
the root zone is so small)
2. fragmented UDP packets or TCP are used as a transport
3. a network is built such that packets entering it through router X 
may prefer a different external link towards a certain destination than 
packet entering it through router Y
4. a customer of this network is connected to two different routers
5. the customer enables per packet load balancing

All of these steps happen in the real world, and are in and of 
themselves not examples of bad engineering. However, the end result can 
be reduced connectivity to one or more anycasted DNS servers under some 
circumstances.

(See my message to dnsop from yesterday 
http://darkwing.uoregon.edu/~llynch/dnsop/msg03105.html for more info, 
reformat using a non proportional font if necessary.)

Now the question is: how do we deal with this? I don't think removing 
anycast wholesale makes sense and/or is feasible. Same thing for 
declaring per packet load balancing an evil practice. A better solution 
would be to give network operators something that enables them to make 
sure load balancing doesn't happen for anycasted destinations. A good 
way to do this would be having an "anycast" or "don't load balance" 
community in BGP, or publication of a list of ASes and/or prefixes that 
shouldn't be load balanced because the destinations are anycast.

> when all members of the PPLB group start and end in the same 

It doesn't make much sense to have multiple links terminate on the same 
router on both ends as then both these routers become single points of 
failure. Often, the end sending out most traffic will have the links 
terminate on one router (so load balancing is possible) while the other 
ends of the links terminate on two or more routers.