Subject: Re: draft-ietf-v6ops-addr-select-ps-02

From: Florian Weimer <[email-masked]>
Date: Sun, 27 Jan 2008 12:26:46 +0100

I've also noticed that the draft fails to mention that Rule 9, when
applied by most of the client population on the Internet, results in
worse performance than no destination address sorting at all.

Rule 9 might offer a significant advantage in private deployments which
use non-IANA address allocation.  In other cases, routing is typically
not hierarchical at all, so that the length of the matching address
prefix is meaningless.

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 28 Jan 2008 09:39:21 +1300

> not hierarchical at all, so that the length of the matching address

Not true if the sites concerned are both using PA space and
happen to have the same ISP. We still have hopes that PA space will
be predominant in IPv6, despite the PI heresy.

Can you describe cases in which this rule is actively harmful?

   Brian

---

From: Florian Weimer <[email-masked]>
Date: Sun, 27 Jan 2008 22:00:19 +0100

* Brian E. Carpenter:

> happen to have the same ISP.

Only if they are within the same PA prefix.  And even then, it's not
clear if this is the best choice for the client (or the server,
off-loading to other networks is no longer possible).

> We still have hopes that PA space will be predominant in IPv6, despite
> the PI heresy.

There's too little experience with IPv6 to be sure.  But I wouldn't be
surprised with round-robin is superior there, too.  With round-robin,
it's possible to add more instances, and they take a share of the load.

And it's a further obstacle to renumbering -- what do you do if the new
prefix has got an unfortunate bit pattern, shifting the loads in
unwanted way?

> Can you describe cases in which this rule is actively harmful?

pool.ntp.org and some Debian mirrors experienced rather asymmetric
traffic loads after getaddrinfo() with Rule 9 support had been rolled
out.

---

From: Rémi Denis-Courmont <[email-masked]>
Date: Sun, 27 Jan 2008 23:21:06 +0200

> Can you describe cases in which this rule is actively harmful?

http://bugs.debian.org/438179 : Rule 9 defeats round-robin DNS.

The practical problem was with IPv4 address space, but as noted, there is no 
reason why it would not apply to IPv6 as well as it gets deployed.

Debian did end up overriding/ignored rule 9 by default.

-- 
Rémi Denis-Courmont
http://www.remlab.net/

---

From: Florian Weimer <[email-masked]>
Date: Sun, 27 Jan 2008 22:31:39 +0100

> Debian did end up overriding/ignored rule 9 by default.

I think this change has itself been overridden, so Rule 9 is active for
IPv4 address selection again.  (These changes have not affected Debian's
stable distribution, Rule 9 has been in use since Debian 4.0/etch.)

---

From: Fred Baker <[email-masked]>
Date: Mon, 28 Jan 2008 00:27:41 -0800

Brian, I'm very perlexed by how you come to that conclusion. Maybe  
you can fill me in.

The text of rule 9 is:

    Rule 9:  Use longest matching prefix.
    When DA and DB belong to the same address family (both are IPv6 or
    both are IPv4): If CommonPrefixLen(DA, Source(DA)) >
    CommonPrefixLen(DB, Source(DB)), then prefer DA.  Similarly, if
    CommonPrefixLen(DA, Source(DA)) < CommonPrefixLen(DB, Source(DB)),
    then prefer DB.

In the Internet, regardless of whether the prefixes are PA or PI,  
there is little if any reason to believe that any two customers  
chosen at random have the same provider. I have appended four  
traceroutes for your (anecdotal) consideration in support of the  
point - I am served by Cox, but you use google mail (which apparently  
has multiple ISPs), Florian is reached through interscholz.net,  
ops.ietf.org is reached through NTT, and ietf.org is reached through  
AT&T. I would encourage you to spend an hour picking random domain  
names out of your mail spool and traceroute to them. If your view of  
the world is at all similar to mine, you will find your path going  
through your network, your provider, their provider, and their  
network, and potentially multiple networks in between the two providers.

If the two communicating networks have different upstream providers,  
what on God's Green Earth leads you to believe that they would have a  
common PA prefix? If they don't have a common prefix, exactly how  
does rule 9 help?

> Can you describe cases in which this rule is actively harmful?

[Fred-Bakers-Computer-3:~] fred% traceroute mail.google.com
traceroute: Warning: mail.google.com has multiple addresses; using  
209.85.199.18
traceroute to googlemail.l.google.com (209.85.199.18), 64 hops max,  
40 byte packets
  1  10.32.244.217 (10.32.244.217)  1.570 ms  1.153 ms  1.104 ms
  2  wsip-98-173-193-1.sb.sd.cox.net (98.173.193.1)  87.274 ms   
49.568 ms  48.215 ms
  3  68.6.13.101 (68.6.13.101)  57.624 ms  47.359 ms  38.188 ms
  4  68.6.13.45 (68.6.13.45)  33.014 ms  32.715 ms  32.097 ms
  5  paltbbrj02-ae0.0.r2.pt.cox.net (68.1.0.235)  51.490 ms  52.489  
ms  41.434 ms
  6  209.85.130.2 (209.85.130.2)  125.613 ms  122.277 ms 209.85.130.6  
(209.85.130.6)  129.134 ms
  7  66.249.95.135 (66.249.95.135)  110.462 ms  124.537 ms  116.237 ms
  8  209.85.250.144 (209.85.250.144)  128.569 ms  130.422 ms  134.411 ms
  9  64.233.174.131 (64.233.174.131)  145.547 ms 64.233.174.129  
(64.233.174.129)  132.366 ms  118.576 ms
10  * 209.85.248.254 (209.85.248.254)  125.765 ms  138.378 ms
11  rv-in-f18.google.com (209.85.199.18)  118.186 ms  109.112 ms *
[Fred-Bakers-Computer-3:~] fred% traceroute
[Fred-Bakers-Computer-3:~] fred% nslookup 209.85.130.2
Server:         68.105.28.13
Address:        68.105.28.13#53

Non-authoritative answer:
*** Can't find 2.130.85.209.in-addr.arpa.: No answer

Authoritative answers can be found from:

[Fred-Bakers-Computer-3:~] fred% traceroute deneb.enyo.de
traceroute to deneb.enyo.de (212.9.189.171), 64 hops max, 40 byte  
packets
  1  10.32.244.217 (10.32.244.217)  1.528 ms  1.131 ms  1.126 ms
  2  wsip-98-173-193-1.sb.sd.cox.net (98.173.193.1)  12.350 ms   
10.628 ms  10.509 ms
  3  68.6.13.101 (68.6.13.101)  14.043 ms  13.115 ms  12.642 ms
  4  68.6.13.45 (68.6.13.45)  18.518 ms  17.599 ms  16.974 ms
  5  paltbbrj02-ae0.0.r2.pt.cox.net (68.1.0.235)  33.845 ms  31.246  
ms  31.796 ms
  6  so-1-2-2.mpr2.sjc2.us.above.net (64.125.29.126)  31.804 ms   
32.963 ms  29.991 ms
  7  so-1-0-0.mpr1.lga5.us.above.net (64.125.26.142)  106.799 ms   
104.833 ms  106.786 ms
  8  so-0-0-0.mpr1.lga5.us.above.net (64.125.27.237)  107.908 ms   
109.646 ms  107.201 ms
  9  so-1-0-0.mpr1.ams1.nl.above.net (64.125.27.186)  192.188 ms   
190.569 ms  192.356 ms
10  pos-9-1.mpr2.fra1.de.above.net (64.125.23.253)  206.446 ms   
201.130 ms  201.177 ms
11  ge-9-7.er2b.fra1.de.above.net (64.125.23.210)  200.881 ms  
ge-9-4.er2b.fra1.de.above.net (64.125.23.206)  199.369 ms  
ge-9-7.er2b.fra1.de.above.net (64.125.23.210)  202.015 ms
12  c12gsr-5-02.intx.fra.interscholz.net (62.93.193.62)  201.813 ms   
208.735 ms  198.146 ms
13  c12gsr-3-02.z10a.stg.interscholz.net (85.236.200.229)  205.121  
ms  207.205 ms  204.092 ms
14  ge-knd-lfnet.z10.stg.interscholz.net (85.236.201.130)  204.035  
ms  201.747 ms  209.954 ms
15  dsl-gw.ispeg.de (212.9.161.26)  202.581 ms  201.286 ms  200.045 ms
16  dsl.enyo.de (213.178.172.64)  227.865 ms  237.674 ms  228.145 ms
17  * *^C
[Fred-Bakers-Computer-3:~] fred% traceroute ops.ietf.org
traceroute to ops.ietf.org (147.28.0.62), 64 hops max, 40 byte packets
  1  10.32.244.217 (10.32.244.217)  1.320 ms  1.063 ms  1.030 ms
  2  wsip-98-173-193-1.sb.sd.cox.net (98.173.193.1)  15.270 ms   
14.536 ms  12.443 ms
  3  68.6.13.117 (68.6.13.117)  17.136 ms  13.310 ms  13.317 ms
  4  68.6.13.49 (68.6.13.49)  17.679 ms  22.767 ms  19.230 ms
  5  paltbbrj01-so000.0.r2.pt.cox.net (68.1.0.199)  43.538 ms  41.442  
ms  43.887 ms
  6  ae-1.r20.snjsca04.us.bb.gin.ntt.net (129.250.5.33)  48.245 ms   
42.201 ms  47.455 ms
  7  p64-0-0-0.r20.sttlwa01.us.bb.gin.ntt.net (129.250.4.22)  72.543  
ms  66.362 ms  81.955 ms
  8  p16-0-0-0.r05.sttlwa01.us.bb.gin.ntt.net (129.250.3.11)  66.021  
ms  72.685 ms  65.238 ms
  9  p1-0.psg.sttlwa01.us.bb.gin.ntt.net (129.250.11.42)  69.071 ms   
64.404 ms  64.297 ms
10  * *^C
[Fred-Bakers-Computer-3:~] fred% traceroute ietf.org
traceroute: Warning: ietf.org has multiple addresses; using  
209.173.53.180
traceroute to ietf.org (209.173.53.180), 64 hops max, 40 byte packets
  1  10.32.244.217 (10.32.244.217)  1.581 ms  1.118 ms  1.095 ms
  2  wsip-98-173-193-1.sb.sd.cox.net (98.173.193.1)  18.068 ms   
12.854 ms  14.602 ms
  3  68.6.13.101 (68.6.13.101)  12.794 ms  11.780 ms  14.710 ms
  4  68.6.13.45 (68.6.13.45)  20.709 ms  28.882 ms  27.386 ms
  5  fed1dsrj02-so010.0.rd.sd.cox.net (68.6.8.13)  18.876 ms  17.096  
ms  22.599 ms
  6  tbr1.la2ca.ip.att.net (12.122.104.38)  79.486 ms  79.698 ms   
79.613 ms
  7  tbr2.la2ca.ip.att.net (12.122.9.146)  80.148 ms  80.635 ms   
84.841 ms
  8  tbr2.sl9mo.ip.att.net (12.122.10.13)  82.377 ms  82.078 ms   
96.295 ms
  9  tbr1.sl9mo.ip.att.net (12.122.9.141)  91.041 ms  81.589 ms   
78.615 ms
10  tbr1.wswdc.ip.att.net (12.122.10.29)  80.846 ms  76.828 ms   
78.812 ms
11  12.123.8.193 (12.123.8.193)  91.766 ms  77.840 ms  77.159 ms
12  12.118.28.10 (12.118.28.10)  79.630 ms  82.614 ms  79.309 ms

---

From: Arifumi Matsumoto <[email-masked]>
Date: Mon, 28 Jan 2008 18:25:53 +0900

Florian,
thank you for pointing out an interesting issue.

This problem can be thought to be conflict of local (client-side) optimization
and remote (server-side) optimization. Dst-rule 9 tries to implement local 
optimization of subsequent communication and breaks DNS round-robin technique
which is optimization of the remote side traffic.

In my solution draft, which was approved as 6man wg item at Vancouver, some
proposed mechanisms for achieving/compensating local optimization are described
and analyzed, such as address policy distrbituion mechanism.

My suggestion for solving this problem is to change default address selection
algorithm to support remote optimization, namely invalidate dst-rule 9, and to
adopt other mechanism for local optimization if necessary.

As everyone mentioned, poor way of local optimization does harm for remote side
and sometimes for local side. Any kind of local optimization should be implemented
carefully by an administrator that figures out what size of address block his
site has, which address block the upstream network has, and what kind of address
selection policy doesn't conflict with DNS round-robin like remote optimization.

Moreover, local optimization is easier to be deployed than remote optimization is.
To implement remote optimization, almost all the nodes that connect to his server
have to support the mechanism for remote optimization. Changing the widely-used
standard document itself seems for me to be the best and possible way to go,
rather than standardizing an additional mechanism for remote optimization and 
convincing every user-OS vendor to implement it.

Best regards,

-- 
Arifumi Matsumoto
    IP Technology Expert Team
    Secure Communication Project
    NTT Information Sharing Platform Laboratories
    E-mail: [email-masked]

---

From: Florian Weimer <[email-masked]>
Date: Mon, 28 Jan 2008 10:44:05 +0100

* Arifumi Matsumoto:

> This problem can be thought to be conflict of local (client-side)
> optimization and remote (server-side) optimization. Dst-rule 9 tries
> to implement local optimization of subsequent communication and breaks
> DNS round-robin technique which is optimization of the remote side
> traffic.
>
> In my solution draft, which was approved as 6man wg item at Vancouver,
> some proposed mechanisms for achieving/compensating local optimization
> are described and analyzed, such as address policy distrbituion
> mechanism.

Does this mean that the 6man WG is responsible for RFC 3484bis?  I had
missed this.  And why is v6ops dealing with
draft-ietf-v6ops-addr-select-ps-02?  Wouldn't this fit 6man better, too?

> adopt other mechanism for local optimization if necessary.

I'm in favor of disabling Rule 9 for IPv4.  If IPv6 is finally deployed,
I guess the prefix distribution will resemble that of IPv4, so Rule 9
isn't that helpful over there, either.  But this is mere speculation.

> As everyone mentioned, poor way of local optimization does harm for
> remote side and sometimes for local side. Any kind of local
> optimization should be implemented carefully by an administrator that
> figures out what size of address block his site has, which address
> block the upstream network has, and what kind of address selection
> policy doesn't conflict with DNS round-robin like remote optimization.

To some extent, it's also possible to put that optimization into the
caching resolver (which might get better topology data via BGP, for
instance).  If the client reorders addresses, this doesn't work anymore.

---

From: Arifumi Matsumoto <[email-masked]>
Date: Mon, 28 Jan 2008 19:47:57 +0900

Florian,

> draft-ietf-v6ops-addr-select-ps-02?  Wouldn't this fit 6man better, too?

Not exactly for RFC 3484bis, but for solving problems related to address
selection described in draft-ietf-v6ops-addr-select-ps-02.

About 2nd question,
we started this work at v6ops because v6ops are motivated for this.
Solution draft moved to 6man mainly because it involves some kind of
protocol work and we cannot do that at operation area.

> isn't that helpful over there, either.  But this is mere speculation.

Now that we have IPv6 PI, I agree that we should expect such a situation.
I'm afraid that leaving another difference between IPv4 and IPv6 spec is
gonna be another obstacle to IPv6 transition. So, disabling Rule 9 only
for IPv4 seems to me not a good idea, as far as IPv6 world doesn't
drastically change from IPv4.

> instance).  If the client reorders addresses, this doesn't work anymore.

If the rule 9 is disabled, that might be another mechanism for local
optimization. However, destination address sorting without knowledge
of source addresses that a querying host has is not helpful in some
cases.

Best regards,

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 29 Jan 2008 10:00:16 +1300

On 2008-01-28 21:27, Fred Baker wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Brian, I'm very perlexed by how you come to that conclusion. Maybe you
> can fill me in.
> 
> The text of rule 9 is:
> 
> In the Internet, regardless of whether the prefixes are PA or PI, there
> is little if any reason to believe that any two customers chosen at
> random have the same provider. 

Of course - but there are circumstances where "at random" doesn't apply.
One case is  that inside an enterprise network that operates more
than one prefix, topology is likely to follow addressing. Another
example: in a geographical area with one dominant provider,
local-to-local traffic isn't quite random. Similarly, thinking longer
term, if any kind of municipal IXP starts handing out prefixes,
local-to-local traffic isn't quite random. For non-local traffic,
the rule *is* effectively a random choice. So as a rule quite far
down in the list, this seems more likely to do good than harm. That's
all I'm saying.

> I have appended four traceroutes for your
> (anecdotal) consideration in support of the point - I am served by Cox,
> but you use google mail (which apparently has multiple ISPs), Florian is
> reached through interscholz.net, ops.ietf.org is reached through NTT,
> and ietf.org is reached through AT&T. I would encourage you to spend an
> hour picking random domain names out of your mail spool and traceroute
> to them. If your view of the world is at all similar to mine, you will
> find your path going through your network, your provider, their
> provider, and their network, and potentially multiple networks in
> between the two providers.

That is certainly true.

> common PA prefix? If they don't have a common prefix, exactly how does
> rule 9 help?

In no way, of course.

    Brian
> traceroute to googlemail.l.google.com (209.85.199.18), 64 hops max, 40
> byte packets
>  2  wsip-98-173-193-1.sb.sd.cox.net (98.173.193.1)  87.274 ms  49.568
> ms  48.215 ms
>  5  paltbbrj02-ae0.0.r2.pt.cox.net (68.1.0.235)  51.490 ms  52.489 ms 
> 41.434 ms
> traceroute to deneb.enyo.de (212.9.189.171), 64 hops max, 40 byte packets
>  2  wsip-98-173-193-1.sb.sd.cox.net (98.173.193.1)  12.350 ms  10.628
> ms  10.509 ms
>  5  paltbbrj02-ae0.0.r2.pt.cox.net (68.1.0.235)  33.845 ms  31.246 ms 
> 31.796 ms
>  6  so-1-2-2.mpr2.sjc2.us.above.net (64.125.29.126)  31.804 ms  32.963
> ms  29.991 ms
>  7  so-1-0-0.mpr1.lga5.us.above.net (64.125.26.142)  106.799 ms  104.833
> ms  106.786 ms
>  8  so-0-0-0.mpr1.lga5.us.above.net (64.125.27.237)  107.908 ms  109.646
> ms  107.201 ms
>  9  so-1-0-0.mpr1.ams1.nl.above.net (64.125.27.186)  192.188 ms  190.569
> ms  192.356 ms
> 10  pos-9-1.mpr2.fra1.de.above.net (64.125.23.253)  206.446 ms  201.130
> ms  201.177 ms
> 13  c12gsr-3-02.z10a.stg.interscholz.net (85.236.200.229)  205.121 ms 
> 207.205 ms  204.092 ms
> 14  ge-knd-lfnet.z10.stg.interscholz.net (85.236.201.130)  204.035 ms 
> 201.747 ms  209.954 ms
>  2  wsip-98-173-193-1.sb.sd.cox.net (98.173.193.1)  15.270 ms  14.536
> ms  12.443 ms
>  5  paltbbrj01-so000.0.r2.pt.cox.net (68.1.0.199)  43.538 ms  41.442 ms 
> 43.887 ms
>  7  p64-0-0-0.r20.sttlwa01.us.bb.gin.ntt.net (129.250.4.22)  72.543 ms 
> 66.362 ms  81.955 ms
>  8  p16-0-0-0.r05.sttlwa01.us.bb.gin.ntt.net (129.250.3.11)  66.021 ms 
> 72.685 ms  65.238 ms
> traceroute: Warning: ietf.org has multiple addresses; using 209.173.53.180
>  2  wsip-98-173-193-1.sb.sd.cox.net (98.173.193.1)  18.068 ms  12.854
> ms  14.602 ms
>  5  fed1dsrj02-so010.0.rd.sd.cox.net (68.6.8.13)  18.876 ms  17.096 ms 
> 22.599 ms
>  6  tbr1.la2ca.ip.att.net (12.122.104.38)  79.486 ms  79.698 ms  79.613 ms
>  7  tbr2.la2ca.ip.att.net (12.122.9.146)  80.148 ms  80.635 ms  84.841 ms
>  8  tbr2.sl9mo.ip.att.net (12.122.10.13)  82.377 ms  82.078 ms  96.295 ms
>  9  tbr1.sl9mo.ip.att.net (12.122.9.141)  91.041 ms  81.589 ms  78.615 ms
> 10  tbr1.wswdc.ip.att.net (12.122.10.29)  80.846 ms  76.828 ms  78.812 ms
> -----BEGIN PGP SIGNATURE-----
> 
> iD8DBQFHnZH+bjEdbHIsm0MRAozqAKCCXuLKmV+8JiXtUCpxnKOITCxnVgCfYtud
> i89+S4Fxu8roS5m85TwYw7o=
> =7HEA
> -----END PGP SIGNATURE-----
>