Subject: ping-pong phenomenon with p2p links & /127 prefixes

From: Fernando Gont <[email-masked]>
Date: Mon, 16 Aug 2010 05:08:01 -0300

Folks,

draft-ietf-ipngwg-p2p-pingpong-00.txt proposes a solution to the
ping-pong problem with point-to-point links, which IMHO is elegant:

> Check the incoming/outgoing interface of the packet.  If the
> interface is the same, is a point-to-point interface and the
> destination address on the packet seems to be on-link (in terms of
> Neighbor Discovery) on the point-to-point interface, the forwarding
> router SHOULD NOT forward the packet.  Also, in this case, the router
> SHOULD NOT generate ICMPv6 redirect message even if the incoming
> packet meets conditions in RFC2461 section 8.2.  The router SHOULD
> generate an ICMPv6 error message instead, with the type field being 1
> (destination unreachable), and the code field being 3 (address
> unreachable).

Then incorporated into RFC 4443 as follows:

>    One specific case in which a Destination Unreachable message is sent
>    with a code 3 is in response to a packet received by a router from a
>    point-to-point link, destined to an address within a subnet assigned
>    to that same link (other than one of the receiving router's own
>    addresses).  In such a case, the packet MUST NOT be forwarded back
>    onto the arrival link.

However, this fix allegedly has big performance implications on routers.
Can anybody comment on this "claim"?

P.S.: This fix doesn't prevent the use of /127s (it's orthogonal), but
I'm wondering about the reasons for which this fix is not the "first
line of defense" for *this* (i.e., ping-pong) vulnerability. -- yes, the
Kohno et al I-D mentions other (additional) reasons for using /127
prefixes o p2p links.

Thanks!

Kind regards,
-- 
Fernando Gont
e-mail: [email-masked] || [email-masked]
PGP Fingerprint: 7809 84F5 322E 45C7 F1C9 3945 96EE A9EF D076 FFF1

---

From: Jeroen Massar <[email-masked]>
Date: Mon, 16 Aug 2010 10:57:40 +0200

On 2010-08-16 10:08, Fernando Gont wrote:
[..]
> P.S.: This fix doesn't prevent the use of /127s (it's orthogonal),

Unless you configure two /128's pointing to the remote side, which will
then thus not be 'on-link for neighbor discovery, the little thing
called the subnet anycast address will make sure that a /127 ptp simply
does not work, unless you have a platform which disables the subnet
anycast address of course.

Greets,
 Jeroen

... who is still wondering why people try to bother with anything not
/64, and how many links they need in their networks. If you are going to
size them a /127, even out of a /64 then you can do 2^63 =
9.223.372.036.854.775.808 /127's which is a bit insane and you will
never need that either as it won't ever fit in any routing table ;)

---

From: [email-masked]
Date: Mon, 16 Aug 2010 11:12:24 +0200 (CEST)

> anycast address of course.

It would seem disabling the subnet anycast is fairly widespread, then.
I have verified the use of /127 on several hardware forwarding platforms
from Cisco and Juniper. /127 works just fine, and prevents the ping-pong.

[One concrete example where /127 works: Juniper T1600 talking to Cisco
CRS-1 on an OC-768/STM-256 link.]

Steinar Haug, Nethelp consulting, [email-masked]

---

From: Jeroen Massar <[email-masked]>
Date: Mon, 16 Aug 2010 11:35:47 +0200

> CRS-1 on an OC-768/STM-256 link.]

It is quite wide-spread indeed, and for instance Linux used to do it
also until a kernel update in 2003 from 2.4.20 -> 2.4.21 and they
finally implemented subnet anycast support(*) and suddenly it all
started breaking as for IPng.nl at the time we used /127's and everybody
with a Linux endpoint who did an upgrade of their kernels suddenly had a
mysterious broken configuration.

Thus, do ask Cisco and Juniper and other vendors where this now 'works'
if this intentional, or if they might finally comply to the IPv6
specifications one day, as then you might better watch out for this as
it will break your network. For the vendors that have it, it might maybe
be an idea to have a 'disable subnetanycast' command or similar so that
one can explicitly mark a prefix that way.

Greets,
 Jeroen

* = http://www.linux-ipv6.org/ml/usagi-users/msg02430.html

---

From: [email-masked]
Date: Mon, 16 Aug 2010 11:41:10 +0200 (CEST)

> one can explicitly mark a prefix that way.

I have no plans to ask Cisco and Juniper about this. I want /127 to
continue working, and couldn't care less about subnet anycast for my
core routers.

---

From: Jeroen Massar <[email-masked]>
Date: Mon, 16 Aug 2010 11:43:54 +0200

> continue working, and couldn't care less about subnet anycast for my

I think you miss my point: they might finally comply with the specs one
day (if you ask or not, others might) and you will have forgotten about
this little subtle problem and upgrade your routers and voila your
network is broken.

If I where you, or anybody else who is using this 'feature' I would be
wary of it and make a big big note for the test-lab to test this before
shoving new versions to the live network.

Greets,
 Jeroen

---

From: Randy Bush <[email-masked]>
Date: Mon, 16 Aug 2010 18:46:18 +0900

> this little subtle problem and upgrade your routers and voila your

then you will join us supporting the /127 document and it won't be a
problem, will it.

randy

---

From: Ole Troan <[email-masked]>
Date: Mon, 16 Aug 2010 11:48:41 +0200

Jeroen,

> one can explicitly mark a prefix that way.

it is intentional.
there is a command to enable support for subnet-router anycast if use of that is desired.

is there _any_ operational experience with the use of the subnet router anycast address?
asking the question another way. is it still a good idea, or was it ever?

(for 6rd we have explicitly required support of it, for discovery of the real BR unicast address, since we're using IPv4 anycast).

cheers,
Ole

---

From: Jeroen Massar <[email-masked]>
Date: Mon, 16 Aug 2010 11:57:28 +0200

[two replies in once before I truly fill up every one's mailboxes ;) ]

> then you will join us supporting the /127 document and it won't be a

When it is changed that way, indeed it won't be a problem any more as
that is then the standard and people can't be bitten by it anymore.

The big 'problem' I have with it that it is yet-another-special case.
Special cases should be kept to a minimum where possible.

On 2010-08-16 11:48, Ole Troan wrote:
[..]
> there is a command to enable support for subnet-router anycast if use

For your platform (which is then a resolved case), but maybe not others.

> is there _any_ operational experience with the use of the subnet

I've never found a real use for it.

> asking the question another way. is it still a good idea, or was it

Currently I don't see the use. The only use seems to be an extra
annoying slide when one is explaining all the 'good stuff about IPv6' ;)

One would almost wonder about fully deprecating the subnet anycast
address.....

Greets,
 Jeroen

---

From: Randy Bush <[email-masked]>
Date: Mon, 16 Aug 2010 19:02:11 +0900

> annoying slide when one is explaining all the 'good stuff about IPv6'

is anyone using ipv6's special anycast at all?

i see use of v4-style anycast in v6, i.e. announcing the same prefix
from varied places in the topology.  but i know of no use of v6's odd
anycast.  but i have a narrow view of the world.

randy

---

From: Fernando Gont <[email-masked]>
Date: Mon, 16 Aug 2010 08:28:56 -0300

> anycast address of course.
[...]

For the most part, i was trying to make it clear that I wasn't asking
about the fix in RFC 4443 from the point-of-view of "the ping-pong issue
is already fixed in RFC 4443... we don't need /127 prefixes!". (i.e.,
I'm not against /127 prefixes... actually, I support the idea).

But I'm still interested in knowing what's the downside of the fix in
RFC 4443 that I cited in my original post. Does it really kill performance?

Thanks!

Kind regards,

---

From: Ole Troan <[email-masked]>
Date: Mon, 16 Aug 2010 13:54:49 +0200

> I'm not against /127 prefixes... actually, I support the idea).

/127 gives you other things too.
 - those who use /31s for IPv4 today can continue the same practice (the resist change argument)
 - operators implicitly know the address of the other end
 - doesn't "waste" space (yeah, yeah... I know)

one could equally just make a convention to use link-locals with fe80::1 and fe80::2
and /128s on each side if one needed global addresses for sources to traceroute etc.

> RFC 4443 that I cited in my original post. Does it really kill performance?

in the forwarding path one would have to check that incoming interface equals outgoing interface (done in existing redirect check anyway) and one would have to verify that the destination address is covered by a connected route on the outgoing interface.
this shouldn't require additional lookups, but would require changes in silicon.

cheers,
Ole

---

From: Mark Smith <[email-masked]>
Date: Mon, 16 Aug 2010 23:08:06 +0930

On Mon, 16 Aug 2010 18:46:18 +0900

> then you will join us supporting the /127 document and it won't be a

Why won't you and the other authors do a proper job with it then? It
doesn't address all the implications that arise. It should, point by
point address, all the issues in RFC3627. It should address the points
I raised here 2 weeks ago. It doesn't read to me as a proper
justification of why the RFCs it contradicts should be contradicted.
e.g. Where is the text explaining the implications to bits 70 and 71,
if there are any, and do they need to be managed, and if so, how?

Saying "it works" is not good enough. A screw driver can be used to
open a paint can, so "it works" as a paint can opener, but eventually
you ruin it and won't be able to use it to undo screws.

/127s break things that are described in existing specifications and
have been implemented. I think the /127 draft needs to justify itself
against each and every one of them.

I personally probably won't use them. But that doesn't mean that I don't
want to understand them and that I don't want to know exactly all the
implications and tradeoffs involved in using them. The draft as it
stands doesn't properly provide that information.

> randy
> --------------------------------------------------------------------
> IETF IPv6 working group mailing list
> [email-masked]
> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> --------------------------------------------------------------------

---

From: Gert Doering <[email-masked]>
Date: Mon, 16 Aug 2010 15:24:02 +0200

Hi,

> this little subtle problem and upgrade your routers and voila your

Cisco understands subnet-anycast, and disables this for /127s.

Gert Doering
        -- NetMaster
-- 
Total number of prefixes smaller than registry allocations:  155817

SpaceNet AG                        Vorstand: Sebastian v. Bomhard
Joseph-Dollinger-Bogen 14          Aufsichtsratsvors.: A. Grundner-Culemann
D-80807 Muenchen                   HRB: 136055 (AG Muenchen)
Tel: +49 (89) 32356-444            USt-IdNr.: DE813185279

---

From: Roger Jørgensen <[email-masked]>
Date: Mon, 16 Aug 2010 19:41:32 +0200 (CEST)

> then you will join us supporting the /127 document and it won't be a

The special case with anycast combined with /127 is such a bad idea that
any work on changing it back to the way it worked before (2003 or so
was it changed on Linux) will get my support for sure!

-- 
---

------------------------------
Roger Jorgensen      | - ROJO9-RIPE  - RJ85P-NORID
[email-masked]           | - The Future is IPv6
-------------------------------------------------------

A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?

---

From: Christopher Morrow <[email-masked]>
Date: Mon, 16 Aug 2010 14:34:39 -0400

> and /128s on each side if one needed global addresses for sources to traceroute etc.

no, ping/monitoring/data-collection fails in this case. (or needs to
be overhauled to collect/test/monitor in new/fun ways)

-chris

---

From: Ole Troan <[email-masked]>
Date: Mon, 16 Aug 2010 20:49:46 +0200

> be overhauled to collect/test/monitor in new/fun ways)

how & why? please, give a more specific example.

cheers,
Ole

---

From: Jared Mauch <[email-masked]>
Date: Mon, 16 Aug 2010 15:57:44 -0400

Please explain how ll would solve the problem first. Maybe the bcp38+1918 thread on nanog on recent days would be instructive. 

Jared Mauch

---

From: Ole Troan <[email-masked]>
Date: Mon, 16 Aug 2010 22:09:27 +0200

Jared,

> Please explain how ll would solve the problem first. Maybe the bcp38+1918 thread on nanog on recent days would be instructive. 

which problem? there are several.
with regards to the NANOG reference, I don't quite see the similarity. I haven't seen any implementation sourcing packets with a link-local source and these getting anywhere.

what I suggested was just to do one of:
1) link-locals only. ICMPs, traceroute etc take a suitably scoped source address from e.g a loopback or some other interface.
2) link-locals plus a global /128 if one wants per interface addresses for ICMPs etc.

BGP peerings and what not could use link-local addresses. e.g:

router A --------------  router B
fe80::1                        fe80::2
dead:beef::1/128     c001:cafe::2/128

no shared subnet between A and B apart from fe80::/10.

cheers,
Ole

---

From: Christopher Morrow <[email-masked]>
Date: Mon, 16 Aug 2010 16:13:48 -0400

> how & why? please, give a more specific example.

please ping my router, it's interface address is: fe80::20e:cff:fe5c:b001/64

my monitoring system can't ping this to ensure liveness of the
interface either :(

---

From: Ole Troan <[email-masked]>
Date: Mon, 16 Aug 2010 22:36:59 +0200

> my monitoring system can't ping this to ensure liveness of the

but they can ping whatever global /128 you put on that interface, so why doesn't that solve the problems?

cheers,
Ole

---

From: [email-masked]
Date: Mon, 16 Aug 2010 22:52:21 +0200 (CEST)

> but they can ping whatever global /128 you put on that interface, so why doesn't that solve the problems?

Because you are then using one set of addresses for protool peerings
and another one for global ping - thus making life more complicated
for the operator.

---

From: Ole Troan <[email-masked]>
Date: Mon, 16 Aug 2010 23:01:08 +0200

> and another one for global ping - thus making life more complicated

is that any more "complicated" (I don't quite understand that argument) than using IS-IS?

cheers,
Ole

---

From: Mark Smith <[email-masked]>
Date: Tue, 17 Aug 2010 06:34:00 +0930

On Mon, 16 Aug 2010 11:48:41 +0200

> asking the question another way. is it still a good idea, or was it ever?

I think it's origins are likely Appletalk, where it was used for Name
Binding Protocol. NBP is for resolving hostnames to autoconfigured
Appletalk node addresses. It was constantly in use anytime an Appletalk
network had a router.

---

From: Randy Bush <[email-masked]>
Date: Tue, 17 Aug 2010 06:15:00 +0900

> and another one for global ping - thus making life more complicated

and is sure to have really fun failure modes that are exciting to debug.

randy

---

From: Jared Mauch <[email-masked]>
Date: Mon, 16 Aug 2010 18:00:56 -0400

Jared Mauch

> is that any more "complicated" (I don't quite understand that argument) than using IS-IS?

Yes.

---

From: Olivier Vautrin <[email-masked]>
Date: Mon, 16 Aug 2010 14:18:06 -0700

It is clear that there is one more action done on the packet with RFC4443. But this has no impact on shipping ASIC based routers. It is difficult to say though if some smaller routers could be impacted.

Another problem with RFC4443 is that it is breaking some application. There could be cases where packet needs to be sent back onto point-to-point links from which they were received.  For example, an LER (Label Edge Router) could just forward the packet solely based on its label without IP resolution.  In this case, if the destination was the LER's egress interface, then the downstream router would do an IP lookup and sent back the packet to its interface (it could be a ping to the LSR egress interface for example).

/Olivier

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Fernando Gont
> Sent: Monday, August 16, 2010 4:29 AM
> To: Jeroen Massar
> Cc: [email-masked]; [email-masked]
> Subject: Re: ping-pong phenomenon with p2p links & /127 prefixes
> 
[...]

---

From: Seiichi Kawamura <[email-masked]>
Date: Tue, 17 Aug 2010 09:02:16 +0900

>> is that any more "complicated" (I don't quite understand that argument) than using IS-IS?

yes. I tried something close to it 5 years ago and it was hell.

Regards,
Seiichi

---

From: Ole Troan <[email-masked]>
Date: Tue, 17 Aug 2010 02:33:13 +0200

> yes. I tried something close to it 5 years ago and it was hell.

how? Jared's "Yes" doesn't exactly help my understanding why this is operationally complex.

cheers,
Ole

---

From: Seiichi Kawamura <[email-masked]>
Date: Tue, 17 Aug 2010 10:49:10 +0900

Hi Ole-san

> how? Jared's "Yes" doesn't exactly help my understanding why this is operationally complex.

Jared's yes may be different from my yes, but

(quoting your exmaple)

> dead:beef::1/128     c001:cafe::2/128

if
   I get a BGP neighbor down message with fe80::2
then
   what address do I ping, trace? I can look at config of router A
   and my address is dead:beef::1. What's the other side's global address?
   If router B isn't mine, I may not have a clue.

The challenge here is that we don't always have the knowledge of
whats on the other side of the router. When you have tons of these
links on one router, this is just making trouble shooting harder.

Even if I did know the other side's global address, monitoring pings
cannot be sent to fe80::2. We'll have to ping c001:cafe::2 and
manually link that status with fe80::2 peering session on the NMS.
I would hate to do that with hundreds of sessions running inside my network.
That's always been a causes mistakes. We want to monitor what's
acutally running and not some alias address.

Regards,
Seiichi

---

From: Christopher Morrow <[email-masked]>
Date: Mon, 16 Aug 2010 21:51:03 -0400

> how? Jared's "Yes" doesn't exactly help my understanding why this is operationally complex.

I should have also asked you to contact your (cisco's) top 10 service
provider and enterprise operations folks, ask them if putting 2x the
interface addresses on interfaces, then attempting to monitor them is
something they are willing/able/trying to do. (really, ask what's
going to pay the bills, see if less complexity or more wins)

in today's world I have a router with 1.2.3.1/31 and another with
1.2.3.0/31 connected across a ptp link. I put these PTR's into my
local (potentially global, but that's a rathole) DNS and my monitoring
system tells me that 1.2.3.1 is alive and well. Sometime later it
tells me that 1.2.3.1 is 'down' (unpingable).

Tomorrow I'll also monitor 2001:db8:1::0/128 and 2001:db8:1::1/128
(which will also appear in DNS for me) on the same respective routers,
across the same respective ptp link. This I'd call, actually,
2001:db8:1::/127, some folks may chose (for religious reasons it turns
out) to call it 2 disconnected /128's. My routers will peer (bgp)
across this /127 (or 2 sets of /128's), they'll put into their IGP the
/127 for reachability info... things will work fine.

I don't have to worry or know about the fe80 addresses (I may not even
have them on the interfaces in question). Why would I add complexity
here? Why would I want to add more records keeping and more overhead?

I'd add that today, on all of the exchange points I see routers
connected to (quote a few) no one uses fe80 addresses for link
addressing, everyone uses globally unique addresses. For all
interprovider connections I see ONLY globally unique addresses used,
not a single person is attempting to interconnect over fe80/link-local
addresses. There is no guarantee of uniqueness with fe80/link-local
address:
o I can not monitor my network (or the interconnects with remote
networks) with link-local addresses
o I can not guarantee that 2 devices not on the same fabric won't
select the same address in link-local space
o I can not use these addresses in my routing protocols (because of
the above point)
o I will not add complexity and extraneous overhead to my operations
to use link-local addresses because of some silly religious reasons,
globally unique addresses work just fine. They even work just fine if
I use 'two disconnected /128's' that I'll happily call (and allocate)
as a /127.
o gymnastics in the address selection/routing world to avoid a problem
that flat shouldn't exist (ping/pong packets) is just silly attempts
to justify the same silly religion. Use a set of 'two disconnected
/128's' that are from the same /127 and be done with it. Seriously.

As an aside, and because it sort of mystifies me that you are asking
these questions Ole... is it because you honestly don't know or
because you are trying to run down the pathways of resistance to an
idea/plan? Have you operated a large network in the past? Have you not
sat with customers fighting to operate their networks in sane
fashions?

-Chris

---

From: Jared Mauch <[email-masked]>
Date: Mon, 16 Aug 2010 23:14:11 -0400

> how? Jared's "Yes" doesn't exactly help my understanding why this is operationally complex.

You're comparing using ISIS as an IGP with utilizing non-unique (link-local) and non-globally-addressable IPs with each other.

I honestly am having a hard time comprehending where you are going.

link-local addresses have a very-limited use (and in some cases no use at all in the backbone that we operate).

While they are useful for communicating on-link, the use (and preference) of a globally unique IP has greater intrinsic value.  In the case of a /127 (we actually set the links as /128's on one vendor to avoid the ping-pong issue) you have two routers (or hosts) trying to talk to each other.  Anything else on-link promotes unnecessary router chatter.

- Jared

---

From: Randy Bush <[email-masked]>
Date: Tue, 17 Aug 2010 14:19:28 +0900

> at all in the backbone that we operate).

i fear we are talking to people who don't go past the head end.  cisco
is big, and folk can get tunnel vision.

randy

---

From: Pekka Savola <[email-masked]>
Date: Tue, 17 Aug 2010 11:20:56 +0300 (EEST)

Hi,

I changed the subject, because the original intent was lost in the 
weeds.

On Mon, 16 Aug 2010, Olivier Vautrin wrote:
> It is clear that there is one more action done on the packet with 
> RFC4443. But this has no impact on shipping ASIC based routers. It 
> is difficult to say though if some smaller routers could be 
> impacted.

This, and what Ole Troan wrote on interface lookup, is interesting.

RFC4443 requires checking that destination address matches the subnet 
prefix.  Is this the hot issue?

Note that pingpong-00 document did not have this requirement; the 
specification was different (incoming/outgoing interface).  Does this 
have different implications on the feasibility of implementation?

FWIW, "Packet may be forwarded back on the received interface" is 
actually, AFAIK, used in certain PE routerscenarios where you ping 
yourself over a p2p link.

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: Ole Troan <[email-masked]>
Date: Tue, 17 Aug 2010 12:02:06 +0200

Seiichi-san,

> acutally running and not some alias address.

yes, I see that point.
how do you troubleshoot when you get a OSPFv3, RIP, or ISIS neighbor down message?
cause then you'd only have a link-local address or a CLNS address. or is BGP troubleshooting different in some way?

this is a solvable problem. it could be done through a management system, better support in routers, a script pinging the link-local address from the router, and I'm sure lots of other solutions.

I'm trying to understand if this is "just" resistance to change (yeah, I know both too little and too much got changed with IPv6.) with the argument being that "this is how we have done it for the last 20 years and we will continue to do it this way whatever argument you make", or if there are real technical and operational issues with link-local only (optionally with /128) p2p links.

we as the IETF community need operator input and we need to understand operational complexities. thanks for replying. 
(and perhaps even Randy with his ever so charming ways also think so.)

cheers,
Ole

---

From: [email-masked]
Date: Tue, 17 Aug 2010 13:11:35 +0200 (CEST)

> cause then you'd only have a link-local address or a CLNS address. or is BGP troubleshooting different in some way?

As of right now, we also have IPv4 addresses on the same links. The traps
we receive normally include enough info (e.g. circuit id, interface name,
IP address, whatever) that we can easily identify the link. Having links
with only IPv6 link-local addresses *and* no further info included in the
traps would be unacceptable.

All our core links are configured with "normal" (global) IPv6 addresses.
We are fully aware of the fact that the routers also use IPv6 link local
addresses as *next hop* for most protocols (e.g. iBGP, IS-IS). We don't
deal with these link local addresses at all under normal circumstances -
instead we deal with the interface names that the routers also helpfully
tell us.

Similarly, all our IPv6 eBGP peerings are configured with global IPv6
addresses - here the IPv6 next hop is also a global IPv6 address.

So, to sum up: yes, we know that the IPv6 link local addresses exist on
our routers, no we don't normally "deal" with these addresses in any way.

---

From: Seiichi Kawamura <[email-masked]>
Date: Tue, 17 Aug 2010 20:35:02 +0900

Hi Ole-san

> how do you troubleshoot when you get a OSPFv3, RIP, or ISIS neighbor down message?

We're sort of drifting away from the original /127 /128 discussion, but
I think this is a good point so I'll make a comment on this one.

My router is configured with
 2001:db8:aaaa:bbbb::bbbb:1
 fe80::bbbb:1
the "bbbb" wich is the 48 bit to 64 bit is copied to the next to the last 16-bit field.
We do this so when we ge an ospf neighbor down, we can guess the global address easily.
When we do show ipv6 ospf neighbor, we can guess the global address from the output.
We still need to figure out the aaaa but that's allocated on a wider scale
versus the /64 which is allocated to each link. (note: ethernet links and not /127)
I don't use /127 internally, but if I did, this addressing scheme will not work.
I'll need some other way to work around.

I don't run RIP (not anymore), and ISIS, but that's how i do things.
Some others do it other ways, like mapping the Ipv4 address to the
interface identifier to achieve the same goal. I don't do that because
someday I hope I won't have to operate dual stack networks and just be
single stack IPv6.

I would loooooooooove to have show ipv6 ospf neighbor give me a global address output.

> cause then you'd only have a link-local address or a CLNS address. or is BGP troubleshooting different in some way?

yes. In two ways.
One, in BGP we only have to deal with a single address (not technically, but operationally
we don't have to care about the link local).
Two, in BGP we have to deal with the unknown. Your neighbor may not disclose all
the information to you. In OSPF, its your own little world most of the time.

BTW, I tried link-local BGP peering once many years ago, and swore to
myself to never do it again.

> this is a solvable problem. it could be done through a management system, better support in routers, a script pinging the link-local address from the router, and I'm sure lots of other solutions.

yes, I understand your point.
There isn't a solution available to me on my current router,
but if there was better support for ospf, I would appreciate the implementation very much.

However, for addressing of links, dealing with the unkown is
complex enough, that having special odd addressing structures
(like not setting a global or setting some odd /128)on that
link does not seem like a choice that would speed up troubleshooting.
That's a risk that many operators would not really want to take.

Even if the link is not used by BGP and only by OSPF,
having a uniform simple design throught a routing domain is
essential in running a 24/365 NOC team. The requirements of
the NOC team ultimately decide the happiness of the customer.

> we as the IETF community need operator input and we need to understand operational complexities. thanks for replying. 

Operators need IETF guidance as well. Especially in times like these.
Thanks for asking :-)

Regards,
Seiichi

---

From: Randy Bush <[email-masked]>
Date: Tue, 17 Aug 2010 20:51:15 +0900

> So, to sum up: yes, we know that the IPv6 link local addresses exist
> on our routers, no we don't normally "deal" with these addresses in
> any way.

and hope we don't have to, because they are not reachable, not uniqie,
have no mapping to the way we think of and name the interfaces, ...

and yes, margorie (a local name for ole), we're stuck in time wanting to
think of and name interfaces in the unique and global way we have for 25
years.  after all, the interfaces are unique and globally reachable.

randy

---

From: Fernando Gont <[email-masked]>
Date: Tue, 17 Aug 2010 08:02:07 -0300

Hi, Pekka,

> Note that pingpong-00 document did not have this requirement;

AFAICT, it does. It says: "....and the destination address on the packet
seems to be on-link (in terms of Neighbor Discovery) on the
point-to-point interface". Or am I missing something?

> the

It seems that the point is not really that of reduced performance, but
rather that complying with this requirement would require a change in
the silicon?

If that's the case (i.e., no real performance implications), then it
looks like an appropriate fix for this issue. -- which does not
necessarily argue against /127 prefixes, as there are other reasons for
using them (or, put another way, let's not correlate *this* with the
fight over /127 prefixes).

> yourself over a p2p link.

Is the echo request/response really forwarded back on the received
interface? (isn't the *response* that is forwarded back on the received
interface?)

Thanks!

Kind regards,

---

From: Truman Boyes <[email-masked]>
Date: Tue, 17 Aug 2010 07:52:23 -0400

> Steinar Haug, Nethelp consulting, [email-masked]

Greetings, 

I share the same sentiments that Steinar has outlined in his post. In the networks that I build, I would prefer to use IPv6 global addresses for peering. I find any position that management systems will reduce the operational complexity of peering with IPv6 link-locals to be academic; the position of security in peering is an interesting one and in some circumstances I would even consider the use of it, but I would certainly not convert any large scale network to using link-local peering without having the necessary code to manage all existing events. 

I think that there will be considerable "resistance to change", and any strong argument for the change needs to be outlined. At this time I have not seen a strong one presented. We (all of us) have been running, building, innovating changes to service provider networks for years and in a market where margins become smaller and smaller, the case would need to be presented where there was significant value in the change to link-local peering. 

Kind regards,
Truman

---

From: Fernando Gont <[email-masked]>
Date: Tue, 17 Aug 2010 10:16:15 -0300

Ole Troan wrote:

>> Thus, do ask Cisco and Juniper and other vendors where this now
>> 'works' if this intentional, or if they might finally comply to the
>> IPv6 specifications one day, as then you might better watch out for
>> this as it will break your network. For the vendors that have it,
>> it might maybe be an idea to have a 'disable subnetanycast' command
>> or similar so that one can explicitly mark a prefix that way.
> 
> it is intentional. there is a command to enable support for
> subnet-router anycast if use of that is desired.

What's the authoritative online documentation (and specific section of
that document) for this behavior?

Thanks!

Kind regards,

---

From: Ole Troan <[email-masked]>
Date: Tue, 17 Aug 2010 17:54:36 +0200

> that document) for this behavior?

I know an oxymoron when I see one. 
http://www.cisco.com/en/US/docs/ios/ipv6/configuration/guide/ip6-addrg_bsc_con.html#wp1052470

cheers,
Ole

---

From: JINMEI Tatuya / 神明達哉 <[email-masked]>
Date: Tue, 17 Aug 2010 11:08:35 -0700

At Tue, 17 Aug 2010 08:02:07 -0300,

> point-to-point interface". Or am I missing something?

(Aside from what is actually documented in the RFC/draft) the
additional check about on-link-ness or subnet matching was
intentionally added and that's what we implemented for KAME/*BSDs.

See, e.g.,
http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/netinet6/ip6_forward.c?rev=1.68&content-type=text/x-cvsweb-markup
(around the comment beginning with "If the incoming interface is equal...")

As commented, we wanted to separate an inevitable loop with a valid
configuration from loops due to
configuration/operational/implementation errors (most typically a
temporary routing loop).  We'd rather catch the latter type of error
easily with usual diagnostic tools such as traceroute so that we can
find and fix it quickly.

---
JINMEI, Tatuya
Internet Systems Consortium, Inc.

---

From: Christian Huitema <[email-masked]>
Date: Tue, 17 Aug 2010 20:19:17 +0000

> and hope we don't have to, because they are not reachable, not uniqie, have no mapping to the way we think of and name the interfaces, ...
>
> and yes, margorie (a local name for ole), we're stuck in time wanting to think of and name interfaces in the unique and global way we have for 25 years.  > after all, the interfaces are unique and globally reachable.
>
> randy

The whole point of IPv6 is to make addresses and prefixes plentiful. If global addresses facilitate the management of peering, why exactly would we not provide sufficient allocations to the network managers? Do we really believe that allocating prefixes to 100,000 or even 1,000,000 interconnection points will deplete the stock? That would just be less than 1 hundred thousandth or one 1 billionth of the available space!

-- Christian Huitema

---

From: James Joyce <[email-masked]>
Date: Wed, 18 Aug 2010 05:44:50 +0900

dear stephen daedalus,

> The whole point of IPv6 is to make addresses and prefixes plentiful.
> If global addresses facilitate the management of peering, why exactly
> would we not provide sufficient allocations to the network managers?

who has said they are not provided?

> Do we really believe that allocating prefixes to 100,000 or even
> 1,000,000 interconnection points will deplete the stock? That would
> just be less than 1 hundred thousandth or one 1 billionth of the
> available space!

this is off the wall.  no one is talking about a lack of global
addresses.  we are talking about protocols which insist on not using
them but use 'local' addresses insteady.

james

---

From: Olivier Vautrin <[email-masked]>
Date: Tue, 17 Aug 2010 15:37:59 -0700

> interface? (isn't the *response* that is forwarded back on the received

No. Again, There could be cases where packet needs to be sent back onto point-to-point links from which they were received.  For example, an LER (Label Edge Router) could just forward the packet solely based on its label without IP resolution.  In this case, if the destination was the LER's egress interface, then the downstream router would do an IP lookup and sent back the packet to its interface (it could be a ping to the LSR egress interface for example).

If the downstream router was using RFC4443, it would break the ping (in this example).

/Olivier

---

From: Pekka Savola <[email-masked]>
Date: Wed, 18 Aug 2010 09:09:36 +0300 (EEST)

> point-to-point interface". Or am I missing something?

Yes, you're right.  I was not reading the text carefully enough. 
(Though "is on-link" vs "is of the same subnet prefix" are 
semantically two subtly different checks. Not sure if it matters in 
this case.)

> fight over /127 prefixes).

This issue was initially brought up by Google IPv6 presentation, 
proxying Juniper's statements, so it would probably best if either of 
them could clarify.

> interface? (isn't the *response* that is forwarded back on the received

You're probably right.  I was likely confused on how this is actually 
done, and right now I can't find any written references on the "p2p 
self ping" troubleshooting technique I seem to recall. Olivier's note 
about the different scenario may still apply.

---

From: Mark Smith <[email-masked]>
Date: Wed, 18 Aug 2010 19:11:02 +0930

On Tue, 17 Aug 2010 11:20:56 +0300 (EEST)

> yourself over a p2p link.

Would that mechanism be described in -

RFC5881 - "Bidirectional Forwarding Detection (BFD) for IPv4 and IPv6
(Single Hop)"

?

I'm aware of a proposal to use it for CPE to test for the
absence/presence of a remote BRAS, which in the case of PPP virtual
circuits, would avoid the BRAS control plane having to respond to LCP
Echo Requests. That's quite attractive with BRASes now being able to
carry multiple 10s of 1000s of subscribers.

Regards,
Mark.

---

From: Miya Kohno <[email-masked]>
Date: Thu, 19 Aug 2010 20:56:37 +0800

Hi Pekka,

>proxying Juniper's statements, so it would probably best if either of 

There is not a straight-forward answer. It can be done without
performance degradation if designed so (e.g. uRPF,
filter-based-forwarding, etc.). In practice, everything costs something
and there are various trade-offs. 

Miya

---

From: Miya Kohno <[email-masked]>
Date: Thu, 19 Aug 2010 20:56:57 +0800

> if there are any, and do they need to be managed, and if so, how?

*Except /127*, we support rfc3627 and the appendix B.2 of rfc5375. They
have properly addressed the implication for using longer prefix than
/64. 

Miya

---

From: Mark Smith <[email-masked]>
Date: Sun, 22 Aug 2010 09:30:56 +0930

Hi Miya,

On Thu, 19 Aug 2010 20:56:57 +0800

> have properly addressed the implication for using longer prefix than

So where is there reference to Appendix B.2 of RFC5375 in the /127
draft? The draft does not mention anything about the 70/71 bit issue,
and that RFC5375, section B 2.4, discusses what to do about it. I happen
to be a contributor to that RFC, so I've read it at least twice, and
I've forgotten that it is there. What about the people who haven't read
it at all? (A reference to "B.2" isn't any good - when I first went to
that section I came across the B.2.2 text saying /127s shouldn't be used
- and that made me quite confused about what you've said above)

RFC3627 effectively provides a minimum checklist of the implications
that need to be specifically addressed in the /127 draft for /127s to
be accepted. If the /127 draft was published as an RFC, it shouldn't be
possible for a reader, who's deciding whether to adopt /127s or not,
has read RFC3627, and is now reading the /127 RFC, to say "so what about
this issue identified in RFC3627?"

Regards,
Mark.

---

From: Miya Kohno <[email-masked]>
Date: Mon, 23 Aug 2010 00:09:35 +0800

Hi Mark,

> - and that made me quite confused about what you've said above)

So except /127 :). Except /127, we support 3627 and 5375 regarding the
analysis for the longer prefixes in general. (We're not specifically
focusing on EUI addressing.) The section 4 of the current draft states
this, but we'll clarify more. 

> this issue identified in RFC3627?"

To clarify and update the usage of /127 is the very reason of the /127
draft. Thank you for your comments, though!

Miya

---

From: Christopher Morrow <[email-masked]>
Date: Sun, 22 Aug 2010 12:30:25 -0400

>> draft? The draft does not mention anything about the 70/71 bit issue,

mark,
what is that issue? that addrarch asks that 70/71 essentially both be
0? What's the harm in them being 1? Taking the pragmatic approach that
'bits are bits' in these cases they are not part of the host-address
so they shouldn't matter, I think.

Curious though, since lots of things seem to be encoded for mysterious
reasons in the 128 bits of an ipv6 address.

-chris

---

From: Mark Smith <[email-masked]>
Date: Mon, 23 Aug 2010 06:38:17 +0930

On Sun, 22 Aug 2010 12:30:25 -0400

> so they shouldn't matter, I think.

I think people are missing the point. I'm only using the 70/71 issue
as an example of something that is discussed in RFC3627, but not
discussed or referenced in the /127 draft. If the /127 draft is a
rebuttal of RFC3627, and that's how I think it is presented, then I
think it needs to at least rebut ever issue relevant to /127s identified
in RFC3627. It doesn't do that, with an example of something missing
being the discussion or reference to what to do about the 70/71 bits.

Other examples - there are probably more - of things I think that should
be discussed, beyond what is in RFC3627 -

o  what is the prefix length of link locals - should it also be /127?

o  do the links need link locals (I'm partly asking because I've seen
some P2P link implementations (tunnels specifically) not have link
locals)?

o  in the ethernet example, is ND NS/NA enabled (the current /127
generally says no)? If not, now is the remote end going to learn the
link layer address of the peer?

o  if /127s mitigate the ND cache exhaustion problem, then why not have
ND NS/NA enabled, making them compliant with the ND RFC, where p2p links
aren't treated as anything special? This rectifies the ethernet issue
too.

o  The anycast router address isn't the only anycast address
that /127s would disable - there are 128 reserved anycast addresses in
a subnet. Another one impacted is the Mobile IPv6 Home-Agents anycast.
So there are 126 left for future possible use. The existing uses may
not be that valuable in point-to-point link scenarios today, but what
are the /127s proponent's advice on what to do of another anycast
address is assigned that is useful on p2p links? 

There are potentially others I think.

> reasons in the 128 bits of an ipv6 address.

It's certainly not unique to IPv6. IPv4 did it of course, with Classes,
including Class E where the forwarding mechanism is different, or 0
network meaning this network, and other protocols like Appletalk and
IPX have done it too. 

Regards,
Mark.

---

From: Randy Bush <[email-masked]>
Date: Mon, 23 Aug 2010 09:06:50 +0900

> If the /127 draft is a rebuttal of RFC3627

and if it isn't?  maybe it's just a bug report on one bit?

> be discussed, beyond what is in RFC3627 -

where is that darned immersion heater?

randy

---

From: Mark Smith <[email-masked]>
Date: Mon, 23 Aug 2010 21:19:09 +0930

On Mon, 23 Aug 2010 09:06:50 +0900

> and if it isn't?  maybe it's just a bug report on one bit?

Well, firstly, this is the text in the /127 draft which seems to
suggest to me it is:

"This document provides rationale for using 127-bit prefix lengths,
   reevaluates the reasons why doing so was considered harmful, and
   specifies that /127 prefixes MUST be supported on inter-router links
   configured for use as point-to-point links."

The MUST in capitals is the key word. The Interface ID has been 64 bits
in length since RFC2373 (July 1998) for global prefixes. Many RFCs rely
on that. This draft is setting itself up to be the new authoritative
statement on an exception to all the prior RFCs that have followed the
64 IID specification. I think therefore it is obligated to address all
the issues that an exception creates.

Alternatively, if it was published as an Informational RFC, I think the
threshold of detail required could be lower.

IMHO, where the real problem lies isn't the /64 prefix length, it's in
the neighbor discovery neighbor solicitation/neighbor advertisement
mechanism, and even more specifically, it's caused by these two
requirements in the Neighbor Discovery RFC, "7.2.2. Sending Neighbor
Solicitations" -

" While waiting for address resolution to complete, the sender MUST,
   for each neighbor, retain a small queue of packets waiting for
   address resolution to complete.  The queue MUST hold at least one
   packet, and MAY contain more.  However, the number of queued packets
   per neighbor SHOULD be limited to some small value.  When a queue
   overflows, the new arrival SHOULD replace the oldest entry.  Once
   address resolution completes, the node transmits any queued packets."

and

"If no Neighbor Advertisement is received after MAX_MULTICAST_SOLICIT
 solicitations, address resolution has failed.  The sender MUST return
 ICMP destination unreachable indications with code 3 (Address
 Unreachable) for each packet queued awaiting address resolution."

The MUST of holding a queue of NS triggering packets, and the MUST of
returning an ICMP destination unreachable create remotely exploitable
state on a router's interface. Switching off NS/NA on links where it
could be (i.e. point-to-point links), mitigates that, however it then
creates the ping-pong problem. /127s is a mitigation for that, however
it isn't a solution for links containing end-nodes i.e. LANs. There'll
still be millions of them that are remotely exploitable from offlink
sources. So /127s are very much a partial rather than full solution.
Lengthening the prefix length/shortening the IID is always a
mitigation, not a solution to the remote exploitable state.

I think there are two realities which reduce the usefulness of
these MUST implement mechanisms -

o  the Internet is best effort, so if a source node wants to be
confident that the packet arrived, it needs to be prepared to
re-transmit it. Queueing it on a router while NS/NA takes place helps
but doesn't avoid a node having to be prepared to retransmit.

o  Also due to the best effort nature of the Internet, there is a
possibility that the ICMP destination unreachable won't make it back to
the original packet's source, with perimeter firewalls being one of the
causes.

If those MUSTs were loosened to SHOULDs or MAYs, then it would be
acceptable to avoid having a router maintain state while the ND
NS/NA transaction takes place.

Once that state requirement is removed, I think there are a number of
ways to solve the layer 3 to layer 2 address mapping function on a link
-

o  have nodes periodically announce themselves via unsolicited
multicasts - the ES-IS model.

o  have nodes register there presence via a two way transaction - the
6lowpan ND optimisation mechanism might be a candidate.

o  use an NS/NA transaction, with the NS containing a magic
cookie in the flow label field or maybe an ND option that is copied
into the NA, and verified by the NS originator. The number of NSes per
second might be rate limited to avoid creating a CPU DoS because of
cookie generation. Source nodes will eventually retransmit their
packets if the NS doesn't occur because the cookie generation rate
limit was exceeded.

o  use an NS/NA transaction, and just accept NAs when ever they arrive.
There are more trust issues here because on the onlink nodes could DoS
the router with NAs - however they have far more of an incentive to not
do it because they're likely to be DoSing the router that provides them
with offlink connectivity. This is essentially a passive/solicited
version of the ES-IS model. It could also be considered a "loose" or
existing ND NS/NA "compatibility" mode of the more "strict" NS/NA cookie
transaction mode.

These mechanisms are applicable to any type of link, would preserve the
simplicity of universal 64 bit IIDs and the other benefits of them e.g.
CGAs, as well as avoiding the ping-pong problem.

> where is that darned immersion heater?

No sure what they do. Can I buy one that runs IPv6?

Regards,
Mark.

---

From: [email-masked]
Date: Mon, 23 Aug 2010 14:11:04 +0200 (CEST)

> CGAs, as well as avoiding the ping-pong problem.

IMHO, the "universality" of 64 bit IIDs went down the drain the moment
router vendors allowed longer than 64 bit netmasks to be configured.

For the routers I am most familiar with (Juniper, Cisco), longer than
64 bit netmasks have been configurable for many years. And such masks
are heavily used for provider backbone links.

The IPv6 standards community can of course continue to pretend a belief
in universal 64 bit IIDs - thus ensuring that they are out of touch
with IPv6 reality...

---

From: Mark Smith <[email-masked]>
Date: Mon, 23 Aug 2010 22:47:36 +0930

> router vendors allowed longer than 64 bit netmasks to be configured.

So how does that prevent those prefix lengths being changed to /64?

> are heavily used for provider backbone links.

As before, how does that prevent those prefix lengths being changed
to /64?

> in universal 64 bit IIDs - thus ensuring that they are out of touch

Maybe that's your reality, but it isn't everybody's.

---

From: Randy Bush <[email-masked]>
Date: Mon, 23 Aug 2010 22:27:27 +0900

>> The IPv6 standards community can of course continue to pretend a
>> belief in universal 64 bit IIDs - thus ensuring that they are out of
>> touch with IPv6 reality...
> 

as you demonstrate so clearly

but those of us who are operators and actually use routers and actually
use /127s etc. kinda like the reality we're experiencing and will play
whack-a-vendor to keep it if we need to

steinar's point is that this draft attempts to codify the operational
reality in the normal way such is done in the ietf

it's really pretty simple

randy

---

From: Mark Smith <[email-masked]>
Date: Mon, 23 Aug 2010 23:16:54 +0930

On Mon, 23 Aug 2010 22:27:27 +0900
Randy Bush <[email-masked]> wrote:

> but those of us who are operators

Why do you assume I'm not? 

> and actually use routers

So that's what they call those things I keep configuring to forward
packets. Thanks for the tip.

> and actually

And all you'll end up with is IPv4 with bigger addresses. You really
should catch up with the useful features of protocols that were
designed in the late 80s / early 90s, like IPX, Appletalk, DECNet and
CLNS.

> reality in the normal way such is done in the ietf

I think the normal way is also to make sure the RFCs can be used to
implement protocols, and can also be used as authoritative references
on how devices that claim compliance with them should behave. That's
what I use them for when lodging bugs with vendors, or providing
feedback to vendors on IPv6 functionality, as I'm doing now with a
couple of CPE vendors.

> it's really pretty simple

Rarely is anything simple.

---

From: Jared Mauch <[email-masked]>
Date: Mon, 23 Aug 2010 09:55:48 -0400

> So how does that prevent those prefix lengths being changed to /64?

Because you would then end up with overlapping address space that is unreachable in a production deployment.

But that would be an operational item and not an standards body item?

- Jared

---

From: [email-masked]
Date: Mon, 23 Aug 2010 17:24:00 +0200 (CEST)

> designed in the late 80s / early 90s, like IPX, Appletalk, DECNet and

For me "more addresses" is the *only* justification for IPv6. All the
other "useful" features are either uninteresting or even *unwanted*.

I'm sort of okay with RA on customer links (though I would much rather
have a more featureful DHCPv6). For my backbone links RA is simply out
of the picture, not even interesting to discuss.

The "failover" capabilities of IPv6 with RA I would much rather solve
with HSRP/VRRP.

etc.

---

From: Miya Kohno <[email-masked]>
Date: Mon, 23 Aug 2010 23:24:16 +0800

IPX, Appletalk, etc. were much sophisticated as LAN protocols, but not
very good in terms of the backbone routing. 

Backbone routing should be optimized for scalability, speed/capacity,
and security; whereas LAN or CPE/Edge routers should be optimized more
for features like plug n' play, subscriber management, etc.

IPv6 protocol theory is more sophisticated than IPv4. But it has far
less consideration for the real backbone operation. I think the recent
confrontation in the 6man/6ops mailing list is due to the different
perspectives between the IPv6 protocol theory and the backbone operation
in practice. 

Miya
-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Mark Smith
Sent: Monday, August 23, 2010 10:47 PM
To: Randy Bush
Cc: [email-masked]; [email-masked]
Subject: Re: ping-pong phenomenon with p2p links & /127 prefixes

---

From: Miya Kohno <[email-masked]>
Date: Mon, 23 Aug 2010 23:29:25 +0800

+1

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
[email-masked]
Sent: Tuesday, August 24, 2010 12:24 AM
To: [email-masked]
Cc: [email-masked]; [email-masked]
Subject: Re: ping-pong phenomenon with p2p links & /127 prefixes

etc.

---

From: Fred Baker <[email-masked]>
Date: Mon, 23 Aug 2010 13:36:42 -0700

On Aug 23, 2010, at 1:15 PM, Joel Jaeggli wrote:

> On 8/23/10 5:11 AM, [email-masked] wrote:
> I think it can be demonstrated that reality includes longer prefixes
> than /64. The document needs to be unequivocal about that fact. or more
> to the point that it is altering that specifications as they stand now.

I agree, but I think I would phrase it differently. The point is that an IPv6 address is a CIDR prefix with an interface ID. As allocated by SLAAC, it will be a /64 prefix and a 64 bit IID. But SLACK is not the only allocation paradigm. Addresses that are allocated by other paradigms are not bound by the SLACK paradigm, only by the need to perform and respond to duplicate address detection and as a result guarantee the local uniqueness of the IID.

---

From: Mark Smith <[email-masked]>
Date: Tue, 24 Aug 2010 06:19:01 +0930

On Mon, 23 Aug 2010 09:55:48 -0400

> Because you would then end up with overlapping address space that is unreachable in a production deployment.

Not necessarily. If I were to deploy /127s, I'd be allocating /64s to
the links.

> But that would be an operational item and not an standards body item?

This has been cross posted to v6ops.

---

From: Mark Smith <[email-masked]>
Date: Tue, 24 Aug 2010 06:41:18 +0930

> other "useful" features are either uninteresting or even *unwanted*.

Just so I know, are you confirming that you've only ever used IPv4, and
know nothing about any other protocols and how they worked?

Your view seems to me to be a bit like saying, "I'm perfectly happy
with my 1970s car, it gets me from A to B, and I see no reason to have
electric windows, anti-lock brakes, electronic fuel injection, or a GPS,
because my 1970s car doesn't have them".

---

From: Jared Mauch <[email-masked]>
Date: Mon, 23 Aug 2010 17:19:26 -0400

> because my 1970s car doesn't have them".

If customers were saying they wanted "network based security" more people
would be offering it.  Problem is that most providers set the price of
$security_thing (either 2547-style vpn, etc..) as something too costly
or external auditors don't sign-off on it, so they say:

give me $dumb_pipe @ 5G/s and toss their own IPSec/VPN/whatnot
device/appliance on-top of it.

The biggest feedback I hear from people about IPv6 (besides the extra
bits for addressses) is "Security", but they generally don't know what
that is outside marketing speak.  Saying that your 1970's car doesn't
have an onboard GPS is correct, but that doesn't mean the customer won't
just get a $200 aftermarket at walmart.  Is that ideal for the $car_mfg?
Surely it is a hit on the profit margin, but overall customers are just
asking for $dumb_pipe @ 5Gb/s @ $low_low_price_per_meg.

- Jared

---

From: Jared Mauch <[email-masked]>
Date: Mon, 23 Aug 2010 17:23:09 -0400

> Not necessarily. If I were to deploy /127s, I'd be allocating /64s to

You may put a /64 on your /127 links in addition, but most people only put
one IP subnet on a link, otherwise they might want redirects ;)

> This has been cross posted to v6ops.

Operationally the vendors may be violating some RFC, so lets publish what is
relevant and working today so we can all move on?  We can deal with
any additional updates and items with "how IPv6" works elsewhere or in a
new document so we can move /127 on p2p links along?

- Jared

---

From: Mark Smith <[email-masked]>
Date: Tue, 24 Aug 2010 07:08:30 +0930

On Mon, 23 Aug 2010 17:23:09 -0400

> one IP subnet on a link, otherwise they might want redirects ;)

I meant reserving a /64 for the link and then configuring a /127 prefix
length on it. If my concerns about /64s were resolved, all I'd need to
do would be change the prefix length back to a /64.

> new document so we can move /127 on p2p links along?

So that leaves the problem still existing on network edge LANs and
virtual P2P links between customer aggregation routers and CPE, of
which there'l be millions. Maybe you, Steinar and Randy don't
have to worry about those types of links, but others of us do.

A complete solution would solve the problem for all link types, not
just mitigate it for point-to-point links in the backbone.

Regards,
Mark.

---

From: Joel Jaeggli <[email-masked]>
Date: Mon, 23 Aug 2010 13:15:15 -0700

---

From: "Manfredi, Albert E" <[email-masked]>
Date: Mon, 23 Aug 2010 16:53:56 -0500

Jared Mauch:

> that is outside marketing speak.

+1, in spades. Nor do these folk seem to appreciate that it's not the network that bears the greatest burden in providing that security. It is the clients. And that this is also true with IPv4. You don't get security if only the network is secure. Conversely, you can get security if the network is not secure.

The /64 limitation only applies for SLAAC. Seems to me that there are many service providers with examples of links where SLAAC isn't applicable. I don't see why models that work with IPv4, like CIDR, must be rejected out of hand. Making IPv6 "less different" from IPv4 can only help its implementation, IMO.

And too, good IPv6 ideas, like multiple addresses per client, also create new problems and new mechanisms for non-reachability, which are still having to be solved.

Bert

---

From: [email-masked]
Date: Mon, 23 Aug 2010 23:57:43 +0200 (CEST)

> know nothing about any other protocols and how they worked?

No. I've used DECnet both as an end user, as a network programmer and
as a network admin. Appletalk as an end user and network admin, XNS 
only as a network admin. CLNS is still alive and well in our backbone,
used for IS-IS. We even have a little bit of CLNS routing, over on the
SDH/WDM side.

> because my 1970s car doesn't have them".

I don't claim to represent all views on IPv6. I *do* claim that a view
that "more addresses" is the only justification for IPv6 is reasonably
widespread. I also claim that in my backbone the IPv6 RA baggage is an
unwanted feature, and I'd much rather solve the problems with a more
featureful DHCPv6, HSRP/VRRP etc.

I'm quite willing to believe that there are large networks that want
IPv6 RA functionality, want LAN segments to be locked to 64 bits, etc.
But please don't assume these views universal - and I won't assume
that *my* views are universal. Thank you.

---

From: Randy Bush <[email-masked]>
Date: Tue, 24 Aug 2010 07:11:16 +0900

> widespread.

you don't want mandatory ipsec, longer battery life, ...? :)

96 more bits, no magic  -- gaurab

the problems existing operators (who just want to add v6 and not go to
the cathedral of ipv6 religion) hit in deployment are 96 more bits and
not all the features they use in ipv4.  dhcp and vrrp, as you mention.
but net management would be nice too, though that is not as much ietf
issue.

again, we need to make it EASY for people to deploy ipv6 on their
networks.

randy

---

From: Mark Smith <[email-masked]>
Date: Tue, 24 Aug 2010 07:37:27 +0930

On Mon, 23 Aug 2010 17:19:26 -0400
Jared Mauch <[email-masked]> wrote:

> just get a $200 aftermarket at walmart.

Well, GPS was only one of the examples I used, and I was envisioning
one that is built into the dash. To continue with the analogy, how many
people would buy and install after-market electric windows, anti-lock
brakes, electronic fuel injection etc.?

>  Is that ideal for the $car_mfg?

I think you haven't really realised who the customers are of IPv6 or
more generally Internet technologies. It isn't the people who pay ISPs
or indirectly pay to access the Internet via corporate networks. They're
customers of Internet services. More specifically they're customers of
the function of allowing applications or processes to intercommunicate
internationally. They don't and shouldn't have to care about the
technology that makes it work. That's what operators are for. Thats why
the "cloud" analogy for networks is a good one - it represents a
concept of "don't worry about that magic inside, just assume your data
will get from one side to the other". (no I'm not talking about "cloud
computing")

The people who buy, deploy and operate the technology of the Internet
are the customers of IPv6. That's not the general public, that's people
like you and I. Our job is to use Internet technologies to deliver
Internet services to our customers. So when I say I think this is what
I'd like IPv6 to do, or it shouldn't work like IPv4 because there are
potentially better ways to do it, I'm the customer. So when I ask for
simplicity, I'm asking for it so that my job is easier and quicker, and
therefore cheaper in time or money for my customers - the end-users of
Internet services. I'm even asking for it so that they don't even have
to employ me to get basic connectivity going on a small scale, even
without understanding the technology. (I'm also asking for it so that I
won't have relatives ask me to help setup their Internet connections at
home - as if I want to do my job unpaid on the weekend as well ...)

Protocols like IPX and Appletalk demonstrated how easily deployable
protocols could be made on a small to medium scale, and they did it in
the 1990s. They define the minimum of what any protocol designed since
then should be able to achieve in user friendliness. IPv4 doesn't.

To paraphrase Soylent Green, "IPv6 customers is operators!"

Regards,
Mark.

---

From: "Manfredi, Albert E" <[email-masked]>
Date: Mon, 23 Aug 2010 17:19:03 -0500

> brakes, electronic fuel injection etc.?

Analogies work both ways. In the '50s, a "great idea" was the helicopter parked in everyone's driveway. Or at the very least, nuclear powered cars. Yet here we are today, still driving on four wheels and four pneumatic tires, with internal combustion engines.

> are the customers of IPv6.

See, that's not entirely the case. They are also people who merely specify the use of IPv6, without appreciating exactly what that means.

And then there are those who must make that happen intelligently, so everything still works right and operates safely.

Bert

---

From: Fred Baker <[email-masked]>
Date: Mon, 23 Aug 2010 15:51:59 -0700

On Aug 23, 2010, at 2:53 PM, Manfredi, Albert E wrote:

> +1, in spades. Nor do these folk seem to appreciate that it's not the network that bears the greatest burden in providing that security. It is the clients.

They also don't get that it doesn't buy you anything if you don't use it, or that it is generally available on IPv4 systems as well. Do a packet capture on your favorite LAN and measure the percentage of IPsec-protected traffic. Then ask yourself how many of those systems in fact implement IPsec code. It's largely about the distribution of certificates and "turning it on".

> And that this is also true with IPv4. You don't get security if only the network is secure. Conversely, you can get security if the network is not secure.
>

---

From: "Hemant Singh (shemant)" <[email-masked]>
Date: Tue, 24 Aug 2010 09:51:41 -0500

-----Original Message-----
Sent: Monday, August 23, 2010 5:58 PM

>that *my* views are universal. Thank you.

Since when does an ND RA locks LAN segments to 64 bits?  I have already
mailed this /127 thread during the IETF 78 week that an RA can be
configured such that the RA does not send any PIO and signals the
clients on the link to send traffic to the default router.  Some SP
networks such as the cable broadband network has host behind two
different cable modems as off-link to each other and such networks do
provision the RA with no PIO.  I also pointed to our document in RFC
5942 that discusses ND off-link.

Hemant

---

From: Christopher Morrow <[email-masked]>
Date: Wed, 25 Aug 2010 00:55:34 -0400

> do would be change the prefix length back to a /64.

this means you're already doing what the draft is asking to codify?
So, in short you support the idea of the draft you just have some
reservations about it covering all the bases you feel are important.

> have to worry about those types of links, but others of us do.

are cable/dsl providers going to provision each p2p link to the end
customer with a /64? won't that just be 2 off-link /128's (today I see
mostly 2 off-link /32's I think?) and PD to send down the prefix the
operator decides is appropriate for in-home use?

If so, the /127 discussion isn't involved here, I think.

> just mitigate it for point-to-point links in the backbone.

I think the only thing that's being addressed in this one draft is
codifying what's happening in practice today. So that vendors don't
accidentally 'fix' the 'bug' that permits /127's to work just fine.
since you stated earlier you already do this, that seems to fit with
your requirements as well, yes?

-chris

---

From: Christopher Morrow <[email-masked]>
Date: Wed, 25 Aug 2010 00:57:12 -0400

> new document so we can move /127 on p2p links along?

I like this as well.. if other link types would like special handling
can't they also gin up some rfc-text and refer to the work done here
as a stepping stone to their answer?

-Chris

---

From: Mark Smith <[email-masked]>
Date: Wed, 25 Aug 2010 18:37:45 +0930

On Mon, 23 Aug 2010 17:19:03 -0500

> Analogies work both ways. In the '50s, a "great idea" was the helicopter parked in everyone's driveway. Or at the very least, nuclear powered cars. Yet here we are today, still driving on four whee…

All the technologies I listed both exist today and are common in cars
that are designed today. Those technologies didn't exist in the 1970s,
or if they did, were probably so prohibitively expensive that they
either weren't even considered for cars or weren't affordable to put
into cars. If you were to design a car today, would you limit it's
design to incorporating technologies that were available 30 years ago?

Possibly it will be surprising to a number of people on this list, but
some of the ideas in IPv6 are over 30 years old, such as single, fixed
size network and node portions, and using link layer addresses as layer
3 node addresses -

"Address Mappings", Jonathan B. Postel, 2 May 1979
http://www.ietf.org/rfc/ien/ien91.txt

---

From: Mark Smith <[email-masked]>
Date: Wed, 25 Aug 2010 19:19:23 +0930

On Wed, 25 Aug 2010 00:55:34 -0400

> this means you're already doing what the draft is asking to codify?

No I'm not.

If somebody was to ask me what my advice was I'd recommend /64s for all
links. I'd also point out the issue that /127s is trying to mitigate.
I'd point out that I think there are feasible solutions that properly
address this issue for all link types. I'd then recommend that if they
want to use /127s, they reserve a /64 for those links, so that if and
when those solutions become available, they can merely change the
prefix length back to /64.

> reservations about it covering all the bases you feel are important.

What I'm against is unnecessary complexity, because, to quote
"anonymous" on slashdot, "complex = more things that can
break". Variable length interface identifiers, if they're unnecessary,
only create more complexity and therefore more things that can break.
If the interface identifier is universally the same size, then it'd be
pretty hard for people to get the size wrong.

The other thing that complexity creates is more choice. Technical
people like choice. The problem with too much choice, however, is
choice paralysis. If there are too many choices, it can become too risky
to make one. With too much choice, it may not be possible to adequately
weigh up the positive and negative consequences of all of the
choices, and in particular the consequences of making a wrong choice.
So not making a choice becomes less risky than making the wrong one.

So, if we have /64s everywhere, and have them not vulnerable to issues
that /127s mitigates (but only on point-to-point links), then I
think /127s are unnecessary. That's one less unnecessary complexity.

> customer with a /64?

It's likely to be the common model. It's the simplest and most widely
available option because it uses no more than the basic IPv6 RA/PIO
mechanism and SLAAC, and therefore would be a baseline functionality
expectation in both CPE and customer aggregation routers.

> won't that just be 2 off-link /128's (today I see
> since you stated earlier you already do this,

No I don't use them, as mentioned earlier.

> that seems to fit with your requirements as well, yes?
> 
> -chris

---

From: Mark Smith <[email-masked]>
Date: Wed, 25 Aug 2010 19:46:55 +0930

On Wed, 25 Aug 2010 00:57:12 -0400

> as a stepping stone to their answer?

If they did, I think they'd be missing the idea of implementing layer 3
to layer 2 address resolution within ICMPv6, instead of following the
IPv4 model of having parallel and separate address resolution protocol
like ARP. (Maybe ARP would have been implemented in ICMP if they hadn't
decided to make it a more general purpose protocol that could suit
Ethernet address resolution for multiple layer 3 protocols, such as
Chaosnet, IPv4 and Xerox PUP)

The idea in IPv6 is to try as much as possible to avoid treating link
types specially, so that for new link types, new layer 3 to
layer 2 protocols don't have to be developed. The only link layer
capabilities Neighbor Discovery Address Resolution requires is unicast
and multicast. If they can be provided, or emulated, such as in the
cast of multicast over point-to-point links, then there isn't any need
to create new protocols and additional complexity (because complex =
more things that can break). It is simpler, when you can, to treat
everything the same.

Regards,
Mark.

---

From: "Manfredi, Albert E" <[email-masked]>
Date: Wed, 25 Aug 2010 14:44:31 -0500

> http://www.ietf.org/rfc/ien/ien91.txt

Doesn't surprise me at all.

I take it, your position is that if an address can have a 64-bit prefix and a 64-bit host ID, then all you should need or want is classful addressing. And only one class at that. Does that sum it up?

What's old is new again. We could almost revert back to RIPv1.

I don't necessarily disagree, and I got that same message from others on the list in the past. And I also wouldn't suggest that prefixes shorter than 64 should be in heavy demand (except in assigning address blocks). It's just that the restriction seems unnecessary, especially given that /127s and /128s are already in use, that CIDR has been invented and everyone knows how it works, and also because the /64 restriction only matters for SLAAC.

Bert

---

From: Mark Smith <[email-masked]>
Date: Sun, 29 Aug 2010 09:04:34 +0930

Hi Bert,

On Wed, 25 Aug 2010 14:44:31 -0500

> I take it, your position is that if an address can have a 64-bit prefix and a 64-bit host ID, then all you should need or want is classful addressing. And only one class at that. Does that sum it u…

When it is possible, it is simpler to have a single sized node address,
which also means the same sized prefix length. Simplicity benefits
everybody - protocol developers, implementers, operators and end-users.

In IPv4 it isn't possible anymore, and hasn't been since around
1981/82 (i.e. somewhere between rfc760 and rfc791), because there
weren't enough addressing bits. Classes were a work around, and were
the first of the "elegant hacks" used to overcome IPv4's limited
address size as the scale of it's deployment started to exceed the
original design assumptions. Fixed length subnets, then variable length
subnets, then classless inter-domain routing were subsequent elegant
hacks to get around these limitations.*

With IPv6 we can have that simplicity back, because the address size
constraint has been removed. We don't need to redeploy the complexities
of IPv4 that we've become comfortable with - we've only become
comfortable with them because we didn't have a choice. We're used to
paying the price of that complexity, and don't notice it, because it
was an essential cost of growing and operating the Internet.

(* if you defined a protocol version change as occurring when the way
packets are processed is changed at a fundamental level, e.g. address
lookups, and software/firmware or hardware replacement needs to take
place to facilitate that change, then each of these "elegant hacks"
could have been a new version of IP - IPv6 might actually be IPv10
(allowing for IPv5 / Internet Stream Protocol))

> I don't necessarily disagree, and I got that same message from others on the list in the past. And I also wouldn't suggest that prefixes shorter than 64 should be in heavy demand (except in assigni…

I think IPv6 "CIDR" i.e. longest match rule across the whole 128 bits is
really only insurance against having to perform a whole of Internet
upgrade, similar to what had to happen when CIDR was introduced, should
the chosen boundary between the network and interface id portions end up
being incorrect. Its likely it won't be necessary to use, as all the
address sizing calculations, e.g. RFC3177, RIR policies etc., are based
on the assumption of 64 bit interface ids. I think that is a wise
decision, however it is only insurance - something you use if the
unexpected occurs.

Regards,
Mark.

---

From: Christopher Morrow <[email-masked]>
Date: Sat, 28 Aug 2010 22:43:21 -0400

> upgrade, similar to what had to happen when CIDR was introduced, should

folk are already holding (internally) /128's for things, I suppose
they could have /64's, but that means dedicating a 'LAN' to some task
(anycast of a nameserver, for instance) when you really only want to
use a 'host' for that.

It means using a 'LAN' for each 'host' you want to anycast... or
losing some flexibility in your service management/offering.

moving back to 'classful' addressing isn't a move forward, or not such
that it's worth the move with the additional overhead incurred. I
agree that making almost all 'LAN' segments a /64 is a fine plan, some
folks may choose other boundaries and in those cases will not get RA
or other things, they may not need those things though.

-Chris

---

From: Mark Smith <[email-masked]>
Date: Sun, 29 Aug 2010 12:56:26 +0930

On Sat, 28 Aug 2010 22:43:21 -0400

> (anycast of a nameserver, for instance) when you really only want to

This is one of the issues. People are being unnecessarily precious about
address space. What real benefit do they gain? It seems that their
IPv4 "must conserve address space" mentality is so ingrained that they
can't even conceive of anything else. That's what makes me think that
they haven't experienced any other protocols that trade addressing bits
for operational convenience. Of course they've probably used Ethernet,
and never or rarely set MAC addresses, so they've already experienced
that sort of operational convenience, but somehow don't realise that it
can also exist at layer 3.

> that it's worth the move with the additional overhead incurred.

That's not what I'm advocating. If I was, I'd be advocating separate
network and node fields in the IPv6 header - as IPX/XNS/Appletalk etc.
implemented.

> I
> agree that making almost all 'LAN' segments a /64 is a fine plan,

So why isn't it a fine plan for all links in a network? We can afford
the address space.

> some

A number of innovations are possible and have been developed in IPv6
because the single and soft boundary between the network and node
portions at the 64 bit mark. These innovations would not have been
possible if that soft boundary hadn't been chosen in the past. If that
soft boundary is now eliminated, then what useful innovations won't be
possible in the future? Maybe somebody will come up with an innovation
that would be of real benefit on point-to-point links between routers,
however it needs 64 bit identifiers. Yet if /127s are widely deployed,
then that innovation may never be worth developing because it can't be
widely deployed. It may even never be thought of, because people will
become ingrained in "point-to-point" links are only /127s.

One of my concerns about /127s is that it'll become the thin edge of
the wedge for also eliminating the /64 boundary on LANs. Then we'll
have spotty SLAAC support, different devices with different prefix
lengths etc., - all the problems we've had with incorrect subnet masks
in IPv4, as well as new ones like SLAAC not working.

Fundamentally, people seem to be wanting to introduce constraints into
IPv6 that seem to have no other reason to be other than "that's how we
did it in IPv4". Yet there doesn't seem to be any questioning of why
those methods had to be implemented in IPv4, and if the design of IPv6
eliminates those IPv4 constraints - and if we'd had a choice in IPv4,
would we have accepted them in the first place.

Regards,
Mark.

---

From: Christopher Morrow <[email-masked]>
Date: Sat, 28 Aug 2010 23:56:07 -0400

> address space. What real benefit do they gain? It seems that their

not precious with address space, precious with opex costs...

 if you put each service address on it's own /64 that may be fine for
some folks but will run afoul of hd-ratio and other measures of how
'full' your use of the address space you'd been allocated may be.

if you put multiple service-addrs on a single /64 you have to
maintenance all of these service-addrs at the same time or suffer
spotty service coverage.

If you value resilient services (dns and ntp for two simple examples)
spotty coverage isn't really an option.

> So why isn't it a fine plan for all links in a network? We can afford

not every link needs it and there are some pretty scary implications
of using /64's on some link types (ptp links, for the /127 example
here), plus complexity and headache that's just not necessary when you
aim to put the 2 ends of the ptp link on well known (to you) and well
defined addresses. SLAAC is just un-useful in this case, RA and ND and
other functions are just a waste of CPU time I want to be used making
RIB/FIB updates go faster.

> that would be of real benefit on point-to-point links between routers,

maybe, though we've been using the same (essentially) technology for
this for ~30 years and ... the largest change has been /30 -> /31
migrations. this part of the network isn't where 'innovation' is
supposed to happen. the edge can do that just fine... operating a
cheap (very, very cheap), fast (very, very fast) and simple core is
the goal. 'innovation' in the core isn't a super good plan for the 2
other parts (cheap/fast).

Today, and in the past number of years, the operational reality and
migration of the industry has been to fast, cheap core wherever
possible. Costs added to operations or capital in the core just don't
happen. Ask VZB/UUNET how many 'network engineers' are doing 'network
engineering' for their global public ip backbone today? far fewer than
in years past. I suspect you'll see the same story at Level3, ATT,
Comcast, Telia, NTT etc...

> widely deployed. It may even never be thought of, because people will

core links are, by numbers, I imagine far fewer than edge links...
Enterprises make up the vast majority of edge network user cases,
equipment buys, etc. Heck, Comcast has ~20m end users, that's got to
be a few orders of magnitude more than their core links, right? :)

> become ingrained in "point-to-point" links are only /127s.

they are.

> the wedge for also eliminating the /64 boundary on LANs. Then we'll

for LANs, SLAAC works fine for getting addresses and default-gw pushed
out. Until ~3 years ago (thanks to thaler, nartens, et-al for the
change then) it was useless for pretty much everything else :(

I don't think folks really want to change LAN architecture... it does
seem to 'just work'.

> did it in IPv4". Yet there doesn't seem to be any questioning of why

"thats how our operations model works today, that's where the money is
driving this train, that's what operations folks are asking for."

> would we have accepted them in the first place.

I do think folks asked these questions, they did so sometime in the
past... they now have implemented a bunch of stuff and want to keep
operating their networks in the manner they are today. Adding more
complications isn't the right direction here.

Understanding that 10+ years ago when ipv6 was designed the world was
a very different place is what we all need to keep in mind. Today
billiions (at least) of dollars depend on a reliable, scalable, useful
network. screwing with that is not in anyone's best interest.  I think
what the authors are trying to do here is to document what they've got
working, why it's working that way and push that forward (since there
seems to be a decent amount of agreement among folks that operate
networks, worldwide) as an acceptable method.

-chris

---

From: Mark Smith <[email-masked]>
Date: Sun, 29 Aug 2010 13:46:02 +0930

I don't agree with any of your points I'm afraid, and I don't think
they'd stand up to any thorough analysis (e.g. /128s verses /64s don't
have any significant impact on HD ratios when an ISP gets a _minimum_ of
a /32 or 4 billion /64s)

However, I don't think this conversation is making any progress. People
do what they want to do, including sometimes making life harder for
themselves than it needs to be. C'est la vie.

On Sat, 28 Aug 2010 23:56:07 -0400

---

From: Fernando Gont <[email-masked]>
Date: Tue, 31 Aug 2010 22:01:07 -0300

Hi, Gert,

> Cisco understands subnet-anycast, and disables this for /127s.

Only for /127s, and automagically?

What about Junipers?

Thanks!

Kind regards,