Subject: Re: [v6ops] draIn ft-ietf-v6ops-design-choices WGLC

From: "Ackermann, Michael" <[email-masked]>
Date: Mon, 6 Apr 2015 23:10:51 +0000

HI Fred

This will be a most beneficial document for those of us contemplating IPv6 deployments.  

A couple questions: 

1.  In Section 2.1.2, second paragraph it states that a network with a large number of link local only interfaces, can just enable an IGP on each router  and not have to assign and trace addresses on each interface.   Does this advantage apply ONLY  to router link local only interfaces? 

2.  Section 2.2.1.   Most of the text in this section seems to suggest that using link-locals for next hop addresses is the best approach.   But there are a couple situations where it is not.   And then at the end it says "Today most operators use GUA's as next hop addresses".      As a neophyte reader I was uncertain what conclusion I should draw?  

3.  Section 2.3.1.    For many of us EIGRP is the IGP of choice.     Is it not acceptable for this protocol to be included in the  Option Table?  

Once again,  a very helpful beneficial document!

Thanks

Mike

-----Original Message-----
From: v6ops [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Sunday, April 05, 2015 2:00 PM
To: [email-masked]
Subject: [v6ops] draft-ietf-v6ops-design-choices WGLC

This is to initiate a two week working group last call of http://tools.ietf.org/html/draft-ietf-v6ops-design-choices.  
Please read it now. If you find nits (spelling errors, minor suggested wording changes, etc), comment to the authors; if you find greater issues, such as disagreeing with a statement or finding additional issues that need to be addressed, please post your comments to the list.

We are looking specifically for comments on the importance of the document as well as its content. If you have read the document and believe it to be of operational utility, that is also an important comment to make.

_______________________________________________
v6ops mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/v6ops

The information contained in this communication is highly confidential and is intended solely for the use of the individual(s) to whom this communication is directed. If you are not the intended recipient, you are hereby notified that any viewing, copying, disclosure or distribution of this information is prohibited. Please notify the sender, by electronic mail or telephone, of any unintended receipt and delete the original message without making any copies.
 
 Blue Cross Blue Shield of Michigan and Blue Care Network of Michigan are nonprofit corporations and independent licensees of the Blue Cross and Blue Shield Association.

---

From: Philip Matthews <[email-masked]>
Date: Tue, 7 Apr 2015 21:10:12 -0400

> 1.  In Section 2.1.2, second paragraph it states that a network with a large number of link local only interfaces, can just enable an IGP on each router  and not have to assign and trace addresses…

Are you asking if the same approach could be taken for host interfaces?  If so, then the question is:  how would you send packets to the host from off-link?  With a router, there is a virtual interface that has a (manually-assigned) global or unique-local address, and so packets can be sent to this address.  Usually hosts do not have such interfaces...

> 2.  Section 2.2.1.   Most of the text in this section seems to suggest that using link-locals for next hop addresses is the best approach.   But there are a couple situations where it is not.   And…

That, despite the specs seeming to say that static-routes should use link-local addresses, that in practice it can sometimes cause problems so most operators use GUAs instead.
Most large providers are large corporations with many employees with varying skill sets. These corporations have established procedures that work for IPv4 and most employees try to apply these IPv4 procedures to IPv6. Anything that causes IPv6 to be different than IPv4 means special procedures must be developed and taught to employees. Using link-local addresses in static routes is a difference, so most large providers don't use them.   Things can be different at a smaller provider where differences can be handled more easily.

> 3.  Section 2.3.1.    For many of us EIGRP is the IGP of choice.     Is it not acceptable for this protocol to be included in the  Option Table?  

The thing about EIGRP is that it is not standardized, but is Cisco proprietary. So I suspect that we could only say "Cisco EIGRP does this" and "Cisco EIGRP does that", which is not a typical thing for IETF documents.  I work for Alcatel-Lucent, and I know we do not support EIGRP on our routers because it is not standardized. Most large providers don't use it for that reason.

Fred, as WG chair and as a Cisco employee, could comment further on whether he thinks the document should cover EIGRP.

> Once again,  a very helpful beneficial document!

Thanks!!  The document started out as my attempt to document many of the questions I had when I first started to design IPv6 networks. I also credit my co-author Victor, who added many useful insights gained from deploying IPv6 when he worked for Rogers Communications.

- Philip

---

From: Mark ZZZ Smith <[email-masked]>
Date: Thu, 9 Apr 2015 00:49:21 +0000 (UTC)

From: Philip Matthews <[email-masked]>
 To: "Ackermann, Michael" <[email-masked]>; Fred Baker (fred) <[email-masked]> 
Cc: v6ops list <[email-masked]> 
 Sent: Wednesday, 8 April 2015, 11:10
 Subject: Re: [v6ops] draIn ft-ietf-v6ops-design-choices WGLC

> This will be a most beneficial document for those of us contemplating IPv6 deployments.  
<snip>

/ Perhaps that means there needs to be a bit more explanation of the value of using link-locals so people can more easily make a value judgement?
/ My understanding is that link-locals were chosen to be used as next-hop values are for two reasons (a) link-local prefixes are always present (as per RFC4291), unlike GUAs or ULAs and (b) they allow less disruptive phasing in and phasing out of GUA/ULA prefixes across a network as static or dynamic routing next-hops won't be impacted by GUA/ULA changes. 
/ Regards,Mark

<snip>

---

From: "Ackermann, Michael" <[email-masked]>
Date: Thu, 9 Apr 2015 14:41:16 +0000

Thanks for the good points Mark!

Documenting the pros and cons of using link locals as next hops, so that informed decisions can be made (especially by those perhaps less knowledgeable on IPv6),  would be a great benefit.       My organization is  just starting to test this, so I would appreciate input on this from those with more actual, practical experiences,  but  from what I have seen so far,  the link local benies would include:

1.       Omnipresence.

2.       Stability and unaffected by external events.

3.       More Secure

So pretty consistent with what you said.

What I do not know is what the drawbacks to using Link Locals for next hop might be?
I also wonder whether the pros and cons change,  if the device is an end host, vs a middle box?
Final related question is whether this situation is affected by the choice of or the presence of any routing protocols?

Thanks again!

Mike

From: Mark ZZZ Smith [mailto:[email-masked]]
Sent: Wednesday, April 08, 2015 8:49 PM
To: Philip Matthews; Ackermann, Michael; Fred Baker (fred)
Cc: v6ops list
Subject: Re: [v6ops] draIn ft-ietf-v6ops-design-choices WGLC

________________________________
From: Philip Matthews <[email-masked]<mailto:[email-masked]>>
To: "Ackermann, Michael" <[email-masked]<mailto:[email-masked]>>; Fred Baker (fred) <[email-masked]<mailto:[email-masked]>>
Cc: v6ops list <[email-masked]<mailto:[email-masked]>>
Sent: Wednesday, 8 April 2015, 11:10
Subject: Re: [v6ops] draIn ft-ietf-v6ops-design-choices WGLC

<snip>

/ Regards,
Mark

<snip>

---

From: Nick Hilliard <[email-masked]>
Date: Thu, 09 Apr 2015 16:32:28 +0100

On 09/04/2015 15:41, Ackermann, Michael wrote:
> Documenting the pros and cons of using link locals as next hops, so that
> informed decisions can be made (especially by those perhaps less
> knowledgeable on IPv6),  would be a great benefit.       My organization is
>  just starting to test this, so I would appreciate input on this from those
> with more actual, practical experiences,  but  from what I have seen so
> far,  the link local benies would include:
> 
> 1.       Omnipresence.

Large-scale provider networks would normally use a GUA as a next-hop for
the bgp prefix and set it to be the address of the PE device which
originated the prefix.  This allows prefix information to be carried in BGP
with NH reachability handled by the IGP.  I.e. IGP is used purely for
loopbacks and iBGP is used for everything else.  This avoids a large number
of troublesome design and scaling problems on many networks.

As a service provider network operator, I would go some lengths to avoid
using link-locals as NH addresses.  They provide no particular advantage
that I'm interested in and have a bunch of serious disadvantages, of which
the above is the main one.  The other main one is that they cause massive
fail when it comes to debugging problems.

With regard to the features that you listed:

- link-local addresses are not omnipresent, even if they are a mandatory
part of the protocol.  Personally, I prefer to disable them if there is an
option to do so.  Obviously this does not work if you're running protocols
which require them (e.g. ospf).

- "Stability and unaffected by external events".  The only stable address
on a router is a loopback.  MAC addresses can change due to e.g. software
upgrades.

- "more secure" is a statement of opinion which lacks context.  They're
unreachable if you're off-link, and if there is a requirement to consider
this as a security feature, infrastructure ACLs could equally be
implemented to give GUA addressing an equivalent level of security.

Nick

---

From: Nick Hilliard <[email-masked]>
Date: Thu, 09 Apr 2015 16:43:15 +0100

On 09/04/2015 16:32, Nick Hilliard wrote:
> originated the prefix.

I should qualify this a bit better: large scale providers running native
ipv6 will do this.  Many large scale providers will run ipv6 on mpls/6pe,
where the bgp NH IP is the ipv4 identifier address of the PE.

Nick

---

From: "Ackermann, Michael" <[email-masked]>
Date: Thu, 9 Apr 2015 15:58:36 +0000

Makes sense.   I can see where your BGP set up would be compelling.  

One of my questions was if the Link Local Pros and Cons would be different for End Hosts vs Middle Boxes.  
Your comments make me think that it may be good for End Hosts, but not so much Middle Boxes,  in particular large scale provider networks.  

You mention that you remove Link Locals wherever possible.    On what platforms is this possible without removing most of IPv6?    Does this cause issues with intrinsic functions such as NDP, etc.?  

Thanks. 

Mike   

-----Original Message-----
From: Nick Hilliard [mailto:[email-masked]] 
Sent: Thursday, April 09, 2015 11:32 AM
To: Ackermann, Michael; Mark ZZZ Smith; Philip Matthews; Fred Baker (fred)
Cc: v6ops list
Subject: Re: [v6ops] draIn ft-ietf-v6ops-design-choices WGLC

On 09/04/2015 15:41, Ackermann, Michael wrote:
> Documenting the pros and cons of using link locals as next hops, so 
> that informed decisions can be made (especially by those perhaps less
>  just starting to test this, so I would appreciate input on this from 
> those with more actual, practical experiences,  but  from what I have 
> seen so far,  the link local benies would include:
> 
> 1.       Omnipresence.

Large-scale provider networks would normally use a GUA as a next-hop for the bgp prefix and set it to be the address of the PE device which originated the prefix.  This allows prefix information to be carried in BGP with NH reachability handled by the IGP.  I.e. IGP is used purely for loopbacks and iBGP is used for everything else.  This avoids a large number of troublesome design and scaling problems on many networks.

As a service provider network operator, I would go some lengths to avoid using link-locals as NH addresses.  They provide no particular advantage that I'm interested in and have a bunch of serious disadvantages, of which the above is the main one.  The other main one is that they cause massive fail when it comes to debugging problems.

- link-local addresses are not omnipresent, even if they are a mandatory part of the protocol.  Personally, I prefer to disable them if there is an option to do so.  Obviously this does not work if you're running protocols which require them (e.g. ospf).

- "Stability and unaffected by external events".  The only stable address on a router is a loopback.  MAC addresses can change due to e.g. software upgrades.

- "more secure" is a statement of opinion which lacks context.  They're unreachable if you're off-link, and if there is a requirement to consider this as a security feature, infrastructure ACLs could equally be implemented to give GUA addressing an equivalent level of security.

Nick

---

From: Nick Hilliard <[email-masked]>
Date: Thu, 09 Apr 2015 22:01:33 +0100

On 09/04/2015 16:58, Ackermann, Michael wrote:
> One of my questions was if the Link Local Pros and Cons would be
> different for End Hosts vs Middle Boxes. Your comments make me think
> that it may be good for End Hosts, but not so much Middle Boxes,  in
> particular large scale provider networks.
> 
> You mention that you remove Link Locals wherever possible.    On what
> platforms is this possible without removing most of IPv6?    Does this
> cause issues with intrinsic functions such as NDP, etc.?

link-locals are necessary on end-hosts due to autoconfig requirements.
Outside this, my use cases tend to be statically configured servers or else
router link addressing.  Neither of these situations particularly need
link-locals.  NDP works fine with GUAs.  Other protocols associated with
autoconfig break (e.g. RA, DAD, etc), but these aren't relevant.

To disable link-locals on a cisco router, use "no ipv6 enable" on your
interface.  It works fine if you don't need it.

Nick

---

From: Mark ZZZ Smith <[email-masked]>
Date: Thu, 9 Apr 2015 22:42:18 +0000 (UTC)

I think it is worth pointing out that Nick's use cases are or are probably Internet Exchange ones, where both he and his customers are highly technically competent. The mechanisms he's disabling by disabling link-locals e.g., DAD, are ones that in an IX environment are in effect provided by thorough attention to detail in both administration and manual configuration, hence the "(e.g. RA, DAD, etc), but these aren't relevant."

I see the target for this draft to be more general cases, where there is less time or resources to spend on providing manual equivalents to these automated protocols that use link-locals.

(One particular bad incident in my past of where there were 5 hosts with duplicate IPv4 addresses pre-IPv4 DAD makes me never want to see DAD become optional again.)

----- Original Message -----
From: Nick Hilliard <[email-masked]>
To: "Ackermann, Michael" <[email-masked]>; Mark ZZZ Smith <[email-masked]>; Philip Matthews <[email-masked]>; Fred Baker (fred) <[email-masked]>
Cc: v6ops list <[email-masked]>
Sent: Friday, 10 April 2015, 7:01
Subject: Re: [v6ops] draIn ft-ietf-v6ops-design-choices WGLC

Nick

---

From: "Ackermann, Michael" <[email-masked]>
Date: Fri, 10 Apr 2015 15:42:12 +0000

It seems that the advice to use Link Locals for next hop (or maybe even to use them at all?),  is situationally dependent.       May be beneficial for most end user organization situations,    but perhaps not beneficial for many larger network operators.      If the document can point out reasons WHY this might be the case (e.g. large BGP implementation),  organizations viewing this document would be in a good position to determine what optimal network designs for  their specific situation.   

----- Original Message -----
From: Nick Hilliard <[email-masked]>
To: "Ackermann, Michael" <[email-masked]>; Mark ZZZ Smith <[email-masked]>; Philip Matthews <[email-masked]>; Fred Baker (fred) <[email-masked]>
Cc: v6ops list <[email-masked]>
Sent: Friday, 10 April 2015, 7:01
Subject: Re: [v6ops] draIn ft-ietf-v6ops-design-choices WGLC

link-locals are necessary on end-hosts due to autoconfig requirements.
Outside this, my use cases tend to be statically configured servers or else router link addressing.  Neither of these situations particularly need link-locals.  NDP works fine with GUAs.  Other protocols associated with autoconfig break (e.g. RA, DAD, etc), but these aren't relevant.

To disable link-locals on a cisco router, use "no ipv6 enable" on your interface.  It works fine if you don't need it.

Nick

---

From: Nick Hilliard <[email-masked]>
Date: Fri, 10 Apr 2015 18:01:08 +0100

On 09/04/2015 23:42, Mark ZZZ Smith wrote:
> I think it is worth pointing out that Nick's use cases are or are
> probably Internet Exchange ones, where both he and his customers are
> highly technically competent. The mechanisms he's disabling by disabling
> link-locals e.g., DAD, are ones that in an IX environment are in effect
> provided by thorough attention to detail in both administration and
> manual configuration, hence the "(e.g. RA, DAD, etc), but these aren't
> relevant."

IXPs are one situation I was thinking of (note that I also do L3 ISP
engineering as well as IXP operations).  General servers installation with
static IP addressing is another case which does not benefit from dynamic
addressing.  YMMV though.  Just because they don't add anything for me
doesn't mean that they don't add anything for you.  Also, server stuff is
slightly separate from network design.

> I see the target for this draft to be more general cases, where there is
> less time or resources to spend on providing manual equivalents to these
> automated protocols that use link-locals.

IPv6 IGPs (e.g. ospf / isis) implicitly use link-locals.  That's fine;
there is technical benefit there because these are link-state protocols and
don't gain benefit from using GUAs.

I don't see any technical benefit for bgp though.  There are two situations
to note: using LLs for session addressing or using them for next-hop.

It's pointless to use them for session addressing because ibgp will
generally talk to an (often non-locally-connected) route reflector so
link-locals are impossible, and ebgp will generally require GUAs by policy
anyway.  In either case, bgp sessions are manually configured (or handled
by provisioning system) rather than auto-discovered locally on a router, so
if you need to manually set up a session, it's probably more obvious to use
GUAs.

LLs don't really work for NH either, if you have a non trivial network.  Do
you really want your network to converge at the speed of your MRAI?  I
don't.  Far more sensible to use your zippy igp for all that sort of thing
and let bgp do the heavy lifting of prefix propagation.

Nick

> ----- Original Message ----- From: Nick Hilliard <[email-masked]> To:
> "Ackermann, Michael" <[email-masked]>; Mark ZZZ Smith
> <[email-masked]>; Philip Matthews <[email-masked]>;
> Fred Baker (fred) <[email-masked]> Cc: v6ops list <[email-masked]> Sent:
> Friday, 10 April 2015, 7:01 Subject: Re: [v6ops] draIn
> ft-ietf-v6ops-design-choices WGLC
> 
>> platforms is this possible without removing most of IPv6?    Does
>> this cause issues with intrinsic functions such as NDP, etc.?
> 
> Outside this, my use cases tend to be statically configured servers or
> else router link addressing.  Neither of these situations particularly
> need link-locals.  NDP works fine with GUAs.  Other protocols associated
> with autoconfig break (e.g. RA, DAD, etc), but these aren't relevant.
>

---

From: Nick Hilliard <[email-masked]>
Date: Fri, 10 Apr 2015 18:02:22 +0100

On 10/04/2015 16:42, Ackermann, Michael wrote:
> It seems that the advice to use Link Locals for next hop (or maybe even
> to use them at all?),  is situationally dependent.

Can you provide some concrete examples of where LLs are more appropriate
than GUAs for BGP NH, and why?  I'm genuinely struggling to see any advantage.

Nick

---

From: Gert Doering <[email-masked]>
Date: Fri, 10 Apr 2015 19:30:44 +0200

Hi,

> than GUAs for BGP NH, and why?  I'm genuinely struggling to see any advantage.

Seconded.

LL NHs for BGP are one of the things causing serious pain today (you peer
with a peer router on the GUA, your BGP process installs the prefix with
the LL NH into the FIB, and if anything breaks in ND for the LL NH, you
blackhole traffic - with GUA NHs, if ND breaks for the GUA, BGP will not 
come up, so "no blackholing").  

And no, this is not a hypothetical example, we currently have at least 
one peer configured with static ND entries on our side because LL ND between
these two boxes is "unreliable", and we've seen blackholing as a consequence
(TAC investigating, but it's... complicated to debug ND on a shared network
with over 600 routers and significant traffic levels).

Gert Doering
        -- NetMaster
-- 
have you enabled IPv6 on something today...?

SpaceNet AG                        Vorstand: Sebastian v. Bomhard
Joseph-Dollinger-Bogen 14          Aufsichtsratsvors.: A. Grundner-Culemann
D-80807 Muenchen                   HRB: 136055 (AG Muenchen)
Tel: +49 (0)89/32356-444           USt-IdNr.: DE813185279

---

From: "Ackermann, Michael" <[email-masked]>
Date: Fri, 10 Apr 2015 18:11:42 +0000

I do not see any advantage in a BGP environment.  

-----Original Message-----
From: Nick Hilliard [mailto:[email-masked]] 
Sent: Friday, April 10, 2015 1:02 PM
To: Ackermann, Michael; Mark ZZZ Smith; Philip Matthews; Fred Baker (fred)
Cc: v6ops list
Subject: Re: [v6ops] draIn ft-ietf-v6ops-design-choices WGLC

On 10/04/2015 16:42, Ackermann, Michael wrote:
> It seems that the advice to use Link Locals for next hop (or maybe 
> even to use them at all?),  is situationally dependent.

Can you provide some concrete examples of where LLs are more appropriate than GUAs for BGP NH, and why?  I'm genuinely struggling to see any advantage.

Nick

---

From: Nick Hilliard <[email-masked]>
Date: Fri, 10 Apr 2015 20:29:57 +0100

> I do not see any advantage in a BGP environment.  

Ok, sure.

Can you now provide some concrete examples of where LLs are more
appropriate than GUAs for BGP NH, and why?

Nick

---

From: Philip Matthews <[email-masked]>
Date: Fri, 10 Apr 2015 20:11:40 -0400

Michael:

I am following this conversation with interest, especially since you seem to be new to IPv6 network design, and thus are exactly the intended audience for this draft.

The questions you are coming up with seem to be very similar to the ones I had when I first started doing IPv6 network design.  Those questions later prompted me to start the draft.

So far, I have not seen anything new in this conversation to add to the draft, but it could well be that I am suffering from "document fatigue". You, on the other hand, are reading it with fresh eyes and may well see things that I just miss.

I encourage you and everyone else on this thread to send me any suggestions you have for improving the document.

- Philip

---

From: "Ackermann, Michael" <[email-masked]>
Date: Mon, 13 Apr 2015 14:34:59 +0000

Thanks Phillip

Yes, this document will be very helpful to those of us just beginning to deploy IPv6.   

Most of the very recent dialogue has centered around LL and BGP.   We do not use BGP, so I will be listening intently, but may not have much to comment on that specific  facet. 

Thanks again

Mike

-----Original Message-----
From: Philip Matthews [mailto:[email-masked]] 
Sent: Friday, April 10, 2015 8:12 PM
To: Ackermann, Michael
Cc: v6ops list; Nick Hilliard; Mark ZZZ Smith; Fred Baker (fred)
Subject: Re: [v6ops] draIn ft-ietf-v6ops-design-choices WGLC

Michael:

- Philip

---

From: JF Tremblay <[email-masked]>
Date: Tue, 14 Apr 2015 10:20:30 -0400

> On Apr 10, 2015, at 3:29 PM, Nick Hilliard <[email-masked]> wrote:
> 

I had the vague impression of having seen this discussion before…. after a bit of digging, this was covered in Chapter 3 of “IPv6 Security”, by Eric Vyncke and Scott Hogg. Pages 97-102 in my paper version. 

While the pros and cons of using LL are discussed, the text doesn’t make a clear recommendation not to use them, so maybe this is where some people get the idea of using LL for single-hop BGP. 

JF