Subject: Re: [lisp] Deployment Document - P-ITR

From: [email-masked] (Noel Chiappa)
Date: Wed, 13 Oct 2010 18:09:10 -0400 (EDT)

> From: Lorand Jakab <[email-masked]>

    > Another thing that maybe should be considered while discussing P-ITRs
    > is the LISP deployment progress (how many networks have migrated to
    > LISP), because it has a heavy influence on the amount of P-ITR traffic.

It's not just the amount of PITR traffic that's a concern; for me, a possibly
even bigger concern is the _potential_ increase in size of the 'legacy' DFZ
routing table - i.e. the routing table which has to be prepared for the
'unconverted' part of the Internet, to allow legacy stuff to talk to LISP
sites. (This has been a concern of mine for some years now...)

The situation is complex, because it depends on all sorts of factors,
including how much aggregability we get in the routing advertisements from
PITRs into the 'legacy' DFZ; whether address space blocks are simply being
converted from legacy (i.e. non-LISP) to LISP (which results in no change in
the routing table size); etc, etc. It might grow, it might shrink, etc, etc.

Alas, I'd really need a really big white-board to lay out all the
scenarios... :-(

    > they aggregate the prefixes of all of their clients, and announce the
    > aggregates from all of their PoPs (anycast). This leads to a continuous
    > decrease in the DFZ routing table.

Maybe... it all depends on whether any announced LISP blocks from those PITRs
are next to each other in the namespace. If not -> no decrease...

    > I've been told that IETF documents should avoid economics

Which might be a huge mistake when it comes to deployment documents, q.v. the
recent discussion on the main IETF list.

	Noel

---

From: Dino Farinacci <[email-masked]>
Date: Thu, 14 Oct 2010 04:23:23 -0700

> are next to each other in the namespace. If not -> no decrease...

I think this *could* be true for IPv4 prefixes. But with IPv6 PI  
allocated blocks, we could make the IPv6 core run really efficiently.

Chairs, I would like to propose IANA to give us not a coarse address  
block for IPv6 EID space but a high-order byte code-point. Similar to  
using 0xff for IPv6 multicast and 0xfe for IPv6 link-local addressing.

This has two benefits:

(1) A very large address space where PITRs can advertise a single  
route into the DFZ.

(2) A very fast method for ITRs and PITRs to tell that a destination  
is a LISP site to
     avoid asking the mapping database system for this answer.

Thoughts?

What would be the procedure to try to do this?

Dino

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 14 Oct 2010 10:48:05 -0400

A few clarifying questions (personal at this point in the process):

If I understand right, we are talking about a block from which EIDs 
would be assigned, and which would not be used for actual routable 
addresses:
1) Why do we need a a very short code?  As far as I can tell, we could 
use a /24, or worst case a /16.  That would still leave plenty of remote 
for organizational delegation to get to assignment to sites of a 
suitable size block.
2) Do we think we can get away with not permitting folks to use their PI 
IPv6 assignments as EIDs?
3) For any definition of a single block for EIDs to give us the 
advantage you site, we have to assume that P-ITRs are actually going to 
advertise the entire EID space, not just a portion of it related to 
their customer set.  That would be very nice, but does not match what I 
read in the deployment document.  I would really need to see the cost 
coverage for that, as there is an even bigger disconnect than just the 
issue of increasing traffic without revenue that I asked about relative 
to EID registrars providing P-ITR services.

Yours,
Joel

On 10/14/2010 7:23 AM, Dino Farinacci wrote:
> (1) A very large address space where PITRs can advertise a single route
> into the DFZ.
>
> (2) A very fast method for ITRs and PITRs to tell that a destination is
> a LISP site to
> _______________________________________________
> lisp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/lisp
>

---

From: Dino Farinacci <[email-masked]>
Date: Thu, 14 Oct 2010 20:18:16 -0700

> would be assigned, and which would not be used for actual routable  

Right.

> 1) Why do we need a a very short code?  As far as I can tell, we  
> could use a /24, or worst case a /16.  That would still leave plenty  
> of remote for organizational delegation to get to assignment to  
> sites of a suitable size block.

I don't want implementations to have to configure these addresses.  
Because as you add more, you'll have to change configurations and  
change ACLs in a lot of places. If you get a large enough space that  
is "well-known", the performance of the system will be better.

> 2) Do we think we can get away with not permitting folks to use  
> their PI IPv6 assignments as EIDs?

We can, but they may not aggregate well and the PITRs will have to  
inject more routes. IPv6 adoption is in its infancy so we can give  
sites which request a new EID-based PI prefix.

> advantage you site, we have to assume that P-ITRs are actually going  
> to advertise the entire EID space, not just a portion of it related  
> to their customer set.  That would be very nice, but does not match  
> what I read in the

Yes, that is the point. You want all the PITRs in the world, by  
default to be anycast servers for the EID space.

> deployment document.  I would really need to see the cost coverage  
> for that, as there is an even bigger disconnect than just the issue  
> of increasing traffic without revenue that I asked about relative to  
> EID registrars providing P-ITR services.

How about this practical situation:

(1) New site becomes LISP enabled with a block out of the well-known  
EID-prefix.
(2) No PITRs have to change.

Today, with IPv4 EID-prefixes, they are manually configured in the  
PITRs to inject into the underlying routing system. Or, maybe in the  
future, this gets automated by "redistributing ALT routes, that are  
first highly aggregated, then injected into the DFZ". The latter is  
scary and dangerous but automated. So choose your poison of manually  
configuring all PITRs with a new EID-prefix when sites go LISP versus  
the auto-redistribution risk.

We could do it much better with IPv6 address allocations.

Dino

>> route

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 14 Oct 2010 23:37:56 -0400

Some comments on items 1 and last, in line...

On 10/14/2010 11:18 PM, Dino Farinacci wrote:
>> use a /24, or worst case a /16. That would still leave plenty of
>> remote for organizational delegation to get to assignment to sites of
>> a suitable size block.
>
> Because as you add more, you'll have to change configurations and change
> ACLs in a lot of places. If you get a large enough space that is
> "well-known", the performance of the system will be better.

Sorry, I am not following this.  Whether we get an 8 bit prefix or a 25 
bit prefix to serve as a base for all EIDs seems to result in the same 
amount of configuration for devices which need this information. 
Getting a longer prefix would seem easier, and equally useful for EIDs.
(Or conversely, we will need a much stronger argument to get a 4 bit 
type code than a 24 bit address prefix. Particularly for an experiment.)

>
>> 2) Do we think we can get away with not permitting folks to use their
>> PI IPv6 assignments as EIDs?
>
> inject more routes. IPv6 adoption is in its infancy so we can give sites
> which request a new EID-based PI prefix.
>
>> to advertise the entire EID space, not just a portion of it related to
>> read in the
> Yes, that is the point. You want all the PITRs in the world, by default
> to be anycast servers for the EID space.
>
>> deployment document. I would really need to see the cost coverage for
>> that, as there is an even bigger disconnect than just the issue of
>> increasing traffic without revenue that I asked about relative to EID
>> registrars providing P-ITR services.
>

I think my question must have been unclear.  The deployment document 
suggests that EID registrars might run PITRs for their blocks.  While I 
am concerned about traffic growth, they are selling a service directly 
related to what they are being paid for.  To the degree that more PITR 
traffic is caused by more LISP users in their space, they ahve gotten 
more money for it.

But, for someone to run PITRs that advertise the entire LISP EID space, 
someone has to pay for the increasing traffic load until LISP becomes 
pervasive.  The fact that technically it is much cleaner if their are 
PITRs like this is nice, but won't pay anyones deployment costs.

If we will get PITRs that do such advertising, then it is very clear 
that we want to have a single prefix for the whole EID space.

Yours,
Joel

---

From: Dino Farinacci <[email-masked]>
Date: Thu, 14 Oct 2010 20:44:11 -0700

>>> could
>> change
> Sorry, I am not following this.  Whether we get an 8 bit prefix or a  
> 25 bit prefix to serve as a base for all EIDs seems to result in the  
> same amount of configuration for devices which need this  
> information. Getting a longer prefix would seem easier, and equally  
> useful for EIDs.

We are not getting an 8-bit prefix assigned by IANA. We are spec'ing a  
well-known high-order byte that is permanently assigned by the IPv6  
Addressing RFC.

>>> for
> suggests that EID registrars might run PITRs for their blocks.   
> While I am concerned about traffic growth, they are selling a

But non-registrars can run PITRs as well. And you can count on the  
data-plane SPs to do so more so then control-plane providers, like a  
registrar that may be an "MSP".

> service directly related to what they are being paid for.  To the  
> degree that more PITR traffic is caused by more LISP users in their  
> space, they ahve gotten more money for it.
>
> But, for someone to run PITRs that advertise the entire LISP EID  
> space, someone has to pay for the increasing traffic load until LISP  
> becomes pervasive.  The fact that technically it is much cleaner if  
> their are PITRs like this is nice, but won't pay anyones deployment  
> costs.

SPs want to attack traffic, that is how they justify cost.

Right.

Dino

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 15 Oct 2010 00:04:17 -0400

I must be dense tonight.
A fixed prefix is a fixed prefix.  whether it is allocated by the IETF 
or IANA does not matter.  And, as far as I can tell, for our purposes, 
whether it is 4 bits, 8 bits, or 25 bits does not matter.  (If it were 
longer than 32 bits there might be some argument for it mattering.)
Without regard to where we direct the request, the shorter a prefix we 
ask for, the more justification we will be asked to provide.

With regard to the PITR deployment, I do not buy "ISPs want to attract 
traffic".  At least not when it comes to traffic which they attract, but 
which they then have to hand back off, to someone else.  Such traffic 
incurs costs.  It is possible that the exchange costs with their peers 
will balance.  But they have to have more traffic handling capacity 
inside to handle that.  It costs money.  So why are they going to run a 
service that costs money?  A service which, if it has problems, will 
cause folks who are not their customers to complain about what they do 
to the traffic?
If we believe this deployment of PITRs is likely (or necessary) then
1) We have to say so in the deployment document
2) I believe we will have to explain much more clearly what the 
incentives are for folks doing this.

Yours,
Joel

On 10/14/2010 11:44 PM, Dino Farinacci wrote:
>> am concerned about traffic growth, they are selling a
>

---

From: Dino Farinacci <[email-masked]>
Date: Thu, 14 Oct 2010 21:28:36 -0700

> I must be dense tonight.
> A fixed prefix is a fixed prefix.  whether it is allocated by the  
> IETF or IANA does not matter.  And, as far as I can tell, for our  
> purposes, whether it is 4 bits, 8 bits, or 25 bits does not matter.   
> (If it were longer than 32 bits there might be some argument for it  
> mattering.)
> Without regard to where we direct the request, the shorter a prefix  
> we ask for, the more justification we will be asked to provide.

Yes, from the PITR's perspective when it originates an EID-prefix in  
the DFZ.

No, for an ITR that wants to know, without asking externally, that a  
destination site is an LISP site.

> With regard to the PITR deployment, I do not buy "ISPs want to  
> attract traffic".  At least not when it comes to traffic which they  
> attract, but which they then have to hand back off, to someone

They want to attrack traffic from their customers destined for LISP  
sites that are either their own customers or their peers customers.

Note the idea of PITR deployment is to have them reside on the edges  
of the network, where the coarse EID-prefixes are sent "outward" and  
not into the core. So they reside in POPs and send the routes toward  
the PE routers.

> else.  Such traffic incurs costs.  It is possible that the exchange  
> costs with their peers will

The traffic is roughly on path.

> balance.  But they have to have more traffic handling capacity  
> inside to handle that.  It costs

They do not. Traffic that comes to them from their sources can be to  
LISP or non-LISP sites. The rate the PITRs get traffic is the rate the  
customer can inject traffic, regardless of the destination type.

> money.  So why are they going to run a service that costs money?  A  
> service which, if it has problems, will cause folks who are not  
> their customers to complain about what they do to the traffic?

They want to provide connectivity to their LISP customers, who want to  
multi-home. So the SP will want to have the PITR encapsulate to the  
ETRs that the LISP customer sites.

> incentives are for folks doing this.

Agree.

Dino

>>> While I

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 15 Oct 2010 08:57:36 -0400

The discussion of PITR deployment, and the creation of an EID base for 
IPv6, raised an interesting quesiton in my head.
Do we need to specify who will function as the top in that case?  If 
there is a single EID prefix, then it seems that someone has to be 
authorized to give out EIDs from that space.  According to some rules. 
WHo do we give that job to?

And if we give that job to someone, do they have to run some ALT 
routers, or does it suffice for them to require that all entities which 
receive direct allocations from that agency must run ALT routers and 
must peer with each other? (We have to ensure the ALT is a connected 
graph, I think.)

Yours,
Joel

PS: Yes, these are operational considerations.  But it seems that if we 
are going to want a common base for v6 EIDs (which seems a good idea to 
me personally) then we need to deal with this.

---

From: Dino Farinacci <[email-masked]>
Date: Fri, 15 Oct 2010 07:56:01 -0700

> The discussion of PITR deployment, and the creation of an EID base  
> for IPv6, raised an interesting quesiton in my head.
> authorized to give out EIDs from that space.  According to some  
> rules. WHo do we give that job to?

Joel, let me offer a suggestion. Can we put this as an agenda item for  
Beijing and have an open discussion about this? I think we all in the  
room, it will be a higher bandwidth channel for communication.

> routers, or does it suffice for them to require that all entities  
> which receive direct allocations from that agency must run ALT  
> routers and must peer with each other? (We have to ensure the ALT is  
> a connected graph, I think.)

Everything is decoupled. A PITR just needs two GRE tunnels to the ALT  
system to anywhere there is an ALT router. A PITR can also use a Map- 
Resolver as well so it is not required to be attached to the ALT.

Who assigns EID-prefixes, who provides map-server services for EID- 
prefixes, who allocates PA-based RLOCs, and all independent from each  
other. That is the huge benefit the level of indirection brings. You  
can change anyone without changing the other.

Dino

>
> Yours,
> Joel
>
> PS: Yes, these are operational considerations.  But it seems that if  
> we are going to want a common base for v6 EIDs (which seems a good  
> idea to me personally) then we need to deal with this.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 15 Oct 2010 11:35:32 -0400

I agree that this needs more discussion.  I sent the note to raise the 
issue, not expecting a quick or simple answer.

(Terry will be chairing the session on Friday at the IETF meeting as I 
have to be on a plane back for family reasons.)

I was not asking in this quesiton about who runs the PITRs.  The 
decoupling there is useful and understandable.
I was asking about the EID side.  The ALT needs to be connected. 
Someone has to run ALT routers for any portion of the EID space that is 
allocated.  And those routers have to have paths to each other.  In the 
Internet, this works by accident by having enough BIG players to hold 
things together.  The initial stage was handled by having a mandated 
central infrastructure.  I see some land mines hidden in this space that 
could cause serious problems at the point where this starts to matter. 
(As a small experiment, it just works.)

Yours,
Joel

On 10/15/2010 10:56 AM, Dino Farinacci wrote:
>> which receive direct allocations from that agency must run ALT routers
>> and must peer with each other? (We have to ensure the ALT is a
>> connected graph, I think.)
>
> system to anywhere there is an ALT router. A PITR can also use a
> Map-Resolver as well so it is not required to be attached to the ALT.
>
> Who assigns EID-prefixes, who provides map-server services for
> EID-prefixes, who allocates PA-based RLOCs, and all independent from
> each other. That is the huge benefit the level of indirection brings.
> You can change anyone without changing the other.
>
> Dino
>
>>
>> Yours,
>> Joel
>>
>> are going to want a common base for v6 EIDs (which seems a good idea
>> to me personally) then we need to deal with this.

---

From: Dino Farinacci <[email-masked]>
Date: Sat, 16 Oct 2010 05:48:50 -0700

> Someone has to run ALT routers for any portion of the EID space that  
> is allocated.  And those routers have to have paths to each other.   
> In the Internet, this works by accident by having enough BIG players  
> to hold things together.  The initial stage was handled by having a  
> mandated central infrastructure.  I see some land mines hidden in  
> this space that could cause serious problems at the point where this  
> starts to matter. (As a small experiment, it just works.)

Agree, this needs discussion and hopefully on a non-Friday afternoon  
session.  ;-)

Dino