Subject: Re: [lisp] EID allocation / ALT base

From: [email-masked] (Noel Chiappa)
Date: Tue, 19 Oct 2010 22:06:04 -0400 (EDT)

> From: "Flinck, Hannu (NSN - FI/Espoo)" <[email-masked]>

    > I think that building on top of "administratively aggregatable"
    > allocation system of the EID space is going to lead to as rigid outcome
    > as the current IANA and RIR allocated addressing solution is.
    > ...
    > Allocation boundaries will degenerate as the networks keep on evolving
    > to meet the real life needs (many times unforeseeable new use cases).

Perhaps... but asking everyone to first renumber into some sort of optimal
allocation scheme, and then stay there, is I think not workable. We just have
to build a system that can deal with scattered allocations.

    > If the ALT routing systems scalability depends exactly on how the
    > original aggregatable EID allocation was done

Your comment co-mingles two separate issues. One is aggregability of LEIDs in
general for purposes of resolution (which I think is a concern); and the
second is aggregability issues which are specific to ALT. The latter is not a
concern to me, because I suspect we will want to replace the ALT long before
LISP gets widely deployed (if LISP does get there), because the ALT has a
number of issues. And needless to say, discarding the ALT means there is no
need to 'solve' any ALT-specific problems.

(As I keep saying, but people seem to keep not hearing, I think the ALT was a
really clever way to build a distributed mapping system without having to
write a whole bunch of new code, but it definitely has some aspects that make
it not the best choice for a _large_ distributed mapping system. So I expect
it to be replaced 'soon' - so why people keep paying any attention at all to
its problems is completely beyond me. There is at least one design out there
for an alternative - LISP-TREE - and the Barcelona guys did some great work,
documented in their paper "LISP-TREE: A DNS Hierarchy to Support the LISP
Mapping System", modelling various approaches, to give a lot of backup to why
that design is a good direction to go. I don't think we'll wind up with that
design exactly - using DNS has some issues - but something like that, yes.)

    > If EIDs were truly PI addresses as touted then binding them into
    > aggregatable (or topological) allocations would restrict the
    > applicability of the independence of these addresses

As to the more interesting point, aggregability of LEIDs in general for
purposes of resolution (i.e. looking up mappings), there are unvoidable
fundamentals.

The completely unrelated organizations A and B may already have been assigned
LEID's E and E+1 by the RIRs, and the question is 'how do you aggregate them
for {LEID->RLOC} binding lookup in a way that scales, but also has other good
properties that people want, like not being tied into a single binding
provider' (assuming, of course, that neither E not E+1 wants to renumber).

We can do scaling, but at the cost of requiring the holder of E+1 to deal
with the same registry as the holder of E - this is similar to the current
DNS situation, where all holders of .COM DNS names have to deal with
Verisign. (Effectively, albeit at one level of organizational remove; since
they interact with registrars, who interact with the registry.)

We have previously discussed (sorry, too lazy to find it in the WG archive)
the possibility of allowing alternate registries, so that if you don't like
registry A, you can deal with registry B (much as in the DNS you can specifiy
more than one authoritative server for a given zone), but it's unclear
whether we will need to do this - the Internet seems to work quite OK with
only one registry for .COM.

Yes, this means that LEID's have some aspect of 'connectedness' to totally
unrelated LEID's, because of the 'aggregation' requirement. However, that's
unavoidable - it's an artifact of the way the names are assigned. We have to
be able to aggregate them in the _directory_, but that doesn't imply any
properties other than that. I mean, abb.com and abc.com have nothing to do
with one another either, right - even though they are 'next to' each other?

The choice is simple, if painful: we have large namespace where we want to be
able to do a directory (look up mappings, really), and we want that directory
to be distributed (as opposed to one giant flat directory). So if we want to
divide it up, to distribute the directory, _and_ we have to take what the
_existing_ allocation of names in that namespace is, we wind up with a
situation where unrelated things share a directory branch.

Would you rather have a giant flat directory?

    > The routing system would become more durable if we would allow
    > reshuffling of EIDs with their aggregation points.

I don't see how. The issue is simply 'how do we build a distributed
resolution system, given names which may be contiguous in a namespace, but
are otherwise unrelated'. That mapping step is completely unrelated to
everything else. We need to make sure that the mapping system has the
properties we need (in terms of speed, robustness, etc as well as user issues
like cost, provider lock-in, etc), yes, but we simply need to work out what
those required properties are, and then we can design the mapping system to
meet them.

    > The xTRs cloud select with the help of Mapping Servers the "optimal"
    > Map Server where to register its EIDs. This selection should be such
    > that it facilitates locality as well as aggregation in a dynamic way.

This seems to be in the context of ALT? Again, I suggest you look at
something like LISP-TREE, to see how an alternative distributed mapping
system would work, and then see if there is some part of your concern that's
generic, and not ALT-specific.

    > In fact, the question for resolving an EID into an RLOC is the one of
    > which of the Mapping Servers know the locator of the EID or the locator
    > of the one who knows the locator of the EID. This doesn't require any
    > fixed allocation of EIDs or grouping of EIDs. (We wrote about this in
    > our proposal for compact routing based mapping system. ... )

It's certainly possibly to build a distributed mapping system in which any
particular mapping can be on any arbitrary server, and there is some
algorithm (along with data) to find it. I'm not sure we need - or want - to
go in that direction, though. I'd want to see more discussion about what the
advantages, and costs, are of such an approach. In particular, is it really
that much more robust, and is any extra robustness worth what is certain to
be a fair amount of extra complexity and overhead (and perhaps slowness too).

    > I would even go so far that the end systems cloud generate their own
    > EIDs (self certified EID would be preferable) and try to register that
    > into the mapping system that ensures that each registered EID is
    > unique.

For LEID's which are either HIP EIDs, or perhap IPvp6 addresses, that might
work. The IPv4 space is not big enough to do that, and very-high quality
support of IPv4 is a necessity. (And no, I don't want to debate that point.)

    > In other words doesn't allow duplicate registrations in the mapping
    > system.

It is one thing to find _a_ mapping _somewhere_ in a distributed database
where mappings are kept on arbitrary servers, _if_ such a mapping exists.
It's something entirely different to basically guarantee that there is no
such mapping _anywhere_ in such a distributed database - at least without
even more overhead, complexity, etc.

    > The issue of using EIDs in the intardomain routing is a separate issue,
    > but easily solvable ... No real issue there.

I'm really at a loss to understand what you mean here, becase if LEID's are
allocated without regard to actual network connection location, routing
tables are going to grow even faster...

	Noel

---

From: Vince Fuller <[email-masked]>
Date: Wed, 20 Oct 2010 00:02:30 -0700

> need to 'solve' any ALT-specific problems.

People seem to be forgetting that the ALT is a logical/virtual network that
is composed of tunnels. If a particular EID block needs to move, it is a
fairly simple matter to reconfigure the tunnel topology or change the Map
Server registration so that the EID block can still be aggregated; none of
the underlying switching topology needs to be touched.

Yes, moving an EID block from an ETR in Sydney to an ETR in London means
that their will be additional latency between the London ETR and a Map
Server (or other ALT aggregation point) near Australia but that additional
latency is limited to a few hundred milliseconds and is only applicable when
doing a database looking (sending a Map Request and receiving a Map Reply,
which occurs only when an ITR needs to find the ETR for the first time). Once
the EID-to-RLOC mapping has been learned by an ITR that needs to communicate
with the ETR, there will be no added latency on the data path between them.

	--Vince

---

From: Dino Farinacci <[email-masked]>
Date: Wed, 20 Oct 2010 11:02:57 -0700

> the underlying switching topology needs to be touched.

I'd like to add to what Vince is saying.

If a site gets an EID-prefix allocation called A, that prefix will  
indicate where the site registers the EID-prefix. Remember, the ALT  
topology follows addressing, unlike the underlying network today.

If that same site needs more EID address space later and gets an  
allocation called B, and B is not in a common power-of-2 block with A,  
then the site will register to another map-server that is aggregating  
for that B space.

In this case, there is no tunnel rehoming on the ALT.

I'd also like to mention that the "ALT" is a logical topology. It is  
spec'ed to run BGP over GRE tunnels. But the way we have implemented  
this, the ALT runs in another VRF. And that VRF can run over any  
physical interface, logical interface, tunnel, VLAN, etc. and any  
routing protocol can be used on those VRF-specific interfaces. We will  
find these flexible options will be used by enterprises who will want  
to run a private mapping database system. And one form of this is no  
ALT at all, just map-servers configured as map-resolvers.

> with the ETR, there will be no added latency on the data path  

And there are many both protocol and implementation techniques to  
reduce how often Map-Requests are sent to the mapping database system.

Dino

>
> 	--Vince
> _______________________________________________
> lisp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/lisp

---

From: "Flinck, Hannu (NSN - FI/Espoo)" <[email-masked]>
Date: Fri, 22 Oct 2010 17:01:31 +0300

Dino, Vince and Noel

I agree what Vince and Dino are saying. No issue there. This shows the
flexibility of ALT. 

And I agree with Noel that that we can not ask everyone to renumber
first. 
I have noted the temporary nature of ALT. However, the operational
practices that ALT reuses (= ISPs know how to operate BGP) are not that
temporary I am afraid. This because that even if the new ways promise
benefits it requires retraining and changes of the supporting systems. 

Noel wrote:

>We can do scaling, but at the cost of requiring the holder of 
>E+1 to deal with the same registry as the holder of E - this 
>is similar to the current DNS situation, where all holders of 
>.COM DNS names have to deal with Verisign.

In the example you give, yes, but it can not be generalized to a case
where there are many unrelated organizations, not only some (like A and
B in your example). 
Let's have n (n is large) independent organizations picking up at random
LEIDs. Now each LEIDs needs to be registered into the distributed
database. All selections of the registers that hold a piece of content
of the full distributed database can be independent from each other. For
example, you may use hashing that randomizes the selection (this is what
the DTHs do), or you may try to locate the most responsive one (maybe
based on the closeness, say using anycast). 

What I was pointing out was the selection of the registers if that is
more descriptive word for what I called as "aggregation points" 

Noel wrote:

>I don't see how. The issue is simply 'how do we build a 
>distributed resolution system, given names which may be 
>contiguous in a namespace, but are otherwise unrelated'.

I think that Vince and Dino gave in their responses a good example how:

Vince wrote:

"If a particular EID block needs to move, it is a fairly simple matter
to reconfigure the tunnel topology or change the Map Server registration
so that the EID block can still be aggregated; none of the underlying
switching topology needs to be touched."

I am  noting that if you can move the EID blocks move them so that the
mapping system will able to scale well. Compact routing is a very good
bench mark for scalability both in terms of the size of the routing
table as well as stretch. (Here need to refer to the RRG write up about
the "Compact routing based mapping system" for the details at this
point.)

Finally I agree with Noel that "... discarding the ALT means there is no
need to 'solve' any ALT-specific problems." 

Best regards
Hannu

>-----Original Message-----
>From: [email-masked] [mailto:[email-masked]] On 
>Behalf Of ext Dino Farinacci
>Sent: Wednesday, October 20, 2010 21:03
>To: Vince Fuller
>Cc: Noel Chiappa; [email-masked]
>Subject: Re: [lisp] EID allocation / ALT base
>
>>>> allocation system of the EID space is going to lead to as rigid 
>>>> outcome as the current IANA and RIR allocated addressing solution 
>>>> is.
>>>> ...
>>>> Allocation boundaries will degenerate as the networks keep on 
>>>> evolving to meet the real life needs (many times unforeseeable new 
>>>> use cases).
>>>
>>> Perhaps... but asking everyone to first renumber into some sort of 
>>> optimal allocation scheme, and then stay there, is I think not 
>>> workable. We just have to build a system that can deal with 
>scattered 
>>> allocations.
>>>
>>>
>>> Your comment co-mingles two separate issues. One is 
>aggregability of 
>>> LEIDs in general for purposes of resolution (which I think is a 
>>> concern); and the second is aggregability issues which are specific 
>>> to ALT. The latter is not a concern to me, because I 
>suspect we will 
>>> want to replace the ALT long before LISP gets widely deployed (if 
>>> LISP does get there), because the ALT has a number of issues. And 
>>> needless to say, discarding the ALT means there is no need 
>to 'solve' 
>>> any ALT-specific problems.
>>
>> People seem to be forgetting that the ALT is a 
>logical/virtual network 
>> that is composed of tunnels. If a particular EID block needs 
>to move, 
>> it is a fairly simple matter to reconfigure the tunnel topology or 
>> change the Map Server registration so that the EID block can 
>still be 
>> aggregated; none of the underlying switching topology needs to be 
>> touched.
>
>If a site gets an EID-prefix allocation called A, that prefix 
>will indicate where the site registers the EID-prefix. 
>Remember, the ALT topology follows addressing, unlike the 
>underlying network today.
>
>If that same site needs more EID address space later and gets 
>an allocation called B, and B is not in a common power-of-2 
>block with A, then the site will register to another 
>map-server that is aggregating for that B space.
>
>I'd also like to mention that the "ALT" is a logical topology. 
>It is spec'ed to run BGP over GRE tunnels. But the way we have 
>implemented this, the ALT runs in another VRF. And that VRF 
>can run over any physical interface, logical interface, 
>tunnel, VLAN, etc. and any routing protocol can be used on 
>those VRF-specific interfaces. We will find these flexible 
>options will be used by enterprises who will want to run a 
>private mapping database system. And one form of this is no 
>> Yes, moving an EID block from an ETR in Sydney to an ETR in London 
>> means that their will be additional latency between the 
>London ETR and 
>> a Map Server (or other ALT aggregation point) near Australia 
>but that 
>> additional latency is limited to a few hundred milliseconds and is 
>> only applicable when doing a database looking (sending a Map Request 
>> and receiving a Map Reply, which occurs only when an ITR 
>needs to find 
>> the ETR for the first time). Once the EID-to-RLOC mapping has been 
>> learned by an ITR that needs to communicate with the ETR, there will 
>> be no added latency on the data path between them.
>
>And there are many both protocol and implementation techniques 
>to reduce how often Map-Requests are sent to the mapping 
>database system.
>
>Dino
>
>>
>> 	--Vince

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 22 Oct 2010 10:11:17 -0400

You refer to an organization picking some random LEID.
I do not believe that is contemplated.
The following is what I understand to be our intentions.

In IPv4, for organizations which have PI space, they can use that for 
LEIDs.  However, the general case will be that organizations request an 
assignment of an LEID.  That assignment will come with a (list of) ALT 
participants who can maintain the mapping information in the ALT for the 
provider.

For the case where an organization has PI that they are using, they will 
either have to register with someone already handling the superblock in 
the ALT, or they will have to arrange (either by running, or paying 
someone to run) for suitable devices to handle that block in the ALT.

In general, the structure is that the ALT structure will mirror the 
address allocation structure.  Even in IPv4, it is presumed that the 
larger number of entities would use assigned EiDs rather than using 
their PIs.  (If the only folks needing LISP were on the order of the 
current PI space we would not be needing to solve an address space 
GROWTH problem.)

There are other cases, outside of the purview of the WG, where other ALT 
operational structures, such as privately operated sets for VPNs, will 
obtain.

Yours,
Joel

On 10/22/2010 10:01 AM, Flinck, Hannu (NSN - FI/Espoo) wrote:
>>>>> From: "Flinck, Hannu (NSN - FI/Espoo)"<[email-masked]>
>>>>

---

From: "Flinck, Hannu (NSN - FI/Espoo)" <[email-masked]>
Date: Fri, 22 Oct 2010 17:31:00 +0300

Hello Joel

I know it is not contemplated. But it might be short sighted to build
from hierarchically administrated allocation structure, though.
 
>In general, the structure is that the ALT structure will 
>mirror the address allocation structure.

Yes, this is fine as long as the allocations are based on virtual
topology that can be changed (or more accurately the
registrar/aggregation points can be changed) if and when the system
takes off and starts to evolve. This is how I understood Vince's mail.  

Best regards
Hannu

>From: ext Joel M. Halpern [mailto:[email-masked]] 
>Sent: Friday, October 22, 2010 17:11
>To: Flinck, Hannu (NSN - FI/Espoo); [email-masked]
>In IPv4, for organizations which have PI space, they can use 
>that for LEIDs.  However, the general case will be that 
>organizations request an assignment of an LEID.  That 
>assignment will come with a (list of) ALT participants who can 
>maintain the mapping information in the ALT for the provider.
>
>For the case where an organization has PI that they are using, 
>they will either have to register with someone already 
>handling the superblock in the ALT, or they will have to 
>arrange (either by running, or paying someone to run) for 
>suitable devices to handle that block in the ALT.
>
>In general, the structure is that the ALT structure will 
>mirror the address allocation structure.  Even in IPv4, it is 
>presumed that the larger number of entities would use assigned 
>EiDs rather than using their PIs.  (If the only folks needing 
>LISP were on the order of the current PI space we would not be 
>needing to solve an address space GROWTH problem.)
>
>There are other cases, outside of the purview of the WG, where 
>other ALT operational structures, such as privately operated 
>sets for VPNs, will obtain.
>
>Yours,
>Joel
>
>> practices that ALT reuses (= ISPs know how to operate BGP) are not 
>> that temporary I am afraid. This because that even if the new ways 
>> promise benefits it requires retraining and changes of the 
>supporting systems.
>>
>> Noel wrote:
>>
>>> is similar to the current DNS situation, where all holders of .COM 
>>> DNS names have to deal with Verisign.
>>
>> where there are many unrelated organizations, not only some (like A 
>> and B in your example).
>> Let's have n (n is large) independent organizations picking up at 
>> random LEIDs. Now each LEIDs needs to be registered into the 
>> distributed database. All selections of the registers that hold a 
>> piece of content of the full distributed database can be independent 
>> from each other. For example, you may use hashing that 
>randomizes the 
>> selection (this is what the DTHs do), or you may try to locate the 
>> most responsive one (maybe based on the closeness, say using 
>anycast).
>>
>>> resolution system, given names which may be contiguous in a 
>>> namespace, but are otherwise unrelated'.
>>
>> to reconfigure the tunnel topology or change the Map Server 
>> registration so that the EID block can still be aggregated; none of 
>> the underlying switching topology needs to be touched."
>>
>>
>> table as well as stretch. (Here need to refer to the RRG write up 
>> about the "Compact routing based mapping system" for the details at 
>> this
>> point.)
>>
>> Finally I agree with Noel that "... discarding the ALT means 
>there is 
>> no need to 'solve' any ALT-specific problems."
>>
>>
>> Best regards
>> Hannu
>>
>>
>>> From: [email-masked] [mailto:[email-masked]] 
>On Behalf 
>>> Of ext Dino Farinacci
>>> indicate where the site registers the EID-prefix.
>underlying 
>>> network today.
>>> allocation called B, and B is not in a common power-of-2 block with 
>>> A, then the site will register to another map-server that is 
>>> aggregating for that B space.
>>>
>>> implemented this, the ALT runs in another VRF. And that VRF can run 
>>> over any physical interface, logical interface, tunnel, VLAN, etc. 
>>> and any routing protocol can be used on those VRF-specific 
>>> interfaces. We will find these flexible options will be used by 
>>> enterprises who will want to run a private mapping database system. 
>>> And one form of this is no ALT at all, just map-servers 
>configured as 
>>> map-resolvers.
>>>