Subject: [RRG] Re: LISP-CONS Default Mapper = Re-Encapsulating Tunnel Router

From: Robin Whittle <[email-masked]>
Date: Fri, 19 Oct 2007 22:04:01 +1000

Hi Dino,

You wrote:

>> This requires the Default Mapper to be an ITR with a full copy of
>> the database - which means you need to use NERD or some other
>> approach like eFIT-APT's or Ivip's to get the updates there.
>
> It could have the full database, it could use any of the mapping
> database schemes already designed, or it could point to another
> default mapper.

OK - I will call an ITR such as this a "RETR" - Re-Encapsulating
Tunnel Router.  According to your message:

  http://psg.com/lists/rrg/2007/msg00476.html

if LISP is deployed with RETRs, ordinary ITRs are configured to send
packets (via a conventional LISP tunnel) to "the default mapper"
(RETR) if the ITR had no mapping for them.

This is a completely changed behavior for the entire LISP system,
since ITRs now will not drop packets.  In your initial message, you
wrote "the default mapper" (RETR) and I assumed there was just one
of these.  So I assumed there was one such RETR, able to tunnel
every packet to an ETR, and that therefore it must have the full
database.

However, you also wrote that if this "default mapper" (RETR)
couldn't tunnel the packet to the proper ETR, it would tunnel it to
"the next mapper along the way to the destination".  I assume this
means: "the next default mapper (Re-Encapsulating Tunnel Router)
along the way to the destination".

If I had focussed on that properly, I would have realised that "the
default mapper" (RETR) didn't necessarily have a full database.

I understand that in the form of LISP with RETRs which you are
describing, ordinary LISP ITRs are caching ITRs - so I think they
resemble eFIT-APT ITRs and Ivip ITRCs.

In your latest message you write that a RETR could have the full
database, so I will refer to two classes of RETR: RETRC (caching
only) and RETRD (full database).

I understand that if a RETRC has another RETR to tunnel packets to
when it has no mapping for the packet, then there is a chain of one
or more RETRs the RETRC can use.

Both RETRCs and ordinary ITRs in this RETR form of LISP are able to
tunnel a packet they have no mapping for to another RETR.

However a RETRC (or the RETRC function of a router which also
functions as an ITR) accepts incoming packets from tunnels, whereas
an ITR accepts raw packets.

Here are some scenarios.  SH is sending host. --- is ordinary
packets.  ==== is tunneled packets:

  (SH)----(ITR1)======(RETRC2)

I don't know if this is part of your arrangement, but this is a
caching RETR with no further RETRs.  The idea would be that RETRC2
may have the needed mapping information and so can tunnel the packet
to the ETR.  If not, the packet will be dropped.

  (SH)----(ITR1)======(RETRC2)======(RETRC3)======(RETRD4)

In this arrangement, the packet might be tunnelled by any one of
ITR1, RETRC2 or RETRC3.  But if none of these have the mapping
information cached, the packet will surely be tunneled correctly to
the ETR by the full database RETRD4.

> If the later, the design is looking like Paul Francis' CRIO which
> fundamentally allows you to reduce table size at the expense of
> stretch. Traditional state/bandwidth tradeoff.

I think CRIO (http://www.ieee-icnp.org/2006/papers/s4a4.pdf) is less
similar to my understanding of the use of LISP with RETRs than is
eFIT-APT or Ivip, since these two, like LISP, are intended to be
incrementally deployable without overall changes in addressing
and/or the capabilities of DFZ routers.

>> If you are doing that, then why not make the ITR or some nearby
>> server a query server for local caching ITRs?  This would be
>> faster than the caching ITRs waiting for the global CAR-CDR
>> network to respond.  Then the architecture would resemble
>> eFIT-APT or Ivip.
>
> I don't understand. And can you explain *briefly*.

I was assuming that a RETR would have a full database - a RETRD -
and that the RETRD would be located in the same ISP's network as the
other ITRs.  I assumed therefore that the RETRD receives a complete
feed of database updates by some system other than CONS - which I
understand is a query-based system and so is not designed to
broadcast all the changes which occur in a mapping database.

In this case, with a copy of the full database in the RETRD
somewhere in the ISP's network, why would any of the ITRs in that
network need to use the global CAR-CDR network?  That would be much
slower and less reliable than using some query system which got
results directly from the nearby RETRD.

Now I understand that RETRs don't necessarily have the full
database, and that while some do, the chain of RETRs may not end
with a full database RETRD - in which case, the system may still
drop packets.

>> As long as your Default Mapper is in the same ISP's network, you
>> are remaining true to the LISP-CONS architecture and the system
>> should never drop or delay a packet.  This puts it in good
>> company with LISP-NERD (sorry about my mistaken statements
>> recently that it relied on DNS and dropped packets), eFIT-APT and
>> Ivip.
>
> I don't understand what you are saying.

All the ITRs in LISP-CONS, as currently defined, are caching ITRs.
If they get a packet they have no mapping information for, they drop
it.  Likewise TRRP ITRs.  So a LISP-CONS or TRRP system drops
packets from sending hosts in upgraded networks, requiring sending
hosts to time-out and try again, leading to problems such as I
explore in detail in:

  http://psg.com/lists/rrg/2007/msg00462.html

I would appreciate it if you looked at the example there and
commented on how realistic you think it is.  I think it is probably
a somewhat worse than average example, but not worst case or totally
unrealistic.

LISP-NERD, eFIT-APT and Ivip do not drop any packets sent by hosts
in upgraded networks.  LISP-CONS and TRRP do.

Anyone using address space mapped by LISP-CONS or TRRP would find
that communication sessions are significantly slower to establish.
I think we should only accept such degradation for the new Internet
architecture if it can be shown that all alternative proposals which
do not drop packets are in some way unworkable.

>> However, as long as the Default Mapper is within the same ISP's
>> network as the caching ITRs which feed it, then it doesn't solve
>> the incremental deployment problem of sending hosts in
>> non-upgraded (having no LISP ITRs) networks being unable to send
>> packets to any host with a LISP-mapped address.
>
> I don't understand what you are saying, your first paragraph says
> "within the same ISP's network" and your second paragraph says
> "however, within the same ISP's network".  So I don't know which
> explanation is for which polarity.  ;-)

In my initial understanding of your LISP with RETRs, the system will
not drop packets from sending hosts in any LISP-upgraded network.
But that was on the assumption that the RETR had a full database -
which I now recognise may not always be the case.

This or any other arrangement which stops LISP-CONS from dropping
packets would remove one major objection to LISP-CONS.

The "However" refers to a second problem which would not be solved
by my understanding of LISP with RETRs.  The RETR form of LISP - at
least when the RETR (I was assuming a solitary full database RETRD)
was in the same same ISP's network as the ITRs which tunnel packets
to it - would not solve another major objection to LISP: that it is
not incrementally deployable due to hosts in non-upgraded networks
being unable to send packets to receiving hosts with LISP-mapped
addresses (EIDs).

My reasoning is that, assuming that each LISP-RETR-equipped ISP
network had its own RETRD, and assuming that the network's ITRs and
one or more RETRs do not accept any packets from sending hosts in
non-LISP-equipped ISP networks, then there is still the major
problem of hosts with LISP-mapped EID addresses not being reachable
from sending hosts in non-LISP-equipped networks.

I could imagine this problem being fixed by making your one or more
RETRDs accept packets from non-upgraded networks:

>> To fix that, you could follow the path Eliot took and deploy one
>> or more ITRs, which know how to tunnel the packets, to which
>> un-tunnelled packets flow from all those non-upgraded networks.
>>
>>    http://psg.com/lists/rrg/2007/msg00260.html
>>    http://psg.com/lists/rrg/2007/msg00264.html
>>    http://psg.com/lists/rrg/2007/msg00288.html
>>
>> If you have a few of them, as Eliot suggested, then this much the
>> same as the Ivip "anycast ITRs in the core" approach.  Then, I
>> think, LISP-CONS would then be potentially incrementally
>> deployable.
>
> The statement saying "LISP-CONS would then be potentially
> incrementally deployable" does not make any sense to me. Old sites
> don't know how to do LISP so they don't use CONS. New sites that
> do use LISP use CONS.

As long as non-LISP sites cannot get packets to an ITR, then sending
hosts in those sites will not be able to reach any host which has a
LISP-mapped address.  This is because, in my understanding of LISP
in general and LISP-CONS in particular (not counting Eliot's
suggestion, which I assume applies only to LISP-NERD), the address
space which is used for EIDs is not advertised in BGP.

As is widely recognised, this creates such a strong disincentive for
anyone considering using a LISP-mapped address (loss of
communication with hosts in all sites which have not implemented
LISP ITRs) that LISP would not be incrementally deployable.

My suggestion was that if you pursued the Re-Encapsulating Tunnel
Router approach, with full databases (actually, you would need to
make these RETRDs accept raw packets, not just encapsulated packets
from ITRs and other RETRs) and combined it with something like Eliot
suggested, you could solve this major problem with LISP-CONS and so
make it incrementally deployable.

Then in this form of LISP there would be no loss of connectivity
from sites which had not yet been upgraded.  Currently, this is true
only of Ivip and Eliot's modified LISP-NERD approach.

As I understand it, eFIT-APT has a major problem with loss of
connectivity.  This seems to have been acknowledged by one of its
proponents recently:

  http://psg.com/lists/rrg/2007/msg00446.html

> Let's don't turn this discussion, again, into how can LISP sites
> talk to non-LISP sites. This thread is about "packet loss during
> mapping cache misses".

We can discuss whatever we like.  You don't have to discuss
LISP-CONS' current inability to ensure reachability from
non-upgraded sites.  I was suggesting a way you could alter LISP to
solve this problem.

Here is my understanding of the possibilities regarding solving the
problem of dropped packets and consequent reliance on sending host
time-outs for communications to proceed.  These points do not
concern the problem of loss of reachability from non-upgraded networks.

If you extend LISP-CONS with caching-only RETRs (RETRCs) then:

  1 - You may reduce the number of dropped packets, depending
      on what extra mapping information the RETRCs have which
      the initial ITR didn't have.

  2 - You don't need to get a full feed of mapping database
      updates to the LISP-enabled site - so the CDR-CAR
      CONS network remains unchanged.

If you extend LISP-CONS with full database RETRDs then:

  1 - You completely eliminate the problem of dropping packets
      from sending hosts in networks which are upgraded with
      this form of LISP.

  2 - You need to devise a method of getting database updates
      to the RETRDs.  Other proposals which have a mechanism
      for this are eFIT-APT (piggybacked on BGP), LISP-NERD
      (ITR requests updates via HTTP from centralised servers
      or perhaps from closer ITRs) or Ivip (I am still working
      on the ambitious Replicator system).

  3 - For any LISP site which has a RETRD, I can't see why
      you would need CONS (the ambitious global CDR-CAR
      network) since this would be slower and less reliable
      than devising a method for local ITRs to get their
      queries answered by the nearby RETRD.

  - Robin

--
to unsubscribe send a message to [email-masked] with the
word 'unsubscribe' in a single line as the message text body.
archive: <http://psg.com/lists/rrg/> & ftp://psg.com/pub/lists/rrg

---

From: Dino Farinacci <[email-masked]>
Date: Fri, 19 Oct 2007 11:39:20 -0700

> This is a completely changed behavior for the entire LISP system,

No, it was always an option. It's the type of mapping system one uses  
that dictate if a packet gets dropped or not.

> and/or the capabilities of DFZ routers.

The reference to CRIO only was for the state/stretch tradeoff.

> results directly from the nearby RETRD.

The ITRs wouldn't need to use the CAR-CDR network directly if they  
had a mapping. The CONS spec says an ITR only sends a Map-Request  
when it gets a cache miss. But if the ITR had 0.0.0.0/0 in it's  
mapping database, it would never have a cache miss.

> space which is used for EIDs is not advertised in BGP.

Okay, so we are changing the subject of this thread.

This was brought up at Nanog as well. Let me put it simply:

o Of course non-LISP site cannot get access to a LISP site if it  
withdraws it's PI
   prefix out of the routing system.

o Of course a non-LISP site today cannot get access to another non- 
LISP site if it
   withdraws it's prefix from the routing system.

So, it's not that LISP is forcing this. We all want to withdraw  
routes so the only way
to get packets to a LISP site from a non-LISP site is to have the  
packets hit an ITR
some where on the path to the LISP site. That could be a proxy ITR  
that does this. Or the LISP site can be smart enough to use an EID  
that *is in* the routing system. Such as a PA address that will  
always continue to be there.

We are working out the details on how to get a LISP site to talk to a  
non-LISP site. In our test rig we have it working already. It is very  
simple by using the following rule:

If the ITR has a cache mapping for a destination site, use it and  
encapsulate the packet with a LISP header. If the ITR does not have a  
cache mapping for a destination site, it does not encapsulate.

That is how I (at cisco) debug the LISP boxes that DaveM/Vince/Darrel/ 
Shep have deployed. They are LISP sites and cisco is not one.

Dino

---

From: Robin Whittle <[email-masked]>
Date: Sun, 21 Oct 2007 13:44:04 +1000

Hi Dino,

I would appreciate you and other people commenting on my packet loss
and sending host time-out example:

>> This is a completely changed behavior for the entire LISP system,

I meant that this is very different from LISP-CONS without
full-database Re-Encapsulating Tunnel Routers - because all packets
are tunnelled to the ETR, whereas without such full-database RETRs,
packets are dropped when there is no mapping information at the
caching ITR.

> mapping database, it would never have a cache miss.

Yes, but my suggestion is that if you have the full mapping database
in a local full-database RETR, then it would make sense to use that
(or a nearby server which also gets the full feed of updates too) as
a local, fast, ~100% contactable, query server.  This means you
don't need the CDR-CAR system.

>   withdraws it's PI prefix out of the routing system.
>
> o Of course a non-LISP site today cannot get access to another
>   non-LISP site if it withdraws it's prefix from the routing
>   system.
>
> So, it's not that LISP is forcing this.

I agree.  I think Eliot was contemplating keeping the prefixes
advertised in perpetuity in order to enable full reachability from
non-upgraded networks for LISP-NERD-mapped addresses.

> We all want to withdraw routes . . .

Not me - and maybe not Eliot.  I would be happy for any of the
ITR-ETR proposals to work with the same or a moderately higher
number of BGP advertised prefixes, as long as, in general, each such
prefix serves the needs of multiple end-users who would try and
probably succeed in getting a prefix if there was no ITR-ETR system
to serve their needs for multihoming and portable address space.
(Apologies to those who wince at this term.)

> . . . so the only way to get packets to a LISP site from a
> non-LISP site is to have the packets hit an ITR some where on the
> path to the LISP site. That could be a proxy ITR that does this.

As far as I can see, the only way a "proxy ITR" could receive
packets from non-upgraded networks is for the prefix these addresses
are part of to be advertised in BGP.  Without a BGP advertisement
for the prefix in which LISP maps address space, the packets will
never leave the border routers of the non-upgraded networks.

So I think there needs to be at least one "proxy ITR" in the world
which advertises the prefix.  The Ivip approach - and I think what
Eliot was exploring - is for there to be multiple such "proxy ITRs"
outside non-upgraded networks, all advertising the prefix, so most
packets don't rely on the one ITR and so most packets have more
optimal paths.

> Or the LISP site can be smart enough to use an EID that *is
> in* the routing system. Such as a PA address that will always
> continue to be there.
>
> We are working out the details on how to get a LISP site to talk
> to a non-LISP site.

There is no problem with a sending host in a network with LISP ITRs
sending packets to hosts with non-LISP mapped addresses, including
those in sites with no LISP ITRs and/or ETRs.  The problem I am
discussing is how to get a packet from a sending host in a network
with no LISP ITRs to an ITR so it can be tunneled to the right ETR
of a destination host with a LISP-mapped address.

 - Robin

---

From: Dino Farinacci <[email-masked]>
Date: Sat, 20 Oct 2007 21:03:44 -0700

> don't need the CDR-CAR system.

Right, you could use NERD to push the mappings to RETRs.

> number of BGP advertised prefixes, as long as, in general, each such

Well, don't forget what one of the goals are, to reduce the size of  
the core routing tables.

> never leave the border routers of the non-upgraded networks.

Right. Either we make them a very small number of prefixes (I say <  
100) or we don't do it. And the only alternative is then to do a NAT  
in the ITR.

> of a destination host with a LISP-mapped address.

Right, understand.

Dino