Subject: [rrg] Arguments in favour of Core-Edge Elimination vs. Separation?

From: Robin Whittle <[email-masked]>
Date: Fri, 22 Jan 2010 12:52:47 +1100

Tony - as far as I know, you are opposed to CES (Core-Edge
Separation) and in favour of CEE (Core-Edge Elimination).

5 or 6 full proposals are CEE and only 4 are CES (LISP, Ivip, TIDR
and RANGER) - so it seems that many active RRG folks fully support
CEE in favor of CES.

The "simple network, sophisticated hosts" paradigm works well in many
respects and I understand there is a perception of elegance in CEE -
which extends this principle further to get hosts to do more routing
and addressing work so the routing system can remain simple.

Introduction of CEE would involve new host protocols and usually new
application code and APIs in order that hosts and networks can have
multihoming TE, portability (of their identifiers) and probably
mobility - while operating entirely from potentially unstable PA
addresses obtained from their one or more ISPs.  (In some schemes,
the ISP's address space only forms the more significant bits of the
host's PI address.)

Routing scalability is achieved by way of the DFZ routers only having
to cope with the prefixes of ISPs - not of any end-user networks.

I hope CEE supporters will argue their case explicitly.  My support
for CES is stated fully in section 4.1 of the Ivip-arch ID.

I think CEE's attraction is based on an over-simplified notion of the
Internet: a routing system with hosts neatly arranged around the
edge.  If I thought this was the reality, I would agree that CEE is
the optimal approach - though I would still have no idea how we could
change to CEE on a voluntary basis from the current arrangement.

However, the Internet's routing system is big and has traffic costs,
delays and risks of lost packets.

Furthermore, an increasing number of hosts are on slow,
less-reliable, and sometimes expensive wireless links.  These
problems are fundamental - most wireless links will never be like
fibre or DSL because the spectrum is shared, the base-stations and
mobile devices are interfering with each other, the timeslots take
time come around and because upstream capacity usually has to be
reserved in advance.

An increasing number of hosts have computational constraints due to
being hand-held devices with very limited battery power.  This is no
problem for simple protocols and caching a few things, but if
cryptographic work is required in the CEE protocols, then I think it
is more of a problem.

I believe the optimal arrangement for the Internet is not the CEE
approach of requiring *all* hosts to take on more responsibilities
for routing and addressing - which generally involves more state,
more computation, delays in initiating contact with other hosts and
the sending and receiving more and/or longer packets.  It also
involves a lot of extra work, packets and therefore delays whenever
the host gets a different (physical, "locator") address.  This can
happen frequently with wireless mobile devices, even when they are
stationary.

These  things might be tolerable and desirable if the Internet's core
had no delays or other costs, and if all hosts were on fast,
inexpensive links and had plenty of CPU power.  But CEE generally
causes more delays even with hosts on fast links, due to the extra
work they need to do to be sure of the identity of the other hosts
they are communicating with and the fact that the lookups and
management packets frequently encounter delays in the core, due to
the size of the Earth and the speed of light in SiO2.

I think CEE would cause unacceptable performance problems and higher
costs for wireless mobile devices - since extra management traffic
has to flow back and forth along this slow and perhaps unreliable
link, frequently before real host-to-host communications begin.  IPv6
makes such management packets longer still.

Some systems CEE architectures involve adding extra stuff to traffic
packets, at least for some packets.  This is further burden on
devices relying on wireless links.

In the future, I think we should expect that most hosts are going to
be wireless mobile devices.

I support the CES approach of retaining the existing host protocols
and instead adding things to the network (ITRs, ETRs, mapping system
etc.) to solve the scaling problem and provide mobility.

My arguments for Core-Edge Separation are in section 4.1:

  http://tools.ietf.org/html/draft-whittle-ivip-arch#section-4.1

Tony - the reason I think you support CEE is this message from mid-2008:

  http://lists.arin.net/pipermail/arin-ppml/2008-June/010988.html

    Please, please, please go read GSE.  You may not like it, you
    may not agree with it, but until you grok it, you haven't seen
    a big chunk of the solution space.

and because I don't recall you writing anything in support of LISP,
Ivip or any other CES proposal.

In the next few weeks I hope to analyse each of the CEE proposals in
detail, to determine to what extent my critique about extra delays,
extra packets etc. applies to them.

    - Robin

---

From: Patrick Frejborg <[email-masked]>
Date: Fri, 22 Jan 2010 10:54:10 +0200

Robin,

think I need to speak up and comment on a few things.

> and RANGER) - so it seems that many active RRG folks fully support

Believe we need both CEE and CES so that the end users can decide
which way to go, either by upgrading the hosts or add a network device
in front of their hosts - both approaches should be promoted, and
perhaps some additional as well

> the ISP's address space only forms the more significant bits of the

Totally new host protocols might not be needed, you could get around
most of the challenges by adding extensions to existing protocols.

> change to CEE on a voluntary basis from the current arrangement.

The voluntary basis issue is the hardest one to crack, it is a valid
statement for both CEE and CES proposals. Think we should remove the
technical nerd hat for a while and put on the shoes of residential
user and the trousers of a CIO in an enterprise, ask ourselves - do
our proposals:
- introduce any new services?
- lower the cost of Internet equipment?
- make Internet connectivity more or less complicated?
- can you use any idealistic arguments, e.g. green values, to promote
a migration?
- introduce some new e.g. API etc that could provide a path to create
new services on?
These end user do not understand nor cares about routing scalability
or that the depletion of IPv4 is soon occurring, simply because they
have a very well working Internet at the moment - they also have a
valid IP address space and the new address space do not provide any
new services that can't be found in the current Internet.

And of course you, put also on the shirt of an ISP and ask the same questions

> cryptographic work is required in the CEE protocols, then I think it

I agree here, do we really need to have cryptographic solution on the
network layer?
We are trying to remove the IP address overload (identifier and
locator) but if we are not careful we could introduce some other
overload mechanism that somebody has to deal with in the future. The
transport layer can take care of things and also the application layer
can take care of things, such as cryptographic

> happen frequently with wireless mobile devices, even when they are

Not *all* host needs to be upgraded, you could design the architecture
in a such way that it is backwards compatible and only part of them
needs to be upgraded. But there is price, you have to change the IP
address allocating policy and that could be a difficult hurdle.

If the host mobility is solved on the network layer it could get
complicated as you describe above. I think the host mobility is an
issue for the transport protocol - if we are trying to solve that on
the network layer IMHO we are overloading the transport and network
layer.

I think MPTCP, http://tools.ietf.org/html/draft-ford-mptcp-multiaddressed-02,
doesn't add that much overhead

Yes, the mobile host-to-host rendezvous mechanism is not provided by
MPTCP but that can you find on the application layer. For real time
applications there is already SIP registrar/proxy in place,
peer-to-peer applications have their own solutions etc. Do we need to
overload the application and network layer?

-- patte

> _______________________________________________
> rrg mailing list
> [email-masked]
> http://www.irtf.org/mailman/listinfo/rrg
>

---

From: Xu Xiaohu <[email-masked]>
Date: Fri, 22 Jan 2010 17:58:16 +0800

<skipped>

> can take care of things, such as cryptographic

In the current Internet architecture, the overlapping of IP address
semantics makes it possible to use uRPF to avoid IP (as the role of ID)
spoofing to some extent. However, in an ID/locator split architecture, ID
spoofing will be much harder to prevent provided there is no any mechanism
for ID authentication (uRPF is useless for ID checking). If cryptographic
identifiers are not used, ID authentication would have to be relied on a
third-party certification infrastructure.

Xiaohu

---

From: Patrick Frejborg <[email-masked]>
Date: Fri, 22 Jan 2010 12:40:20 +0200

> third-party certification infrastructure.
I think that depends upon the ID/locator split architecture, i.e how
the split is carried out.

If you do the ID/locator split within the network layer, then you
might need the mechanism you described above.

But if you do a regional-edge/global-core locator split within the
network layer and move the ID element to the transport layer the ISP
can still do uRPF on both locators.

The architectural question is - do the ID authentication issue belong
to the network or to the hosts (or the application they use), where
should the ID authentication be applied?

-- patte

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 23 Jan 2010 09:07:31 +1300

On 2010-01-22 23:40, Patrick Frejborg wrote:
...
> should the ID authentication be applied?

It seems to me that fraud mainly occurs at application level, or at least
at the level of games played with DNS names. So authentication at those
levels is needed quite regardless of what is done at network level.
On that view, authenticating a network or transport ID is irrelevant.

I think that authentication at the network ID level is important to
to ensure that the physical origin of unwanted traffic can be checked
and/or proved. This does seem to be more of an issue when there's a
loc/id split, because the locator becomes a free parameter, and the
combination of ingress filtering and locator RPF doesn't help prove
an ID.

     Brian

---

From: Robin Whittle <[email-masked]>
Date: Sat, 23 Jan 2010 16:06:09 +1100

Short versions:         CEE and CES architectures are vastly
                        different in how benefit arises from effort.

                        With CES, end-user networks get instant full
                        benefits and routing scaling benefits are in
                        direct proportion to how many such end-user
                        networks adopt it.

                        With CEE, there's no really usable set of
                        benefits for end-user networks until pretty
                        much all the hosts in the Net adopt the CEE
                        upgrades too.  Only then can there be any
                        real routing scaling benefits.

                        ASCII art diagrams of the cumulative
                        effort -> benefit relationships for CES
                        and CEE.

Hi patte,

> perhaps some additional as well

I am not suggesting that if someone wants to implement a host-based
CEE approach that they should be prevented from doing so - nor are
you implying that I did.

>From a routing scaling point of view (ignoring my concerns about
burdening all hosts with extra responsibilities) I think the trouble
with CEE approaches is that they only provide scaling benefits when
an adopting network can either:

  1 - Give up its current PI space and work from the PA space from
      its one, two or more ISPs.

or

  2 - Migrate from reliance on its current (non-portable, no
      multi-homing) PI space to the new system of identity (AKA edge)
      addresses in a way which provides portability, multihoming and
      TE.

It is reasonable to assume that every end-user network (which wants
or needs portability, multihoming etc.) which we need to adopt a
scalable solution will require continual uninterrupted communications
with all hosts, including those which have no adopted the new system.

We need the great majority of these networks to adopt a scalable
solution, because there are believed to be so many of them (millions
to ten million I guess) that even a small number using conventional
PI space threatens the scalability of the DFZ.

With this assumption, and with the assumption that multihoming for
only a subset of sessions is of no real value, then point 1 can't
happen unless all the hosts on the Net adopt the new system - because
ordinary hosts can't communicate with (or at least initiate
communications with) this networks' hosts via their new edge
identities (which is the only way the adopting network's hosts can be
reached once they no longer have PI space).

Even if the end-user network was ready to accept 10% of hosts not
being able to reach their hosts, which seems unlikely, this means
such networks can only give up their PI space once 90% of the hosts
in the world adopt the same system.

Perhaps I am mistaken in assuming these things about CEE systems -
there are quite a few of them and I am yet to read those proposals in
detail.

So, from the point of view of routing scalability, I don't see the
point in encouraging end-user networks to adopt CEE unless we could
be confident that everyone would adopt it.  As far as I know,
networks can only switch of their PI space, or get truly usable
portability, multihoming etc. when essentially all other hosts are
using the same system.

> most of the challenges by adding extensions to existing protocols.

OK - I agree, but even altering applications is very difficult.
Also, I think it would be a nightmare to have an application with two
different ways of communicating with other hosts - the conventional
approach and some CEE approach.

For a host to use the new CEE system, as far as I know, its
applications all need to be rewritten to use it as well, and the
stack and its API needs to be changed as well.

There may be CEE architectures which work entirely with existing API
and applications, but I can't think of one now that would work.

So its not just a network deciding to adopt a new system, upgrading
their operating systems etc.  It also requires that all their
applications be rewritten.   I can't imagine how this would ever
happen - especially since I can see how mobility, scalable
portability, multihoming etc. can be done with a CES system and no
host changes at all.

> statement for both CEE and CES proposals. 

I think the CEE proposals differ enormously from CES proposals in
their ability to meet the constraints which arise from the need for
voluntary adoption.  The only attempt I know of to state these
constraints is my page:

  http://www.firstpr.com.au/ip/ivip/RRG-2009/constraints/

This is my text, refined after some RRG discussions.  I don't know of
anyone who thinks these constraints are invalid, though 8, 9 and 10
are sliding scale constraints and the rest are absolute, so there is
debate about how much 8, 9 and 10 matter on a case-by-case basis.

A CES architecture which does not significantly affect performance,
security etc. could satisfy all 10 constraints.  I think Ivip would
satisfy them all.  I think LISP-ALT or any system which relies on a
global query server system for mapping should be able to satisfy all
but point 9 (performance) - due to the "initial packet delay"
problem.  With ALT, this is actually a problem of "initial packets
get dropped until the one is sent after the ITR gets the mapping".

No CEE architecture can meet any of the constraints 1 to 5.

> Think we should remove the

I agree.  I think that in order to solve the routing scalability
problem, we need to construct a perfectly beneficial Trojan Horse.
We can generally assume they won't do anything to their networks
purely for the purpose of helping with scalable routing.  So we need
to give them something provides concrete and immediate benefits, and
which will also achieve our scalable routing goals if adopted
sufficiently widely.

The requirements for getting people to adopt the Horse depend on the
nature of the architecture.

For CES, we only need it to be adopted by the subset of end-user
networks we are trying to serve - those which want or need
portability, multihoming and TE.  This is as long as the scheme is
self-contained with its own PTRs (LISP) or DITRs (Ivip).  In
practice, it would also be best if networks which weren't using EID
addresses (LISP) or SPI addresses (Ivip) also installed ITRs - since
that would make the whole thing work better than relying on PTRs and
DITRs.

Still, there is no need, with a CES, to do any of these things:

  1 - Change host stacks or apps in any hosts whatsoever.

  2 - Change routers or hosts in any networks not adopting the new
      kind of space.

in order to:

  1 - Give full portability, multihoming etc. benefits to the
      adopting networks.

  2 - Achieve routing scalability goals in direct proportion to
      the number of end-user networks which adopt the system.

With a CEE, it is totally different.

As far as I know, no network which adopts it can get portability,
multihoming etc. benefits for all its packets until all other hosts
in the world adopt the system too.

As far as I know, no network would find these benefits of any real
use unless they applied to all, or almost all (~99%+), their
communications.

As far as I know, there's no way any end-user network with PI space
would abandon it, and so provide scalable routing benefits, until
almost all their communications were with upgraded hosts.

Here are some ASCII art diagrams for the scalable routing benefit vs.
effort curves for the two types of proposal:

    ^
 B  |        *       Core-Edge Separation (CES)
 e  |       **
 n  |      ***
 e  |     ****
 f  |    *****
 i  |   ******
 t  |  *******
    | ********
    |*********
    0--------->
      Effort ~= level of adoption

    ^
 B  |        *       Core-Edge Elimination (CEE)
 e  |        *
 n  |        *       (IPv6 adoption too.)
 e  |        *
 f  |        *
 i  |        *
 t  |        *
    |        *
    |*********
    0--------->
      Effort ~= level of adoption

CEE gives no really usable benefits to adopters until all other
networks adopt it.  There are no scalable routing benefits until
firstly these really usable benefits exist and secondly, this enables
those with PI space to release it.

So our CES-style Trojan Horse is a lot easier to get people to adopt,
since it provides them with immediate benefits, irrespective of the
rate of adoption - and it produces scalable routing benefits to the
exact degree to which it is adopted.

A CEE Trojan Horse is not going to be widely enough adopted to ever
reach the required nearly 100% adoption rates, because it takes
effort, eats hay etc. and doesn't do anything really useful until
everyone in the Net, every host, adopts it as well.

> And of course you, put also on the shirt of an ISP and ask the same questions

Indeed. As far as I know, the only possible conclusion is that the
best CES architecture could be adopted widely, and that even the best
CEE architecture (in terms of scalable routing benefits, or benefits
to adopters once everyone else adopts it) would be impossible to have
adopted by more than a tiny fraction of networks.

> I agree here, do we really need to have cryptographic solution on the

HIP is a CEE and it uses crypto, but perhaps it is optional:

  http://tools.ietf.org/html/rfc4423

I intend to look more closely at the CEE RRG proposals to see what
they require of hosts.  I think many of them involve nonces, which is
not such a problem.

> locator) 

Not me - Ivip and other Core-Edge Separation architecture still
retain the IP address as both the identifier and locator for each
host.  Its just that a subset of the address space (SPI for Ivip, EID
for LISP) is interpreted differently by special devices (ITRs) which
are able to get the packets to ETRs anywhere in the Net.  So for SPI
space, the actual IP address doesn't imply anything about where the
destination host is.  Nonetheless, it is still used by the routing
system to get the packet to its destination.  Its just that ITRs and
the mapping system are more flexible and scalable than the BGP DFZ's
way of deciding where to send the packet.

> but if we are not careful we could introduce some other

. . .

> address allocating policy and that could be a difficult hurdle.

As far as I know, a CEE converted host doesn't derive any benefits of
portability, multihoming etc. with packets arriving from a non-CEE
host.  So if you want all your traffic to be portable, multihomable
(which I assume any network would require before they considered the
thing at all useful) then this is only going to happen once all other
hosts in the world are upgraded too.

> the network layer IMHO we are overloading the transport and network

The only way I know of doing mobility, without service interruption
every time the MN gets a new address, is my TTR Mobility approach:

  http://www.firstpr.com.au/ip/ivip/TTR-Mobility.pdf

This requires a CES architecture, but doesn't require mapping changes
every time the host gets a new address.  It would work for IPv4 and
IPv6, enable communication with normal protocols to all other hosts,
and enable the host to be behind one or more layers of NAT.  This
would provide the MN with one or more IPv4 addresses or IPv6 /64
prefixes no matter where it was, with generally optimal paths.  It
requires infrastructure in the network itself, but not in the access
networks.  I think this could be a commercially feasible and
attractive service.

> doesn't add that much overhead

But that requires host stacks at both ends be upgraded and that the
applications are written to use MPTCP.  I am not sure how well it
would work with a host which could be connected to any address,
including behind NAT.  MPTCP is probably not much good for VoIP,
streaming media or other protocols which naturally work well with UDP.

> overload the application and network layer?

I think extending a scalable routing CES system with TTRs
(Translating Tunnel Routers), or building a CES system just for the
TTR system would both work fine - and the only host changes would be
some extra software for tunneling and a few other things in the MN
itself.

  - Robin

---

From: Patrick Frejborg <[email-masked]>
Date: Sat, 23 Jan 2010 14:53:41 +0200

>      addresses in a way which provides portability, multihoming and

I prefer approach 2, think this is the best way to go since that what
has been done in the cell-phone network - the subscriber owns the
number and can choose which ever service provider he/she prefers. So
we should keep this model in the Internet as well.

> approach and some CEE approach.

You might be able to design the CEE architecture in such a way that
most applications do not see the difference, if the approach is enough
backwards compatible. Of course you get into trouble with applications
that uses IP-addresses in their payload - but these applications are
already having troubles with NAT traversal

> stack and its API needs to be changed as well.

Nope, not all CEE approaches

> and applications, but I can't think of one now that would work.

Well, there is one that might work around the problem

> portability, multihoming etc. can be done with a CES system and no

Agree, if every application needs to be rewritten we are in serious trouble

> No CEE architecture can meet any of the constraints 1 to 5.

Had a look on those and some comments

1. Think all proposals suffer from this, why upgrade in the first
place - what do the end user gain? It is really hard to find
disruptive elements in the proposals

2. Yes, no problem

3. Agree, has been taken into account and obeyed

4. Almost, I do have issues with some applications when it goes
outside an ALOC realm

5. Disagree here, both approaches should be adopted because there is
the cache issue that is scaring the crap out of me. More about the
cache later on. Also other benefits than routing scalability in
Internet can be achieved if you take this core-edge split to the
hosts' stack. See what Microsoft Research have done with their VL2
work
http://research.microsoft.com/pubs/80693/vl2-sigcomm09-final.pdf
I'm not working for MS but here is an example what can be done if you
don't exclude a host stack upgrade

> which will also achieve our scalable routing goals if adopted

Yes, and I believe that the  Trojan Horse is called MPTCP :-)
My guess is that MPTCP offers something interesting for the end-users
and we do have an opportunity to smuggle our Achilles in the hosts
when MPTCP is getting deployed

> in the world adopt the system too.
Nope, not true. By using MPTCP you can migrate current multi-homing
towards multi-pathing. No tweaking with BGP load-balancing anymore,
let the transport protocol take care of the load-balancing and
multi-homing. And this can be done in a smooth way, also it should
lower the costs of networking devices.

It is true that you would need a lot of host stacks to be upgraded
before the benefits can be harvested, thus CEE needs a CES solution to
speed up the adaption. Then you argue, as you do, that it is
sufficient with a CES solution only. But here I see a paradox and I
might be wrong, so please correct me if I miss something.

The more popular a CES solution becomes the more stress it will create
on the ETR (that becomes ITR for returning traffic). If the CES
solution becomes popular and gets widely deployed, e.g. if broadband
routers will include an ITR stack it will increase cache sizes on the
ETRs in front of popular sites. Today broadband address space is quite
well aggregated in the DFZ but if you put an ITR on every broadband
router the currently aggregated PA-address space will get
de-aggregated on the ETRs in front of popular sites. Of course the
core network will have a smaller FIB but how large cached FIB is
needed on the popular ETRs? One million, two millions, will it scale?
What about the housekeeping of the caches? What are the costs of these
large cache ETRs - it is not trivial to do housekeeping with two
millions cache entries, or is it?
Will the content providers adapt to this technology - if the content
sites becomes unreliable due to cache housekeeping will they adapt or
do they rather stay with the current solution - so they don't risk
their business model?
If the cache solution is not 100% reliable and proven the content
providers will most likely not accept this technology and then the CES
approaches will not be adapted.
Can you get a better story if you combine a CES with CEE from day one,
explaining the benefits of it, also remove the address space
constraints, so that the content providers can have simpler&cheaper
Internet connectivity and they can reach more customers in the future?

Only more questions....and I might misunderstand something...

-- patte

---

From: Scott Brim <[email-masked]>
Date: Sun, 24 Jan 2010 11:46:09 -0500

Robin Whittle allegedly wrote on 01/21/2010 20:52 EST:
> application code and APIs

These are much less of a problem than they used to be.  However,
in any case improvements to the routing system should not depend on
changes to endpoints.  A change to routing architecture that makes no
difference in routing scalability until every endpoint in a domain is
updated isn't a wise idea.

> happen frequently with wireless mobile devices, even when they are

Better to leave mobility out of your argument.  For a mobile node,
everything you mention is going to happen anyway, and during handoff,
getting a new IP address is one of the smaller problems (compared to,
e.g. authentication and authorization and the infrastructure to support
it).  Also there are mitigation mechanisms whereby a node often doesn't
need a new address.

But you have omitted the burden on the routing domains that serve those
endpoints -- the burden on the edge networks' administration.  As the
network changes its upstream connectivity, it has to adjust its policies
to handle changes in addresses given to endpoints.

This is another example of why you want to decouple progress in the
endpoints (multihoming, multipathing, mobility, identification,
federated identity etc.) from routing and addressing architecture
(scaling, robustness, churn, reachability, complexity, administrative
overhead etc.).  They may benefit each other, but neither should depend
on the other, and each should be motivated on its own merits.

> I agree here, do we really need to have cryptographic solution on the

Generally I agree.  Validation of various sorts is between consenting
parties, which may be routers, tunnel endpoints, session endpoints,
applications, whatever.  How much validation to have on which particular
packets should be up to the communicating pairs.  Validation will be
done at a higher layer anyway if at all.  Consider video distribution --
the main bandwidth consumer on mobile networks today -- which runs
multiple connections with a unifying validation scheme.

> can take care of things, such as cryptographic

Right, and if the upper layers have to do something anyway, why should
it be done in a lower layer?  The end-to-end principle applies inside a
stack as well as across a network.  The good reasons for providing a
function at a lower layer are if it cannot be done at a higher layer
(access to certain information is needed), or if it is certain that all
higher layer clients should use that function.

> doesn't add that much overhead

:-D but let's not forget that MPTCP is just one mechanism.  We already
have application-layer mechanisms in place that may never be abandoned
in favor of multipath transport.  Also let's get away from the idea that
a whole node ("host") is what is mobile.  The future is virtual.
Individual sessions will change their apparent location independently of
each other.  They need to be able to do so arbitrarily, so simply adding
session classes (e.g. "voice") to a whole node mobility protocol isn't
enough.

> third-party certification infrastructure.

But that's the way it is today.  Communication endpoints are at higher
layers, and those that care will always check the validity of the other
endpoints' identifiers.  They do not depend on, or even expect, support
from lower layers for that.  End-to-end argument goes here.

> should the ID authentication be applied?

See arguments above about when it is appropriate for a lower layer to
provide a function for the use of a higher layer.

Brian E Carpenter allegedly wrote on 01/22/2010 15:07 EST:
> and/or proved. 

The advantage of using a node ID is that a lower layer (network layer)
can take care of housekeeping before delivering packets to the actual
recipient (higher layer), and the higher layer can trust that everything
is okay.  Disregarding the end-to-end argument and the trust problems,
you still have the issue of virtualization.  When you're talking to me,
you don't get to know what my physical origin is.  Any of the sessions
we have open can move individually from box to box, and even from
virtual box to virtual box.  The only constant identifiers are
associated with the session or group of sessions.  Those identifiers can
be ad hoc and temporary once initial AAA is taken care of.

Noel Chiappa allegedly wrote on 01/23/2010 11:26 EST:
>     > From: Patrick Frejborg <[email-masked]>
> 
> Support for multiple PA addresses does certainly help with the multi-homing
> aspects, but does it help with number (address) portability at all? 

Just for clarity: Multipath transport does not assume or rely on the
endpoint receiving multiple PA addresses from a network.

Scott

---

From: Robin Whittle <[email-masked]>
Date: Mon, 25 Jan 2010 22:06:24 +1100

Short version:    Continuing discussion of the adoption problems
                  inherent in CEE and whether CEE is needed if a
                  CES architecture can "do everything".  Also a
                  perceived problem with (ITR?) cache unreliability,
                  which I think may be based on a misunderstanding.

Hi patte,

>>      addresses in a way which provides portability, multihoming and

In point 2 I should have written PA space.  There are two classes of
existing end-user network which might want portability, multihoming
and TE:

  1 - Those who have it already with PI space.

  2 - Those who don't have it since they have PA space.

So my original 1 and 2 are not a choice an end-user network makes -
it is two types of transition for two classes of existing end-user
network.

> we should keep this model in the Internet as well.

That is my point 1 - the end-user network already has "their own" PI
space.  But in CEE, the new kind of scalable space is not IP
addresses of the original kind.  So they can't keep their space and
make it portable, multihomable.  All CEE schemes involve two separate
namespaces - one for Locator (physical) and one for Identifier (logical).

As far as I know, for any CEE (I think including your hIPv4 proposal,
which I am yet to read in detail) no network at present has any space
of the new Identifier type which a CEE will provide.  So neither of
my two points involve the end-user network retaining their existing
address space as their new, portable, multihomable, Identifier space.

In point 1, they give up their PI space and adopt multiple prefixes
of PA space, for the "Locator (physical)" addresses for their hosts.
 They gain a patch of the new Identifier range of addresses, which is
in a totally different namespace from IP addresses - it might not
even be a binary number.  In point 2, they may keep their current PA
address space, and get another set of PA addresses from a second ISP
if they start multihoming.  They migrate their hosts and applications
to a new set of "addresses" of the Identifier kind, just as for point 1.

> already having troubles with NAT traversal

I hope to review the CEE proposals which the RRG is considering, plus
HIP, in various respects - including what host changes would be required.

> Nope, not all CEE approaches.

Which one can run with unmodified API and applications?  If they do
this, then the apps are using things which are the same number of
bits as IP addresses to refer to other hosts.  Yet the whole idea of
CEE is to have the apps using a new kind of address - an Identifier.
 If so, then the Identifiers are of the same numeric type and size as
the IP addresses which are used for "Locators" - however the two are
in different namespaces.  I can't think of such a system, but I need
to read about them in more detail.

> Well, there is one that might work around the problem

The trouble is, I don't think there are any such CEE architectures.

> Agree, if every application needs to be rewritten we are in serious trouble

Exactly.

Yet the purpose of a CEE is to have all applications using purely
Identifiers, and being generally unaware of how the stack decides
which physical address(es) - Locator(s) - to use when sending packets
to the identified host the application is communicating with.

> Had a look on those and some comments

OK - thanks.

> place - what do the end user gain? 

If they gain portability, multihoming and TE when they either
couldn't get it before, or couldn't afford it, then this is a good
motivating factor - however, to be "compelling" these benefits have
to work for packets coming from all hosts, not just those in networks
which have already adopted the CEE or CES architecture.

> It is really hard to find disruptive elements in the proposals

All the proposals involve effort.  CES involves no host changes and
minimal end-user network changes.  CEE involves upgrading at least
the stacks and (I think) probably the apps in all hosts - which is so
disruptive as to be generally prohibitive.

> 2. Yes, no problem

??

  2 - Therefore, it must provide multihoming, TE and
      portability for the adopting networks in a
      manner which fully supports communications with
      all hosts - including all hosts in non-upgraded
      networks.

A CES architecture such as LISP with its PTRs or Ivip with its DITRs
can do this: provide portability, multihoming and inbound TE for
packets sent by *all* hosts, not just those in networks which have
adopted LISP-mapped EID address space or Ivip-mapped SPI space.

No CEE can do it - since they only provide the benefits for packets
sent by hosts in networks which have already adopted the CEE
architecture.  This will initially be close to 0% and even if 90% or
99% of other networks adopted it, there would still be 10% or 1% of
hosts for which communications are not supported with portability,
multihoming and TE.

I think its fair to say that most networks would only regard the
benefits as being worthwhile once they applied to all communications,
or some very high percentage of them, like 99.99%.

> 3. Agree, has been taken into account and obeyed

Maybe I need to make this more explicit.  Its not just that a host in
a network which has adopted the CES or CEE architecture might be able
to do old-style communications with non-upgraded hosts - it must be
able to support portability, multihoming and TE with them too.  CES
architectures can do this.  CEE architectures can't.

> 4. Almost, I do have issues with some applications when it goes

Googling "ALOC" leads me to your hIPv4 summary:

  http://www.ietf.org/mail-archive/web/rrg/current/msg05529.html

which I intend to write a critique of, once I have written some others.

> cache later on.

I guess by "both approaches" you mean a network-based CES and a
host-based CEE.

> Also other benefits than routing scalability in

I don't clearly understand this.

All sorts of things are possible with host-upgrades.  Maybe not all
those things are a good idea to enforce on all Internet hosts,
especially those on slow wireless links.

The difficulty with host upgrades is if the system requires them to
be made to all hosts in the world in order to provide the benefits.
This is the case with all CEE architectures.

> when MPTCP is getting deployed

The MPTCP horse wouldn't get through the castle gates.  It doesn't do
anything useful and its name is hard to pronounce.

It doesn't support existing applications.

Maybe if some people write applications which use it, then that's
great because for those applications, maybe the hosts can run OK on
scalable PA address space and multihome with PA space from two or
more ISPs.  But that is not enough to give real multihoming,
portability etc.  The only way to do it is in a way which supports
existing applications and existing host stacks.

> lower the costs of networking devices.

Yes, but it only works with applications which have been written (or
rewritten) to use MPTCP - and some applications can't use MPTCP.

So there's nothing you can do with MPTCP to help with scalable
routing in a gutsy way, which will really provide multihoming,
portability etc. for existing applications.  To do that you need to
have a simple change, which supports all existing applications with
these benefits.

If this was a clean-slate design, then by all means, MPTCP would be a
part of the system - but we are working with a billion or more
end-users who each use dozens of applications.

> speed up the adaption. 

But if the CES can provide all the portability, multihoming and TE
benefits, then why do we need CEE?

A CES architecture such as Ivip can optionally involve host changes
when it is the cheapest way of providing the ITR function - to build
it into the sending host.  But this is not a requirement - nor does
it resemble a CEE scheme.

> Then you argue, as you do, that it is
> on the ETR (that becomes ITR for returning traffic). 

The device which performs the ETR functions is not necessarily being
an ITR for outgoing packets.  ITRs in Ivip could be in various
places, including in sending hosts.

In Ivip, the ETR normally only de-tunnels the traffic packet - and
then knows how to forward it to the destination network.  The ETR
sometimes engages in communication with the ITR for the purposes of
PMTUD management - but this is just for encapsulation tunneling.  In
the long term, or perhaps from the start, Ivip is intended to use
Modified Header Forwarding for tunneling - which involves no PMTUD
problems.  Then the ETR function is very simple - just revert the
header to normal and forward the packet to the destination network.

> If the CES
> ETRs in front of popular sites. 

It would be possible to put an ITR function in DSL etc. routers.
This would be cheaper than building ITRs into the ISP's network.

I don't clearly understand what you wrote about cache sizes. ETRs
don't have caches - either in LISP or Ivip.  ITRs cache mapping.  In
Ivip, the mapping is for a micronet of SPI space (an arbitrary number
of IPv4 addresses or IPv6 /64s) to a single ETR address.  LISP maps
each EID prefix to one or more ETR addresses, with priorities to help
with multihoming and with weightings to implement load sharing
(inbound TE for the destination network).

> Today broadband address space is quite
> de-aggregated on the ETRs in front of popular sites. 

I don't understand this at all.

If you have a million DSL customers like today, each DSL router has
an IPv4 address - a single IPv4 address of PA space from the ISP's
large (short) prefixes which it scalably advertised in the DFZ.

The ISP could decide to put an ITR in each DSL router (by specifying
the use of certain DSL modems etc.), rather than spend money on
adding ITR functionality to existing internal routers or by
implementing ITRs on servers in the internal routing system.  There's
no problem with this, at least for Ivip.

With Ivip, the ITRs request mapping directly or indirectly from full
database QSDs in the ISP - and so get the mapping reliably and within
a few tens of milliseconds.   Each such ITR function needs to cache
the ETR addresses of all the destination hosts which are on SPI space
 which any of the home/office computers are sending packets to.
That's fine - there may be 10 PCs on the LAN, and over any time
period such as 10 minutes, they may be sending packets to 1000 or
more hosts, most of which (in a nearly ubiquitous adoption scenario)
are on SPI addresses.  If each such host is in a different micronet
(some destination hosts would probably be on the same micronet), then
the ITR is caching mapping for 1000 micronets or so.  That's no
problem for the CPUs which are in broadband routers.  If it was
10,000 it might be trickier - and 100,000 would probably be excessive.

But the ITR function can always drop things from its cache and
request the mapping again if it needs it.

What you wrote:

  "the currently aggregated PA-address space will get
   de-aggregated on the ETRs in front of popular sites."

doesn't mean anything to me.  Can you give a fuller description of
the problem you foresee?

> Of course the

I don't understand this.

> millions cache entries, or is it?

ETRs don't cache anything.

When you are discussing a broadband router, I assume you mean a
customer like today, on a single IPv4 address via DSL, fibre or HFC
cable.  The broadband router does NAT to support potentially many PCs
on the local LAN.

This broadband router is not on SPI space.  It is not multihomed.  It
uses ordinary PA space like it does today, which is perfectly
scalable.  The fact that it has an ITR function built into it - which
is a choice, not a requirement in Ivip - just means the ITR function
needs to cache ETR addresses according to whatever is needed due to
the packets sent by the various PCs in the LAN.

This broadband router is not operating as an ETR.

If you wanted to multihome a network via DSL and HFC cable, then in
principle you could make a special "broadband router" to do it.

Then, the box would get IP address XX from ISP-A (DSL) and IP address
YY from ISP-B (HFC).  I assume for this example that these are fixed
IP addresses.  Then the box will perform ITR functions as previously
noted.  It would also perform the functions of two ETRs - one on XX
and one on YY.  An external company hired by the end-user network
would be given the credentials to control the mapping for this
network's one or more micronets - and then would control the mapping
to implement multihoming service restoration, and if both links were
working, inbound TE as noted below.

Now the box can support its PCs on the LAN with SPI addresses, which
are portable, multihomable etc.  With Ivip, it can do inbound TE by
splitting up the SPI addresses to which the incoming packets are
addressed into two or more different micronets and then mapping one
micronet to the XX ETR and the other to the YY.  Ivip can't
load-share traffic addressed to a single micronet.  (LISP can load
share traffic addressed to a single EID prefix, including potentially
a single IPv4 address.)  So there would need to be at least 2 SPI
addresses and the incoming packets would need to be addressed some to
one and the rest to the other.

There's no such thing as "cache" in ETRs.  The ITR function is just
the same as before without multihoming and ETR functions.  The box
would also be making outgoing TE decisions about its outgoing
packets, since it has two uplinks.

With encapsulation tunneling, the ITR function would have two
outgoing addresses - XX and YY.  It could decide which link to send
out encapsulated packets on as it chooses.  It doesn't matter to the
ETR at the destination network which ITR the packet comes through.
The host at the destination network doesn't know or care either.

Perhaps your understanding of what CES involves is different from mine.

> do they rather stay with the current solution - so they don't risk

I don't understand the problem you are referring to.

> approaches will not be adapted.

Maybe you are asking what happens if ITR caches are unreliable.
There's no reason why they should be unreliable.  Do you have any
specific problems in mind, for Ivip or for LISP?

> Only more questions....and I might misunderstand something...

Maybe so - it is difficult discussing this stuff by email.
Face-to-face and pen and paper would be a lot easier!

If you describe the problem you foresee I will respond ASAP.

  - Robin

---

From: Patrick Frejborg <[email-masked]>
Date: Mon, 25 Jan 2010 16:03:10 +0200

Hi Robin

> it is two types of transition for two classes of existing end-user

Ok,

PI-addresses is the way to go, an enterprise should have PI-addresses
- regardless if multi-homed or not. Drop PA addresses, they are suited
only for residential users, IMHO.

> address space as their new, portable, multihomable, Identifier space.

Think we have a misunderstanding here with syntax, perhaps you are
referring to LISP's locator and identifier definition when I'm using
RRG's syntax. For me an IP-address is a locator - is it an edge or
core IP-address it is always a locator. In hIPv4 the identifier is
found in the transport layer, a 32-bit value provided by SCTP or
MPTCP.

HIP can provide a host identifier functionality and I see HIP's role
more or less related to Cloud Computing - it could provide a mechanism
to do the ultimate decoupling - instead of building network based VPNs
why not build that with HIP? You could identify your workstations and
server resources with HIP and if the computing resources
(applications, storage, databases etc) is connected to HIP identifier
perhaps you could move your computing resources from one data center
provider to another without doing anything at the network layer - if
the transport or application layer is encrypted, if needed.

> Perhaps your understanding of what CES involves is different from mine.

It might be the case, I'm no expert on the CES solutions and have
studied a little bit LISP. But I'll try to explain my concern on the
ETR cache issue. Having a look on the LISP header I think the ETR
needs to remove the UDP and LISP header, also replace the Source and
Destination Routing Locator values with the inner header's Source and
Destination EID values so that the host receives a "normal" IPv4 (or
IPv6)  header, right?

If the ETR doesn't need to remove the non-IPv4 headers then host needs
to understand LISP and then you have made changes to the host stack.

For the returning traffic the host will use normal  IPv4 (or IPv6)
headers and send it back to the ETR, which becomes an ITR now. Think
there needs to be a cache table so that this ITR can assemble the
outer, UDP and LISP headers - the host can not know those headers
unless it is upgraded and if the host is upgraded it is no longer a
full blown CES solution.

That's why I think there is a cache in the ETRs for returning traffic,
but the returning traffic doesn't need be looked up by a mapping
solution, it is populated by the initiating traffic from e.g.
broadband sites (if the broadband routers uses a CES solution).

But perhaps I miss something here...

> Face-to-face and pen and paper would be a lot easier!

Agree, a whiteboard would be great - think we also are mixing with the
definition of locators and identifiers.
I'll send you off-list a presentation  that I did back in December, it
might get you up to speed with hIPv4 - I realized then that a host
upgrade only is not preferable, need to team up with a CES
(map&encpasulate) solution to speed up to the migration pace. Perhaps
we can sort this out...

-- patte

---

From: Robin Whittle <[email-masked]>
Date: Tue, 26 Jan 2010 02:53:44 +1100

Short version:   Clarifying how an ITR function in a box has nothing
                 to do with an ETR function in the same box - and how
                 ETRs don't cache anything.

                 Also, a new PPT file from patte about hIPv4 is at:

                 http://www.firstpr.com.au/ip/files/PF/

Hi patte,

> only for residential users, IMHO.

I was not writing about a choice an end-user network could make -
just that before the scalable routing system is introduced, existing
networks fall into one of these two classes.

No matter whether an end-user network started with PI or PA space,
with Ivip it will use SPI (Scalable PI) space.  In case 1, it may let
its current PI space go, and adopt SPI space it rents from a MAB
(Mapped Address Block) company, or it may convert its own PI space
into a MAB and use it all for itself.  In case 2, it abandons its PI
space and rents SPI space from a MAB company.

In my previous message I wrote how in both cases, with a CEE
architecture, the new address space is of a different kind (different
namespace) then the IP addresses used for both PI and PA space - so
in both cases, the network's upgraded hosts have their applications
referring to that host, and addressing other hosts, by using the new
kind of Identifier addresses, no matter what IP addresses the hosts
may be using at the time.

> found in the transport layer, a 32-bit value provided by SCTP or

I am yet to read hIPv4 so I can't be sure about this. I am attempting
to explain my understanding of CEE architecture, and I previously
decided that hIPv4 is a CEE architecture.

> the transport or application layer is encrypted, if needed.

I find cloud computing, such as at the Rackspace Cloud, quite
astounding.  More on this below my signature.

> Destination EID values so that the host receives a "normal" IPv4 (or

There's no replacement of anything.  Just strip off the outer IP
header, the UDP header and then the LISP header and what remains is
the exact packet which arrived at the ITR.  Then the ETR knows how to
forward this naked traffic packet to the destination network, which
is presumably topologically close-by.  I think the TTL is
decremented, with an appropriate alteration of the IPv4 header
checksum, but that happens on all routers anyway.

This is for a packet going from host-A to host-B

> to understand LISP and then you have made changes to the host stack.

The destination host host-B has no alterations, and can't see any of
the tunneling operation.  It receives a packet identical to that
which the ITR received, apart from the TTL being decremented by the
ETR and any routers between the ETR and the destination host.  Apart
from the TTL value, this identical to the packet sent by host-A.

> headers and send it back to the ETR, which becomes an ITR now. 

Not necessarily.  The ITR function could be in the sending host
(Ivip, not LISP, except LISP-MN), or in any other router.  The packet
from host-B to host-A doesn't necessarily go back through, or even
near, the device which performed the ETR function on the first packet.

If host-A is on a conventional address (ordinary PI or PA) then
there's no need for an ITR anyway.  If it is on SPI address (Ivip -
or for LISP "EID" space) then at some stage the host-B to host-A
packet will be encapsulated by an ITR.  In Ivip, if there is no ITR
function in host-B, then the packet may be processed by an ITR
function in the  host-B's network (perhaps the same device which did
the ETR function for the first packet) or a router in the ISP network
(likewise - maybe the ETR was in the ISP network, rather than
host-B's network).  If there is no ITR in that path, then the packet
will go out of the ISP and be forwarded to the nearest DITR (Default
ITR in the DFZ - which in LISP is called a Proxy Tunnel Router).
That encapsulates the packet and tunnels it to the correct ETR.

> Think

ITR functions are much the same in principle no matter where it is
done.  The ITR caches mapping information, and if it doesn't have
mapping for a micronet covering the destination address, say of
host-A for the second packet in this example, then it sends a mapping
request to get this.

In Ivip, the ITR buffers the packet for a few tens of milliseconds at
most, since the mapping comes quickly and reliably from a local full
database query server, such as in the same ISP or end-user network.

With LISP-ALT, the ITR drops the packet and sends a request into the
global ALT overlay network, directly or via a local Map Resolver.
The reply comes back - via the Internet.  Since this is a global
network and the path in the ALT system could be longer than an RTT
across the the Earth and back, the ITR does not buffer the packet.
Instead, once it has received the mapping (which might take a short
time, but will often be 400ms to maybe a second or so - assuming no
packets are lost) then whenever the host sends another packet to the
same destination, the ITR encapsulates that one - and it is tunneled
to the ETR.

An ITR function may be in the same box (a router or a server) as an
ETR function, but the two functions don't communicate in Ivip - or as
far as I know, in LISP.

> But perhaps I miss something here...

The ITR operation is independent of any other packet which might have
passed through the box in the other direction, such as to an ETR
function.  In Ivip, if both host-A and host-B are on SPI addresses,
and if they both have their outgoing packets handled by nearby ITR-A
and ITR-B, then ITR-A does one lookup - and gets the mapping for the
micronet which encompasses host-B's address.  When the second packet
is sent, ITR-B does a similar lookup, but with host-A's address, and
gets mapping for the micronet which covers host-A's address.

It doesn't matter where the ETR functions are performed - and ETRs
don't cache anything.

> (map&encpasulate) solution to speed up to the migration pace. Perhaps

OK - I have made the new presentation available at

  http://www.firstpr.com.au/ip/files/PF/

which is where I put an earlier presentation from March last year.

  - Robin

A "server" (formerly a "slice") is a running instance of an operating
system, with its data and all, CPU registers, RAM - everything.  This
can be be moved from one physical server (which runs many of such
things) to another, with only a few seconds break - though the
end-user doesn't do this.  The OS and apps itself doesn't know its
happened, as far as I know.  The end-user can halt the "server" and
make a copy of it (all its hard drive contents) as a backup.  Then it
can be restarted and the the backup can be run as another "server" -
which is identical, but has a different global unicast address and
private LAN IP address.

So a single fully configured "server" can be cloned as often as
required, all via the HTTPS web interface - each one resulting in a
different server instance, probably (or if you want, definitely)
running on another physical server.  The "server" can be running any
one of many 64 bit OSes - I use CentOS.  There is some kind of API so
the creation of these "servers" can be under program control, from
another "server" of course.

---

From: Patrick Frejborg <[email-masked]>
Date: Tue, 26 Jan 2010 09:00:03 +0200

Hi Robin,

> That encapsulates the packet and tunnels it to the correct ETR.

Ok, got it. An ETR has never a cache but for the returning traffic you
might need an ITR (could be most likely the ETR for the initiating
session). So, yes, there is no cache for an ETR -  the cache is always
on the ITR.

> ETR function, but the two functions don't communicate in Ivip - or as
Yes, agreed and that what is what I was referring to - an ITR in front
of a popular server (that might be an ETR for the initiating traffic
from a client). But I stop using that term, instead "ITR in front of a
server" is the term.

Then to the issue. If you take the CES functionality to the xDSL
routers you will get a lot more entries on the ITR in front of a
server at a popular site. The reason is that the xDSL prefixes are
quite well aggregated in today's DFZ. But once you apply the CES
solution on the xDSL router it will carve up the aggregated (PA)
prefixes on the ITR in front of popular servers.
E.g. if an ISP uses today a /20 aggregate for their xDSL connections
and the residential users are allocated /29 prefixes for their
networks. If the residential users would deploy a CES solution in
their xDSL routers you will have lot more entries in the ITR in front
of popular servers - in the current architecture you have only /20
entry in the FIB but in the CES architecture you can't predict how
many entries you will have in the FIB - it depends if, when and how
many xDSL users are accessing your server. And if your site is very
popular do you need more FIB entries than today's router do have, just
in case the CES solution gets very popular and deployed very close to
the edge?

This is my concern and the reason why I think a CES solution also
should include a CEE approach.

Or do I still miss something?

-- patte

---

From: Robin Whittle <[email-masked]>
Date: Tue, 26 Jan 2010 23:22:25 +1100

Short version:   Trying to clarify that putting an ITR function
                 in a DSL router does not mean the DSL customer
                 is using SPI space.  DSL etc. customers will
                 keep using PA space, except for a few who
                 want to multihome.

                 Any existing DSL etc. customer which wants to
                 multihome will still use their existing PA
                 address, and get another one from another
                 link such as WiMAX, HFC or fibre.  But then
                 their hosts will be using SPI addresses.

                 Very few home or SOHO customers will want or
                 need portability or multihoming, since their
                 existing service is quite reliable enough.

Hi patte,

> session). So, yes, there is no cache for an ETR -  the cache is always

Yes - the ITR and ETR functions are separate.  The ITR
caches mapping  and the ETR doesn't.  Ivip ETRs don't have
anything to do with mapping.

(LISP ETRs are typically the authoritative source of
mapping for the one or more end-user networks whose
packets they handle - so if there are two such ETRs in
different ISPs, both are typically authoritative for the
mapping of this end-user network's EID prefixes, as far
as I know.)

> server at a popular site. 

No - the DSL services will still be on conventional PA
space, as they today.  The ITRs in a popular site
(whether they are separate devices, or ITR functions
in the sending hosts) don't need to look up mapping or
tunnel packets for all this plethora of packets going
out to large numbers of these DSL PA IP addresses.

The addition of an ITR function in the DSL router
doesn't change this.  Just because a sending host has
an ITR function, or some router in a network X has
ITRs, doesn't mean that this host or any host in
network X is on SPI space (EID for LISP).

In my previous message I contemplated a special
router for a SOHO situation whereby it implemented
two ETRs, one for the DSL link and one for the HFC
cable link.  Then, the hosts on that SOHO network
would be on SPI addresses, since this is now a
multihomed end-user network.

ITRs in busy server sites will be tunneling packets
to such hosts - to whatever hosts are on SPI addresses.

Ordinary home and SOHO users are probably fine with
their current PI space.  So I am not suggesting that
they adopt SPI space either for portability or for
multihoming and/or inbound TE.

However, there will be hosts in many end-user networks
which do have SPI space, such as those of larger
businesses and other organisations which need
portability and/or multihoming.  So a significant
proportion of packets transacted across the DFZ will
be for SPI addresses - they will be tunneled across
the DFZ.

TTR mobility involves each mobile host having at least
one IPv4 address, which would be a single micronet -
or an IPv6 /64.   This is SPI space.  I am sure that in
the future, most Internet hosts will be hand-held mobile
devices (probably most on IPv6) and I believe the TTR
Mobility architecture is the only one which can provide
global mobility of these addresses, no matter where the
MN is connected, including behind NAT.

So this will be a lot of hosts on SPI addresses and in
general, in the long-term future, any busy web-site or
hosting company etc. is likely to be sending out a lot
of its packets addressed to SPI addresses and therefore
which will be tunneled to ETR addresses.

For most DSL, fibre, WiMAX etc. mass-market non-
physically-mobile customers, I believe the existing PI
space arrangements will remain just fine.

> The reason is that the xDSL prefixes are

This is not the case.  I hope my explanation above
makes this clear.

> Or do I still miss something?

Yes.  Most DSL etc. customers will remain just as they
are today.  Putting an ITR in their DSL router is just
to save on the cost of the alternative - putting an ITR
in the ISP network (or not having such an ITR, and letting
the packets go out to a DITR in the DFZ).

Having an ITR function is to handle packets these DSL hosts
send to other hosts which are on SPI addresses.  It does not
necessarily mean that the DSL service or the hosts behind
its router are using SPI addresses.

  - Robin

---

From: Patrick Frejborg <[email-masked]>
Date: Tue, 26 Jan 2010 19:49:43 +0200

Hi Robin,

finally got an understanding of CES, sorry for taking so long. Let see
if I got it right this time...

For a conventional host A (PA address) -> CES enabled B host:
Actually the closer the ITR functionality (DITR/PTR) is to the xDSL
routers the better the cache scales, since most likely the residential
users access the same services and the cache will not grow that much.
The more CES enabled sites a DITR/PTR has to handle the more stress it
will put on the ITR cache - so here is a dependency between
geographically distributed DITRs/PTRs and amount of CES enabled sites.
The cache scalability is all about to get the DITRs/PTRs in optimal
places.

In a nutshell, CES is all about taking out the multi-homed PI
addresses from the DFZ - transforming them into PA-addresses (RLOCs,
which are aggregated) and provide a multi-homing solution for IPv6 so
it will not further expand the size of DFZ.

Thanks for your patient and education - my sincere apologizes for
being slow on learning. But is has been useful for me, sorry for
wasting yours and Noels time.

-- patte

---

From: Robin Whittle <[email-masked]>
Date: Wed, 27 Jan 2010 13:15:57 +1100

Short version:      Discussion of placement of ITRs closer to the
                    sending hosts.

                    Why I think DITRs/PTRs will initially handle most
                    packets and why ISPs will increasingly be
                    motivated to install their won ITRs.

Hi patte,

> For a conventional host A (PA address) -> CES enabled B host:

Host A could be on a PI address too.

> users access the same services and the cache will not grow that much.

Yes.  Generally speaking, the smaller and cheaper you can make an
ITR, the more you can have of them, the closer they can be to the
sending hosts and so the less mapping each ITR will have to cache and
the fewer packets each one will need to tunnel.

Ivip is the only CES architecture to have optional ITR functions in
the sending host (ITFH).  (LISP-MN mobile hosts have this option, but
I think that this is the wrong approach to mobility.  The TTR
Mobility [1] architecture would work OK with LISP and would be far
better.)

If the sending host can have an ITR function added, and if it is:

  1 - On conventional (PA or PI) or SPI space - not behind NAT, and:

  2 - Has fast, reliable, low-cost access to one or ideally two
      full database QSDs.

then it would be good to put the ITR function there.

For Modified Header Forwarding (MHF) [2] to operate, all the routers,
including DFZ routers between this new ITR location and all ETRs
would need to be upgraded.  For the rest of this discussion I will
assume the ITRs use encapsulation rather than MHF.

I will probably add an option for an ITR or ITFH (to be behind NAT.
It would make a two-way tunnel to a caching query server QSC outside
NAT, on a conventional address, and use keepalives to maintain the
tunnel.

Placing the ITR in the sending host typically involves no hardware
costs at all.  One minor disadvantage is that there are map query and
response packets going back and forth, and occasionally a map update
packet to the ITR.  Also, the encapsulated packet is longer, and if
the shortest MTU between it and the ETR is the PPOE link which a DSL
modem uses, then this would result in slightly shorter packets being
able to be sent by the sending host applications.

I don't suggest putting an ITR function on a sending host which
relies on a wireless link, such as a 3G link.  Maybe it would be OK
on a WiFi or WiMAX link which was consistently fast, reliable and
inexpensive.

With these caveats, the best place to put the ITR function is in the
sending host.

> The cache scalability is all about to get the DITRs/PTRs in optimal

When Ivip/LISP is first introduced, I think DITRs/PTRs will handle
most or all of the packets.

When an end-user network is using SPI/EID space (and so is using one
or two ETRs) it will presumably also have an ITR somewhere.  It may
have the ITR in the same box(es) as the ETR(s) but it may rely on
the ISP to have an ITR.   It doesn't really need an ITR, since it
could always let the packets to to DITRs/PTRs.

Over time, as the CES architecture becomes more widely used, I would
think that ISPs would want to introduce their own ITRs, partly to
ensure their customers' packets which are sent to SPI/EID addresses
are handled reliably, locally, rather than depending on potentially
busy DITRs/ITRs.

For any ISP which has ETRs in its network - or has any of its
customers running their own ETRs, such as at their own home or
office, using the the PA address which comes with the DSL etc.
service - then these ISPs have some SPI/EID using customers.

Presumably, some of their customers are going to be sending packets
to these SPI addresses.  The ISP doesn't necessarily need to provide
an ITR, but by doing so, it avoids the expense of those packets going
out to the DFZ to the nearest DITR/PTR and then coming back,
tunneled, to the ETRs in the ISP's network.

Over time, I would expect low cost ITRs made from ordinary servers
and freely available software would be common in ISP and larger
end-user networks.  Also, conventional routers may be upgraded to do
this, and new ones would probably have ITR functions built in.  At
the same time, host operating systems would either have ITR functions
built in and optionally enabled, or be available as separate packages
which could be installed.

Any hosting farm would be keen to get ITR functions in each such host
OS, since it has large volumes of outgoing packets, and a growing
proportion will be to hosts with SPI addresses.

In the long term, I expect DITRs/PTRs to handle a small proportion of
packets - but at the start, I think they would handle most of them.

If the DITR only handled MABs (Mapped Address Blocks - DFZ advertised
prefixes containing SPI space) containing micronets which were mapped
to ETRs in a certain area, such as Australia, then it would suffice
to have the one or more DITRs for these MABs in Australia.  However,
the whole idea of a CES architecture is to enable these SPI (EID for
LISP) addresses to be used anywhere.  So in general, DITRs/PTRs need
to be scattered around the Net, topologically speaking, to generally
minimise the extra path length between any ITR and any ETR, and to
spread the load reasonably evenly between them.

> it will not further expand the size of DFZ.

This is part of it.  I will write my attempt to summarise the purpose
of CES in a separate message.

> wasting yours and Noels time.

I don't think it is a waste of time, since I guess there are a few
lurkers who are also finding it hard to understand these things.

  - Robin

[1] http://www.firstpr.com.au/ip/ivip/#mobile

[2] http://tools.ietf.org/html/draft-whittle-ivip-etr-addr-forw
    http://www.firstpr.com.au/ip/ivip/PLF-for-IPv6/