Subject: Re: [RRG] Moving the problem to the global mapping system - fast-push

From: Robin Whittle <[email-masked]>
Date: Fri, 11 Apr 2008 02:11:33 +1000

Short version:   There are many benefits to having fast-push of
                 mapping, so it is not just a simple trade-off.

                 For instance, with Ivip, there is no need to try
                 to anticipate all the ways end-users might need
                 to detect multihoming failure, or to make decisions
                 about how to change mapping, when to restore the
                 original mapping, how long a loss of connectivity
                 constitutes and "outage" etc.  The end-users do
                 all this themselves.

                 LISP, APT and TRRP developers need to try to make
                 their systems really complex and costly in order to
                 attempt to do as big a subset as possible of the
                 things end-users might want or need.

Hi Michael,

You wrote:

>> However, Ivip's mapping distribution system is completely different
>> from how BGP works and is optimised to handle much greater flows of
>> information, faster and more reliably than BGP.
>>
>>   http://tools.ietf.org/html/draft-whittle-ivip-db-fast-push-00
> 
> Hi Robin, you often refer to Ivip's "fast" mapping distribution method
> as an unabashedly positive thing. 

It is a very good thing in terms of what it does.  The challenge, of
course, is to show that it is worth the effort, and that there is a
business case for it being built.  There is lots of work to do, but
the above ID gives a pretty good idea of what I plan.  Quite a lot
of that ID is stuff which is repeated in the summary and analysis
document:

  http://www.firstpr.com.au/ip/ivip/Ivip-summary.pdf

The gutsy stuff is on pages 33 to 48.

> However, it seems to me that you've simply picked a different
> point the trade-off space -- by increasing the speed of mapping
> distribution, you increase the amount of mapping traffic as well.

I have two comments about this.

Firstly, it is not simply a trade-off where one gets more of one
thing and less of another.  Fast push clearly involves some extra
effort compared to something slower, but the benefits include:

  Modular separation of the multihoming restoration functions.

  Reduction in the size of the mapping information.

  Reduced ITR and ETR functionality.

  Greater security through simplification and modularization.

  Better support for the TTR model of mobility: IPv4 and IPv6
  mobility with generally optimal path lengths.

For a fuller discussion of these benefits, please see page 4 of
Ivip-summary.pdf.

The modularity is absolutely crucial.  All the other schemes provide
very limited options for the end-user to have their mapping changed
- by having more elaborate mapping data than Ivip's single ETR
address, and by having a *lot* more functionality in the ITRs and
ETRs.  Yet no matter how elaborate you make this stuff, you still
won't be able to do what some or many end-users actually want.

In a recent message, Scott Brim wrote a set of questions, which I
think the developers of LISP, APT and TRRP all need to worry about,
such as:

   Who discovers the problem, and how is it dealt with?

   What does the remote ITR do in response?

   How many intermediaries and systems have to be involved?

   How long does it take for the entire path, end-to-end, to
   stabilize again?

   Also, how far is the operational change flooded?

   To multiple mapping points in anticipation?

   To just the ones with traffic flowing?

   What happens when the situation is corrected?

Also, the whole question of "How much of a break in connectivity to
an ETR constitutes an "outage" requiring ITRs to select a different
ETR?" is likely to be answered by different end-users.  However, to
to add sufficient options into the mapping system and the ITR
functionality to cover most of the end-users' divergent needs would
be completely unworkable

How could 1000 ITRs currently sending data to the one ETR monitor
its reachability on a second by second basis without overloading it
with probe packets and with its need to respond to them all?  They
can't.

The Ivip system doesn't have to be designed to provide the answers
to such questions.  For every conceivable end-user and situation,
the end-users get what they want since they do it themselves -
however they like.  (As long as they are happy with a 5 second
latency in mapping changes, and with luck, this might be reduced to
2 or 3 seconds in the future.  LISP, APT and TRRP ITRs are not going
to respond much faster.)

With Ivip, the end-user has one multihoming monitoring system -
which won't overly burden the ETR, no matter how many ITRs are
sending data to it.  They set up their system (probably provided by
another company) to change the mapping according to whatever
criteria they like, including how long a loss of connectivity
constitutes an "outage".

Similarly, how are LISP, APT or TRRP ITRs going to rapidly respond
to the initial ETR coming on-stream again - for those end-user
networks who want the traffic to go back that way, but not for
end-user networks who want to avoid the first ETR for a good while
longer, or perhaps forever?

There's no way of telling what end-users will actually want and
need, but you can be sure end-users will will sometimes, or often,
want or need to changing the mapping to some arrangement they just
thought of right now.

Only Ivip can achieve this.  All the other systems are far to slow
at getting their mapping out to ITRs - unless with ALT or TRRP the
caching times were really short, in which case this places an
unreasonable burden on many ITRs and on the global query server system.

Although with Ivip, end-users need to supply their own reachability
detection system, and typically have it make their mapping
decisions, at least they can do whatever they like - and the
map-encap system doesn't have to invest any costly ITR-ETR
functionality trying to anticipate the sorts of things end-users
will actually want to do.

So fast-push makes Ivip modular and extensible to purposes end-users
really want, and which we cannot necessarily imagine at present.
Since there is a small fee per update, with Ivip we don't have to
fuss so much over "excessive" numbers of updates.

The average reptile might think it a pretty uninspiring trade-off to
change into a warm blooded creature, wasting all that precious
energy and so having to eat more, when it could be lazing in the sun
instead.  But this is not simply a choice in a linear sliding scale
between one thing and another: it enables a bunch of things to
happen which otherwise wouldn't happen, including much more
efficient enzymes, faster and more precisely tuned processes in
neurons, more optimised proteins and more efficient organs at every
level of the organism.  That leads to greater speed in cold
climates, new brain development possibilities etc.  Birds (10,000
species) and mammals (5,400) are doing pretty well compared to
reptiles (8,200).

A trade-off usually involves some kind of compromise.  The new thing
only does one good thing and it costs something to achieve it.

If a new thing does about five really good things - major conceptual
benefits, not just marginal improvements in existing things - and
involves some costs, then its not just a trade-off, it may be a
really great idea and a key part of the design which leads the total
system to be a far better solution than all the alternatives.

I don't agree that by increasing the speed of mapping distribution
that Ivip necessarily increases the amount of mapping traffic.  If
you souped up APT's flooding technique (initially it was a BGP
flooding technique) to send the mapping data everywhere in 5
seconds, this wouldn't increase the amount of mapping data to send.

APT would be much improved by this, and perhaps people would want to
send more mapping changes, but that is a separate issue.

One thing you could do, as I do with Ivip, is only send a single ETR
address.  There's no need for multiple options, ITRs (Default
Mappers in APT) making decisions independently etc. when the
end-user can control the mapping in 5 seconds.  So that reduces the
amount of data per mapping change.  Perhaps, as a result, there
would need to be more mapping changes, so maybe there isn't an
overall saving in the quantity of data to be sent - it depends on
how people would actually want to use it.

With APT, you have no obvious way of charging end-users per mapping
update, so "profligate" mapping changes (however defined) leads you
to exactly the same problem which bedevils BGP today: unfair cost
burdens on people who derive no benefit from a change instigated by
an end-user network.  Likewise with LISP-NERD.

Ivip's fast-push structure makes it easy for the RUASes (Root Update
Authorisation Systems) to charge per update, and so gain the revenue
they need to run most of the fast-push system.  Then, there is not
such a bad problem with "profligate", since the charges will be set
to ensure the costs are more than covered.  Then the end-users whose
changes would be considered "profligate" in an uncharged system are
not being antisocial - they are simply paying at least their fair
share of the costs of the fast push system.

In fact, it is a bit trickier that that - since there are also costs
for the far end of the fast push system which is run by ISPs etc.
Also, whoever runs full database ITRs and query servers is going to
have costs due to each update.

The Ivip business case is not complete, but it does have the
potential to charge end-users per update, and so avoid the "tragedy
of the commons" situation which is a key part of today's routing
scaling problem.

The fast-push system is far more efficient at moving updates around
the planet than BGP.  This is because it has clearly defined inputs
and as many outputs as required, with minimal fast processing by
Replicators in between - whereas BGP has to be a mesh network of
independent devices which process the information and pass a
potentially new version of it to each of their neighbours.

> So even if each update can be handled more efficiently,
> there could be many, many more of them. So you may not have a win
> overall in terms of (processing per update) * (number of updates) *
> (number of devices doing the processing), even versus BGP. Notice I said
> you /may/ not -- if you can come up with some defensible worst-case
> numbers, people may be convinced. Also, people may disagree with me that
> this is a good metric. But I think you'll find yourself fighting an
> uphill battle.

It does seem to be uphill, since no other proposal attempts this.

At least Ivip has a birds-of-feather similarity to APT, in that both
involve hybrid push-pull systems, with local pull and robust notify
systems beyond wherever operators terminate the push system.

I think the benefits of fast-push are very powerful.

I think the task of fanning mapping data out reliably in a few
seconds, across the globe, even to a few hundred thousand sites,
doesn't seem impossible or impractical - given that we have this
marvellous network which can send packets from one side of the
planet to the other, quite reliably, for very low costs, in about
0.15 seconds.  To make the Replicator servers, we have cheap PCs
with multi-core CPUs, gigabytes of RAM for peanuts (the Replicator
needs little RAM and no hard drive) and gigabit Ethernet interfaces
on the motherboard.  The actual quantity of mapping data is not all
that high, even when there are a few billion end users doing
mobility, since they only generally need to send a mapping update
when they move 1000km or so.

  http://psg.com/lists/rrg/2008/msg00535.html 	
  http://psg.com/lists/rrg/2008/msg00832.html

Perhaps the fast push system would tempt people to do a lot of TE,
in the crude, non-explicit fashion which Ivip allows.  That is fine
- 5 second latency control of incoming traffic by controlling the
mapping of single IPv4 addresses or micronets of them.  Those
end-users will always be paying their way on the fast push system,
so whatever rate they do it at is good for business.  (Except as
noted above with the update burden on full database ITRs and query
servers - and on any ITRs which are currently handling traffic for
the micronets they are changing the mapping of.)

>> When trying to frame questions about something like the "mapping
>> function", as we get down to questions which provide meaningful
>> answers regarding important principles and implementation details, I
>> think the questions only tend to make sense if certain architectural
>> attributes can be assumed.
> 
> Actually, I tend to agree with you on that. I quite enjoyed your post on
> the subject. =) However, I think it's possible to set some quantitative
> constraints on possible designs. In particular, setting constraints that
> the design must be able to solve the problems we are setting out to
> solve (without re-introducing them elsewhere) seems quite reasonable to me.

Sure, that is a good constraint.

However, moving mapping changes (for instance each one due to a
change in reachability, or a desire to do TE) to a much more
efficient system than BGP does lead to a much better situation.  If
the new system was as costly and difficult as BGP, then simply
moving the changes somewhere else wouldn't achieve much.

>> My message a week ago:
>>
>>    http://psg.com/lists/rrg/2008/msg01091.html
>>    Taxonomy: 25 questions
>>
>> had several questions which teased out commonalities and differences
>> in the mapping systems of these proposals.
> 
> I saw that post, in fact, you can expect something from me or Dan in
> reply (probably Dan) in the next few days.

OK. Bill Herrin corrected various things I didn't know or got wrong
about TRRP - to which I haven't responded yet:

  http://psg.com/lists/rrg/2008/msg01094.html

If you can do the same for APT - and ideally if someone corrects any
mistakes I made about ALT and NERD - then at some stage I will
rewrite the questions and answers and put it on my site as a kind of
comparison between the current proposals.  Also, maybe someone can
suggest more questions to ask.

  - Robin

--
to unsubscribe send a message to [email-masked] with the
word 'unsubscribe' in a single line as the message text body.
archive: <http://psg.com/lists/rrg/> & ftp://psg.com/pub/lists/rrg

---

From: Michael Meisel <[email-masked]>
Date: Thu, 10 Apr 2008 14:12:41 -0700

>                  constitutes and "outage" etc.  The end-users do

I looked up trade-off in the dictionary, and this is what I got: "a 
balance achieved between two desirable but incompatible features; a 
compromise".

So yes, of course you get unique, desirable features with your fast push 
system. I'm just trying to point out that you lose an incompatible, 
desirable feature: less control traffic and less processing of mapping 
updates. The rest of your e-mail makes a lot of arguments that Ivip's 
features are more desirable, and more than worth the cost. You may be 
able to convince people of this, but only if you can also convince 
people that it can actually work in practice, given the extra traffic 
generated.

>                  things end-users might want or need.

I can't speak about the other proposals, but the only thing APT intends 
to do for end-users is make sure the Internet keeps working as the 
expect, hopefully at the prices they expect. In fact, optimally, they 
wouldn't even know APT exists! APT is intended specifically to solve the 
routing scalability problem in transit networks. And this is my point 
about constraints -- primarily, we need to make sure we're solving the 
problem at hand! Extra features are a bonus, but should come second.

> criteria they like, including how long a loss of connectivity

It is going to take a VERY convincing argument, including real data 
analysis and/or simulations, to convince me that you can allow all end 
users to flood the whole network with mapping changes at the full 
capacity of their connection without causing a massive, network wide 
DDoS attack. Think about why we still need TCP congestion control on 
individual hosts on today's network. And that's just for point-to-point 
traffic!

> fuss so much over "excessive" numbers of updates.

Assuming you are actually billing the person responsible. What if an 
unsuspecting user is generating a massive number of mapping updates 
because of a virus?

> seconds, this wouldn't increase the amount of mapping data to send.

The issue is not how fast a given update can be propagated, but how many 
of these you allow. Ivip's desirable features do not just rely on 
getting a single update out in 5 seconds, but getting ANY update out in 
5 seconds. This means having no rate limit on updates. This is the major 
problem, as I see it.

> systems beyond wherever operators terminate the push system.

True. =)

> moving the changes somewhere else wouldn't achieve much.

But what I'm trying to say is, if the new system is an order of 
magnitude more efficient, but has to handle an order of magnitude more 
control traffic, you're back where you started.

-Michael

---

From: Robin Whittle <[email-masked]>
Date: Fri, 11 Apr 2008 11:06:10 +1000

Short version:     Discussion of critique that Ivip would not be
                   able to cope with a flood of mapping updates.

                   I think the charge per update model answers that
                   critique - but APT doesn't have such
                   arrangements.

                   Business models for deploying ITRs in APT or Ivip
                   which handle packets sent by non-upgraded
                   networks to APT-mapped or Ivip-mapped addresses.

                   Discussion of what sort of end-user networks the
                   various map-encap schemes are really aiming to
                   support - and therefore what number, since the
                   human population is presumably not going to keep
                   growing forever.

Hi Michael,

> balance achieved between two desirable but incompatible features; a

Thanks.  I didn't look it up - but the idea is that a properly
constructed fast-push system is a much better deal than a compromise.

> updates. 

The idea is that the updates are a lot easier to process than what
happens between BGP routers today.

Also, I think that current mass-market CPU and RAM technologies will
make short work of processing each update.

Likewise the Replicator servers will be cheap, highly reliable,
PC-like devices which will handle the update packets pretty
snappily.  They need to do some crypto on the received and sent
packets, but are basically choosing packets and fanning out their
entire payloads, not fussing with the internal details of particular
updates.  Each packet's payload would be 1200 bytes or so, which is
about 100 IPv4 mapping changes.

> The rest of your e-mail makes a lot of arguments that Ivip's

It is novel and a tricky thing to do, but it doesn't look
impractical to me.  Ideally I would have time to complete the
Internet Drafts and write code for the Launch and Replicator servers
in the next few months.  However all this will have to happen more
gradually.

If some folks want to do a programming project - I think this would
be interesting and valuable research.

> wouldn't even know APT exists! 

By "end-users" I meant those who run networks or have address space
mapped by the map-encap scheme.

I sense that APT is aimed pretty much at largish end-users similar
to those who have the resources and the need to get PI space today.
 If so, then APT may have pretty modest goals in terms of the total
number of end-user networks supported.  There are never going to be
tens of millions of such end-users, much less the billions some
people are keen for map-encap schemes to handle efficiently.

I think the other map-encap schemes aim for larger numbers of
end-user networks - and therefore presumably networks or assignments
of addresses for millions, hundreds of millions, billions or
whatever of end-users who are much smaller (financially, number of
people, address requirements, traffic requirements etc.) than
current PI using end-user networks today.

However, the more I look at discussions about LISP-ALT, the more I
see it being tangled up in ISPs and physical end-user networks of
the PI type as we know them today.  So I can't see how LISP-ALT is
going to be a light, easy, system for hundreds of millions of
end-users to have their own little EID space and have it mapped to
whatever ETR address(es) they like.

Ivip is intended to be used for end-user networks of all sizes -
from big global corporate networks to individual IPv4 addresses (or
IPv6 /64s) for cell-phones for billions of end-users.  It is not
just for small, or medium, or big "end-user networks".

I think the only way we can get a map-encap solution to the routing
scaling problem to be widely enough deployed is to make it a very
attractive form of address space for end-users of all sizes.  If it
is perceived as being second-class in some way, or not suitable for
the big networks, then many other networks will steer well clear of
it - in part because they intend to be big one day.

> APT is intended specifically to solve the

I agree.  The TTR approach to mobility can be added to any of the
map-encap schemes without any architectural change:

  http://www.firstpr.com.au/ip/ivip/#mobile  <<< New diagrams

The TTR behaves like an ETR, and the TTR <--> Mobile Node stuff can
be regarded as a separate system.  There could be multiple such
systems - no single technical standard.  As long as the TTR behaves
like an ETR, it can be used with any of the map-encap systems.
There is no way to detect or prevent TTR systems and their mobility
arrangements with mobile nodes being used with any of the map-encap
systems.

Its just that this TRR approach to mobility would work best with a
scheme in which end-users could change the mapping pretty quickly.
Probably, a few minutes would be OK.  Half an hour would be pretty
bad, but probably not prohibitively slow.  A few seconds would be
much better - which is what Ivip aims for.  The mapping change is
only to direct packets to a TTR which is closer to the current
access network than the TTR to which the packets are currently being
tunneled to.  A mapping change is not needed for maintaining
connectivity every time the mobile node gets a new access network or
care of address.

> DDoS attack. 

Here is the argument:  End-users with mapped address space (for
their big fixed, multihomed university network, or for their
consumer cell-phone's public, stable, mobile IP address - or IPv6
/64) are going to be paying some fee, such as  0.5 to 10 cents, per
mapping change.

That money flows to the RUASes which run the Launch servers and most
of the global cross-linked tree-structured Replicator network.  So
most of the fast-push system is run as a profitable business, with
the cost per update set high enough to ensure viability and profits.
 That cost will be pretty low per update, especially once the system
is scaled to hundred of millions of end-users.  Only an end-user
with sufficient financial motivation is going to fire off mapping
updates at a high rate - and if they do, the RUASes will be making
more money!

Some will, either because they are flying around the world on
aircraft and really want to use TTRs close to wherever they are now.
 But that will be only a small proportion of end-users at any one
time, and the mapping changes are not absolutely required - it is
just to minimise path lengths by choosing a TTR closer to their
current access network's connection to the rest of the Net.

Others will issue lots of mapping changes for frequent incoming TE
management of their presumably high-volume networks.  They too will
be doing so because it is worthwhile - more value to them than the fee.

Your paragraph above is a real challenge for APT.  You have no
charging mechanism for updated - and you also have some pretty
tricky work to do to ensure each ISP only admits updates from the
genuine end-user whose network the address space is part of.

   How are you going to scale the security arrangements to a
   authenticate each mapping update which is sent to an ISP
   by someone who pretends to be the end-user whose network
   uses the address space to which the update applies?

   Each ISP trusts every other ISP, so if a single ISP allows a
   hacker to send updates about mapping for some end-user
   network's address space, how are other ISPs going to know this
   is a hack, and stop this particular subset of updates to their
   Default Mappers?

   Since there is no charge per update, how are you going to
   prevent or deter the profligate sending of updates, which
   would need to be flooded to the entire APT island - or to
   the entire APT system if all APT ISPs linked together into a
   single global APT system?

> Think about why we still need TCP congestion control on individual 
> hosts on today's network. And that's just for point-to-point
> traffic!

In Ivip, each RUAS directly or indirectly authenticates each update
by it being admitted to their system after the end-user
authenticates themselves with either a username and password, or a
challenge-response crypto transaction.

http://tools.ietf.org/html/draft-whittle-ivip-db-fast-push-00#page-33

Then, they are charged per update.

> unsuspecting user is generating a massive number of mapping updates

They need to keep their authentication details secure.

But your question is similar to "What if a virus or worm gets the
end-user's PayPal or online banking username and password?"

If, for some reason, their mapping change authentication details
were misused in this way and this was a serious enough problem to
have a workaround for, their RUAS, or whatever UAS they directly
interface to, could be configured to reject more than a certain rate
of updates, and to notify them by email and expect a response,
before allowing more.

> 5 seconds. This means having no rate limit on updates. This is the major

It is not a problem if there is a small fee for each update.

>> systems beyond wherever operators terminate the push system.

I am critical of APT's non-charging approach to accepting updates to
be flooded across the APT network - and of the highly decentralised
authentication and admission system for those updates, which I think
LISP-ALT has a problem with too.

However, I think APT is the best of the map-encap systems other than
Ivip, because it gives a reasonably flexible arrangement of pushing
to a certain point and then using fast, highly reliable, local query
servers to pull data to the majority of ITRs, which are cheap
caching ITRs.

LISP-NERD can't scale well.

LISP-ALT and TRRP will delay initial packets.  So their mapped
address space will suck to a degree sufficient to cause most
end-user networks not to adopt it.

All the map-encap schemes other than Ivip give end-users slow
control over their mapping (unless ALT or NERD ran with extremely
short cache times, which places unreasonable burdens on the rest of
the network).

If APT had a more centralised and therefore robust method of
authenticating mapping changes, and charged a fee per change, then I
think it would be much more scalable.

If APT had fast push, then your Default Mappers wouldn't need to
fuss over reachability, complex mapping data, making decisions etc.
 The end-users could do all that much better themselves, and then
simply change the mapping to whatever ETR they liked.  Then the
mapping data could be much simpler - a single ETR address.
(However, simplifying the mapping like this to a single ETR address
removes the explicitly load sharing function of ITRs which APT, LISP
and TRRP currently provide.)

> control traffic, you're back where you started.

The fast-push system will be many orders of magnitude more efficient
than BGP.  It will have a charging model so it is a good business
proposition, without (ideally) the tragedy of the commons problem
which is at the heart of the BGP routing scaling problem we are
trying to solve.

I don't have a complete business model for why some operator of a
full database ITRs or query servers in Bolivia would want to have
these devices receiving and crunching on zillions of TE-related
mapping changes for micronets of east-Asian end-user networks
(assuming few Bolivian users are communicating with those end-user
networks, which is not assured).

However, whoever is pumping out those mapping changes is paying per
change, and presumably doing it because it is a cost-effective way
of balancing their traffic over multiple upstream links, in
real-time - or because they really want it for mobility.

If these TE-happy and mobile end-user network people (customers
directly or indirectly of the RUAS which runs the MAB which their
mapped address space is a part of) find some ITR operators are not
taking any notice of their updates, then perhaps there is a business
case for some of their fees to flow via the RUAS to pay reluctant
ITR operators to accept these changes.  In other words, the RUAS
would make itself more attractive to its customers by offering
payment to any ITR operators who needed inducement to accept every
mapping change the RUAS puts into the fast push system.

At least Ivip has a partially developed business model for mapping
change fees, for running the fast push mapping distribution system,
and for charging end-user networks by traffic volumes for packets
handled by the OITRDs (Open ITRs in the DFZ) which are operated by
the RUAS.

With APT, I think you need a business case why your equivalent of
OITRDs would be deployed widely.  For instance, an APT ISP in Chile
advertises the various BGP prefixes which cover all the APT-mapped
space, attracting packets from non-APT networks nearby.  Many other
APT ISPs do this as well, some in Chile, some in neighbouring
countries - so there are these "open" ITRs all over the world.
These "open" ITRs need to be widely scattered, near non-upgraded
networks, to ensure generally optimal paths for packets being sent
from those networks to whatever ETR in some APT network the address
space is mapped to.

Yet why should an ISP in Chile pay for their own ITR, and its
connectivity, to attract packets from nearby ISPs and to tunnel them
to ETRs in APT ISP networks all over the world?

Ivip has a business model where RUASes charge end-user networks for
the traffic which flows over the OITRDs they deploy.  End-users
choose Ivip-mapped address space, in part, according to which RUAS
is responsible for the MAB (Mapped Address Block) in which their
space resides - in part according to how extensive and capable their
OITRD system is and in part according to what the RUAS charges for
traffic on those OITRDs.  So competitive forces ensure RUASes are
motivated to deploy a nice, widespread, reasonably high capacity set
of OITRDs around the world.

An OITRD, or its equivalent in APT ("Proxy Tunnel Router" is the
LISP equivalent) may be a router by name, but it is probably going
to be implemented as a server, at least for the initial years of
introduction while such features are not a part of big-iron routers.

The OITRD just needs to be sitting somewhere at a peering point.
This costs money to own and run the server, and to pay for
connectivity to routers at the peering point - but it is not like
paying for dedicated fibre links to routers in far-flung places.  So
the cost of handling X amount of traffic in an OITRD will be far
less than the cost of an ISP actually providing access to the
Internet for X amount of traffic.  Still, there needs to be some
business case for deploying OITRDs or their equivalents widely
enough that they handle traffic from non-upgraded networks without
being overloaded, and ideally without excessively long paths to
whatever ETR the traffic is tunneled to.

  - Robin

  - Robin