Subject: [RRG] Moving towards a new set of Networking APIs

From: RJ Atkinson <[email-masked]>
Date: Thu, 14 Aug 2008 08:59:06 -0400

Folks,

For some long while it has seemed to me that the C/C++ programming
community needs a widely available, openly specified, abstracted
networking API.   I've been advocating this for years now in
various talks various places.

Such an API ought to be a functional replacement for the BSD
Sockets API.  Historically, BSD Sockets shipped before the DNS
existed, so that API was forced to be much less abstract than
it ought to have been.  A new API would replace IP addresses
with FQDNs, transport protocol and port with Service Names,
and so forth.

Java has long had just such an API.  Nearly all Java programmers
use that abstracted API in preference to the lower-level
networking APIs also available in some Java implementations.
This has meant that nearly all Java programs are oblivious
to the differences between IPv4 and IPv6 (and a range of other
low-level details irrelevant to nearly all applications).

If such an API were designed properly, it would be entirely
oblivious to (and agnostic towards) IPv4, IPv6, and any of
the proposals here that I am familiar with.  So this notion
of adding such an openly specified API is entirely neutral
with respect to any specific proposal forward.

I am told that Stuart Cheshire advocated adding such an
openly specified API during the IAB Plenary in Dublin.
If that report is accurate, one could do worse than to talk
with him about Apple's experience with their abstracted API.

One imagines this could be implemented either as an OS interface
or as a library, whichever way an implementer chose to use.

So I'm fully in agreement with tli.  We ought to move to a new
networking API for C/C++ programs.   Once such an abstract API is
specified, then the existing Sockets API ought to be marked as
deprecated, although obviously implementers could (and very likely
would) continue to support the BSD Sockets API as well. [1]

I'm not familiar with Apple's C/C++ abstracted networking API,
but that might be one candidate for putting together an openly
specified networking API in an RFC.

I'm not sure whether this topic is in/out of charter for the
Routing RG.  The IAB's Internet Architecture list might be
an alternative venue for this thread if it doesn't belong here.

Yours,

Ran Atkinson
[email-masked]

PS:  ILNP can work fine with the BSD Sockets API, so I really
don't think that any "legacy" applications need to change to
work properly over ILNP.  I do think that most applications
would benefit from using a more abstracted API, whether one
is using IPv4, IPv6, both, or something else.

[1]  IETF documents, even standards-track documents, aren't laws.
On a good day they are just strong recommendations.  No implementer
is forced to follow any RFC, although many implementers choose to
follow most RFCs most of the time -- out of enlightened self-interest.

--
to unsubscribe send a message to [email-masked] with the
word 'unsubscribe' in a single line as the message text body.
archive: <http://psg.com/lists/rrg/> & ftp://psg.com/pub/lists/rrg

---

From: Robin Whittle <[email-masked]>
Date: Sat, 16 Aug 2008 21:40:53 +1000

Short version:   I think it would be wrong to assume we can solve
                 or should try to solve the routing scaling problem
                 with a proposal which requires the vast majority of
                 hosts to run seriously changed OSes and
                 applications.

                 There are other approaches to the routing scaling
                 problem (including Ivip and LISP with PTRs) which
                 involve no host changes and which will be much
                 easier to have large numbers of end-user networks
                 adopt.  One reason is that these proposals provide
                 full multithoming benefits to 100% of incoming
                 traffic for all adoptors, including especially
                 early adoptors, without depending (as your
                 host-based proposal does) on how many other users
                 adopt the new scheme.

Hi Ran,

I understand some of the attraction of rearranging network
programming strictly along the lines of text-based FQDNs as
identifiers for every communication, with applications to use only
these - and with the OS to figure out how to do it, via whatever
facilities it has its disposal: IP addresses, multihoming  and
mobility arrangements, hooks into systems which spread the load of
traffic over multiple paths etc.

   (Still, how do you do DNS?  You need to rely on fixed IP
    addresses for some of that - DNS can't depend on DNS.)

I suppose that by creating the appropriate underlying mechanisms,
the OS could do the communication in ways which would lead to an
indefinitely scalable routing system - perhaps even keeping the
current BGP arrangement and having purely PA addresses for all
end-user networks, and without any router-based core-edge separation
arrangement such as map-encap.

However, I can't see how such a system could be introduced so as to
be widely adopted, because there are very few immediate benefits to
those who adopt it: the application and operating system
programmers.  There are also very high up-front costs for those
people who must do the work of adopting it.

I can't see how a proposal such as yours could provide what is
needed for widespread adoption: robust benefits for all adoptors
irrespective of the level of adoption so far.

Each application would need to be compilable via its existing
network APIs and with the new one, making it harder still to
maintain and debug.

For some applications, the new API might be fundamentally simpler to
use than the current one.  For others - especially those which rely
on IP addresses of correspondent hosts at present - it would be much
harder to program the application using the new API, unless that
aspect of the application (which might be its entire mode of
operation) would gain no benefits such as working with the new
scheme's approach to multihoming or mobility. (In that case, why
bother rewriting it to the new API?)

I think the primary reason you want the new FQDN-centric networking
API to be very widely adopted is to solve the routing and scaling
problem.  It may also have appeal of elegance to you and to others,
and in some scenarios it may be easier to use than the current APIs,
libraries etc.

The trouble is that if this is to be *the* way of solving the
routing scaling problem, it can only be effective if the vast
majority of applications and OSes adopt it.

Broadly speaking we need a routing system which scales to millions
(some insist billions) of multihomable end-user networks, rather
than the current situation of 10s of thousands or so (my guess).
Let's assume there are 100k multihomed end-user networks today,
using a large fraction of the 260k currently advertised prefixes.
(The real figure is no-doubt less.)

So you even if the new system places no extra load on the routing
system (which is unlikely to be the case) you need about 90%
adoption (very broadly speaking) to retain the current burden on the
BGP system when you have 1M end-user networks.  If you want to scale
it to 100M end-user networks, then very broadly, you need 99.9%
adoption.  (It is more complex than this of course, and I think the
adoption requirements are still more onerous.)

So in order to achieve your goal, you need virtually all
applications and OSes using the new API.

I assume the reason you would go to all this trouble is to avoid
arguably kludge-like alternatives such as map-encap or any other
router-based core-edge separation system, including Six/One Router.

As I understand it then, your proposed solution to the routing
scaling problem would be entirely host-based in the data plane.
There may be special additions to the DNS system etc. but there
would be no special router functions, no ITRs, ETRs etc.

Based on this potentially erroneous understanding of what you are
proposing, I think your system only provides substantial benefits to
its adoptors - programmers of applications - (assuming the OS work
has already been done, which is a similar chicken and egg problem)
once the system is in very wide use.

If 10% of all web clients use the new system and 10% of the web
servers (percentage of total clients and servers in use) then on
average only 1% of the communications will have any benefit in terms
of the new scalable approach to communication.  For each user with
an upgraded client or server, they will only have 10% of their
communications running with the new system.

It would be different if a particular mode of communication was
purely between hosts running a single piece of application code - a
single actual application program. Then by upgrading that single
program, and by having existing users update to it (assuming their
OSes were already upgraded), then all users would have 100% of this
particular type of communication using the new system.

Assuming the more common scenario of diverse separate applications
and servers all communicating with each other, a proposal such as
yours faces very difficult barriers to adoption.

Let's say you somehow had all OSes upgraded.  One day, everyone
might be using upgraded code, but for the period such as 4 years
afterwards, at most 50% of computers would have the new API.

Now lets say there were 10 kinds of client and 10 kinds of server in
use - each with 10% adoption. (In reality, there may be 50% of
users covered by a single program, which could be an easier
situation than what I present here.)

Let's say you get 2 of the servers and 2 of the client applications
upgraded.  On average, only 10% of server installations will be
upgraded, since only half the computers on which a server is running
will have an upgraded OS.  Likewise, only 10% of client
installations can use the new API.  So you have only enabled 1% of
traffic to use the new system.  This is with investing 20% of the
effort you would need to get everything upgraded.

So you would have done this massive amount of work - 20% of the
final task of upgrading everything - and there are benefits to only
10% of each adopting user's traffic.  You would have have had
essentially no impact on the scalable routing problem, because 99%
of the traffic (for this type of client-server communication) is
still using current methods.

I think the only way of solving the scalable routing problem is by
devising a solution for it which provides immediate, substantial,
benefits to everyone who adopts it.  This requires that the benefits
do not depend at all on how many other people have adopted it.

In principle, LISP with PTRs (Proxy Tunnel Routers) or Ivip with
OITRDs (Open ITRs in the DFZ) does just this.  The people who adopt
this new form of end-user network address space immediately get
multihoming in a scalable fashion for 100% of their incoming traffic
(without having to have their own BGP prefix for each end-user
network).   Most wouldn't care about how scalable their newly
adopted system is.

The reason they would adopt it is that it gives them multihoming in
on a much less expensive basis than today's BGP-based techniques
allow.  In particular, it would enable them to cost-effectively
multihome a small number of addresses, such as one or a few IPv4
addresses (or IPv6 /64s) - far less expensively than current
techniques allow.

The key to this is PTRs and OITRDs.  All adoptors, including the
very first, get full multihoming for all incoming traffic, even if
no-one else has adopted the system at all.

The only way your approach can encourage application programmers or
application users to adopt your new API is by whatever benefits it
brings to them.  They generally won't care that their system will be
less of a burden on the BGP system.   Yet being a host-based system,
with no equivalent of PTRs or OITRDs, the benefits each adoptor gets
depends entirely on what proportion of hosts they communicate also
have the updates (both to the relevant client or server application
and to the OS).

I think it would be extraordinarily difficult to motivate
application programmers to add this new API to their code.  They
need to keep the old API, library etc. interface for many years,
because it will be years before a sufficient number of hosts have
the upgraded OSes.

The basic problem is that even after they have made the effort, the
benefits which accrue to their users depend entirely on how many
other people (whose hosts they communicate with) have also adopted
the updated OS and the updated API in the relevant applications.

I am not saying there is no cause for devising a new API an some new
lower level protocols to support it.

What I am arguing is that this would be an extraordinarily highly
strung - risky, unlikely to succeed - approach to solving the
scalable routing problem.

If I though there was no other way of solving the routing scalability
problem, I guess I would support your approach - whilst being
mindful of the great challenges we would face getting it to the 90%
to 99% adoption levels needed to make the required difference to the
routing scaling problem.

However I am sure there are other approaches to solving the routing
scaling problem.

While they are not architecturally as pure and elegant as your
proposed system, LISP, APT, Ivip, TRRP and Six/One Router are all
intended to solve the scaling problem without requiring any OS or
application changes in hosts.

I am convinced that any of these systems would be easier to
introduce than what you suggest.

The proposals require changes as follows:

   LISP
   APT
   Ivip4e (encapsulation)
   Ivip6e (encapsulation)
   TRRP

      Changes:     New devices or new functions in
                   some border routers and/or some
                   internal routers and or (PTRs and
                   OITRDs) some other routers.

      Unchanged:   All protocols.
                   All host OSes and applications.
                   All remaining routers.

      Benefits:    (Assuming they all have a PTR, OITRD
                   etc. system to support packets from
                   non-upgraded networks.)

                   Immediate benefits to all adoptors,
                   independent of the the proportion of
                   other people who have adopted it yet.

  Ivip4f (ETR Address Forwarding)
  Ivip6f (Prefix Label Forwarding)

      As above, except that most or all core routers also
      need a modest upgrade - I guess one which could be
      done as a firmware update for many more modern routers.
      This makes it harder to introduce at all, but would be a
      more efficient and generally simpler approach: no PMTUD
      problems and no packet overhead.  In the long term,
      moderate upgrades to all routers is easy.

  Translation: Six/One Router (IPv6 only)

     Changes: and Unchanged: as for LISP, APT etc. above except
     there are no PTRs, OITRDs etc.

      Benefits:    As with your proposed scheme, there is no
                   support for doing multihoming with packets
                   coming from non-upgraded networks or hosts.

                   So the benefits to adoptors depend entirely
                   on how many other networks have adopted it.
                   Initially, this is probably too low to
                   motivate many people to adopt it, so adoption
                   would always remain low.

                   It would be very hart to get adoption up to
                   such high levels (99.9%?) where there was
                   very little traffic coming in not being
                   multihomed.  Only then would it be possible
                   for end-user networks to rely on the new scheme.

  Something like you suggest, with a new API:

      Changes:     All host OSes and applications, due to
                   the need to use a bunch of new protocols.
                   (Maybe also some enhancements to DNS.)

      Unchanged:   All routers.

      Benefits:    For end-users, depend very much on the
                   proportion of upgraded hosts their host
                   is communicating with.

                   The benefit starts off being very close to
                   zero, and it is hard to see how these benefits
                   are sufficient to get even 50% of people to
                   adopt it.  Even then, it is questionable how
                   much benefit the mulithoming is if it works
                   only for packets sent by other upgraded hosts.

                   Robust multihoming can only be achieved
                   *after* essentially all hosts in the world
                   have upgraded OSes and when all the
                   applications in use on other hosts have also
                   been upgraded.

Your approach is probably more elegant, since it would transform the
Internet into what it arguably always should have been - and because
it involves no "in-the-network" changes.

However, I am sure an approach such as your approach will never be
adopted.  The barriers are too high.  The benefits to the early
adoptors are too few.  The scalable routing benefits only
materialise when the vast majority of OSes and applications are
upgraded.

However, if LISP, APT, Ivip4/6e and TRRP work as intended, the
scalable routing benefits materialise immediately as more and more
people adopt the system and so don't need conventionally managed BGP
space in order to be multihomed.  Also, those early adoptors get
multihoming for 100% of their traffic, due to PTBs, OITRDs or
however it is in each system.

I think Six/One Router (if it operated as intended, and I am not yet
convinced it would) could provide benefits to adoptors in proportion
to how many other networks have adopted it.

> an alternative venue for this thread if it doesn't belong here.

I think any scalable routing proposal which depends for its
effectiveness on ubiquitous host OS changes - and worse still,
application changes - is a non-starter, especially when those
changes will be so hard to get people to adopt, due to the fact they
are very disruptive and provide very few benefits until most other
people have adopted them.

Since the RRG has other alternatives which promise scalable routing
without the hurdles of new OSes and applications, I think that one
of these approaches will be the optimal choice over any proposal
such as yours.

  - Robin

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 18 Aug 2008 09:32:02 +1200

Front-posting since this is a very slight change of tack.

I agree with Ran, and with the points made by Christian
undre the "On identifiers..." thread. But I think we should
think of this as something slightly more proactive than an
API; it's almost what is sometimes called a convergence
sublayer (and in Java, it's a run-time system). In any case,
with the correct abstraction, it could free up the discussion
of the routing layer from the current requirement to deliver
handles that look like IP addresses to the upper layers.
So I think it should be pursued but probably not on this
list.

   Brian