Subject: Re: The internet architecture

From: [email-masked] (Noel Chiappa)
Date: Mon, 29 Dec 2008 11:28:52 -0500 (EST)

> From: John Day <[email-masked]>

    > Multihoming is fundamentally a routing problem.

I have been thinking this for some time too, and it's especially true/clear
when the multi-homing in question is site multi-homing, and not
host-multihoming (which is much rarer, is my impression). You clearly have
two alternative paths to the destination, and have to pick.

Which raises the question of 'why not have the routing do it', and that's a
very valid question. In a routing architecture which had better - read
non-manually configured - _scopes_ for routing information, and more
automatic aggregation (or hiding, to use the more general concept), I think I
would agree that probably it should be hidden in the routing. (The "probably"
is because I'd have to see the actual details.)

However, we have to 'go to war with the army we have', and the current routing
architecture (which includes the _functional interface_ with the _rest_ of the
architecture, not just how it's arranged internally - and the former is
basically impossible to change) makes it impossible to do that.

    > It is a problem of routing not be able to recognize that two points of
    > attachment go to the same place. Portraying it as anything else is just
    > deluding yourself.

I would agree with this, except I defer to the 'get down off an elephant'
principle. If both points of attachment are bound to a single transport level
entity, then it ought to be relatively easy, and not involve the routing at
all, to detect that both points of attachment lead to the same entity.

	Noel
_______________________________________________
Ietf mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ietf

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 30 Dec 2008 08:55:20 +1300

Noel,

>     > Multihoming is fundamentally a routing problem.

(snip)

> all, to detect that both points of attachment lead to the same entity.

It ought to be, but unfortunately we have confounded the transport entity
namespace with the network entity namespace with the point of attachment
namespace.

    Brian

---

From: Christian Huitema <[email-masked]>
Date: Mon, 29 Dec 2008 14:55:53 -0800

> namespace with the network entity namespace with the point of attachment

Not really. Many applications are actively managing their network connections, and for a good reason. A network connection is not an interface to an abstract "unified network". On the contrary, a network interface implements a contract with a specific network.

Take the example of a laptop with Bluetooth, Wi-Fi, WIMAX and 3G. Four connections, with four different providers. Wi-Fi, through the access point, communicates with a broadband provider, maybe a cable company such as Comcast. WIMAX communicates with the Internet through a wireless provider, maybe Clearwire. 3G also offer some kind of Internet access, possibly through a different provider such as AT&T. And Bluetooth typically does not communicate with the Internet, but provides access to some local devices. You will note that the different providers have different rules for managing traffic. Behind each interface lies a different contract, a different type of service.

Is it possible to manage all these interfaces as if they were a single abstract point of attachment? Maybe. That would require a common management system. Can that management system be part of "the network"? Frankly, I doubt it. The management system will have to make arbitration between different services, deciding which part of the traffic goes which way. These decisions have economic consequences. Do you really believe that different providers will delegate these economic decisions to some kind of cooperative distributed system? If that was realistic, we would have network wide QOS by now...

On the other hand, the end system is in a very good position to implement these arbitrations. It has direct access to the requirement of the applications, and to the terms of each specific interface contract. Moreover, it can actually measure the quality of the offered service, allowing for informed real time decisions.

We can debate which part of the end system should implement these decisions, whether it should be the application or a common transport layer. I can see arguments either way. But the business reality essentially precludes an implementation in the network layer. Even if we did reengineer the network layer to implement a clean separation between identifiers and locators, the business reality will still be there. We will end up with separate identifiers for the different "provider contracts", and applications, or the transport layers, will have to arbitrage between these contracts.

-- Christian Huitema

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 30 Dec 2008 13:48:42 +1300

Hi Christian,

> Not really. Many applications are actively managing their network connections, and for a good reason. A network connection is not an interface to an abstract "unified network". On the contrary, a n…

It seems to me that you're agreeing with me. It's exactly because the three
namespaces I mentioned are mashed together by TCP/IP that applications have
to do what you describe, rather than just saying "open a connection to
Christian's laptop."

    Brian

---

From: Christian Huitema <[email-masked]>
Date: Mon, 29 Dec 2008 16:56:19 -0800

> > Not really. Many applications are actively managing their network
> connections, and for a good reason. A network connection is not an
> interface to an abstract "unified network". On the contrary, a network
> interface implements a contract with a specific network.
>

If "Christian's laptop" is the "transport" name space, and if the network entity namespace use different network entity names to designate the various "network contracts", then, yes, we probably agree. Although I am not sure that we should place too much emphasis on the name of physical entities like "Christian's laptop". What if the application process migrates from my laptop to my desktop?

-- Christian Huitema

---

From: Tony Finch <[email-masked]>
Date: Tue, 30 Dec 2008 03:02:59 +0000

On Mon, 29 Dec 2008, Noel Chiappa wrote:
>
> I have been thinking this for some time too, and it's especially
> true/clear when the multi-homing in question is site multi-homing, and
> not host-multihoming (which is much rarer, is my impression).

Most networkable consumer electronics ships with at least two network
interfaces. Host multihoming is only rare because it doesn't work.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
WIGHT PORTLAND: SOUTHEAST BACKING EAST 3 OR 4, OCCASIONALLY 5. SLIGHT,
OCCASIONALLY MODERATE IN PORTLAND. MAINLY FAIR. MODERATE OR GOOD, OCCASIONALLY
POOR.
_______________________________________________
Ietf mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ietf

---

From: Marshall Eubanks <[email-masked]>
Date: Mon, 29 Dec 2008 23:19:56 -0500

> interfaces. Host multihoming is only rare because it doesn't work.

Why do you say it doesn't work ? I have found host multihoming to be  
very useful at
times, especially if the local network uses several address blocks for  
whatever reason. (Note that of course
there is still a router routing this.)

And, if you use a software based router (which many do) that is  
strictly speaking host multihoming.

Regards
Marshall

---

From: Tony Finch <[email-masked]>
Date: Tue, 30 Dec 2008 17:53:44 +0000

On Mon, 29 Dec 2008, Marshall Eubanks wrote:
>>> true/clear when the multi-homing in question is site multi-homing,
>>> and not host-multihoming (which is much rarer, is my impression).
>>
> Why do you say it doesn't work ?

The kind of multihoming I am talking about is where you are using multiple
links for redundncy or resilience - i.e. the same reasons for site
multihoming. I did not mean other kinds of multihoming, such as multiple
addresses on the same link (which I don't think should be called
multihoming) nor links which connect to different networks - which
includes VPNs and gateways/routers. (The site-level equivalent of the
latter is private peering.)

The kind of multihoming I mean is the kind that trivially supports
mobility as a degenerate case, since mobility is just repeated failover.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
ROCKALL: SOUTHEASTERLY 5 OR 6, OCCASIONALLY 7, BUT 4 IN NORTH AT FIRST. SLIGHT
OR MODERATE, OCCASIONALLY ROUGH. RAIN OR SHOWERS. MODERATE OR GOOD.
_______________________________________________
Ietf mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ietf

---

From: Tony Finch <[email-masked]>
Date: Thu, 1 Jan 2009 05:22:03 +0000

On Tue, 30 Dec 2008, Michael Richardson wrote:
> Tony Finch <[email-masked]> wrote:
> >
> > The kind of multihoming I am talking about is where you are
> > using multiple links for redundncy or resilience - i.e. the
> > same reasons for site multihoming.
>
>   I think it says something to the extreme *rarity* of host multihoming
> (for systems that initiate connections) that there is even this confusion.
>   You can be multihomed with one physical interface.

NOT in the sense that I am talking about.

In fact the use of the term "multihoming" to describe multiple IP
addresses on the same link is a serious problem with practical
consequences.

I've been asked twice now in private email to clarify what I mean, so this
is going to turn into a massive rant about how the current Internet
architecture - as it is deployed, and as it seems to be developing - has a
completely broken idea of how to address endpoints. The multiple meanings
of the word "multihoming" relate directly to the multiple points in the
rant.

> This is rather more common for machines that mostly accept connections
> (web servers), as you can do relatively simple source address based
> policy routing.

In my experience policy routing is not what people use multiple addresses
on the same link for. Multiple addresses on the same link are used for
virtual hosting for application protocols that don't signal
application-level addresses within the application protocol. The canonical
example is HTTP over SSL, but POP and IMAP have the same problem. (People
sometimes hack around this design error in POP and IMAP by embedding the
virtual domain in the username, which is yet another example of why every
application protocol needs its own addressing architecture.) To
re-iterate, one computer providing multiple different services on the same
IP address is NOT multihoming, in the same way that one computer providing
multiple services on different port numbers is not multihoming.

The dual scenario is multiple computers providing the same service on
different IP addresses. The simplest deployment is to scale up on the
cheap by relying on round-robin DNS. In this case there are multiple
links, but they are often on the same layer 2 segment, so again not
multihoming in the sense that I meant. If you scale up further, the first
thing you do is get proper resilience with a load-balancing router (which
probably requires the architecturally impure NAT). If you require more
than one point of presence, the next step (beyond layer 2 techniques like
trunking ethernet vlans between multiple sites) is IP routing tricks, i.e.
anycast. For serious wide-area services, you are likely to use location-
and availibity-sensitive DNS to deirect users to the right instance.

Note that NONE of these techniques use the architecturally-supported idea
of multihoming. In fact most of them deliberately avoid it because it does
not work!

The Internet's idea of multihoming as supported by the architecture is NOT
what I consider to be multihoming. (The lack of support for my idea of
multihoming makes Internet connections klunky, fragile, and immobile, but
we all know that, and should be embarrassed by it.) The Internet's idea of
multihoming is reasonably precisely encapsulated by RFC 3484. This
specification breaks existing practice and fails to do what it is designed
to.

OK, so what is Internet multihoming? If the DNS resolves a hostname to
multiple IP addresses then those are assumed to be multiple links to the
same host.

Unfortunately there is no way for a client to make an informed decision
about which IP address to choose. RFC 3484 specifies how to make an
UNINFORMED decision, or at best, how one could in principle inform the
decision. However in order to be informed, a host needs to be hooked into
the routing infrastructure - but the Internet architecture says that the
"dumb" network need not explain its workings to the intelligent but
ignorant hosts.

As a result, having multiple addreses for the same hostname on different
links does not work. It never worked before RFC 3484, and though RFC 3484
tries to fix it, it fails because of the lack of routing information. What
is worse is it breaks DNS round robin - which I admit is a hack, but it's
a hack with 15 years of deployment, therefore not to be broken without
warning. Naïve implementations have broken round-robin DNS because RFC
3484 ignores it.

So to summarize:

A host has no way to use multiple links to provide redundancy or
resilience, without injecting a PI route into the DFZ - like the
anycasted root name servers.

Given multiple addresses for the same hostname, a client has no way to
make an informed decision about which is the best to connect to. This is
why hosts that support IPv6 do not work as well as IPv4-only hosts.

The Internet addressing architecture has a built-in idea that there is one
instance of each application per host, and applications are identified by
protocols (port numbers). There is no support for multiple instances of
the same application per host (i.e. virtual hosting) unless the
application has its own addressing.

There is no support for distributing an application across multiple hosts
(or multiple links to the same host) because address selection is blind to
availability and reachability - whether you consider them to be binary or
fractional values. If you try to use it you are either relying on the
non-kosher round-robin DNS, or you are likely to suffer failed or
sub-optimal connections.

In practice multihomed services (services with multiple redundant links to
the public Internet) do not use any of the techniques described in RFCs as
host multihoming, and often use techniques that are contrary to the
architecture or outright protocol violations (e.g. Akamai's use of CNAME
chains).

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
FORTIES: NORTHERLY 4 OR 5, OCCASIONALLY 6 LATER IN EAST. SLIGHT, BECOMING
MODERATE OR ROUGH. SHOWERS. MAINLY GOOD.