Subject: [Tsvwg] RE: multihoming issues via SCTP

From: Coene Lode <[email-masked]>
Date: Tue, 11 Sep 2001 11:07:05 +0200

Some comments below.

> -----Original Message-----
> From: Iljitsch van Beijnum [mailto:[email-masked]]
> Sent: maandag 10 september 2001 17:17
> To: Coene Lode
> Cc: [email-masked]'; [email-masked]'
> Subject: Re: multihoming issues via SCTP
> 
> 
> On Mon, 10 Sep 2001, Coene Lode wrote:
> 
> > This document describes issues of the Stream Control Transmission
> > Protocol (SCTP)[RFC2960] in regard to multihoming on the 
> Internet. It
> > explores cases where through situations in the internet, single
> > points-of-failure can occur even when using multihoming and what the
> > impact is of multihoming on the host routing tables.
> 
> The main problem is the idea of paths from an interface on 
> one host to an
> interface on another host. This could work if the hosts are 
> both connected
> to two networks that are not interconnected, but not on anything
> resembling the Internet.
> 
> For instance, consider host A having interface 1 connected to 
> ISP X and
> interface 2 to ISP Y. By coincidence host B it is 
> communicating with, is
> connected to the same ISPs, but the other way around: 
> interface 1 to Y and
> interface 2 to X. So now host A tries to connect to host B 
> over interface
> 1, ISP X, ISP Y and interface 1 of host B (A1->X->Y-B1) and also
> A2->Y->X->B2. This should work just fine, although two ISPs 
> instead of one
> in each path may not give the best possible performance.
> 
> But now there are three ways in which the connection can 
> fail: either of
> the ISPs may fail, or the connection between them.
> 
> This means there have to be more paths:
> 
> A1->X->Y->B1
> A1->X->B2
> A2->Y->B1
> A2->Y->X->B2

yes,  but does routing allow them to be used? We got the impression that at
a certain time you can have only one path through the network for a certain
destination address. 

> 
> However, this assumes both that the output interface stays 
> the same over
> the life time of a connection and that the end host is the 
> one making the
> decision how packets are routed to the outside. Both are 
> unlikely.

Well, but that is how it is done with SCTP. SCTP specifies the destination
address  and IP should try to route based on that IP address to B. If B1 is
taken as destination address then the source address should be the
corresponding A1(and that is something done by IP in host A). Host A has to
make a routing decision based on the destination address whether it likes it
or not.

> If an
> output interface fails, hosts may simply reroute outgoing 
> packets over the
> other interface. 

That is a possibility, but that depends on who finds it first. If SCTP
detects that the interface has gone(via the heartbeats that aren't
acknowledge), then it will changeover to another destination address(thus
taking the other interface)
If on the other hand routing finds this, then routing will reroute
everything over the other interface(but with the same destination address as
before(and giving SCTP the impression that the path presently used is still
up: so SCTP has been fooled into believing that both paths are still up) 
The question for me is: Does it happen that the host changes it routing
tables?

> And in most cases, routers will reroute packets over
> another path if a connection to an ISP fails. So the other 
> side will often
> still see incoming traffic from a remote address that is no longer
> reachable for outgoing packets.

Uhh, doesn't that mean that then the alternate address should be used
instead of the original addresses? 
And as far as I understand something from routing and routing protocols,
doesn't that take time? 
The fact is that I come from a background where rerouting(around failures)
is immediate and we don't wait for routing protocol till they found some
route around. In such a configuration, using the concept of paths through
the network is indeed a stupid concept, as there are in theory a infinite
number of paths through the network(every router has 2 or more routes
towards a certain destination at any time). But that is not what we have
been told what routing does in IP networks. Changing routes in IP networks
seem to involve first searching for them and waiting for it till the routing
protocol converges. And it is during that convergence faze that the packets
will be routed to probably the middle of nowhere(or better said: they are
dropped)
If I happen to be wrong in this assesment, then let me know. That would mean
that we could sleep better, put our trust in routing  and that all the work
in SCTP concerning multihoming wasn't really necessary.

> 
> Also, I don't like the idea of a protocol sending keepalives 
> over a path
> it is otherwise not using. This is a waste of resources and 
> can trigger
> inefficient use of different kinds of caches. 

Well the alternate routes are not only used for heartbeats, but if the SACK
don't get back, then the alternative routes are used for the SACK, so they
are not really idle in the strictest sense of the word.
What about caches: well caches (in my very humble opinion) shouldn't be
there in the first place, especially the implicit ones, as you point out,
they will contain irrelevant info. Well good, if they contain irrelevant
info, what the hell are they doing anyway in may path apart from having
absolutely no idea what is going on. The same is applicable for NAT's, that
is also for SCTP not such a nice sight, especially if the SCTP multihoming
is involved.

> Also, it won't 
> do much good
> unless there are three or more paths of which at least two fail, a
> situation that is not likely to be very common.

Murphy law: Failures of links, interfaces and routers will happen.(once upon
a time, we believed this tale that failures wouldn't happen, but then
reality and a lot of users thought us otherwise)

> Iljitsch van Beijnum

Yours sincerly,
Lode

_______________________________________________
tsvwg mailing list
[email-masked]
http://www1.ietf.org/mailman/listinfo/tsvwg

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 11 Sep 2001 11:53:25 +0200 (CEST)

> a certain time you can have only one path through the network for a certain

If we define a path as being from a source interface to a destination
interface, then the number of paths is only limited by the product of the
number of interfaces on both sides. Obviously, many of those paths will
share a lot of infrastructure, but they won't be identical.

I think the main problem is that in most cases a default route will be
used, which is tied to one interface. That means the paths originating at
the different interfaces are effectively identical.

> address  and IP should try to route based on that IP address to B.

Right. So the decision about the output path is done in IP and SCTP
probably can't control this.

> If B1 is taken as destination address then the source address should be the
> corresponding A1(and that is something done by IP in host A).

I suppose it is possible to do this, although TCP doesn't.

> taking the other interface)

The fact that host B is suddenly sending packets to interface 2 instead of
1 of host A, doesn't influence outgoing packets on host A. Assuming B1 and
B2 are both routed over the same interface (which is likely unless the
host has a full routing table, something that never happens today) host A
will have to detect the failure of interface 1 and reroute over interface
2.

> The question for me is: Does it happen that the host changes it routing

Depends on the host. I think most will do this, but it never hurts to
experiment. If this doesn't happen automatically when using defaults, it
is always possible to run RIP.

> instead of the original addresses?

If the router does this, how would SCTP know a different source address
has to be used?

> And as far as I understand something from routing and routing protocols,

Of course. Depends on the interface, though: some can detect failures very
fast (less than a second) while others won't see failures at all and there
has to be a timeout in the routing protocol. This takes 30 to 180 seconds.

The former is likely in point-to-point links, the latter in switched
environments (Ethernet, ATM).

> Changing routes in IP networks

There are many different routing protocols, all with different properties.
For instance, RIP waits for minutes to decide that a destination is
unreachable and then removes it from the routing table. OSPF and IS-IS on
the other hand, will detect all failures in 30 seconds, and interfaces
that go down immediately. Then the update is flooded through the network
and all routers recompute the topology. In BGP the detection process is
much the same but each router recomputes the routing changes individually.
So in BGP it may take a few moments for routing to stabilize.

Routers do not necessarily drop packets while updating routing
information, but they may route them to unreachable destinations or in
loops for a while.

> are not really idle in the strictest sense of the word.

Well if everything is working, then the other paths aren't used, right?

> there in the first place,

Some routers use a "route cache" where the most recently used routing
information is stored. ARP or neighbor discovery are also a form of
cacheing. Why should processing all of this be triggered just to see if a
path is available if you don't intend to use it?

> > Also, it won't do much good

> reality and a lot of users thought us otherwise)

I'm not saying failures won't happen, just that in most cases the
heartbeats won't help you. If there are only two paths, you'll have to use
the second one when the first one fails. So why bother to see if the
second path is alive? You'll find out soon enough. If you have three paths
and one fails, then each of the others still work so no need to use
heartbeats again. Only when two paths fail at the same time, it helps to
know which secondary path has also failed so you know to avoid that one
and pick the other.

If doing something and doing nothing yield the same results, the latter is
preferable.

Iljitsch

---

From: Michael Tuexen <[email-masked]>
Date: Tue, 11 Sep 2001 12:01:56 +0200

Dear all,
see my comments below.
Best regards
Michael

> share a lot of infrastructure, but they won't be identical.
RFC 2960 defines a path as a destination transport address.
> probably can't control this.
That is why the above definition was used in RFC 2960.
> know which secondary path has also failed so you know to avoid that one
Well, you have the option to disable heartbeats in SCTP, if you do not
like them.