Subject: RE: [tsvwg] multihoming with asymmetric endpoints

From: [email-masked]
Date: Fri, 15 Nov 2002 05:34:10 +0200

Hi Lev!

	I have some comments (inline)...

> -----Original Message-----
> From: ext [email-masked] [mailto:[email-masked]]
> Sent: 14 November, 2002 20:47
> To: [email-masked]
> Cc: [email-masked]
> Subject: [tsvwg] multihoming with asymmetric endpoints
> 
> 
> Hello Lode,
> 
> I have to find a solution of the problem with multihoming.
> 
> The case is referenced in draft-coene-sctp-multihome-03.txt. Our
> configuration is as shown on Figure 2.1.3.
> 
>     +------------+           *~~~~~~~~~*            +------------+
>     | Endpoint A |          *   Cloud   *           | Endpoint B |
>     |      1.2   +---------+ 1.1<--+     |          |            |
>     |            |         |       |->3.1|----------+ 3.2        |
>     |      2.2   +---------+ 2.1<--+     |          |            |
>     |            |          *           *           |            |
>     +------------+           *~~~~~~~~~*            +------------+
> 
> We are unable currently to apply another IP address to the Endpoint B.
> Endpoint A may only choose interface based on destination IP.
> 
> About such case section 2.1 said:
>     "In IP implementations the outgoing interface of 
> multihomed hosts is
>     often determined by the destination IP address. The 
> mapping is done
>     by a lookup in a routing table maintained by the operating
>     system. Therefore the outgoing interface is not 
> determined by SCTP.
>     Using such implementations, it should be noted that a multihomed
>     host cannot make use of the multiple local IP addresses 
> if the peer
>     is singlehomed. The multihomed host has only one path and will
>     normally use only one of its interfaces to send the SCTP datagrams
>     to the peer. If this physical path fails, the IP routing table in
>     the multihome host has to be changed. This problem is out of scope
>     for SCTP."
> 
> From other side the RFC2960 said:
>    "An endpoint SHOULD transmit reply chunks (e.g., SACK, 
> HEARTBEAT ACK,
>    etc.) to the same destination transport address from which it
>    received the DATA or control chunk to which it is replying".
> 
> The problem is following:
> The Endpoint A sends all the packages with 1.2 as IP source. 
> If 1.2-1.1
> broken it uses
> 2.2-2.1. (OS changes its routing table). Such behavior required in the
> draft.

	So the point here is that if the OS at Endpoint A changes its routing table it should start sending from network interface 2.2 setting its source IP address to that of interface 2.2.

	This way everything goes alright. Sending packets from one network interface setting the IP source address to the one used by another is a bad idea. It leads to problems such as the one you show here...

	I'm not an expert, but I would say that if you bind to INADDR_ANY the source IP address should be set to that of the physical source used. If you set the source IP address by yourself, you should take care with what you are doing... :-/

> The Endpoint B verifies two paths and defines that path 
> 3.2-1.2 failed. It
> starts to send data using 3.2-2.2. But it still must response on all
> HEARTBEAT and DATA chunks to the IP from arrived packets, i.e 
> 1.2. Those

	Here is the mistake... If packets go out from 2.2, then you should set the IP address consequently (or let the OS do it for you)...

	I hope it helps (and I hope I'm not wrong either :->).

	BR Iván Arias Rodríguez

> packages will not pass to the Endpoint A without special 
> arrangement in the
> network.
> I do not think that such special arrangement are always 
> possible. Or if
> they possible why to have the multihoming?
> Or the conclusion is the singlehomed host do not allow to use 
> multihoming
> at all?
> 
> Best regards,
> Lev
> 
> _______________________________________________
> tsvwg mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/tsvwg
> 
_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

---

From: Coene Lode <[email-masked]>
Date: Fri, 15 Nov 2002 17:04:24 +0100

Hello Lev,

Well Andreas and Ivan(thanks) beat me to it but they are essentially right.
It all boils down to the selection of the correct source address. If doen
correctly then the return msg will arrive on the interface that is still
connected/active... 

yours sincerly,
Lode

Siemens atea
atealaan 34          2200 Herentals, Belgium
E-mail: [email-masked]
Tel: +32-14-252081
Fax: +32-14-253212

> From: [email-masked]
> [mailto:[email-masked]]
> Sent: Friday, November 15, 2002 4:34 AM
> To: [email-masked]; [email-masked]
> Cc: [email-masked]
> Subject: RE: [tsvwg] multihoming with asymmetric endpoints
> 
> 
> 	Hi Lev!
> 
> 	So the point here is that if the OS at Endpoint A 
> changes its routing table it should start sending from 
> network interface 2.2 setting its source IP address to that 
> of interface 2.2.
> 
> 	This way everything goes alright. Sending packets from 
> one network interface setting the IP source address to the 
> one used by another is a bad idea. It leads to problems such 
> as the one you show here...
> 
> 	I'm not an expert, but I would say that if you bind to 
> INADDR_ANY the source IP address should be set to that of the 
> physical source used. If you set the source IP address by 
> yourself, you should take care with what you are doing... :-/
> 
> 	Here is the mistake... If packets go out from 2.2, then 
> you should set the IP address consequently (or let the OS do 
> it for you)...
>

---

From: Coene Lode <[email-masked]>
Date: Fri, 15 Nov 2002 17:31:56 +0100

A few comments and (hopefully )some explanations below..

> From: JinyangSHI [mailto:[email-masked]]
> Sent: Friday, November 15, 2002 1:34 PM
> To: [email-masked]; [email-masked]; 
> [email-masked]
> Cc: [email-masked]; [email-masked]
> Subject: Re:RE: [tsvwg] multihoming with asymmetric endpoints
> Importance: High
> 
> 
> hi,
> 
> >	This way everything goes alright. 
> >	I would say that if you bind to INADDR_ANY the source IP
> >address should be set to that of the physical source used. If you set
> >the source IP address by yourself, you should take care with what you
> >are doing... :-/
> 
> I think this is a problem of source-addreee selection.
> 
> We maybe have two strategies to select a source address for a patket: 
>    First one is just to look up in the routing table 
> according to the destination; 
>    but sometimes the route-table maintained by the OS can't 
> recognize the destination and will thus make some mistakes, 
> so in such cases, we need to select the source address by 
> ourselves, that is, to select the network interface by ourselves.
> 
>     I think Lev's problem is under the second strategy: If 
> the destination is 3.2, we will always select 1.2 as the 
> source addesss by ourselves. In such case, even the the IP 
> packet sent via 2.2 interface, the IP source address will 
> still be 1.2.
>     As a result, endpoint A go on sending patkets via 2.2 
> interface, these packet reach endpoint B, then B send SACK 
> back to 1.1, and all the SACK can't reach A, so transfering 
> fail. However the HeartBeat and HeartBeat-Ack between 
> 2.2--3.2 can still communicate as normal, the association is 
> always on, but we we can't transfer data from A->B.

to be more specific, the heartbeat send from B  to A will not fail(dest 2.2,
source 3.2 -> heartbeat from A to B will get back(dest = 3.2, source 2.2),
but a heartbeat from A to B using dest = 3.2 and source 1.2 will fail.
However that is based on the assumption that the heartbeact ack will take
source and dest from the received heartbeat(and switch them)(= a
implementation issue) 
A implmentation could do it differently and end up still working but with a
not so good source address.

> 
> But i think if we use the second strategy and select 1.1 as 
> the source address, we should let the packet sent via the 
> same interface.
> --------------------
> I had done a similar experiment before, based on Linux 
> environment. I take the first strategy and just let the OS 
> select the source IP address by itself.
> 
>     |  10.4.0.1  +---------+ x.x<--+     |          |            |
>     |            |         |       |->x.x|----------+ 10.2.0.1   |
>     |  10.3.0.1  +---------+ x.x<--+     |          |            |
>     |            |          *           *           |            |
> The following is the ethereal trace:
> 
> [01] INIT     chunk from 10.4.0.1 to 10.2.0.1;
> [02] INIT-ACK chunk from 10.2.0.1 to 10.4.0.1.
> 
> In the INIT chunk, we can find two IPv4 addresses: 10.3.0.1 
> and 10.4.0.1.
> Then send a message :
> 
> [03] DATA chunk from 10.4.0.1 to 10.2.0.1, 
> [04] SACK chunk from 10.2.0.1 to 10.4.0.1, 
> 
> [05] HEARTBEAT     chunk from 10.4.0.1 to 10.2.0.1, 
> [06] HEARTBEAT-ACK chunk from 10.2.0.1 to 10.4.0.1, 
> 
> [07] HEARTBEAT     chunk from 10.2.0.1 to 10.3.0.1, 
> [08] HEARTBEAT-ACK chunk from 10.4.0.1 to 10.2.0.1.
> 
> [09] HEARTBEAT     chunk from 10.2.0.1 to 10.4.0.1, 
> [10] HEARTBEAT-ACK chunk from 10.4.0.1 to 10.2.0.1.
> 
> The association is established between A and B, and data is 
> passing through. We can find that the active interface on 
> endpoint A now is eth1(10.4.0.1). Then we disable the 
> currently active interface (eth 1: 10.4.0.1 ):
> 
> [11] DATA chunk from 10.3.0.1 to 10.2.0.1, 
> [12] SACK chunk from 10.2.0.1 to 10.3.0.1, 
> 
> We look into the HEARTBEAT :
> 
> [13] HEARTBEAT chunk from 10.3.0.1 to 10.2.0.1, 
> [14] HEARTBEAT ACK chunk from 10.2.0.1 to 10.3.0.1, 
> [15] HEARTBEAT chunk from 10.2.0.1 to 10.4.0.1, 
> [16] No HEARTBEAT ACK
> [17] HEARTBEAT chunk from 10.2.0.1 to 10.3.0.1, 
> [18] HEARTBEAT ACK chunk from 10.3.0.1 to 10.2.0.1.
> 
> The association is still established between A and B, and 
> data is passing through. We can find that the active 
> interface on endpoint A now is eth0(10.3.0.1).
> 
> --------------------
> From the ethereal trace, we can have two conclusions:
> 
>     One, From [11], we can see that Lev's problem never 
> happen, because we use fisrt strategy.
>     Two, From [08]. When endponit A receives HEARTBEAT[07] 
> from 10.2.0.1 to 10.3.0.1, then it sends HEARTBEAT ACK[08] 
> back to 10.0.2.1, and with source address 10.4.0.1 selected 
> by route table, other than 10.3.0.1. I think this isn't 
> reasonable, in other experiments we have find it will lead to 
> mistakes, therefore in some case we need the second strategy, 
> that is, to selece source address by ourselves.

Yep, that's right
Maybe the routing software detect that interface 10.4.0.1 was deactivated,
and only then changed its routing table to used 10.3.0.1 as source for
outgoing packets. (just a guess on my part)
Because it could turn that it was not the link directly connected to the
interface that failed but a link further along the path. And then the host
has no idea that that particular path failed(the failed link is not visible)

To check that theory out, may an extra router box between 10.4.0.1 and x.x,
and then really plug the cable between the box and x.x.

> 
> Randy had mentioned source-address-selection in their book, 
> but he assumes that we will always know the route table, i 
> think it seems that in most time we may not alawys konw it.
> 
> So we try to study two problems: in whick case we need to use 
> the second strategy, and how we select the source address? 
> Especially we maybe nerver know how the OS maintains the route table.

We are clearly into implementation issues that can make the protocol work or
not.
> 
> I am not sure if i express myself clearly, wish for more 
> guidance and discussion -:)

Well, I tried, Hope I was clear enough(and that I added to the discussion). 

> 
> Best regards
> Yang
> 11/15
> 
> ---------------------------------
> Jinyang SHI
> Mail:  [email-masked]
> 
> Office:(86-10) 65392828-2759
> Nokia(China) R&D Center
> 
> Lab:   (86-10) 62283757
> Broadband Network Research Center,
> National Laboratory of Switching Technology and 
> Telecommunication Networks,
> Beijing University of Posts and Telecommunications(BUPT), P.R.China
> 

Thank you for supplying us with the result of your tests.

yours sincerly,
Lode

Siemens atea

---

From: [email-masked]
Date: Tue, 19 Nov 2002 07:48:50 +0200

Hi Lode, Yang,
thanks for your explanations. See some comments below.

Best regards,
Lev.

                      Coene Lode                                                                            
                      <Lode.Coene@sieme         To:      "'JinyangSHI'" <[email-masked]>,                 
                      ns.atea.be>               [email-masked], [email-masked],     
                                                Coene Lode <[email-masked]>                     
                      15/11/02 18:31            cc:      [email-masked], [email-masked]                       
                                                Subject: RE: [tsvwg] multihoming with asymmetric endpoints  

>not so good source address.
[lev]Such behavior requires from application on singlehomed side to
correlate received HEARTBEAT ACK with sent HEARTBEAT based on Heartbeat
Info field. Field may contain the identification of peer transport address.
>has no idea that that particular path failed(the failed link is not
visible)
[lev] It is true that described method provides only link redundancy, but
not a network redundancy.
It seems that network redundancy (without special preparations in network
e.g RIP) can be achieved only (a) if SCTP on multihomed side can control
the outcoming interface according to source IP or (b) singlehomed endpoint
converted to multihomed with assigning of additional IP. Both methods are
actually described in the implementation guide.
I think that if nor (a) nor (b) available, there is still the choice for
multihomed side to use or not to use multihomed local IP addresses (not as
it proposed in the implementation guide). The successful interworking will
depend on application behaviour on singlehomed side.
We are going to try with usage of two IP address by multihomed side.