Subject: [multipathtcp] Dealing with EBGP

From: Costin Raiciu <[email-masked]>
Date: Mon, 3 Aug 2009 15:33:30 +0100

Hi,

I kept wondering if "default on" MPTCP does break unmodified eBGP. I  
need to ask the following questions, to get things straight in my  
head. Experts (like Iljistch) please step in :)

1. Does eBGP bind the local source address before making a  
connection? If so, we don't have a problem - in our current proposal  
MPTCP will not even attempt to setup multiple flows
when the host locally binds a specific interface.

2. Are addresses used by the border routers "routable"? I assume they  
may well be, but they in fact don't need to. In the end, all you need  
for a next hop is some address  of a neighboring router (even  
ethernet will do, and so will local ip addresses). If that were the  
case, then multipath TCP would simply fail to find multiple paths -  
unless there are redundant links between the same servers (this seems  
unlikely).

Thanks,
Costin

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 4 Aug 2009 13:16:24 +0200

On 3 aug 2009, at 16:33, Costin Raiciu wrote:

> I kept wondering if "default on" MPTCP does break unmodified eBGP.

Don't worry about it. There is no real overlap between boxes running  
BGP and boxes that would benefit from MPTCP.

Especially if we provide an opt-out API call then software BGP  
implementations will use this quickly and there shouldn't be any  
problems even if people run BGP on boxes that enable MPBGP by default.

> 1. Does eBGP bind the local source address before making a connection?

Yes, it usually does.

> If so, we don't have a problem - in our current proposal MPTCP will  
> not even attempt to setup multiple flows

Strange.

> 2. Are addresses used by the border routers "routable"?

What is "routable"?

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 4 Aug 2009 13:52:24 +0200

> Sure, but that requires changes to BGP implementations. The big win  
> is if we can deploy it now and get only benefits, without breaking  
> any important apps.

Right, but in the BGP case I would expect people to simply turn off  
MPTCP for the whole box.

> As in can I ping a border router from my machine? Does it have a  
> public address?

Yes, the addresses are as good as always public and usually pingable,  
although if you're paranoid you may want to restrict incoming ping  
packets etc towards your router's addresses.

---

From: Costin Raiciu <[email-masked]>
Date: Tue, 4 Aug 2009 12:30:44 +0100

On 4 Aug 2009, at 12:16, Iljitsch van Beijnum wrote:

> Don't worry about it. There is no real overlap between boxes  
> running BGP and boxes that would benefit from MPTCP.
I agree with that, it doesn't make sense to implement mptcp on IOS.  
However, if the router is running on Linux (there are a few out there  
that use XORP or Quagga or something else, I imagine),
>> connection?
>> If so, we don't have a problem - in our current proposal MPTCP  
>> will not even attempt to setup multiple flows

Arguable. If an application only wants to accept connections destined  
for ip a, why should we advertise ip B to the other end. However, if  
the app binds INADDR_ANY, it seems natural it doesn't care and  
therefore we could use all addresses.

---

From: Joe Touch <[email-masked]>
Date: Tue, 04 Aug 2009 07:05:04 -0700

Costin Raiciu wrote:
>...
> for ip a, why should we advertise ip B to the other end. However, if the
> app binds INADDR_ANY, it seems natural it doesn't care and therefore we
> could use all addresses.

The app would need to:
	a) use wildcard addresses in its API (INADDR_ANY)
	b) never use the local endpoint used (i.e., never look
	at the socket's actual addresses)

This goes for whichever end of the connection it assumes can have
multiple addresses - local or remote.

Joe

---

From: Aleksi Suhonen <[email-masked]>
Date: Wed, 05 Aug 2009 05:53:04 +0300

Hi,

Costin Raiciu wrote:
> need to ask the following questions, to get things straight in my head. 
> Experts (like Iljistch) please step in :)

Iljitsch already responded, but I thought I'd pitch in too. It also 
sounds to me like you're already on the right path. I just wanted to 
reiterate a few points.

> If so, we don't have a problem - in our current proposal MPTCP will not 
> even attempt to setup multiple flows

With best current practices, EBGP binds the source address. However, 
with lazy configuration practices (often the case), it doesn't.

EBGP connections are(*) between two different operators. When the 
operators are peering at an Internet Exchange Point, the address space 
belongs to neither of the two operators. When the EBGP session's purpose 
is private peering or transit, the address space belongs to one of the 
operators according to the terms of their agreement.

A common practice is that an operator doesn't accept anyone else's 
address space in their IGP. This means that without the EBGP session, 
either (and sometimes neither) one of the operators wouldn't have any 
route to the addresses used by the EBGP/TCP connection, save for the 
directly connected routers.

When the connection between the two EBGP speakers fails, the EBGP 
session should die as fast as possible, so that alternative routes would 
be sought. EBGP suffers slightly from TCP's attempts to survive short 
connection outages. MPTCP might prolong this suffering.

> for a next hop is some address  of a neighboring router (even ethernet 
> will do, and so will local ip addresses). If that were the case, then 
> multipath TCP would simply fail to find multiple paths - unless there 
> are redundant links between the same servers (this seems unlikely).

Some operators protect their customer EBGP sessions by putting them on a 
different link than where the traffic flows. In those cases it also 
makes sense to use non-routable addresses on that link.

As Iljitsch said, it's also common practice to filter external traffic 
directed at exchange point address space, to protect them from attacks.

One thing to keep in mind is that EBGP sessions usually(*) use a TTL of 
one. MPTCP might well find alternative paths between the speakers, BUT 
the others might have more hops than the proper path.

3. IBGP

You didn't write anything about IBGP, but I'm going to write something 
anyway. :-)

The best practice is that IBGP sessions are bound to logical interfaces 
(loopbacks) instead of physical interfaces (such as ethernets) so that 
the failure of any single link won't affect the IBGP session as long as 
alternative paths exist. IBGP relies on IGP (usually OSPF or IS-IS) to 
spread the routes for the logical interfaces and to find alternative 
paths quickly.

If an IBGP speaker is totally isolated from the rest of the network, the 
IBGP sessions should die as quickly as possible so that the network will 
converge to alternative (external) paths as quickly as possible. This 
too might be something MPTCP could prolong and hinder.

Again, lazy configurations will bind IBGP sessions to either IPADDR_ANY 
or to some physical interface. This is where MPTCP could save those 
sessions from the failure of a single interface, but it should make sure 
to do a better job at it than IGP would.

My two cents,

-- 
	Aleksi Suhonen, Research Assistant
	Department of Communications Engineering
	Tampere University of Technology

(*) 99% of the time

---

From: [email-masked]
Date: 05 Aug 2009 10:07:04 +0100

>multiple addresses - local or remote.

Agreed

> My concern is that there are (and could/should be) applications that
> expect a TCP connection to go down when the local IP address is down.
> TCP is a service for a bytestream between an IP address/port pair, it
is
> not defined as binding to something else that happens to map to an
> IP/address port pair right now.
> 
> The implications if this would be:
> 
>        - assume multipath TCP opens connections using a primary
>        (initial) socket pair
> 
>        - as long as that socketpair is available*, the connection
>        stays up; when not, the connection is torn down
> 
> * = where "available" could mean just that both sides still think that
> the IP address is bound to one of their interfaces.
> 
> I.e., you REALLY do not want the following case:
> 
> 
>        A: adr 11, adr 12
>        B: adr 23, adr 24
> 
> A adr 11 port 52394 connects to B adr 23 port 80
> A and B open a second TCP connection to use in the multipath set,
e.g.,
> from A 12 -> B 24
> A ifconfig's adr 11 down
> C ifconfig's adr 11 up
> Now what?

Well A needs not do anything in particular. It will timeout the subflow it 
had running on adr 11, and then remove the state associated with it. 
B may try to retransmit data to A's 11 and it will end up at C. Here it will
get a reset -> so B will kill the subflow too.

The problem, as I see it, is when applications use TCP to test path 
liveness and measure path properties: e.g. EBGP does the first; I assume 
there are dumb apps that use app level pings on top of TCP to measure rtt, 
for instance.

In these cases, any type of multipath distorts the measurements; adding a 
single additional path makes the measurements meaningless. It seems that 
for EBGP this may not be an issue, because it binds a specific address. 
Can't tell what it will be like for other apps.

Costin

---

From: [email-masked]
Date: 05 Aug 2009 10:11:36 +0100

Aleksi,

Thanks for the clarifications, much appreciated.
Costin