From iljitsch@muada.com Wed May 6 10:48:38 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 115CA28C1AA for ; Wed, 6 May 2009 10:48:38 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.065 X-Spam-Level: X-Spam-Status: No, score=-6.065 tagged_above=-999 required=5 tests=[AWL=0.534, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id x1h6NRmA3zXq for ; Wed, 6 May 2009 10:48:37 -0700 (PDT) Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by core3.amsl.com (Postfix) with ESMTP id 3D2BB3A70EE for ; Wed, 6 May 2009 10:42:05 -0700 (PDT) Received: from claw.it.uc3m.es (claw.it.uc3m.es [163.117.139.66]) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n46Hh1C2099044 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO) for ; Wed, 6 May 2009 19:43:02 +0200 (CEST) (envelope-from iljitsch@muada.com) Message-Id: <832190C7-1AC2-4360-A47B-91AA218200C5@muada.com> From: Iljitsch van Beijnum To: multipathtcp@ietf.org Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Wed, 6 May 2009 19:43:02 +0200 References: <20090506173905.C4A7E3A6FB3@core3.amsl.com> X-Mailer: Apple Mail (2.930.3) Subject: [multipathtcp] draft-van-beijnum-1e-mp-tcp-00 X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 May 2009 17:48:38 -0000 Hi all, Here's my one-ended multipath TCP draft. Please let me know what you think. Iljitsch Begin forwarded message: > Filename: draft-van-beijnum-1e-mp-tcp > Revision: 00 > Title: One-ended multipath TCP > Creation_date: 2009-05-06 > WG ID: Independent Submission > Number_of_pages: 21 > Abstract: > Normal TCP/IP operation is for the routing system to select a best > path that remains stable for some time, and for TCP to adjust to the > properties of this path to optimize throughput. A multipath TCP > would be able to either use capacity on multiple paths, or > dynamically find the best performing path, and therefore reach higher > throughput. By adapting to the properties of several paths through > the usual congestion control algorithms, a multipath TCP shifts its > traffic to less congested paths, leaving more capacity available for > traffic that can't move to another path on more congested paths. And > when a path fails, this can be detected and worked around by TCP much > more quickly than by waiting for the routing system to repair the > failure. > This memo specifies a multipath TCP that is implemented on the > sending host only, without requiring modifications on the receiving > host. From marcelo@it.uc3m.es Wed May 6 11:10:20 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 8EF2C28C220 for ; Wed, 6 May 2009 11:10:20 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.508 X-Spam-Level: X-Spam-Status: No, score=-5.508 tagged_above=-999 required=5 tests=[AWL=1.091, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id l-YAJf9z8C3C for ; Wed, 6 May 2009 11:10:13 -0700 (PDT) Received: from smtp03.uc3m.es (smtp03.uc3m.es [163.117.176.133]) by core3.amsl.com (Postfix) with ESMTP id B347228C22D for ; Wed, 6 May 2009 11:03:55 -0700 (PDT) Received: from marcelo-bagnulos-macbook-pro.local (175.pool85-59-202.dynamic.orange.es [85.59.202.175]) by smtp03.uc3m.es (Postfix) with ESMTP id 7E9EF7326CD for ; Wed, 6 May 2009 20:05:21 +0200 (CEST) Message-ID: <4A01D161.7060108@it.uc3m.es> Date: Wed, 06 May 2009 20:05:21 +0200 From: marcelo bagnulo braun User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: multipathtcp@ietf.org Content-Type: multipart/mixed; boundary="------------010401040306030108090307" Subject: [multipathtcp] Fwd: I-D Action:draft-van-beijnum-1e-mp-tcp-00.txt]] X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 May 2009 18:10:20 -0000 This is a multi-part message in MIME format. --------------010401040306030108090307 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit with a link to draft, which comes handy sometimes... (especially when people actually want to read it :-) -------- Mensaje original -------- Asunto: I-D Action:draft-van-beijnum-1e-mp-tcp-00.txt Fecha: Wed, 6 May 2009 10:45:02 -0700 (PDT) De: Internet-Drafts@ietf.org Responder a: internet-drafts@ietf.org Para: i-d-announce@ietf.org A New Internet-Draft is available from the on-line Internet-Drafts directories. Title : One-ended multipath TCP Author(s) : I. van Beijnum Filename : draft-van-beijnum-1e-mp-tcp-00.txt Pages : 21 Date : 2009-05-06 Normal TCP/IP operation is for the routing system to select a best path that remains stable for some time, and for TCP to adjust to the properties of this path to optimize throughput. A multipath TCP would be able to either use capacity on multiple paths, or dynamically find the best performing path, and therefore reach higher throughput. By adapting to the properties of several paths through the usual congestion control algorithms, a multipath TCP shifts its traffic to less congested paths, leaving more capacity available for traffic that can't move to another path on more congested paths. And when a path fails, this can be detected and worked around by TCP much more quickly than by waiting for the routing system to repair the failure. This memo specifies a multipath TCP that is implemented on the sending host only, without requiring modifications on the receiving host. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-van-beijnum-1e-mp-tcp-00.txt Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ Below is the data which will enable a MIME compliant mail reader implementation to automatically retrieve the ASCII version of the Internet-Draft. --------------010401040306030108090307 Content-Type: Message/External-body; name="draft-van-beijnum-1e-mp-tcp-00.txt" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="draft-van-beijnum-1e-mp-tcp-00.txt" Content-Type: text/plain Content-ID: <2009-05-06103905.I-D@ietf.org> --------------010401040306030108090307 Content-Type: text/plain; name="Parte del mensaje adjunto" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="Parte del mensaje adjunto" _______________________________________________ I-D-Announce mailing list I-D-Announce@ietf.org https://www.ietf.org/mailman/listinfo/i-d-announce Internet-Draft directories: http://www.ietf.org/shadow.html or ftp://ftp.ietf.org/ietf/1shadow-sites.txt --------------010401040306030108090307-- From janardhan.iyengar@fandm.edu Thu May 7 07:29:56 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 00EDE3A703E for ; Thu, 7 May 2009 07:29:56 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WIpAGOrsq7B8 for ; Thu, 7 May 2009 07:29:55 -0700 (PDT) Received: from zimfe2.fandm.edu (zimfe2.fandm.edu [155.68.1.75]) by core3.amsl.com (Postfix) with ESMTP id A7C7F3A6D97 for ; Thu, 7 May 2009 07:29:54 -0700 (PDT) Received: from surutti.fandm.edu (dhcp-155-68-60-80.fandm.edu [155.68.60.80]) by zimfe2.fandm.edu (Postfix) with ESMTP id 75CC41168040; Thu, 7 May 2009 10:31:19 -0400 (EDT) Message-ID: <4A02F0B7.6080202@fandm.edu> Date: Thu, 07 May 2009 10:31:19 -0400 From: Janardhan Iyengar Organization: Franklin & Marshall College User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: marcelo bagnulo braun References: <4A01D161.7060108@it.uc3m.es> In-Reply-To: <4A01D161.7060108@it.uc3m.es> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: nasif ekiz , multipathtcp@ietf.org, "Paul D. Amer" , Randall Stewart , Preethi Natarajan , Ertugrul YILMAZ Subject: Re: [multipathtcp] Fwd: I-D Action:draft-van-beijnum-1e-mp-tcp-00.txt]] X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: janardhan.iyengar@fandm.edu List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 May 2009 14:29:56 -0000 Marcelo, Thanks for the draft, I've seen presentations and discussions about it, and I have wanted to see this written up for a while now. With regard to the statement: "In addition, SCTP as defined today [RFC2960] does not accommodate the concurrent use of multiple paths. Additional paths are purely used for backup purposes." let me encourage you to look up Concurrent Multipath Tranfer (CMT) with SCTP. While this work used SCTP for CMT, many of the issues are not SCTP-specific; they apply to multipath-TCP as well. A list follows: + Our Transactions on Networking paper from 2006 (http://www.fandm.edu/jiyengar/papers/cmt-ton2006.pdf) discusses several of the concerns that you bring up here, only in the context of SCTP. We discuss managing reordering and other concerns and present algorithms for managing those concerns. Our work on CMT applies here quite directly. + You list 4 challenges and questions, and say that you address the first two (scheduling, reordering). We have looked extensively at those two and also the third one (receiver buffer constraints), and our results are documented. For receive buffer (rbuf) considerations and retransmission policies, I would encourage you to see our Computer Communications paper from 2007 (http://www.fandm.edu/jiyengar/papers/rbuf-compcomm2007.pdf). + We also looked into using CMT for fault tolerance, extended CMT mechanisms for fault tolerance, and found that CMT can do much better than simple-old-SCTP in detecting and responding to failures. We have documented our design and our experimental results in our IFIP Networking paper from 2008 (http://www.fandm.edu/jiyengar/papers/cmtpf-networking2008.pdf). + For other design considerations that are not covered in these papers, a list of unresolved issues, and for other related work, please see my dissertation on "End-to-End Concurrent Multipath Transfer Using Transport Layer Multihoming" from 2006: http://www.fandm.edu/jiyengar/papers/iyengar-dissertation.pdf I like to think that we did quite a bit of work in this area already, and while there are unresolved issues still around, I also like to think that we got some important things down :-) It would be very disheartening to see work that is pretty much exactly the same not using our previous work. I am happy to discuss this further! (I will go read your draft more carefully later today ...) regards, - jana marcelo bagnulo braun wrote: > > with a link to draft, which comes handy sometimes... (especially when > people actually want to read it :-) > > > > -------- Mensaje original -------- > Asunto: I-D Action:draft-van-beijnum-1e-mp-tcp-00.txt > Fecha: Wed, 6 May 2009 10:45:02 -0700 (PDT) > De: Internet-Drafts@ietf.org > Responder a: internet-drafts@ietf.org > Para: i-d-announce@ietf.org > > > > A New Internet-Draft is available from the on-line Internet-Drafts > directories. > > Title : One-ended multipath TCP > Author(s) : I. van Beijnum > Filename : draft-van-beijnum-1e-mp-tcp-00.txt > Pages : 21 > Date : 2009-05-06 > > Normal TCP/IP operation is for the routing system to select a best > path that remains stable for some time, and for TCP to adjust to the > properties of this path to optimize throughput. A multipath TCP > would be able to either use capacity on multiple paths, or > dynamically find the best performing path, and therefore reach higher > throughput. By adapting to the properties of several paths through > the usual congestion control algorithms, a multipath TCP shifts its > traffic to less congested paths, leaving more capacity available for > traffic that can't move to another path on more congested paths. And > when a path fails, this can be detected and worked around by TCP much > more quickly than by waiting for the routing system to repair the > failure. > > This memo specifies a multipath TCP that is implemented on the > sending host only, without requiring modifications on the receiving > host. > > A URL for this Internet-Draft is: > http://www.ietf.org/internet-drafts/draft-van-beijnum-1e-mp-tcp-00.txt > > Internet-Drafts are also available by anonymous FTP at: > ftp://ftp.ietf.org/internet-drafts/ > > Below is the data which will enable a MIME compliant mail reader > implementation to automatically retrieve the ASCII version of the > Internet-Draft. > > > > > > ------------------------------------------------------------------------ > > _______________________________________________ > multipathtcp mailing list > multipathtcp@ietf.org > https://www.ietf.org/mailman/listinfo/multipathtcp -- Janardhan Iyengar Assistant Professor, Computer Science Franklin & Marshall College http://www.fandm.edu/jiyengar From marcelo@it.uc3m.es Thu May 7 07:52:31 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id CB1453A69DB for ; Thu, 7 May 2009 07:52:31 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.351 X-Spam-Level: X-Spam-Status: No, score=-6.351 tagged_above=-999 required=5 tests=[AWL=0.248, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jz7Ba4ZHl9qa for ; Thu, 7 May 2009 07:52:30 -0700 (PDT) Received: from smtp02.uc3m.es (smtp02.uc3m.es [163.117.176.132]) by core3.amsl.com (Postfix) with ESMTP id AFA2D28C2C2 for ; Thu, 7 May 2009 07:52:10 -0700 (PDT) Received: from marcelo-bagnulos-macbook-pro.local (216.pool85-53-129.dynamic.orange.es [85.53.129.216]) by smtp02.uc3m.es (Postfix) with ESMTP id 51C456C0F68; Thu, 7 May 2009 16:53:34 +0200 (CEST) Message-ID: <4A02F5ED.7090902@it.uc3m.es> Date: Thu, 07 May 2009 16:53:33 +0200 From: marcelo bagnulo braun User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: janardhan.iyengar@fandm.edu References: <4A01D161.7060108@it.uc3m.es> <4A02F0B7.6080202@fandm.edu> In-Reply-To: <4A02F0B7.6080202@fandm.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-TM-AS-Product-Ver: IMSS-7.0.0.3116-5.6.0.1016-16628.000 Cc: nasif ekiz , multipathtcp@ietf.org, "Paul D. Amer" , Randall Stewart , Preethi Natarajan , Ertugrul YILMAZ Subject: Re: [multipathtcp] Fwd: I-D Action:draft-van-beijnum-1e-mp-tcp-00.txt]] X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 May 2009 14:52:31 -0000 Hi Jana, the current idea is to submit a BOF proposal for Stockholm to propose the formation of a WG to define the multipath extensions for TCP. It would be great to have people that have been doing work on this involved. I will circulate the current version of the BOF proposal shortly, so please coment if you think that the relevant aspects are being included. At this stage in this effort, we are looking forward to integrate interested parties, so if you have any suggestions or related work that you would like to be included as part of this effort, please say so, so we can work it out. Regards, marcelo Janardhan Iyengar escribió: > Marcelo, > > Thanks for the draft, I've seen presentations and discussions about > it, and I have wanted to see this written up for a while now. > > With regard to the statement: "In addition, SCTP as > defined today [RFC2960] does not accommodate the concurrent use of > multiple paths. Additional paths are purely used for backup > purposes." > let me encourage you to look up Concurrent Multipath Tranfer (CMT) > with SCTP. While this work used SCTP for CMT, many of the issues are > not SCTP-specific; they apply to multipath-TCP as well. A list follows: > > + Our Transactions on Networking paper from 2006 > (http://www.fandm.edu/jiyengar/papers/cmt-ton2006.pdf) discusses > several of the concerns that you bring up here, only in the context of > SCTP. We discuss managing reordering and other concerns and present > algorithms for managing those concerns. Our work on CMT applies here > quite directly. > > + You list 4 challenges and questions, and say that you address the > first two (scheduling, reordering). We have looked extensively at > those two and also the third one (receiver buffer constraints), and > our results are documented. For receive buffer (rbuf) considerations > and retransmission policies, I would encourage you to see our Computer > Communications paper from 2007 > (http://www.fandm.edu/jiyengar/papers/rbuf-compcomm2007.pdf). > > + We also looked into using CMT for fault tolerance, extended CMT > mechanisms for fault tolerance, and found that CMT can do much better > than simple-old-SCTP in detecting and responding to failures. We have > documented our design and our experimental results in our IFIP > Networking paper from 2008 > (http://www.fandm.edu/jiyengar/papers/cmtpf-networking2008.pdf). > > + For other design considerations that are not covered in these > papers, a list of unresolved issues, and for other related work, > please see my dissertation on "End-to-End Concurrent Multipath > Transfer Using Transport Layer Multihoming" from 2006: > http://www.fandm.edu/jiyengar/papers/iyengar-dissertation.pdf > > > I like to think that we did quite a bit of work in this area already, > and while there are unresolved issues still around, I also like to > think that we got some important things down :-) It would be very > disheartening to see work that is pretty much exactly the same not > using our previous work. > > I am happy to discuss this further! (I will go read your draft more > carefully later today ...) > > regards, > - jana > > > marcelo bagnulo braun wrote: >> >> with a link to draft, which comes handy sometimes... (especially when >> people actually want to read it :-) >> >> >> >> -------- Mensaje original -------- >> Asunto: I-D Action:draft-van-beijnum-1e-mp-tcp-00.txt >> Fecha: Wed, 6 May 2009 10:45:02 -0700 (PDT) >> De: Internet-Drafts@ietf.org >> Responder a: internet-drafts@ietf.org >> Para: i-d-announce@ietf.org >> >> >> >> A New Internet-Draft is available from the on-line Internet-Drafts >> directories. >> >> Title : One-ended multipath TCP >> Author(s) : I. van Beijnum >> Filename : draft-van-beijnum-1e-mp-tcp-00.txt >> Pages : 21 >> Date : 2009-05-06 >> >> Normal TCP/IP operation is for the routing system to select a best >> path that remains stable for some time, and for TCP to adjust to the >> properties of this path to optimize throughput. A multipath TCP >> would be able to either use capacity on multiple paths, or >> dynamically find the best performing path, and therefore reach higher >> throughput. By adapting to the properties of several paths through >> the usual congestion control algorithms, a multipath TCP shifts its >> traffic to less congested paths, leaving more capacity available for >> traffic that can't move to another path on more congested paths. And >> when a path fails, this can be detected and worked around by TCP much >> more quickly than by waiting for the routing system to repair the >> failure. >> >> This memo specifies a multipath TCP that is implemented on the >> sending host only, without requiring modifications on the receiving >> host. >> >> A URL for this Internet-Draft is: >> http://www.ietf.org/internet-drafts/draft-van-beijnum-1e-mp-tcp-00.txt >> >> Internet-Drafts are also available by anonymous FTP at: >> ftp://ftp.ietf.org/internet-drafts/ >> >> Below is the data which will enable a MIME compliant mail reader >> implementation to automatically retrieve the ASCII version of the >> Internet-Draft. >> >> >> >> >> >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> multipathtcp mailing list >> multipathtcp@ietf.org >> https://www.ietf.org/mailman/listinfo/multipathtcp > From janardhan.iyengar@fandm.edu Thu May 7 07:58:04 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id EC84F3A68D0 for ; Thu, 7 May 2009 07:58:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MyFAo7YxRzCb for ; Thu, 7 May 2009 07:58:04 -0700 (PDT) Received: from zimfe2.fandm.edu (zimfe2.fandm.edu [155.68.1.75]) by core3.amsl.com (Postfix) with ESMTP id DA6123A683A for ; Thu, 7 May 2009 07:58:03 -0700 (PDT) Received: from surutti.fandm.edu (dhcp-155-68-60-80.fandm.edu [155.68.60.80]) by zimfe2.fandm.edu (Postfix) with ESMTP id 0D91E116804A; Thu, 7 May 2009 10:59:31 -0400 (EDT) Message-ID: <4A02F752.70603@fandm.edu> Date: Thu, 07 May 2009 10:59:30 -0400 From: Janardhan Iyengar Organization: Franklin & Marshall College User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: marcelo bagnulo braun References: <4A01D161.7060108@it.uc3m.es> <4A02F0B7.6080202@fandm.edu> <4A02F5ED.7090902@it.uc3m.es> In-Reply-To: <4A02F5ED.7090902@it.uc3m.es> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: nasif ekiz , multipathtcp@ietf.org, "Paul D. Amer" , Randall Stewart , Preethi Natarajan , Ertugrul YILMAZ Subject: Re: [multipathtcp] Fwd: I-D Action:draft-van-beijnum-1e-mp-tcp-00.txt]] X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: janardhan.iyengar@fandm.edu List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 May 2009 14:58:05 -0000 Hi Marcelo, A BoF sounds great! I will be glad to look at the BoF proposal when it comes out. > [...] so if you have any suggestions or related work that > you would like to be included as part of this effort, please say so, [...] A good bunch of our previous work on CMT is as related as anything can ever be. There. I said so. :-) I look forward to seeing the BoF proposal, - jana -- Janardhan Iyengar Assistant Professor, Computer Science Franklin & Marshall College http://www.fandm.edu/jiyengar From marcelo@it.uc3m.es Thu May 7 08:19:47 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 76A573A6C73 for ; Thu, 7 May 2009 08:19:47 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.353 X-Spam-Level: X-Spam-Status: No, score=-6.353 tagged_above=-999 required=5 tests=[AWL=0.246, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5pBdfvumhvRA for ; Thu, 7 May 2009 08:19:46 -0700 (PDT) Received: from smtp02.uc3m.es (smtp02.uc3m.es [163.117.176.132]) by core3.amsl.com (Postfix) with ESMTP id 3E1683A6C1F for ; Thu, 7 May 2009 08:19:45 -0700 (PDT) Received: from marcelo-bagnulos-macbook-pro.local (216.pool85-53-129.dynamic.orange.es [85.53.129.216]) by smtp02.uc3m.es (Postfix) with ESMTP id 60F1F6BA037 for ; Thu, 7 May 2009 17:21:11 +0200 (CEST) Message-ID: <4A02FC62.8080503@it.uc3m.es> Date: Thu, 07 May 2009 17:21:06 +0200 From: marcelo bagnulo braun User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: multipathtcp@ietf.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-TM-AS-Product-Ver: IMSS-7.0.0.3116-5.6.0.1016-16628.000 Subject: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 May 2009 15:19:47 -0000 Hi, We are planning to submit a BOF request for the next IETF. This is a first draft of the bof proposal. Comments are welcome, of course. Multipath TCP (MPTCP) BOF Recent research has suggested that by making better use of the multiple paths between endpoints we can better utilise the resources within the network and thus improve resource pooling. In particular, recent results include models of how multipath-capable end-systems might respond to congestion signals, and what the potential benefits might be (see [1] for a more details and references). If end systems could spread their load across multiple paths in the right way, with the right reaction to the right congestion signals from the network, then traffic would quickly move away from congested or failed links in favor of available uncongested links. It is fairly intuitive to see that when multipath transport is used in conjunction with congestion-dependent load-distribution, it is possible to push more traffic through the network and to achieve enhanced fault tolerance. Traffic will flow throught any available path that has spare capacity, filling unused resources, while moving away from failures and congested resources. The result is a more flexible network, capable of functioning well, even when the actual traffic matrix differs substantially from the traffic matrix envisaged when the network was designed for. Currently used transport protocols use only one path at a time for sending data. In particular TCP and DCCP use only one path and they respond to congestion reducing their sending rate. SCTP on the other hand, is aware of multiple paths, but it uses only one at the time, reserving the alternative paths as backups in case of failure. The proposed work is to define the mechanisms to enable transport protocols to use multiple paths simultaneously and to distribute the load among the subflows of each path based on congestion. In particular, the proposed work is to extend TCP to support simultaneous data exchange through multiple paths. Two complementary approaches are proposed: On one hand, to define a two-ended multipath TCP that uses multiple addresses a-la SCTP to obtain multiple paths and that requires support from both endpoints to use them. On the other hand, single ended multipath TCP, where only one of the endpoints involved in the communication is multipath aware and relies on multipath routing capabilities provided by the IP routing to deliver data towards a legacy TCP endpoint. These two approaches provide a smooth deployment model, starting with the one-ended multipath TCP, that only requires one end to deploy it and it results in immediate benefits, and then when multipath TCP support becomes pervasive, using the two-ended multipath TCP to obtain all the potential benefits. The proposed work items are: - A set of architectural guidelines for congestion dependent multipath transport protocols. Since different transport protocols can potentially benefit from this approach, this document describes the motivations and the general approach that should be followed to enable congestion dependent multipath transport. - Extensions to current TCP to support the one-ended mutipath TCP. - Extensions to current TCP to support the two-ended mutipath TCP. - A threat analysis for multipath TCP - An extended API describing how applications can access to multipath information and use it. The initial goal is that the documents produced by the WG will be either experimental or informational. However, if when the documents are ready the technology seems mature enough, the WG will consider if it is appropriate to ask the IESG to advance the document as a Proposed Standard. With respect to the work on the support for multipath in TCP, the only acceptable modifications to the TCP packet format are in the form of new TCP options. [1] D. Wischik, M. Handley, M. Bagnulo, The resource pooling principle, ACM SIGCOMM Computer Communication Review archive, October 2008 From swb@employees.org Thu May 7 10:30:39 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id C6C403A6C97 for ; Thu, 7 May 2009 10:30:39 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.528 X-Spam-Level: X-Spam-Status: No, score=-6.528 tagged_above=-999 required=5 tests=[AWL=0.071, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Lw3jJ++LXhPF for ; Thu, 7 May 2009 10:30:38 -0700 (PDT) Received: from sj-iport-6.cisco.com (sj-iport-6.cisco.com [171.71.176.117]) by core3.amsl.com (Postfix) with ESMTP id BD7253A6C89 for ; Thu, 7 May 2009 10:30:38 -0700 (PDT) X-IronPort-AV: E=Sophos;i="4.40,311,1238976000"; d="scan'208";a="300351745" Received: from sj-dkim-2.cisco.com ([171.71.179.186]) by sj-iport-6.cisco.com with ESMTP; 07 May 2009 17:32:07 +0000 Received: from sj-core-2.cisco.com (sj-core-2.cisco.com [171.71.177.254]) by sj-dkim-2.cisco.com (8.12.11/8.12.11) with ESMTP id n47HW5Rv011586; Thu, 7 May 2009 10:32:05 -0700 Received: from xbh-rtp-211.amer.cisco.com (xbh-rtp-211.cisco.com [64.102.31.102]) by sj-core-2.cisco.com (8.13.8/8.13.8) with ESMTP id n47HVxCl026553; Thu, 7 May 2009 17:32:04 GMT Received: from xfe-rtp-201.amer.cisco.com ([64.102.31.38]) by xbh-rtp-211.amer.cisco.com with Microsoft SMTPSVC(6.0.3790.1830); Thu, 7 May 2009 13:32:00 -0400 Received: from sbrim-mbp.local ([10.86.255.104]) by xfe-rtp-201.amer.cisco.com with Microsoft SMTPSVC(6.0.3790.1830); Thu, 7 May 2009 13:31:59 -0400 Message-ID: <4A031B0F.4030606@employees.org> Date: Thu, 07 May 2009 13:31:59 -0400 From: Scott Brim User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: marcelo bagnulo braun References: <4A02FC62.8080503@it.uc3m.es> In-Reply-To: <4A02FC62.8080503@it.uc3m.es> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 07 May 2009 17:31:59.0720 (UTC) FILETIME=[B9FCAE80:01C9CF39] Authentication-Results: sj-dkim-2; header.From=swb@employees.org; dkim=neutral Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 May 2009 17:30:39 -0000 Hi Marcelo. (1) Up in the introductory material: As you've heard me argue, in addition to the obvious benefits of multipath TCP, I believe it also indirectly provides the best approach to path failure handling (better than probing, in particular). (2) Under work items, I don't see the work on the routers, or anyway intermediate agents, to provide information to a single-ended multipath TCP. From marcelo@it.uc3m.es Thu May 7 10:43:48 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 7091B3A6E3E for ; Thu, 7 May 2009 10:43:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.354 X-Spam-Level: X-Spam-Status: No, score=-6.354 tagged_above=-999 required=5 tests=[AWL=0.245, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Cso4fErJ2+Gx for ; Thu, 7 May 2009 10:43:47 -0700 (PDT) Received: from smtp01.uc3m.es (smtp01.uc3m.es [163.117.176.131]) by core3.amsl.com (Postfix) with ESMTP id 7F4CE3A6C73 for ; Thu, 7 May 2009 10:43:47 -0700 (PDT) Received: from marcelo-bagnulos-macbook-pro.local (214.pool85-53-145.dynamic.orange.es [85.53.145.214]) by smtp01.uc3m.es (Postfix) with ESMTP id 025ECB4D796; Thu, 7 May 2009 19:45:04 +0200 (CEST) Message-ID: <4A031E1E.5050900@it.uc3m.es> Date: Thu, 07 May 2009 19:45:02 +0200 From: marcelo bagnulo braun User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Scott Brim References: <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> In-Reply-To: <4A031B0F.4030606@employees.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-TM-AS-Product-Ver: IMSS-7.0.0.3116-5.6.0.1016-16628.000 Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 May 2009 17:43:48 -0000 Scott Brim escribió: > Hi Marcelo. > > (1) Up in the introductory material: As you've heard me argue, in > addition to the obvious benefits of multipath TCP, I believe it also > indirectly provides the best approach to path failure handling (better > than probing, in particular). > agree, i will include some text on that, thanks > (2) Under work items, I don't see the work on the routers, or anyway > intermediate agents, to provide information to a single-ended multipath TCP. > > > > mmm, i am not sure i understnad that you mean. the single ended approach requires the mptpc capable end to be able to select different paths. this can be done either b giving some hint to multipath capable routers or there are some other means, such as selecting different next hops, or tunneling to different egress routers. Is that what you mean? Cause, you seem to imply that the routers need to give information to the mptcp capable endpoint, which i am not sure i fully understand Regards, marcelo From Pasi.Sarolahti@nokia.com Thu May 7 13:56:34 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 9203F3A68F1 for ; Thu, 7 May 2009 13:56:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aZWLwrcVAPAo for ; Thu, 7 May 2009 13:56:33 -0700 (PDT) Received: from mgw-mx09.nokia.com (smtp.nokia.com [192.100.105.134]) by core3.amsl.com (Postfix) with ESMTP id 4600928C287 for ; Thu, 7 May 2009 13:56:25 -0700 (PDT) Received: from esebh106.NOE.Nokia.com (esebh106.ntc.nokia.com [172.21.138.213]) by mgw-mx09.nokia.com (Switch-3.2.6/Switch-3.2.6) with ESMTP id n47KvFQG023335; Thu, 7 May 2009 15:57:53 -0500 Received: from esebh102.NOE.Nokia.com ([172.21.138.183]) by esebh106.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.3959); Thu, 7 May 2009 23:57:42 +0300 Received: from vaebh101.NOE.Nokia.com ([10.160.244.22]) by esebh102.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.3959); Thu, 7 May 2009 23:57:41 +0300 Received: from [192.168.0.102] ([10.162.252.114]) by vaebh101.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.3959); Thu, 7 May 2009 23:57:28 +0300 In-Reply-To: <4A02FC62.8080503@it.uc3m.es> References: <4A02FC62.8080503@it.uc3m.es> Mime-Version: 1.0 (Apple Message framework v753.1) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <869CB6AE-1804-4691-8DE1-B3FF6D314C07@nokia.com> Content-Transfer-Encoding: 7bit From: Pasi Sarolahti Date: Thu, 7 May 2009 23:57:23 +0300 To: ext marcelo bagnulo braun X-Mailer: Apple Mail (2.753.1) X-OriginalArrivalTime: 07 May 2009 20:57:28.0173 (UTC) FILETIME=[6E5151D0:01C9CF56] X-Nokia-AV: Clean Cc: "multipathtcp@ietf.org" Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 May 2009 20:56:34 -0000 Hi, A wiki page has been established at http://trac.tools.ietf.org/area/ tsv/trac/wiki/MultipathTcp, including the BOF description below, and links to some additional material. Feel free to add more pointers to related material/discussion there (such as the links from Jana, that I didn't yet add). - Pasi On May 7, 2009, at 18:21, ext marcelo bagnulo braun wrote: > Hi, > > We are planning to submit a BOF request for the next IETF. > This is a first draft of the bof proposal. > > Comments are welcome, of course. > > > Multipath TCP (MPTCP) BOF > > Recent research has suggested that by making better use of the > multiple paths between endpoints we can better utilise the > resources within the network and thus improve resource pooling. > In particular, recent results include models of how > multipath-capable end-systems might respond to congestion signals, > and what the potential benefits might be (see [1] for a more details > and references). > > If end systems could spread their load across multiple paths in the > right way, with the right reaction to the right congestion signals > from the network, then traffic would quickly move away from > congested or failed links in favor of available uncongested links. > > It is fairly intuitive to see that when multipath transport is used > in conjunction with congestion-dependent load-distribution, it is > possible to push more traffic through the network and to achieve > enhanced fault tolerance. Traffic will flow throught any available > path that has spare capacity, filling unused resources, while moving > away from failures and congested resources. > The result is a more flexible network, capable of functioning well, > even when the actual traffic matrix differs substantially from the > traffic matrix envisaged when the network was designed for. > > Currently used transport protocols use only one path at a time > for sending data. In particular TCP and DCCP use only one path and > they respond to congestion reducing their sending rate. SCTP > on the other hand, is aware of multiple paths, but it uses only > one at the time, reserving the alternative paths as backups > in case of failure. > > The proposed work is to define the mechanisms to enable > transport protocols to use multiple paths simultaneously > and to distribute the load among the subflows of each path based > on congestion. > > In particular, the proposed work is to extend TCP to support > simultaneous data exchange through multiple paths. > Two complementary approaches are proposed: On one hand, > to define a two-ended multipath TCP that uses multiple addresses a- > la SCTP > to obtain multiple paths and that requires support from both > endpoints to use them. > On the other hand, single ended multipath TCP, where only > one of the endpoints involved in the communication > is multipath aware and relies on multipath routing > capabilities provided by the IP routing to deliver data towards > a legacy TCP endpoint. > These two approaches provide a smooth deployment model, starting > with the one-ended multipath TCP, that only requires one end to > deploy it > and it results in immediate benefits, and then when multipath > TCP support becomes pervasive, using the two-ended multipath TCP > to obtain all the potential benefits. > > The proposed work items are: > - A set of architectural guidelines for congestion dependent > multipath transport protocols. Since different transport protocols > can potentially benefit from this approach, this document describes > the motivations and the general approach that should be followed > to enable congestion dependent multipath transport. > - Extensions to current TCP to support the one-ended mutipath TCP. > - Extensions to current TCP to support the two-ended mutipath TCP. > - A threat analysis for multipath TCP > - An extended API describing how applications can > access to multipath information and use it. > > The initial goal is that the documents produced by the WG will be > either experimental or informational. However, if when the documents > are ready the technology seems mature enough, the WG will consider > if it is appropriate to ask the IESG to advance the document as a > Proposed Standard. > > With respect to the work on the support for multipath in TCP, the > only acceptable modifications to the TCP packet format are in the form > of new TCP options. > > [1] D. Wischik, M. Handley, M. Bagnulo, The resource pooling > principle, > ACM SIGCOMM Computer Communication Review archive, October 2008 > > > _______________________________________________ > multipathtcp mailing list > multipathtcp@ietf.org > https://www.ietf.org/mailman/listinfo/multipathtcp From Scott.Brim@gmail.com Thu May 7 14:14:54 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id EEFC93A6BFB for ; Thu, 7 May 2009 14:14:54 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XZEUqSEzt3K3 for ; Thu, 7 May 2009 14:14:54 -0700 (PDT) Received: from sj-iport-6.cisco.com (sj-iport-6.cisco.com [171.71.176.117]) by core3.amsl.com (Postfix) with ESMTP id 19E613A697A for ; Thu, 7 May 2009 14:14:54 -0700 (PDT) X-IronPort-AV: E=Sophos;i="4.40,312,1238976000"; d="scan'208";a="300487143" Received: from rtp-dkim-2.cisco.com ([64.102.121.159]) by sj-iport-6.cisco.com with ESMTP; 07 May 2009 21:16:22 +0000 Received: from rtp-core-1.cisco.com (rtp-core-1.cisco.com [64.102.124.12]) by rtp-dkim-2.cisco.com (8.12.11/8.12.11) with ESMTP id n47LGLGm002162; Thu, 7 May 2009 17:16:21 -0400 Received: from cisco.com (ssh-rtp-2.cisco.com [64.102.8.172]) by rtp-core-1.cisco.com (8.13.8/8.13.8) with ESMTP id n47LGL5r010880; Thu, 7 May 2009 21:16:21 GMT Date: Thu, 7 May 2009 17:16:19 -0400 From: Scott Brim To: marcelo bagnulo braun Message-ID: <20090507211619.GA35382@cisco.com> Mail-Followup-To: Scott Brim , marcelo bagnulo braun , multipathtcp@ietf.org References: <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <4A031E1E.5050900@it.uc3m.es> User-Agent: Mutt/1.5.19 (2009-01-05) Authentication-Results: rtp-dkim-2; header.From=Scott.Brim@gmail.com; dkim=neutral Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 May 2009 21:14:55 -0000 Excerpts from marcelo bagnulo braun on Thu, May 07, 2009 07:45:02PM +0200: > Scott Brim escribió: >> Hi Marcelo. >> >> (1) Up in the introductory material: As you've heard me argue, in >> addition to the obvious benefits of multipath TCP, I believe it >> also indirectly provides the best approach to path failure handling >> (better than probing, in particular). >> > agree, i will include some text on that, thanks > >> (2) Under work items, I don't see the work on the routers, or >> anyway intermediate agents, to provide information to a >> single-ended multipath TCP. >> > mmm, i am not sure i understnad that you mean. the single ended > approach requires the mptpc capable end to be able to select > different paths. this can be done either b giving some hint to > multipath capable routers or there are some other means, such as > selecting different next hops, or tunneling to different egress > routers. Is that what you mean? > > Cause, you seem to imply that the routers need to give information > to the mptcp capable endpoint, which i am not sure i fully > understand In all cases except the simplest, the endpoint would need to receive hints from some agent -- an egress router or some other intermediary that has knowledge of network state. It's not enough for the endpoint to give hints to the router. Even if the endpoint tunnels to an egress router, it needs an indication that the egress router is up and can deliver the packets to the destination address. So it's not enough to work on the endpoint -- don't you need that agent? Thanks ... Scott From iljitsch@muada.com Thu May 7 14:44:08 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 777333A6AC8 for ; Thu, 7 May 2009 14:44:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.919 X-Spam-Level: X-Spam-Status: No, score=-5.919 tagged_above=-999 required=5 tests=[AWL=0.680, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FXrjDLFaGtwE for ; Thu, 7 May 2009 14:44:07 -0700 (PDT) Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by core3.amsl.com (Postfix) with ESMTP id 6C5F23A696F for ; Thu, 7 May 2009 14:44:07 -0700 (PDT) Received: from [192.168.0.193] (static-167-138-7-89.ipcom.comunitel.net [89.7.138.167] (may be forged)) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n47LimDs012546 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Thu, 7 May 2009 23:44:49 +0200 (CEST) (envelope-from iljitsch@muada.com) Message-Id: <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> From: Iljitsch van Beijnum To: Scott Brim In-Reply-To: <20090507211619.GA35382@cisco.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Thu, 7 May 2009 23:44:49 +0200 References: <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> X-Mailer: Apple Mail (2.930.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 May 2009 21:44:08 -0000 On 7 mei 2009, at 23:16, Scott Brim wrote: > In all cases except the simplest, the endpoint would need to receive > hints from some agent -- an egress router or some other intermediary > that has knowledge of network state. It's not enough for the endpoint > to give hints to the router. Even if the endpoint tunnels to an > egress router, it needs an indication that the egress router is up and > can deliver the packets to the destination address. So it's not > enough to work on the endpoint -- don't you need that agent? No. The proof of the pudding is in the eating. In principle, the host sends packets over a certain path, if ACKs come back the path works, if they don't it doesn't. In practice things are going to be a bit more complex if we want to have multipath routers that work together with multipath TCP, because the multipath TCP needs to write a value in the packet somewhere that the router is going to look at. If everyone agrees on where that field is and what values are legal, it ends there, but if there are different options in different networks the host would need to know which options to use in a particular network. But the situation where a good deal of routing information is exposed to hosts wouldn't be good, in my opinion. The great thing about multipath TCP is that it doesn't need that info, because it can determine which paths are working and how good they are for itself by virtue of simply sending data packets. From marcelo@it.uc3m.es Thu May 7 15:39:08 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 86BE43A6CB6 for ; Thu, 7 May 2009 15:39:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.355 X-Spam-Level: X-Spam-Status: No, score=-6.355 tagged_above=-999 required=5 tests=[AWL=0.244, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KmMObfbIhy9Z for ; Thu, 7 May 2009 15:39:07 -0700 (PDT) Received: from smtp02.uc3m.es (smtp02.uc3m.es [163.117.176.132]) by core3.amsl.com (Postfix) with ESMTP id 7C18F3A6ABE for ; Thu, 7 May 2009 15:39:07 -0700 (PDT) Received: from marcelo-bagnulos-macbook-pro.local (214.pool85-53-145.dynamic.orange.es [85.53.145.214]) by smtp02.uc3m.es (Postfix) with ESMTP id 7D349656123; Fri, 8 May 2009 00:40:32 +0200 (CEST) Message-ID: <4A03635F.4010107@it.uc3m.es> Date: Fri, 08 May 2009 00:40:31 +0200 From: marcelo bagnulo braun User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Iljitsch van Beijnum References: <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> In-Reply-To: <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-TM-AS-Product-Ver: IMSS-7.0.0.3116-5.6.0.1016-16628.001 Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 May 2009 22:39:08 -0000 Iljitsch van Beijnum escribió: > On 7 mei 2009, at 23:16, Scott Brim wrote: > >> In all cases except the simplest, the endpoint would need to receive >> hints from some agent -- an egress router or some other intermediary >> that has knowledge of network state. It's not enough for the endpoint >> to give hints to the router. Even if the endpoint tunnels to an >> egress router, it needs an indication that the egress router is up and >> can deliver the packets to the destination address. So it's not >> enough to work on the endpoint -- don't you need that agent? > > No. The proof of the pudding is in the eating. In principle, the host > sends packets over a certain path, if ACKs come back the path works, > if they don't it doesn't. > > In practice things are going to be a bit more complex if we want to > have multipath routers that work together with multipath TCP, because > the multipath TCP needs to write a value in the packet somewhere that > the router is going to look at. If everyone agrees on where that field > is and what values are legal, it ends there, but if there are > different options in different networks the host would need to know > which options to use in a particular network. > right. I understand that one of the points Scott is raising is whehter the charter should cover the mptcp-router interaction. I am not sure what the answer would be. Regards, marcelo > But the situation where a good deal of routing information is exposed > to hosts wouldn't be good, in my opinion. The great thing about > multipath TCP is that it doesn't need that info, because it can > determine which paths are working and how good they are for itself by > virtue of simply sending data packets. > _______________________________________________ > multipathtcp mailing list > multipathtcp@ietf.org > https://www.ietf.org/mailman/listinfo/multipathtcp > From Scott.Brim@gmail.com Thu May 7 16:03:35 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 38AF13A68A3 for ; Thu, 7 May 2009 16:03:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xLAmNDGZpeyG for ; Thu, 7 May 2009 16:03:34 -0700 (PDT) Received: from rtp-iport-1.cisco.com (rtp-iport-1.cisco.com [64.102.122.148]) by core3.amsl.com (Postfix) with ESMTP id 2B1853A67B3 for ; Thu, 7 May 2009 16:03:34 -0700 (PDT) X-IronPort-AV: E=Sophos;i="4.40,313,1238976000"; d="scan'208";a="43728297" Received: from rtp-dkim-1.cisco.com ([64.102.121.158]) by rtp-iport-1.cisco.com with ESMTP; 07 May 2009 23:05:02 +0000 Received: from rtp-core-2.cisco.com (rtp-core-2.cisco.com [64.102.124.13]) by rtp-dkim-1.cisco.com (8.12.11/8.12.11) with ESMTP id n47N51SQ020490 for ; Thu, 7 May 2009 19:05:01 -0400 Received: from cisco.com (ssh-rtp-2.cisco.com [64.102.8.172]) by rtp-core-2.cisco.com (8.13.8/8.13.8) with ESMTP id n47N51WM025065 for ; Thu, 7 May 2009 23:05:01 GMT Date: Thu, 7 May 2009 19:04:52 -0400 From: Scott Brim To: multipathtcp@ietf.org Message-ID: <20090507230452.GC35382@cisco.com> Mail-Followup-To: Scott Brim , multipathtcp@ietf.org References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> User-Agent: Mutt/1.5.19 (2009-01-05) Authentication-Results: rtp-dkim-1; header.From=Scott.Brim@gmail.com; dkim=neutral Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 May 2009 23:03:35 -0000 Excerpts from Iljitsch van Beijnum on Thu, May 07, 2009 11:44:49PM +0200: > In practice things are going to be a bit more complex if we want to have > multipath routers that work together with multipath TCP, because the > multipath TCP needs to write a value in the packet somewhere that the > router is going to look at. If everyone agrees on where that field is and > what values are legal, it ends there, but if there are different options > in different networks the host would need to know which options to use in > a particular network. Good point -- the endpoint doesn't need to know which paths actually work, just which ones might work. However, how does the endpoint know even that? How does it know how to cause packets to take different egresses? I may be going too far too fast, but if the interesting cases involve some agent giving the endpoint that information, requirements for that agent should be documented, and thus in the charter. No?. Excerpts from marcelo bagnulo braun on Fri, May 08, 2009 12:40:31AM +0200: > I understand that one of the points Scott is raising is whehter the > charter should cover the mptcp-router interaction. > I am not sure what the answer would be. It makes sense to me to have those interactions in the charter. I'm asking if there is a component missing that should be included, to support them. Thanks much ... Scott From touch@ISI.EDU Thu May 7 16:08:31 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 279F13A6A56 for ; Thu, 7 May 2009 16:08:31 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yDSHMZfsFH1O for ; Thu, 7 May 2009 16:08:30 -0700 (PDT) Received: from vapor.isi.edu (vapor.isi.edu [128.9.64.64]) by core3.amsl.com (Postfix) with ESMTP id 1832A3A67FD for ; Thu, 7 May 2009 16:08:30 -0700 (PDT) Received: from [75.217.216.156] (156.sub-75-217-216.myvzw.com [75.217.216.156]) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id n47N9TJa028248; Thu, 7 May 2009 16:09:31 -0700 (PDT) Message-ID: <4A036A29.7020005@isi.edu> Date: Thu, 07 May 2009 16:09:29 -0700 From: Joe Touch User-Agent: Thunderbird 2.0.0.21 (Windows/20090302) MIME-Version: 1.0 To: Scott Brim , multipathtcp@ietf.org References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <20090507230452.GC35382@cisco.com> In-Reply-To: <20090507230452.GC35382@cisco.com> X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 May 2009 23:08:31 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Scott Brim wrote: > Excerpts from Iljitsch van Beijnum on Thu, May 07, 2009 11:44:49PM +0200: >> In practice things are going to be a bit more complex if we want to have >> multipath routers that work together with multipath TCP, because the >> multipath TCP needs to write a value in the packet somewhere that the >> router is going to look at. If everyone agrees on where that field is and >> what values are legal, it ends there, but if there are different options >> in different networks the host would need to know which options to use in >> a particular network. > > Good point -- the endpoint doesn't need to know which paths actually > work, just which ones might work. However, how does the endpoint know > even that? How does it know how to cause packets to take different > egresses? I may be going too far too fast, but if the interesting > cases involve some agent giving the endpoint that information, > requirements for that agent should be documented, and thus in the > charter. No?. IMO, this goes too far. I would expect it would be useful enough to talk about "multi-address TCP" and avoid talking about "paths" when they're not part of the architecture, nor are they knowable without doing things like pathchar or traceroute. It's useful enough to talk about TCP supporting multiple concurrent endpoint addresses, and to be *agnostic* with respect to varying path characteristics (not so much to tune to them, but to not be de-tuned by them). Joe -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkoDaikACgkQE5f5cImnZrvXRQCgyZ9IIjK6gEq2veMuTtl05npb SDgAni7GWZ6MKwrG1gbfGlAFP18m7mkZ =68uN -----END PGP SIGNATURE----- From fred@cisco.com Thu May 7 17:55:33 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id B65523A68A0 for ; Thu, 7 May 2009 17:55:33 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.477 X-Spam-Level: X-Spam-Status: No, score=-106.477 tagged_above=-999 required=5 tests=[AWL=0.122, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MJ8hFij9hUnz for ; Thu, 7 May 2009 17:55:32 -0700 (PDT) Received: from sj-iport-1.cisco.com (sj-iport-1.cisco.com [171.71.176.70]) by core3.amsl.com (Postfix) with ESMTP id D24023A659C for ; Thu, 7 May 2009 17:55:32 -0700 (PDT) X-IronPort-AV: E=Sophos;i="4.40,313,1238976000"; d="scan'208";a="182566347" Received: from sj-dkim-1.cisco.com ([171.71.179.21]) by sj-iport-1.cisco.com with ESMTP; 08 May 2009 00:57:01 +0000 Received: from sj-core-5.cisco.com (sj-core-5.cisco.com [171.71.177.238]) by sj-dkim-1.cisco.com (8.12.11/8.12.11) with ESMTP id n480v1wg018263; Thu, 7 May 2009 17:57:01 -0700 Received: from stealth-10-32-244-220.cisco.com (stealth-10-32-244-220.cisco.com [10.32.244.220]) by sj-core-5.cisco.com (8.13.8/8.13.8) with ESMTP id n480v1q7021700; Fri, 8 May 2009 00:57:01 GMT Message-Id: <86CDA096-93A8-4B59-8E77-90BF74BC1377@cisco.com> From: Fred Baker To: Iljitsch van Beijnum In-Reply-To: <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Thu, 7 May 2009 17:56:58 -0700 References: <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> X-Mailer: Apple Mail (2.930.3) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; l=372; t=1241744221; x=1242608221; c=relaxed/simple; s=sjdkim1004; h=Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version; d=cisco.com; i=fred@cisco.com; z=From:=20Fred=20Baker=20 |Subject:=20Re=3A=20[multipathtcp]=20Draft=20Multipath=20TC P=20(MPTCP)=20BOF=20description |Sender:=20; bh=N7CsrOs79xEE+PDH2puofnFwopjqCEZazuxJ5imk76E=; b=mVz53Ew9pS2IdLl1RseX+7OM1qp589SHC0C/lqcfAECqP/RXXR2WLW3Pp9 iBbzeD7QH16XeISGWsJTeeP2XTFRBqoB77mrBp/csmRZz3MVVBicAjCkkutN jn+ue5i0db+WtNk3tG52f9ptyN7QFeIHyAPrnJ4uKz1swlJLqDnQI=; Authentication-Results: sj-dkim-1; header.From=fred@cisco.com; dkim=pass ( sig from cisco.com/sjdkim1004 verified; ); Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 May 2009 00:55:33 -0000 On May 7, 2009, at 2:44 PM, Iljitsch van Beijnum wrote: > No. The proof of the pudding is in the eating. In principle, the > host sends packets over a certain path, if ACKs come back the path > works, if they don't it doesn't. I buy that. That tells us something about source address pairs, which determine routing. How about finer-grain routing issues? From michael.scharf@ikr.uni-stuttgart.de Fri May 8 01:19:05 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 8356A3A6E9C for ; Fri, 8 May 2009 01:19:05 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.249 X-Spam-Level: X-Spam-Status: No, score=-2.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1eDouFgifR34 for ; Fri, 8 May 2009 01:19:04 -0700 (PDT) Received: from mailsrv.ikr.uni-stuttgart.de (mailsrv.ikr.uni-stuttgart.de [129.69.170.2]) by core3.amsl.com (Postfix) with ESMTP id C6A7C3A6D44 for ; Fri, 8 May 2009 01:19:04 -0700 (PDT) Received: from netsrv1.ikr.uni-stuttgart.de (netsrv1-c [10.11.12.12]) by mailsrv.ikr.uni-stuttgart.de (Postfix) with ESMTP id ED155439F4; Fri, 8 May 2009 10:20:31 +0200 (CEST) Received: from ikr.uni-stuttgart.de (cnode03 [10.21.20.3]) by netsrv1.ikr.uni-stuttgart.de (Postfix) with SMTP id D528EBC07E; Fri, 8 May 2009 10:20:31 +0200 (CEST) Received: by ikr.uni-stuttgart.de (sSMTP sendmail emulation); Fri, 8 May 2009 10:20:31 +0200 Date: Fri, 8 May 2009 10:20:31 +0200 From: Michael Scharf To: marcelo bagnulo braun Message-ID: <20090508082031.GA6067@ikr.uni-stuttgart.de> References: <4A02FC62.8080503@it.uc3m.es> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline In-Reply-To: <4A02FC62.8080503@it.uc3m.es> User-Agent: Mutt/1.5.18 (2008-05-17) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 May 2009 08:19:05 -0000 On Thu, 07 May 2009 at 17:21:06, marcelo bagnulo braun wrote: > - An extended API describing how applications can > access to multipath information and use it. IMHO this statement suggests that that API mainly allows applications to query information about the multiple paths. Is this the expected purpose? Isn't there a need for an information flow in the reverse direction as well, i. e., a (optional) API that enables applications to control and configure the selection and usage of multiple paths? If so, one could describe the scope of the API more explicitly. Michael From micchie@sfc.wide.ad.jp Fri May 8 02:16:48 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 02D3F3A706E for ; Fri, 8 May 2009 02:16:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.905 X-Spam-Level: X-Spam-Status: No, score=0.905 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_JP=1.244, HOST_EQ_JP=1.265, HTML_MESSAGE=0.001, RELAY_IS_203=0.994] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Wj92MmuU4OWs for ; Fri, 8 May 2009 02:16:46 -0700 (PDT) Received: from mail.sfc.wide.ad.jp (mail.sfc.wide.ad.jp [203.178.142.146]) by core3.amsl.com (Postfix) with ESMTP id 5898F3A7086 for ; Fri, 8 May 2009 02:16:46 -0700 (PDT) Received: from epi.ht.sfc.keio.ac.jp (unknown [IPv6:2001:200:1c0:2800:21f:5bff:fe81:455f]) by mail.sfc.wide.ad.jp (Postfix) with ESMTPSA id 0E3364C0A8 for ; Fri, 8 May 2009 18:22:10 +0900 (JST) Message-ID: <4A03F8D6.7080709@sfc.wide.ad.jp> Date: Fri, 08 May 2009 18:18:14 +0900 From: Michio HONDA User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; ja-JP-mac; rv:1.9.1b3pre) Gecko/20090223 Thunderbird/3.0b2 MIME-Version: 1.0 To: multipathtcp@ietf.org References: <4A01D161.7060108@it.uc3m.es> In-Reply-To: <4A01D161.7060108@it.uc3m.es> Content-Type: multipart/alternative; boundary="------------010905020307040002080303" Subject: Re: [multipathtcp] Fwd: I-D Action:draft-van-beijnum-1e-mp-tcp-00.txt]] X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 May 2009 09:16:48 -0000 This is a multi-part message in MIME format. --------------010905020307040002080303 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Hi, I have a bit of comments on this draft. Sec. 3.5 Fairness and TCP friendliness >Until more analysis has been performed and/or there is more experience with multipath TCP, a multipath TCP implementation SHOULD limit itself to using no more than 3 subflows concurrently. How the decision to allow 3 subflows is made? For example, HTTP spec. (RFC 2116 Sec.8.1.4) describes the simultaneous connections should not use more than 2 connections. The reason contains not only receiver workload, but also congestion control. Hence I think you should mention how multipath TCP (using 3 subflows) deals with such criterions. In addition, it would be good to allow more subflows based on shared bottleneck detection like mTCP(https://www.usenix.org/events/usenix04/tech/general/full_papers/zhang/zhang_html/) Best regards, Michio (5/7/09 3:05 AM), marcelo bagnulo braun wrote: > > with a link to draft, which comes handy sometimes... (especially when > people actually want to read it :-) > > > > -------- Mensaje original -------- > Asunto: I-D Action:draft-van-beijnum-1e-mp-tcp-00.txt > Fecha: Wed, 6 May 2009 10:45:02 -0700 (PDT) > De: Internet-Drafts@ietf.org > Responder a: internet-drafts@ietf.org > Para: i-d-announce@ietf.org > > > > A New Internet-Draft is available from the on-line Internet-Drafts > directories. > > Title : One-ended multipath TCP > Author(s) : I. van Beijnum > Filename : draft-van-beijnum-1e-mp-tcp-00.txt > Pages : 21 > Date : 2009-05-06 > > Normal TCP/IP operation is for the routing system to select a best > path that remains stable for some time, and for TCP to adjust to the > properties of this path to optimize throughput. A multipath TCP > would be able to either use capacity on multiple paths, or > dynamically find the best performing path, and therefore reach higher > throughput. By adapting to the properties of several paths through > the usual congestion control algorithms, a multipath TCP shifts its > traffic to less congested paths, leaving more capacity available for > traffic that can't move to another path on more congested paths. And > when a path fails, this can be detected and worked around by TCP much > more quickly than by waiting for the routing system to repair the > failure. > > This memo specifies a multipath TCP that is implemented on the > sending host only, without requiring modifications on the receiving > host. > > A URL for this Internet-Draft is: > http://www.ietf.org/internet-drafts/draft-van-beijnum-1e-mp-tcp-00.txt > > Internet-Drafts are also available by anonymous FTP at: > ftp://ftp.ietf.org/internet-drafts/ > > Below is the data which will enable a MIME compliant mail reader > implementation to automatically retrieve the ASCII version of the > Internet-Draft. > > > > > ------------------------------------------------------------------------ > > _______________________________________________ > multipathtcp mailing list > multipathtcp@ietf.org > https://www.ietf.org/mailman/listinfo/multipathtcp > -- Michio Honda E-mail: micchie@sfc.wide.ad.jp, micchie@ht.sfc.keio.ac.jp Website: http://www.micchie.net/ --------------010905020307040002080303 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Hi,

I have a bit of comments on this draft. 
   
Sec. 3.5 Fairness and TCP friendliness
>Until more  analysis has been performed and/or there is more experience with multipath TCP, a multipath TCP implementation SHOULD limit itself to  using no more than 3 subflows concurrently.

How the decision to allow 3 subflows is made?
For example, HTTP spec. (RFC 2116 Sec.8.1.4) describes the simultaneous connections should not use more than 2 connections. 
The reason contains not only receiver workload, but also congestion control. 
Hence I think you should mention how multipath TCP (using 3 subflows) deals with such criterions. 

In addition, it would be good to allow more subflows based on shared bottleneck detection like mTCP(https://www.usenix.org/events/usenix04/tech/general/full_papers/zhang/zhang_html/)

Best regards,
Michio

(5/7/09 3:05 AM), marcelo bagnulo braun wrote:

with a link to draft, which comes handy sometimes... (especially when people actually want to read it :-)



-------- Mensaje original --------
Asunto:     I-D Action:draft-van-beijnum-1e-mp-tcp-00.txt
Fecha:     Wed, 6 May 2009 10:45:02 -0700 (PDT)
De:     Internet-Drafts@ietf.org
Responder a:     internet-drafts@ietf.org
Para:     i-d-announce@ietf.org



A New Internet-Draft is available from the on-line Internet-Drafts directories.

    Title           : One-ended multipath TCP
    Author(s)       : I. van Beijnum
    Filename        : draft-van-beijnum-1e-mp-tcp-00.txt
    Pages           : 21
    Date            : 2009-05-06

Normal TCP/IP operation is for the routing system to select a best
path that remains stable for some time, and for TCP to adjust to the
properties of this path to optimize throughput.  A multipath TCP
would be able to either use capacity on multiple paths, or
dynamically find the best performing path, and therefore reach higher
throughput.  By adapting to the properties of several paths through
the usual congestion control algorithms, a multipath TCP shifts its
traffic to less congested paths, leaving more capacity available for
traffic that can't move to another path on more congested paths.  And
when a path fails, this can be detected and worked around by TCP much
more quickly than by waiting for the routing system to repair the
failure.

This memo specifies a multipath TCP that is implemented on the
sending host only, without requiring modifications on the receiving
host.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-van-beijnum-1e-mp-tcp-00.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft. 





_______________________________________________
multipathtcp mailing list
multipathtcp@ietf.org
https://www.ietf.org/mailman/listinfo/multipathtcp
  


-- 
Michio Honda
E-mail: micchie@sfc.wide.ad.jp, micchie@ht.sfc.keio.ac.jp
Website: http://www.micchie.net/
--------------010905020307040002080303-- From baford@mpi-sws.org Fri May 8 02:36:55 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 56FAC28C187 for ; Fri, 8 May 2009 02:36:55 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.249 X-Spam-Level: X-Spam-Status: No, score=-6.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id K1zpguMLsYOJ for ; Fri, 8 May 2009 02:36:53 -0700 (PDT) Received: from apollo.mpi-sb.mpg.de (swsao0807.mpi-sb.mpg.de [139.19.1.50]) by core3.amsl.com (Postfix) with ESMTP id 7084F28C10B for ; Fri, 8 May 2009 02:36:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mpi-sb.mpg.de; s=mail200803; h=Cc:Message-Id:From:To: In-Reply-To:Content-Type:Content-Transfer-Encoding:Mime-Version: Subject:Date:References; bh=vuntb6/tEt+Vgu/kjhWrqYcWpfl0ST9+I0VV 4dMqE5c=; b=TIJSh0+ED8AxqL8yd2tjVEyOnRckTZl5QdjsgXyFZOZkSd8Yp9dL tLRWyLEAxxuAs2MS4lK76bbQCUP+UefGyGSYKBuN5YfW0b6bI7+tYYgWSUJmtnwd 2NK1N8EgWXfEgqOKSTLnkt4UyaV7aHX0LM6t4XmwrjM4KAnC6XDmGyQ= Received: from newzak.mpi-sb.mpg.de ([139.19.1.28]:43647) by apollo.mpi-sb.mpg.de (envelope-from ) with esmtp (Exim 4.69) id 1M2MX2-0007Cm-N1; Fri, 08 May 2009 11:38:19 +0200 Received: from adsl-62-167-26-228.adslplus.ch ([62.167.26.228]:61926 helo=[192.168.1.100]) by newzak.mpi-sb.mpg.de with esmtpsa (TLS-1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.63) (envelope-from ) id 1M2MX2-0001DB-6P; Fri, 08 May 2009 11:38:16 +0200 Message-Id: From: Bryan Ford To: Scott Brim In-Reply-To: <20090507230452.GC35382@cisco.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Fri, 8 May 2009 11:38:15 +0200 References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <20090507230452.GC35382@cisco.com> X-Mailer: Apple Mail (2.930.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 May 2009 09:36:55 -0000 On May 8, 2009, at 1:04 AM, Scott Brim wrote: > Excerpts from Iljitsch van Beijnum on Thu, May 07, 2009 11:44:49PM > +0200: >> In practice things are going to be a bit more complex if we want to >> have >> multipath routers that work together with multipath TCP, because the >> multipath TCP needs to write a value in the packet somewhere that the >> router is going to look at. If everyone agrees on where that field >> is and >> what values are legal, it ends there, but if there are different >> options >> in different networks the host would need to know which options to >> use in >> a particular network. > > Good point -- the endpoint doesn't need to know which paths actually > work, just which ones might work. However, how does the endpoint know > even that? How does it know how to cause packets to take different > egresses? I may be going too far too fast, but if the interesting > cases involve some agent giving the endpoint that information, > requirements for that agent should be documented, and thus in the > charter. No?. But this brings up another issue - why should such a router/transport signaling mechanism be specific to multipath TCP? For example, draft- van-beijnum-1e-mp-tcp-00 proposes a new TCP option with some fields for router interaction. But what if multipath SCTP also wants to use the same router facilities to discover paths or indicate which paths to use? Or multipath DCCP, SST, XYZ...? Why should such network layer extensions be made specific to a particular transport, when they clearly could be useful to any multipath transport? Of course, looking at the practicalities, this line of reasoning might suggest new IP options instead of new TCP options, and we know the difficulties of adding new IP options (i.e., routers and NATs choke on them). On the other hand, we also know the difficulties of adding new TCP options (i.e., there isn't any option space left). Figuring out how to extend the network to support such types of signaling seems to me to fit most clearly in the domain of, e.g., NSIS, not a BOF or WG specific to any particular transport. Bryan > > > Excerpts from marcelo bagnulo braun on Fri, May 08, 2009 12:40:31AM > +0200: >> I understand that one of the points Scott is raising is whehter the >> charter should cover the mptcp-router interaction. >> I am not sure what the answer would be. > > It makes sense to me to have those interactions in the charter. I'm > asking if there is a component missing that should be included, to > support them. > > Thanks much ... Scott > _______________________________________________ > multipathtcp mailing list > multipathtcp@ietf.org > https://www.ietf.org/mailman/listinfo/multipathtcp From iljitsch@muada.com Fri May 8 05:26:38 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id A7E403A69C8 for ; Fri, 8 May 2009 05:26:38 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.1 X-Spam-Level: X-Spam-Status: No, score=-6.1 tagged_above=-999 required=5 tests=[AWL=0.499, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Y9guXitspIfH for ; Fri, 8 May 2009 05:26:37 -0700 (PDT) Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by core3.amsl.com (Postfix) with ESMTP id 302CA3A7115 for ; Fri, 8 May 2009 05:25:47 -0700 (PDT) Received: from [163.117.139.66] ([163.117.139.66]) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n48CQaBT018052 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Fri, 8 May 2009 14:26:42 +0200 (CEST) (envelope-from iljitsch@muada.com) Message-Id: From: Iljitsch van Beijnum To: Fred Baker In-Reply-To: <86CDA096-93A8-4B59-8E77-90BF74BC1377@cisco.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Fri, 8 May 2009 14:26:23 +0200 References: <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <86CDA096-93A8-4B59-8E77-90BF74BC1377@cisco.com> X-Mailer: Apple Mail (2.930.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 May 2009 12:26:38 -0000 On 8 mei 2009, at 2:56, Fred Baker wrote: >> In principle, the host sends packets over a certain path, if ACKs >> come back the path works, if they don't it doesn't. > I buy that. That tells us something about source address pairs, > which determine routing. How about finer-grain routing issues? Like what? From iljitsch@muada.com Fri May 8 05:36:54 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id D9D843A714F for ; Fri, 8 May 2009 05:36:54 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.116 X-Spam-Level: X-Spam-Status: No, score=-6.116 tagged_above=-999 required=5 tests=[AWL=0.483, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bGEhSBpSDH36 for ; Fri, 8 May 2009 05:36:54 -0700 (PDT) Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by core3.amsl.com (Postfix) with ESMTP id 0B6AC3A7115 for ; Fri, 8 May 2009 05:36:34 -0700 (PDT) Received: from [163.117.139.66] ([163.117.139.66]) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n48CbSGE018123 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Fri, 8 May 2009 14:37:34 +0200 (CEST) (envelope-from iljitsch@muada.com) Message-Id: From: Iljitsch van Beijnum To: Bryan Ford In-Reply-To: Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Fri, 8 May 2009 14:37:15 +0200 References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <20090507230452.GC35382@cisco.com> X-Mailer: Apple Mail (2.930.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 May 2009 12:36:55 -0000 On 8 mei 2009, at 11:38, Bryan Ford wrote: > But this brings up another issue - why should such a router/ > transport signaling mechanism be specific to multipath TCP? For > example, draft-van-beijnum-1e-mp-tcp-00 proposes a new TCP option > with some fields for router interaction. It does say this: It is not expected that routers will make routing decisions directly based on the path indication option, as this option occurs deep inside the packet and not in a fixed place. (However, the text before that basically assumes that routers do look at this field so your conclusion isn't unreasonable.) > But what if multipath SCTP also wants to use the same router > facilities to discover paths or indicate which paths to use? Or > multipath DCCP, SST, XYZ...? Why should such network layer > extensions be made specific to a particular transport, when they > clearly could be useful to any multipath transport? It shouldn't, of course. If we want routers to make a path selection decision based on information from a multipath transport protocol, we need a field in packets to convey this information. And it should be standardized so all layers and boxes involved know how to interpret the field. > Of course, looking at the practicalities, this line of reasoning > might suggest new IP options instead of new TCP options, and we know > the difficulties of adding new IP options (i.e., routers and NATs > choke on them). Indeed. > On the other hand, we also know the difficulties of adding new TCP > options (i.e., there isn't any option space left). Figuring out how > to extend the network to support such types of signaling seems to me > to fit most clearly in the domain of, e.g., NSIS, not a BOF or WG > specific to any particular transport. I only had a quick look at NSIS, but isn't this mostly about signaling and signaling protocols? AFAIK RSVP recognizes flows, presumably using port numbers or maybe through the IPv6 flow label. In the former case, we have a problem because port numbers can't arbitrarily changed for an existing session and/or NATs change them in unpredictable ways. The flow label and DSCP bits also have existing uses and limitations but perhaps these could be reused for this if done carefully. Alternatively, the path selection field could be included in a layer below IP, which makes it easier to come up with something that's not backward compatible because the encapsulation containing the field can be stripped away when the packet leaves the multipath-aware part of the network. As the signaling part of this shares a lot with QoS I think it makes sense to go to NSIS for that, but there's also interactions with the internet layer/area and routing, as well as transport. From marcelo@it.uc3m.es Fri May 8 06:02:30 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 514D93A7162 for ; Fri, 8 May 2009 06:02:30 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YqQprGcJOw2I for ; Fri, 8 May 2009 06:02:29 -0700 (PDT) Received: from smtp01.uc3m.es (smtp01.uc3m.es [163.117.176.131]) by core3.amsl.com (Postfix) with ESMTP id 316DD3A6855 for ; Fri, 8 May 2009 06:02:28 -0700 (PDT) Received: from marcelo-bagnulos-macbook-pro.local (186.Red-83-55-62.dynamicIP.rima-tde.net [83.55.62.186]) by smtp01.uc3m.es (Postfix) with ESMTP id 6C0C0B540D0; Fri, 8 May 2009 15:03:53 +0200 (CEST) Message-ID: <4A042DB9.5040608@it.uc3m.es> Date: Fri, 08 May 2009 15:03:53 +0200 From: marcelo bagnulo braun User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Scott Brim , multipathtcp@ietf.org References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <20090507230452.GC35382@cisco.com> In-Reply-To: <20090507230452.GC35382@cisco.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-TM-AS-Product-Ver: IMSS-7.0.0.3116-5.6.0.1016-16628.005 Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 May 2009 13:02:30 -0000 Scott Brim escribió: > Excerpts from Iljitsch van Beijnum on Thu, May 07, 2009 11:44:49PM +0200: > >> In practice things are going to be a bit more complex if we want to have >> multipath routers that work together with multipath TCP, because the >> multipath TCP needs to write a value in the packet somewhere that the >> router is going to look at. If everyone agrees on where that field is and >> what values are legal, it ends there, but if there are different options >> in different networks the host would need to know which options to use in >> a particular network. >> > > Good point -- the endpoint doesn't need to know which paths actually > work, just which ones might work. However, how does the endpoint know > even that? How does it know how to cause packets to take different > egresses? I may be going too far too fast, but if the interesting > cases involve some agent giving the endpoint that information, > requirements for that agent should be documented, and thus in the > charter. No?. > mmm, i would say that if an endpoint is multipaht capable and have some way to express different paths, it simply does it. If the underlying routing infrastrucutre doesn't support that, the endpoint is not worth than sending all through a single path (assuming we can actually manage to make MPTCP fair to other single path tcps) In other words, MPTCP marks the packets and hopes the IP (locally in the host or a router somehwehre else) will use that mark to send the packet through one path or another. If there is no such capability, the endpoint is not worse than the normal single path. > > Excerpts from marcelo bagnulo braun on Fri, May 08, 2009 12:40:31AM +0200: > >> I understand that one of the points Scott is raising is whehter the >> charter should cover the mptcp-router interaction. >> I am not sure what the answer would be. >> > > It makes sense to me to have those interactions in the charter. I'm > asking if there is a component missing that should be included, to > support them. > > My take is that there will be many different ways to provide IP support to MPTCP. I don't know if it would make sense to write them down in an informational draft. But i fail to see the need for a new component, if i understnad correctly what you say. OTOH, it seems clear that more description of how MPTCP interacts with multiapth IP is needed. Regards, marcelo > Thanks much ... Scott > _______________________________________________ > multipathtcp mailing list > multipathtcp@ietf.org > https://www.ietf.org/mailman/listinfo/multipathtcp > > From marcelo@it.uc3m.es Fri May 8 06:08:35 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id AFB563A7151 for ; Fri, 8 May 2009 06:08:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 24daFQTsibeM for ; Fri, 8 May 2009 06:08:34 -0700 (PDT) Received: from smtp01.uc3m.es (smtp01.uc3m.es [163.117.176.131]) by core3.amsl.com (Postfix) with ESMTP id 455253A714F for ; Fri, 8 May 2009 06:08:33 -0700 (PDT) Received: from marcelo-bagnulos-macbook-pro.local (186.Red-83-55-62.dynamicIP.rima-tde.net [83.55.62.186]) by smtp01.uc3m.es (Postfix) with ESMTP id 0A6AEB4935B; Fri, 8 May 2009 15:10:00 +0200 (CEST) Message-ID: <4A042F28.5050107@it.uc3m.es> Date: Fri, 08 May 2009 15:10:00 +0200 From: marcelo bagnulo braun User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Joe Touch References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <20090507230452.GC35382@cisco.com> <4A036A29.7020005@isi.edu> In-Reply-To: <4A036A29.7020005@isi.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-TM-AS-Product-Ver: IMSS-7.0.0.3116-5.6.0.1016-16628.005 Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 May 2009 13:08:35 -0000 Joe Touch escribió: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > > > Scott Brim wrote: > >> Excerpts from Iljitsch van Beijnum on Thu, May 07, 2009 11:44:49PM +0200: >> >>> In practice things are going to be a bit more complex if we want to have >>> multipath routers that work together with multipath TCP, because the >>> multipath TCP needs to write a value in the packet somewhere that the >>> router is going to look at. If everyone agrees on where that field is and >>> what values are legal, it ends there, but if there are different options >>> in different networks the host would need to know which options to use in >>> a particular network. >>> >> Good point -- the endpoint doesn't need to know which paths actually >> work, just which ones might work. However, how does the endpoint know >> even that? How does it know how to cause packets to take different >> egresses? I may be going too far too fast, but if the interesting >> cases involve some agent giving the endpoint that information, >> requirements for that agent should be documented, and thus in the >> charter. No?. >> > > IMO, this goes too far. I would expect it would be useful enough to talk > about "multi-address TCP" and avoid talking about "paths" when they're > not part of the architecture, nor are they knowable without doing things > like pathchar or traceroute. > But there are many situations and probably very interesting ones, where the node has multipaht paths and does not have multiple addresses. For instnace, think about a server located in a big multihomed site using PI addresses. This server is likely to benefit from multipath TCP but it only has a single address. In addition, even if you do have multiple addresses, you still need some support from the routing infrastrcuture. consider the following case. One endpoint A has multiple addresses and the other endpoint B has a single address. When a node A with multiple addresses sends packets, it will include different source addresses, but all packets will carry the same destiantion address. How can the nade A influence the egress path? clearly, today changing the source address has only one potential effect, that is that the packets are dropped due to ingress filters, but it does no have the ffect of changing the exit path So even in the multiaddress case, we need support from the IP layer. The support in all the cases (multiple addres MPTCP or single address MPTCP is the same. That the IP layer is capable of looking into a path identifier included in the packet 8either the course address or other magic bits) and selects a different path based on those bits. regards, marcelo > It's useful enough to talk about TCP supporting multiple concurrent > endpoint addresses, and to be *agnostic* with respect to varying path > characteristics (not so much to tune to them, but to not be de-tuned by > them). > > Joe > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.9 (MingW32) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iEYEARECAAYFAkoDaikACgkQE5f5cImnZrvXRQCgyZ9IIjK6gEq2veMuTtl05npb > SDgAni7GWZ6MKwrG1gbfGlAFP18m7mkZ > =68uN > -----END PGP SIGNATURE----- > _______________________________________________ > multipathtcp mailing list > multipathtcp@ietf.org > https://www.ietf.org/mailman/listinfo/multipathtcp > > From marcelo@it.uc3m.es Fri May 8 06:10:32 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 44BC93A714F for ; Fri, 8 May 2009 06:10:32 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZWYgc6eleL5d for ; Fri, 8 May 2009 06:10:31 -0700 (PDT) Received: from smtp02.uc3m.es (smtp02.uc3m.es [163.117.176.132]) by core3.amsl.com (Postfix) with ESMTP id 7A8DB3A6BED for ; Fri, 8 May 2009 06:10:31 -0700 (PDT) Received: from marcelo-bagnulos-macbook-pro.local (186.Red-83-55-62.dynamicIP.rima-tde.net [83.55.62.186]) by smtp02.uc3m.es (Postfix) with ESMTP id AECE765640B; Fri, 8 May 2009 15:11:58 +0200 (CEST) Message-ID: <4A042F9E.4030700@it.uc3m.es> Date: Fri, 08 May 2009 15:11:58 +0200 From: marcelo bagnulo braun User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Michael Scharf References: <4A02FC62.8080503@it.uc3m.es> <20090508082031.GA6067@ikr.uni-stuttgart.de> In-Reply-To: <20090508082031.GA6067@ikr.uni-stuttgart.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-TM-AS-Product-Ver: IMSS-7.0.0.3116-5.6.0.1016-16628.005 Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 May 2009 13:10:32 -0000 Michael Scharf escribió: > On Thu, 07 May 2009 at 17:21:06, marcelo bagnulo braun wrote: > >> - An extended API describing how applications can >> access to multipath information and use it. >> > > IMHO this statement suggests that that API mainly allows applications > to query information about the multiple paths. Is this the expected > purpose? > > Isn't there a need for an information flow in the reverse direction as > well, i. e., a (optional) API that enables applications to control and > configure the selection and usage of multiple paths? > > If so, one could describe the scope of the API more explicitly. > makes sense to me do you care to provide new text? It would be very mch appreciated thanks, marcelo > Michael > > From marcelo@it.uc3m.es Fri May 8 06:14:09 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id D486D3A6CE4 for ; Fri, 8 May 2009 06:14:09 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lFE0WeRE6VPA for ; Fri, 8 May 2009 06:14:09 -0700 (PDT) Received: from smtp02.uc3m.es (smtp02.uc3m.es [163.117.176.132]) by core3.amsl.com (Postfix) with ESMTP id A0EFF3A6BED for ; Fri, 8 May 2009 06:14:08 -0700 (PDT) Received: from marcelo-bagnulos-macbook-pro.local (186.Red-83-55-62.dynamicIP.rima-tde.net [83.55.62.186]) by smtp02.uc3m.es (Postfix) with ESMTP id 71E0265A04B; Fri, 8 May 2009 15:15:35 +0200 (CEST) Message-ID: <4A043077.6070309@it.uc3m.es> Date: Fri, 08 May 2009 15:15:35 +0200 From: marcelo bagnulo braun User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Bryan Ford References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <20090507230452.GC35382@cisco.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-TM-AS-Product-Ver: IMSS-7.0.0.3116-5.6.0.1016-16628.005 Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 May 2009 13:14:09 -0000 Bryan Ford escribió: > > On May 8, 2009, at 1:04 AM, Scott Brim wrote: > >> Excerpts from Iljitsch van Beijnum on Thu, May 07, 2009 11:44:49PM >> +0200: >>> In practice things are going to be a bit more complex if we want to >>> have >>> multipath routers that work together with multipath TCP, because the >>> multipath TCP needs to write a value in the packet somewhere that the >>> router is going to look at. If everyone agrees on where that field >>> is and >>> what values are legal, it ends there, but if there are different >>> options >>> in different networks the host would need to know which options to >>> use in >>> a particular network. >> >> Good point -- the endpoint doesn't need to know which paths actually >> work, just which ones might work. However, how does the endpoint know >> even that? How does it know how to cause packets to take different >> egresses? I may be going too far too fast, but if the interesting >> cases involve some agent giving the endpoint that information, >> requirements for that agent should be documented, and thus in the >> charter. No?. > > But this brings up another issue - why should such a router/transport > signaling mechanism be specific to multipath TCP? For example, > draft-van-beijnum-1e-mp-tcp-00 proposes a new TCP option with some > fields for router interaction. But what if multipath SCTP also wants > to use the same router facilities to discover paths or indicate which > paths to use? Or multipath DCCP, SST, XYZ...? Why should such > network layer extensions be made specific to a particular transport, > when they clearly could be useful to any multipath transport? agree, the question is which path selector bits we use? > > Of course, looking at the practicalities, this line of reasoning might > suggest new IP options instead of new TCP options, and we know the > difficulties of adding new IP options (i.e., routers and NATs choke on > them). On the other hand, we also know the difficulties of adding new > TCP options (i.e., there isn't any option space left). Figuring out > how to extend the network to support such types of signaling seems to > me to fit most clearly in the domain of, e.g., NSIS, not a BOF or WG > specific to any particular transport. any ideas of which bits could these be? Regards, marcelo > > Bryan > >> >> >> Excerpts from marcelo bagnulo braun on Fri, May 08, 2009 12:40:31AM >> +0200: >>> I understand that one of the points Scott is raising is whehter the >>> charter should cover the mptcp-router interaction. >>> I am not sure what the answer would be. >> >> It makes sense to me to have those interactions in the charter. I'm >> asking if there is a component missing that should be included, to >> support them. >> >> Thanks much ... Scott >> _______________________________________________ >> multipathtcp mailing list >> multipathtcp@ietf.org >> https://www.ietf.org/mailman/listinfo/multipathtcp > > _______________________________________________ > multipathtcp mailing list > multipathtcp@ietf.org > https://www.ietf.org/mailman/listinfo/multipathtcp > From aongenae@gmail.com Fri May 8 06:30:19 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 7217F28C1B2 for ; Fri, 8 May 2009 06:30:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tgRt8Lnh4ORl for ; Fri, 8 May 2009 06:30:18 -0700 (PDT) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.158]) by core3.amsl.com (Postfix) with ESMTP id 2896528C1C6 for ; Fri, 8 May 2009 06:30:18 -0700 (PDT) Received: by fg-out-1718.google.com with SMTP id d23so122160fga.18 for ; Fri, 08 May 2009 06:31:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:subject:from:to:cc :in-reply-to:references:content-type:date:message-id:mime-version :x-mailer; bh=IeEXu88xZHZATiTw92oBxabO6Qa+ITsrhIJgHX4T7fA=; b=sB3C7sr5xx0k59tW4g67uUhJeT4Fo+v6bjWnlHXFx63i8YP7ZO72/85NWNsPZf9UVC vYzQh9b8VbzhUh3rP61fW0ovbBejGAAVU1GnmpteVmdKg03uhGbJx515J1avvz66JHom 02cuYwFDTU7Qpizuz0YFnmSngnHODD9QCKopM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=subject:from:to:cc:in-reply-to:references:content-type:date :message-id:mime-version:x-mailer; b=U0FEDqUEgeNkb6KL4Msgu1eENi8KNVTm55LbhYu6kkR4kCmQI4kn7GWGAfWeoHIhHQ N4Y9+CS/i0IzvEcgTtYUl7zZRFrK73ftI4QooaM/3jxHWd4lCs/22fDcjW8CSYPJkILT pkhIAUfh2j2pJPGrRJGSw0FctlGhh5XBsBbHA= Received: by 10.86.95.8 with SMTP id s8mr3674106fgb.24.1241789503913; Fri, 08 May 2009 06:31:43 -0700 (PDT) Received: from ?192.168.1.12? ([91.177.75.226]) by mx.google.com with ESMTPS id 4sm1039396fge.8.2009.05.08.06.31.41 (version=SSLv3 cipher=RC4-MD5); Fri, 08 May 2009 06:31:42 -0700 (PDT) From: Arnaud Ongenae To: Iljitsch van Beijnum In-Reply-To: References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <20090507230452.GC35382@cisco.com> Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-uiqG9ixXKFnmRTND8nUV" Date: Fri, 08 May 2009 15:31:37 +0200 Message-Id: <1241789497.4055.8.camel@nhuynhth-laptop> Mime-Version: 1.0 X-Mailer: Evolution 2.26.1 Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 May 2009 13:30:19 -0000 --=-uiqG9ixXKFnmRTND8nUV Content-Type: text/plain Content-Transfer-Encoding: quoted-printable >=20 > > Of course, looking at the practicalities, this line of reasoning =20 > > might suggest new IP options instead of new TCP options, and we know =20 > > the difficulties of adding new IP options (i.e., routers and NATs =20 > > choke on them). >=20 > Indeed. >=20 As I understand, you want to add an option in TCP so that it has great chance to pass through NAT/Firewall/etc. (look as much as possible at the original TCP). If the new option might be discarded, why do you want so much to look like TCP and do not support SCTP (and Jana's work) to achieve multipath transfer ? (maybe this point was already discussed elsewhere do not hesitate to redirect me if needed) Arnaud Ongenae UCL (Belgium) student --=-uiqG9ixXKFnmRTND8nUV Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEABECAAYFAkoENDIACgkQHpJ+LMVPWRIdAwCfSVx+f1K1IeE2HbsrdV2Qnyy0 Zm8An0F131stYJoP/v3Qb6KUxnhdgK2j =hyZ8 -----END PGP SIGNATURE----- --=-uiqG9ixXKFnmRTND8nUV-- From michael.scharf@ikr.uni-stuttgart.de Fri May 8 07:12:51 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id B50E23A7134 for ; Fri, 8 May 2009 07:12:51 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.249 X-Spam-Level: X-Spam-Status: No, score=-2.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QJteiQ0V+ynH for ; Fri, 8 May 2009 07:12:51 -0700 (PDT) Received: from mailsrv.ikr.uni-stuttgart.de (mailsrv.ikr.uni-stuttgart.de [129.69.170.2]) by core3.amsl.com (Postfix) with ESMTP id 4E2483A716D for ; Fri, 8 May 2009 07:12:42 -0700 (PDT) Received: from netsrv1.ikr.uni-stuttgart.de (netsrv1-c [10.11.12.12]) by mailsrv.ikr.uni-stuttgart.de (Postfix) with ESMTP id 7718743A03; Fri, 8 May 2009 16:14:10 +0200 (CEST) Received: from ikr.uni-stuttgart.de (cnode03 [10.21.20.3]) by netsrv1.ikr.uni-stuttgart.de (Postfix) with SMTP id 5F428BC07E; Fri, 8 May 2009 16:14:10 +0200 (CEST) Received: by ikr.uni-stuttgart.de (sSMTP sendmail emulation); Fri, 8 May 2009 16:14:10 +0200 Date: Fri, 8 May 2009 16:14:10 +0200 From: Michael Scharf To: marcelo bagnulo braun Message-ID: <20090508141410.GA22483@ikr.uni-stuttgart.de> References: <4A02FC62.8080503@it.uc3m.es> <20090508082031.GA6067@ikr.uni-stuttgart.de> <4A042F9E.4030700@it.uc3m.es> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline In-Reply-To: <4A042F9E.4030700@it.uc3m.es> User-Agent: Mutt/1.5.18 (2008-05-17) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 May 2009 14:12:51 -0000 On Fri, 08 May 2009 at 15:11:58, marcelo bagnulo braun wrote: >>> - An extended API describing how applications can >>> access to multipath information and use it. >> >> If so, one could describe the scope of the API more explicitly. >> > > makes sense to me > do you care to provide new text? It would be very mch appreciated What about this text: "- An extended API describing how applications can use multipath transport. The API should also provide access to multipath information and enable the control of the usage." One simple example for such control would be an extension of the "NO_DELAY" socket option, i. e., applications could explicitly state that they prefer a path with mimimum latency. But there are probably some other useful control primitives as well. Michael From iljitsch@muada.com Fri May 8 07:23:10 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 23DE23A6C38 for ; Fri, 8 May 2009 07:23:10 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.131 X-Spam-Level: X-Spam-Status: No, score=-6.131 tagged_above=-999 required=5 tests=[AWL=0.468, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rjEXKP6EaMcv for ; Fri, 8 May 2009 07:23:09 -0700 (PDT) Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by core3.amsl.com (Postfix) with ESMTP id 237C83A68BF for ; Fri, 8 May 2009 07:23:08 -0700 (PDT) Received: from [163.117.139.66] (claw.it.uc3m.es [163.117.139.66]) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n48EO3WW018982 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Fri, 8 May 2009 16:24:08 +0200 (CEST) (envelope-from iljitsch@muada.com) Message-Id: From: Iljitsch van Beijnum To: Arnaud Ongenae In-Reply-To: <1241789497.4055.8.camel@nhuynhth-laptop> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Fri, 8 May 2009 16:23:49 +0200 References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <20090507230452.GC35382@cisco.com> <1241789497.4055.8.camel@nhuynhth-laptop> X-Mailer: Apple Mail (2.930.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 May 2009 14:23:10 -0000 On 8 mei 2009, at 15:31, Arnaud Ongenae wrote: > As I understand, you want to add an option in TCP so that it has great > chance to pass through NAT/Firewall/etc. (look as much as possible at > the original TCP). For the one-ended multipath TCP draft it must look like normal TCP because otherwise the receiver (which we assume isn't multipath capable) won't be able to process the packet. In this draft, the TCP option is there in case both ends are multipath- capable, so that when doing RTT measurements it's possible to see which path the other side used so it's possible to have different RTTs and therefore RTOs for the different paths. > If the new option might be discarded, why do you want > so much to look like TCP and do not support SCTP (and Jana's work) to > achieve multipath transfer ? (maybe this point was already discussed > elsewhere do not hesitate to redirect me if needed) I think a multipath-aware IP layer, which the one-ended multipath TCP needs, should also be available to other multipath transports, like SCTP with CMT (concurrent multipath). From iljitsch@muada.com Fri May 8 07:40:11 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 7FCD63A68EE for ; Fri, 8 May 2009 07:40:11 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.145 X-Spam-Level: X-Spam-Status: No, score=-6.145 tagged_above=-999 required=5 tests=[AWL=0.454, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3U+nhmvhqL4W for ; Fri, 8 May 2009 07:40:10 -0700 (PDT) Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by core3.amsl.com (Postfix) with ESMTP id 467633A684C for ; Fri, 8 May 2009 07:40:10 -0700 (PDT) Received: from claw.it.uc3m.es (claw.it.uc3m.es [163.117.139.66]) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n48EZ28U019087 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Fri, 8 May 2009 16:35:08 +0200 (CEST) (envelope-from iljitsch@muada.com) Message-Id: From: Iljitsch van Beijnum To: janardhan.iyengar@fandm.edu In-Reply-To: <4A02F0B7.6080202@fandm.edu> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Fri, 8 May 2009 16:34:49 +0200 References: <4A01D161.7060108@it.uc3m.es> <4A02F0B7.6080202@fandm.edu> X-Mailer: Apple Mail (2.930.3) Cc: nasif ekiz , multipathtcp@ietf.org, "Paul D. Amer" , Randall Stewart , Preethi Natarajan , Ertugrul YILMAZ Subject: Re: [multipathtcp] Fwd: I-D Action:draft-van-beijnum-1e-mp-tcp-00.txt]] X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 May 2009 14:40:11 -0000 Jana, Thanks for all the pointers. I'll go through them to see if there are any that I haven't seen before. On 7 mei 2009, at 16:31, Janardhan Iyengar wrote: > Marcelo, > With regard to the statement: "In addition, SCTP as > defined today [RFC2960] does not accommodate the concurrent use of > multiple paths. Additional paths are purely used for backup > purposes." > let me encourage you to look up Concurrent Multipath Tranfer (CMT) > with SCTP. While this work used SCTP for CMT, many of the issues are > not SCTP-specific; Right. However, note that RFC 2960 says that only one path is used to send traffic, the other path is only used for retransmissions. (I'll have to check with RFC 4960.) > + Our Transactions on Networking paper from 2006 (http://www.fandm.edu/jiyengar/papers/cmt-ton2006.pdf > ) discusses several of the concerns that you bring up here, only in > the context of SCTP. Right, I'm familiar with this paper. > + You list 4 challenges and questions, and say that you address > the first two (scheduling, reordering). We have looked extensively > at those two and also the third one (receiver buffer constraints), > and our results are documented. Note that for SCTP the send and receive buffer issues are different when there are multiple streams. > I like to think that we did quite a bit of work in this area > already, and while there are unresolved issues still around, I also > like to think that we got some important things down :-) It would > be very disheartening to see work that is pretty much exactly the > same not using our previous work. I agree. But SCTP results don't always translate into TCP results. And there's still plenty of work to do to apply the research to implementations. From preethi.cis@gmail.com Fri May 8 10:08:02 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 323A93A69D3 for ; Fri, 8 May 2009 10:08:02 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VsJ7GbOwnTre for ; Fri, 8 May 2009 10:08:01 -0700 (PDT) Received: from qw-out-2122.google.com (qw-out-2122.google.com [74.125.92.24]) by core3.amsl.com (Postfix) with ESMTP id 4A8663A672F for ; Fri, 8 May 2009 10:08:01 -0700 (PDT) Received: by qw-out-2122.google.com with SMTP id 3so1457959qwe.31 for ; Fri, 08 May 2009 10:09:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:from:to:cc:references:subject :date:message-id:mime-version:content-type:content-transfer-encoding :x-mailer:thread-index:in-reply-to:x-mimeole; bh=RKib83RsVOjdLME25vqRIBbLItZU49Dkv9KSOiZp4eM=; b=tOJ3pcUcZCPCqZajKReBCl5fsTwFK5XOQlV9dZUqEQbB8OCj770JlP6xJCHk948An5 2d0olwVDW63eR8zPumohTtco0UxgctYkONj2fYqZr8OkZyBiQ1pOpvh9bv96vEkcbNQN RCzY8qoABwQ5qKnoHsM1+OxZVuVFSe8mNMVsE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:cc:references:subject:date:message-id:mime-version :content-type:content-transfer-encoding:x-mailer:thread-index :in-reply-to:x-mimeole; b=iEhf/qGSNHBPHYOEFotO5T/9lfMXrHssKIJZ9k96oN6suZMBpO1fQyftB3oXgLbgrq dgtm/KV7rwa1isSxjz1TtBCXolpTG/fnIXf8aPONKjkzPx0Et/GOZ/h+fgEss7CdREVG jTJxd9I12dz42lJDhqXlj3HO+V+ZA2xatoJoM= Received: by 10.142.135.16 with SMTP id i16mr1599518wfd.238.1241802567353; Fri, 08 May 2009 10:09:27 -0700 (PDT) Received: from prenatarwxp01 (dhcp-171-71-57-16.cisco.com [171.71.57.16]) by mx.google.com with ESMTPS id b39sm2060622rvf.3.2009.05.08.10.09.26 (version=SSLv3 cipher=RC4-MD5); Fri, 08 May 2009 10:09:26 -0700 (PDT) From: "Preethi Natarajan" To: "'Iljitsch van Beijnum'" , "'Arnaud Ongenae'" References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es><4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><20090507230452.GC35382@cisco.com><1241789497.4055.8.camel@nhuynhth-laptop> Date: Fri, 8 May 2009 10:09:25 -0700 Message-ID: <001e01c9cfff$bde8f1e0$103947ab@cisco.com> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 11 Thread-Index: AcnP6LrL/KA6vij5Slq/HEaNqeVy2AAFL5JA In-Reply-To: X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3350 Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 May 2009 17:08:02 -0000 > -----Original Message----- > From: multipathtcp-bounces@ietf.org > [mailto:multipathtcp-bounces@ietf.org] On Behalf Of Iljitsch > van Beijnum > On 8 mei 2009, at 15:31, Arnaud Ongenae wrote: > > > If the new option might be discarded, why do you want so > much to look > > like TCP and do not support SCTP (and Jana's work) to achieve > > multipath transfer ? (maybe this point was already > discussed elsewhere > > do not hesitate to redirect me if needed) > > I think a multipath-aware IP layer, which the one-ended > multipath TCP needs, should also be available to other > multipath transports, like SCTP with CMT (concurrent multipath). I suppose what Arnaud is asking is the motivation to modify TCP such that the modified TCP requires new middlebox considerations when other multihome-capable and multipath-aware transports exist. I think this is different from whether or not a multipath transport requires multipath-aware IP. Preethi From iljitsch@muada.com Fri May 8 10:24:00 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 8A86D3A67F1 for ; Fri, 8 May 2009 10:24:00 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.159 X-Spam-Level: X-Spam-Status: No, score=-6.159 tagged_above=-999 required=5 tests=[AWL=0.440, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id x5iDvgAYeLRQ for ; Fri, 8 May 2009 10:23:59 -0700 (PDT) Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by core3.amsl.com (Postfix) with ESMTP id 9E5DF3A6A8C for ; Fri, 8 May 2009 10:23:59 -0700 (PDT) Received: from claw.it.uc3m.es (claw.it.uc3m.es [163.117.139.66]) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n48HNteh019970 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Fri, 8 May 2009 19:23:56 +0200 (CEST) (envelope-from iljitsch@muada.com) Message-Id: <97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com> From: Iljitsch van Beijnum To: "Preethi Natarajan" In-Reply-To: <001e01c9cfff$bde8f1e0$103947ab@cisco.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Fri, 8 May 2009 19:23:58 +0200 References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es><4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><20090507230452.GC35382@cisco.com><1241789497.4055.8.camel@nhuynhth-laptop> <001e01c9cfff$bde8f1e0$103947ab@cisco.com> X-Mailer: Apple Mail (2.930.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 May 2009 17:24:00 -0000 On 8 mei 2009, at 19:09, Preethi Natarajan wrote: > I suppose what Arnaud is asking is the motivation to modify TCP such > that > the modified TCP requires new middlebox considerations when other > multihome-capable and multipath-aware transports exist. The motivations are better performance and redundancy without requiring application changes or any changes at the receiver. If people put middleboxes in their networks that break MPTCP then obviously they won't enjoy the benefits. That is a choice that everyone needs to make for themselves. From preethi.cis@gmail.com Fri May 8 10:47:35 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 7E23A3A717A for ; Fri, 8 May 2009 10:47:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HY5rOZ2jOh1Z for ; Fri, 8 May 2009 10:47:34 -0700 (PDT) Received: from rv-out-0506.google.com (rv-out-0506.google.com [209.85.198.234]) by core3.amsl.com (Postfix) with ESMTP id 5B4F128C1F7 for ; Fri, 8 May 2009 10:46:05 -0700 (PDT) Received: by rv-out-0506.google.com with SMTP id g37so1135978rvb.49 for ; Fri, 08 May 2009 10:47:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:from:to:cc:references:subject :date:message-id:mime-version:content-type:content-transfer-encoding :x-mailer:thread-index:in-reply-to:x-mimeole; bh=HU3v6/hWPc05WDCrcd/LtV3aCtiy9sd0djl9bcWGbBg=; b=DITBk37zW+KHEnmo72CW671eHg0urA5EmsiFtinm6645hX9P94oNnBJWYAiMNul9AU YT89yTEbUAha5O0TMunfqZDBvuMkBRbWX+IehAo5LEUQtEO043S4Rwsl4lZcU2yUenxt vETKSFik9Nx0rDOzYOuPPVStJarF58rKe3OMQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:cc:references:subject:date:message-id:mime-version :content-type:content-transfer-encoding:x-mailer:thread-index :in-reply-to:x-mimeole; b=CP7YPBWlaz9kNKQ+cGw7SeNmVUaXkQER5MZqYuGNkaCYhw4W9jbPvsb1EUILiOzvh9 KHwHgsmIt4YZG0+TORIaKve82QROT2LaeVRtr9F+M/ND55KlOoStlvZXyrAUpG3nqNhR ZKuX45ZWBIqcaKdYFhjJVcIH9PunfygYnhG2o= Received: by 10.141.162.1 with SMTP id p1mr1766030rvo.259.1241804846614; Fri, 08 May 2009 10:47:26 -0700 (PDT) Received: from prenatarwxp01 (dhcp-171-71-57-16.cisco.com [171.71.57.16]) by mx.google.com with ESMTPS id g31sm2816038rvb.33.2009.05.08.10.47.25 (version=SSLv3 cipher=RC4-MD5); Fri, 08 May 2009 10:47:25 -0700 (PDT) From: "Preethi Natarajan" To: "'Iljitsch van Beijnum'" References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es><4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><20090507230452.GC35382@cisco.com><1241789497.4055.8.camel@nhuynhth-laptop> <001e01c9cfff$bde8f1e0$103947ab@cisco.com> <97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com> Date: Fri, 8 May 2009 10:47:24 -0700 Message-ID: <001f01c9d005$0c583110$103947ab@cisco.com> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 11 Thread-Index: AcnQAc42Zosor8X8S+iKkhxhK5l9mwAAViFA In-Reply-To: <97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3350 Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 May 2009 17:47:35 -0000 > -----Original Message----- > From: Iljitsch van Beijnum [mailto:iljitsch@muada.com] > Sent: Friday, May 08, 2009 10:24 AM > To: Preethi Natarajan > Cc: 'Arnaud Ongenae'; multipathtcp@ietf.org > > The motivations are better performance and redundancy without > requiring application changes or any changes at the receiver. Do you mean the application mods to open an SCTP (or whatever) socket instead of TCP socket vs. mods to TCP stacks and middleboxes? From iljitsch@muada.com Fri May 8 13:07:58 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id E467C28C1C1 for ; Fri, 8 May 2009 13:07:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.941 X-Spam-Level: X-Spam-Status: No, score=-5.941 tagged_above=-999 required=5 tests=[AWL=0.658, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zuuVMJPe+HET for ; Fri, 8 May 2009 13:07:58 -0700 (PDT) Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by core3.amsl.com (Postfix) with ESMTP id 79C6B3A6E57 for ; Fri, 8 May 2009 13:07:55 -0700 (PDT) Received: from [192.168.2.4] (static-167-138-7-89.ipcom.comunitel.net [89.7.138.167] (may be forged)) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n48K8tFE020929 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Fri, 8 May 2009 22:08:55 +0200 (CEST) (envelope-from iljitsch@muada.com) Message-Id: From: Iljitsch van Beijnum To: "Preethi Natarajan" In-Reply-To: <001f01c9d005$0c583110$103947ab@cisco.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Fri, 8 May 2009 22:08:56 +0200 References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es><4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><20090507230452.GC35382@cisco.com><1241789497.4055.8.camel@nhuynhth-laptop> <001e01c9cfff$bde8f1e0$103947ab@cisco.com> <97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com> <001f01c9d005$0c583110$103947ab@cisco.com> X-Mailer: Apple Mail (2.930.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 May 2009 20:07:59 -0000 On 8 mei 2009, at 19:47, Preethi Natarajan wrote: >> The motivations are better performance and redundancy without >> requiring application changes or any changes at the receiver. > Do you mean the application mods to open an SCTP (or whatever) socket > instead of TCP socket vs. mods to TCP stacks and middleboxes? We know for sure that if applications are going to switch from TCP to SCTP to gain multipath benefits, this creates deployment issues. Suppose that a browser wants to load pages from a website. How does the browser know if the server also supports SCTP? For IPv4 vs IPv6 we could solve this using the DNS, but for TCP vs SCTP this is harder. Also, I believe current SCTP needs the application to tell it the addresses for the other side. Then there is the issue that SCTP must of course be implemented on both sides, while the big feature in the one-ended MPTCP is that it can work if it's only implemented in the sender. Then we trade this off against the possible gain of not having to change middleboxes. However, you assume that incompatible middleboxes exist. While I'm sure this will be the case a good percentage of the time, it's not a universal truism. And you assume that this same middlebox would be compatible with SCTP. That is a signficant leap of faith, work for SCTP NAT is still going on in the behave wg. So I think there is a place for multipath TCP in general and the one- ended variety in particular. From aongenae@gmail.com Fri May 8 14:45:07 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 71E6E28C1DF for ; Fri, 8 May 2009 14:45:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bg3gKBvdqcvL for ; Fri, 8 May 2009 14:45:01 -0700 (PDT) Received: from ey-out-2122.google.com (ey-out-2122.google.com [74.125.78.25]) by core3.amsl.com (Postfix) with ESMTP id 531473A6FEC for ; Fri, 8 May 2009 14:45:01 -0700 (PDT) Received: by ey-out-2122.google.com with SMTP id 9so511658eyd.31 for ; Fri, 08 May 2009 14:46:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:subject:from:to:cc :in-reply-to:references:content-type:date:message-id:mime-version :x-mailer; bh=XQg1HwV4qe39wLTjncJ7i7JIRCT9QxV5J5Zft6ckbyM=; b=N0l16jjQOroo/V6jXVAh/W3yi8XzVqYK7TzIajLMziVMeiqKzH0Z+Qtksx04IHszmV bjooo4qhDy0VWciVeM7RHrI3KuVHM9hUh0jFiHeZegq6/jcEG/kQGsfo20PHYChCd/9v RTAzL1A5VNfA581+adoIZsxhcd+FSwZQrqA7g= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=subject:from:to:cc:in-reply-to:references:content-type:date :message-id:mime-version:x-mailer; b=U8HJL/ueoMOhbwGe0zThT3H4yOgyzqRvXDXz+mGOPxcZXjunQPxHF8ypJmU3FRgXtJ yW7dRasfa9U25hsV5RCridojeH9u6fmrx96Z7BMxf6zFHZmankMoB95A3yFRaeFAiAJ5 rQg0Ij8757CtkzkngXfCMFBOWSN7FNEqmiiGQ= Received: by 10.216.18.84 with SMTP id k62mr1925777wek.126.1241819186761; Fri, 08 May 2009 14:46:26 -0700 (PDT) Received: from ?192.168.1.12? ([91.177.75.226]) by mx.google.com with ESMTPS id 24sm2502159eyx.43.2009.05.08.14.46.24 (version=SSLv3 cipher=RC4-MD5); Fri, 08 May 2009 14:46:25 -0700 (PDT) From: Arnaud Ongenae To: Iljitsch van Beijnum In-Reply-To: References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <20090507230452.GC35382@cisco.com> <1241789497.4055.8.camel@nhuynhth-laptop> <001e01c9cfff$bde8f1e0$103947ab@cisco.com> <97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com> <001f01c9d005$0c583110$103947ab@cisco.com> Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-ebvJj8UId4pWKf9wmETv" Date: Fri, 08 May 2009 23:40:43 +0200 Message-Id: <1241818843.26211.67.camel@nhuynhth-laptop> Mime-Version: 1.0 X-Mailer: Evolution 2.26.1 Cc: multipathtcp@ietf.org, Olivier Bonaventure Subject: [multipathtcp] TCP modification vs switching to SCTP X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 May 2009 21:45:07 -0000 --=-ebvJj8UId4pWKf9wmETv Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable A new thread for a question out of the scope of=20 "Draft Multipath TCP (MPTCP) BOF description" TCP modification vs switching to SCTP (if it has already been discussed elsewhere do not hesitate to redirect me) I believe that by introducing this modification to TCP (one-end MTCP) you will give argument to those who do not want to switch to SCTP to stay on TCP. I think that we should create an hybrid transition protocol to produce the need to switch from TCP. This hybrid protocol exist and was presented in : Transparent TCP-to-SCTP translation shim layer=20 from RW Bickhart thesis [1] Why not concentrate multipath effort on SCTP which has the benefit to natively support multihomed host and as Jana said research as been done on the field. There are more than CMT, while doing my master thesis in the field I discover LS-SCTP[2] or cmpSCTP[3] proposing different interesting solutions to multipath issues. When you take a look at this website (that you certainly know): http://www.isoc.org/briefings/017/ you realize that using SCTP is more than simple multihomed capabilities. I, personally, believe that letting SCTP to be alone to handle multipath could boost his integration in the current internet. By defending TCP modification, don't you think that those new protocols could play the same role as NAT for IPv4 address starvation ? Maybe it's an hard comparison but I miss IPv6.=20 On Fri, 2009-05-08 at 22:08 +0200, Iljitsch van Beijnum wrote: > On 8 mei 2009, at 19:47, Preethi Natarajan wrote: >=20 > >> The motivations are better performance and redundancy without > >> requiring application changes or any changes at the receiver. >=20 > > Do you mean the application mods to open an SCTP (or whatever) socket > > instead of TCP socket vs. mods to TCP stacks and middleboxes? >=20 > We know for sure that if applications are going to switch from TCP to =20 > SCTP to gain multipath benefits, this creates deployment issues. =20 > Suppose that a browser wants to load pages from a website. How does =20 > the browser know if the server also supports SCTP? For IPv4 vs IPv6 we =20 > could solve this using the DNS, but for TCP vs SCTP this is harder. absolutely, this is not a trivial issue >=20 > Also, I believe current SCTP needs the application to tell it the =20 > addresses for the other side. Then there is the issue that SCTP must =20 > of course be implemented on both sides, while the big feature in the =20 > one-ended MPTCP is that it can work if it's only implemented in the =20 > sender. >=20 > Then we trade this off against the possible gain of not having to =20 > change middleboxes. However, you assume that incompatible middleboxes =20 > exist. While I'm sure this will be the case a good percentage of the =20 > time, it's not a universal truism. >=20 > And you assume that this same middlebox would be compatible with SCTP. should be compatible :( > =20 > That is a signficant leap of faith, work for SCTP NAT is still going =20 > on in the behave wg. NAT is a poison on the internet (to me) >=20 > So I think there is a place for multipath TCP in general and the one-=20 > ended variety in particular. If you only consider multipath, yes, there is a place, especially if you want the benefit as soon as possible, because this new protocol work without modifying one end, you may implement it for yourself and already use it. regards, Arnaud Ongenae UCL (Belgium) student [1] http://www.cis.udel.edu/~amer/PEL/poc/pdf/BickhartMSthesis.pdf [2] Ahmed Abd El Al, Tarek Saadawi, and Myung Lee. Ls-sctp: a bandwidth aggregation technique for stream control transmission protocol. Computer Communications, 27(10):1012=E2=80=931024, 2004. Protocol Engineering for Wired and Wireless Networks. [3] J. Liao, J. Wang, and X. Zhu. A multi-path mechanism for reliable voip transmission over wireless networks. Comput. Netw., 52(13):2450=E2=80=932460, 2008. --=-ebvJj8UId4pWKf9wmETv Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEABECAAYFAkoEptEACgkQHpJ+LMVPWRK89gCgsxMaFc7bBkw1vF/t623y18cp KdoAoJS5oLn15Nn4BmSGVHVizsG+ei3N =6Svv -----END PGP SIGNATURE----- --=-ebvJj8UId4pWKf9wmETv-- From preethi.cis@gmail.com Fri May 8 15:01:35 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id BDFEE3A67FD for ; Fri, 8 May 2009 15:01:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id W-MqBGa5eMsR for ; Fri, 8 May 2009 15:01:34 -0700 (PDT) Received: from rv-out-0506.google.com (rv-out-0506.google.com [209.85.198.225]) by core3.amsl.com (Postfix) with ESMTP id BF51B3A6C43 for ; Fri, 8 May 2009 15:01:03 -0700 (PDT) Received: by rv-out-0506.google.com with SMTP id g37so1223352rvb.49 for ; Fri, 08 May 2009 15:02:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:from:to:cc:references:subject :date:message-id:mime-version:content-type:content-transfer-encoding :x-mailer:in-reply-to:thread-index:x-mimeole; bh=TsdPbXycmGOltTUwhD0lpP9qsYVNQIU6vqT6cZeZJzo=; b=NNJs0+GfrSuz5MlgOwrE6dYLPBOm2JO9uA2/LR5Stroo7h49ZVR5cN3E7eCww7hl6p B0vBOFlDL0ZHEmzofjmZLE40mFoGYk7std4GK9mxLz+iNXtdDfk542VhRscW6hXuCH7E cGpltT25DB0UUcdQYArQ3Wvq3wqFjRzXLfEGY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:cc:references:subject:date:message-id:mime-version :content-type:content-transfer-encoding:x-mailer:in-reply-to :thread-index:x-mimeole; b=RhePsVo2SFXxphC+BLXMh7byT5eUU4ny0n5EdnMx150MgbOdCWeZdy8I6hSCFarBcG TqlHHW+guBo96KznDx4ndx1OgdUTdBOafO+3cjGxFWl6mcJmgKGSkStFAJL7oDEPdZRV cA/9DpgEt7Ccs12i0mBJivmdWS8rQJVtSzUWY= Received: by 10.142.231.7 with SMTP id d7mr1674762wfh.124.1241820150944; Fri, 08 May 2009 15:02:30 -0700 (PDT) Received: from prenatarwxp01 (dhcp-171-71-57-16.cisco.com [171.71.57.16]) by mx.google.com with ESMTPS id 22sm235673wfd.39.2009.05.08.15.02.29 (version=SSLv3 cipher=RC4-MD5); Fri, 08 May 2009 15:02:30 -0700 (PDT) From: "Preethi Natarajan" To: "'Iljitsch van Beijnum'" References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es><4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><20090507230452.GC35382@cisco.com><1241789497.4055.8.camel@nhuynhth-laptop> <001e01c9cfff$bde8f1e0$103947ab@cisco.com> <97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com> <001f01c9d005$0c583110$103947ab@cisco.com> Date: Fri, 8 May 2009 15:02:29 -0700 Message-ID: <000001c9d028$aea92640$103947ab@cisco.com> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 11 In-Reply-To: Thread-Index: AcnQGNW1yZzoic1mQjCTaLFwW8Q7GwABVXKw X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3350 Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 May 2009 22:01:35 -0000 > -----Original Message----- > From: Iljitsch van Beijnum [mailto:iljitsch@muada.com] > Sent: Friday, May 08, 2009 1:09 PM > To: Preethi Natarajan > Cc: 'Arnaud Ongenae'; multipathtcp@ietf.org > Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF > description > > Suppose that a browser wants to load pages from a website. > How does the browser know if the server also supports SCTP? > For IPv4 vs IPv6 we could solve this using the DNS, but for > TCP vs SCTP this is harder. > DNS has been proposed to identify transport protocols in the past, for ex. see RFC 3263. From what I hear, SRV records have worked quite well for SIP. > > Then we trade this off against the possible gain of not > having to change middleboxes. However, you assume that > incompatible middleboxes exist. While I'm sure this will be > the case a good percentage of the time, it's not a universal truism. When NATs on different paths have to talk to each other to do the same translation, I would assume that this requires changes to NAT implementations... > > So I think there is a place for multipath TCP in general and > the one- ended variety in particular. I am sure it is. I am just wondering why the draft should be tied to TCP, when (i) the one-ended multipath transmission could be adapted to any transport, and (ii) some of the issues listed in draft-van-beijnum-1e-mp-tcp-00 has been researched in-depth and solved for another transport. The motivation to consider just TCP cannot be NAT traversal, since, MPTCP itself might not work with the existing NAT implementations? Preethi From janardhan.iyengar@fandm.edu Fri May 8 20:57:43 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 4E5963A6A50 for ; Fri, 8 May 2009 20:57:43 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jKRPCU1YleVk for ; Fri, 8 May 2009 20:57:42 -0700 (PDT) Received: from zimfe2.fandm.edu (zimfe2.fandm.edu [155.68.1.75]) by core3.amsl.com (Postfix) with ESMTP id 2DCFE3A6940 for ; Fri, 8 May 2009 20:57:41 -0700 (PDT) Received: from surutti.fandm.edu (pool-71-173-241-83.hrbgpa.east.verizon.net [71.173.241.83]) by zimfe2.fandm.edu (Postfix) with ESMTP id A4CED1168035; Fri, 8 May 2009 23:59:08 -0400 (EDT) Message-ID: <4A04FF8B.2060800@fandm.edu> Date: Fri, 08 May 2009 23:59:07 -0400 From: Janardhan Iyengar Organization: Franklin & Marshall College User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Iljitsch van Beijnum References: <4A01D161.7060108@it.uc3m.es> <4A02F0B7.6080202@fandm.edu> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: nasif ekiz , multipathtcp@ietf.org, "Paul D. Amer" , Randall Stewart , Preethi Natarajan , Ertugrul YILMAZ Subject: Re: [multipathtcp] Fwd: I-D Action:draft-van-beijnum-1e-mp-tcp-00.txt]] X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: janardhan.iyengar@fandm.edu List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 May 2009 03:57:43 -0000 Hi Illjitsch, I will also pull out other related work and add it into the wiki. > Right. However, note that RFC 2960 says that only one path is used to > send traffic, the other path is only used for retransmissions. (I'll > have to check with RFC 4960.) That is correct. >> + You list 4 challenges and questions, and say that you address the >> first two (scheduling, reordering). We have looked extensively at >> those two and also the third one (receiver buffer constraints), and >> our results are documented. > > Note that for SCTP the send and receive buffer issues are different when > there are multiple streams. Can you say some more about which issues you think may be different? Any solution for SCTP send/receive buffer issues must handle the single-stream case, which is essentially collapses, as far as buffer issues are concerned, down to the issues of an ordered TCP stream. >> I like to think that we did quite a bit of work in this area already, >> and while there are unresolved issues still around, I also like to >> think that we got some important things down :-) It would be very >> disheartening to see work that is pretty much exactly the same not >> using our previous work. > > I agree. But SCTP results don't always translate into TCP results. And > there's still plenty of work to do to apply the research to > implementations. As far as implementation is concerned, CMT has been part of FreeBSD since FreeBSD 7.0 (sysctls net.inet.sctp.cmt*). And if you count simulators, CMT is part of the ns2 simulator (ns/sctp/). As far as TCP/SCTP differences are concerned: Pulling words from the BoF announcement, the first proposed work item of multipath-TCP is: "A set of architectural guidelines for congestion dependent multipath transport protocols. Since different transport protocols can potentially benefit from this approach, this document describes the motivations and the general approach that should be followed to enable congestion dependent multipath transport." Surely SCTP falls into the category described here. Architecturally, I expect that there will be a significant amount of overlap; it doesn't matter if it is TCP or SCTP. Our design is one point in the design space of multipath transport. I fully agree that the devil in the details will have to be dealt with, and there will be many differences there between TCP and SCTP; we don't have to go very far, SACK differences are significant enough! Having said that, I expect that *at this stage*, when we are probably still having architectural discussions, there will be more similarities than differences. - jana -- Janardhan Iyengar Assistant Professor, Computer Science Franklin & Marshall College http://www.fandm.edu/jiyengar From janardhan.iyengar@fandm.edu Fri May 8 21:01:30 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id BCFD93A67F9 for ; Fri, 8 May 2009 21:01:30 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id L4p5RkbiRhR8 for ; Fri, 8 May 2009 21:01:29 -0700 (PDT) Received: from zimfe2.fandm.edu (zimfe2.fandm.edu [155.68.1.75]) by core3.amsl.com (Postfix) with ESMTP id C1AC53A6B96 for ; Fri, 8 May 2009 21:01:29 -0700 (PDT) Received: from surutti.fandm.edu (pool-71-173-241-83.hrbgpa.east.verizon.net [71.173.241.83]) by zimfe2.fandm.edu (Postfix) with ESMTP id 5EC921168035; Sat, 9 May 2009 00:02:57 -0400 (EDT) Message-ID: <4A05006F.2090305@fandm.edu> Date: Sat, 09 May 2009 00:02:55 -0400 From: Janardhan Iyengar Organization: Franklin & Marshall College User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Scott Brim References: <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> In-Reply-To: <4A031B0F.4030606@employees.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: janardhan.iyengar@fandm.edu List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 May 2009 04:01:30 -0000 Scott, > (1) Up in the introductory material: As you've heard me argue, in > addition to the obvious benefits of multipath TCP, I believe it also > indirectly provides the best approach to path failure handling (better > than probing, in particular). Fully agree. We have results to demonstrate this! - jana -- Janardhan Iyengar Assistant Professor, Computer Science Franklin & Marshall College http://www.fandm.edu/jiyengar From janardhan.iyengar@fandm.edu Fri May 8 22:55:30 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 400093A6A7F for ; Fri, 8 May 2009 22:55:30 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Q3k5pPrq8ffF for ; Fri, 8 May 2009 22:55:29 -0700 (PDT) Received: from zimfe2.fandm.edu (zimfe2.fandm.edu [155.68.1.75]) by core3.amsl.com (Postfix) with ESMTP id 139BA3A69F2 for ; Fri, 8 May 2009 22:55:28 -0700 (PDT) Received: from surutti.fandm.edu (pool-71-173-241-83.hrbgpa.east.verizon.net [71.173.241.83]) by zimfe2.fandm.edu (Postfix) with ESMTP id E5FB21168036; Sat, 9 May 2009 01:56:56 -0400 (EDT) Message-ID: <4A051B25.4040702@fandm.edu> Date: Sat, 09 May 2009 01:56:53 -0400 From: Janardhan Iyengar Organization: Franklin & Marshall College User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Iljitsch van Beijnum References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es><4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><20090507230452.GC35382@cisco.com><1241789497.4055.8.camel@nhuynhth-laptop> <001e01c9cfff$bde8f1e0$103947ab@cisco.com> <97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com> <001f01c9d005$0c583110$103947ab@cisco.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: janardhan.iyengar@fandm.edu List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 May 2009 05:55:30 -0000 Hi Iljitsch, > We know for sure that if applications are going to switch from TCP to > SCTP to gain multipath benefits, this creates deployment issues. Suppose > that a browser wants to load pages from a website. How does the browser > know if the server also supports SCTP? For IPv4 vs IPv6 we could solve > this using the DNS, but for TCP vs SCTP this is harder. Simultaneous TCP and SCTP connects. Yes, it is a hack, but it is a solution. > Also, I believe current SCTP needs the application to tell it the > addresses for the other side. SCTP peers exchange information about all addresses in the handshake; the application *does not* tell the initiator all the addresses for the other side. The API is exactly the same as with TCP/UDP; in fact, Ryan Bickhart's thesis (referenced in an earlier email by Arnaud) exploited exactly this idea to give multihoming and fault tolerance benefits to any application by transparently mapping application-level TCP socket calls to SCTP function calls in-kernel. > Then there is the issue that SCTP must of > course be implemented on both sides, while the big feature in the > one-ended MPTCP is that it can work if it's only implemented in the sender. But to be useful, one-ended MPTCP will need to be understood by routers/middleboxes. > Then we trade this off against the possible gain of not having to change > middleboxes. However, you assume that incompatible middleboxes exist. > While I'm sure this will be the case a good percentage of the time, it's > not a universal truism. > > And you assume that this same middlebox would be compatible with SCTP. > That is a signficant leap of faith, work for SCTP NAT is still going on > in the behave wg. The ongoing work, if I am right, is for supporting multihoming, not basic support (which is trivial). And as I see it, that same work will become the template for multipathTCP as well, when you move to the "two-ended" mulipath case. - jana -- Janardhan Iyengar Assistant Professor, Computer Science Franklin & Marshall College http://www.fandm.edu/jiyengar From janardhan.iyengar@fandm.edu Fri May 8 23:08:50 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 9784A3A6D2E for ; Fri, 8 May 2009 23:08:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3J1fSiiFsMEQ for ; Fri, 8 May 2009 23:08:49 -0700 (PDT) Received: from zimfe2.fandm.edu (zimfe2.fandm.edu [155.68.1.75]) by core3.amsl.com (Postfix) with ESMTP id 668AA3A6CED for ; Fri, 8 May 2009 23:08:49 -0700 (PDT) Received: from surutti.fandm.edu (pool-71-173-241-83.hrbgpa.east.verizon.net [71.173.241.83]) by zimfe2.fandm.edu (Postfix) with ESMTP id 227B21168033; Sat, 9 May 2009 02:10:16 -0400 (EDT) Message-ID: <4A051E47.7090400@fandm.edu> Date: Sat, 09 May 2009 02:10:15 -0400 From: Janardhan Iyengar Organization: Franklin & Marshall College User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Preethi Natarajan References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es><4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><20090507230452.GC35382@cisco.com><1241789497.4055.8.camel@nhuynhth-laptop> <001e01c9cfff$bde8f1e0$103947ab@cisco.com> <97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com> <001f01c9d005$0c583110$103947ab@cisco.com> <000001c9d028$aea92640$103947ab@cisco.com> In-Reply-To: <000001c9d028$aea92640$103947ab@cisco.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: janardhan.iyengar@fandm.edu List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 May 2009 06:08:50 -0000 A couple of thoughts on the draft/BoF: 1/ "Two-ended" multipath, where both endpoints are aware: There are several architectural and design considerations that will be common to TCP, SCTP, DCCP, SST etc., and as I see it, there is no reason why this discussion should not include all transports. My proposal: scope the BoF more broadly. Interested parties can then speak up about multipath for different transports and different points in the design space. There will be shared goals, and lessons to be learned from one another. Perhaps there can be a shared design considerations document, and then separate documents per transport. Perhaps two per transport. 2/ "One-ended" multipath seems very unclear to me... there is benefit in employing this "path" option in *any* transport. This should be a transport-neutral option, below the traditional transport. (Perhaps in the shim6 shim layer? I am not sure yet...) I am not convinced that there is a point in having this as a *TCP* option. I also do not understand how this is on the deployment path to two-ended TCP. They seem like two different beasts: one of them needs router support to get going, the other one needs endpoint/NAT support to get going. Don't get me wrong: I love the idea of having signaling between endpoints and routers to enable more control over multipath scheduling at the transport, but that seems like a different trajectory than the "two-ended" multipath. Others have said this, and I agree that "one-sided" multipath with router support does not seem to be scoped right... it seems to belong elsewhere, under the transport and independent of it. Or am I missing something? - jana -- Janardhan Iyengar Assistant Professor, Computer Science Franklin & Marshall College http://www.fandm.edu/jiyengar From alan.ford@roke.co.uk Sat May 9 07:26:39 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id B5C553A69F6 for ; Sat, 9 May 2009 07:26:39 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.599 X-Spam-Level: X-Spam-Status: No, score=-3.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5foK70uoO87t for ; Sat, 9 May 2009 07:26:38 -0700 (PDT) Received: from rsys001x.roke.co.uk (rsys001x.roke.co.uk [193.118.201.108]) by core3.amsl.com (Postfix) with ESMTP id 8468A3A6D22 for ; Sat, 9 May 2009 07:26:37 -0700 (PDT) Received: from rsys005a.comm.ad.roke.co.uk ([193.118.193.85]) by rsys001x.roke.co.uk (8.13.1/8.13.1) with ESMTP id n49ES0RK009147 for ; Sat, 9 May 2009 15:28:00 +0100 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----_=_NextPart_001_01C9D0B2.5A511706" Date: Sat, 9 May 2009 15:27:59 +0100 Message-ID: <2181C5F19DD0254692452BFF3EAF1D6804E606B5@rsys005a.comm.ad.roke.co.uk> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: I-D ACTION:draft-ford-mptcp-multiaddressed-00.txt thread-index: AcnQslla0Mr85BCCTV+wqdK1rvVexw== From: "Ford, Alan" To: X-roke-co-uk-MailScanner: Found to be clean X-roke-co-uk-MailScanner-SpamCheck: X-roke-co-uk-MailScanner-From: alan.ford@roke.co.uk Subject: [multipathtcp] FW: I-D ACTION:draft-ford-mptcp-multiaddressed-00.txt X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 May 2009 14:26:39 -0000 This is a multi-part message in MIME format. ------_=_NextPart_001_01C9D0B2.5A511706 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi all, Please find below information about another Multipath TCP draft that has just been published. This draft covers a different part of the multipath space to Iljitsch's draft. This is a work-in-progress proposal for a "2-ended" multipath TCP, where each endpoint is multihomed, multiaddressed, and speaks the multipath TCP protocol. This document currently concentrates on protocol features for enabling this operation; work on congestion coupling/receive buffers/etc is ongoing and will follow in later versions. Regards, Alan -----Original Message----- From: i-d-announce-bounces@ietf.org [mailto:i-d-announce-bounces@ietf.org] On Behalf Of Internet-Drafts@ietf.org Sent: 08 May 2009 23:45 To: i-d-announce@ietf.org Subject: I-D ACTION:draft-ford-mptcp-multiaddressed-00.txt A New Internet-Draft is available from the on-line Internet-Drafts=20 directories. Title : TCP Extensions for Multipath Operation with Multiple Addresses Author(s) : A. Ford, C. Raiciu, M. Handley, S. Barre Filename : draft-ford-mptcp-multiaddressed-00.txt Pages : 30 Date : 2009-5-7 =09 Often endpoints are connected by multiple paths, but the nature of TCP/IP restricts communications to a single path per socket. Resource usage within the network would be more efficient were these multiple paths able to be used concurrently. This should enhance user experience through higher throughput and improved resilience to network failure. This document presents extensions to TCP in order to transparently provide this multi-path functionality at the transport layer, if at least one endpoint is multi-addressed. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ford-mptcp-multiaddressed-00.t xt Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ Below is the data which will enable a MIME compliant mail reader implementation to automatically retrieve the ASCII version of the Internet-Draft. ------_=_NextPart_001_01C9D0B2.5A511706 Content-Type: application/octet-stream; name="draft-ford-mptcp-multiaddressed-00.URL" Content-Transfer-Encoding: base64 Content-Description: draft-ford-mptcp-multiaddressed-00.URL Content-Disposition: attachment; filename="draft-ford-mptcp-multiaddressed-00.URL" W0ludGVybmV0U2hvcnRjdXRdDQpVUkw9ZnRwOi8vZnRwLmlldGYub3JnL2ludGVybmV0LWRyYWZ0 cy9kcmFmdC1mb3JkLW1wdGNwLW11bHRpYWRkcmVzc2VkLTAwLnR4dA0K ------_=_NextPart_001_01C9D0B2.5A511706-- From touch@ISI.EDU Sat May 9 07:47:07 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 985C43A6A0C for ; Sat, 9 May 2009 07:47:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.807 X-Spam-Level: X-Spam-Status: No, score=0.807 tagged_above=-999 required=5 tests=[BAYES_40=-0.185, DATE_IN_PAST_12_24=0.992] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id elPU2+HhGsL3 for ; Sat, 9 May 2009 07:47:06 -0700 (PDT) Received: from vapor.isi.edu (vapor.isi.edu [128.9.64.64]) by core3.amsl.com (Postfix) with ESMTP id 941093A6914 for ; Sat, 9 May 2009 07:47:06 -0700 (PDT) Received: from [192.168.1.40] (pool-72-81-98-82.phlapa.east.verizon.net [72.81.98.82]) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id n49EmKKf012982; Sat, 9 May 2009 07:48:22 -0700 (PDT) Message-ID: <4A046242.4040104@isi.edu> Date: Fri, 08 May 2009 09:48:02 -0700 From: Joe Touch User-Agent: Thunderbird 2.0.0.21 (Windows/20090302) MIME-Version: 1.0 To: marcelo bagnulo braun References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <20090507230452.GC35382@cisco.com> <4A036A29.7020005@isi.edu> <4A042F28.5050107@it.uc3m.es> In-Reply-To: <4A042F28.5050107@it.uc3m.es> X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 May 2009 14:47:07 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 > Joe Touch escribió: .. >> IMO, this goes too far. I would expect it would be useful enough to talk >> about "multi-address TCP" and avoid talking about "paths" when they're >> not part of the architecture, nor are they knowable without doing things >> like pathchar or traceroute. > > But there are many situations and probably very interesting ones, > where the node has multipaht paths and does not have multiple addresses. > For instnace, think about a server located in a big multihomed site > using PI addresses. This server is likely to benefit from multipath TCP > but it only has a single address. Then you have different problems, and you need to separate them: | single endpoint | multi endpoint -------------+-------------------+---------------- single path | sp/se | sp/me -------------+-------------------+---------------- multi path | mp/se | mp/me -------------+-------------------+---------------- Existing TCP assumes sp/se. If you solve mp/me, you solve sp/me as a degenerate case. So basically there are two distinct problems: 1) making TCP work when there are multiple paths which has *nothing* to do with whether there are are multiple endpoints or not e.g., a single TCP connection can experience two different paths given multipath routing 2) making TCP work with multiple endpoints which has *nothing* to do with whether there are multiple paths, since they different endpoints can easily experience the same path properties - including the same bottleneck. You can't assume that two different IP addresses (or other TCP endpoint differentiators) result in paths that do not share properties or bottlenecks. this *does* include striping, i.e., concurrent use of alternate endpoints to increase BW; that is NOT the case for single-address. note that 'endpoint' could mean IP address or port, or anything else that could cause path properties to vary - keep in mind that policy routing peeks at the entire IP and often TCP headers. Note that when I say "multiple paths" I mean paths with different characteristics, e.g., MTU, RTT, BW, loss. IMO, multiendpoint is a potentially important problem that may be ready to solve via IETF work at this time. However, you need to be careful to never assume that having multiple endpoints means distinct paths - if they're not distinct in bottleneck, then you'll end up making TCP fight with itself. If you're going to solve multipath, then you need to solve it in the single endpoint case first, IMO. If you can't solve it there, there's no point in conflating the issue with multiendpoint. > In addition, even if you do have multiple addresses, you still need > some support from the routing infrastrcuture. IMO, you do not need support from the routing infrastructure at all. TCP probes to detect path properties without interacting with routing right now. You need to separate that out when you get distinct sets of values. All you need to know is that you are getting a correlation between segments being sent and varying path properties. > When a node A with multiple addresses sends packets, it will include > different source addresses, but all packets will carry the same > destiantion address. That's ONE way to do this. There are others. > How can the nade A influence the egress path? By varying the source address, source port, dest address, dest port, or just about anything else in the IP or TCP header. Joe -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkoEYkIACgkQE5f5cImnZrtEzwCgqTmskXCmmAKi/SJHpw5A5ZRj eukAnRkO0vPBnEwDf1wzJ1VG1wQs46dz =1KaA -----END PGP SIGNATURE----- From iljitsch@muada.com Sat May 9 08:01:41 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 8B21B28C0F8 for ; Sat, 9 May 2009 08:01:41 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.962 X-Spam-Level: X-Spam-Status: No, score=-5.962 tagged_above=-999 required=5 tests=[AWL=0.637, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id H1L0VUUjquRA for ; Sat, 9 May 2009 08:01:40 -0700 (PDT) Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by core3.amsl.com (Postfix) with ESMTP id B08E03A6E71 for ; Sat, 9 May 2009 08:00:32 -0700 (PDT) Received: from [192.168.2.4] (static-167-138-7-89.ipcom.comunitel.net [89.7.138.167] (may be forged)) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n49F0A6b027163 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Sat, 9 May 2009 17:00:11 +0200 (CEST) (envelope-from iljitsch@muada.com) Message-Id: From: Iljitsch van Beijnum To: Michio HONDA In-Reply-To: <4A03F8D6.7080709@sfc.wide.ad.jp> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Sat, 9 May 2009 17:00:13 +0200 References: <4A01D161.7060108@it.uc3m.es> <4A03F8D6.7080709@sfc.wide.ad.jp> X-Mailer: Apple Mail (2.930.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Fwd: I-D Action:draft-van-beijnum-1e-mp-tcp-00.txt]] X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 May 2009 15:01:41 -0000 Hi Michio, Thanks for your comments. On 8 mei 2009, at 11:18, Michio HONDA wrote: >> >Until more analysis has been performed and/or there is more >> experience with multipath TCP, a multipath TCP implementation >> SHOULD limit itself to using no more than 3 subflows concurrently. > How the decision to allow 3 subflows is made? Well, it needs to be low enough that in the case where a multipath TCP session is sharing a bottleneck with a regular TCP session where all the multipath subflows flow through that same bottleneck, the impact to the regular TCP session is limited to something reasonable. On the other hand, it doesn't make much sense to have a multipath TCP with only one subflow. And two subflows could easily create a number of special cases, so for testing purposes three would seem like a reasonable number. But like I said, this needs more analysis and experimentation. > For example, HTTP spec. (RFC 2116 Sec.8.1.4) describes the > simultaneous connections should not use more than 2 connections. > The reason contains not only receiver workload, but also congestion > control. > Hence I think you should mention how multipath TCP (using 3 > subflows) deals with such criterions. We are working on a paper about this, others are working on this as well. > In addition, it would be good to allow more subflows based on shared > bottleneck detection like mTCP(https://www.usenix.org/events/usenix04/tech/general/full_papers/zhang/zhang_html/ > ) From that paper: "Intuitively, a match means the two subflows enter fast retransmit around the same time. This also means packets from the two flows are dropped at about the same time, so it is likely they share the same congested link." However, this is way too simple. Congestion can present itself in very many different ways. For instance, there may not even any packet loss at all, just an increase in RTT. If the congestion is because of a fixed high utilization or if a queuing strategy like RED is in use, or if the bandwidth of the congested link is much higher than the send rate, losses will very likely not be correlated. Only when there is a burst that is enough to fill up the buffer and long enough to span the time between the arrivals of two packets from two subflows will there be a loss on two subflows at the same time. However, if shared bottlenecks could be detected that would make the fairness issues much less problematic, so maybe this warrants more investigation. Iljitsch From salvatore.loreto@ericsson.com Sat May 9 09:37:12 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 6519A3A6AD4 for ; Sat, 9 May 2009 09:37:12 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.386 X-Spam-Level: X-Spam-Status: No, score=-6.386 tagged_above=-999 required=5 tests=[AWL=-0.137, BAYES_00=-2.599, HELO_EQ_SE=0.35, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tTyJXk+a4hPl for ; Sat, 9 May 2009 09:37:11 -0700 (PDT) Received: from mailgw4.ericsson.se (mailgw4.ericsson.se [193.180.251.62]) by core3.amsl.com (Postfix) with ESMTP id 19A7D3A689B for ; Sat, 9 May 2009 09:37:10 -0700 (PDT) X-AuditID: c1b4fb3e-b7bc1ae00000675c-e7-4a05b18e34bc Received: from esealmw128.eemea.ericsson.se (Unknown_Domain [153.88.253.124]) by mailgw4.ericsson.se (Symantec Mail Security) with SMTP id E9.3B.26460.E81B50A4; Sat, 9 May 2009 18:38:38 +0200 (CEST) Received: from esealmw126.eemea.ericsson.se ([153.88.254.170]) by esealmw128.eemea.ericsson.se with Microsoft SMTPSVC(6.0.3790.1830); Sat, 9 May 2009 18:38:38 +0200 Received: from mail.lmf.ericsson.se ([131.160.11.50]) by esealmw126.eemea.ericsson.se with Microsoft SMTPSVC(6.0.3790.1830); Sat, 9 May 2009 18:38:37 +0200 Received: from nomadiclab.lmf.ericsson.se (nomadiclab.lmf.ericsson.se [131.160.33.3]) by mail.lmf.ericsson.se (Postfix) with ESMTP id D0D5B245F; Sat, 9 May 2009 19:38:37 +0300 (EEST) Received: from nomadiclab.lmf.ericsson.se (localhost [127.0.0.1]) by nomadiclab.lmf.ericsson.se (Postfix) with ESMTP id 9CCA5219F7; Sat, 9 May 2009 19:38:37 +0300 (EEST) Received: from localhost.localdomain (localhost [127.0.0.1]) by nomadiclab.lmf.ericsson.se (Postfix) with ESMTP id 3ED19219CF; Sat, 9 May 2009 19:38:37 +0300 (EEST) Message-ID: <4A05B18C.5030205@ericsson.com> Date: Sat, 09 May 2009 19:38:36 +0300 From: Salvatore Loreto User-Agent: Thunderbird 2.0.0.21 (X11/20090320) MIME-Version: 1.0 To: Preethi Natarajan References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es><4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><20090507230452.GC35382@cisco.com><1241789497.4055.8.camel@nhuynhth-laptop> <001e01c9cfff$bde8f1e0$103947ab@cisco.com> <97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com> <001f01c9d005$0c583110$103947ab@cisco.com> <000001c9d028$aea92640$103947ab@cisco.com> In-Reply-To: <000001c9d028$aea92640$103947ab@cisco.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV using ClamSMTP X-OriginalArrivalTime: 09 May 2009 16:38:38.0084 (UTC) FILETIME=[9A7D4C40:01C9D0C4] X-Brightmail-Tracker: AAAAAA== Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 May 2009 16:37:12 -0000 Preethi Natarajan wrote: > > > >> -----Original Message----- >> From: Iljitsch van Beijnum [mailto:iljitsch@muada.com] >> Sent: Friday, May 08, 2009 1:09 PM >> To: Preethi Natarajan >> Cc: 'Arnaud Ongenae'; multipathtcp@ietf.org >> Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF >> description >> >> Suppose that a browser wants to load pages from a website. >> How does the browser know if the server also supports SCTP? >> For IPv4 vs IPv6 we could solve this using the DNS, but for >> TCP vs SCTP this is harder. >> >> > > DNS has been proposed to identify transport protocols in the past, for ex. > see RFC 3263. From what I hear, SRV records have worked quite well for SIP. > actually there is already an interesting proposal, take a look at those drafts: http://www.ietf.org/internet-drafts/draft-wood-tae-specifying-uri-transports-05.txt http://www.ietf.org/internet-drafts/draft-jennings-http-srv-05.txt /Sal From baford@mpi-sws.org Sat May 9 10:03:08 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 51D883A6987 for ; Sat, 9 May 2009 10:03:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.249 X-Spam-Level: X-Spam-Status: No, score=-6.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xWN4IEbAJQ9D for ; Sat, 9 May 2009 10:03:04 -0700 (PDT) Received: from apollo.mpi-sb.mpg.de (apollo.mpi-sb.mpg.de [139.19.1.50]) by core3.amsl.com (Postfix) with ESMTP id AC0523A6A72 for ; Sat, 9 May 2009 10:03:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mpi-sb.mpg.de; s=mail200803; h=Cc:Message-Id:From:To: In-Reply-To:Content-Type:Content-Transfer-Encoding:Mime-Version: Subject:Date:References; bh=nQxaAqm0rn+hBeK2+SSm2w217cZXxWY6YNZg i9N9RWU=; b=srneabtOyhD4d33vhMKkX+8gjFaPRq8/DedLEb2UVzqsrOMKzXuw qedK4Zcc+JknubysMgT55bT/99cMW7JgqgHlYYRgjaUOKzNxEdH76eSuUkH4m2Ob 8BKF51jKeEGg6F4x1Qqgolz9WDMLeWvYwucHnXq0wl4n3Bc8eO6oTqM= Received: from newzak.mpi-sb.mpg.de ([139.19.1.28]:54553) by apollo.mpi-sb.mpg.de (envelope-from ) with esmtp (Exim 4.69) id 1M2pyM-0005uD-Ke; Sat, 09 May 2009 19:04:30 +0200 Received: from adsl-89-217-69-158.adslplus.ch ([89.217.69.158]:61274 helo=[192.168.1.100]) by newzak.mpi-sb.mpg.de with esmtpsa (TLS-1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.63) (envelope-from ) id 1M2pyM-00027T-1I; Sat, 09 May 2009 19:04:26 +0200 Message-Id: From: Bryan Ford To: Salvatore Loreto In-Reply-To: <4A05B18C.5030205@ericsson.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Sat, 9 May 2009 19:04:25 +0200 References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es><4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><20090507230452.GC35382@cisco.com><1241789497.4055.8.camel@nhuynhth-laptop> <001e01c9cfff$bde8f1e0$103947ab@cisco.com> <97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com> <001f01c9d005$0c583110$103947ab@cisco.com> <000001c9d028$aea92640$103947ab@cisco.com> <4A05B18C.5030205@ericsson.com> X-Mailer: Apple Mail (2.930.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 May 2009 17:03:08 -0000 On May 9, 2009, at 6:38 PM, Salvatore Loreto wrote: > Preethi Natarajan wrote: >>> -----Original Message----- >>> From: Iljitsch van Beijnum [mailto:iljitsch@muada.com] Sent: >>> Friday, May 08, 2009 1:09 PM >>> To: Preethi Natarajan >>> Cc: 'Arnaud Ongenae'; multipathtcp@ietf.org >>> Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF >>> description >>> >>> Suppose that a browser wants to load pages from a website. How >>> does the browser know if the server also supports SCTP? For IPv4 >>> vs IPv6 we could solve this using the DNS, but for TCP vs SCTP >>> this is harder. >>> >>> >> >> DNS has been proposed to identify transport protocols in the past, >> for ex. >> see RFC 3263. From what I hear, SRV records have worked quite well >> for SIP. >> > actually there is already an interesting proposal, take a look at > those drafts: > http://www.ietf.org/internet-drafts/draft-wood-tae-specifying-uri-transports-05.txt > http://www.ietf.org/internet-drafts/draft-jennings-http-srv-05.txt Proposals like these are trying to take URIs in exactly the wrong direction, IMO, by trying to encode _more_ fragile information about "how exactly to access some object", rather than less as they should (i.e., what URNs try to do). When someone gives you a URI or posts it on their web page, what you generally want is for your web (or SIP or bittorrent...) client to use that as the minimal information necessary to find and contact the desired service or object, and then negotiate online what specific suite of communication protocols and optional protocol features should be used for continuing communication. That way the URI works when either the client or the server don't support some alternative transport or other protocol variation (i.e., they can fall back on a more basic protocol), in addition to when they do. When you encode a requirement to use a particular transport or protocol suite into the URI, the URI won't work at all unless both parties fully understand the indicated protocol suite. There may be legitimate corner case uses of such mechanisms to encode protocol suites and options and such into URIs, but I haven't yet seen any convincing ones. As far as I can tell the right and only solution to the general problem is to come up with a better way to negotiate protocol suites and features dynamically. Bryan > > /Sal > _______________________________________________ > multipathtcp mailing list > multipathtcp@ietf.org > https://www.ietf.org/mailman/listinfo/multipathtcp From baford@mpi-sws.org Sat May 9 10:38:24 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id D8C263A6923 for ; Sat, 9 May 2009 10:38:24 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.249 X-Spam-Level: X-Spam-Status: No, score=-6.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Cd1r3IuFn4jh for ; Sat, 9 May 2009 10:38:23 -0700 (PDT) Received: from apollo.mpi-sb.mpg.de (swsao0807.mpi-sb.mpg.de [139.19.1.50]) by core3.amsl.com (Postfix) with ESMTP id E338B3A6AA2 for ; Sat, 9 May 2009 10:38:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mpi-sb.mpg.de; s=mail200803; h=Cc:Message-Id:From:To: In-Reply-To:Content-Type:Content-Transfer-Encoding:Mime-Version: Subject:Date:References; bh=t1nZp/kD9gGacX++OLoATY9VKglStdqofPeK Vv7nCiw=; b=mXaL3bjifhcHFG8P4jFGXJilk3kNtzyCJRLbIc/rqZaHAVvr4rWg Je35y2aL5ts5j4y+vqqVcj5SKNnlcsphcCxwv+uvK2A8+IlFfL5DYFJCiThakoOq uWKt131A/fNtCaYJZx7LtdS7r8L7SuJbupP9NYw1yHgTDDf/IHcClts= Received: from newzak.mpi-sb.mpg.de ([139.19.1.28]:41161) by apollo.mpi-sb.mpg.de (envelope-from ) with esmtp (Exim 4.69) id 1M2qWY-0005t5-Fu; Sat, 09 May 2009 19:39:49 +0200 Received: from adsl-89-217-69-158.adslplus.ch ([89.217.69.158]:61713 helo=[192.168.1.100]) by newzak.mpi-sb.mpg.de with esmtpsa (TLS-1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.63) (envelope-from ) id 1M2qWX-0002JZ-Nj; Sat, 09 May 2009 19:39:46 +0200 Message-Id: From: Bryan Ford To: Arnaud Ongenae In-Reply-To: <1241818843.26211.67.camel@nhuynhth-laptop> Content-Type: text/plain; charset=WINDOWS-1252; format=flowed; delsp=yes Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Apple Message framework v930.3) Date: Sat, 9 May 2009 19:39:44 +0200 References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <20090507230452.GC35382@cisco.com> <1241789497.4055.8.camel@nhuynhth-laptop> <001e01c9cfff$bde8f1e0$103947ab@cisco.com> <97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com> <001f01c9d005$0c583110$103947ab@cisco.com> <1241818843.26211.67.camel@nhuynhth-laptop> X-Mailer: Apple Mail (2.930.3) Cc: multipathtcp@ietf.org, Olivier Bonaventure Subject: Re: [multipathtcp] TCP modification vs switching to SCTP X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 May 2009 17:38:24 -0000 On May 8, 2009, at 11:40 PM, Arnaud Ongenae wrote: > A new thread for a question out of the scope of > "Draft Multipath TCP (MPTCP) BOF description" > > TCP modification vs switching to SCTP (if it has already been =20 > discussed > elsewhere do not hesitate to redirect me) I think this "multipath TCP versus multipath SCTP" question is =20 important, but still misses a larger issue: SCTP and TCP are different =20= transports providing substantially different application-visible =20 semantics, but we would really like to have multipath support for =20 _both_ of those transports (and perhaps others). It's a pain for TCP =20= applications to switch to SCTP because it involves programmers =20 learning a new API with different semantics: e.g., the application has =20= to break byte streams up into (reasonable-size) messages and use =20 different system calls to tag them with a stream number and send them =20= on an SCTP connection. On the other hand, SCTP provides features that =20= some applications really want that TCP doesn't, such as a partially-=20 ordered connection where one stream's data can pass another stream's. =20= Furthermore, these semantics are pretty much totally orthogonal to =20 multipath issues: the transport functionality that needs to be =20 modified for multipath is fairly independent of and readily separable =20= from the transport functionality that provides application-visible =20 semantics like TCP's byte streams or SCTP's message-oriented multi-=20 streams. The modifications required for multipath support in general =20= mainly affect the transport's congestion control functionality, and =20 many research projects and deployed protocols have demonstrated how =20 congestion control can be cleanly separated from the application-=20 semantic part of the transport: e.g., the Congestion Manager =20 architecture, several prior variants of TCP and SCTP (e.g., Split-TCP, =20= pTCP, mTCP, LS-SCTP), and my own Structured Stream Transport (SST). So if we're going to start a BOF or WG about multipath something-or-=20 other, why should it be restricted to adding multipath to one =20 particular transport (and then get into debates about which transport =20= that should be), when what we really need is a clean way to add =20 multipath to _any_ transport? Whether to do that by literally =20 splitting the transport into multiple layers, as the HotNets-08 paper =20= Jana and I wrote proposes = (http://brynosaurus.com/pub/net/logjam-abs.html=20 ), or simply by developing a principled approach to adding multipath =20 support incrementally to existing "monolithic" transports while =20 considering the needs of multiple transports, amounts to technical =20 details that need to be discussed by whatever BOF or WG gets formed. =20= Expanding the BOF charter's scope from "multipath TCP" to "multipath =20 transport" wouldn't even need to mean that all transports will =20 necessarily get equal attention or that solutions for each transport =20 would be developed in lock-step; it just means recognizing that there =20= is more than one transport that applications and users care about and =20= that would like multipath support. Bryan > I believe that by introducing this modification to TCP (one-end MTCP) > you will give argument to those who do not want to switch to SCTP to > stay on TCP. I think that we should create an hybrid transition =20 > protocol > to produce the need to switch from TCP. This hybrid protocol exist and > was presented in : > Transparent TCP-to-SCTP translation shim layer > from RW Bickhart thesis [1] > > Why not concentrate multipath effort on SCTP which has the benefit to > natively support multihomed host and as Jana said research as been =20 > done > on the field. There are more than CMT, while doing my master thesis in > the field I discover LS-SCTP[2] or cmpSCTP[3] proposing different > interesting solutions to multipath issues. > > When you take a look at this website (that you certainly know): > http://www.isoc.org/briefings/017/ > you realize that using SCTP is more than simple multihomed =20 > capabilities. > I, personally, believe that letting SCTP to be alone to handle =20 > multipath > could boost his integration in the current internet. > > By defending TCP modification, don't you think that those new =20 > protocols > could play the same role as NAT for IPv4 address starvation ? Maybe =20= > it's > an hard comparison but I miss IPv6. > > > On Fri, 2009-05-08 at 22:08 +0200, Iljitsch van Beijnum wrote: >> On 8 mei 2009, at 19:47, Preethi Natarajan wrote: >> >>>> The motivations are better performance and redundancy without >>>> requiring application changes or any changes at the receiver. >> >>> Do you mean the application mods to open an SCTP (or whatever) =20 >>> socket >>> instead of TCP socket vs. mods to TCP stacks and middleboxes? >> >> We know for sure that if applications are going to switch from TCP to >> SCTP to gain multipath benefits, this creates deployment issues. >> Suppose that a browser wants to load pages from a website. How does >> the browser know if the server also supports SCTP? For IPv4 vs IPv6 =20= >> we >> could solve this using the DNS, but for TCP vs SCTP this is harder. > > absolutely, this is not a trivial issue >> >> Also, I believe current SCTP needs the application to tell it the >> addresses for the other side. Then there is the issue that SCTP must >> of course be implemented on both sides, while the big feature in the >> one-ended MPTCP is that it can work if it's only implemented in the >> sender. >> >> Then we trade this off against the possible gain of not having to >> change middleboxes. However, you assume that incompatible middleboxes >> exist. While I'm sure this will be the case a good percentage of the >> time, it's not a universal truism. >> >> And you assume that this same middlebox would be compatible with =20 >> SCTP. > > should be compatible :( > >> >> That is a signficant leap of faith, work for SCTP NAT is still going >> on in the behave wg. > > NAT is a poison on the internet (to me) > >> >> So I think there is a place for multipath TCP in general and the one- >> ended variety in particular. > > If you only consider multipath, yes, there is a place, especially if =20= > you > want the benefit as soon as possible, because this new protocol work > without modifying one end, you may implement it for yourself and =20 > already > use it. > > > regards, > > Arnaud Ongenae > UCL (Belgium) student > > > [1] http://www.cis.udel.edu/~amer/PEL/poc/pdf/BickhartMSthesis.pdf > > [2] Ahmed Abd El Al, Tarek Saadawi, and Myung Lee. Ls-sctp: a =20 > bandwidth > aggregation technique for stream control transmission protocol. =20 > Computer > Communications, > 27(10):1012=961024, 2004. Protocol Engineering for Wired and Wireless > Networks. > > [3] J. Liao, J. Wang, and X. Zhu. A multi-path mechanism for reliable > voip transmission > over wireless networks. Comput. Netw., 52(13):2450=962460, 2008. > > _______________________________________________ > multipathtcp mailing list > multipathtcp@ietf.org > https://www.ietf.org/mailman/listinfo/multipathtcp From iljitsch@muada.com Sat May 9 11:07:28 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 340653A6E69 for ; Sat, 9 May 2009 11:07:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.981 X-Spam-Level: X-Spam-Status: No, score=-5.981 tagged_above=-999 required=5 tests=[AWL=0.618, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iBzA6O1OMJeF for ; Sat, 9 May 2009 11:07:27 -0700 (PDT) Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by core3.amsl.com (Postfix) with ESMTP id 06B853A6E43 for ; Sat, 9 May 2009 11:07:26 -0700 (PDT) Received: from [192.168.2.4] (static-167-138-7-89.ipcom.comunitel.net [89.7.138.167] (may be forged)) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n49I8QSE028269 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Sat, 9 May 2009 20:08:27 +0200 (CEST) (envelope-from iljitsch@muada.com) Message-Id: <0D6E7BC4-17AF-4F5C-9107-ABA2B729683F@muada.com> From: Iljitsch van Beijnum To: Bryan Ford In-Reply-To: Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Sat, 9 May 2009 20:08:29 +0200 References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <20090507230452.GC35382@cisco.com> <1241789497.4055.8.camel@nhuynhth-laptop> <001e01c9cfff$bde8f1e0$103947ab@cisco.com> <97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com> <001f01c9d005$0c583110$103947ab@cisco.com> <1241818843.26211.67.camel@nhuynhth-laptop> X-Mailer: Apple Mail (2.930.3) Cc: multipathtcp@ietf.org Subject: [multipathtcp] TCP/SCTP and BoF description, was: Re: TCP modification vs switching to SCTP X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 May 2009 18:07:28 -0000 On 9 mei 2009, at 19:39, Bryan Ford wrote: > SCTP and TCP are different transports providing substantially > different application-visible semantics, Another difference is that TCP is extremely simple, while SCTP has a more complex structure with its chunks. This and the CRC32 checksum make SCTP processing more CPU intensive than TCP processing. I don't know how big the difference is, but I'd say that SCTP isn't appropriate in very constrained environments (although the constrained environments from a few years ago, such as cell phones, are much more capable today, but there are devices that are even _more_ constrained) or very high-performance environments. Even on my 2 GHz dual core Mac I need to use jumboframes to saturate gigabit ethernet over TCP. > So if we're going to start a BOF or WG about multipath something-or- > other, why should it be restricted to adding multipath to one > particular transport (and then get into debates about which > transport that should be), when what we really need is a clean way > to add multipath to _any_ transport? There are three parts to all of this: 1. Congestion control. We haven't talked about this much yet, but there is a lot of work to do here, which of course applies equally to TCP, SCTP and other transports using TCP-like congestion control 2. Multipath mechanisms. TCP doesn't have these, and we now have two drafts proposing them. There are papers and even implementations skinning this cat in different ways. We currently have a single address and a multiaddress proposal. Would of course be nice to be able to do both, as each has limitations that the other doesn't have. SCTP does multiaddress multipath and it could be extended with the single address multipath similar to TCP but the details will be different (option space etc) (2a: TCP, 2b: SCTP) 3. Interaction with routing. In the current draft, the assumption is that the host acting as a TCP sender sends packets belonging to different subflows in different directions, without help from the routing system. However, the routing system has many more opportunities to make path choices than hosts so if we can take advantage of that we get many more paths and thus better multipath. So the question is, what should the BoF be about? 1, 2a, 2b, 3? Just 2a? More than just 2a but less than everything? My take: From a practical perspective, I can tell you that putting congestion control and routing protocol people in the same room isn't always very effective. I think we have a pretty good handle on the 2a issues, so we can certainly do some good engineering in this area in the IETF. If there is significant interest in 2b, then perhaps 2a and 2b could be done at the same time. I don't think we can unleash modified transport protocols that may be used for 85% of the internet's traffic without addressing congestion control, so I guess 1 needs to be in scope. However, we should curb are ambitions in this area, or we won't have any usable results for a long time. If we can get the "first do no harm" part of the multipath congestion control changes done, then the rest is probably best left to the research community rather than the IETF. As for 3: someone made the argument that changing IP and routing would be a hard sell to the IESG and it would need to happen elsewhere anyway. I agree. It would be more effective to wait with this work until parts 1 and 2 are in good shape, and then figure out if, how and when 3 should be done. From john@jlc.net Sat May 9 11:41:40 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 534643A68F2 for ; Sat, 9 May 2009 11:41:40 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.84 X-Spam-Level: X-Spam-Status: No, score=-5.84 tagged_above=-999 required=5 tests=[AWL=0.759, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Y44+biTmjVf6 for ; Sat, 9 May 2009 11:41:39 -0700 (PDT) Received: from mailhost.jlc.net (mailhost.jlc.net [199.201.159.9]) by core3.amsl.com (Postfix) with ESMTP id 850AF3A69B7 for ; Sat, 9 May 2009 11:41:12 -0700 (PDT) Received: by mailhost.jlc.net (Postfix, from userid 104) id 5154233C6A; Sat, 9 May 2009 14:42:41 -0400 (EDT) Date: Sat, 9 May 2009 14:42:41 -0400 From: John Leslie To: Iljitsch van Beijnum Message-ID: <20090509184241.GT32848@verdi> References: <1241789497.4055.8.camel@nhuynhth-laptop> <001e01c9cfff$bde8f1e0$103947ab@cisco.com> <97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com> <001f01c9d005$0c583110$103947ab@cisco.com> <1241818843.26211.67.camel@nhuynhth-laptop> <0D6E7BC4-17AF-4F5C-9107-ABA2B729683F@muada.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <0D6E7BC4-17AF-4F5C-9107-ABA2B729683F@muada.com> User-Agent: Mutt/1.4.1i Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] TCP/SCTP and BoF description, was: Re: TCP modification vs switching to SCTP X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 May 2009 18:41:40 -0000 Iljitsch van Beijnum wrote: > > There are three parts to all of this: > > 1. Congestion control. We haven't talked about this much yet, but > there is a lot of work to do here, which of course applies equally to > TCP, SCTP and other transports using TCP-like congestion control Absolutely! This is work (IMHO) best done under an Experimental RFC, and I am hoping we can produce such a beast, describing the experiment. (That could, of course, be IRTF work, but an IETF WG could make much faster progress.) > 2. Multipath mechanisms. TCP doesn't have these, and we now have two > drafts proposing them. There are papers and even implementations > skinning this cat in different ways. We currently have a single > address and a multiaddress proposal. Would of course be nice to be > able to do both, as each has limitations that the other doesn't have. > SCTP does multiaddress multipath and it could be extended with the > single address multipath similar to TCP but the details will be > different (option space etc) (2a: TCP, 2b: SCTP) We shouldn't gloss over these differences. I believe a TCP-only implementation can be done more quickly, and be deployed much more quickly for a large-scale experiment (which I believe necessary to establish _practical_ congestion-control principles). > 3. Interaction with routing. In the current draft, the assumption is > that the host acting as a TCP sender sends packets belonging to > different subflows in different directions, without help from the > routing system. However, the routing system has many more > opportunities to make path choices than hosts so if we can take > advantage of that we get many more paths and thus better multipath. I disagree that more paths necessarily means better multipath. In particular, congestion control becomes exponentially harder when the endpoints don't know from moment to moment which paths are being used. Also, deploying _any_ routing change to the actual big-I Internet will be a slow process. I most strongly advise against putting it in any critical path of work we hope to accomplish. > So the question is, what should the BoF be about? 1, 2a, 2b, 3? Just > 2a? More than just 2a but less than everything? > > My take: > > From a practical perspective, I can tell you that putting congestion > control and routing protocol people in the same room isn't always very > effective. An understatment I'd be proud of! > I think we have a pretty good handle on the 2a issues, so we can > certainly do some good engineering in this area in the IETF. I agree, and I'd hesitate to crowd too much more into early milestones. > If there is significant interest in 2b, then perhaps 2a and 2b could be > done at the same time. This is a good question. I see a deployment benefit in having a single code base for both, but I can't imagine how to get there. Absent a single code base, it would seem we could get faster deployment by taking them serially instead of in parallel. > I don't think we can unleash modified transport protocols that may be > used for 85% of the internet's traffic without addressing congestion > control, so I guess 1 needs to be in scope. However, we should curb > our ambitions in this area, or we won't have any usable results for a > long time. If we can get the "first do no harm" part of the multipath > congestion control changes done, then the rest is probably best left > to the research community rather than the IETF. I wouldn't go quite that far: we will need field experiments, which will progress faster in IETF than IRTF. Also "no harm" is in the eye of the beholder, and we need to better quantify such a promise. > As for 3: someone made the argument that changing IP and routing would > be a hard sell to the IESG and it would need to happen elsewhere > anyway. I agree. It would be more effective to wait with this work > until parts 1 and 2 are in good shape, and then figure out if, how and > when 3 should be done. I agree. Changing _anything_ about routing would have to coordinate with Routing Area, which would greatly complicate the process. I believe we can implement (2a) _entirely_ without changes to routing or even interaction with middleboxes. If I am right in that belief, scoping our work (initially) to (1) and (2a) will get us into congestion-control experiments much more quickly. (If I haven't been clear, let me state explicitly: we can't possibly satisfy everyone's congestion-control desires by arguing positions: we will need experimental results.) -- John Leslie From Pasi.Sarolahti@nokia.com Mon May 11 01:11:08 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 93F1D3A6F1D for ; Mon, 11 May 2009 01:11:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id W7DsNj6GLLuu for ; Mon, 11 May 2009 01:11:07 -0700 (PDT) Received: from mgw-mx09.nokia.com (smtp.nokia.com [192.100.105.134]) by core3.amsl.com (Postfix) with ESMTP id 1AF383A67B0 for ; Mon, 11 May 2009 01:10:46 -0700 (PDT) Received: from esebh105.NOE.Nokia.com (esebh105.ntc.nokia.com [172.21.138.211]) by mgw-mx09.nokia.com (Switch-3.2.6/Switch-3.2.6) with ESMTP id n4B8Bub8029671; Mon, 11 May 2009 03:12:14 -0500 Received: from vaebh102.NOE.Nokia.com ([10.160.244.23]) by esebh105.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.3959); Mon, 11 May 2009 11:12:06 +0300 Received: from vaebh101.NOE.Nokia.com ([10.160.244.22]) by vaebh102.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.3959); Mon, 11 May 2009 11:12:06 +0300 Received: from [172.21.30.100] ([172.21.30.100]) by vaebh101.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.3959); Mon, 11 May 2009 11:12:03 +0300 In-Reply-To: <1241818843.26211.67.camel@nhuynhth-laptop> References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <20090507230452.GC35382@cisco.com> <1241789497.4055.8.camel@nhuynhth-laptop> <001e01c9cfff$bde8f1e0$103947ab@cisco.com> <97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com> <001f01c9d005$0c583110$103947ab@cisco.com> <1241818843.26211.67.camel@nhuynhth-laptop> Mime-Version: 1.0 (Apple Message framework v753.1) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Pasi Sarolahti Date: Mon, 11 May 2009 11:11:57 +0300 To: ext Arnaud Ongenae X-Mailer: Apple Mail (2.753.1) X-OriginalArrivalTime: 11 May 2009 08:12:03.0427 (UTC) FILETIME=[2AB03330:01C9D210] X-Nokia-AV: Clean Cc: "multipathtcp@ietf.org" , Olivier Bonaventure Subject: Re: [multipathtcp] TCP modification vs switching to SCTP X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 May 2009 08:11:08 -0000 On May 9, 2009, at 0:40, ext Arnaud Ongenae wrote: > Why not concentrate multipath effort on SCTP which has the benefit to > natively support multihomed host and as Jana said research as been > done > on the field. There are more than CMT, while doing my master thesis in > the field I discover LS-SCTP[2] or cmpSCTP[3] proposing different > interesting solutions to multipath issues. There is one common practical reason: I think multihomed SCTP (or any protocol that explicitly communicates multihomed IP addresses inside transport header) does not work over NATs, even if encapsulated inside UDP. I think it is possible to design multipath TCP so that it can be made to work over existing NATs. - Pasi From Pasi.Sarolahti@nokia.com Mon May 11 01:23:23 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 50F533A6B79 for ; Mon, 11 May 2009 01:23:23 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id l74BKLCp2sv8 for ; Mon, 11 May 2009 01:23:22 -0700 (PDT) Received: from mgw-mx03.nokia.com (smtp.nokia.com [192.100.122.230]) by core3.amsl.com (Postfix) with ESMTP id 3B3E23A68C5 for ; Mon, 11 May 2009 01:23:22 -0700 (PDT) Received: from vaebh105.NOE.Nokia.com (vaebh105.europe.nokia.com [10.160.244.31]) by mgw-mx03.nokia.com (Switch-3.2.6/Switch-3.2.6) with ESMTP id n4B8OBKi006723; Mon, 11 May 2009 11:24:44 +0300 Received: from vaebh102.NOE.Nokia.com ([10.160.244.23]) by vaebh105.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.3959); Mon, 11 May 2009 11:24:33 +0300 Received: from vaebh101.NOE.Nokia.com ([10.160.244.22]) by vaebh102.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.3959); Mon, 11 May 2009 11:24:32 +0300 Received: from [172.21.30.100] ([172.21.30.100]) by vaebh101.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.3959); Mon, 11 May 2009 11:24:28 +0300 In-Reply-To: <4A051E47.7090400@fandm.edu> References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es><4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><20090507230452.GC35382@cisco.com><1241789497.4055.8.camel@nhuynhth-laptop> <001e01c9cfff$bde8f1e0$103947ab@cisco.com> <97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com> <001f01c9d005$0c583110$103947ab@cisco.com> <000001c9d028$aea92640$103947ab@cisco.com> <4A051E47.7090400@fandm.edu> Mime-Version: 1.0 (Apple Message framework v753.1) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <2AEF47F0-83EC-4BD9-84B8-C700F663C495@nokia.com> Content-Transfer-Encoding: 7bit From: Pasi Sarolahti Date: Mon, 11 May 2009 11:24:22 +0300 To: "janardhan.iyengar@fandm.edu" X-Mailer: Apple Mail (2.753.1) X-OriginalArrivalTime: 11 May 2009 08:24:28.0237 (UTC) FILETIME=[E6A12FD0:01C9D211] X-Nokia-AV: Clean Cc: "multipathtcp@ietf.org" Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 May 2009 08:23:23 -0000 On May 9, 2009, at 9:10, ext Janardhan Iyengar wrote: > A couple of thoughts on the draft/BoF: > > 1/ "Two-ended" multipath, where both endpoints are aware: There > are several architectural and design considerations that will be > common to TCP, SCTP, DCCP, SST etc., and as I see it, there is no > reason why this discussion should not include all transports. This might be a sensible idea: there is an expired draft on multipath/ mobile DCCP (draft-kohler-dccp-mobility-02) that is a bit similar to the multipath TCP variant with implicit address management. - Pasi From Rolf.Winter@nw.neclab.eu Mon May 11 01:31:53 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 351433A6CD8 for ; Mon, 11 May 2009 01:31:53 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.565 X-Spam-Level: X-Spam-Status: No, score=-1.565 tagged_above=-999 required=5 tests=[AWL=1.034, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TZn83l7-5Z7n for ; Mon, 11 May 2009 01:31:52 -0700 (PDT) Received: from smtp0.neclab.eu (smtp0.neclab.eu [195.37.70.41]) by core3.amsl.com (Postfix) with ESMTP id 08A743A68C5 for ; Mon, 11 May 2009 01:31:48 -0700 (PDT) Received: from localhost (localhost.localdomain [127.0.0.1]) by smtp0.neclab.eu (Postfix) with ESMTP id 2585E2C017B31; Mon, 11 May 2009 10:33:18 +0200 (CEST) X-Virus-Scanned: Amavisd on Debian GNU/Linux (atlas2.office) Received: from smtp0.neclab.eu ([127.0.0.1]) by localhost (atlas2.office [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8qZDccDN1FPI; Mon, 11 May 2009 10:33:18 +0200 (CEST) Received: from VENUS.office (mx2.office [192.168.24.15]) by smtp0.neclab.eu (Postfix) with ESMTP id EB5F82C00C51C; Mon, 11 May 2009 10:33:02 +0200 (CEST) X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Mon, 11 May 2009 10:33:00 +0200 Message-ID: <547F018265F92642B577B986577D671C882A39@VENUS.office> In-Reply-To: <0D6E7BC4-17AF-4F5C-9107-ABA2B729683F@muada.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [multipathtcp] TCP/SCTP and BoF description, was: Re: TCP modification vs switching to SCTP Thread-Index: AcnQ0UGbhlGqSzJ4Sc6YojUBad44qQBP81wQ References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es><4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><20090507230452.GC35382@cisco.com><1241789497.4055.8.camel@nhuynhth-laptop><001e01c9cfff$bde8f1e0$103947ab@cisco.com><97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com><001f01c9d005$0c583110$103947ab@cisco.com><1241818843.26211.67.camel@nhuynhth-laptop> <0D6E7BC4-17AF-4F5C-9107-ABA2B729683F@muada.com> From: "Rolf Winter" To: "Iljitsch van Beijnum" , "Bryan Ford" Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] TCP/SCTP and BoF description, was: Re: TCP modification vs switching to SCTP X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 May 2009 08:31:53 -0000 Regarding the scope of the BoF, I agree with Iljitsch here. Multi-path = TCP, or any transport for that matter, that uses multiple address pairs = from more than a single operator will use different paths which are as = diverse as BGP will allow them to be. So, not having any interaction = with routers/routing will already make multi-path transport an = interesting thing to look at. Once we know how to do the multi-path TCP = bit right, we can come back and see whether we can make it even better = assuming that routing is involved. But unless we understand congestion = control across multiple subflows and the gory engineering details of the = protocol, broadening the scope beyond the purely transport related = issues will unnecessarily complicate things at this point. NEC Europe Limited | Registered Office: NEC House, 1 Victoria Road, = London W3 6BL | Registered in England 2832014=20 =20 > -----Original Message----- > From: multipathtcp-bounces@ietf.org [mailto:multipathtcp- > bounces@ietf.org] On Behalf Of Iljitsch van Beijnum > Sent: Samstag, 9. Mai 2009 20:08 > To: Bryan Ford > Cc: multipathtcp@ietf.org > Subject: [multipathtcp] TCP/SCTP and BoF description,was: Re: TCP > modification vs switching to SCTP >=20 > On 9 mei 2009, at 19:39, Bryan Ford wrote: >=20 > > SCTP and TCP are different transports providing substantially > > different application-visible semantics, >=20 > Another difference is that TCP is extremely simple, while SCTP has a > more complex structure with its chunks. This and the CRC32 checksum > make SCTP processing more CPU intensive than TCP processing. I don't > know how big the difference is, but I'd say that SCTP isn't > appropriate in very constrained environments (although the constrained > environments from a few years ago, such as cell phones, are much more > capable today, but there are devices that are even _more_ constrained) > or very high-performance environments. Even on my 2 GHz dual core Mac > I need to use jumboframes to saturate gigabit ethernet over TCP. >=20 > > So if we're going to start a BOF or WG about multipath something-or- > > other, why should it be restricted to adding multipath to one > > particular transport (and then get into debates about which > > transport that should be), when what we really need is a clean way > > to add multipath to _any_ transport? >=20 > There are three parts to all of this: >=20 > 1. Congestion control. We haven't talked about this much yet, but > there is a lot of work to do here, which of course applies equally to > TCP, SCTP and other transports using TCP-like congestion control >=20 > 2. Multipath mechanisms. TCP doesn't have these, and we now have two > drafts proposing them. There are papers and even implementations > skinning this cat in different ways. We currently have a single > address and a multiaddress proposal. Would of course be nice to be > able to do both, as each has limitations that the other doesn't have. > SCTP does multiaddress multipath and it could be extended with the > single address multipath similar to TCP but the details will be > different (option space etc) (2a: TCP, 2b: SCTP) >=20 > 3. Interaction with routing. In the current draft, the assumption is > that the host acting as a TCP sender sends packets belonging to > different subflows in different directions, without help from the > routing system. However, the routing system has many more > opportunities to make path choices than hosts so if we can take > advantage of that we get many more paths and thus better multipath. >=20 > So the question is, what should the BoF be about? 1, 2a, 2b, 3? Just > 2a? More than just 2a but less than everything? >=20 > My take: >=20 > From a practical perspective, I can tell you that putting congestion > control and routing protocol people in the same room isn't always very > effective. I think we have a pretty good handle on the 2a issues, so > we can certainly do some good engineering in this area in the IETF. If > there is significant interest in 2b, then perhaps 2a and 2b could be > done at the same time. >=20 > I don't think we can unleash modified transport protocols that may be > used for 85% of the internet's traffic without addressing congestion > control, so I guess 1 needs to be in scope. However, we should curb > are ambitions in this area, or we won't have any usable results for a > long time. If we can get the "first do no harm" part of the multipath > congestion control changes done, then the rest is probably best left > to the research community rather than the IETF. >=20 > As for 3: someone made the argument that changing IP and routing would > be a hard sell to the IESG and it would need to happen elsewhere > anyway. I agree. It would be more effective to wait with this work > until parts 1 and 2 are in good shape, and then figure out if, how and > when 3 should be done. > _______________________________________________ > multipathtcp mailing list > multipathtcp@ietf.org > https://www.ietf.org/mailman/listinfo/multipathtcp From iljitsch@muada.com Mon May 11 03:31:52 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id E9F773A67B0 for ; Mon, 11 May 2009 03:31:52 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.54 X-Spam-Level: X-Spam-Status: No, score=-4.54 tagged_above=-999 required=5 tests=[AWL=2.059, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 94uZfj6CHFHz for ; Mon, 11 May 2009 03:31:07 -0700 (PDT) Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by core3.amsl.com (Postfix) with ESMTP id 27CB53A6CEB for ; Mon, 11 May 2009 03:30:47 -0700 (PDT) Received: from w210022.ccupm.upm.es (w210022.ccupm.upm.es [138.100.210.22]) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n4BAVg7B041771 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Mon, 11 May 2009 12:31:43 +0200 (CEST) (envelope-from iljitsch@muada.com) Message-Id: <9FC0003F-68D6-47FF-8B4A-6BAB20A2A59E@muada.com> From: Iljitsch van Beijnum To: Preethi Natarajan In-Reply-To: <000001c9d028$aea92640$103947ab@cisco.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Mon, 11 May 2009 12:31:48 +0200 References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es><4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><20090507230452.GC35382@cisco.com><1241789497.4055.8.camel@nhuynhth-laptop> <001e01c9cfff$bde8f1e0$103947ab@cisco.com> <97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com> <001f01c9d005$0c583110$103947ab@cisco.com> <000001c9d028$aea92640$103947ab@cisco.com> X-Mailer: Apple Mail (2.930.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 May 2009 10:31:53 -0000 On 9 mei 2009, at 0:02, Preethi Natarajan wrote: >> Suppose that a browser wants to load pages from a website. >> How does the browser know if the server also supports SCTP? >> For IPv4 vs IPv6 we could solve this using the DNS, but for >> TCP vs SCTP this is harder. > DNS has been proposed to identify transport protocols in the past, > for ex. > see RFC 3263. From what I hear, SRV records have worked quite well > for SIP. That's because you can't do SIP without it. For HTTP it hasn't been successful. Also note that SRV records only supply a port number and a weight/ priority, not a protocol number. So without modifications, it wouldn't be possible to use SRV records to differentiate between TCP and SCTP. >> So I think there is a place for multipath TCP in general and >> the one- ended variety in particular. > I am sure it is. I am just wondering why the draft should be tied to > TCP, So if this BoF results in a wg, and that wg would take on this work for both TCP and SCTP, how would that work? Would we have different documents for TCP and for SCTP, or one document? Is anyone volunteering to spend time on this for SCTP? From preethi.cis@gmail.com Mon May 11 08:52:17 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 1496E3A6CA7 for ; Mon, 11 May 2009 08:52:17 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gM7a+Y1uuteW for ; Mon, 11 May 2009 08:52:16 -0700 (PDT) Received: from rv-out-0506.google.com (rv-out-0506.google.com [209.85.198.236]) by core3.amsl.com (Postfix) with ESMTP id 80FF53A67E1 for ; Mon, 11 May 2009 08:52:15 -0700 (PDT) Received: by rv-out-0506.google.com with SMTP id g37so1961758rvb.49 for ; Mon, 11 May 2009 08:53:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:from:to:cc:references:subject :date:message-id:mime-version:content-type:content-transfer-encoding :x-mailer:thread-index:in-reply-to:x-mimeole; bh=YioVETNfid0P1NxIuorE+u1GTGHiFJbGlhRTKcdzwUU=; b=MMQqQO7LEtLOKG0VfT9mPMF/K/+QCmmkEw9P2Ro/PfxO2v0qR9BZ78FeF0TteI6kdo loVLxX5bwTlGgUl01OYPIN0LDnenGiwIGRdgVfA6YEY6m26r/aAI38mGPNpyW4HOFgBL rjcLceyBoBuKkj/eCOJBJ+oPLgCVfwuGdqBio= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:cc:references:subject:date:message-id:mime-version :content-type:content-transfer-encoding:x-mailer:thread-index :in-reply-to:x-mimeole; b=TcP+O+MLwiUnE0SCiAQ0g3J0aqiRqW6dIRz7cgBWtn7hzhmmVg6twMnGSsfSaYZAKl 85OqTR+t+NwZib7ydI4PpWYDdVgQadXJpclQzjNd2TuLuxTUyjsUxIVHFuPV69qQ53Fg +0p0qlR8WrCuGIoLuvCBa1TX4/F1RYzuw/0hA= Received: by 10.142.79.12 with SMTP id c12mr2839494wfb.325.1242057224817; Mon, 11 May 2009 08:53:44 -0700 (PDT) Received: from prenatarwxp01 (dhcp-171-71-57-16.cisco.com [171.71.57.16]) by mx.google.com with ESMTPS id 30sm10914122wfa.35.2009.05.11.08.53.43 (version=SSLv3 cipher=RC4-MD5); Mon, 11 May 2009 08:53:44 -0700 (PDT) From: "Preethi Natarajan" To: "'Iljitsch van Beijnum'" , "'Bryan Ford'" References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es><4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><20090507230452.GC35382@cisco.com><1241789497.4055.8.camel@nhuynhth-laptop><001e01c9cfff$bde8f1e0$103947ab@cisco.com><97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com><001f01c9d005$0c583110$103947ab@cisco.com><1241818843.26211.67.camel@nhuynhth-laptop> <0D6E7BC4-17AF-4F5C-9107-ABA2B729683F@muada.com> Date: Mon, 11 May 2009 08:53:43 -0700 Message-ID: <000b01c9d250$a9c311c0$103947ab@cisco.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 11 Thread-Index: AcnQ0Tv96AhepfJKQZWpCXJni/76AAA8PSgg In-Reply-To: <0D6E7BC4-17AF-4F5C-9107-ABA2B729683F@muada.com> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3350 Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] TCP/SCTP and BoF description, was: Re: TCP modification vs switching to SCTP X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 May 2009 15:52:17 -0000 > -----Original Message----- > From: multipathtcp-bounces@ietf.org > [mailto:multipathtcp-bounces@ietf.org] On Behalf Of Iljitsch > van Beijnum > Sent: Saturday, May 09, 2009 11:08 AM > To: Bryan Ford > > Another difference is that TCP is extremely simple, while > SCTP has a more complex structure with its chunks. This and > the CRC32 checksum make SCTP processing more CPU intensive > than TCP processing. I don't know how big the difference is, > but I'd say that SCTP isn't appropriate in very constrained > environments 1. CRC32c might be an intensive operation. Nonetheless CRC32c was chosen for its superior error detection mechanisms. 2. As you mention, the notion of "very constrained envrionments" has been a moving target -- what we think is "very constrained" today might not be in the future. My larger point -- why are _we_ trying to decide which transport would be best for today's/tomorrow's requirements? Instead, I would prefer that the consumers (app developers) make that decision. If the app developer doesn't want the compute-intensive SCTP and wants to use DCCP or XYZ, then so be it. However, this can happen only if the BoF or WG is cordial to transports other than TCP... > > DNS has been proposed to identify transport protocols in > the past, for > > ex. > > see RFC 3263. From what I hear, SRV records have worked > quite well for > > SIP. > > That's because you can't do SIP without it. For HTTP it > hasn't been successful. I was under the impression that nobody has even tried to do this with SCTP... there are a few upcoming proposals but thats about it? I would say its premature to say that it hasn't been successful. > So if this BoF results in a wg, and that wg would take on > this work for both TCP and SCTP, how would that work? Would > we have different documents for TCP and for SCTP, or one > document? Is anyone volunteering to spend time on this for SCTP? I know that there are volunteers for SCTP, and am sure there will be volunteers for other transports as well. Preethi From L.Wood@surrey.ac.uk Mon May 11 09:19:21 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 5E2FF3A680E for ; Mon, 11 May 2009 09:19:21 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jcKO93XpSCCD for ; Mon, 11 May 2009 09:19:20 -0700 (PDT) Received: from mail115.messagelabs.com (mail115.messagelabs.com [195.245.231.179]) by core3.amsl.com (Postfix) with SMTP id B61593A68B4 for ; Mon, 11 May 2009 09:19:19 -0700 (PDT) X-VirusChecked: Checked X-Env-Sender: L.Wood@surrey.ac.uk X-Msg-Ref: server-6.tower-115.messagelabs.com!1242058844!39707413!5 X-StarScan-Version: 6.0.0; banners=-,-,- X-Originating-IP: [131.227.102.140] Received: (qmail 28153 invoked from network); 11 May 2009 16:20:49 -0000 Received: from ads40.surrey.ac.uk (HELO ads40.surrey.ac.uk) (131.227.102.140) by server-6.tower-115.messagelabs.com with SMTP; 11 May 2009 16:20:49 -0000 Received: from ads31.surrey.ac.uk ([131.227.120.131]) by ads40.surrey.ac.uk with Microsoft SMTPSVC(6.0.3790.3959); Mon, 11 May 2009 17:20:47 +0100 Received: from [192.168.1.209] ([86.3.114.249]) by ads31.surrey.ac.uk over TLS secured channel with Microsoft SMTPSVC(6.0.3790.3959); Mon, 11 May 2009 17:20:46 +0100 Message-Id: <1566C013-B291-48E7-A826-7667C1B0502C@surrey.ac.uk> From: Lloyd Wood To: "Preethi Natarajan" In-Reply-To: <000b01c9d250$a9c311c0$103947ab@cisco.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Mon, 11 May 2009 17:20:46 +0100 References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es><4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><20090507230452.GC35382@cisco.com><1241789497.4055.8.camel@nhuynhth-laptop><001e01c9cfff$bde8f1e0$103947ab@cisco.com><97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com><001f01c9d005$0c583110$103947ab@cisco.com><1241818843.26211.67.camel@nhuynhth-laptop> <0D6E7BC4-17AF-4F5C-9107-ABA2B729683F@muada.com> <000b01c9d250$a9c311c0$103947ab@cisco.com> X-Mailer: Apple Mail (2.930.3) X-OriginalArrivalTime: 11 May 2009 16:20:46.0693 (UTC) FILETIME=[70B78150:01C9D254] Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] TCP/SCTP and BoF description, was: Re: TCP modification vs switching to SCTP X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 May 2009 16:19:21 -0000 CRC32c isn't intensive compared to e.g. message digests. The choice of a better checksum mechanism for SCTP was predicated on the fact that the ones-complement checksum used in TCP/UDP is very weak, and doesn't catch swapped words - which is how echo servers worked way back when. Swap the addresses, the ones-complement checksum doesn't even notice. See RFC3309 for some reasons for choosing CRC32c over the previous Adler32 (which is still better than the UDP/TCP checksums). The size and complexity of the SCTP state machine and whether its features are desirable or necessary are better embedded arguments against SCTP than its choice of checksum to protect itself. On 11 May 2009, at 16:53, Preethi Natarajan wrote: >> Another difference is that TCP is extremely simple, while >> SCTP has a more complex structure with its chunks. This and >> the CRC32 checksum make SCTP processing more CPU intensive >> than TCP processing. I don't know how big the difference is, >> but I'd say that SCTP isn't appropriate in very constrained >> environments > > 1. CRC32c might be an intensive operation. Nonetheless CRC32c was > chosen for > its superior error detection mechanisms. > 2. As you mention, the notion of "very constrained envrionments" has > been a > moving target -- what we think is "very constrained" today might not > be in > the future. DTN work: http://info.ee.surrey.ac.uk/Personal/L.Wood/saratoga/ From preethi.cis@gmail.com Mon May 11 10:42:16 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 298463A686C for ; Mon, 11 May 2009 10:42:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JDOTUwqCrdz2 for ; Mon, 11 May 2009 10:42:15 -0700 (PDT) Received: from rv-out-0506.google.com (rv-out-0506.google.com [209.85.198.232]) by core3.amsl.com (Postfix) with ESMTP id 2D6953A67DB for ; Mon, 11 May 2009 10:42:15 -0700 (PDT) Received: by rv-out-0506.google.com with SMTP id g37so1996900rvb.49 for ; Mon, 11 May 2009 10:43:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:from:to:cc:references:subject :date:message-id:mime-version:content-type:content-transfer-encoding :x-mailer:thread-index:in-reply-to:x-mimeole; bh=fy0AlV8yOeSPGoieDGX54IssBufNz6TpZAT4L9cdizY=; b=EO0thEZieBEqBD/PIvCKqGlgGFsr3gnrTddU4GURtHIy3pwcqmmNXC9JBSytwlMXs/ SqaCUzSP9q/01bqJTLTcd0bW7GdGO7DPosQlQsGAKj9GAHvP4SuW/33NBi52ssf/2SSc fleetAeybh4DB8rA2WfufAb8IhY5NIDDvvz8k= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:cc:references:subject:date:message-id:mime-version :content-type:content-transfer-encoding:x-mailer:thread-index :in-reply-to:x-mimeole; b=klfkynAlrPBcQ65CiJpJYL+EfQYYqEXNE9zRNR1F51XigeY/xbCfMs9ljlUfdnKZkZ p1ObI/V3X2eMllvNDq+jjVN3ljiiABx49MaW01EFQIpOGKeFFRSjgZlx2zu4HxhobpGv IZN7gUIqhYoCjEaBf34ElP2vgDgzouMk1oSPQ= Received: by 10.142.212.21 with SMTP id k21mr2892107wfg.65.1242063824608; Mon, 11 May 2009 10:43:44 -0700 (PDT) Received: from prenatarwxp01 (dhcp-171-71-57-16.cisco.com [171.71.57.16]) by mx.google.com with ESMTPS id 24sm15627718wfc.37.2009.05.11.10.43.43 (version=SSLv3 cipher=RC4-MD5); Mon, 11 May 2009 10:43:43 -0700 (PDT) From: "Preethi Natarajan" To: "'Lloyd Wood'" References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es><4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><20090507230452.GC35382@cisco.com><1241789497.4055.8.camel@nhuynhth-laptop><001e01c9cfff$bde8f1e0$103947ab@cisco.com><97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com><001f01c9d005$0c583110$103947ab@cisco.com><1241818843.26211.67.camel@nhuynhth-laptop> <0D6E7BC4-17AF-4F5C-9107-ABA2B729683F@muada.com> <000b01c9d250$a9c311c0$103947ab@cisco.com> <1566C013-B291-48E7-A826-7667C1B0502C@surrey .ac.uk> Date: Mon, 11 May 2009 10:43:42 -0700 Message-ID: <001b01c9d260$073ab0b0$103947ab@cisco.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 11 Thread-Index: AcnSVHNIux4ibUOyT9mcUwkMTXBpZAACcnHA In-Reply-To: <1566C013-B291-48E7-A826-7667C1B0502C@surrey.ac.uk> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3350 Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] TCP/SCTP and BoF description, was: Re: TCP modification vs switching to SCTP X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 May 2009 17:42:16 -0000 > -----Original Message----- > From: Lloyd Wood [mailto:L.Wood@surrey.ac.uk] > Sent: Monday, May 11, 2009 9:21 AM > To: Preethi Natarajan > Cc: Lloyd Wood; 'Iljitsch van Beijnum'; 'Bryan Ford'; > multipathtcp@ietf.org > Subject: Re: [multipathtcp] TCP/SCTP and BoF description, > was: Re: TCP modification vs switching to SCTP > > The size and complexity of the SCTP state machine and whether > its features are desirable or necessary are better embedded > arguments against SCTP than its choice of checksum to protect itself. > SCTP is more complex since SCTP offers more services/features than TCP. But, there are apps/environments for which SCTP's features improve performance in spite of the associated complexity... > DTN work: http://info.ee.surrey.ac.uk/Personal/L.Wood/saratoga/ As I mentioned before we shouldn't be trying to decide which transport would be most suitable for tomorrow's app requirements. For example, Saratoga was invented because TCP wasn't good enough for certain environments, and I doubt if these environments were even envisioned when TCP was designed... Personally, I think that we should open up the playing field so that app developers have a variety of transports to choose from... Preethi From marcelo@it.uc3m.es Mon May 11 11:49:35 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id A3EA63A6A8C for ; Mon, 11 May 2009 11:49:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.356 X-Spam-Level: X-Spam-Status: No, score=-6.356 tagged_above=-999 required=5 tests=[AWL=0.243, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cyGZLFWOK1va for ; Mon, 11 May 2009 11:49:34 -0700 (PDT) Received: from smtp01.uc3m.es (smtp01.uc3m.es [163.117.176.131]) by core3.amsl.com (Postfix) with ESMTP id F15573A6946 for ; Mon, 11 May 2009 11:49:33 -0700 (PDT) Received: from marcelo-bagnulos-macbook-pro.local (166.pool85-53-132.dynamic.orange.es [85.53.132.166]) by smtp01.uc3m.es (Postfix) with ESMTP id 884BFB49775; Mon, 11 May 2009 20:51:02 +0200 (CEST) Message-ID: <4A087396.4050001@it.uc3m.es> Date: Mon, 11 May 2009 20:51:02 +0200 From: marcelo bagnulo braun User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Joe Touch References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <20090507230452.GC35382@cisco.com> <4A036A29.7020005@isi.edu> <4A042F28.5050107@it.uc3m.es> <4A046242.4040104@isi.edu> In-Reply-To: <4A046242.4040104@isi.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-TM-AS-Product-Ver: IMSS-7.0.0.3116-5.6.0.1016-16636.001 Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 May 2009 18:49:35 -0000 Joe Touch escribió: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > > >> Joe Touch escribió: >> > .. > >>> IMO, this goes too far. I would expect it would be useful enough to talk >>> about "multi-address TCP" and avoid talking about "paths" when they're >>> not part of the architecture, nor are they knowable without doing things >>> like pathchar or traceroute. >>> >> But there are many situations and probably very interesting ones, >> where the node has multipaht paths and does not have multiple addresses. >> For instnace, think about a server located in a big multihomed site >> using PI addresses. This server is likely to benefit from multipath TCP >> but it only has a single address. >> > > Then you have different problems, and you need to separate them: > > | single endpoint | multi endpoint > -------------+-------------------+---------------- > single path | sp/se | sp/me > -------------+-------------------+---------------- > multi path | mp/se | mp/me > -------------+-------------------+---------------- > I am confused now, what do you mean by multiple endpoint? So far, i understood we were talking about to endpoints communicating through TCP and where there were multiple paths between them. the question was whether we can use different address pairs (assigned to the two endpoints involved) as path selectors, but i am afriad we went out of synch and you have something else in mind > Existing TCP assumes sp/se. > > If you solve mp/me, you solve sp/me as a degenerate case. > > So basically there are two distinct problems: > > 1) making TCP work when there are multiple paths > > which has *nothing* to do with whether > there are are multiple endpoints or not > > e.g., a single TCP connection can experience > two different paths given multipath routing > well, i think there is andditional case which is to make TCP use the multiple paths simultaneously, which TCP doesn't do currently > 2) making TCP work with multiple endpoints > > which has *nothing* to do with whether > there are multiple paths, since they > different endpoints can easily experience > the same path properties - including > the same bottleneck. You can't assume > that two different IP addresses (or other > TCP endpoint differentiators) result in > paths that do not share properties or > bottlenecks. > > no, i can't be certain that they will not share the smae bottle neck, but what may be done is that if they do have the smae bottleneck, they behave as single TCP flow, while if they do not have the same bottle neck, they actually use BW availbale. > this *does* include striping, i.e., > concurrent use of alternate endpoints > to increase BW; that is NOT the case for > single-address. > why not? It is possible to use a different path selector, set by TCP, that allows different packets belonging to the same TCP connectio to be routed through different paths. > note that 'endpoint' could mean IP address > or port, or anything else that could cause > path properties to vary - keep in mind that > policy routing peeks at the entire IP and often > TCP headers. > > i am having difficuties to match this with the previous sentence... I mean, we agree that we can have a single address and use other fileds than the address as path selector? > Note that when I say "multiple paths" I mean paths with different > characteristics, e.g., MTU, RTT, BW, loss. > > right > IMO, multiendpoint is a potentially important problem that may be ready > to solve via IETF work at this time. However, you need to be careful to > never assume that having multiple endpoints means distinct paths - if > they're not distinct in bottleneck, then you'll end up making TCP fight > with itself. > > exactly, this is critical in this effort. > If you're going to solve multipath, then you need to solve it in the > single endpoint case first, IMO. If you can't solve it there, there's no > point in conflating the issue with multiendpoint. > > i am still having problems to understand your description of the problem, i.e. the multiendpoint vs, the multipath how would you scope the proble only to deal with multipaht and not with multiendpoint? >> In addition, even if you do have multiple addresses, you still need >> some support from the routing infrastrcuture. >> > > IMO, you do not need support from the routing infrastructure at all. TCP > probes to detect path properties without interacting with routing right now. > > You need to separate that out when you get distinct sets of values. All > you need to know is that you are getting a correlation between segments > being sent and varying path properties. > > exactly, that is what i call a path selector a path selector is something in the packets that TCP sets and that other part of the network looks at and sends packets with different paths selectors through different paths. For TCP, all it is needed is that the packets with the same path selector are routed through the same path (or at least as stabel as it is today in the single path case). also, it is useful, that packets with different path selectors are routed through different paths but if this doesn't happen, the systme should work as a single path TCP >> When a node A with multiple addresses sends packets, it will include >> different source addresses, but all packets will carry the same >> destiantion address. >> > > That's ONE way to do this. There are others. > > such as? (this is sincere question. We have been trying to understand how to do this in shim6 for many years and we couldn't find very nice answers) >> How can the nade A influence the egress path? >> > > By varying the source address, source port, dest address, dest port, or > just about anything else in the IP or TCP header. > > right, but in the case the destination node has a single address, all the rest of the fields are not currently cosndiered by routing, so we do require some modification in routing, afaict (but i would love to be convinced otherwise) Regards, marcelo > Joe > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.9 (MingW32) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iEYEARECAAYFAkoEYkIACgkQE5f5cImnZrtEzwCgqTmskXCmmAKi/SJHpw5A5ZRj > eukAnRkO0vPBnEwDf1wzJ1VG1wQs46dz > =1KaA > -----END PGP SIGNATURE----- > > > From marcelo@it.uc3m.es Mon May 11 12:04:15 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id CCA5E3A6CE4 for ; Mon, 11 May 2009 12:04:15 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.358 X-Spam-Level: X-Spam-Status: No, score=-6.358 tagged_above=-999 required=5 tests=[AWL=0.241, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id juNsCFoICkx3 for ; Mon, 11 May 2009 12:04:15 -0700 (PDT) Received: from smtp01.uc3m.es (smtp01.uc3m.es [163.117.176.131]) by core3.amsl.com (Postfix) with ESMTP id A8F153A6A6A for ; Mon, 11 May 2009 12:04:14 -0700 (PDT) Received: from marcelo-bagnulos-macbook-pro.local (166.pool85-53-132.dynamic.orange.es [85.53.132.166]) by smtp01.uc3m.es (Postfix) with ESMTP id 03F37B715CB; Mon, 11 May 2009 21:05:38 +0200 (CEST) Message-ID: <4A087701.7090502@it.uc3m.es> Date: Mon, 11 May 2009 21:05:37 +0200 From: marcelo bagnulo braun User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Preethi Natarajan References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es><4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><20090507230452.GC35382@cisco.com><1241789497.4055.8.camel@nhuynhth-laptop> <001e01c9cfff$bde8f1e0$103947ab@cisco.com> <97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com> <001f01c9d005$0c583110$103947ab@cisco.com> <000001c9d028$aea92640$103947ab@cisco.com> In-Reply-To: <000001c9d028$aea92640$103947ab@cisco.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-TM-AS-Product-Ver: IMSS-7.0.0.3116-5.6.0.1016-16636.001 Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 May 2009 19:04:15 -0000 Preethi Natarajan escribió: > > > >> -----Original Message----- >> From: Iljitsch van Beijnum [mailto:iljitsch@muada.com] >> Sent: Friday, May 08, 2009 1:09 PM >> To: Preethi Natarajan >> Cc: 'Arnaud Ongenae'; multipathtcp@ietf.org >> Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF >> description >> >> Suppose that a browser wants to load pages from a website. >> How does the browser know if the server also supports SCTP? >> For IPv4 vs IPv6 we could solve this using the DNS, but for >> TCP vs SCTP this is harder. >> >> > > DNS has been proposed to identify transport protocols in the past, for ex. > see RFC 3263. From what I hear, SRV records have worked quite well for SIP. > > >> Then we trade this off against the possible gain of not >> having to change middleboxes. However, you assume that >> incompatible middleboxes exist. While I'm sure this will be >> the case a good percentage of the time, it's not a universal truism. >> > > When NATs on different paths have to talk to each other to do the same > translation, I would assume that this requires changes to NAT > implementations... > > >> So I think there is a place for multipath TCP in general and >> the one- ended variety in particular. >> > > I am sure it is. I am just wondering why the draft should be tied to TCP, > when (i) the one-ended multipath transmission could be adapted to any > transport, and (ii) some of the issues listed in > draft-van-beijnum-1e-mp-tcp-00 has been researched in-depth and solved for > another transport. The motivation to consider just TCP the motivation for considering TCP is that the vast majority of traffc uses TCP and not SCTP. that seems to be a fact and it seems that defining multipaht capabilities to TCP would provide additional benefits for those users. > cannot be NAT > traversal, since, MPTCP itself might not work with the existing NAT > implementations? > As described in the proposed BOF, there are two possible approaches to MPTCP, the one ended one and the two ended one the two ended one works fine with nats The one ended one, woudl require modificatiosn to the NATs of the ends that supports it. this means that if a site decides to deploy the one ended version of MPTCP in their servers, they will need tp update their NATs. There is no need to update the NAT on the side that is unchanged. So, changes are only required by the party deploying the solution, which doesn't seem to be the case in SCTP, since afaiu, if the peer has a nat, it no longer works. Regards, marcelo > Preethi > > _______________________________________________ > multipathtcp mailing list > multipathtcp@ietf.org > https://www.ietf.org/mailman/listinfo/multipathtcp > > From marcelo@it.uc3m.es Mon May 11 12:06:47 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 994173A6AFF for ; Mon, 11 May 2009 12:06:47 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.359 X-Spam-Level: X-Spam-Status: No, score=-6.359 tagged_above=-999 required=5 tests=[AWL=0.240, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RYM9CB+0HSew for ; Mon, 11 May 2009 12:06:46 -0700 (PDT) Received: from smtp02.uc3m.es (smtp02.uc3m.es [163.117.176.132]) by core3.amsl.com (Postfix) with ESMTP id 01B623A6A6A for ; Mon, 11 May 2009 12:06:42 -0700 (PDT) Received: from marcelo-bagnulos-macbook-pro.local (166.pool85-53-132.dynamic.orange.es [85.53.132.166]) by smtp02.uc3m.es (Postfix) with ESMTP id A47B465A38E; Mon, 11 May 2009 21:07:49 +0200 (CEST) Message-ID: <4A087784.1040307@it.uc3m.es> Date: Mon, 11 May 2009 21:07:48 +0200 From: marcelo bagnulo braun User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: janardhan.iyengar@fandm.edu References: <4A01D161.7060108@it.uc3m.es> <4A02F0B7.6080202@fandm.edu> <4A04FF8B.2060800@fandm.edu> In-Reply-To: <4A04FF8B.2060800@fandm.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-TM-AS-Product-Ver: IMSS-7.0.0.3116-5.6.0.1016-16636.001 Cc: nasif ekiz , multipathtcp@ietf.org, "Paul D. Amer" , Randall Stewart , Preethi Natarajan , Ertugrul YILMAZ Subject: Re: [multipathtcp] Fwd: I-D Action:draft-van-beijnum-1e-mp-tcp-00.txt]] X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 May 2009 19:06:47 -0000 Janardhan Iyengar escribió: > Hi Illjitsch, > > I will also pull out other related work and add it into the wiki. > >> Right. However, note that RFC 2960 says that only one path is used to >> send traffic, the other path is only used for retransmissions. (I'll >> have to check with RFC 4960.) > > That is correct. > >>> + You list 4 challenges and questions, and say that you address >>> the first two (scheduling, reordering). We have looked extensively >>> at those two and also the third one (receiver buffer constraints), >>> and our results are documented. >> >> Note that for SCTP the send and receive buffer issues are different >> when there are multiple streams. > > Can you say some more about which issues you think may be different? > Any solution for SCTP send/receive buffer issues must handle the > single-stream case, which is essentially collapses, as far as buffer > issues are concerned, down to the issues of an ordered TCP stream. > >>> I like to think that we did quite a bit of work in this area >>> already, and while there are unresolved issues still around, I also >>> like to think that we got some important things down :-) It would >>> be very disheartening to see work that is pretty much exactly the >>> same not using our previous work. >> >> I agree. But SCTP results don't always translate into TCP results. >> And there's still plenty of work to do to apply the research to >> implementations. > > As far as implementation is concerned, CMT has been part of FreeBSD > since FreeBSD 7.0 (sysctls net.inet.sctp.cmt*). And if you count > simulators, CMT is part of the ns2 simulator (ns/sctp/). > > As far as TCP/SCTP differences are concerned: > Pulling words from the BoF announcement, the first proposed work item > of multipath-TCP is: > > "A set of architectural guidelines for congestion dependent > multipath transport protocols. Since different transport protocols > can potentially benefit from this approach, this document describes > the motivations and the general approach that should be followed > to enable congestion dependent multipath transport." > > Surely SCTP falls into the category described here. Architecturally, I > expect that there will be a significant amount of overlap; it doesn't > matter if it is TCP or SCTP. Our design is one point in the design > space of multipath transport. > > I fully agree that the devil in the details will have to be dealt > with, and there will be many differences there between TCP and SCTP; > we don't have to go very far, SACK differences are significant > enough! Having said that, I expect that *at this stage*, when we are > probably still having architectural discussions, there will be more > similarities than differences. > i agree that it would be possible to have at least one part of the arch document to be common there may also be other points on common, such as the interaction with the multipaht routing. Regards, marcelo > - jana > From marcelo@it.uc3m.es Mon May 11 12:08:32 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 28CAD3A6E09 for ; Mon, 11 May 2009 12:08:32 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.36 X-Spam-Level: X-Spam-Status: No, score=-6.36 tagged_above=-999 required=5 tests=[AWL=0.239, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5hMFdNqs0Jvz for ; Mon, 11 May 2009 12:08:31 -0700 (PDT) Received: from smtp02.uc3m.es (smtp02.uc3m.es [163.117.176.132]) by core3.amsl.com (Postfix) with ESMTP id EB0A33A6D39 for ; Mon, 11 May 2009 12:08:30 -0700 (PDT) Received: from marcelo-bagnulos-macbook-pro.local (166.pool85-53-132.dynamic.orange.es [85.53.132.166]) by smtp02.uc3m.es (Postfix) with ESMTP id 240F46B9CC9; Mon, 11 May 2009 21:09:36 +0200 (CEST) Message-ID: <4A0877EF.7090306@it.uc3m.es> Date: Mon, 11 May 2009 21:09:35 +0200 From: marcelo bagnulo braun User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: janardhan.iyengar@fandm.edu References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es><4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><20090507230452.GC35382@cisco.com><1241789497.4055.8.camel@nhuynhth-laptop> <001e01c9cfff$bde8f1e0$103947ab@cisco.com> <97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com> <001f01c9d005$0c583110$103947ab@cisco.com> <4A051B25.4040702@fandm.edu> In-Reply-To: <4A051B25.4040702@fandm.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-TM-AS-Product-Ver: IMSS-7.0.0.3116-5.6.0.1016-16636.001 Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 May 2009 19:08:32 -0000 Janardhan Iyengar escribió: > Hi Iljitsch, > >> We know for sure that if applications are going to switch from TCP to >> SCTP to gain multipath benefits, this creates deployment issues. >> Suppose that a browser wants to load pages from a website. How does >> the browser know if the server also supports SCTP? For IPv4 vs IPv6 >> we could solve this using the DNS, but for TCP vs SCTP this is harder. > > Simultaneous TCP and SCTP connects. Yes, it is a hack, but it is a > solution. > >> Also, I believe current SCTP needs the application to tell it the >> addresses for the other side. > > SCTP peers exchange information about all addresses in the handshake; > the application *does not* tell the initiator all the addresses for > the other side. The API is exactly the same as with TCP/UDP; in fact, > Ryan Bickhart's thesis (referenced in an earlier email by Arnaud) > exploited exactly this idea to give multihoming and fault tolerance > benefits to any application by transparently mapping application-level > TCP socket calls to SCTP function calls in-kernel. > >> Then there is the issue that SCTP must of course be implemented on >> both sides, while the big feature in the one-ended MPTCP is that it >> can work if it's only implemented in the sender. > > > But to be useful, one-ended MPTCP will need to be understood by > routers/middleboxes. right, but only by the middleboxes in control of the side supporting the OmTCP. I don't think there is need for middlebox support on the side that is not MTCP capable. So, the whole point is that if one end updates their servers and potentially their middleboxes, they can support this. Regards, marcelo > > >> Then we trade this off against the possible gain of not having to >> change middleboxes. However, you assume that incompatible middleboxes >> exist. While I'm sure this will be the case a good percentage of the >> time, it's not a universal truism. >> >> And you assume that this same middlebox would be compatible with >> SCTP. That is a signficant leap of faith, work for SCTP NAT is still >> going on in the behave wg. > > The ongoing work, if I am right, is for supporting multihoming, not > basic support (which is trivial). And as I see it, that same work will > become the template for multipathTCP as well, when you move to the > "two-ended" mulipath case. > > - jana > From marcelo@it.uc3m.es Mon May 11 12:17:23 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id D4C803A6F5E for ; Mon, 11 May 2009 12:17:23 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.362 X-Spam-Level: X-Spam-Status: No, score=-6.362 tagged_above=-999 required=5 tests=[AWL=0.237, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cP40CyjyHbKM for ; Mon, 11 May 2009 12:17:22 -0700 (PDT) Received: from smtp03.uc3m.es (smtp03.uc3m.es [163.117.176.133]) by core3.amsl.com (Postfix) with ESMTP id 54DFE3A6F5C for ; Mon, 11 May 2009 12:17:22 -0700 (PDT) Received: from marcelo-bagnulos-macbook-pro.local (166.pool85-53-132.dynamic.orange.es [85.53.132.166]) by smtp03.uc3m.es (Postfix) with ESMTP id E41317EF012; Mon, 11 May 2009 21:18:50 +0200 (CEST) Message-ID: <4A087A1A.4090303@it.uc3m.es> Date: Mon, 11 May 2009 21:18:50 +0200 From: marcelo bagnulo braun User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: janardhan.iyengar@fandm.edu References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es><4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><20090507230452.GC35382@cisco.com><1241789497.4055.8.camel@nhuynhth-laptop> <001e01c9cfff$bde8f1e0$103947ab@cisco.com> <97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com> <001f01c9d005$0c583110$103947ab@cisco.com> <000001c9d028$aea92640$103947ab@cisco.com> <4A051E47.7090400@fandm.edu> In-Reply-To: <4A051E47.7090400@fandm.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-TM-AS-Product-Ver: IMSS-7.0.0.3116-5.6.0.1016-16636.001 Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 May 2009 19:17:23 -0000 Janardhan Iyengar escribió: > > A couple of thoughts on the draft/BoF: > > 1/ "Two-ended" multipath, where both endpoints are aware: There are > several architectural and design considerations that will be common to > TCP, SCTP, DCCP, SST etc., and as I see it, there is no reason why > this discussion should not include all transports. > right, i agree with that > My proposal: scope the BoF more broadly. Interested parties can then > speak up about multipath for different transports and different points > in the design space. There will be shared goals, and lessons to be > learned from one another. Perhaps there can be a shared design > considerations document, right, that seems in the scope for the proposed charter imho > and then separate documents per transport. Perhaps two per transport. > i am not sure about this one. I am not sure if a single group would be the ideal place to discuss all transports > > 2/ "One-ended" multipath seems very unclear to me... there is benefit > in employing this "path" option in *any* transport. This should be a > transport-neutral option, below the traditional transport. (Perhaps > in the shim6 shim layer? I am not sure yet...) I am not convinced > that there is a point in having this as a *TCP* option. mmm, the comment about the TCP options was general in the charter. I mean, the sentence that says that we can only do tcp options applies to both two ended and single ended. In other words, it is scoping the kind of changes we can do to TCP, in general. > I also do not understand how this is on the deployment path to > two-ended TCP. They seem like two different beasts: one of them needs > router support to get going, the other one needs endpoint/NAT support > to get going. which one is which? any transport that handles multiple paths need some way to send different packets throug different path, this is true for MPTCP (one ended or two ended), tru for SCTP and true for shim6 If multiple addresses are used, somehtings work, but still we need to deal with the case of changing the source address that needs to have some impact on routing. I mean, the two ended case, need updating both ends and probably some routing of the involved sites, the one ended flavour, needs updating one side, including the server, the routing and the middle boxes. They have different deployment vectors. In the one ended case, the site that deploys it, gets inmediate benefit (while more limited than the two ended version) In the two ended version, more benefits can be achived, but needs both ends supporting it. > Don't get me wrong: I love the idea of having signaling between > endpoints and routers to enable more control over multipath scheduling > at the transport, but that seems like a different trajectory than the > "two-ended" multipath. not really, what is needed is that the routing take into account something defined by the MPTCP layer in order to define the egress router. > Others have said this, and I agree that "one-sided" multipath with > router support does not seem to be scoped right... it seems to belong > elsewhere, under the transport and independent of it. Or am I missing > something? > mmm, have you read the draft? regards, marcelo > - jana > From marcelo@it.uc3m.es Mon May 11 12:24:24 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id C151D3A6946 for ; Mon, 11 May 2009 12:24:24 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.363 X-Spam-Level: X-Spam-Status: No, score=-6.363 tagged_above=-999 required=5 tests=[AWL=0.236, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2Uby7X9MTuiU for ; Mon, 11 May 2009 12:24:23 -0700 (PDT) Received: from smtp01.uc3m.es (smtp01.uc3m.es [163.117.176.131]) by core3.amsl.com (Postfix) with ESMTP id 07D403A692F for ; Mon, 11 May 2009 12:24:22 -0700 (PDT) Received: from marcelo-bagnulos-macbook-pro.local (166.pool85-53-132.dynamic.orange.es [85.53.132.166]) by smtp01.uc3m.es (Postfix) with ESMTP id E7D12B73D16; Mon, 11 May 2009 21:25:51 +0200 (CEST) Message-ID: <4A087BBF.6010005@it.uc3m.es> Date: Mon, 11 May 2009 21:25:51 +0200 From: marcelo bagnulo braun User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Bryan Ford References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <20090507230452.GC35382@cisco.com> <1241789497.4055.8.camel@nhuynhth-laptop> <001e01c9cfff$bde8f1e0$103947ab@cisco.com> <97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com> <001f01c9d005$0c583110$103947ab@cisco.com> <1241818843.26211.67.camel@nhuynhth-laptop> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-TM-AS-Product-Ver: IMSS-7.0.0.3116-5.6.0.1016-16636.001 Cc: multipathtcp@ietf.org, Olivier Bonaventure Subject: Re: [multipathtcp] TCP modification vs switching to SCTP X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 May 2009 19:24:24 -0000 Bryan Ford escribió: > On May 8, 2009, at 11:40 PM, Arnaud Ongenae wrote: >> A new thread for a question out of the scope of >> "Draft Multipath TCP (MPTCP) BOF description" >> >> TCP modification vs switching to SCTP (if it has already been discussed >> elsewhere do not hesitate to redirect me) > > I think this "multipath TCP versus multipath SCTP" question is > important, but still misses a larger issue: SCTP and TCP are different > transports providing substantially different application-visible > semantics, exactly! > but we would really like to have multipath support for _both_ of those > transports (and perhaps others). sure > It's a pain for TCP applications to switch to SCTP because it involves > programmers learning a new API with different semantics: e.g., the > application has to break byte streams up into (reasonable-size) > messages and use different system calls to tag them with a stream > number and send them on an SCTP connection. On the other hand, SCTP > provides features that some applications really want that TCP doesn't, > such as a partially-ordered connection where one stream's data can > pass another stream's. Furthermore, these semantics are pretty much > totally orthogonal to multipath issues: the transport functionality > that needs to be modified for multipath is fairly independent of and > readily separable from the transport functionality that provides > application-visible semantics like TCP's byte streams or SCTP's > message-oriented multi-streams. The modifications required for > multipath support in general mainly affect the transport's congestion > control functionality, and many research projects and deployed > protocols have demonstrated how congestion control can be cleanly > separated from the application-semantic part of the transport: e.g., > the Congestion Manager architecture, several prior variants of TCP and > SCTP (e.g., Split-TCP, pTCP, mTCP, LS-SCTP), and my own Structured > Stream Transport (SST). > > So if we're going to start a BOF or WG about multipath > something-or-other, why should it be restricted to adding multipath to > one particular transport (and then get into debates about which > transport that should be), i don't think we need that debate As you say, it is a good idea to have them in all the transport protocols. > when what we really need is a clean way to add multipath to _any_ > transport? Whether to do that by literally splitting the transport > into multiple layers, as the HotNets-08 paper Jana and I wrote > proposes (http://brynosaurus.com/pub/net/logjam-abs.html), or simply > by developing a principled approach to adding multipath support > incrementally to existing "monolithic" transports while considering > the needs of multiple transports, amounts to technical details that > need to be discussed by whatever BOF or WG gets formed. Expanding the > BOF charter's scope from "multipath TCP" to "multipath transport" > wouldn't even need to mean that all transports will necessarily get > equal attention or that solutions for each transport would be > developed in lock-step; it just means recognizing that there is more > than one transport that applications and users care about and that > would like multipath support. there is a tradeoff here between having enough arhcitectural perspective and having a charter so broad that is impossible to have concrete output. I perosnally think that a reasonable middleground is what Jana has proposed. We can have an architectural document about multipath transports in general, and focus this partiuclar effort in multiapth tcp. If after, based on the architectural document, other efforts for other transport protoco spring, it woudl be a great otucome. I am very concerned that if we loose the focus of the work, the effort will get nowhere Regards, marcelo > > > Bryan > >> I believe that by introducing this modification to TCP (one-end MTCP) >> you will give argument to those who do not want to switch to SCTP to >> stay on TCP. I think that we should create an hybrid transition protocol >> to produce the need to switch from TCP. This hybrid protocol exist and >> was presented in : >> Transparent TCP-to-SCTP translation shim layer >> from RW Bickhart thesis [1] >> >> Why not concentrate multipath effort on SCTP which has the benefit to >> natively support multihomed host and as Jana said research as been done >> on the field. There are more than CMT, while doing my master thesis in >> the field I discover LS-SCTP[2] or cmpSCTP[3] proposing different >> interesting solutions to multipath issues. >> >> When you take a look at this website (that you certainly know): >> http://www.isoc.org/briefings/017/ >> you realize that using SCTP is more than simple multihomed capabilities. >> I, personally, believe that letting SCTP to be alone to handle multipath >> could boost his integration in the current internet. >> >> By defending TCP modification, don't you think that those new protocols >> could play the same role as NAT for IPv4 address starvation ? Maybe it's >> an hard comparison but I miss IPv6. >> >> >> On Fri, 2009-05-08 at 22:08 +0200, Iljitsch van Beijnum wrote: >>> On 8 mei 2009, at 19:47, Preethi Natarajan wrote: >>> >>>>> The motivations are better performance and redundancy without >>>>> requiring application changes or any changes at the receiver. >>> >>>> Do you mean the application mods to open an SCTP (or whatever) socket >>>> instead of TCP socket vs. mods to TCP stacks and middleboxes? >>> >>> We know for sure that if applications are going to switch from TCP to >>> SCTP to gain multipath benefits, this creates deployment issues. >>> Suppose that a browser wants to load pages from a website. How does >>> the browser know if the server also supports SCTP? For IPv4 vs IPv6 we >>> could solve this using the DNS, but for TCP vs SCTP this is harder. >> >> absolutely, this is not a trivial issue >>> >>> Also, I believe current SCTP needs the application to tell it the >>> addresses for the other side. Then there is the issue that SCTP must >>> of course be implemented on both sides, while the big feature in the >>> one-ended MPTCP is that it can work if it's only implemented in the >>> sender. >>> >>> Then we trade this off against the possible gain of not having to >>> change middleboxes. However, you assume that incompatible middleboxes >>> exist. While I'm sure this will be the case a good percentage of the >>> time, it's not a universal truism. >>> >>> And you assume that this same middlebox would be compatible with SCTP. >> >> should be compatible :( >> >>> >>> That is a signficant leap of faith, work for SCTP NAT is still going >>> on in the behave wg. >> >> NAT is a poison on the internet (to me) >> >>> >>> So I think there is a place for multipath TCP in general and the one- >>> ended variety in particular. >> >> If you only consider multipath, yes, there is a place, especially if you >> want the benefit as soon as possible, because this new protocol work >> without modifying one end, you may implement it for yourself and already >> use it. >> >> >> regards, >> >> Arnaud Ongenae >> UCL (Belgium) student >> >> >> [1] http://www.cis.udel.edu/~amer/PEL/poc/pdf/BickhartMSthesis.pdf >> >> [2] Ahmed Abd El Al, Tarek Saadawi, and Myung Lee. Ls-sctp: a bandwidth >> aggregation technique for stream control transmission protocol. Computer >> Communications, >> 27(10):1012–1024, 2004. Protocol Engineering for Wired and Wireless >> Networks. >> >> [3] J. Liao, J. Wang, and X. Zhu. A multi-path mechanism for reliable >> voip transmission >> over wireless networks. Comput. Netw., 52(13):2450–2460, 2008. >> >> _______________________________________________ >> multipathtcp mailing list >> multipathtcp@ietf.org >> https://www.ietf.org/mailman/listinfo/multipathtcp > > _______________________________________________ > multipathtcp mailing list > multipathtcp@ietf.org > https://www.ietf.org/mailman/listinfo/multipathtcp > From iljitsch@muada.com Mon May 11 12:32:16 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id E41843A68D7 for ; Mon, 11 May 2009 12:32:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2QiI5kuX8pSz for ; Mon, 11 May 2009 12:32:16 -0700 (PDT) Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by core3.amsl.com (Postfix) with ESMTP id 70BE628C170 for ; Mon, 11 May 2009 12:31:52 -0700 (PDT) Received: from [192.168.2.4] (static-167-138-7-89.ipcom.comunitel.net [89.7.138.167] (may be forged)) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n4BJWtdK045146 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Mon, 11 May 2009 21:32:56 +0200 (CEST) (envelope-from iljitsch@muada.com) Message-Id: From: Iljitsch van Beijnum To: "Preethi Natarajan" In-Reply-To: <000b01c9d250$a9c311c0$103947ab@cisco.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Mon, 11 May 2009 21:32:59 +0200 References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es><4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><20090507230452.GC35382@cisco.com><1241789497.4055.8.camel@nhuynhth-laptop><001e01c9cfff$bde8f1e0$103947ab@cisco.com><97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com><001f01c9d005$0c583110$103947ab@cisco.com><1241818843.26211.67.camel@nhuynhth-laptop> <0D6E7BC4-17AF-4F5C-9107-ABA2B729683F@muada.com> <000b01c9d250$a9c311c0$103947ab@cisco.com> X-Mailer: Apple Mail (2.930.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] TCP/SCTP and BoF description, was: Re: TCP modification vs switching to SCTP X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 May 2009 19:32:17 -0000 On 11 mei 2009, at 17:53, Preethi Natarajan wrote: > 1. CRC32c might be an intensive operation. Nonetheless CRC32c was > chosen for > its superior error detection mechanisms. I understand. However, there are times when spending CPU cycles (I'm assuming the SCTP checksum calculation can't be offloaded to hardware) needed to do this are not worth the benefit it provides. > My larger point -- why are _we_ trying to decide which transport > would be > best for today's/tomorrow's requirements? Instead, I would prefer > that the > consumers (app developers) make that decision. Certainly, I agree. > I was under the impression that nobody has even tried to do this with > SCTP... there are a few upcoming proposals but thats about it? Like I said, SRV records only have a port number, not a protocol number. >> So if this BoF results in a wg, and that wg would take on >> this work for both TCP and SCTP, how would that work? Would >> we have different documents for TCP and for SCTP, or one >> document? Is anyone volunteering to spend time on this for SCTP? > I know that there are volunteers for SCTP, and am sure there will be > volunteers for other transports as well. Ok, let's think about this. With regard to the single ended case: in addition to the multipath congestion control that is common to all types of multipath with "normal" congestion control, the tricky parts here are how to handle fast retransmit, how to handle stalls because of receive buffer depletion and correlating incoming ACKs with the path the ACKed packets were sent over. Without having studied SCTP in detail, it seems to me that the one- ended multipath TCP draft could easily be applied to SCTP with minor changes. This could be done during the TCP work or immediately after so a single document can apply to both if there are no incompatibilities. If the SCTP stuff requires a lot of extra text, it probably makes more sense to make a separate document. AS for the two-ended multiaddress multipath: SCTP already does all the protocol work, the only thing that's needed is apply the multipath congestion control. Iljitsch From john@jlc.net Mon May 11 12:49:54 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 775DC3A686C for ; Mon, 11 May 2009 12:49:54 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.853 X-Spam-Level: X-Spam-Status: No, score=-5.853 tagged_above=-999 required=5 tests=[AWL=0.746, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id J6rMZMY4uXRZ for ; Mon, 11 May 2009 12:49:53 -0700 (PDT) Received: from mailhost.jlc.net (mailhost.jlc.net [199.201.159.9]) by core3.amsl.com (Postfix) with ESMTP id 939453A6A98 for ; Mon, 11 May 2009 12:49:53 -0700 (PDT) Received: by mailhost.jlc.net (Postfix, from userid 104) id BE3C633C26; Mon, 11 May 2009 15:51:22 -0400 (EDT) Date: Mon, 11 May 2009 15:51:22 -0400 From: John Leslie To: marcelo bagnulo braun Message-ID: <20090511195122.GA14375@verdi> References: <4A031B0F.4030606@employees.org> <001e01c9cfff$bde8f1e0$103947ab@cisco.com> <97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com> <001f01c9d005$0c583110$103947ab@cisco.com> <000001c9d028$aea92640$103947ab@cisco.com> <4A087701.7090502@it.uc3m.es> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4A087701.7090502@it.uc3m.es> User-Agent: Mutt/1.4.1i Cc: multipathtcp@ietf.org Subject: [multipathtcp] Working through NATs X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 May 2009 19:49:54 -0000 marcelo bagnulo braun wrote: > > As described in the proposed BOF, there are two possible approaches to > MPTCP, the one ended one and the two ended one: > the two ended one works fine with nats I quite agree, though I'm not sure we have any consensus on that. As I see it, so long as each "path" is distinguished by a unique source/destination pair, the routing structure can do all the work, and we can even arrange for SYNs to travel "first" outbound through any NAT(s). > The one ended one, would require modifications to the NATs of the ends > that supports it. I'm not quite that pessimistic. The one-ended version can't, of course, distinguish paths by source/destination pairs, but the chief implementation I see would be content providers with two (or more) upstream paths and no NATs in them. The content-receiving end could have an unmodified NAT, which would be blissfully ignorant of which path a packet takes. It's not immediately clear to me that there's any point in designing special-purpose NATs for the one-ended case... > this means that if a site decides to deploy the one ended version of > MPTCP in their servers, they will need to update their NATs. Again, I'm not quite so pessimistic; but indeed, if a content-provider has a NAT on one or more upstream paths, it would need to find what address/port that NAT translates to. This may be challenging for an unmodified NAT; but it's not impossible by definition... > There is no need to update the NAT on the side that is unchanged. Agreed. -- John Leslie From touch@ISI.EDU Mon May 11 13:38:08 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id C9F743A6B95 for ; Mon, 11 May 2009 13:38:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id W++IEfSmNkKU for ; Mon, 11 May 2009 13:38:07 -0700 (PDT) Received: from vapor.isi.edu (vapor.isi.edu [128.9.64.64]) by core3.amsl.com (Postfix) with ESMTP id 564E33A683E for ; Mon, 11 May 2009 13:38:07 -0700 (PDT) Received: from [172.20.32.138] (Extlanl3840-01.nsf.gov [198.181.231.14]) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id n4BKdDWP012606; Mon, 11 May 2009 13:39:15 -0700 (PDT) Message-ID: <4A088CF1.6070201@isi.edu> Date: Mon, 11 May 2009 13:39:13 -0700 From: Joe Touch User-Agent: Thunderbird 2.0.0.21 (Windows/20090302) MIME-Version: 1.0 To: marcelo bagnulo braun References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <20090507230452.GC35382@cisco.com> <4A036A29.7020005@isi.edu> <4A042F28.5050107@it.uc3m.es> <4A046242.4040104@isi.edu> <4A087396.4050001@it.uc3m.es> In-Reply-To: <4A087396.4050001@it.uc3m.es> X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 May 2009 20:38:08 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 > Joe Touch escribió: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> >> >>> Joe Touch escribió: >>> >> .. >> >>>> IMO, this goes too far. I would expect it would be useful enough to talk >>>> about "multi-address TCP" and avoid talking about "paths" when they're >>>> not part of the architecture, nor are they knowable without doing things >>>> like pathchar or traceroute. >>>> >>> But there are many situations and probably very interesting ones, >>> where the node has multipaht paths and does not have multiple addresses. >>> For instnace, think about a server located in a big multihomed site >>> using PI addresses. This server is likely to benefit from multipath TCP >>> but it only has a single address. >>> >> >> Then you have different problems, and you need to separate them: >> >> | single endpoint | multi endpoint >> -------------+-------------------+---------------- >> single path | sp/se | sp/me >> -------------+-------------------+---------------- >> multi path | mp/se | mp/me >> -------------+-------------------+---------------- >> > > I am confused now, what do you mean by multiple endpoint? More than one socket pair - more than one port and/or address. > So far, i understood we were talking about to endpoints communicating > through TCP and where there were multiple paths between them. the > question was whether we can use different address pairs (assigned to the > two endpoints involved) as path selectors, but i am afriad we went out > of synch and you have something else in mind You can have different paths when you have different: - endpoint addresses - port numbers - or neither Paths do not have anything to do with addresses per se. >> Existing TCP assumes sp/se. >> >> If you solve mp/me, you solve sp/me as a degenerate case. >> >> So basically there are two distinct problems: >> >> 1) making TCP work when there are multiple paths >> >> which has *nothing* to do with whether >> there are are multiple endpoints or not >> >> e.g., a single TCP connection can experience >> two different paths given multipath routing >> > well, i think there is andditional case which is to make TCP use the > multiple paths simultaneously, which TCP doesn't do currently That's case (1) above. A single TCP connection can already see - and use - - multiple paths simultaneously, and trips badly when it happens. E.g., if long packets go one way and short ones go another, this will mess up TCP's RTT calculation. >> 2) making TCP work with multiple endpoints >> >> which has *nothing* to do with whether >> there are multiple paths, since they >> different endpoints can easily experience >> the same path properties - including >> the same bottleneck. You can't assume >> that two different IP addresses (or other >> TCP endpoint differentiators) result in >> paths that do not share properties or >> bottlenecks. >> >> > no, i can't be certain that they will not share the smae bottle neck, > but what may be done is that if they do have the smae bottleneck, they > behave as single TCP flow, while if they do not have the same bottle > neck, they actually use BW availbale. When they share a bottleneck, they can easily end up with WORSE than a single TCP of BW, because the two TCP 'sides' can fight with each other. You'd be probing the bottleneck from both sides, and a probe on one side can affect the other, etc. >> this *does* include striping, i.e., >> concurrent use of alternate endpoints >> to increase BW; that is NOT the case for >> single-address. >> > why not? > > It is possible to use a different path selector, set by TCP, that > allows different packets belonging to the same TCP connectio to be > routed through different paths. Yes - that can increase path BW, but not endpoint BW, AFAICT. >> note that 'endpoint' could mean IP address >> or port, or anything else that could cause >> path properties to vary - keep in mind that >> policy routing peeks at the entire IP and often >> TCP headers. >> >> > i am having difficuties to match this with the previous sentence... I > mean, we agree that we can have a single address and use other fileds > than the address as path selector? Yes, we're agreeing. >> Note that when I say "multiple paths" I mean paths with different >> characteristics, e.g., MTU, RTT, BW, loss. >> >> > right > >> IMO, multiendpoint is a potentially important problem that may be ready >> to solve via IETF work at this time. However, you need to be careful to >> never assume that having multiple endpoints means distinct paths - if >> they're not distinct in bottleneck, then you'll end up making TCP fight >> with itself. >> >> > exactly, this is critical in this effort. > >> If you're going to solve multipath, then you need to solve it in the >> single endpoint case first, IMO. If you can't solve it there, there's no >> point in conflating the issue with multiendpoint. >> >> > i am still having problems to understand your description of the > problem, i.e. the multiendpoint vs, the multipath Multi-endpoint uses different socket pairs at one or more endpoint. Multipath experiences two different paths at the same time. Those two paths may or may not be correlated with the use of different endpoints. > how would you scope the proble only to deal with multipaht and not with multiendpoint? You make TCP figure out 'path' by 'path characteristics', not by assuming it from differences in endpoint IDs. Which is important, because you can use different endpoint IDs and end up with the same path. You need to find out path agnostically. >>> In addition, even if you do have multiple addresses, you still need >>> some support from the routing infrastrcuture. >>> >> >> IMO, you do not need support from the routing infrastructure at all. TCP >> probes to detect path properties without interacting with routing right now. >> >> You need to separate that out when you get distinct sets of values. All >> you need to know is that you are getting a correlation between segments >> being sent and varying path properties. >> >> > > exactly, that is what i call a path selector > a path selector is something in the packets that TCP sets and that > other part of the network looks at and sends packets with different > paths selectors through different paths. For TCP, all it is needed is > that the packets with the same path selector are routed through the > same path (or at least as stabel as it is today in the single path > case). That's reasonable to assume, but isn't always true. > also, it is useful, that packets with different path > selectors are routed through different paths but if this doesn't > happen, the systme should work as a single path TCP That's not reasonable to assume. >>> When a node A with multiple addresses sends packets, it will include >>> different source addresses, but all packets will carry the same >>> destiantion address. >>> >> >> That's ONE way to do this. There are others. >> >> > > such as? > (this is sincere question. We have been trying to understand how to > do this in shim6 for many years and we couldn't find very nice answers) Use different source ports. Use different destination addresses. Use different destination ports. >>> How can the nade A influence the egress path? >>> >> >> By varying the source address, source port, dest address, dest port, or >> just about anything else in the IP or TCP header. >> >> > right, but in the case the destination node has a single address, all > mean, we agree that we can have a single address and use other fileds > than the address as path selector? Yes. What I'm having problems with is assuming that different selectors - - of any type - can assume different paths. Joe > > Regards, marcelo > > >> Joe >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v1.4.9 (MingW32) >> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org >> >> iEYEARECAAYFAkoEYkIACgkQE5f5cImnZrtEzwCgqTmskXCmmAKi/SJHpw5A5ZRj >> eukAnRkO0vPBnEwDf1wzJ1VG1wQs46dz >> =1KaA >> -----END PGP SIGNATURE----- >> >> >> -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkoIjPAACgkQE5f5cImnZrvc7gCgjfAKgarnOnbNHrEfNjmV7s7F u24AnAz3LgMqhvxfD6J4RvJvQTEixtF0 =UoHK -----END PGP SIGNATURE----- From iljitsch@muada.com Mon May 11 13:59:02 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id A28DB28C16A for ; Mon, 11 May 2009 13:59:02 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GE5LZG6NKJkq for ; Mon, 11 May 2009 13:59:01 -0700 (PDT) Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by core3.amsl.com (Postfix) with ESMTP id 7D3253A6D8B for ; Mon, 11 May 2009 13:59:01 -0700 (PDT) Received: from [192.168.2.4] (static-167-138-7-89.ipcom.comunitel.net [89.7.138.167] (may be forged)) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n4BL045A045812 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Mon, 11 May 2009 23:00:06 +0200 (CEST) (envelope-from iljitsch@muada.com) Message-Id: <3D4CFA5B-A4CD-43C1-BCC4-57BA3383466B@muada.com> From: Iljitsch van Beijnum To: Joe Touch In-Reply-To: <4A088CF1.6070201@isi.edu> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Mon, 11 May 2009 23:00:08 +0200 References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <20090507230452.GC35382@cisco.com> <4A036A29.7020005@isi.edu> <4A042F28.5050107@it.uc3m.es> <4A046242.4040104@isi.edu> <4A087396.4050001@it.uc3m.es> <4A088CF1.6070201@isi.edu> X-Mailer: Apple Mail (2.930.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 May 2009 20:59:02 -0000 On 11 mei 2009, at 22:39, Joe Touch wrote: > Paths do not have anything to do with addresses per se. [...] > When they share a bottleneck, they can easily end up with WORSE than a > single TCP of BW, because the two TCP 'sides' can fight with each > other. No, that shouldn't reduce throughput unless there is synchronization. In fact, in the classic sawtooth case it should even improve throughput somewhat. But are you saying that when MPTCP invokes a number of paths, we MUST be able to determine which ones share bottlenecks? Detecting identical paths shouldn't be a huge problem: here packets from different subflows should (almost) never be reordered relative to each other. However, if the paths only partially overlap the part that's different can still introduce reordering so looking at that won't help. And despite the paper that (I think) Jana posted a link to a few days ago, I don't think correlating the losses is robust enough to detect the shared bottlenecks. I guess it would be possible to back off on one path and then see what happens to others and repeat this until it's clear whether or not there is a strong correlation, but this would take significant time, be fairly complex, and not work very well on links with very many sessions where the impact of a few subflows isn't visible above the noise. So I think we should proceed under the assumption that shared bottlenecks will happen and multipath congestion control should behave in a way that isn't unduly harmful when it does. If only we had an internet with different service classes, we could relegate some sessions to a lower than best effort class so we don't get in the way of normal traffic but still have the extra bandwidth when it's available. BTW, I would prefer to stay away from the "endpoint" terminology because it's not entirely clear what it means. In shim6 a path is defined as a (source, dest) address pair. This is probably not a good use of the word in the context here, but "address pair" is very clear. Rather than just IP addresses we can also talk about transport addresses, which is address + port. A logical path is whatever we put a labal on, whether it maps to something different than another logical path or not. I've also used "physical path" to talk about the actual route that packets follow through the network although this is of course on layer 3 not layer 1. From touch@ISI.EDU Mon May 11 14:09:18 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id DF03E3A68AE for ; Mon, 11 May 2009 14:09:18 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id I8eEs8g0P3Hl for ; Mon, 11 May 2009 14:09:17 -0700 (PDT) Received: from vapor.isi.edu (vapor.isi.edu [128.9.64.64]) by core3.amsl.com (Postfix) with ESMTP id CCB6D3A67EA for ; Mon, 11 May 2009 14:09:17 -0700 (PDT) Received: from [172.20.32.138] (Extlanl3840-01.nsf.gov [198.181.231.14]) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id n4BLAMRV020468; Mon, 11 May 2009 14:10:24 -0700 (PDT) Message-ID: <4A08943D.1080100@isi.edu> Date: Mon, 11 May 2009 14:10:21 -0700 From: Joe Touch User-Agent: Thunderbird 2.0.0.21 (Windows/20090302) MIME-Version: 1.0 To: Iljitsch van Beijnum References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <20090507230452.GC35382@cisco.com> <4A036A29.7020005@isi.edu> <4A042F28.5050107@it.uc3m.es> <4A046242.4040104@isi.edu> <4A087396.4050001@it.uc3m.es> <4A088CF1.6070201@isi.edu> <3D4CFA5B-A4CD-43C1-BCC4-57BA3383466B@muada.com> In-Reply-To: <3D4CFA5B-A4CD-43C1-BCC4-57BA3383466B@muada.com> X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 May 2009 21:09:19 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Iljitsch van Beijnum wrote: > On 11 mei 2009, at 22:39, Joe Touch wrote: > >> Paths do not have anything to do with addresses per se. > > [...] > >> When they share a bottleneck, they can easily end up with WORSE than a >> single TCP of BW, because the two TCP 'sides' can fight with each other. > > No, that shouldn't reduce throughput unless there is synchronization. In > fact, in the classic sawtooth case it should even improve throughput > somewhat. > > But are you saying that when MPTCP invokes a number of paths, we MUST be > able to determine which ones share bottlenecks? Yes. > Detecting identical paths shouldn't be a huge problem: here packets from > different subflows should (almost) never be reordered relative to each > other. > > However, if the paths only partially overlap the part that's different > can still introduce reordering so looking at that won't help. Right. > And despite the paper that (I think) Jana posted a link to a few days > ago, I don't think correlating the losses is robust enough to detect the > shared bottlenecks. > > I guess it would be possible to back off on one path and then see what > happens to others and repeat this until it's clear whether or not there > is a strong correlation, but this would take significant time, be fairly > complex, and not work very well on links with very many sessions where > the impact of a few subflows isn't visible above the noise. > > So I think we should proceed under the assumption that shared > bottlenecks will happen and multipath congestion control should behave > in a way that isn't unduly harmful when it does. Yes. Why does that not mean that you're back to single-path TCP behavior? > If only we had an internet with different service classes, we could > relegate some sessions to a lower than best effort class so we don't get > in the way of normal traffic but still have the extra bandwidth when > it's available. > > BTW, I would prefer to stay away from the "endpoint" terminology because > it's not entirely clear what it means. It has a known meaning in TCP - I'm talking about a socket (an address/port pair). Joe -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkoIlD0ACgkQE5f5cImnZrv5NQCgsJfZwcZyjAbOw5y2hpaq21/B 7D0AoN2Sm0ytMGAiR9Nx4eq2VOQWF5cI =sbcI -----END PGP SIGNATURE----- From marcelo@it.uc3m.es Mon May 11 15:21:34 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 7F5A93A69C3 for ; Mon, 11 May 2009 15:21:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.364 X-Spam-Level: X-Spam-Status: No, score=-6.364 tagged_above=-999 required=5 tests=[AWL=0.235, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id K8ReSdx9XGXv for ; Mon, 11 May 2009 15:21:32 -0700 (PDT) Received: from smtp03.uc3m.es (smtp03.uc3m.es [163.117.176.133]) by core3.amsl.com (Postfix) with ESMTP id 8E5603A6A1C for ; Mon, 11 May 2009 15:20:59 -0700 (PDT) Received: from marcelo-bagnulos-macbook-pro.local (166.pool85-53-132.dynamic.orange.es [85.53.132.166]) by smtp03.uc3m.es (Postfix) with ESMTP id 409697EEF73; Tue, 12 May 2009 00:22:27 +0200 (CEST) Message-ID: <4A08A522.3060300@it.uc3m.es> Date: Tue, 12 May 2009 00:22:26 +0200 From: marcelo bagnulo braun User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Joe Touch References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <20090507230452.GC35382@cisco.com> <4A036A29.7020005@isi.edu> <4A042F28.5050107@it.uc3m.es> <4A046242.4040104@isi.edu> <4A087396.4050001@it.uc3m.es> <4A088CF1.6070201@isi.edu> In-Reply-To: <4A088CF1.6070201@isi.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-TM-AS-Product-Ver: IMSS-7.0.0.3116-5.6.0.1016-16636.002 Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 May 2009 22:21:34 -0000 Joe Touch escribió: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > >> Joe Touch escribió: >> >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA1 >>> >>> >>> >>> >>>> Joe Touch escribió: >>>> >>>> >>> .. >>> >>> >>>>> IMO, this goes too far. I would expect it would be useful enough to talk >>>>> about "multi-address TCP" and avoid talking about "paths" when they're >>>>> not part of the architecture, nor are they knowable without doing things >>>>> like pathchar or traceroute. >>>>> >>>>> >>>> But there are many situations and probably very interesting ones, >>>> where the node has multipaht paths and does not have multiple addresses. >>>> For instnace, think about a server located in a big multihomed site >>>> using PI addresses. This server is likely to benefit from multipath TCP >>>> but it only has a single address. >>>> >>>> >>> Then you have different problems, and you need to separate them: >>> >>> | single endpoint | multi endpoint >>> -------------+-------------------+---------------- >>> single path | sp/se | sp/me >>> -------------+-------------------+---------------- >>> multi path | mp/se | mp/me >>> -------------+-------------------+---------------- >>> >>> >> I am confused now, what do you mean by multiple endpoint? >> > > More than one socket pair - more than one port and/or address. > from the application perspective, i would expect that it all looks like a single normal TCP socket. I mean, i would expect that any modification we make is transparent to the apps, so apps can take advantage of it without requiring any modification. that doesn't mean we need to limit ourselves to a single address pair, but that the different address pairs are managed by the MPTCP layer, in a transparent manner would that make sense? > >> So far, i understood we were talking about to endpoints communicating >> through TCP and where there were multiple paths between them. the >> question was whether we can use different address pairs (assigned to the >> two endpoints involved) as path selectors, but i am afriad we went out >> of synch and you have something else in mind >> > > You can have different paths when you have different: > - endpoint addresses > - port numbers > - or neither > > Paths do not have anything to do with addresses per se. > > Right But, in some cases, having different address pairs, does imply to have different paths. But in any case, in the general situation, i think we need to accommodate multiple path selectors, which can be any kind of bits, from address pairs, ports, options, or other bits in the header >>> Existing TCP assumes sp/se. >>> >>> If you solve mp/me, you solve sp/me as a degenerate case. >>> >>> So basically there are two distinct problems: >>> >>> 1) making TCP work when there are multiple paths >>> >>> which has *nothing* to do with whether >>> there are are multiple endpoints or not >>> >>> e.g., a single TCP connection can experience >>> two different paths given multipath routing >>> >>> >> well, i think there is andditional case which is to make TCP use the >> multiple paths simultaneously, which TCP doesn't do currently >> > > That's case (1) above. A single TCP connection can already see - and use > - - multiple paths simultaneously, and trips badly when it happens. E.g., > if long packets go one way and short ones go another, this will mess up > TCP's RTT calculation. > > but they don't use it simultaneously afaict and if they do, TCP as you say is not designed to do that. The proposal is to allow TCP to be aware that packets are being routed through alternative paths, characterize each of these paths based on congestion and decide which packets need to be forwarded through each path. >>> 2) making TCP work with multiple endpoints >>> >>> which has *nothing* to do with whether >>> there are multiple paths, since they >>> different endpoints can easily experience >>> the same path properties - including >>> the same bottleneck. You can't assume >>> that two different IP addresses (or other >>> TCP endpoint differentiators) result in >>> paths that do not share properties or >>> bottlenecks. >>> >>> >>> >> no, i can't be certain that they will not share the smae bottle neck, >> but what may be done is that if they do have the smae bottleneck, they >> behave as single TCP flow, while if they do not have the same bottle >> neck, they actually use BW availbale. >> > > When they share a bottleneck, they can easily end up with WORSE than a > single TCP of BW, because the two TCP 'sides' can fight with each other. > right, the idea here is to try to use coupling of the congestion control windows to avoid this negative effect, as per Kelly's work. > You'd be probing the bottleneck from both sides, and a probe on one side > can affect the other, etc. > > >>> this *does* include striping, i.e., >>> concurrent use of alternate endpoints >>> to increase BW; that is NOT the case for >>> single-address. >>> >>> >> why not? >> >> It is possible to use a different path selector, set by TCP, that >> allows different packets belonging to the same TCP connectio to be >> routed through different paths. >> > > Yes - that can increase path BW, but not endpoint BW, AFAICT. > > how come? mmm, i think i am probably doing some unstated assumptions here... I am assuming multihoming somewhere, probably in one of the sites involved in the the communication. I mean, if congestion control is limiting the amount of data in flight, then if we have an additional path, we will be able to push more traffic, right? >>> note that 'endpoint' could mean IP address >>> or port, or anything else that could cause >>> path properties to vary - keep in mind that >>> policy routing peeks at the entire IP and often >>> TCP headers. >>> >>> >>> >> i am having difficuties to match this with the previous sentence... I >> mean, we agree that we can have a single address and use other fileds >> than the address as path selector? >> > > Yes, we're agreeing. > > >>> Note that when I say "multiple paths" I mean paths with different >>> characteristics, e.g., MTU, RTT, BW, loss. >>> >>> >>> >> right >> >> >>> IMO, multiendpoint is a potentially important problem that may be ready >>> to solve via IETF work at this time. However, you need to be careful to >>> never assume that having multiple endpoints means distinct paths - if >>> they're not distinct in bottleneck, then you'll end up making TCP fight >>> with itself. >>> >>> >>> >> exactly, this is critical in this effort. >> >> >>> If you're going to solve multipath, then you need to solve it in the >>> single endpoint case first, IMO. If you can't solve it there, there's no >>> point in conflating the issue with multiendpoint. >>> >>> >>> >> i am still having problems to understand your description of the >> problem, i.e. the multiendpoint vs, the multipath >> > > Multi-endpoint uses different socket pairs at one or more endpoint. > right, this is not what i have in mind when i wrote the proposed charter the idea is that from the socket perspective, this is exactly like a TCP connection > Multipath experiences two different paths at the same time. Those two > paths may or may not be correlated with the use of different endpoints. > > if you mean that different paths may or may not be related with having different address pairs, i agree that it is not universally true, but it does seems to be true in many cases (large enough to construct on top of that imho) >> how would you scope the proble only to deal with multipaht and not with multiendpoint? >> > > You make TCP figure out 'path' by 'path characteristics', not by > assuming it from differences in endpoint IDs. exactly TCP will characterize each path available based on the characteristics it can measure, in particular congestion along each path Now, then we need an interface that translates the paths as seen by TCP and the packet on the wire. I call this the path selector. These are some bits that reflect the notion of which path TCP thinks it is being used. So, at some point the path selector must be used by the IP or routing so that path with different path selectors, actually flow through a different path, makes sense? > Which is important, > because you can use different endpoint IDs and end up with the same > path. You need to find out path agnostically. > > agree >>>> In addition, even if you do have multiple addresses, you still need >>>> some support from the routing infrastrcuture. >>>> >>>> >>> IMO, you do not need support from the routing infrastructure at all. TCP >>> probes to detect path properties without interacting with routing right now. >>> >>> You need to separate that out when you get distinct sets of values. All >>> you need to know is that you are getting a correlation between segments >>> being sent and varying path properties. >>> >>> >>> >> exactly, that is what i call a path selector >> a path selector is something in the packets that TCP sets and that >> other part of the network looks at and sends packets with different >> paths selectors through different paths. For TCP, all it is needed is >> that the packets with the same path selector are routed through the >> same path (or at least as stabel as it is today in the single path >> case). >> > > That's reasonable to assume, but isn't always true. > > >> also, it is useful, that packets with different path >> selectors are routed through different paths but if this doesn't >> happen, the systme should work as a single path TCP >> > > That's not reasonable to assume. > > what is not reasonable? that path with different selector will be routed through different paths or that if this doesn't happen the multiple flows should behave as a single TCP flow? >>>> When a node A with multiple addresses sends packets, it will include >>>> different source addresses, but all packets will carry the same >>>> destiantion address. >>>> >>>> >>> That's ONE way to do this. There are others. >>> >>> >>> >> such as? >> (this is sincere question. We have been trying to understand how to >> do this in shim6 for many years and we couldn't find very nice answers) >> > > Use different source ports. Use different destination addresses. Use > different destination ports. > > >>>> How can the nade A influence the egress path? >>>> >>>> >>> By varying the source address, source port, dest address, dest port, or >>> just about anything else in the IP or TCP header. >>> >>> >>> >> right, but in the case the destination node has a single address, all >> mean, we agree that we can have a single address and use other fileds >> than the address as path selector? >> > > Yes. What I'm having problems with is assuming that different selectors > - - of any type - can assume different paths. > > right So, we have multiple options here, but there is no universal solution that applies to all scenarios. In many cases, using different address pairs will result in different paths. Not always, but i think that it will happen in enough amount of cases to make it useful The other option is to have different next hops. Suppose a multihomed site, and that we can either use different egress router as next hops. In this case, the path selector is something local in the host, that TCP tells IP that a given packet needs to use one next hop and another packet should use a different next hop. We can even use tunnels to the egress routers and take it from there. So, there is myriad of ways we can do this. Probably none of them will be universal. So, i am not sure if we need to describe them or just leave it open for people to use whatever suits them. From the discussion so far, it seems that at least we need to be more explicit about the relation between MPTCP and IP/routing. Regards, marcelo > Joe > > >> Regards, marcelo >> >> >> >>> Joe >>> -----BEGIN PGP SIGNATURE----- >>> Version: GnuPG v1.4.9 (MingW32) >>> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org >>> >>> iEYEARECAAYFAkoEYkIACgkQE5f5cImnZrtEzwCgqTmskXCmmAKi/SJHpw5A5ZRj >>> eukAnRkO0vPBnEwDf1wzJ1VG1wQs46dz >>> =1KaA >>> -----END PGP SIGNATURE----- >>> >>> >>> >>> > > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.9 (MingW32) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iEYEARECAAYFAkoIjPAACgkQE5f5cImnZrvc7gCgjfAKgarnOnbNHrEfNjmV7s7F > u24AnAz3LgMqhvxfD6J4RvJvQTEixtF0 > =UoHK > -----END PGP SIGNATURE----- > > From fred@cisco.com Mon May 11 16:04:32 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id D3D863A68FB for ; Mon, 11 May 2009 16:04:32 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.477 X-Spam-Level: X-Spam-Status: No, score=-106.477 tagged_above=-999 required=5 tests=[AWL=0.122, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 10nhi63wngVA for ; Mon, 11 May 2009 16:04:32 -0700 (PDT) Received: from sj-iport-6.cisco.com (sj-iport-6.cisco.com [171.71.176.117]) by core3.amsl.com (Postfix) with ESMTP id EF8B63A67DA for ; Mon, 11 May 2009 16:04:31 -0700 (PDT) X-IronPort-AV: E=Sophos;i="4.41,177,1241395200"; d="scan'208";a="302571898" Received: from sj-dkim-4.cisco.com ([171.71.179.196]) by sj-iport-6.cisco.com with ESMTP; 11 May 2009 23:06:03 +0000 Received: from sj-core-2.cisco.com (sj-core-2.cisco.com [171.71.177.254]) by sj-dkim-4.cisco.com (8.12.11/8.12.11) with ESMTP id n4BN63P9028992; Mon, 11 May 2009 16:06:03 -0700 Received: from stealth-10-32-244-221.cisco.com (stealth-10-32-244-221.cisco.com [10.32.244.221]) by sj-core-2.cisco.com (8.13.8/8.13.8) with ESMTP id n4BN62EV027265; Mon, 11 May 2009 23:06:03 GMT Message-Id: From: Fred Baker To: Iljitsch van Beijnum In-Reply-To: Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Mon, 11 May 2009 16:06:00 -0700 References: <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <86CDA096-93A8-4B59-8E77-90BF74BC1377@cisco.com> X-Mailer: Apple Mail (2.930.3) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; l=908; t=1242083163; x=1242947163; c=relaxed/simple; s=sjdkim4002; h=Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version; d=cisco.com; i=fred@cisco.com; z=From:=20Fred=20Baker=20 |Subject:=20Re=3A=20[multipathtcp]=20Draft=20Multipath=20TC P=20(MPTCP)=20BOF=20description |Sender:=20; bh=UDVb5SBD8VhA0FS4rwGxRO2dGxWVm+LhaBdwI0xBM7E=; b=lc7iOK49/EKXvobNy+i6vlig8zskJuFUPL6yKQ6sueYPCGB3l+XnAKMgnA MekNXPm0EBG10XqRJpN31C6NrFYyffLCBuC1uMW1WylqHMAJoj8l4nvzgsPR MhVunlhOTQ; Authentication-Results: sj-dkim-4; header.From=fred@cisco.com; dkim=pass ( sig from cisco.com/sjdkim4002 verified; ); Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 May 2009 23:04:32 -0000 On May 8, 2009, at 5:26 AM, Iljitsch van Beijnum wrote: > On 8 mei 2009, at 2:56, Fred Baker wrote: > >>> In principle, the host sends packets over a certain path, if ACKs >>> come back the path works, if they don't it doesn't. > >> I buy that. That tells us something about source address pairs, >> which determine routing. How about finer-grain routing issues? > > Like what? Like what if I have multiple paths through the same domain? The same source/destination address pair has multiple routes that it can use? For the sake of argument, we could postulate that they have different AS paths between the local ISPs or at least discrete routing paths. What I would like to understand is what signals MPTCP would like to give the network and what it would like the network to do with them. Using different address pairs is a pretty heavy club, but may be workable. From touch@ISI.EDU Mon May 11 18:31:10 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 138123A6807 for ; Mon, 11 May 2009 18:31:10 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RWBYHQSHfuAG for ; Mon, 11 May 2009 18:31:08 -0700 (PDT) Received: from vapor.isi.edu (vapor.isi.edu [128.9.64.64]) by core3.amsl.com (Postfix) with ESMTP id DFB843A6A03 for ; Mon, 11 May 2009 18:31:02 -0700 (PDT) Received: from [75.198.135.139] (139.sub-75-198-135.myvzw.com [75.198.135.139]) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id n4C1WFrw016896; Mon, 11 May 2009 18:32:17 -0700 (PDT) Message-ID: <4A08D19F.1080404@isi.edu> Date: Mon, 11 May 2009 18:32:15 -0700 From: Joe Touch User-Agent: Thunderbird 2.0.0.21 (Windows/20090302) MIME-Version: 1.0 To: marcelo bagnulo braun References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <20090507230452.GC35382@cisco.com> <4A036A29.7020005@isi.edu> <4A042F28.5050107@it.uc3m.es> <4A046242.4040104@isi.edu> <4A087396.4050001@it.uc3m.es> <4A088CF1.6070201@isi.edu> <4A08A522.3060300@it.uc3m.es> In-Reply-To: <4A08A522.3060300@it.uc3m.es> X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2009 01:31:10 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 ... >>>> Then you have different problems, and you need to separate them: >>>> >>>> | single endpoint | multi endpoint >>>> -------------+-------------------+---------------- >>>> single path | sp/se | sp/me >>>> -------------+-------------------+---------------- >>>> multi path | mp/se | mp/me >>>> -------------+-------------------+---------------- >>>> >>> I am confused now, what do you mean by multiple endpoint? >>> >> >> More than one socket pair - more than one port and/or address. >> > from the application perspective, i would expect that it all looks > like a single normal TCP socket. I mean, i would expect that any > modification we make is transparent to the apps, so apps can take > advantage of it without requiring any modification. The goal is to make this all transparent to apps, regardless of what's done. The question is "how many problems are you trying to solve at once", or, more specifically, which problems are you trying to solve? >>> through TCP and where there were multiple paths between them. the >>> question was whether we can use different address pairs (assigned to the >>> two endpoints involved) as path selectors, but i am afriad we went out >>> of synch and you have something else in mind >>> >> >> You can have different paths when you have different: >> - endpoint addresses >> - port numbers >> - or neither >> >> Paths do not have anything to do with addresses per se. >> >> > > Right > But, in some cases, having different address pairs, does imply to have different paths. Different addresses MAY result in different paths, but doesn't have to. IMO, you need to *test* for that condition, not assume it. > But in any case, in the general situation, i think we need to > accommodate multiple path selectors, which can be any kind of bits, from > address pairs, ports, options, or other bits in the header Sure. >>>> Existing TCP assumes sp/se. >>>> >>>> If you solve mp/me, you solve sp/me as a degenerate case. >>>> >>>> So basically there are two distinct problems: >>>> >>>> 1) making TCP work when there are multiple paths >>>> >>>> which has *nothing* to do with whether >>>> there are are multiple endpoints or not >>>> >>>> e.g., a single TCP connection can experience >>>> two different paths given multipath routing >>>> >>> well, i think there is andditional case which is to make TCP use the >>> multiple paths simultaneously, which TCP doesn't do currently >>> >> >> That's case (1) above. A single TCP connection can already see - and use >> - - multiple paths simultaneously, and trips badly when it happens. E.g., >> if long packets go one way and short ones go another, this will mess up >> TCP's RTT calculation. > > but they don't use it simultaneously afaict They can. > and if they do, TCP as you say is not designed to do that. TCP isn't. "MULTIPATH", IMO, is. Or if that's not what this is focusing on, you're using the wrong word - - you mean "multiendpoint". > The > proposal is to allow TCP to be aware that packets are being routed > through alternative paths, characterize each of these paths based on > congestion and decide which packets need to be forwarded through each path. You need to determine whether there are multiple paths. I don't accept the assertion that "different endpoints == different paths", because neither => nor <= is true ('implies', in either direction). ... >>>> this *does* include striping, i.e., >>>> concurrent use of alternate endpoints >>>> to increase BW; that is NOT the case for >>>> single-address. >>>> >>> why not? >>> >>> It is possible to use a different path selector, set by TCP, that >>> allows different packets belonging to the same TCP connectio to be >>> routed through different paths. >>> >> >> Yes - that can increase path BW, but not endpoint BW, AFAICT. >> >> > how come? > mmm, i think i am probably doing some unstated assumptions here... > I am assuming multihoming somewhere, probably in one of the sites > involved in the the communication. > > I mean, if congestion control is limiting the amount of data in flight, > then if we have an additional path, we will be able to push more > traffic, right? If there are two ****PATHS****. You don't have two PATHS when you have two ENDPOINTS. They're just not the same thing. ... >>> i am still having problems to understand your description of the >>> problem, i.e. the multiendpoint vs, the multipath >>> >> >> Multi-endpoint uses different socket pairs at one or more endpoint. > > right, this is not what i have in mind when i wrote the proposed charter > the idea is that from the socket perspective, this is exactly like a TCP connection There are two different viewpoints: 1) the endpoints 2) the network The network can see two different socket pairs, even if the endpoints see one socket pair. >> Multipath experiences two different paths at the same time. Those two >> paths may or may not be correlated with the use of different endpoints. >> >> > if you mean that different paths may or may not be related with > having > different address pairs, i agree that it is not universally true, but it > does seems to be true in many cases (large enough to construct on top of > that imho) TCP was symmetric in most cases, until the web came along. Designing for the currently common case is a disaster waiting to happen. If you can test for distinct paths, do so. If not, then don't design for it based on an assumption. Or maybe we should call this "ethernet TCP", since -- in most cases -- it'll run over ethernet, right? >>> how would you scope the proble only to deal with multipaht and not with multiendpoint? >>> >> >> You make TCP figure out 'path' by 'path characteristics', not by >> assuming it from differences in endpoint IDs. > exactly TCP will characterize each path available based on the > characteristics > it can measure, in particular congestion along each path > Now, then we need an interface that translates the paths as seen by > TCP and the packet on the wire. I call this the path selector. These are > some bits that reflect the notion of which path TCP thinks it is being > used. So, at some point the path selector must be used by the IP or > routing so that path with different path selectors, actually flow > through a different path, makes sense? Sounds like you're trying to bury routing inside of TCP to me. ... >>>>> In addition, even if you do have multiple addresses, you still need >>>>> some support from the routing infrastrcuture. >>>>> >>>> IMO, you do not need support from the routing infrastructure at all. TCP >>>> probes to detect path properties without interacting with routing right now. >>>> >>>> You need to separate that out when you get distinct sets of values. All >>>> you need to know is that you are getting a correlation between segments >>>> being sent and varying path properties. >>>> >>>> >>> exactly, that is what i call a path selector >>> a path selector is something in the packets that TCP sets and that >>> other part of the network looks at and sends packets with different >>> paths selectors through different paths. For TCP, all it is needed is >>> that the packets with the same path selector are routed through the >>> same path (or at least as stabel as it is today in the single path >>> case). >> >> That's reasonable to assume, but isn't always true. >> >> >>> also, it is useful, that packets with different path >>> selectors are routed through different paths but if this doesn't >>> happen, the systme should work as a single path TCP >>> >> >> That's not reasonable to assume. >> >> > what is not reasonable? > that path with different selector will be routed through different > paths Yes. > or that if this doesn't happen the multiple flows should > behave as a single TCP flow? No - I agree with that. ... >>>>> How can the nade A influence the egress path? >>>>> >>>> By varying the source address, source port, dest address, dest port, or >>>> just about anything else in the IP or TCP header. >>>> >>>> >>> right, but in the case the destination node has a single address, all >>> mean, we agree that we can have a single address and use other fileds >>> than the address as path selector? >>> >> >> Yes. What I'm having problems with is assuming that different selectors >> - - of any type - can assume different paths. >> >> > right > > So, we have multiple options here, but there is no universal solution > that applies to all scenarios. > > In many cases, using different address pairs will result in different > paths. Not always, but i think that it will happen in enough amount of > cases to make it useful > The other option is to have different next hops. Suppose a multihomed > site, and that we can either use different egress router as next hops. > In this case, the path selector is something local in the host, that TCP > tells IP that a given packet needs to use one next hop and another > packet should use a different next hop. We can even use tunnels to the > egress routers and take it from there. Nope. Different hops tells you that the path diverges, but not that the path properties differ - or that the defining path properties aren't dominated by a shared path downstream. > So, there is myriad of ways we can do this. Probably none of them > will be universal. > So, i am not sure if we need to describe them or just leave it open > for people to use whatever suits them. > > From the discussion so far, it seems that at least we need to be more > explicit about the relation between MPTCP and IP/routing. Yes, but that's exactly what concerns me. Joe -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkoI0Z8ACgkQE5f5cImnZruSHACfTQ9YpxJBbaZgZzeTnoZsfT2n 1V8An3XSQVZkKWm9Shls4zKEna1gRtvV =Kx9h -----END PGP SIGNATURE----- From marcelo@it.uc3m.es Mon May 11 23:45:36 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 1FAAF3A6BB1 for ; Mon, 11 May 2009 23:45:36 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.365 X-Spam-Level: X-Spam-Status: No, score=-6.365 tagged_above=-999 required=5 tests=[AWL=0.234, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wd8C5APGCNMK for ; Mon, 11 May 2009 23:45:35 -0700 (PDT) Received: from smtp02.uc3m.es (smtp02.uc3m.es [163.117.176.132]) by core3.amsl.com (Postfix) with ESMTP id CFFEE3A68B2 for ; Mon, 11 May 2009 23:45:34 -0700 (PDT) Received: from marcelo-bagnulos-macbook-pro.local (166.pool85-53-132.dynamic.orange.es [85.53.132.166]) by smtp02.uc3m.es (Postfix) with ESMTP id 5EE23656734; Tue, 12 May 2009 08:46:59 +0200 (CEST) Message-ID: <4A091B61.7060606@it.uc3m.es> Date: Tue, 12 May 2009 08:46:57 +0200 From: marcelo bagnulo braun User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Fred Baker References: <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <86CDA096-93A8-4B59-8E77-90BF74BC1377@cisco.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-TM-AS-Product-Ver: IMSS-7.0.0.3116-5.6.0.1016-16636.005 Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2009 06:45:36 -0000 Fred Baker escribió: > > On May 8, 2009, at 5:26 AM, Iljitsch van Beijnum wrote: > >> On 8 mei 2009, at 2:56, Fred Baker wrote: >> >>>> In principle, the host sends packets over a certain path, if ACKs >>>> come back the path works, if they don't it doesn't. >> >>> I buy that. That tells us something about source address pairs, >>> which determine routing. How about finer-grain routing issues? >> >> Like what? > > Like what if I have multiple paths through the same domain? The same > source/destination address pair has multiple routes that it can use? > For the sake of argument, we could postulate that they have different > AS paths between the local ISPs or at least discrete routing paths. > > What I would like to understand is what signals MPTCP would like to > give the network and what it would like the network to do with them. right, this is exactly the point the idea as i understand it is: MPTCP will characterize different paths based on congestion. It will then associate these paths to a different path selector (those signals you say above) and what is expected from the IP layer is that different path selectors are routed through different paths, if available. > Using different address pairs is a pretty heavy club, but may be workable. Agree. However, in my shim6 experience, people were very reluctant to use source address dependent routing. AFICT, we have the following options: - source dst address - different local interfaces (from the MPTCP host) - different next hops (from the MPTCP host) - different tunnels to different egress routers (from the MPTCP host) - the flow label (only in IPv6) - DSCP bits - magical use of fragmentation fields - some new IP option (sigh) - some new TCP option (even more sigh) and all of them are very far from being a nice solution i am afraid. Regards, marcelo > > _______________________________________________ > multipathtcp mailing list > multipathtcp@ietf.org > https://www.ietf.org/mailman/listinfo/multipathtcp > From marcelo@it.uc3m.es Tue May 12 00:23:05 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 5000F3A67FD for ; Tue, 12 May 2009 00:23:05 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.367 X-Spam-Level: X-Spam-Status: No, score=-6.367 tagged_above=-999 required=5 tests=[AWL=0.232, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 95xf2h8wL2Lo for ; Tue, 12 May 2009 00:23:03 -0700 (PDT) Received: from smtp02.uc3m.es (smtp02.uc3m.es [163.117.176.132]) by core3.amsl.com (Postfix) with ESMTP id E98B93A69FC for ; Tue, 12 May 2009 00:23:02 -0700 (PDT) Received: from marcelo-bagnulos-macbook-pro.local (166.pool85-53-132.dynamic.orange.es [85.53.132.166]) by smtp02.uc3m.es (Postfix) with ESMTP id 0E6806BA067; Tue, 12 May 2009 09:24:31 +0200 (CEST) Message-ID: <4A09242F.6080505@it.uc3m.es> Date: Tue, 12 May 2009 09:24:31 +0200 From: marcelo bagnulo braun User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Joe Touch References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <20090507230452.GC35382@cisco.com> <4A036A29.7020005@isi.edu> <4A042F28.5050107@it.uc3m.es> <4A046242.4040104@isi.edu> <4A087396.4050001@it.uc3m.es> <4A088CF1.6070201@isi.edu> <4A08A522.3060300@it.uc3m.es> <4A08D19F.1080404@isi.edu> In-Reply-To: <4A08D19F.1080404@isi.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-TM-AS-Product-Ver: IMSS-7.0.0.3116-5.6.0.1016-16636.005 Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2009 07:23:05 -0000 Joe Touch escribió: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > ... > >>>>> Then you have different problems, and you need to separate them: >>>>> >>>>> | single endpoint | multi endpoint >>>>> -------------+-------------------+---------------- >>>>> single path | sp/se | sp/me >>>>> -------------+-------------------+---------------- >>>>> multi path | mp/se | mp/me >>>>> -------------+-------------------+---------------- >>>>> >>>>> >>>> I am confused now, what do you mean by multiple endpoint? >>>> >>>> >>> More than one socket pair - more than one port and/or address. >>> >>> >> from the application perspective, i would expect that it all looks >> like a single normal TCP socket. I mean, i would expect that any >> modification we make is transparent to the apps, so apps can take >> advantage of it without requiring any modification. >> > > The goal is to make this all transparent to apps, regardless of what's done. > The question is "how many problems are you trying to solve at once", or, > more specifically, which problems are you trying to solve? > > We agree that using multiple paht if available improve the performance of the communication (i.e. BW, fault tolerance), right? The idea is to allow TCP to get these benefits, without involving the apps. >>>> through TCP and where there were multiple paths between them. the >>>> question was whether we can use different address pairs (assigned to the >>>> two endpoints involved) as path selectors, but i am afriad we went out >>>> of synch and you have something else in mind >>>> >>>> >>> You can have different paths when you have different: >>> - endpoint addresses >>> - port numbers >>> - or neither >>> >>> Paths do not have anything to do with addresses per se. >>> >>> >>> >> Right >> But, in some cases, having different address pairs, does imply to have different paths. >> > > Different addresses MAY result in different paths, but doesn't have to. > IMO, you need to *test* for that condition, not assume it. > > that is one option. the other option is that if you have multiple addresses, you try to use them. In the best case you will get different paths with different bottlenecks and better performance. In the worse case, you are like a single TCP. Of course, in order to take this approach, we must make sure that two flows of a single TCP don't fight against each other making things worse. >> But in any case, in the general situation, i think we need to >> accommodate multiple path selectors, which can be any kind of bits, from >> address pairs, ports, options, or other bits in the header >> > > Sure. > > >>>>> Existing TCP assumes sp/se. >>>>> >>>>> If you solve mp/me, you solve sp/me as a degenerate case. >>>>> >>>>> So basically there are two distinct problems: >>>>> >>>>> 1) making TCP work when there are multiple paths >>>>> >>>>> which has *nothing* to do with whether >>>>> there are are multiple endpoints or not >>>>> >>>>> e.g., a single TCP connection can experience >>>>> two different paths given multipath routing >>>>> >>>>> >>>> well, i think there is andditional case which is to make TCP use the >>>> multiple paths simultaneously, which TCP doesn't do currently >>>> >>>> >>> That's case (1) above. A single TCP connection can already see - and use >>> - - multiple paths simultaneously, and trips badly when it happens. E.g., >>> if long packets go one way and short ones go another, this will mess up >>> TCP's RTT calculation. >>> >> but they don't use it simultaneously afaict >> > > They can. > > >> and if they do, TCP as you say is not designed to do that. >> > > TCP isn't. "MULTIPATH", IMO, is. > > Or if that's not what this is focusing on, you're using the wrong word > - - you mean "multiendpoint". > > mmm, i am not sure. multiendpoint seem to assume different address paris or at least different port pairs, but it is possible that the path selector is a different next hop, in which case, it hardly fits in the multiendpoint definition (as i understand it) the goal is multipath. We are using multiendpoint as one of the tools to get multipath, without guarantees that we will get it, if we do so. Would that be a good description? >> The >> proposal is to allow TCP to be aware that packets are being routed >> through alternative paths, characterize each of these paths based on >> congestion and decide which packets need to be forwarded through each path. >> > > You need to determine whether there are multiple paths. I don't accept > the assertion that "different endpoints == different paths", because > neither => nor <= is true ('implies', in either direction). > ... > Let me see if i get what you mean, Let's suppose we have a MPTCP host. This host is configured to use a given path selector. Let' consider two cases: option a) different address pairs option b) different next hops. Now, in the case of different address pairs, there are two different cases. When the source address changes and when the dst address changes. If the intra domain routing of the site that is hosting the MPTCP node has been modified to support source address dependent routing, the mptcp host knows that using different soruce address will result in different paths. Using differnet dest address, if available, it is not clear what effect that will have. there may result in more path diversity or not. So, in this case, i think it is reaosnable to assume that if multiple addresses are available , at least idfferent soruce addresses will result in different paths. In the option b with a different next hop, it is reasonable to assume that if mptcp is deployed in this scenario, the different next hops will result in different egress paths. In this case, there is no guarantee that there is path diversity in the reverse path. does this makes sense to you? What i am trying to say is that mptcp is likely to be deployed using a path selector that would result in path diversity. If not, it would not be exposed to mptcp i.e. nobody will take the burden of configuring the paht selectors in the mptcp if no path diversity is available. >>>>> this *does* include striping, i.e., >>>>> concurrent use of alternate endpoints >>>>> to increase BW; that is NOT the case for >>>>> single-address. >>>>> >>>>> >>>> why not? >>>> >>>> It is possible to use a different path selector, set by TCP, that >>>> allows different packets belonging to the same TCP connectio to be >>>> routed through different paths. >>>> >>>> >>> Yes - that can increase path BW, but not endpoint BW, AFAICT. >>> >>> >>> >> how come? >> mmm, i think i am probably doing some unstated assumptions here... >> I am assuming multihoming somewhere, probably in one of the sites >> involved in the the communication. >> >> I mean, if congestion control is limiting the amount of data in flight, >> then if we have an additional path, we will be able to push more >> traffic, right? >> > > If there are two ****PATHS****. You don't have two PATHS when you have > two ENDPOINTS. They're just not the same thing. > ... > >>>> i am still having problems to understand your description of the >>>> problem, i.e. the multiendpoint vs, the multipath >>>> >>>> >>> Multi-endpoint uses different socket pairs at one or more endpoint. >>> >> right, this is not what i have in mind when i wrote the proposed charter >> the idea is that from the socket perspective, this is exactly like a TCP connection >> > > There are two different viewpoints: > > 1) the endpoints > > 2) the network > > The network can see two different socket pairs, even if the endpoints > see one socket pair. > > ok, >>> Multipath experiences two different paths at the same time. Those two >>> paths may or may not be correlated with the use of different endpoints. >>> >>> >>> >> if you mean that different paths may or may not be related with >> having >> different address pairs, i agree that it is not universally true, but it >> does seems to be true in many cases (large enough to construct on top of >> that imho) >> > > TCP was symmetric in most cases, until the web came along. > > Designing for the currently common case is a disaster waiting to happen. > If you can test for distinct paths, do so. If not, then don't design for > it based on an assumption. > > what would be the disaster? I mean, i think it is a sin equanum condition to require that if multiple subflows run over the same path, they do not fight against each other. I mean, if we design in such a way that in the case there is not path diversity, or in the case there is path diversity but both paths share the same bottleneck, things are not worse than in the single path tcp, we should be ok, don't you think? > Or maybe we should call this "ethernet TCP", since -- in most cases -- > it'll run over ethernet, right? > > :-) >>>> how would you scope the proble only to deal with multipaht and not with multiendpoint? >>>> >>>> >>> You make TCP figure out 'path' by 'path characteristics', not by >>> assuming it from differences in endpoint IDs. >>> >> exactly TCP will characterize each path available based on the >> characteristics >> it can measure, in particular congestion along each path >> Now, then we need an interface that translates the paths as seen by >> TCP and the packet on the wire. I call this the path selector. These are >> some bits that reflect the notion of which path TCP thinks it is being >> used. So, at some point the path selector must be used by the IP or >> routing so that path with different path selectors, actually flow >> through a different path, makes sense? >> > > Sounds like you're trying to bury routing inside of TCP to me. > > but routing doens't characterize routes based on congestion. the only one that knows about congestion is the transport and that is what we need to make a wise split of the load. So, clearly we are trying to get a closer relationship between routing and congestion control, but each of them still performs its role i think. What needs to be defined is the interface between routing and MPTCP, and that interface is the path selector we have been disucssing so far. > ... > >>>>>> In addition, even if you do have multiple addresses, you still need >>>>>> some support from the routing infrastrcuture. >>>>>> >>>>>> >>>>> IMO, you do not need support from the routing infrastructure at all. TCP >>>>> probes to detect path properties without interacting with routing right now. >>>>> >>>>> You need to separate that out when you get distinct sets of values. All >>>>> you need to know is that you are getting a correlation between segments >>>>> being sent and varying path properties. >>>>> >>>>> >>>>> >>>> exactly, that is what i call a path selector >>>> a path selector is something in the packets that TCP sets and that >>>> other part of the network looks at and sends packets with different >>>> paths selectors through different paths. For TCP, all it is needed is >>>> that the packets with the same path selector are routed through the >>>> same path (or at least as stabel as it is today in the single path >>>> case). >>>> >>> That's reasonable to assume, but isn't always true. >>> >>> >>> >>>> also, it is useful, that packets with different path >>>> selectors are routed through different paths but if this doesn't >>>> happen, the systme should work as a single path TCP >>>> >>>> >>> That's not reasonable to assume. >>> >>> >>> >> what is not reasonable? >> that path with different selector will be routed through different >> paths >> > > Yes. > > >> or that if this doesn't happen the multiple flows should >> behave as a single TCP flow? >> > > No - I agree with that. > > ... > >>>>>> How can the nade A influence the egress path? >>>>>> >>>>>> >>>>> By varying the source address, source port, dest address, dest port, or >>>>> just about anything else in the IP or TCP header. >>>>> >>>>> >>>>> >>>> right, but in the case the destination node has a single address, all >>>> mean, we agree that we can have a single address and use other fileds >>>> than the address as path selector? >>>> >>>> >>> Yes. What I'm having problems with is assuming that different selectors >>> - - of any type - can assume different paths. >>> >>> >>> >> right >> >> So, we have multiple options here, but there is no universal solution >> that applies to all scenarios. >> >> In many cases, using different address pairs will result in different >> paths. Not always, but i think that it will happen in enough amount of >> cases to make it useful >> The other option is to have different next hops. Suppose a multihomed >> site, and that we can either use different egress router as next hops. >> In this case, the path selector is something local in the host, that TCP >> tells IP that a given packet needs to use one next hop and another >> packet should use a different next hop. We can even use tunnels to the >> egress routers and take it from there. >> > > Nope. Different hops tells you that the path diverges, but not that the > path properties differ - or that the defining path properties aren't > dominated by a shared path downstream. > > agree >> So, there is myriad of ways we can do this. Probably none of them >> will be universal. >> So, i am not sure if we need to describe them or just leave it open >> for people to use whatever suits them. >> >> From the discussion so far, it seems that at least we need to be more >> explicit about the relation between MPTCP and IP/routing. >> > > Yes, but that's exactly what concerns me. > > i understand. We should try to present the assumptions more explciitly, so we can have a discussion. Regards, marcelo > Joe > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.9 (MingW32) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iEYEARECAAYFAkoI0Z8ACgkQE5f5cImnZruSHACfTQ9YpxJBbaZgZzeTnoZsfT2n > 1V8An3XSQVZkKWm9Shls4zKEna1gRtvV > =Kx9h > -----END PGP SIGNATURE----- > > From iljitsch@muada.com Tue May 12 04:06:10 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 23A6B3A6781 for ; Tue, 12 May 2009 04:06:10 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ee7zLbb2NO+a for ; Tue, 12 May 2009 04:06:09 -0700 (PDT) Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by core3.amsl.com (Postfix) with ESMTP id 4A6C73A6C91 for ; Tue, 12 May 2009 04:06:02 -0700 (PDT) Received: from claw.it.uc3m.es (claw.it.uc3m.es [163.117.139.66]) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n4CB72im050935 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Tue, 12 May 2009 13:07:03 +0200 (CEST) (envelope-from iljitsch@muada.com) Message-Id: <5DC12C17-2FC5-4728-A3BB-C3EDA3369640@muada.com> From: Iljitsch van Beijnum To: Joe Touch In-Reply-To: <4A08943D.1080100@isi.edu> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Tue, 12 May 2009 13:07:07 +0200 References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <20090507230452.GC35382@cisco.com> <4A036A29.7020005@isi.edu> <4A042F28.5050107@it.uc3m.es> <4A046242.4040104@isi.edu> <4A087396.4050001@it.uc3m.es> <4A088CF1.6070201@isi.edu> <3D4CFA5B-A4CD-43C1-BCC4-57BA3383466B@muada.com> <4A08943D.1080100@isi.edu> X-Mailer: Apple Mail (2.930.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2009 11:06:10 -0000 On 11 mei 2009, at 23:10, Joe Touch wrote: >> But are you saying that when MPTCP invokes a number of paths, we >> MUST be >> able to determine which ones share bottlenecks? > Yes. Ok. I think it can't be done reliably. (And note that in the one-ended case the multiple paths must always converge towards the interface holding the remote address.) Also, why would we care? If we simply limit the number of active subflows to something modest we're not doing anything that HTTP isn't doing, or that users aren't doing. >> So I think we should proceed under the assumption that shared >> bottlenecks will happen and multipath congestion control should >> behave >> in a way that isn't unduly harmful when it does. > Yes. Why does that not mean that you're back to single-path TCP > behavior? Because it's acceptable to be more aggressive than a single NewReno flow by a factor 2 or 3 or so. >> BTW, I would prefer to stay away from the "endpoint" terminology >> because >> it's not entirely clear what it means. > It has a known meaning in TCP - I'm talking about a socket (an > address/port pair). That terminology may be useful in normal TCP but in multipath TCP we need to be more precise. From iljitsch@muada.com Tue May 12 04:16:21 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id CDED43A689E for ; Tue, 12 May 2009 04:16:21 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Bheh64K6ZoEL for ; Tue, 12 May 2009 04:16:21 -0700 (PDT) Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by core3.amsl.com (Postfix) with ESMTP id B684C3A69A0 for ; Tue, 12 May 2009 04:16:20 -0700 (PDT) Received: from claw.it.uc3m.es (claw.it.uc3m.es [163.117.139.66]) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n4CBGlHE051022 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Tue, 12 May 2009 13:16:50 +0200 (CEST) (envelope-from iljitsch@muada.com) Message-Id: <419C5E60-0153-465B-90DC-73FDE43719CA@muada.com> From: Iljitsch van Beijnum To: Fred Baker In-Reply-To: Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Tue, 12 May 2009 13:16:52 +0200 References: <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <86CDA096-93A8-4B59-8E77-90BF74BC1377@cisco.com> X-Mailer: Apple Mail (2.930.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2009 11:16:21 -0000 On 12 mei 2009, at 1:06, Fred Baker wrote: > Like what if I have multiple paths through the same domain? Why would the domain be noteworthy? I can imagine the situation where both ends are multihomed to different ISPs and two paths that they use are completely disjoint at all levels. I can also imagine the situation where the ends are actually single homed and all hops are identicial except one, which consists of set of bundled links between two routers. Because of the limitations on how ECMP works, it could easily be that one link in the bundle is saturated while the other one as a lot of unused capacity. There are also situations in between. For instance, the packets between two neighboring ASes could flow through different interconnect locations. Or there could be path diversity within the AS. All that multipath TCP sees is different RTTs and loss rates for the different subflows. > What I would like to understand is what signals MPTCP would like to > give the network and what it would like the network to do with them. > Using different address pairs is a pretty heavy club, but may be > workable. Well, ideally we would have some bits X somewhere in the IP header (less ideal: a non-IP header) that tell a router "if you have multiple choices in the FIB for this destination, pick #X". However, I think it's premature to start usurping header bits before we have a good handle on how issues like fast retransmit, buffer depletion and congestion control coupling between the subflows would work. From touch@ISI.EDU Tue May 12 05:18:04 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id A3FDE3A689E for ; Tue, 12 May 2009 05:18:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JjpahHErpNrF for ; Tue, 12 May 2009 05:18:03 -0700 (PDT) Received: from vapor.isi.edu (vapor.isi.edu [128.9.64.64]) by core3.amsl.com (Postfix) with ESMTP id DD8A93A6ADA for ; Tue, 12 May 2009 05:18:03 -0700 (PDT) Received: from [172.20.32.138] (Extlanl3840-01.nsf.gov [198.181.231.14]) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id n4CCJG5m002857; Tue, 12 May 2009 05:19:18 -0700 (PDT) Message-ID: <4A096944.8060501@isi.edu> Date: Tue, 12 May 2009 05:19:16 -0700 From: Joe Touch User-Agent: Thunderbird 2.0.0.21 (Windows/20090302) MIME-Version: 1.0 To: Iljitsch van Beijnum References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es> <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es> <20090507211619.GA35382@cisco.com> <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <20090507230452.GC35382@cisco.com> <4A036A29.7020005@isi.edu> <4A042F28.5050107@it.uc3m.es> <4A046242.4040104@isi.edu> <4A087396.4050001@it.uc3m.es> <4A088CF1.6070201@isi.edu> <3D4CFA5B-A4CD-43C1-BCC4-57BA3383466B@muada.com> <4A08943D.1080100@isi.edu> <5DC12C17-2FC5-4728-A3BB-C3EDA3369640@muada.com> In-Reply-To: <5DC12C17-2FC5-4728-A3BB-C3EDA3369640@muada.com> X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2009 12:18:04 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Iljitsch van Beijnum wrote: > On 11 mei 2009, at 23:10, Joe Touch wrote: > >>> But are you saying that when MPTCP invokes a number of paths, we MUST be >>> able to determine which ones share bottlenecks? > >> Yes. > > Ok. I think it can't be done reliably. > > (And note that in the one-ended case the multiple paths must always > converge towards the interface holding the remote address.) > > Also, why would we care? > > If we simply limit the number of active subflows to something modest > we're not doing anything that HTTP isn't doing, or that users aren't doing. > >>> So I think we should proceed under the assumption that shared >>> bottlenecks will happen and multipath congestion control should behave >>> in a way that isn't unduly harmful when it does. > >> Yes. Why does that not mean that you're back to single-path TCP behavior? > > Because it's acceptable to be more aggressive than a single NewReno flow > by a factor 2 or 3 or so. No, that exceeds the recommendations in standards-track documents for TCP. >>> BTW, I would prefer to stay away from the "endpoint" terminology because >>> it's not entirely clear what it means. > >> It has a known meaning in TCP - I'm talking about a socket (an >> address/port pair). > > That terminology may be useful in normal TCP but in multipath TCP we > need to be more precise. No, you need to be precise about what you're talking about. A socket is very precise for TCP, just insufficient for MPTCP. Joe -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkoJaUQACgkQE5f5cImnZrvmYQCfarG8HFT0uP611NknsRonQIpN cTQAoNKoARoMWVQHpAgJjO+d+TI+Iqut =V+PI -----END PGP SIGNATURE----- From john@jlc.net Tue May 12 09:55:01 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id A94AD3A68B4 for ; Tue, 12 May 2009 09:55:01 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.866 X-Spam-Level: X-Spam-Status: No, score=-5.866 tagged_above=-999 required=5 tests=[AWL=0.733, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id m7tBUwMi6cL1 for ; Tue, 12 May 2009 09:55:00 -0700 (PDT) Received: from mailhost.jlc.net (mailhost.jlc.net [199.201.159.9]) by core3.amsl.com (Postfix) with ESMTP id AFCC23A6C43 for ; Tue, 12 May 2009 09:55:00 -0700 (PDT) Received: by mailhost.jlc.net (Postfix, from userid 104) id 6D05633C31; Tue, 12 May 2009 12:56:31 -0400 (EDT) Date: Tue, 12 May 2009 12:56:31 -0400 From: John Leslie To: Iljitsch van Beijnum Message-ID: <20090512165631.GB14375@verdi> References: <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <20090507230452.GC35382@cisco.com> <4A036A29.7020005@isi.edu> <4A042F28.5050107@it.uc3m.es> <4A046242.4040104@isi.edu> <4A087396.4050001@it.uc3m.es> <4A088CF1.6070201@isi.edu> <3D4CFA5B-A4CD-43C1-BCC4-57BA3383466B@muada.com> <4A08943D.1080100@isi.edu> <5DC12C17-2FC5-4728-A3BB-C3EDA3369640@muada.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5DC12C17-2FC5-4728-A3BB-C3EDA3369640@muada.com> User-Agent: Mutt/1.4.1i Cc: multipathtcp@ietf.org, Joe Touch Subject: [multipathtcp] Shared Bottleneck X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2009 16:55:01 -0000 Iljitsch van Beijnum wrote: > On 11 mei 2009, at 23:10, Joe Touch wrote: >> [Iljitsch van Beijnum wrote:] >> >>> But are you saying that when MPTCP invokes a number of paths, we >>> MUST be able to determine which ones share bottlenecks? > >> Yes. > > Ok. I think it can't be done reliably. Which makes it a suboptimal goal. I would advise against accepting it. (This does not mean we shouldn't have goal(s) of mitigating problems _caused_by_ shared bottlenecks; merely that we shouldn't state a goal as "avoid any shared bottlenecks".) > (And note that in the one-ended case the multiple paths must always > converge towards the interface holding the remote address.) That's not quite true; but it will usually be the case. > Also, why would we care? Exactly! Let's list a few reasons we might wish to avoid shared bottlenecks. I'll start: 1) oscillation (it's all to easy to design congestion control that will oscillate between two paths, confusing congestion signals for all and giving less overall bandwidth than a single path); 2) over-saturation of the "innocent" end's link (the textbook example is one user at an enterprise with marginal bandwidth can ruin the experience of other users at that enterprise); 3) too much retransmission of lost packets (leading to both loss of bandwidth and choppy response). (Others, please feel free to add...) > If we simply limit the number of active subflows to something modest > we're not doing anything that HTTP isn't doing, or that users aren't > doing. "Something modest" may be in the eyes of the beholder. Iljitsch, I believe is thinking three; HTTP used to think four but now is two; in the extreme case we could get arbitrarily close to one... Is it worth extensive discussion _before_ the BoF whether our target should be one or three? Also, does anyone want to claim that the target should be _less_ than one? >>> So I think we should proceed under the assumption that shared >>> bottlenecks will happen and multipath congestion control should >>> behave in a way that isn't unduly harmful when it does. > >> Yes. Why does that not mean that you're back to single-path TCP >> behavior? > > Because it's acceptable to be more aggressive than a single NewReno > flow by a factor 2 or 3 or so. Even if it isn't, there's a benefit to content-providers of fault tolerance and load balancing which chooses the best path for each flow. >>> BTW, I would prefer to stay away from the "endpoint" terminology >>> because it's not entirely clear what it means. > >> It has a known meaning in TCP - I'm talking about a socket (an >> address/port pair). > > That terminology may be useful in normal TCP but in multipath TCP we > need to be more precise. In particular, the application has no need to deal with separate "sockets" for a single "flow" over the multiple paths. -- John Leslie From iljitsch@muada.com Tue May 12 14:59:42 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id A9D553A6AD1 for ; Tue, 12 May 2009 14:59:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dhragxk+L5VZ for ; Tue, 12 May 2009 14:59:41 -0700 (PDT) Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by core3.amsl.com (Postfix) with ESMTP id 7AA6D3A6A2F for ; Tue, 12 May 2009 14:59:41 -0700 (PDT) Received: from [192.168.2.4] (static-167-138-7-89.ipcom.comunitel.net [89.7.138.167] (may be forged)) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n4CM0mS2055734 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Wed, 13 May 2009 00:00:48 +0200 (CEST) (envelope-from iljitsch@muada.com) Message-Id: <8C275BE5-B1BF-49AE-81A2-CB1876037012@muada.com> From: Iljitsch van Beijnum To: John Leslie In-Reply-To: <20090512165631.GB14375@verdi> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Wed, 13 May 2009 00:00:53 +0200 References: <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <20090507230452.GC35382@cisco.com> <4A036A29.7020005@isi.edu> <4A042F28.5050107@it.uc3m.es> <4A046242.4040104@isi.edu> <4A087396.4050001@it.uc3m.es> <4A088CF1.6070201@isi.edu> <3D4CFA5B-A4CD-43C1-BCC4-57BA3383466B@muada.com> <4A08943D.1080100@isi.edu> <5DC12C17-2FC5-4728-A3BB-C3EDA3369640@muada.com> <20090512165631.GB14375@verdi> X-Mailer: Apple Mail (2.930.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Shared Bottleneck X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2009 21:59:42 -0000 On 12 mei 2009, at 18:56, John Leslie wrote: > Let's list a few reasons we might wish to avoid shared bottlenecks. > I'll start: > 1) oscillation (it's all to easy to design congestion control that > will > oscillate between two paths, confusing congestion signals for all > and giving less overall bandwidth than a single path); Right, oscillations are never good although I don't think less total bandwidth will happen easily. > 2) over-saturation of the "innocent" end's link (the textbook example > is one user at an enterprise with marginal bandwidth can ruin the > experience of other users at that enterprise); The trouble with Reno congestion control is that the amount of loss on a link goes up with the square of the number of flows. So if everyone starts using 3 subflows average packet loss will go up by a factor 9. Unfortunately it's hard to replace Reno because if you're less aggressive you get less bandwidth so it doesn't pay. > (Others, please feel free to add...) Unfairness. If we both share the bottleneck with a regular flow we would get about a 50% share of the bandwidth each (small print applies) but if I use 3 subflows through that same bottleneck I'll get 75% and you 25%. Although not a huge deal, managing paths that provide no benefits creates unnecessary extra overhead. >> If we simply limit the number of active subflows to something modest >> we're not doing anything that HTTP isn't doing, or that users aren't >> doing. > "Something modest" may be in the eyes of the beholder. Iljitsch, > I believe is thinking three; HTTP used to think four but now is two; > in the extreme case we could get arbitrarily close to one... Yes, it is in the eye of the beholder but don't forget that the internet is built on the principle of "one cuts from thick wood" which is a Dutch expression meaning that we don't waste our time with subleties but only care about getting things approximately right. > Is it worth extensive discussion _before_ the BoF whether our > target should be one or three? First of all, the 3 in draft-van-beijnum-1e-mp-tcp-00.txt is just a placeholder. Do we agree that the magic number is the number of MSSes per RTT that the sum of the cwnds of the subflows belonging to a session is allowed to grow? (We then want to limit the per-subflow cwnd growth to >= 0 and <= 1, the former to avoid one subflow eating up another's cwnd, which would create oscillations, and the latter to avoid that we just grow a single subflow cwnd by a factor X of the normal increase, because that would give us the best performance but no multipath.) In order to be no slower than a regular TCP session using the fastest path, we MUST grow the subflow over the fastest path by 1 MSS per RTT. So if we want to use additional subflows, the aggressiveness factor needs to be > 1. Opinions? I'm saying 3 for now, which is on the high side, but within reason, IMO. Don't forget that if I implement 9000-byte jumboframes I get no less than 36 times more bandwidth than users with 1500-byte packets. Compared to that, a factor 3 is quite modest. > Also, does anyone want to claim that the target should be _less_ > than one? So a multipath TCP would be required to be slower than a regular TCP session? I'm not seeing the big selling point for multipath here... >> it's acceptable to be more aggressive than a single NewReno >> flow by a factor 2 or 3 or so. > Even if it isn't, there's a benefit to content-providers of fault > tolerance and load balancing which chooses the best path for each > flow. That's true. From c.raiciu@cs.ucl.ac.uk Wed May 13 03:32:27 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 5A3613A6FB6 for ; Wed, 13 May 2009 03:32:27 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OiyeMGV9hSjp for ; Wed, 13 May 2009 03:32:25 -0700 (PDT) Received: from bells2.cs.ucl.ac.uk (bells2.cs.ucl.ac.uk [128.16.5.33]) by core3.amsl.com (Postfix) with ESMTP id 4D44A28C1C6 for ; Wed, 13 May 2009 03:32:01 -0700 (PDT) Received: from blackie2.cs.ucl.ac.uk ([128.16.68.58]) by bells2.cs.ucl.ac.uk with esmtpsa (TLSv1:AES128-SHA:128) (C.Raiciu authenticated) (Exim 4.54) id 1M4CIK-0008XH-Iz for multipathtcp@ietf.org; Wed, 13 May 2009 12:06:40 +0100 Mime-Version: 1.0 (Apple Message framework v753.1) References: Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <9AC7B18B-5167-4A14-ADCB-2DABC282C07D@cs.ucl.ac.uk> Content-Transfer-Encoding: 7bit From: Costin Raiciu Date: Wed, 13 May 2009 11:33:28 +0100 To: multipathtcp@ietf.org X-Mailer: Apple Mail (2.753.1) Subject: [multipathtcp] We don't need to detect shared bottlenecks X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 May 2009 10:32:27 -0000 Hi, As this is my first message on this list, I'd like to introduce myself: I am a PhD student at UCL working with Mark Handley and Damon Wischik on multipath TCP. Theory tells us (e.g. Frank Kelly's paper "joint routing and rate control") that we don't need to detect shared bottlenecks explicitly. The rough idea is to make an the ensemble of multipath subflows act as a single one, always assuming they share a bottleneck. Let me give you an example; say we have two sublfows on exactly the same path, with windows w1 and w2, and our strategy is to increase w1 with 1 whenever w1+w2 packets are acknowledged on subflow 1, and with 1 whenever w1 +w2 packets are acknowledged on subflow 2. When we get a drop on subflow 1, we reduce w1 with (w1+w2)/2; if w1 is less than one after the decrease, set it to one. The same goes for subflow 2. This simple strategy makes the aggregate behave like a single tcp flow, with exactly the same aggressiveness. It is true that traffic will move from one flow to the other depending on the , but why should we care? Assuming we can manage reordering of packets, this should be the same as a single flow. As for traffic oscillations at the bottleneck: this will not oscillate worse than TCP already does. So far we have assumed there is a shared bottleneck, and were fair to TCP there. What is we have two completely disjoint paths? The net effect will be that traffic will be pushed towards the paths with the lower drop probability, therefore achieving resource pooling (see "the resource pooling principle" ccr paper for more). As a net effect, if everybody runs multipath tcp, there will be large swades of the network where the drop probability is the same. The discussion in this latter case is more involved (we are currently running experiments to fill the picture). Cheers. Costin From baford@mpi-sws.org Wed May 13 04:45:00 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 22C4F3A6983 for ; Wed, 13 May 2009 04:45:00 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.249 X-Spam-Level: X-Spam-Status: No, score=-6.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wsJuOHCw9E+o for ; Wed, 13 May 2009 04:44:52 -0700 (PDT) Received: from hera.mpi-sb.mpg.de (hera.mpi-sb.mpg.de [139.19.1.49]) by core3.amsl.com (Postfix) with ESMTP id 702C53A67A8 for ; Wed, 13 May 2009 04:44:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mpi-sb.mpg.de; s=mail200803; h=Cc:Message-Id:From:To: In-Reply-To:Content-Type:Content-Transfer-Encoding:Mime-Version: Subject:Date:References; bh=/6F00fIklF6JdloztwxTQ/mbKX7yspQbS8LU RlbVHFE=; b=t8Q3a+i2ofpQzpKlAClYexViMFJ4F4crzRIepiuT4Iq3MWMgBh3y wLAUuffKjXfljJzaA4AS9vwwmC46SihjF8Vvts3jjQodkYsSCmYDiA1Ki7XONJf4 hyulDNhDyQ+zcn8Ug0a8JOwQpoi5HpFmeoLMrQMpxJe/iWxzDO7ZU+Q= Received: from newmaniac.mpi-sb.mpg.de ([139.19.1.26]:35497) by hera.mpi-sb.mpg.de (envelope-from ) with esmtp (Exim 4.69) id 1M4Cug-00081g-RE; Wed, 13 May 2009 13:46:21 +0200 Received: from adsl-84-226-8-58.adslplus.ch ([84.226.8.58]:60009 helo=[192.168.1.100]) by newmaniac.mpi-sb.mpg.de with esmtpsa (TLS-1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.63) (envelope-from ) id 1M4Cuf-0001RF-Vf; Wed, 13 May 2009 13:46:18 +0200 Message-Id: <2FA66F9F-D86F-412A-AA5B-BFCF383704BE@mpi-sws.org> From: Bryan Ford To: John Leslie In-Reply-To: <20090512165631.GB14375@verdi> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Wed, 13 May 2009 13:46:16 +0200 References: <5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com> <20090507230452.GC35382@cisco.com> <4A036A29.7020005@isi.edu> <4A042F28.5050107@it.uc3m.es> <4A046242.4040104@isi.edu> <4A087396.4050001@it.uc3m.es> <4A088CF1.6070201@isi.edu> <3D4CFA5B-A4CD-43C1-BCC4-57BA3383466B@muada.com> <4A08943D.1080100@isi.edu> <5DC12C17-2FC5-4728-A3BB-C3EDA3369640@muada.com> <20090512165631.GB14375@verdi> X-Mailer: Apple Mail (2.930.3) Cc: multipathtcp@ietf.org, Joe Touch Subject: Re: [multipathtcp] Shared Bottleneck X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 May 2009 11:45:00 -0000 On May 12, 2009, at 6:56 PM, John Leslie wrote: > Let's list a few reasons we might wish to avoid shared bottlenecks. > I'll start: > > 1) oscillation (it's all to easy to design congestion control that > will > oscillate between two paths, confusing congestion signals for all > and giving less overall bandwidth than a single path); I've never quite understood exactly what the problem is here. Any reasonable congestion control algorithm for the Internet must be stable in the presence of multiple competing flows sharing the same bottleneck: Reno does this by ensuring via its AIMD response that competing flows converge toward giving each flow a more-or-less even share (at least among flows of similar RTT; different RTTs as we know affect fairness but AFAIK do not affect stability). What does the congestion control algorithm care if two competing flows happen to be fed from the same data source? That only affects the contents of the packets that get sent, not congestion control behavior. If single- path congestion control is stable in the presence of competing independent flows, then a multi-path transport that directs data from one source over multiple otherwise-independent congestion control flows should be equally stable. Maybe you're thinking of a situation where the data source doesn't provide enough data to keep all active flows network-bottlenecked, but even in such situations it seems it should be easy to ensure stability (assuming the underlying congestion control algorithm is stable) merely by not doing something really stupid like trying to shift _all_ data from one path to another just because the first path experiences some congestion. There are at least two "reasonable approaches" I can think of offhand to distributing data among only partially- bottlenecked flows that should trivially ensure stability (again assuming the underlying CC algorithm is stable for independent data sources), and I'm sure there are plenty more. (1) impose a consistent total order on the flows, and always max out the first before sending any data on the second, max out the second before sending data on the third, etc. Then for given source load and network conditions, the same first set of flows will always be network- bottlenecked and behave like independent TCP flows, and the remaining flows will not be network-bottlenecked and thus CC will be irrelevant. This approach might be what you want in asymmetrical "primary/backup link" or "cheap/expensive link" situations. (2) always distribute traffic evenly across all flows up to the point there's no more data to distribute or a given flow becomes network- bottlenecked. Then the network-bottlenecked flows will remain consistently network-bottlenecked and will behave like independent congestion controlled flows even if they compete with each other at some bottlenecks, while for non-network-bottlenecked flows congestion control is irrelevant because it just won't kick in. \Presumably somebody must have already analyzed these types of scenarios formally and/or experimentally, or am I missing something fundamental here? > 2) over-saturation of the "innocent" end's link (the textbook example > is one user at an enterprise with marginal bandwidth can ruin the > experience of other users at that enterprise); > > 3) too much retransmission of lost packets (leading to both loss of > bandwidth and choppy response). These are just instances of the same "aggressiveness amplification" issue you get whenever you use multiple possibly-competing TCP flows to support one logical activity. The same thing happens at application level with HTTP's two-concurrent-connections convention, or with Bittorrent's many-concurrent-connections behavior, especially when most or all of those connections happen to be bottlenecked at the same home or enterprise broadband link. It's not in any way specific to multipath TCP or multipath transports; it's just an artifact of TCP's model of competition and fairness. We routinely live with this problem in the case of HTTP and Bittorrent; why is it suddenly so critical that multipath TCP (or multipath SCTP or whatever) has to solve it before multipath transports can be designed or deployed? And any solution this BOF/WG might come up with for multipath transport probably wouldn't solve the more general problem of fairness and aggressiveness amplification, e.g., in the bittorrent case where the problem results from many different flows to different and administratively unrelated endpoints. I'm not saying that we shouldn't work on solving TCP's aggressiveness amplification or fairness problem for multi-flow usages (either by multi-flow transports or multi-flow applications), just that it seems quite independent and orthogonal to the details of any particular multipath transport, and proposed solutions should ideally consider not just the multipath transport case but also the more general point- to-multipoint case that bittorrent exhibits. Unless there's a really good reason not to - e.g., if the former is possible and the latter is simply not - but I don't believe that and haven't seen any argument to that effect. Cheers, Bryan From baford@mpi-sws.org Wed May 13 07:52:55 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 13BA53A69D6 for ; Wed, 13 May 2009 07:52:55 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.249 X-Spam-Level: X-Spam-Status: No, score=-6.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ydgHlt8whV7A for ; Wed, 13 May 2009 07:52:54 -0700 (PDT) Received: from hera.mpi-sb.mpg.de (infao0809.mpi-sb.mpg.de [139.19.1.49]) by core3.amsl.com (Postfix) with ESMTP id AE8BD3A67A7 for ; Wed, 13 May 2009 07:52:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mpi-sb.mpg.de; s=mail200803; h=Cc:Message-Id:From:To: In-Reply-To:Content-Type:Content-Transfer-Encoding:Mime-Version: Subject:Date:References; bh=RnPCygZy+chPC3aPSpkNLR2/z6k2op3WN3pP znwdMjc=; b=d3TpNh45cWSWdwKmRqN0eM6Kf0DAHh7QukeueWVgIvXSvL1j3K0E vfpJj+pl20urKtYHyPziHCh2njgnxJjsXgnL6ukryL93o05q9x6F3Hzo5mm19XJk R4GG7QxU2+ENQ5fdBOFI4gynDJBmNd5yhZYZLxINLJGJCka4uPXOFFk= Received: from infao0525.mpi-sb.mpg.de ([139.19.1.26]:42288 helo=newmaniac.mpi-sb.mpg.de) by hera.mpi-sb.mpg.de (envelope-from ) with esmtp (Exim 4.69) id 1M4Fqe-0000ip-Lf; Wed, 13 May 2009 16:54:23 +0200 Received: from adsl-84-226-8-58.adslplus.ch ([84.226.8.58]:61012 helo=[192.168.1.100]) by newmaniac.mpi-sb.mpg.de with esmtpsa (TLS-1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.63) (envelope-from ) id 1M4Fqe-0002aa-4e; Wed, 13 May 2009 16:54:20 +0200 Message-Id: <8FDA96DF-96F4-4BFC-AD35-93BC389421C0@mpi-sws.org> From: Bryan Ford To: Costin Raiciu In-Reply-To: <9AC7B18B-5167-4A14-ADCB-2DABC282C07D@cs.ucl.ac.uk> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Wed, 13 May 2009 16:54:18 +0200 References: <9AC7B18B-5167-4A14-ADCB-2DABC282C07D@cs.ucl.ac.uk> X-Mailer: Apple Mail (2.930.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] We don't need to detect shared bottlenecks X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 May 2009 14:52:55 -0000 On May 13, 2009, at 12:33 PM, Costin Raiciu wrote: > Hi, > > As this is my first message on this list, I'd like to introduce > myself: I am a PhD student at UCL working with Mark Handley and > Damon Wischik on multipath TCP. > > Theory tells us (e.g. Frank Kelly's paper "joint routing and rate > control") that we don't need to detect shared bottlenecks > explicitly. The rough idea is > to make an the ensemble of multipath subflows act as a single one, > always assuming they share a bottleneck. > > Let me give you an example; say we have two sublfows on exactly the > same path, with windows w1 and w2, and our strategy is to increase > w1 with 1 whenever > w1+w2 packets are acknowledged on subflow 1, and with 1 whenever > w1+w2 packets are acknowledged on subflow 2. When we get a drop on > subflow 1, > we reduce w1 with (w1+w2)/2; if w1 is less than one after the > decrease, set it to one. The same goes for subflow 2. > > This simple strategy makes the aggregate behave like a single tcp > flow, with exactly the same aggressiveness. > It is true that traffic will move from one flow to the other > depending on the , but why should we care? Assuming we can manage > reordering of packets, this should be the same as > a single flow. > > As for traffic oscillations at the bottleneck: this will not > oscillate worse than TCP already does. > > So far we have assumed there is a shared bottleneck, and were fair > to TCP there. What is we have two completely disjoint paths? The net > effect will be that traffic will > be pushed towards the paths with the lower drop probability, > therefore achieving resource pooling (see "the resource pooling > principle" ccr paper for more). > As a net effect, if everybody runs multipath tcp, there will be > large swades of the network where the drop probability is the same. > The discussion in this latter case is more involved (we are > currently running experiments to fill the picture). I'm somewhat familiar with this joint congestion control work, and it sounds very promising to me - I only wish someone would write a version of that paper that's a bit more readable by non-theory folks (hint, hint ;) ). This absolutely needs to be followed up with more practical experimentation, regardless of whether this BOF/WG proves to be the right place for that to happen. One outstanding question I have is whether this joint rate control approach extends easily to non-Reno congestion control schemes (e.g., TFRC, Microsoft's CTCP, and other "TCP-friendly" alternatives), or explicit schemes like XCP/RCP. Another question is whether this or a similar approach to tying multiple congestion control loops together could apply naturally not just to multipath transports but to multipath _applications_, as in HTTP or BitTorrent for example. It seems to me that this should be relatively easy to accomplish if the multiple flows are to be tied together at a given sender, regardless of whether the flows are to the same receiver (e.g., multipath transport, or multi-connection HTTP) or to different receivers (e.g., BitTorrent). For example, if an OS's TCP stack provided a simple API extension by which applications could indicate that a certain set of TCP connections should be treated jointly for congestion control purposes, then applications like web browsers and BitTorrent clients could use the same mechanism independent of whether TCP or any transport supports multipath natively between a given pair of hosts. How to tie flows together at the receiver side is not so obvious, nor is how receiver-side flow joining would interact with sender-side flow joining when the two sides don't necessarily agree on a common set of send and receive endpoints to be tied together (again, BitTorrent). But regardless of whether or how these problems might be solved, if this BOF/WG is going to spend a lot of time trying to solve the multipath fairness/ agressiveness problem like Frank Kelly's work tries to do, it needs to consider this problem not just in the context of multipath transports (which are still completely nascent and thus really have no economic/ industry impetus at the moment) but in the context of multipath applications (which are already widespread and for which a solution could therefore could provide a much bigger immediate impact). Bryan > > Cheers. > Costin > > > _______________________________________________ > multipathtcp mailing list > multipathtcp@ietf.org > https://www.ietf.org/mailman/listinfo/multipathtcp From iljitsch@muada.com Wed May 13 08:24:09 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 3D62F3A6909 for ; Wed, 13 May 2009 08:24:09 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WhHK-VQQzrZQ for ; Wed, 13 May 2009 08:24:08 -0700 (PDT) Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by core3.amsl.com (Postfix) with ESMTP id 3F07F3A65A5 for ; Wed, 13 May 2009 08:24:08 -0700 (PDT) Received: from [192.168.2.4] (static-167-138-7-89.ipcom.comunitel.net [89.7.138.167] (may be forged)) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n4DFO3tR063518 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Wed, 13 May 2009 17:24:09 +0200 (CEST) (envelope-from iljitsch@muada.com) Message-Id: <6D11F360-2658-428A-9755-5478AE418919@muada.com> From: Iljitsch van Beijnum To: Costin Raiciu In-Reply-To: <9AC7B18B-5167-4A14-ADCB-2DABC282C07D@cs.ucl.ac.uk> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v935.3) Date: Wed, 13 May 2009 17:24:08 +0200 References: <9AC7B18B-5167-4A14-ADCB-2DABC282C07D@cs.ucl.ac.uk> X-Mailer: Apple Mail (2.935.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] We don't need to detect shared bottlenecks X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 May 2009 15:24:09 -0000 On 13 mei 2009, at 12:33, Costin Raiciu wrote: > say we have two sublfows on exactly the same path, with windows w1 > and w2, and our strategy is to increase w1 with 1 whenever > w1+w2 packets are acknowledged on subflow 1, and with 1 whenever > w1+w2 packets are acknowledged on subflow 2. Are you sure? The way I understood this approach is that the different congestion windows for the different subflows grow with the standard 1 MSS per RTT independently, but then when there is congestion: > When we get a drop on subflow 1, > we reduce w1 with (w1+w2)/2; if w1 is less than one after the > decrease, set it to one. The same goes for subflow 2. This is the opposite approach of what I talked about the other day where we may limit the cwnd increase but the decrease after a drop is the normal cwnd/2. There are three problems with the approach you outline: 1. Suppose w1 is 1000 and w2 is 2000. After a drop on subflow 1 you would have to decrease w1 by 1500, making it -500. That doesn't work. 2. cwnd in isolation is meaningless. I can have 10000 bytes/s with a cwnd of only 10 bytes (RTT 1 ms) or with a cwnd of 2500 (250 ms). Adding up cwnds that have different RTTs creates meaningless results. I believe in your tests you saw this reflected in the fact that higher RTT paths are preferred in this scheme. 3. Growing fast and backing off hard makes the sawtooth cycles faster so the network becomes burstier. Iljitsch From c.raiciu@cs.ucl.ac.uk Wed May 13 10:12:58 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id BA4463A6906 for ; Wed, 13 May 2009 10:12:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id US2NL93YxPvx for ; Wed, 13 May 2009 10:12:58 -0700 (PDT) Received: from bells2.cs.ucl.ac.uk (bells2.cs.ucl.ac.uk [128.16.5.33]) by core3.amsl.com (Postfix) with ESMTP id DA3AA3A68D9 for ; Wed, 13 May 2009 10:12:57 -0700 (PDT) Received: from blackie2.cs.ucl.ac.uk ([128.16.68.58]) by bells2.cs.ucl.ac.uk with esmtpsa (TLSv1:AES128-SHA:128) (C.Raiciu authenticated) (Exim 4.54) id 1M4IY9-0008bp-5a for multipathtcp@ietf.org; Wed, 13 May 2009 18:47:25 +0100 Mime-Version: 1.0 (Apple Message framework v753.1) In-Reply-To: <6D11F360-2658-428A-9755-5478AE418919@muada.com> References: <9AC7B18B-5167-4A14-ADCB-2DABC282C07D@cs.ucl.ac.uk> <6D11F360-2658-428A-9755-5478AE418919@muada.com> Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Costin Raiciu Date: Wed, 13 May 2009 18:14:20 +0100 To: multipathtcp@ietf.org X-Mailer: Apple Mail (2.753.1) Subject: Re: [multipathtcp] We don't need to detect shared bottlenecks X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 May 2009 17:12:58 -0000 Hi, > Are you sure? > > The way I understood this approach is that the different congestion > windows for the different subflows grow with the standard 1 MSS per > RTT independently, but then when there is congestion: To get resource pooling you must couple the increases too. There are other ways that you could achieve fairness, e.g. increase with 1/2 per RTT, but those don't get you resource pooling. > >> When we get a drop on subflow 1, >> we reduce w1 with (w1+w2)/2; if w1 is less than one after the >> decrease, set it to one. The same goes for subflow 2. > > This is the opposite approach of what I talked about the other day > where we may limit the cwnd increase but the decrease after a drop > is the normal cwnd/2. > > There are three problems with the approach you outline: > > 1. Suppose w1 is 1000 and w2 is 2000. After a drop on subflow 1 you > would have to decrease w1 by 1500, making it -500. That doesn't work. I agree there will be "clipping effects" because you can't set cwnd less than 1. To mitigate this, you can tweak the increase/decrease parameters so that you are still tcp fair, but avoid clipping most of the time. E.g. increase with 1/2 packet per RTT (for the whole connection) and drop with 1/4 of the total cwnd. > > 2. cwnd in isolation is meaningless. I can have 10000 bytes/s with > a cwnd of only 10 bytes (RTT 1 ms) or with a cwnd of 2500 (250 ms). > Adding up cwnds that have different RTTs creates meaningless > results. I believe in your tests you saw this reflected in the fact > that higher RTT paths are preferred in this scheme. This is true. Because of the RTT bias, it looks like we will have to introduces rates in this computation, instead of cwnds. > > 3. Growing fast and backing off hard makes the sawtooth cycles > faster so the network becomes burstier. Not if you choose milder parameters. Anyways, bursty is not always bad; for one thing, it breaks synchronization between flows. Costin From iljitsch@muada.com Wed May 13 14:18:58 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id C85ED3A6B87 for ; Wed, 13 May 2009 14:18:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DRDcr5J9kuGJ for ; Wed, 13 May 2009 14:18:52 -0700 (PDT) Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by core3.amsl.com (Postfix) with ESMTP id B54C03A6A1F for ; Wed, 13 May 2009 14:18:51 -0700 (PDT) Received: from [192.168.2.4] (static-167-138-7-89.ipcom.comunitel.net [89.7.138.167] (may be forged)) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n4DLJSVw066676 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Wed, 13 May 2009 23:19:30 +0200 (CEST) (envelope-from iljitsch@muada.com) Message-Id: From: Iljitsch van Beijnum To: Costin Raiciu In-Reply-To: Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v935.3) Date: Wed, 13 May 2009 23:19:33 +0200 References: <9AC7B18B-5167-4A14-ADCB-2DABC282C07D@cs.ucl.ac.uk> <6D11F360-2658-428A-9755-5478AE418919@muada.com> X-Mailer: Apple Mail (2.935.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] We don't need to detect shared bottlenecks X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 May 2009 21:18:58 -0000 On 13 mei 2009, at 19:14, Costin Raiciu wrote: > To get resource pooling you must couple the increases too. No, I don't think so... coupling one should be enough. > There are other ways that you could achieve fairness, e.g. increase > with 1/2 per RTT, but those don't get you resource pooling. (For those who haven't read the resource pooling paper: the idea is if you have a bunch of links (paths) where each user can use a subset of the links, we make the users react to congestion on each link by shifting traffic to less congested ones such that the set of links behaves like a single big one.) Even if you don't couple at all so an MPTCP takes the same amount of capacity on a given link, you get weak resource pooling because the additional path makes the transfer complete faster so the link we're interested in sees less traffic than in the single path case. >> 1. Suppose w1 is 1000 and w2 is 2000. After a drop on subflow 1 you >> would have to decrease w1 by 1500, making it -500. That doesn't work. > I agree there will be "clipping effects" because you can't set cwnd > less than 1. An unfortunate side effect of this is that it makes the behavior of the algorithm hard to model. > This is true. Because of the RTT bias, it looks like we will have to > introduces rates in this computation, instead of cwnds. Right. However, I think with all these changes you're now so far from Reno that you've basically created a completely new cc scheme, and we'd have to carefully model it or run experiments to see how it behaves compared to Reno. Not having done that, I expect that your scheme (TCP London?) can't match the performance of (New)Reno on the fastest path in all cases, so there could be situations where enabling MPTCP reduces performance. That would be a problem. >> 3. Growing fast and backing off hard makes the sawtooth cycles >> faster so the network becomes burstier. > Not if you choose milder parameters. Anyways, bursty is not always > bad; for one thing, it breaks synchronization between flows. Burstier means more losses for the same utilization and buffering, which means reduced performance in a loss based cc. From c.raiciu@cs.ucl.ac.uk Thu May 14 01:37:58 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 672733A6A62 for ; Thu, 14 May 2009 01:37:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.299 X-Spam-Level: X-Spam-Status: No, score=-2.299 tagged_above=-999 required=5 tests=[AWL=-0.300, BAYES_00=-2.599, J_CHICKENPOX_47=0.6] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rnyM-PJSUUYQ for ; Thu, 14 May 2009 01:37:57 -0700 (PDT) Received: from bells2.cs.ucl.ac.uk (bells2.cs.ucl.ac.uk [128.16.5.33]) by core3.amsl.com (Postfix) with ESMTP id 6614D3A6F21 for ; Thu, 14 May 2009 01:37:57 -0700 (PDT) Received: from blackie2.cs.ucl.ac.uk ([128.16.68.58]) by bells2.cs.ucl.ac.uk with esmtpsa (TLSv1:AES128-SHA:128) (C.Raiciu authenticated) (Exim 4.54) id 1M4Wz1-0008jk-M3; Thu, 14 May 2009 10:12:07 +0100 In-Reply-To: <8FDA96DF-96F4-4BFC-AD35-93BC389421C0@mpi-sws.org> References: <9AC7B18B-5167-4A14-ADCB-2DABC282C07D@cs.ucl.ac.uk> <8FDA96DF-96F4-4BFC-AD35-93BC389421C0@mpi-sws.org> Mime-Version: 1.0 (Apple Message framework v753.1) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Costin Raiciu Date: Thu, 14 May 2009 09:39:27 +0100 To: Bryan Ford X-Mailer: Apple Mail (2.753.1) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] We don't need to detect shared bottlenecks X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 May 2009 08:37:58 -0000 Hi, > One outstanding question I have is whether this joint rate control > approach extends easily to non-Reno congestion control schemes > (e.g., TFRC, Microsoft's CTCP, and other "TCP-friendly" > alternatives), or explicit schemes like XCP/RCP. We are finding it difficult to retrofit it to Reno :), so I don't know about the others. As I hinted in the previous email, the RTT dependence of throughput makes it difficult to use cwnd to get resource pooling. If all RTTs are the same it works, otherwise not necessarily. I would guess its easier for protocols where RTTs do not determine achieved throughput, but I don't know for sure. > > Another question is whether this or a similar approach to tying > multiple congestion control loops together could apply naturally > not just to multipath transports but to multipath _applications_, > as in HTTP or BitTorrent for example. It seems to me that this > should be relatively easy to accomplish if the multiple flows are > to be tied together at a given sender, regardless of whether the > flows are to the same receiver (e.g., multipath transport, or multi- > connection HTTP) or to different receivers (e.g., BitTorrent). For > example, if an OS's TCP stack provided a simple API extension by > which applications could indicate that a certain set of TCP > connections should be treated jointly for congestion control > purposes, then applications like web browsers and BitTorrent > clients could use the same mechanism independent of whether TCP or > any transport supports multipath natively between a given pair of > hosts. How to tie flows together at the receiver side is not so > obvious, nor is how receiver-side flow joining would interact with > sender-side flow joining when the two sides don't necessarily agree > on a common set of send and receive endpoints to be tied together > (again, BitTorrent). But regardless of whether or how these > problems might be solved, if this BOF/WG is going to spend a lot of > time trying to solve the multipath fairness/agressiveness problem > like Frank Kelly's work tries to do, it needs to consider this > problem not just in the context of multipath transports (which are > still completely nascent and thus really have no economic/industry > impetus at the moment) but in the context of multipath applications > (which are already widespread and for which a solution could > therefore could provide a much bigger immediate impact). These are interesting observations. Mark has a student working on multi-server http, which is the opposite of what you propose: the receiver will fetch different data from different replicated servers, downloading more from servers that are closer. Here there is no explicit coupling of CC, but still we get job level resource pooling - users with fast paths will tend to push the download of web objects on those paths. Since web objects are small/moderately sized, the busy links will be utilized less for users with fast alternatives. If, however, web pages were infinite in size (or requests will follow immediately after each downloaded page, in an infinite loop) we don't get resource pooling anymore: the users with fast links will still utilize the busy links all the time,getting an overall unfair share of the network. It is an interesting idea to couple the flows at the sender with different receivers like you propose in BitTorrent. It seems like the effect would be to get resource pooling in that case too: if there are multiple replicas of all files, traffic will flow mostly on uncongested links. This could have perverse outcomes too, for users that only have congested links to the data sources: they will get very little throughput. Also, at first glance, this seems to break the tit-for-tat incentive scheme. Taking a step further, one could couple CC all flows coming out of a given machine, which would get you resource pooling, but likely with disastrous effects :). Cheers Costin From micchie@sfc.wide.ad.jp Thu May 14 02:23:03 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 87CF93A6B79 for ; Thu, 14 May 2009 02:23:03 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.904 X-Spam-Level: X-Spam-Status: No, score=0.904 tagged_above=-999 required=5 tests=[AWL=0.001, BAYES_00=-2.599, HELO_EQ_JP=1.244, HOST_EQ_JP=1.265, RELAY_IS_203=0.994] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wesg9p+qMBP4 for ; Thu, 14 May 2009 02:23:02 -0700 (PDT) Received: from mail.sfc.wide.ad.jp (mail.sfc.wide.ad.jp [203.178.142.146]) by core3.amsl.com (Postfix) with ESMTP id 4F5433A67D4 for ; Thu, 14 May 2009 02:23:02 -0700 (PDT) Received: from epi.ht.sfc.keio.ac.jp (unknown [IPv6:2001:200:1c0:2800:21f:5bff:fe81:455f]) by mail.sfc.wide.ad.jp (Postfix) with ESMTPSA id 5B2D74C76B for ; Thu, 14 May 2009 18:29:36 +0900 (JST) Message-ID: <4A0BE352.7050209@sfc.wide.ad.jp> Date: Thu, 14 May 2009 18:24:34 +0900 From: Michio HONDA User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; ja-JP-mac; rv:1.9.1b3pre) Gecko/20090223 Thunderbird/3.0b2 MIME-Version: 1.0 To: multipathtcp@ietf.org References: <9AC7B18B-5167-4A14-ADCB-2DABC282C07D@cs.ucl.ac.uk> <6D11F360-2658-428A-9755-5478AE418919@muada.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [multipathtcp] We don't need to detect shared bottlenecks X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 May 2009 09:23:03 -0000 Hi, (5/14/09 6:19 AM), Iljitsch van Beijnum wrote: > On 13 mei 2009, at 19:14, Costin Raiciu wrote: > >> To get resource pooling you must couple the increases too. > > No, I don't think so... coupling one should be enough. I agree with Iljitsch. > Let me give you an example; say we have two sublfows on exactly the same path, with windows w1 and w2, and our strategy is to increase w1 with 1 > whenever > w1+w2 packets are acknowledged on subflow 1, and with 1 whenever w1+w2 packets are acknowledged on subflow 2. When we get a drop on > subflow 1, > we reduce w1 with (w1+w2)/2; if w1 is less than one after the decrease, set it to one. The same goes for subflow 2. I don't think this approach works, because it looks that behavior on one subflow affects the other subflows, reducing overall performance of the bundle of subflows. > >> There are other ways that you could achieve fairness, e.g. increase >> with 1/2 per RTT, but those don't get you resource pooling. This approach is appropriate (more exactly, increase with (1/2)^2 per RTT based on the TCP response function). Costin, could you please why this approach doesn't get resource pooling? In addition, now, the discussion mixes up the resource pooling and the shared bottleneck. In fact, resource-pooling congestion control and shared bottleneck aware congestion control work similarly, however, we would need separate consideration between them, because fairness in the multipath case is not clear now. Do we actually need to share resources equally between different bottlenecks? We might not need to share equally resources across different bottleneck. > > (For those who haven't read the resource pooling paper: the idea is if > you have a bunch of links (paths) where each user can use a subset of > the links, we make the users react to congestion on each link by > shifting traffic to less congested ones such that the set of links > behaves like a single big one.) > > Even if you don't couple at all so an MPTCP takes the same amount of > capacity on a given link, you get weak resource pooling because the > additional path makes the transfer complete faster so the link we're > interested in sees less traffic than in the single path case. > >>> 1. Suppose w1 is 1000 and w2 is 2000. After a drop on subflow 1 you >>> would have to decrease w1 by 1500, making it -500. That doesn't work. > >> I agree there will be "clipping effects" because you can't set cwnd >> less than 1. > > An unfortunate side effect of this is that it makes the behavior of > the algorithm hard to model. > >> This is true. Because of the RTT bias, it looks like we will have to >> introduces rates in this computation, instead of cwnds. > > Right. > > However, I think with all these changes you're now so far from Reno > that you've basically created a completely new cc scheme, and we'd > have to carefully model it or run experiments to see how it behaves > compared to Reno. > > Not having done that, I expect that your scheme (TCP London?) can't > match the performance of (New)Reno on the fastest path in all cases, > so there could be situations where enabling MPTCP reduces performance. > That would be a problem. > >>> 3. Growing fast and backing off hard makes the sawtooth cycles >>> faster so the network becomes burstier. > >> Not if you choose milder parameters. Anyways, bursty is not always >> bad; for one thing, it breaks synchronization between flows. > > Burstier means more losses for the same utilization and buffering, > which means reduced performance in a loss based cc. > _______________________________________________ > multipathtcp mailing list > multipathtcp@ietf.org > https://www.ietf.org/mailman/listinfo/multipathtcp -- Michio Honda E-mail: micchie@sfc.wide.ad.jp, micchie@ht.sfc.keio.ac.jp Website: http://www.micchie.net/ From David.Ros@telecom-bretagne.eu Thu May 14 07:41:47 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 4FB323A6AE1 for ; Thu, 14 May 2009 07:41:47 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.124 X-Spam-Level: X-Spam-Status: No, score=-3.124 tagged_above=-999 required=5 tests=[AWL=-0.875, BAYES_00=-2.599, HELO_EQ_FR=0.35] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EX4AR9HbLirO for ; Thu, 14 May 2009 07:41:46 -0700 (PDT) Received: from laposte.rennes.enst-bretagne.fr (laposte.rennes.enst-bretagne.fr [192.44.77.17]) by core3.amsl.com (Postfix) with ESMTP id C4F8C28C27D for ; Thu, 14 May 2009 07:41:35 -0700 (PDT) Received: from localhost (localhost.localdomain [127.0.0.1]) by laposte.rennes.enst-bretagne.fr (8.13.7/8.13.7/2006.08.14) with ESMTP id n4EEh23e014527; Thu, 14 May 2009 16:43:02 +0200 Received: from l1.rennes.enst-bretagne.fr (l1.rennes.enst-bretagne.fr [192.44.77.3]) by laposte.rennes.enst-bretagne.fr (8.13.7/8.13.7/2008.01.11) with ESMTP id n4EEguC5014508; Thu, 14 May 2009 16:43:00 +0200 Received: from [10.0.1.2] (ARennes-351-1-57-109.w82-126.abo.wanadoo.fr [82.126.61.109]) (authenticated bits=0) by l1.rennes.enst-bretagne.fr (8.13.1/8.13.1) with ESMTP id n4EEgtwi030220 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Thu, 14 May 2009 16:42:56 +0200 Message-Id: From: David Ros To: Costin Raiciu In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed; delsp=yes Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Apple Message framework v930.3) Date: Thu, 14 May 2009 16:43:11 +0200 References: <9AC7B18B-5167-4A14-ADCB-2DABC282C07D@cs.ucl.ac.uk> <6D11F360-2658-428A-9755-5478AE418919@muada.com> X-Mailer: Apple Mail (2.930.3) X-Virus-Scanned: amavisd-new at enst-bretagne.fr Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] We don't need to detect shared bottlenecks X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 May 2009 14:41:47 -0000 Le 13 mai 09 =E0 19:14, Costin Raiciu a =E9crit : > >> >> 3. Growing fast and backing off hard makes the sawtooth cycles =20 >> faster so the network becomes burstier. > Not if you choose milder parameters. Anyways, bursty is not always =20 > bad; for one thing, it breaks synchronization between flows. > Hi, Could you please elaborate a little on this? (I'm surely missing something obvious, but what you say regarding =20 burstiness seems to somewhat contradict what some people [including =20 us] have found, at least in the context of high-speed TCPs) Regards, David. From iljitsch@muada.com Thu May 14 12:04:19 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id A689D3A6C62 for ; Thu, 14 May 2009 12:04:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tKaypahLX5ec for ; Thu, 14 May 2009 12:04:18 -0700 (PDT) Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by core3.amsl.com (Postfix) with ESMTP id DA47D3A6DA6 for ; Thu, 14 May 2009 12:02:56 -0700 (PDT) Received: from [192.168.2.4] (static-167-138-7-89.ipcom.comunitel.net [89.7.138.167] (may be forged)) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n4EJ43va074285 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO) for ; Thu, 14 May 2009 21:04:04 +0200 (CEST) (envelope-from iljitsch@muada.com) Message-Id: From: Iljitsch van Beijnum To: multipathtcp@ietf.org Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v935.3) Date: Thu, 14 May 2009 21:04:09 +0200 X-Mailer: Apple Mail (2.935.3) Subject: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 May 2009 19:04:19 -0000 In the BoF proposal it talks about "a set of architectural guidelines for congestion dependent multipath transport protocols". We now have two drafts for multipath TCP and SCTP could also be one of those with relatively small changes. And then DCCP with some more changes. And over beverages in Stockholm I can explain why I think cngestion dependent multipath is useful for inelastic traffic such as a/v streaming over RTP. So if we have a shared architecture, does it make sense to implement the multipath congestion control up to five separate times? Wouldn't it make more sense to define an abstraction layer between the transport protocol and the congestion control algorithm? That would make it much easier to make transports multipath-capable by simply linking them against the appropriate MPCC library. It would also make it much easier to work on multipath congestion control because the congestion control mechanism can then be a simple state machine that reacts to different events inside the transport protocol that the transport protocol exposes through a standardized interface rather than code spread all over the transport protocol implementation. From janardhan.iyengar@fandm.edu Thu May 14 13:54:45 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 857E23A6D27 for ; Thu, 14 May 2009 13:54:45 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nQHzMjHdlo6A for ; Thu, 14 May 2009 13:54:44 -0700 (PDT) Received: from zimfe2.fandm.edu (zimfe2.fandm.edu [155.68.1.75]) by core3.amsl.com (Postfix) with ESMTP id 5ED993A6B95 for ; Thu, 14 May 2009 13:54:44 -0700 (PDT) Received: from surutti.fandm.edu (unknown [155.68.152.30]) by zimfe2.fandm.edu (Postfix) with ESMTP id 7D8AB1168039; Thu, 14 May 2009 16:56:16 -0400 (EDT) Message-ID: <4A0C8570.8010001@fandm.edu> Date: Thu, 14 May 2009 16:56:16 -0400 From: Janardhan Iyengar Organization: Franklin & Marshall College User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: marcelo bagnulo braun References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es><4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><20090507230452.GC35382@cisco.com><1241789497.4055.8.camel@nhuynhth-laptop> <001e01c9cfff$bde8f1e0$103947ab@cisco.com> <97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com> <001f01c9d005$0c583110$103947ab@cisco.com> <4A051B25.4040702@fandm.edu> <4A0877EF.7090306@it.uc3m.es> In-Reply-To: <4A0877EF.7090306@it.uc3m.es> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: janardhan.iyengar@fandm.edu List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 May 2009 20:54:45 -0000 >>> Then there is the issue that SCTP must of course be implemented on >>> both sides, while the big feature in the one-ended MPTCP is that it >>> can work if it's only implemented in the sender. >> >> >> But to be useful, one-ended MPTCP will need to be understood by >> routers/middleboxes. > right, but only by the middleboxes in control of the side supporting the > OmTCP. I don't think there is need for middlebox support on the side > that is not MTCP capable. > So, the whole point is that if one end updates their servers and > potentially their middleboxes, they can support this. That assumes that the entity that updates the servers also controls _all_ middleboxes that will see incomplete connection information. What happens if I want to employ MPTCP and my ISP uses NATs? - jana -- Janardhan Iyengar Assistant Professor, Computer Science Franklin & Marshall College http://www.fandm.edu/jiyengar From janardhan.iyengar@fandm.edu Thu May 14 14:14:41 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 08F0D28C167 for ; Thu, 14 May 2009 14:14:41 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sMeEImellPwR for ; Thu, 14 May 2009 14:14:40 -0700 (PDT) Received: from zimfe2.fandm.edu (zimfe2.fandm.edu [155.68.1.75]) by core3.amsl.com (Postfix) with ESMTP id CCB4C28C17D for ; Thu, 14 May 2009 14:14:39 -0700 (PDT) Received: from surutti.fandm.edu (unknown [155.68.152.30]) by zimfe2.fandm.edu (Postfix) with ESMTP id 9EE58116802D; Thu, 14 May 2009 17:16:12 -0400 (EDT) Message-ID: <4A0C8A1C.608@fandm.edu> Date: Thu, 14 May 2009 17:16:12 -0400 From: Janardhan Iyengar Organization: Franklin & Marshall College User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: marcelo bagnulo braun References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es><4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><20090507230452.GC35382@cisco.com><1241789497.4055.8.camel@nhuynhth-laptop> <001e01c9cfff$bde8f1e0$103947ab@cisco.com> <97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com> <001f01c9d005$0c583110$103947ab@cisco.com> <000001c9d028$aea92640$103947ab@cisco.com> <4A051E47.7090400@fandm.edu> <4A087A1A.4090303@it.uc3m.es> In-Reply-To: <4A087A1A.4090303@it.uc3m.es> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] Draft Multipath TCP (MPTCP) BOF description X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: janardhan.iyengar@fandm.edu List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 May 2009 21:14:41 -0000 >> My proposal: scope the BoF more broadly. Interested parties can then >> speak up about multipath for different transports and different points >> in the design space. There will be shared goals, and lessons to be >> learned from one another. Perhaps there can be a shared design >> considerations document, > > right, that seems in the scope for the proposed charter imho I am not able to read it in the charter. "In particular, the proposed work is to extend TCP to support simultaneous data exchange through multiple paths" pretty clearly seems to restrict the charter to MPTCP only. >> and then separate documents per transport. Perhaps two per transport. >> > > i am not sure about this one. I am not sure if a single group would be > the ideal place to discuss all transports It is not ideal if you scope the group to be a "TCP" group, and not a "transport" group. There are ideas that are beyond specific transports---Bryan mentioned our HotNets08 paper that I am hoping to present at this IETF that would fit this bill perfectly---that need to be discussed. Sure we want to deal with engineering issues at the IETF, but it is important for the engineering effort to be focussed in a mutually agreed upon direction. Opening up the floor to discussion about multipath transport is necessary, IMHO. And if we agree on how multipath should be done, interested parties could write up drafts on how different transports would map to it. I understand the concern about scoping too broadly and not getting anything done, but the concern on the other side is that scoping too narrowly may yield suboptimal designs for the long term. >> Others have said this, and I agree that "one-sided" multipath with >> router support does not seem to be scoped right... it seems to belong >> elsewhere, under the transport and independent of it. Or am I missing >> something? >> > mmm, have you read the draft? Yes, I did. And I am still not clear on why one-sided "options" are TCP options; from where I'm seeing it, the option should be transport independent. What am I missing? - jana -- Janardhan Iyengar Assistant Professor, Computer Science Franklin & Marshall College http://www.fandm.edu/jiyengar From janardhan.iyengar@fandm.edu Thu May 14 14:33:33 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id E85163A6B7A for ; Thu, 14 May 2009 14:33:33 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KhsqSTV+RDQf for ; Thu, 14 May 2009 14:33:33 -0700 (PDT) Received: from zimfe2.fandm.edu (zimfe2.fandm.edu [155.68.1.75]) by core3.amsl.com (Postfix) with ESMTP id AF2D53A6B46 for ; Thu, 14 May 2009 14:33:32 -0700 (PDT) Received: from surutti.fandm.edu (unknown [155.68.152.30]) by zimfe2.fandm.edu (Postfix) with ESMTP id 596C51168039; Thu, 14 May 2009 17:35:05 -0400 (EDT) Message-ID: <4A0C8E89.9080000@fandm.edu> Date: Thu, 14 May 2009 17:35:05 -0400 From: Janardhan Iyengar Organization: Franklin & Marshall College User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: John Leslie References: <1241789497.4055.8.camel@nhuynhth-laptop> <001e01c9cfff$bde8f1e0$103947ab@cisco.com> <97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com> <001f01c9d005$0c583110$103947ab@cisco.com> <1241818843.26211.67.camel@nhuynhth-laptop> <0D6E7BC4-17AF-4F5C-9107-ABA2B729683F@muada.com> <20090509184241.GT32848@verdi> In-Reply-To: <20090509184241.GT32848@verdi> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] TCP/SCTP and BoF description, was: Re: TCP modification vs switching to SCTP X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: janardhan.iyengar@fandm.edu List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 May 2009 21:33:34 -0000 John, >> 2. Multipath mechanisms. TCP doesn't have these, and we now have two >> drafts proposing them. There are papers and even implementations >> skinning this cat in different ways. We currently have a single >> address and a multiaddress proposal. Would of course be nice to be >> able to do both, as each has limitations that the other doesn't have. >> SCTP does multiaddress multipath and it could be extended with the >> single address multipath similar to TCP but the details will be >> different (option space etc) (2a: TCP, 2b: SCTP) > > We shouldn't gloss over these differences. I believe a TCP-only > implementation can be done more quickly, and be deployed much more > quickly for a large-scale experiment (which I believe necessary to > establish _practical_ congestion-control principles). That assumes that there will be a single mechanism that will apply equally to all transports, and we are not there yet. My design for CMT is different from the one proposed in the draft for MPTCP, and it will be worthwhile discussing the differences (CMT uses a single sequence number space, MPTCP and SST use two sequence spaces; there are arguments for both sides.) TCP considerations will be different than ones for SCTP (option space limitations, for instance), and I expect they will be *very* different for DCCP (different ack mechanisms). Again, I think that this BoF should be broadly scoped to allow for multiple parallel/competing architectures. We should be cautious not to scope the group too narrowly yet. I sincerely think that there are shared questions here that cut across transports, and there will be significant differences. Architecturally, this conversation is exactly where we should discuss where multipath belongs on the stack. Why not have a common flow layer for *all* transports that does multipath? We don't have to agree to split up the transport (as Bryan and I have proposed), but it will help us see the questions here from a perspective that is beyond TCP. And we can then map the answers back to TCP. Incidentally, this is how I remember the "Resource-Pooling" paper as being scoped. - jana -- Janardhan Iyengar Assistant Professor, Computer Science Franklin & Marshall College http://www.fandm.edu/jiyengar From jta@fe.up.pt Thu May 14 16:03:07 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 9C30A3A686C for ; Thu, 14 May 2009 16:03:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.299 X-Spam-Level: X-Spam-Status: No, score=-6.299 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Rnq5QqsCU5Il for ; Thu, 14 May 2009 16:03:07 -0700 (PDT) Received: from dax.ee.ucl.ac.uk (dax.ee.ucl.ac.uk [128.40.42.12]) by core3.amsl.com (Postfix) with ESMTP id B754E3A67EA for ; Thu, 14 May 2009 16:03:06 -0700 (PDT) Received: from [192.168.0.5] (79-68-253-0.dynamic.dsl.as9105.com [79.68.253.0]) (authenticated bits=0) by dax.ee.ucl.ac.uk (8.13.8/8.13.8) with ESMTP id n4EN3RER014386 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 15 May 2009 00:03:32 +0100 (BST) Message-ID: <4A0CA1D0.4040802@fe.up.pt> Date: Thu, 14 May 2009 23:57:20 +0100 From: =?ISO-8859-1?Q?Jo=E3o_Taveira_Ara=FAjo?= Organization: UCL User-Agent: Mozilla-Thunderbird 2.0.0.19 (X11/20090103) MIME-Version: 1.0 To: Iljitsch van Beijnum References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-UCL_EE-MailScanner-Information: Please contact mailhelp@ee.ucl.ac.uk for more information X-MailScanner-ID: n4EN3RER014386 X-UCL_EE-MailScanner: Found to be clean X-UCL_EE-MailScanner-From: jta@fe.up.pt Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: j.araujo@ucl.ac.uk List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 May 2009 23:03:07 -0000 Iljitsch van Beijnum wrote: > So if we have a shared architecture, does it make sense to implement > the multipath congestion control up to five separate times? > > Wouldn't it make more sense to define an abstraction layer between the > transport protocol and the congestion control algorithm? This makes sense and reminds me of Bryan's case for decoupling CC and transport semantics in "Breaking up the transport logjam". But are we drifting away from the original motivation for extending TCP for multipath, namely for deployment purposes. Is the purpose of the eventual WG to work on multipath as a general concept and attempt to retrofit these results to transport protocols, or to take into account the limitations TCP and retain backwards compatibility whilst enabling multipath? Cheers, Joao. From c.raiciu@cs.ucl.ac.uk Thu May 14 16:12:53 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 6B05D28C12F for ; Thu, 14 May 2009 16:12:53 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FbKb8dH1+LdC for ; Thu, 14 May 2009 16:12:52 -0700 (PDT) Received: from bells2.cs.ucl.ac.uk (bells2.cs.ucl.ac.uk [128.16.5.33]) by core3.amsl.com (Postfix) with ESMTP id 333633A6DEA for ; Thu, 14 May 2009 16:12:52 -0700 (PDT) Received: from [82.45.130.212] (helo=[172.24.121.200]) by bells2.cs.ucl.ac.uk with esmtpsa (TLSv1:AES128-SHA:128) (C.Raiciu authenticated) (Exim 4.54) id 1M4kdP-000908-8d for multipathtcp@ietf.org; Fri, 15 May 2009 00:46:43 +0100 Mime-Version: 1.0 (Apple Message framework v753.1) In-Reply-To: <4A0BE352.7050209@sfc.wide.ad.jp> References: <9AC7B18B-5167-4A14-ADCB-2DABC282C07D@cs.ucl.ac.uk> <6D11F360-2658-428A-9755-5478AE418919@muada.com> <4A0BE352.7050209@sfc.wide.ad.jp> Content-Type: multipart/mixed; boundary=Apple-Mail-5--104826152 Message-Id: <6B03E9AF-C96F-4606-9754-9F2629C4C80F@cs.ucl.ac.uk> From: Costin Raiciu Date: Fri, 15 May 2009 00:14:22 +0100 To: multipathtcp@ietf.org X-Mailer: Apple Mail (2.753.1) Subject: Re: [multipathtcp] We don't need to detect shared bottlenecks X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 May 2009 23:12:53 -0000 --Apple-Mail-5--104826152 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Hi, > > (5/14/09 6:19 AM), Iljitsch van Beijnum wrote: >> On 13 mei 2009, at 19:14, Costin Raiciu wrote: >> >>> To get resource pooling you must couple the increases too. >> >> No, I don't think so... coupling one should be enough. > I agree with Iljitsch. > > Let me give you an example; say we have two sublfows on exactly > the same path, with windows w1 and w2, and our strategy is to > increase w1 with 1 > whenever > > w1+w2 packets are acknowledged on subflow 1, and with 1 whenever > w1+w2 packets are acknowledged on subflow 2. When we get a drop on > > subflow 1, > > we reduce w1 with (w1+w2)/2; if w1 is less than one after the > decrease, set it to one. The same goes for subflow 2. > > I don't think this approach works, because it looks that behavior > on one subflow affects the other subflows, reducing overall > performance of the bundle of subflows. That is the whole idea with resource pooling; people will tend to use better paths always, therefore increasing the efficiency of the network. Nobody guarantees that if you do multipath at any given user you will get double the throughput. > >> >>> There are other ways that you could achieve fairness, e.g. >>> increase with 1/2 per RTT, but those don't get you resource pooling. > This approach is appropriate (more exactly, increase with (1/2)^2 > per RTT based on the TCP response function). > Costin, could you please why this approach doesn't get resource > pooling? Below is an example of why not coupling the increases doesn't achieve resource pooling. I attach are three plots obtained by running different multipath cc algorithms in a simple cwnd simulator (that I tested against a real implementation, so I have a bit of confidence it does not produce garbage :) ). The algos tested are coupling both increase and decrease, coupling only decrease, and not coupling anything. The first two algorithms are both fair at the bottleneck (they increase and decrease like a single flow); the last one is not fair, and is just given as reference. In all runs there are two paths, one that drops 1 packet in 1000, and one that drops 1 packet in 500 (the drops are periodic to get prettier graphs; we get the same qualitative output for random drops). To get rate-level resource pooling, the CC algorithm must push all its traffic on the path with fewer drops. If everybody does this, the net effect is that large parts of the network will have the same drop probability! As the graphs show, the coupled algorithm comes closest to this goal, though it's not perfect: it still probes the lossier path, albeit at a slow rate. The version with coupled decreases only is much more aggressive on the increase for the bad path than the coupled version, that is why it gets more window on that in the end. Thus it is worse at resource pooling. Finally, the uncoupled version is even worse. --Apple-Mail-5--104826152 Content-Transfer-Encoding: base64 Content-Type: image/png; x-unix-mode=0644; name=coupled_increases.png Content-Disposition: inline; filename=coupled_increases.png iVBORw0KGgoAAAANSUhEUgAAAoAAAAHgCAMAAAACDyzWAAABKVBMVEX///8AAACgoKD/AAAAwAAA gP/AAP8A7u7AQADu7gAgIMD/wCAAgECggP+AQAD/gP8AwGAAwMAAYIDAYIAAgABA/4AwYICAYABA QEBAgAAAAICAYBCAYGCAYIAAAMAAAP8AYADjsMBAwIBgoMBgwABgwKCAAACAAIBgIIBgYGAgICAg QEAgQIBggCBggGBggICAgEAggCCAgICgoKCg0ODAICAAgIDAYACAwODAYMDAgADAgGD/QAD/QECA wP//gGD/gIDAoADAwMDA/8D/AAD/AP//gKDAwKD/YGAA/wD/gAD/oACA4OCg4OCg/yDAAADAAMCg ICCgIP+AIACAICCAQCCAQICAYMCAYP+AgADAwAD/gED/oED/oGD/oHD/wMD//wD//4D//8BUJrxz AAASE0lEQVR4nO2di3abOhAA8YH//+Y2tgEJ9NiH8IZ45uQ2rMVKK2muk7ipNU0AAAAAAAAAAAAA AABfxePJ63N0LfCFvLV7ORhcC3wh2xMgAkIEb/OeH4/9QQAXWgkfLwnXB+w+fz7zVsWyQMW7H4cv wb+/5Ngh75X5u4vdvwQj4B/N/OXFvr9gZ1+3f3nJ4UPeK/NWxQ5Jh68HASEUBIQB2F8ZRkDwo305 L011juxLh3uzvjLCMyCEsL0mlwo498i7cFbgS4d7UxRQ2YWzAl863Jz1RWEEhBAQEGLZ/nLMqiAC QigICKEgIISCgBAKAkIoCAh+7D8EIyD48fxLSQQEO8V/pqHswlmBLx3uTWLeLuDSI+/CWYEvHe5N 8ssIfA8IAWz/UM3eg7MAX7qWuX+Lusf5op79FGv6SKHz+tEb/iWg561aEBABi4PIBPy2X0ZAwN8m YP1BIQiIgMVBELAEAiLg0HQtw1d/vpuAMwKOTNeCgAg4NF0LAiLg0HQtCIiAI9O3Sc7ZTwPpxfN6 ns+PHRq9yXOeU+6q0T6y4ObIrmR12dIu7KuBgAiIgAiIgEYQEAEREAER0EZa4U0EnL9TwLk1c9cO 6Ax6nROS/OYDAiLgBwXMD6lRp59AQATUGPR4rM+A+0OK9DMIiICqc0IKB9X8sHUq5FTkJmDlw96o S547pTTbZ3/BxsHnfmW9Rndlph34r47i91iL/wLqkQmFgL6Cv03Anw+NgNsXXwQsN3oLRsC+hKWz 4jahENBXMAIKBCycFbcJhYC+ghFQzaUCNhdW3ihJ7iWId3JEwb3OLxDwdMusnJN5NRAQAREQAREQ ARHwOwSUFjluP0cIKBD0zwlY+LhgBxAQAREQAREQAREQAREQAW0Czn9XwN5OlNv3bl0OGQYXNjY7 b84JAREQAREQAREQARFQLuBp7EqRc2cGNh/ajfYOBQJaCroseUTZg3cAAT0dIqB7BxDQ0yECuncA AT0dIqB7B8IEHDd34ZZVGl3r+XUCjt8BBDTtFAKO2gEENO0UAo7aAQQ07RQCjtqBjwj4uqFdoWTL xk+/LGCrSEl785ZZqoltcHujq3MEvI+AU69gBERA1za1b5kRMP1AQP02IODAHUBA/TYg4MAd+ISA r5HqReSfyu2OVdP37NymC2fjSzaXnX8aumAIOHY9+wW7ZuNLNpedfxq6YAg4dj37Bbtm40s2l51/ GrpgCDh2PfsFu2bjSzaXnX8aumCXCzhP7dok02vPvbfqnZ4LnTvWU1Bwq7HXuS/ZXHZzh3w7cH8B e1vSSy60dws276SkYHPP3WRz2ZV1GrEDCFho7xZs3klJweaeu8nmsivrNGIHELDQ3i3YvJOSgs09 d5PNZVfWacQOIGChvVuweSclBZt77iaby66s04gdGCDgs/sLBKwH46ZfL6B+t6DaXn6hRbLNhp6b K9yblnyO9h0Y8CblCCiJm/s1I6DQv9IxDQgoiZv7NSOg0D8E1G+OYL9mBBT5t31kZ8U5BHxTWque kOfkZntz5EYd5s0pFdTsrNnetLdXWdOcamOhRZiZLX9vB9RnxeVHtjqfAesPafdf68dhLTSlyQZQ 3a9KVk2t17lqB4SZggntsfqsuKFfglXTH7ffp5tVpRm2uVuAJlk1tV7nqh0QZgomtMcDzopDQPf9 qmTV1Hqdq3ZAmCmY0B7Hvgyjmv64/T7drCrNsM3dAjTJqqn1OlftgDBTMKE9Dnwhuvmodj9V0z81 antTbfM8de+vJM+T5GZ7Zc2Re9MetAMIOCFgceTetAftAAJOCFgcuTftQTuAgBMCFkfuTXvQDowQ cJlrAs42AetK7BTazfY2EsqNeXuroHmr2zCb/R7NyHljs7LmyOVpu7ot7MDvFFDZsmtU7bk5ckuw TvW9gpr5tmRJZa2eu5Vd11jYAQSsLJxkmwUFNfNtyZLKWj13K7uusbADCFhZOMk2Cwpq5tuSJZW1 eu5Wdl1jYQcQsLJwkm0WFNTMtyVLKmv13K3susbCDnxewNen2iJZp+dZG/M2d3a63WhOllT20aVy dYuACOhdKle3CIiA3qVydYuACOhdKle31wk4NwWsFWSd3vBVXal32G7vF4SAPx9hAupneM2qVt4x qT2UoL11S7Oxm9xbx/4iI6BphtesKgJqGsd3i4AIqGgc3+1nBOyu2nZro9GQOe8VaDbbJYiwYHuy uTLZUn24EQER8GrHEHBCQPlSfbgRARHwasciBJwTAd+7//rDsjYIOKayrxVwG6z24vP1Auoabyfg JKrMvlSS7UHAcTIgoCYTAREQARHQO6Y8ufywoLKvFPA5SGuzshkc722qOYkzy8n52vQSetXIxhwx G+2w3qUyCihcDQREQAREQAREQAT8QwLO7+4REAF/tYDHIidBi6Bdnfwq2iWgdkx7sst7+VIVWuSL LJ8QAiIgAiIgAiIgAv5NAev23ULA8jaotnkyC2hw4DiscakE0/68gK93J3+f1oCACPhhAbeDarIH ERABXTtgOCfkdEwDAiLgJwRcvwRnB9X8nBW3rNPdN3J+OYiACNiY8H93HupnwORT/gyoFjDb/e36 /Fgyg7T9cFFqnAs9H5JPbLOY8xmddF0nkeQch6g+dp5NbZTCw3NlKeZCz8alSldDVJN9BxDwAAIe VuPXCJj8EIKACPh5AddD4mo/hCAgAl4rYImXgKt0e4ENAcuzkghYWsx8WiMF7KmBgAiIgAg4ISAC nha50i0CIiACIuDrAgE/KeC0C7jdAzAhIASDgBAKAkIoHxBw++Zz3r1DQHiBgBAKAkIoCAihjBdw TgWcERCaXC3gVBYQ4AUCQigICKH4BZyXsoCbhfkHQAoCQigICKEgIITiF3B5Cfj8gWN7ERoBQcY4 AadEwCkX8AX2wRkEhFAQEEJBQAgFASEUBIRQEBBCQUAIZYyAy/qrz/vfgiAgSLhMwOkkIMAZBIRQ EBBCQUAIBQEhFP+blCMgOPAf04CA4MBwVhwCwjgsZ8U9srPilh/Wt0BIfhsaAaFDfvClLONp3/EZ 8EfA01/DrQICNBjyJRgBwcqQH0IQEKwMeRkGAcHKkBeiERCsICCEgoAQCgJCKAgIoVwpIH8LAl0Q EEJBQAgFASGUKwRcfy8LAaELAkIoCAihICCEMlTAGQFBySgBp/UNivbfw8c+6IOAEAoCQigICKEg IISCgBAKAkIoCAihDBLw5/ooIEAfBIRQEBBCQUAIZdBb9P5cH34RAUAAAkIoCAihICCEMlLAGQFB y1AB9/8AZCAghIKAEAoCQihXCDiiLvgSDAfVpO9SjoDgw3RMQ5aOgODAIGDptMyfawQEPd6Tkp5n xSEgWNCeFbfdnX8POE0ICFZUP4QUrhAQXOi+B5ymwllxEwKCGY2A++swaToCgoMBL0RPCAhmEBBC QUAIBQEhFASEUEYKOPGrWKAFASEUBIRQEBBCcQv4dG8X0F0QfBcICKEgIISCgBDKMAEXBAQDCAih ICCEgoAQCgJCKAgIoSAghIKAEMooARcEBAsDBQTQg4AQCgJCKAgIoSAghIKAEAoCQigICKEgIISC gBAKAkIoCAihICCEMkjA91EhAErUZ8Ud36QcAcGD6ay49JgGBAQPCAihWM6KS49rfSyPBwKCCdNZ cY/syFaeAcGH+qw4vgTDSNxnxSEgeHCfFYeA4IEXoiEUBIRQEBBCQUAIBQEhlDECThMCggkEhFAQ EEJBQAgFASEUBIRQEBBCQUAIBQEhFASEUBAQQkFACAUBIRQEhFAQEEJBQAgFASEUBIRQEBBCQUAI BQEhFASEUBAQQkFACAUBIRQEhFAQEEJBQAgFASEUBIRQEBBC0Z0TMu3vVb6mIyB40B1WOKVHNLzS ERA8KI5pSJ4BkwcREDwYvgRnB9W8zoqbEBD06M+KO17wDAhOEBBC8X4JRkBwMeasuAkBwYb3hejN OwQECwgIoSAghIKAEAoCQigICKEgIISCgBAKAkIoCAihICCEgoAQCgJCKIMFREPQgYAQCgJCKAgI oSAghIKAEAoCQigICKGMFXBBQNCBgBAKAkIoCAihICCEgoAQCgJCKAgIoSAghDJUwAUBQclYAfmr OFCCgBAKAkIoCAihICCEoj+mIX+TcgQEF6az4pJjGhAQXBjOikNAGIfloJr0uNblsZ0V9/MyNAKC HNNZcY/Ho/IM+HoOBNDgPaoLAcEFAkIo3rPiEBBcDHwhGgFBDwJCKAgIoSAghDJOwOXOAt617vsz UMDnn77uwrhr3fcHAZ/cte77g4BP7lr3/UHAJ3et+/4g4JO71n1/EPDJXeu+Pwj45K513x8EfHLX uu/PpQI2ovxNPMRpx3efOYijCtNO6wKmTcl1llG5xxZMtpVpLox5XezXh6A63jABC33fR8DGMyAC 2q4RsBumDQhYAgELEQJWbkTAU99N5W4iYNqUXV8mYMtGcaQRsP4NMAIWIgSURAhYGBYB9QECdkDA Y4CAxaA6/IUCLos8Et6oaeyEyaDLUcByU36dupgHlSHdgTk6qSoMK/NaKtf10lsTvFJAeSS9UdPY CVPL6s8JaVN+nT4xVp4lBwfm6PSkKgwrz/61NamX3pogAiIgAiKgJEDAAgiIgP3SrxTwEJtWSLOU CKiPBghYX4fK3MWLgoAIiIAIKAkQsAACImCxp18gYP7ybhYth0iYdsg7voCsDNddeV6Um6a0KbnO 6sjvyRr0wXGdTAt6XjXLMqV9VtahNfcpC+oTvFDAerRMh0iWdsg7PnMZw+VQd3pnOmByndVRuccW nNbJsqDNPNW6dNZBPvf6BBEQAREQActlImAXBERAyQzDBTyt8i0ETJuO19cI2LZRGGkEPGTeQMBH droIAiKgZFJDBcyiQysCImApGCpgdkzDoRUBEbAUDBUwO6jmcNrXPujxpcik0l5U7SSfVv4CsjVc DgJWmpLrrI40qDZIg8I6lZewtaDNNTyHlcnX5iW5LgTFUv8HurPiNgTfAy71aJk6UauTWqM5PPRa a1rKjdU7TUFhnfQL2swTh80V6PxZCIqlPj8h4CRqQsDfIuDhS/ChFQER8GIBJ9kPIQhoCBBQzV8W sLCAFwvYs1EUaQTsTR4BTxEC9iIEPA+LgIYAASUgYOVPBAwVcMlfwz1EUyU63bicu6w2NsKpGi55 r4embUql62U6pB+n4g5ki9ZdmOYyTeXWyryWynV9HtmIx5QrBUw/if8nbt2oaTyG9bvWRzpNpetl OqTXp+IOzFH5Uz8sTbKxJvXSiw1ZCwIKmhAwfxgBEVAWIeCxIAREwO8TsO4jApZDj4D9RUHA7RMC lkMEPEUIaIkQMB0XAa0BAlYpCZi860UanKLlFFVubPaiDPP35MiXpNJUvl6Oj5en0pikMDBH+UYo lqk8ydaa6BYla7lCwDRapmqUBHl0StM0dsJs1Oqdx6bSw0slOL5LS61FGpijfCPkyySgOnfRomQt CFi689hUehgB39cIqAkRsBsKQMB6o22hRwpYa2gFVwg4Yl0qLLVrBETASoSACIiAhRYElDSVHkbA wnW8gOnLPdlrP4VoqkTnNE2jOKzXnTfVMiqBScDqyvQmL49M61JBL2C1ZbCAADoQEEJBQAgFASEU BIRQEBBCQUAIBQEhFASEUBAQQkFACAUBIRQEhFDCBAzIvFWxLFA1o/Um5VcO7M68VbEsUC2heUzD hQP7M29VLAtUS0DAP5t5i2L/y/c4nBUHYEcv4GOynTAHMID8SzDAh0FAiMXydRsAAAAAAADgL6D/ mXh9/fudKe/gdV+eJsre7tQOWxxLkXmXMdcX19RDTuvfjGmHLKcYXmAxvCr4vv2dKe9grTFNE2Xv icphi2NpM28w5v7XW8ohk0zDNE8plpeYTQI+1izNoI/1iUxb8paoHnaADLapWm2wjvkwC/jONCxt IcUooPZpcx/tsX+SJh7SZNkHc1XDnlKUmeoxt50xTDPfU3Hmep9pZW3TLKYYZFrztJwGFiZNhXqF ApqGXb9PMWaaxqzYJ83Uj1kVXqqRfshyikWmpARl2v7/gFJAfXYuoDrRNaRlqqcUTaZ+zPcPBIYh 1x8l1EOWUywyGXLMg/oENK2vtWBfptUGx566nP90sYeerCnqH/eNP7jbf+KPyCynXJy5CmHbl48X CwAAAAAAAAAAAADg5B9z3OP97/LJugAAAABJRU5ErkJggg== --Apple-Mail-5--104826152 Content-Transfer-Encoding: base64 Content-Type: image/png; x-unix-mode=0644; name=uncoupled_increases.png Content-Disposition: inline; filename=uncoupled_increases.png iVBORw0KGgoAAAANSUhEUgAAAoAAAAHgCAMAAAACDyzWAAABKVBMVEX///8AAACgoKD/AAAAwAAA gP/AAP8A7u7AQADu7gAgIMD/wCAAgECggP+AQAD/gP8AwGAAwMAAYIDAYIAAgABA/4AwYICAYABA QEBAgAAAAICAYBCAYGCAYIAAAMAAAP8AYADjsMBAwIBgoMBgwABgwKCAAACAAIBgIIBgYGAgICAg QEAgQIBggCBggGBggICAgEAggCCAgICgoKCg0ODAICAAgIDAYACAwODAYMDAgADAgGD/QAD/QECA wP//gGD/gIDAoADAwMDA/8D/AAD/AP//gKDAwKD/YGAA/wD/gAD/oACA4OCg4OCg/yDAAADAAMCg ICCgIP+AIACAICCAQCCAQICAYMCAYP+AgADAwAD/gED/oED/oGD/oHD/wMD//wD//4D//8BUJrxz AAARmklEQVR4nO2di3ajOgwA4cD/f/Nuk0Bs/ECSRUTamT03VYgl22JCmrS7d5oAAAAAAAAAAAAA AAD+FvN/nrc/XwE+y9O+CfkgjoeAXAAhhO0leEqugjPAGEoH0y8/kV3nz2d+1WJpUHU0Av7mzHsv NnkTgoC/M/Pmi329YGev2zdfcviU35X5VYt1SYc/DwJCKAgIDtg/GkZAGEf7cV6aOjjzWDp8N9sn I1wBIYT9M7lUwOWMvMTgCsbS4bupCqgsMbiCsXT4crYPhREQQkBAiGX/4ZhVQQSEUBAQQkFACAUB IRQEhFAQEMaxvwlGQBgn/y15Ze7g1GPp8N1U/5qGssTgCsbS4btJzHsLuJ6RlxhcwVg6fDfJLyPw PSAEsP9FNXuFwQWMpcOX8xRw5N9qQUAYgV9GgO8GASEUBIRQEBBCQUAIBQEhFASEUBAQQkFACAUB IRQEhFAQEEJBQAgFASEUBIRQEBBCQUAIBQEhFASEUBAQQkFACAUBIZTvFnA5HyIfJy3mMbNqLo91 fKSQpQYCqot5zHwrb9wKIeDQOASMqIGA6mIeM9/KG7dCCDg0DgEjaiCgupjHzLfyxq0QAg6NQ8CI Ggi4j0HAiBoIuI9BwIgaCLiPQcCIGgqD9n+ONfm3CBHQNu5W3rgVulbA6j/Jj4C2cbfyxq3Q9S/B COg08628cSv0kZfgh4NzeuykxLLdLPV7rcOLJWiU9irvGjS6Y9z41GmDoE1uK5KfgR97FuU/Lf2y T3UFREAE7ATXvwQjIAK6CGh9E4KACOgioPVjGAREQB8BayAgAn6DgEt9Q419XnPSO01dYgX02Piz mFTAjzwliqA6GAEREAEREAGNICACIiACbsUQsAICImCUgO/Jls699iBJIGnIQPnrgtGN56ndNoi9 aR+WbMT0PEZABERABERAIwiIgAiIgCdtQEAEREAEREAEvEDAgZP+lwSUFXLqVecSUAQIiIAIiIAI aAQBERABEbDeT21FBBzaNQIi4JNDP0722Ty8yAOLBory1wTNE2ypqGmsfqBHv79OwGmRB4bta8pf E7QENFW8VECPfiMgAiIgAiIgAkZ7h4AIiIAIiIAI+EEBDytv7KM+yCu4nYCuG3fZq8eTtaNbfQwC IiACIiACIiACIiACIiACIuAXC7hkN6ctbuxzuycJ5C1aLOW9ApE3A6XP9ips01D+UknNguryQgXs HX79OQusVw1hea/Af+N56bO9ipsykG87JwiIgAiIgAiIgJcGCIiAY6cZAb9UwNqqRPt0PNeu5S9Q cmxlIgFlcgjybWcAAREQARGwsxYERMAPBwiIgKEBAiJgaICAVwm4ZDfv5dXvNQ9vexIEhz4Upd8j TOUHg/IsLZUTrFlZXij5EWynsZ3uiE6MKmjPUQafE3Dqb+h4WLFvxQhL+cGgfOARNq8w5t00xouX OppvCxDw6qB8AAGTAAGvDsoHEDAJEPDqoHwAAZMAAa8OygcQMAkQ8OqgfAABk+BCARtbEO5TsRWP ER8PBgRs5DQelZR5rahX3dbffIbaCAQMChDw+QUBgwIEfH5RCDj/53k771kIaA0Q8PlFLuDTvmnO MhDQGiDg84tBwOQCiIDmAAGfX3TfA76ugMlLcKZj8/cQsjn3NeUrKg6XQWu7y1lQKzYYlGtRLrz1 6wfKydSNrQZTfyGtxp/Osdeob29eZo2Au2ut7wGbHwFWv5w9ccvgtA+aYoNBOY9+4QMb7/XX3K/m QrSFyhrN7anehFQiBNwDBOzUaG5P9z3gNB1fghFwDxCwU6O5PY2A789h3gcRcA8QsFOjuT3XD6IR ULfwgY0joEzAxs4E+7RqUD1XFwg47u/QxkUCagU6X0i/zQiIgAiIgAhYqYGA9/AOARHwHsHoxhEQ AYeC0Y0jYEvAw9+TbqyqsSLNeViyjF6Qr8wxmDRjirNk3Xg+6/NPselG/U5QHK5O221zq0b1yCvw F/AgXe9pIT099fZP52MOAnoH5Vo6Y9w2ns+qa6xmamV+v/HNU4GACIiACIiACIiACIiACIiACPgr BJR3ZtiZOAFtq9u88Vuri0AIiIAIiIAIiIAIiIAIiIAIiIAIqBPw9dPxbbLlHaRraKxoOQSK83BM HSrmHpSLmuqHBwVsFTrvTuN/da7N726muT1XAZOLX02y3r0icNVg0gVlDdWYzhHBDs0CjhfyPwNT +8grQEAEREAEREAjCIiACKjoAwIioPg8SIJrBHQP/Dee1bhKQHMhBERARXcQEAER0AgCIiACKvqA gAjoeh4QEAFfvBrxnGwT8LAqBLxGQFX93y5gYy/tnTW66qrB1AiW9pHyoU66yRufjesaK57au/HN U4GACIiACIiARhAQARFQ0QcEREDJPhEQAWWIBWzvTOZMR4wBAQeDci3ihYvX2tndVugKATXdRUAE REAEREAElBbzChAQAREQAZ8goGrhCOgs4PL4PYT/f1YElCwcAS8QcMoEbO9FJMfvpmOavtBpY20r 0uVbQcAQEHADAUNAwA0EDAEBNxAwBATcQMAQEHDDW8BVLOBUvfd3BdwPqwudNta2IgT8xSDgBgKG gIAbCgHn/7y/HNIRUAUCbsgF/NHu/3+vL8d0BFSBgBsIGAICbui+B3wKmLwGz9sL8rKs/+37f4uA EhDwhzn7dk40/nlTvQImAj4OOC4Ufi+qNyHbLQKCF7rvAScEBF80As7ba3D1YxgEBAN+H0QjIBhA QAgFASEUBIRQEBBCQUAIBQEhFCcBf8zbBVwfRwYXBn8DLwEfvwaTCgggAQEhFASEUBAQQkFACAUB IRRPAVcEBC0ICKEgIISCgBAKAkIoCAihOAu4bgI+vj6C6X1gv/NgndIj2+F8XDY4TyhL7HfKI+rM tTrOVlSYqZ+h6Flj2Z3thXb7J/AVcPr5g4AIiIAIiIAIiID9zJ8AAeuZCIiA03EwAiLgkVEBQ1tS GZettpyhfsalk2dHpON0PbuxgI1xCFg9goAIiIAIeA4CKibPjiAgAiIgAt6tJQh4MvlNBVwePwR+ CbimAma9yRvVDiTjKmZXXZfOeZh8qIR9w51M3XKcF3lNt3/+cxJwWhcEREAEREAEREAEREAE/IZu TwiIgAh4q5Yg4BcL+NKvFDA7cHrGFefN3tXWnEMC2jciEdDpeePZ7ZMTK5oTAevNREAEtLQEAREQ AREQAREQAREQAQVz+gi4vgRcNwHXpoDliT48fBzXUeQ0UCaslSMnPZQUlWZOp5Or57x3t10FnNaX etutUcDpOO6TLZmKI79cwNBuIyACIuC9WoKACIiACIiACPgbBMwlLLbaOOPjLUmKWjPrq71YBrOA 0mXfrdsIiIAIeK+WIOCtBZznx80PCIiAHxdwfgmYHUNABPyQgHNyBURABPy4gPtLcHoVfOq4bn8t /VE2FXBN11n5Ueu7Af1xRVA+VO+qJHMtM/sllIF83H7Ec9m37fY6z6tewCx4XQG3mtuXRMC0N71n oHZc+VCjJYLMXiAd51BiX9uxb4PLvm23H3cQEAG/S8DDSzACIuCHBNw+f6m8CUFABLxewLqVCIiA CIiACJgKmOlX2+pJMHrGLSWyp42XgOpxlwh4224jIAKGdhsBETC02wiIgKHdRkAEDO02AiJgaLcv FHDNFMyc3O90gvTOVDxcHqmXEI1rC/j+nQD15P0T0xvXaUF1OfVAOi64294CTq84F/1wbisnuhrs d8prVOeqdQik485XYZj8uIrTzH3ceQvUy75ltxHQcCYR8Gz9ysYgIAJGdRsBDWcSAc/Wr2wMAiJg VLcR0HAmEfBs/crGfEjA7EC3E8Pdr7VEObl62d2iwwLa57y+20nPbItEQAQc6TYCCk4pAp5t2N5t BBScUgQ827C92wgoOKUIeLZhe7cRUHBKEfBsw/Zu31jA/zfpLyVUd10G5bi1Nq4sUSu630lW0suc ihL9TMlDSdHqw73MZP29GapzStd2GKfudn9755lOAr6PrJmEoqddPeiOKx+q55VPbFlmEkjGSXdk v3qqS1y7vdpgWyYCIqB3t1WZCIiA3t1WZSIgAnp3W5WJgAjo3W1VpoeAj2LnAlpbUvP4pMQxU/4Z wfGMj3wKZC9xPJps92oBHbqtWiQCNhLsXfU5MdnRZLsIiIAI6LhPBGwk2Lvqc2Kyo8l2ERABEdBx nwjYSLB31efEZEeT7SKgVsC1XN6aHi2DdNxUlClLTP1alafC6eRlifOpJDtSJKRzTmu2YenOg7qt aJWXgAlrTcLKaZjOgrT76cNliZOi+2oUk5clRFNJdmSZs95JSdGIbqtahYBaGRDQtVUIqJUBAV1b hYBaGRDQtVUIqJUBAV1bhYBaGRDQtVUfFtCz+4paohKpqK1Mi0YXCji24au6rVokAiIgAiIgAiIg AiIgAiIgAiLgLxSwYWMn2Mf1NyYqWj0f1SfHeeZam0FYVJR50gJR0N/w5d1WNcZdwJaEtZZMvWAf 9y6lvWolQXucPVNbVJTZaIGsZ/fotqoxCIiACIiACIiACIiACIiAvgJqz2R13FSOE7dk0k7eXk4n kJ5ktT3v9ZuXHdBtVWMQEAEREAERUGTb/Lyd30kIiIAfE/Ap3s/N20AERMBPCThPCIiAgQJOu4DJ a/DP6/GqFPCsmcdxNQfKEkIB7WeyF9xYwIBuKxqTfjsnFnA+XAEL/6oS1m1MguY2Wg40MzvBaKbn 5Io5652UFI3otqoxlisgAiIgAiLgXxNwnp8fwBQfwyAgAn5CwCoIiIAIiIAIWKIWsLmNQxn1mZxa KzFkWjQyCpjOqRNw0s7Zmcq+bGGAgAiIgAiIgEYQUBEgIAIiIAIiIAIiIAL+egGrZFO/j2RBOU6z H2nRtTgizmyXOClaOdrPLFvQy6wGrt2296ydiYAIiIBZUI5DQARsgYCKogiIgAiIgAMtKR66pCX6 My4VcFIUnbI7J4u8RMBLeoaAjpnaogiIgAh4t24jIAIiYBaU4xAQAVsgoKIoAl4i4IqACBgq4FgF +NsgIISCgBAKAkIoCAihICCEgoAQCgJCKAgIoSAghIKAEAoCQigICKEgIISCgBAKAkIoCAihICCE goAQCgJCKAgIoSAghIKAEAoCQigICKEMC6j5hxEAjowL6LMO+KMgIISCgBAKAkIoCAihICCEgoAQ CgJCKGoB5wfvdASEEQwC5ukICCPYroBJOgLCCKYrYPISPK/z4LeR8GfJL2aqxD3iCghDICCEMvoS jIAwhP4KyJsQcIQPoiEUBIRQEBBCQUAIBQEhFASEUBAQQkFACGVUQP5eOgwxLKDPMuCvgoAQCgJC KAgIoSAghIKAEAoCQigICKEgIISCgBAKAkIoCAihICCEgoAQCgJCKAgIoSAghIKAEAoCQigICKEg IISCgBAKAkIoCAihICCEMvxPcwCMgIAQCgJCKAgIoSAghIKAEAoCQigICKEgIISCgBAKAkIoCAih ICCEgoAQCgJCKAgIoSAghIKAEAoCQigICKGECRiQ+VWLpUHNjP+ETDyc+VWLpUGthPn538cnHs/8 qsXSoFYCAv7azK9Y7H/5ktfgGWAMvYDzpM8CcCJ/CQb4MAgIsVhetwEAAAAAAAB+A/r3xNvn369M eYHnuDxNlL2P1E5bnUuR+S1zbh+uqaectp+Maaespxg+YDF8Kvga/sqUF9jWmKaJst+Jymmrc2kz v2DO94+3lFMmmYZtFimWj5hNAs5blmbSebuQaZe8J6qndZDBtlWrDdY5Z7OAr0xDayspRgG1l833 bPP7izTxkCbLPpirmrZIUWaq59zPjGGb+TkVZ27jTJ21bbOaYpBpy9NSTCxMmirrFQpomnb7PsWY aZqzYZ80Uz9nU3ipRvop6ykWmZIlKNPezwGlgPrsXEB14tCUlq0WKZpM/ZyvNwSGKbe3Euop6ykW mQw55knHBDT117rgsUyrDQPndMj5Ty/2UMmaon67b3zjbn/HH5FZT7k4cxPCdl4+vlgAAAAAAAAA AAAAgEH+AWX1+frXlUGKAAAAAElFTkSuQmCC --Apple-Mail-5--104826152 Content-Transfer-Encoding: base64 Content-Type: image/png; x-unix-mode=0644; name=uncoupled.png Content-Disposition: inline; filename=uncoupled.png iVBORw0KGgoAAAANSUhEUgAAAoAAAAHgCAMAAAACDyzWAAABKVBMVEX///8AAACgoKD/AAAAwAAA gP/AAP8A7u7AQADu7gAgIMD/wCAAgECggP+AQAD/gP8AwGAAwMAAYIDAYIAAgABA/4AwYICAYABA QEBAgAAAAICAYBCAYGCAYIAAAMAAAP8AYADjsMBAwIBgoMBgwABgwKCAAACAAIBgIIBgYGAgICAg QEAgQIBggCBggGBggICAgEAggCCAgICgoKCg0ODAICAAgIDAYACAwODAYMDAgADAgGD/QAD/QECA wP//gGD/gIDAoADAwMDA/8D/AAD/AP//gKDAwKD/YGAA/wD/gAD/oACA4OCg4OCg/yDAAADAAMCg ICCgIP+AIACAICCAQCCAQICAYMCAYP+AgADAwAD/gED/oED/oGD/oHD/wMD//wD//4D//8BUJrxz AAAQS0lEQVR4nO2djZqiuhJF4YP3f+aZVlF+QpKqJOyga9073aLUppKso6OnT2cYAAAAAAAAAAAA AAAAforxwfO7uhf4QV7aPR0U9wI/yPsJEAFBwcu8x//Hz50ARVglHJ8SLnf4fb6+8lbNMkHBs8fd S3D/LWsvea/Kvpv9vAQj4JdWdt7s6wV787rdecvyS96r8lbNVimHnwcBQQoCQgX8nwwjIJRj/Thv XVp45bJyuDfLJyM8A4KE92dyawGnFNuIwg7KyuHeBAU0RhR2UFYON2f5UBgBQQICgpb3vxzzKoiA IAUBQQoCgpRWAk7L/1c3P0fLwfvm6sTnvYEHNjenIRWWSki3E2xufe8mwdVOsDlvwmZ2tgnmsPKE 1awcZmd1FgKaEgYENAxoVbbNXZ2FgKaEAQFDCX9vghEwa4oRsIGAf/b9fQzTuYBTXwKm20kJuE3w tLNO8Am4KtvOjkfAWA/BhPd/prEIuB0QAkYTELCWgMOwFnBOgYAIWFvA5e+APAMmptjUDgImE97/ oVpwQCIBp90i7W+GHx229zoSdmHJhGQ7qat8dTt5CQ8Bp89bYNOAELCrFe+sncyEP/em5YcREPDO K95ZOxcMCAG7WvHO2kFA/QQh4N0FnDwdxcocE+QMCydYpviCdlrOzgUDQkAERMBjRwiIgAiIgF8l 4HS4GR1efEzhRx0JyXasCZ21Uz6/ZQnTtuwkFwEREAERsEU7CIiACJicHQREwC8T8HxWzod3vDcr IRqWTEi2Y03orJ2wPtclTMGy/U2bgM99Qla/hgEB+23nCwXcblKDgH23830CjuPyDIiAN2jn6wRc NorbbFSz/7VIxzamUHORjqIJx7FlJfjaKU+oY3N2gnN2VAMaxmk0CLgyL/J3wBbDm0I3ETDW2T0E /LtpEfD9bIeAivVKJzhnRzUg898Bh8BLMAJetl7pBOfsqAbkEzCxV1yL4U2hmwgY6+xbBTwaeSJg 0Jl1R6nWTxLCU+xLSLZjSnCGhdfLmWBQo9kKJf952J2IgJUSEDDWDgJ62kHAau0goKcdBKzWzkUC TgkByyeoZkIyLHTu1Fc7t57fxxcEtCUsfzpp59bz+/iCgLaE5U8n7dx6fh9fENCWsPzppJ1bz+/j CwLaEpY/nbRz6/l9fKksYN7SpVs/WeaaCcl2ggnrsuIBmcJSCeWzc/GAnt8REAFFA3p+R0AEFA3o +R0BEVA0oOf3xgJOxcObwmHlCcUrXmFAlrC0MzXbMSSUzU5rAcvXq1VC+Yp/czuGhLJ2ELCTFe+s HUNCWTsI2MmKd9aOIaGsnRYCpoeYHIUjYRdmWfVoQlbJde1cNDsXDOj5HQFTCVkl17Vz0excMKDn dwRMJWSVXNfORbNzwYCe3xEwlZBVcl07F83OBQN6fkfAZEL4pqyda2bnggE9v2sEPN50lrUKo52r EhCQdqQJCEg70gQEpB1pQk0BN/Gv7+c3k30mE5Jh1yZ01k75/FZNOLsXAasldNZO+fxWTTi7FwGr JXTWTvn8Vk04uxcBqyV01k75/FZNOLu3CwGn8E3fbMsSOmvnxJliAZ0J6xPW9/YhoG+Zq4Z9XTur BGdY1YSzcxHwW9tZJTjDqiacnYuA39rOKsEZVjXh7FwE/NZ2VgnOsKoJZ+c2EnD4XGUwtBwuaxV2 i4RkWPiEHtrJSqj5S8qf8U3X69rF7yEBATf+RbdpeMY3Xa9rF7+HBATc+IeACFg3wbNX3HiyV9wz vul6Xbv4PSR8s4DjNI35Ar6e+rZbtrqfAZNTYZmrHhKq/mN07SgumNTzXPNecZVegvudK2cCAsYS agj4ehZEwHACAsYSKgpY6WOYfufKmYCAsYRaAh6N3ApY1L1uk79UWPLeuw2oo3YQMCMsee/dBtRR OwiYEZa8924D6qgdBMwIS957twF11E49Af/ivnCCsu6924A6auc6AcPdJ2+Wh1VNcI6ialg4QTa2 sgQEREAEzBxe+RTLFgkBz24iIAJ+nYDZbQyGe51h1ybQjjkBAb94xTtrBwF/bcU7awcBf23FO2sH AX9txTtr53IBB8NNZ1mrsOTE99BOZ7PjS0BAQwICIiACdhDWpYDrT6GrzdWtbg4d9NDvzbPZQcDm U8zN2OwgYPMp5mZsdhCw+RRzMzY7CNh8irkZm53qAj6iP1d7PXg8mj5H74dW925qXGFNEvKaS4a1 SRiOCZ6wJgmr5tbLvd/qyAwCImBWwqo5BERAw5QgoDcMAZ1hTRJWzfUi4FlHw7HGFdYkIa+5ZFib hOGY4Al7l00VE4ZAAgLaE/KaQ0AEbJSQ1xwCImCjhLzmEBABGyXkNYeAUgEB8kFAkIKAIAUBQQoC ghTTRjXL7ygP/ZZ8BAQPjo1qwuUICB4c+4SEt2lAQPDgeAnebFTz1nEn4Pz6Oj9uzfP2jvf9n8cN py+HQzjs7PTN4ydh2Rd73Jq3F9tc5PDoobdA9RAJy7/Y8fTIBB1n+6R6MyPG2f5cbPv4OM72jWqG IbhT0rR9+kNABAxd7HA6Ag6uiyGgZbarCLh6E4KAAwJaZruKgMsmcbE3IZ+TERABQxc7nF73g+jP A9uWU0bN56fPRQLuqueEgIHT16u6P9wuU0LAedtbsHqIhMUuFjjcTa9BwJPq+gLOCIiAn2wERMBP bwiYCQIiYHg+RQIu0cPmf/O8PTgczmeHu9rQYSo7Xt0y7IaNhqY/shpDeV8IeMW63qZRBERAaaMI iIDSRhEQAaWN3lfA14/CvN+inQ0pPMLTxzOGlMo2HVYNM61r7uG8D4tnZ0xvo9XIHBUCtgsrXZuY UQiIgP5sBPwcImC7sNK1iRmFgCEB7dO1ezx8WDJdxkNXdao2fnr80DdjLbMrr0ZFAQcEzGo0a8wI iIDZh6laBIwcIiACSlejroBnI0xNyBw9dEzXHH001WhmX8ZV7zdMuRoIiIAIeO2QEbCn1UBABPwC Af/0m2xD2jHsD7OdiNcaD6uG9dxorDa9GueLY+4LAREQATOGfB7V1breptFYLQIiYPOwWC0CImDz sFjtbQU0DWlO3OGvLTmsGqZsNE5KwJJa4yECtgtDwIxDBGwXhoAZhwjYLgwBMw6rCRi9TlWqZrds 9DspET9AnwJGB/ktArYcR8PswivtyxFQBgLOCIiA5mwERMB02M8L2M/bxdQEDacHt3q/fds373V+ STkCOhpBwNkkYGybBgR0NIKAs/klGAErNoKAs+cl+Llf4fq+EQFdjSDgf0bHM+CYfgas+JNFgZ/+ MYSlGxlOH00t831/FqvaalQIa/MSjID3CbuNgJY3IQh4n7DbCGj5GAYB7xN2HwGDUi4C7q+SrVDw cDg5rJEdb/RsslzZ9wtrvxr7MATcHs7Rwx6dQUAEPMu+XxgCIqA07KYCTtPh5/ER8JZh3yBgvOm8 w7Jq03QN0UczR2VaitQg86ozw2rMWNVsBIw3ajoZAcuXFgGjj2aOCgGzDxEw3qjpZAQsX9rGAg6W Q+PpuWFZE1KyFM5Gqg6y5Yw1bRQBc05GwGaNImDOyQjYrFEEzDkZAZs1WknAz0p1P53xS4VPRsBm jdYVcNheZ7B1Ga82ZefXDpX7unCQzcJqj3o335tDBERABETA4uwfF/DYcrLp6MnHxyseRh9P2moa pDIsnm151BoWPURABLQ+ioAI6AiLZ1seRUAEdITFsy2PIiACOsLi2ZZH7ycgQC4ICFIQEKQgIEip J+CAgGAHAUEKAoIUBAQpCAhSqgoIYAUBQQoCghQEBCkICFIcG9Wsf0s5AkIZrm0a9uUICF4cAh63 aUBA8FK6U9JDRwQED9sns6zzlxvvux5fJ/QDJ6Y3IYFbj68ICF5sfwcchuBecQgIXiwCfj6H2ZUj IHip8UH0NCAgOEFAkIKAIAUBQQoCgpRqAgJ4QECQgoAgBQFBShUBeQsCXhAQpCAgSEFAkIKAIAUB QQoCghQEBCkICFIQEKQgIEhBQJBSQ0B+HBXcICBIQUCQgoAgBQFBCgKCFAQEKQgIUhAQpCAgSKkk IIAPBAQpCAhSEBCkICBIMe8VF/gl5QgIblx7xe22aUBAcIOAIMWzV9y43StuRkBw4dorbtxs2fp3 AwHBj3mvuONLMAKCnwp7xSEg+KmwVxwCgp8KH0QjIPhBQJCCgCAFAUEKAoKUKgJW6QR+EgQEKQgI UhAQpCAgSEFAkIKAIAUBQQoCghQEBCkICFIQEKQgIEhBQJCCgCAFAUEKAoKUGgJWaQR+EwQEKQgI UhAQpCAgSEFAkIKAIAUBQUq5gHwODQVUELBOI/CbICBIQUCQgoAgBQFBim2fkOHzu8rf5QgIBdg2 KxzWWzS8yhEQCjBs07B6BlyXIyAU4HgJ3mxUM45z4V8j4Wex7xW3v8EzIBSCgCCl9CUYAaGI8r3i EBAK4INokIKAIAUBQQoCghQEBCkICFIQEKQgIEhBQJCCgCAFAUEKAoIUBAQpCAhSEBCkICBIQUCQ goAgBQFBCgKCFAQEKQgIUhAQpCAgSEFAkIKAIKVYQPyDEhAQpCAgSEFAkIKAIAUBQQoCghQEBCkI CFIQEKTYt2nY/ZJyBIQSXHvFrbdpQEAowbFXHAJCPTwb1ay3ax1n025fAB9ce8WN48gzIFSjeKsu BIQSEBCkFO8Vh4BQAh9EgxQEBCkICFIQEKQgIEhBQJCCgCAFAUEKAoIUBAQpCAhSEBCkICBIQUCQ goAgBQFBCgKCFAQEKQgIUhAQpCAgSCkVcEZAKKFYwDptwK+CgCAFAUEKAoIUBAQpCAhSEBCkICBI QUCQgoAgBQFBCgKCFAQEKQgIUswCjpvdRRAQynAIuDlCQCjC9wz4OUJAKML1DLh+Cbbt9gXwwbpX 3KrwfYtnQCgCAUFK8Utw3Xbg17A/A/ImBCrCB9EgBQFBCgKCFAQEKQgIUhAQpCAgSEFAkIKAIAUB QQoCghQEBCkICFIQEKQU/5Z8gBIQEKQgIEhBQJCCgCAFAUEKAoIUBAQpCAhSEBCkICBIQUCQgoAg BQFBCgKCFAQEKQgIUhAQpCAgSEFAkIKAIEUmoKDyVs0yQacVm19SfuGFiytv1SwTdFaw3abhuguX V96qWSborAABv7byFs3+l2+9XStAGXYBx4H9CUHG9iUY4GIQELR4XrcBAAAAAAAAvgH7e+Ll8+9X ZX7A87xtWVb1+0zrZYPXMlTe5ZrLh2vmSw7LvxmzXjJc4viAxfGp4Ov0V2V+wNLjuiyr+lNovGzw WtbKG1zz86+3jJdcVTqGeSjxfMTsEnBcqiwXHZcnMmvL70LzZSvI4Buq1wbvNUe3gK9Kx9QGSpwC Wp82P1cbP99yC3dledU7c02XPZQYK83XfK+MY5jbNc2uXM5zzaxvmMESh0xLnZXDhTOLhkC/mQK6 Lrv8PcVZ6brmiX25lfZrngqfq5H9kuESj0yrFoxln38GjALaq7cCmguLLukZ6qHEUmm/5usNgeOS y1sJ8yXDJR6ZHDXui5YJ6Jpfb8NllV4bCta0yPmrm90leUvMb/edb9z97/gVleGSxpWLEL51ubxZ AAAAAAAAAAAAAIBC/gGVhXGyH2sHlwAAAABJRU5ErkJggg== --Apple-Mail-5--104826152 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; format=flowed --Apple-Mail-5--104826152-- From john@jlc.net Thu May 14 16:16:17 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id F0C8F3A6B7F for ; Thu, 14 May 2009 16:16:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.902 X-Spam-Level: X-Spam-Status: No, score=-5.902 tagged_above=-999 required=5 tests=[AWL=0.697, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Qb8iJaHro6FK for ; Thu, 14 May 2009 16:16:15 -0700 (PDT) Received: from mailhost.jlc.net (mailhost.jlc.net [199.201.159.9]) by core3.amsl.com (Postfix) with ESMTP id AE0E428C110 for ; Thu, 14 May 2009 16:16:11 -0700 (PDT) Received: by mailhost.jlc.net (Postfix, from userid 104) id 2CDC633C64; Thu, 14 May 2009 19:17:45 -0400 (EDT) Date: Thu, 14 May 2009 19:17:45 -0400 From: John Leslie To: Janardhan Iyengar Message-ID: <20090514231745.GB70521@verdi> References: <001e01c9cfff$bde8f1e0$103947ab@cisco.com> <97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com> <001f01c9d005$0c583110$103947ab@cisco.com> <1241818843.26211.67.camel@nhuynhth-laptop> <0D6E7BC4-17AF-4F5C-9107-ABA2B729683F@muada.com> <20090509184241.GT32848@verdi> <4A0C8E89.9080000@fandm.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4A0C8E89.9080000@fandm.edu> User-Agent: Mutt/1.4.1i Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] TCP/SCTP and BoF description, was: Re: TCP modification vs switching to SCTP X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 May 2009 23:16:17 -0000 Janardhan Iyengar wrote: > [John Leslie wrote:] >> [Iljitsch van Beijnum wrote:] > >>> 2. Multipath mechanisms. TCP doesn't have these, and we now have two >>> drafts proposing them. There are papers and even implementations >>> skinning this cat in different ways. We currently have a single >>> address and a multiaddress proposal. Would of course be nice to be >>> able to do both, as each has limitations that the other doesn't have. >>> SCTP does multiaddress multipath and it could be extended with the >>> single address multipath similar to TCP but the details will be >>> different (option space etc) (2a: TCP, 2b: SCTP) >> >> We shouldn't gloss over these differences. I believe a TCP-only >> implementation can be done more quickly, and be deployed much more >> quickly for a large-scale experiment (which I believe necessary to >> establish _practical_ congestion-control principles). > > That assumes that there will be a single mechanism that will apply > equally to all transports, and we are not there yet. I think we're in violent agreement there. > My design for CMT is different from the one proposed in the draft for > MPTCP, and it will be worthwhile discussing the differences (CMT uses > a single sequence number space, MPTCP and SST use two sequence spaces; > there are arguments for both sides.) Exactly the sort of thing we need to discuss under the first Charter item... " " - A set of architectural guidelines for congestion dependent " multipath transport protocols. Since different transport protocols " can potentially benefit from this approach, this document describes " the motivations and the general approach that should be followed " to enable congestion dependent multipath transport. > TCP considerations will be different than ones for SCTP (option > space limitations, for instance), and I expect they will be *very* > different for DCCP (different ack mechanisms). I tend to agree. > Again, I think that this BoF should be broadly scoped to allow for > multiple parallel/competing architectures. We should be cautious > not to scope the group too narrowly yet. Is that first Charter item too narrow? > I sincerely think that there are shared questions here that cut across > transports, and there will be significant differences. Architecturally, > this conversation is exactly where we should discuss where multipath > belongs on the stack. Why not have a common flow layer for *all* > transports that does multipath? I don't follow how we'd get there. Implementations already pretty thoroughly obscure "layer" boundaries. While I am very sympathetic to the idea of Transport layers that know how to do multipath, I am not convinced that "one true way" is even a desirable goal. > We don't have to agree to split up the transport (as Bryan and I have > proposed), but it will help us see the questions here from a > perspective that is beyond TCP. That would be good, during the architecture discussion. > And we can then map the answers back to TCP. _Iff_ we (ever) agree on the "one true way"... (Seriously, I don't think we're any more likely to converge on "one true way" that other groups are to converge on a single "congestion control" algorithm.) > Incidentally, this is how I remember the "Resource-Pooling" paper > as being scoped. We're talking "engineering" here, not academic papers. We tend to avoid areas where we need to debate what would be "best." I'm very happy to see interest by academic types in this subject. But unless there's a place for engineers to dive in and code something, I fear the project would drag on forever. :^( -- John Leslie From baford@mpi-sws.org Thu May 14 23:21:45 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id A10DC3A67DB for ; Thu, 14 May 2009 23:21:45 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.249 X-Spam-Level: X-Spam-Status: No, score=-6.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KlqFn-7qrIG6 for ; Thu, 14 May 2009 23:21:44 -0700 (PDT) Received: from hera.mpi-sb.mpg.de (infao0809.mpi-sb.mpg.de [139.19.1.49]) by core3.amsl.com (Postfix) with ESMTP id 5FA083A6CB4 for ; Thu, 14 May 2009 23:21:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mpi-sb.mpg.de; s=mail200803; h=References:Message-Id:From:To: In-Reply-To:Content-Type:Content-Transfer-Encoding:Mime-Version: Subject:Date:Cc; bh=oVOy5LMwgUYqcqfto22oIfGbY0+ofolqYFO6r/iWKx4=; b=nJQQqdgHV47OZF7Q4OXwVP+6sxyN834xntMZVPiiIDmCa8gd2+DMohTvBIAmFu ACulLjfZvXd3YXJQ8QhZQYxujhJrXk+zExxdLcNI23R4OkjWEjb7081So5H46agV z26NfapgDbFJ32S4OMVaQWCOJDmhwZH4htJoFoxK5spqY= Received: from newmaniac.mpi-sb.mpg.de ([139.19.1.26]:39419) by hera.mpi-sb.mpg.de (envelope-from ) with esmtp (Exim 4.69) id 1M4qow-0000Jw-Ku; Fri, 15 May 2009 08:23:05 +0200 Received: from adsl-84-227-28-216.adslplus.ch ([84.227.28.216]:62236 helo=[192.168.1.103]) by newmaniac.mpi-sb.mpg.de with esmtpsa (TLS-1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.63) (envelope-from ) id 1M4qov-0004Ex-W7; Fri, 15 May 2009 08:23:02 +0200 References: <4A0CA1D0.4040802@fe.up.pt> Message-Id: <61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> From: Bryan Ford To: "j.araujo@ucl.ac.uk" In-Reply-To: <4A0CA1D0.4040802@fe.up.pt> Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes Content-Transfer-Encoding: quoted-printable X-Mailer: iPod Mail (5H11) Mime-Version: 1.0 (iPod Mail 5H11) Date: Fri, 15 May 2009 08:22:38 +0200 Cc: "multipathtcp@ietf.org" Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 May 2009 06:21:45 -0000 On May 15, 2009, at 12:57 AM, Jo=C3=A3o Taveira Ara=C3=BAjo = =20 wrote: > Iljitsch van Beijnum wrote: >> So if we have a shared architecture, does it make sense to =20 >> implement the multipath congestion control up to five separate times? >> >> Wouldn't it make more sense to define an abstraction layer between =20= >> the transport protocol and the congestion control algorithm? > This makes sense and reminds me of Bryan's case for decoupling CC =20 > and transport semantics in "Breaking up the transport logjam". But =20 > are we drifting away from the original motivation for extending TCP =20= > for multipath, namely for deployment purposes. Is the purpose of the =20= > eventual WG to work on multipath as a general concept and attempt to =20= > retrofit these results to transport protocols, or to take into =20 > account the limitations TCP and retain backwards compatibility =20 > whilst enabling multipath? If the latter is the case, then I think that the scope of this WG =20 should include ONLY single-ended TCP, because that's the only approach =20= we know of so far that preserves any kind of backward compatibility. =20= A two-ended MPTCP will encounter all the same deployment challenges =20 that, say, CMT SCTP does, including the huge practical problem of =20 legacy NATs in the path. Bryan > > > Cheers, > Joao. > > > > _______________________________________________ > multipathtcp mailing list > multipathtcp@ietf.org > https://www.ietf.org/mailman/listinfo/multipathtcp From Pasi.Sarolahti@nokia.com Fri May 15 01:06:58 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id ACC213A6E1D for ; Fri, 15 May 2009 01:06:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MrFYy2HUu+w7 for ; Fri, 15 May 2009 01:06:58 -0700 (PDT) Received: from mgw-mx03.nokia.com (smtp.nokia.com [192.100.122.230]) by core3.amsl.com (Postfix) with ESMTP id 9C1DD3A6CA0 for ; Fri, 15 May 2009 01:06:57 -0700 (PDT) Received: from esebh105.NOE.Nokia.com (esebh105.ntc.nokia.com [172.21.138.211]) by mgw-mx03.nokia.com (Switch-3.2.6/Switch-3.2.6) with ESMTP id n4F88O5D026677; Fri, 15 May 2009 11:08:25 +0300 Received: from esebh102.NOE.Nokia.com ([172.21.138.183]) by esebh105.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.3959); Fri, 15 May 2009 11:07:53 +0300 Received: from vaebh101.NOE.Nokia.com ([10.160.244.22]) by esebh102.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.3959); Fri, 15 May 2009 11:07:53 +0300 Received: from [172.21.30.100] ([172.21.30.100]) by vaebh101.NOE.Nokia.com with Microsoft SMTPSVC(6.0.3790.3959); Fri, 15 May 2009 11:07:52 +0300 In-Reply-To: <61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> References: <4A0CA1D0.4040802@fe.up.pt> <61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> Mime-Version: 1.0 (Apple Message framework v753.1) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Pasi Sarolahti Date: Fri, 15 May 2009 11:07:51 +0300 To: ext Bryan Ford X-Mailer: Apple Mail (2.753.1) X-OriginalArrivalTime: 15 May 2009 08:07:52.0795 (UTC) FILETIME=[3EF3CAB0:01C9D534] X-Nokia-AV: Clean Cc: "multipathtcp@ietf.org" , "j.araujo@ucl.ac.uk" Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 May 2009 08:06:58 -0000 On May 15, 2009, at 9:22, ext Bryan Ford wrote: > If the latter is the case, then I think that the scope of this WG > should include ONLY single-ended TCP, because that's the only approach > we know of so far that preserves any kind of backward compatibility. > A two-ended MPTCP will encounter all the same deployment challenges > that, say, CMT SCTP does, including the huge practical problem of > legacy NATs in the path. I believe two-ended version can be made to work over many NAT configurations, if you don't do the dynamic address management by explicitly passing the addresses in TCP options. I think this implicit address management was one of the variants discussed in the two-ended draft. This is important, because the ability to go through NATs is one of the most important benefits of doing this in TCP vs. other protocols in my opinion. - Pasi From Rolf.Winter@nw.neclab.eu Fri May 15 04:24:41 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 408B43A6916 for ; Fri, 15 May 2009 04:24:41 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.772 X-Spam-Level: X-Spam-Status: No, score=-1.772 tagged_above=-999 required=5 tests=[AWL=0.827, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2f-JNHDBU+Tt for ; Fri, 15 May 2009 04:24:40 -0700 (PDT) Received: from smtp0.neclab.eu (smtp0.neclab.eu [195.37.70.41]) by core3.amsl.com (Postfix) with ESMTP id 0436E3A6781 for ; Fri, 15 May 2009 04:24:40 -0700 (PDT) Received: from localhost (localhost.localdomain [127.0.0.1]) by smtp0.neclab.eu (Postfix) with ESMTP id 0FC3D2C0203D7; Fri, 15 May 2009 13:26:13 +0200 (CEST) X-Virus-Scanned: Amavisd on Debian GNU/Linux (atlas2.office) Received: from smtp0.neclab.eu ([127.0.0.1]) by localhost (atlas2.office [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lFxQRdsfs0+Z; Fri, 15 May 2009 13:26:12 +0200 (CEST) Received: from VENUS.office (mx1.office [192.168.24.3]) by smtp0.neclab.eu (Postfix) with ESMTP id DD1922C00035C; Fri, 15 May 2009 13:25:57 +0200 (CEST) Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-MimeOLE: Produced By Microsoft Exchange V6.5 Date: Fri, 15 May 2009 13:25:55 +0200 Message-ID: <547F018265F92642B577B986577D671C8FF016@VENUS.office> In-Reply-To: <4A0C8E89.9080000@fandm.edu> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [multipathtcp] TCP/SCTP and BoF description, was: Re: TCP modification vs switching to SCTP Thread-Index: AcnU2+OnRoc+/VkoRO+IbKGE00fn5wAczS4w References: <1241789497.4055.8.camel@nhuynhth-laptop> <001e01c9cfff$bde8f1e0$103947ab@cisco.com> <97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com> <001f01c9d005$0c583110$103947ab@cisco.com> <1241818843.26211.67.camel@nhuynhth-laptop> <0D6E7BC4-17AF-4F5C-9107-ABA2B729683F@muada.com><20090509184241.GT32848@verdi> <4A0C8E89.9080000@fandm.edu> From: "Rolf Winter" To: , "John Leslie" Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] TCP/SCTP and BoF description, was: Re: TCP modification vs switching to SCTP X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 May 2009 11:24:41 -0000 When I read parallel and competing architectures I read IRTF not IETF. = Thinking about it, when I read competing architectures I am not sure = this still qualifies for being a BoF.=20 > Again, I think that this BoF should be broadly scoped to allow for > multiple parallel/competing architectures. We should be cautious not > to scope the group too narrowly yet. I sincerely think that there are > shared questions here that cut across transports, and there will be > significant differences. Architecturally, this conversation is = exactly > where we should discuss where multipath belongs on the stack. Why not > have a common flow layer for *all* transports that does multipath? We > don't have to agree to split up the transport (as Bryan and I have > proposed), but it will help us see the questions here from a > perspective that is beyond TCP. And we can then map the answers back > to TCP. Incidentally, this is how I remember the "Resource-Pooling" > paper as being scoped. >=20 > - jana >=20 > -- > Janardhan Iyengar > Assistant Professor, Computer Science > Franklin & Marshall College > http://www.fandm.edu/jiyengar > _______________________________________________ > multipathtcp mailing list > multipathtcp@ietf.org > https://www.ietf.org/mailman/listinfo/multipathtcp From john@jlc.net Fri May 15 05:09:53 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 6DC643A70C9 for ; Fri, 15 May 2009 05:09:53 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.924 X-Spam-Level: X-Spam-Status: No, score=-5.924 tagged_above=-999 required=5 tests=[AWL=0.675, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Sxo1ENAzOJV5 for ; Fri, 15 May 2009 05:09:52 -0700 (PDT) Received: from mailhost.jlc.net (mailhost.jlc.net [199.201.159.9]) by core3.amsl.com (Postfix) with ESMTP id 72E363A70C8 for ; Fri, 15 May 2009 05:09:52 -0700 (PDT) Received: by mailhost.jlc.net (Postfix, from userid 104) id 96DD833C6E; Fri, 15 May 2009 08:11:25 -0400 (EDT) Date: Fri, 15 May 2009 08:11:25 -0400 From: John Leslie To: Pasi Sarolahti Message-ID: <20090515121125.GD70521@verdi> References: <4A0CA1D0.4040802@fe.up.pt> <61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.1i Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 May 2009 12:09:53 -0000 Pasi Sarolahti wrote: > On May 15, 2009, at 9:22, ext Bryan Ford wrote: >> On May 15, 2009, at 12:57 AM, Jo??o Taveira Ara??jo wrote: >> >>> Is the purpose of the eventual WG to work on multipath as a general >>> concept and attempt to retrofit these results to transport protocols, >>> or to take into account the limitations TCP and retain backwards >>> compatibility whilst enabling multipath? The latter seems more appropriate. (The former is too open-ended for an IETF WG, though quite appropriate for an IRTF Research Group.) >> If the latter is the case, then I think that the scope of this WG >> should include ONLY single-ended TCP, because that's the only approach >> we know of so far that preserves any kind of backward compatibility. I can't agree that we don't "know of" two-ended multipath approaches that can preserve backward compatibility. >> A two-ended MPTCP will encounter all the same deployment challenges >> that, say, CMT SCTP does, including the huge practical problem of >> legacy NATs in the path. It's exactly things like NATs in the path that make the deployment challenges so different! > I believe two-ended version can be made to work over many NAT > configurations, if you don't do the dynamic address management by > explicitly passing the addresses in TCP options. That is one promising approach. We will no doubt consider others. Legacy NATs will almost certainly ignore stuff we might pass in TCP options. > I think this implicit address management was one of the variants > discussed in the two-ended draft. This is important, because the > ability to go through NATs is one of the most important benefits of > doing this in TCP vs. other protocols in my opinion. The one-ended version, happily, goes through NATs at the unmodified end quite nicely. Alton Ford's draft-ford-mptcp-multiaddressed proposes an "Implicit Path Management" mode where "subflows" look to NATs just like another TCP connection, but are "joined" to an existing flow and the group of subflows appears as a single TCP connection to the application layer. Thus Pasi and I have a (IMHO) reasonable belief that a mptcp WG could design an extension to TCP that manages both one-ended and two-ended multipath TCP with full NAT traversal. But if others think that belief unreasonable, perhaps we _should_ limit ourselves to the one-ended case... IMHO, discussing the architectural document in an open-ended way is the right start. Committing to follow through on multipath for TCP (only) feels safe enough to me. Committing to multipath for other transports doesn't feel safe. YMMV, of course... -- John Leslie From baford@mpi-sws.org Fri May 15 05:10:05 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 467FE3A70D0 for ; Fri, 15 May 2009 05:10:05 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.249 X-Spam-Level: X-Spam-Status: No, score=-6.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jA7XnvAyOB1c for ; Fri, 15 May 2009 05:10:03 -0700 (PDT) Received: from hera.mpi-sb.mpg.de (hera.mpi-sb.mpg.de [139.19.1.49]) by core3.amsl.com (Postfix) with ESMTP id 993853A70C8 for ; Fri, 15 May 2009 05:10:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mpi-sb.mpg.de; s=mail200803; h=Cc:Message-Id:From:To: In-Reply-To:Content-Type:Content-Transfer-Encoding:Mime-Version: Subject:Date:References; bh=8+dYT5sKvBmboisY8oSVOwI/ubiIKOfFdN5D HiBc24E=; b=UCo8tYC3XP7OBs/o373MKSRqhl/4S4bCFWckEfPpX0VY0Kr9oYNb eL35NQpvClsMfgRuDADndCK2lrTwecGFfPcukffDwTlZ/eE2lgOtvcB3jAjS/lC1 rnId9cannAXvNgIUZZrxfGbj7TEyr4LqBTPszlXIEFOv8Qgih8t/ujw= Received: from infao0525.mpi-sb.mpg.de ([139.19.1.26]:35820 helo=newmaniac.mpi-sb.mpg.de) by hera.mpi-sb.mpg.de (envelope-from ) with esmtp (Exim 4.69) id 1M4wGC-0006gs-OR; Fri, 15 May 2009 14:11:35 +0200 Received: from adsl-84-227-28-216.adslplus.ch ([84.227.28.216]:61071 helo=[192.168.1.100]) by newmaniac.mpi-sb.mpg.de with esmtpsa (TLS-1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.63) (envelope-from ) id 1M4wGC-000612-6t; Fri, 15 May 2009 14:11:32 +0200 Message-Id: From: Bryan Ford To: Pasi Sarolahti In-Reply-To: Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Fri, 15 May 2009 14:11:31 +0200 References: <4A0CA1D0.4040802@fe.up.pt> <61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> X-Mailer: Apple Mail (2.930.3) Cc: "multipathtcp@ietf.org" , "j.araujo@ucl.ac.uk" Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 May 2009 12:10:05 -0000 On May 15, 2009, at 10:07 AM, Pasi Sarolahti wrote: > On May 15, 2009, at 9:22, ext Bryan Ford wrote: > >> If the latter is the case, then I think that the scope of this WG >> should include ONLY single-ended TCP, because that's the only >> approach >> we know of so far that preserves any kind of backward compatibility. >> A two-ended MPTCP will encounter all the same deployment challenges >> that, say, CMT SCTP does, including the huge practical problem of >> legacy NATs in the path. > > I believe two-ended version can be made to work over many NAT > configurations, if you don't do the dynamic address management by > explicitly passing the addresses in TCP options. I think this > implicit address management was one of the variants discussed in the > two-ended draft. This is important, because the ability to go > through NATs is one of the most important benefits of doing this in > TCP vs. other protocols in my opinion. How do you deal with TCP sequence numbers then? Or by "many NAT configurations" did you mean only those that don't track TCP sequence numbers and expect them to work as they normally do in TCP? How many NAT configurations is that, I wonder, especially given the increasing ubiquity of DPI technology that expects to see full TCP streams? (For that matter, you don't even need NAT to have this problem: I bet many non-NAT firewalls will do something bad with TCP streams that appear to the firewall to be missing sequence numbers because they got sent on another path.) Bryan > > - Pasi > From c.raiciu@cs.ucl.ac.uk Fri May 15 05:39:40 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 48FA53A6D9F for ; Fri, 15 May 2009 05:39:40 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.499 X-Spam-Level: X-Spam-Status: No, score=-2.499 tagged_above=-999 required=5 tests=[AWL=0.100, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CMluzuuPCdPp for ; Fri, 15 May 2009 05:39:39 -0700 (PDT) Received: from bells2.cs.ucl.ac.uk (bells2.cs.ucl.ac.uk [128.16.5.33]) by core3.amsl.com (Postfix) with ESMTP id 61B303A6A80 for ; Fri, 15 May 2009 05:39:39 -0700 (PDT) Received: from blackie2.cs.ucl.ac.uk ([128.16.68.58]) by bells2.cs.ucl.ac.uk with esmtpsa (TLSv1:AES128-SHA:128) (C.Raiciu authenticated) (Exim 4.54) id 1M4xDt-0009Cc-3E for multipathtcp@ietf.org; Fri, 15 May 2009 14:13:13 +0100 Mime-Version: 1.0 (Apple Message framework v753.1) In-Reply-To: References: <4A0CA1D0.4040802@fe.up.pt> <61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <5C4FB334-B163-49C4-BC94-775247962B34@cs.ucl.ac.uk> Content-Transfer-Encoding: 7bit From: Costin Raiciu Date: Fri, 15 May 2009 13:41:09 +0100 To: multipathtcp@ietf.org X-Mailer: Apple Mail (2.753.1) Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 May 2009 12:39:40 -0000 I also believe we can make the two-ended version work with NATs. If the one ended version could work, so will the two ended one. Details below. There are three issues: 1. Will new option types be allowed through nats and other middleboxes? If one new option type goes through, probably all will go through. I would personally bet new options do go through, but don't have numbers to back this up. The options two ended multipath really needs are "mpath capable", "token", and "global sequence number". If it gets them through, it is superior to single-ended (see below at point 3). If new options almost never make it through, two ended multipath must pretend it is single ended, even if both ends do this: it can't declare it supports mpath, there is no token and no global sequence number; thus the two ended version is forced to stripe the global sequence space across the subflows it has, behaving like the single ended one. 2. address management Here one can simply open subflows and see if they succeed (this is implicit management). If they don't it's maybe because the endpoint is behind a nat, so if we want to use the extra address it must be signaled otherwise to the other end. To get this working now, we must use options. For ipv4 addresses, there is still room in the options space to add at least 1 per packet. For ipv6, we can just use shim6 to discover alternate paths; since we are talking about deploying now, let's leave ipv6 out of this. Thus, if we can use new option types, we can do address management too for two ended multipath. 3. sequence numbers The two-ended multipath has subflows which look over the wire as normal tcp connections, i.e. with contiguous sequence spaces, so they will pass through boxes like normalizers. The single ended one or CMT wont. Its unclear how many normalizers are deployed; Mark had understood from discussions with guys from the industry that many people use them (how many, don't know). Finally, the one-ended one, and cmt for that matter, have the following issue: when a packet is lost and retransmitted on a different path, there is no way to know which of the paths delivered the packet unless one adds "options" to the packets. This is a more serious problem than not knowing whether the original or retransmitted packet got there in traditional tcp, as it places the host in the impossibility of deciding whether the path that carried the retransmitted packet has lost or delivered the packet. Cheers, Costin From c.raiciu@cs.ucl.ac.uk Fri May 15 05:56:46 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 69D8F3A69CF for ; Fri, 15 May 2009 05:56:46 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.524 X-Spam-Level: X-Spam-Status: No, score=-2.524 tagged_above=-999 required=5 tests=[AWL=0.075, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kNREybLxuLmv for ; Fri, 15 May 2009 05:56:45 -0700 (PDT) Received: from bells2.cs.ucl.ac.uk (bells2.cs.ucl.ac.uk [128.16.5.33]) by core3.amsl.com (Postfix) with ESMTP id 965A93A6D74 for ; Fri, 15 May 2009 05:56:35 -0700 (PDT) Received: from blackie2.cs.ucl.ac.uk ([128.16.68.58]) by bells2.cs.ucl.ac.uk with esmtpsa (TLSv1:AES128-SHA:128) (C.Raiciu authenticated) (Exim 4.54) id 1M4xUH-0009D3-A4 for multipathtcp@ietf.org; Fri, 15 May 2009 14:30:09 +0100 Mime-Version: 1.0 (Apple Message framework v753.1) In-Reply-To: References: <9AC7B18B-5167-4A14-ADCB-2DABC282C07D@cs.ucl.ac.uk> <6D11F360-2658-428A-9755-5478AE418919@muada.com> Content-Type: text/plain; charset=ISO-8859-1; delsp=yes; format=flowed Message-Id: <5C9609D3-60D2-431D-8712-50EB9730A613@cs.ucl.ac.uk> Content-Transfer-Encoding: quoted-printable From: Costin Raiciu Date: Fri, 15 May 2009 13:58:06 +0100 To: multipathtcp@ietf.org X-Mailer: Apple Mail (2.753.1) Subject: Re: [multipathtcp] We don't need to detect shared bottlenecks X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 May 2009 12:56:46 -0000 Hi, My remark comes from Damon Wischik's paper called "Buffer sizing =20 theory for bursty TCP flows" where he shows that bursty TCP traffic =20 is less prone to synchronization for different buffer sizes. Cheers, Costin On 14 May 2009, at 15:43, David Ros wrote: > > Le 13 mai 09 =E0 19:14, Costin Raiciu a =E9crit : > >> >>> >>> 3. Growing fast and backing off hard makes the sawtooth cycles =20 >>> faster so the network becomes burstier. >> Not if you choose milder parameters. Anyways, bursty is not always =20= >> bad; for one thing, it breaks synchronization between flows. >> > > Hi, > > Could you please elaborate a little on this? > > (I'm surely missing something obvious, but what you say regarding =20 > burstiness seems to somewhat contradict what some people [including =20= > us] have found, at least in the context of high-speed TCPs) > > Regards, > > David. > From alan.ford@roke.co.uk Fri May 15 05:59:07 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 079F93A6AC5 for ; Fri, 15 May 2009 05:59:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.599 X-Spam-Level: X-Spam-Status: No, score=-3.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WkgOtRf27ok4 for ; Fri, 15 May 2009 05:59:06 -0700 (PDT) Received: from rsys001x.roke.co.uk (rsys001x.roke.co.uk [193.118.201.108]) by core3.amsl.com (Postfix) with ESMTP id 7CE3F3A6956 for ; Fri, 15 May 2009 05:59:05 -0700 (PDT) Received: from rsys005a.comm.ad.roke.co.uk ([193.118.193.85]) by rsys001x.roke.co.uk (8.13.1/8.13.1) with ESMTP id n4FD0Pqd015902; Fri, 15 May 2009 14:00:27 +0100 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Fri, 15 May 2009 14:00:21 +0100 Message-ID: <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [multipathtcp] MPCC and different transports Thread-Index: AcnVVlexN8uR0X4/QpG1iiFhoME3mQABPpuA References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> From: "Ford, Alan" To: "Bryan Ford" X-roke-co-uk-MailScanner: Found to be clean X-roke-co-uk-MailScanner-SpamCheck: X-roke-co-uk-MailScanner-From: alan.ford@roke.co.uk Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 May 2009 12:59:07 -0000 Bryan, > On May 15, 2009, at 10:07 AM, Pasi Sarolahti wrote: > > I believe two-ended version can be made to work over many NAT > > configurations, if you don't do the dynamic address management by > > explicitly passing the addresses in TCP options. I think this > > implicit address management was one of the variants discussed in the > > two-ended draft. This is important, because the ability to go > > through NATs is one of the most important benefits of doing this in > > TCP vs. other protocols in my opinion. >=20 > How do you deal with TCP sequence numbers then? Or by "many NAT > configurations" did you mean only those that don't track TCP sequence > numbers and expect them to work as they normally do in TCP? How many > NAT configurations is that, I wonder, especially given the increasing > ubiquity of DPI technology that expects to see full TCP streams? (For > that matter, you don't even need NAT to have this problem: I bet many > non-NAT firewalls will do something bad with TCP streams that appear > to the firewall to be missing sequence numbers because they got sent > on another path.) In draft-ford-mptcp-multiaddressed we describe using two sequence spaces, so that each individual TCP connection that makes up a multipath connection appears to be a complete and independent TCP connection to a legacy NAT. The "implicit path management" as proposed in that draft allows new connections to be added without the need for source address knowledge. Very interested in hearing you thoughts about feasibility issues, of course! Regards, Alan From baford@mpi-sws.org Fri May 15 21:49:00 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 3EC303A68E4 for ; Fri, 15 May 2009 21:49:00 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.249 X-Spam-Level: X-Spam-Status: No, score=-6.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BFtXjG-OIpZL for ; Fri, 15 May 2009 21:48:58 -0700 (PDT) Received: from hera.mpi-sb.mpg.de (infao0809.mpi-sb.mpg.de [139.19.1.49]) by core3.amsl.com (Postfix) with ESMTP id 7923A3A6B8B for ; Fri, 15 May 2009 21:48:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mpi-sb.mpg.de; s=mail200803; h=Cc:Message-Id:From:To: In-Reply-To:Content-Type:Content-Transfer-Encoding:Mime-Version: Subject:Date:References; bh=tP+sND4eqDzdTfZ2ummY3T/t+AHq6iE51brb 4JAkOWo=; b=pHqc5fjmqG2OaG5suMETEa7SHNwj6clTuwY5ixLHVok0fA6543F5 Lkm3iVEP/jshOudwsX0jX6+WV5Elgp9z1iycjed6G0QJitnnXtls+3+dvFptSsg+ oeKNr6kU9xzE+moxk8nmS6qZAEBQ/bhGbE+pYcdU1iGJNcX8Hrqhc04= Received: from infao0525.mpi-sb.mpg.de ([139.19.1.26]:53754 helo=newmaniac.mpi-sb.mpg.de) by hera.mpi-sb.mpg.de (envelope-from ) with esmtp (Exim 4.69) id 1M5Bqt-0003oR-0G; Sat, 16 May 2009 06:50:30 +0200 Received: from adsl-84-227-28-216.adslplus.ch ([84.227.28.216]:63441 helo=[192.168.1.100]) by newmaniac.mpi-sb.mpg.de with esmtpsa (TLS-1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.63) (envelope-from ) id 1M5Bqs-0002jM-Dj; Sat, 16 May 2009 06:50:26 +0200 Message-Id: <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> From: Bryan Ford To: "Ford, Alan" In-Reply-To: <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Sat, 16 May 2009 06:50:25 +0200 References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> X-Mailer: Apple Mail (2.930.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 May 2009 04:49:00 -0000 On May 15, 2009, at 3:00 PM, Ford, Alan wrote: > Bryan, >> On May 15, 2009, at 10:07 AM, Pasi Sarolahti wrote: >>> I believe two-ended version can be made to work over many NAT >>> configurations, if you don't do the dynamic address management by >>> explicitly passing the addresses in TCP options. I think this >>> implicit address management was one of the variants discussed in the >>> two-ended draft. This is important, because the ability to go >>> through NATs is one of the most important benefits of doing this in >>> TCP vs. other protocols in my opinion. >> >> How do you deal with TCP sequence numbers then? Or by "many NAT >> configurations" did you mean only those that don't track TCP sequence >> numbers and expect them to work as they normally do in TCP? How many >> NAT configurations is that, I wonder, especially given the increasing >> ubiquity of DPI technology that expects to see full TCP streams? >> (For >> that matter, you don't even need NAT to have this problem: I bet many >> non-NAT firewalls will do something bad with TCP streams that appear >> to the firewall to be missing sequence numbers because they got sent >> on another path.) > > In draft-ford-mptcp-multiaddressed we describe using two sequence > spaces, so that each individual TCP connection that makes up a > multipath > connection appears to be a complete and independent TCP connection > to a > legacy NAT. The "implicit path management" as proposed in that draft > allows new connections to be added without the need for source address > knowledge. OK, I see where you're going. If you care about getting through DPI firewalls/NATs semi-reliably, though, I strongly suspect you'll have to scrap the resync packet mechanism described in section 4.4.3, as it seems extremely likely that many firewalls/NATs will balk in some way if you try to leave what looks to them like a hole in the TCP sequence space (which is what the resync mechanism will do). Instead I see no reliable alternative but to make each subflow individually faithful to TCP semantics, including retransmitting any data that was previously transmitted on that subflow and lost - i.e., retransmitting the data _on the same subflow_ even if the data has since been successfully transmitted and ACKed on a different subflow. Thus, shifting previously transmitted data among subflows would imply some semantically unnecessary data retransmissions, but this shouldn't be a performance problem as long as it doesn't happen too often, and it presumably shouldn't need to happen often if the transmit scheduler is designed with a bit of care. That still leaves all the issues with the new TCP options. Besides the issues you mention (e.g., having to fall back on single-path TCP if a NAT or firewall removes or rejects unknown options), there's a potentially more insidious one: what if a NAT or firewall allows unknown TCP options through but also mucks with the TCP sequence number space, e.g., by fragmenting, defragmenting, or realigning TCP segments along the way? I don't have any concrete data on this, but since many middleboxes fully reassemble the original TCP byte stream in order to do reliable DPI, it seems very likely that they may treat the TCP stream's original segment boundaries as advisory at best. For example, what if a middlebox receives a 2000-byte segment on its input side and fragments it into two 1000-byte segments for transmission on the smaller-MTU output side link, in the process simply duplicating any unkown TCP options? The second output segment will then have an incorrect Data Sequence Number option (the same as the first, rather than the first + 1000 as it should be), confusing or possibly corrupting the MPTCP connection. Similarly, what if two 1000-byte segments arrive at the middlebox out of order on a low MTU link, the middlebox buffers the first until the hole is filled so it can do DPI properly, and then it retransmits that data as one 2000-byte segment on the higher-MTU outgoing link? Those two 1000-byte segments might have had two different unrelated Data Sequence Numbers from MPTCP's perspective, but only one of those sequence numbers will get through, and the second 1000 bytes will get put in the wrong place in the higher-level stream. It might arguably be reasonable for us to hope (and pray!!!) that all or practically all deployed middleboxes either (a) drop all unknown TCP options, which castrates MPTCP by falling back on single-path mode but at least still works, or (b) pass through unknown TCP options but never modify the sequence numbering. It would be nice if this were the case, and might be a semi-reasonable expectation based on the likelihood that most middleboxes either process TCP segments packet-by- packet without doing much if anything to sequence numbers, or else fully split/proxy the TCP connection and in the process throw away unknown options since they're effectively acting as a TCP endpoint. But given all the innumerable types of weird behavior we know middleboxes to exhibit, this strikes me as wishful thinking. At the very least, someone would need to do a fairly large-scale test of the behavior of TCP with unknown options over a wide variety of deployed NATs, before we could have much confidence that this approach to an MPTCP would be reliable. Especially since the potential cost of guessing wrong on this count includes not just reversion to single- path behavior or even lost connectivity, but silent (and possibly very non-deterministic) application data corruption! Bryan > > Very interested in hearing you thoughts about feasibility issues, of > course! > > Regards, > Alan > From iljitsch@muada.com Sat May 16 05:47:35 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 55B5C3A67D2 for ; Sat, 16 May 2009 05:47:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Yo7KH0SQ0DIr for ; Sat, 16 May 2009 05:47:34 -0700 (PDT) Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by core3.amsl.com (Postfix) with ESMTP id 52E703A6808 for ; Sat, 16 May 2009 05:47:30 -0700 (PDT) Received: from [192.168.2.4] (static-167-138-7-89.ipcom.comunitel.net [89.7.138.167] (may be forged)) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n4GCmJfS090926 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Sat, 16 May 2009 14:48:21 +0200 (CEST) (envelope-from iljitsch@muada.com) Message-Id: From: Iljitsch van Beijnum To: Bryan Ford In-Reply-To: Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v935.3) Date: Sat, 16 May 2009 14:48:27 +0200 References: <4A0CA1D0.4040802@fe.up.pt> <61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> X-Mailer: Apple Mail (2.935.3) Cc: "multipathtcp@ietf.org" , "j.araujo@ucl.ac.uk" Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 May 2009 12:47:35 -0000 On 15 mei 2009, at 14:11, Bryan Ford wrote: > How do you deal with TCP sequence numbers then? Or by "many NAT > configurations" did you mean only those that don't track TCP > sequence numbers and expect them to work as they normally do in > TCP? How many NAT configurations is that, I wonder, especially > given the increasing ubiquity of DPI technology that expects to see > full TCP streams? Yes, that's a good question. On the one hand it could be that they insist on seeing all packets. However, even if the NAT/firewall saw the packet, the destination may not see it because it gets lost between the NAT/FW and the destination. So the window of sequence number space that is allowed through must track the acknowledgments from the destination. So that would suggest that there is a good chance that if the destination ACKs something that the NT/FW didn't see the window still progresses and there is no trouble. In this case, it would be possible to use a single sequence number space if we make sure that the NAT/FWs get to see enough packets to progress their windows, which would be about one per path per window. From iljitsch@muada.com Sat May 16 06:11:31 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 033633A6E65 for ; Sat, 16 May 2009 06:11:31 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id S+Yz1QGWsh7s for ; Sat, 16 May 2009 06:11:30 -0700 (PDT) Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by core3.amsl.com (Postfix) with ESMTP id 801433A6E86 for ; Sat, 16 May 2009 06:10:59 -0700 (PDT) Received: from [192.168.2.4] (static-167-138-7-89.ipcom.comunitel.net [89.7.138.167] (may be forged)) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n4GDBvXf091010 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Sat, 16 May 2009 15:11:58 +0200 (CEST) (envelope-from iljitsch@muada.com) Message-Id: <177A89C1-4D53-4212-BD0E-C68F8A6B2128@muada.com> From: Iljitsch van Beijnum To: Costin Raiciu In-Reply-To: <5C4FB334-B163-49C4-BC94-775247962B34@cs.ucl.ac.uk> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v935.3) Date: Sat, 16 May 2009 15:12:04 +0200 References: <4A0CA1D0.4040802@fe.up.pt> <61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <5C4FB334-B163-49C4-BC94-775247962B34@cs.ucl.ac.uk> X-Mailer: Apple Mail (2.935.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 May 2009 13:11:31 -0000 On 15 mei 2009, at 14:41, Costin Raiciu wrote: > since we are talking about deploying now, let's leave ipv6 out of > this. Absolutely not. IPv4 is walking on its last legs, deploying new IPv4-only technology now is almost criminal. > Finally, the one-ended one, and cmt for that matter, have the > following issue: when a packet is lost and retransmitted on a > different path, there is no way to know which of the paths delivered > the packet unless one adds "options" to the packets. This is a more > serious problem than not knowing whether the original or > retransmitted packet got there in traditional tcp, as it places the > host in the impossibility of deciding whether the path that carried > the retransmitted packet has lost or delivered the packet. There are two types of retransmissions: fast and RTO-based. In the case of a packet being delivered after the RTO expires and a retransmission has occurred, I think we can safely just pretent the original packet was lost, because it was delayed _a lot_. In theory then the retransmission on another path could be lost and we progress the cwin of that path incorrectly, but the circumstances under which this would happen are very unusual: a very large RTT increase on one path and then being extremely unlucky (normal packet loss is < 1%) or having massive congestion on the other path. If it's congestion we can safely assume another packet will drop and the cwin will shrink posthaste. The case where a packet that was presumed to be lost after fast retransmit was triggered and then delivered after all should be more common, as it doesn't require extremely large RTT increases. However, this should still be rare because it requires some significant reordering in the network. Because reordering is actively avoided, it makes sense that when it occurs, it's in large amounts because of a network path that uses parallelism that doesn't care about reordering so it will happen a lot, which would be bad for performance because of the constant cwin halvings. However, in that case, we shouldn't just see regular occurrences of three duplicate ACKs, but also larger numbers of single and double duplicate ACKs. Statistically, the difference between reordering and losses should be fairly obvious. Loss (assuming 6 pkts before a fast retransmitted packet is received): * * * * * * * * * * * * * * * * * * * * * * * * * * 1 2 3 4 5 6 7 8 9 Reordering: * * * * * * * * * * * * * * * 1 2 3 4 5 6 7 8 9 The reordering issue can be solved by simply upping the fast retransmit threshold. There are a few papers that look at this. Not sure if they make the behavior dynamic to preserve fast retransmit as much as possible, though. If you use a multiaddress multipath solution and you retransmit over a different path, depending on how the ACKs work, you may be able to detect the difference between the original packet and the retransmission through the addresses. From iljitsch@muada.com Sat May 16 06:21:44 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id A41AB3A6782 for ; Sat, 16 May 2009 06:21:44 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OgSt0U8pWMWx for ; Sat, 16 May 2009 06:21:43 -0700 (PDT) Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by core3.amsl.com (Postfix) with ESMTP id 833133A6ACF for ; Sat, 16 May 2009 06:21:06 -0700 (PDT) Received: from [192.168.2.4] (static-167-138-7-89.ipcom.comunitel.net [89.7.138.167] (may be forged)) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n4GDM3kH091045 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Sat, 16 May 2009 15:22:11 +0200 (CEST) (envelope-from iljitsch@muada.com) Message-Id: From: Iljitsch van Beijnum To: Bryan Ford In-Reply-To: <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v935.3) Date: Sat, 16 May 2009 15:22:11 +0200 References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> X-Mailer: Apple Mail (2.935.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 May 2009 13:21:44 -0000 On 16 mei 2009, at 6:50, Bryan Ford wrote: > That still leaves all the issues with the new TCP options. A discussion that a bunch of us had offline: would it be useful to exchange signaling information in-band at the beginning of a subflow? I.e., I set up something that looks like a new TCP session to middleboxes, but in reality it's an additional subflow for an existing session. Rather than negotiate all the housekeeping in options, I simply do that in-band. When the negotiation is finished, I start exchanging data. In fact, I could do that for all packets. The first n bytes following the TCP header could be the second sequence number. Obviously deep packet inspection boxes wouldn't like that, but if we can't do options, we must maintain the sequence numbering and we can't touch the data then maybe using that path for multipath isn't such a good idea after all. > But given all the innumerable types of weird behavior we know > middleboxes to exhibit, this strikes me as wishful thinking. The good thing here is that we don't really care that we can't get through a given middlebox. Obviously we want to get through as many as we can in order to be able to use as many paths as we can, but we always have the default path so losing additional paths is something that we can live with. The only issue would be middleboxes that do stuff that makes MPTCP work incorrectly. This could happen when the options and sequence numbers get out of alignment like you describe, so this situation may need to be detected. From john@jlc.net Sat May 16 06:42:49 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 163F53A6CA8 for ; Sat, 16 May 2009 06:42:49 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.935 X-Spam-Level: X-Spam-Status: No, score=-5.935 tagged_above=-999 required=5 tests=[AWL=0.664, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DI7IjwJ4gn9Q for ; Sat, 16 May 2009 06:42:48 -0700 (PDT) Received: from mailhost.jlc.net (mailhost.jlc.net [199.201.159.9]) by core3.amsl.com (Postfix) with ESMTP id 0C8273A67B1 for ; Sat, 16 May 2009 06:42:48 -0700 (PDT) Received: by mailhost.jlc.net (Postfix, from userid 104) id C2CF333C7A; Sat, 16 May 2009 09:44:22 -0400 (EDT) Date: Sat, 16 May 2009 09:44:22 -0400 From: John Leslie To: Bryan Ford Message-ID: <20090516134422.GE70521@verdi> References: <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> User-Agent: Mutt/1.4.1i Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 May 2009 13:42:49 -0000 Bryan Ford wrote: > > On May 15, 2009, at 3:00 PM, Ford, Alan wrote: >> >> In draft-ford-mptcp-multiaddressed we describe using two sequence >> spaces, so that each individual TCP connection that makes up a >> multipath connection appears to be a complete and independent TCP >> connection to a legacy NAT. The "implicit path management" as >> proposed in that draft allows new connections to be added without the >> need for source address knowledge. > > If you care about getting through DPI firewalls/NATs semi-reliably, > though, I strongly suspect you'll have to scrap the resync packet > mechanism described in section 4.4.3, as it seems extremely likely > that many firewalls/NATs will balk in some way if you try to leave > what looks to them like a hole in the TCP sequence space (which is > what the resync mechanism will do)... We're getting far beyond the usual BoF stage here -- speculating on what (future?) NATs might do. (I don't believe checking for holes in TCP sequence space is common today -- how would you ever recover synchronization if the NAT starts diddling?) > I don't have any concrete data on this, but since many middleboxes > fully reassemble the original TCP byte stream in order to do reliable > DPI, It seems unlikely that all that "many" middleboxes do deep packet inspection. And I most sincerely hope that we don't include waging an arms-escalation war with deep-packet-inspection in our charter. Deep-packet-inspection is something governments do, and they're likely to take offense if we try to play in that playground. :^( > it seems very likely that they may treat the TCP stream's original > segment boundaries as advisory at best. For example, what if a > middlebox receives a 2000-byte segment on its input side and fragments > it into two 1000-byte segments for transmission on the smaller-MTU > output side link, in the process simply duplicating any unkown TCP > options? I can barely imagine how many things that would break! > ... It might arguably be reasonable for us to hope (and pray!!!) > that all or practically all deployed middleboxes either (a) drop > all unknown TCP options, which castrates MPTCP by falling back on > single-path mode but at least still works, or (b) pass through > unknown TCP options but never modify the sequence numbering. NAT traversal is being considered in many IETF WGs. I haven't heard this level of speculation in other WGs. You are describing things that do serious violence to the very _basis_ of TCP; and you seem to be suggesting that we should avoid any changes to TCP that might be broken by broken middleboxes. I dissent. > ... But given all the innumerable types of weird behavior we know > middleboxes to exhibit, this strikes me as wishful thinking. At the > very least, someone would need to do a fairly large-scale test of the > behavior of TCP with unknown options over a wide variety of deployed > NATs, before we could have much confidence that this approach to an > MPTCP would be reliable. ... which I believe is the intent of our charter: making an Experimental protocol. -- John Leslie From baford@mpi-sws.org Sat May 16 12:21:42 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 57A8528C142 for ; Sat, 16 May 2009 12:21:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.249 X-Spam-Level: X-Spam-Status: No, score=-6.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Xm7ZJS6RpCsb for ; Sat, 16 May 2009 12:21:41 -0700 (PDT) Received: from hera.mpi-sb.mpg.de (infao0809.mpi-sb.mpg.de [139.19.1.49]) by core3.amsl.com (Postfix) with ESMTP id C0EFE3A68D9 for ; Sat, 16 May 2009 12:21:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mpi-sb.mpg.de; s=mail200803; h=Cc:Message-Id:From:To: In-Reply-To:Content-Type:Content-Transfer-Encoding:Mime-Version: Subject:Date:References; bh=dE4ClaWSOJOep7avURmVX6yEzy4a2LvXxr6w bk/h05c=; b=BeeBC5T8IyOQQsrC0ay4Qp3UoYE4jdGaCrKI3hlFbrGFqY4dAhru jbftq7m1DZqAbjwBq7iUpya+VE6qffqpAWSaRv2jAJyQlD4FSalYc8xmh4Jv7MQr Alpu7/hSb7Am6tIKf+nyXe+oV7Cac2zqP4y5glrts8kOJ6cwOUZchJE= Received: from newmaniac.mpi-sb.mpg.de ([139.19.1.26]:58313) by hera.mpi-sb.mpg.de (envelope-from ) with esmtp (Exim 4.69) id 1M5PTR-0008O8-JZ; Sat, 16 May 2009 21:23:12 +0200 Received: from adsl-84-227-28-216.adslplus.ch ([84.227.28.216]:61803 helo=[192.168.1.100]) by newmaniac.mpi-sb.mpg.de with esmtpsa (TLS-1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.63) (envelope-from ) id 1M5PTQ-0005Ub-Ls; Sat, 16 May 2009 21:23:09 +0200 Message-Id: <8FA18689-5954-4813-AEF5-480FC68A9E76@mpi-sws.org> From: Bryan Ford To: Iljitsch van Beijnum In-Reply-To: Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Sat, 16 May 2009 21:23:05 +0200 References: <4A0CA1D0.4040802@fe.up.pt> <61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> X-Mailer: Apple Mail (2.930.3) Cc: "multipathtcp@ietf.org" , "j.araujo@ucl.ac.uk" Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 May 2009 19:21:42 -0000 On May 16, 2009, at 2:48 PM, Iljitsch van Beijnum wrote: > On 15 mei 2009, at 14:11, Bryan Ford wrote: >> How do you deal with TCP sequence numbers then? Or by "many NAT >> configurations" did you mean only those that don't track TCP >> sequence numbers and expect them to work as they normally do in >> TCP? How many NAT configurations is that, I wonder, especially >> given the increasing ubiquity of DPI technology that expects to see >> full TCP streams? > > Yes, that's a good question. > > On the one hand it could be that they insist on seeing all packets. > However, even if the NAT/firewall saw the packet, the destination > may not see it because it gets lost between the NAT/FW and the > destination. So the window of sequence number space that is allowed > through must track the acknowledgments from the destination. So that > would suggest that there is a good chance that if the destination > ACKs something that the NT/FW didn't see the window still progresses > and there is no trouble. I agree that this is fairly reasonable behavior for a middlebox that merely tracks TCP flows on a packet-by-packet basis and doesn't need to or try to reconstruct the original semantic byte stream that's being transmitted via TCP. But some middleboxes do need to reconstruct the original byte stream: e.g., firewalls that parse HTTP or other application-level headers and filter traffic based on them; virus scanners that scan for virus signatures within payloads, etc. If such a firewall has a policy that involves analyzing the reconstructed TCP byte stream as a condition on allowing the connection to continue or complete, then it seems unlikely that the firewall would just wave on through any holes in the sequence number space: more likely, after filling its reorder buffer space it will simply block the stream until the holes are filled, which may be never if the stream is an MPTCP stream. Further, many middleboxes even have the capability to insert or remove bytes in TCP streams, and supporting that capability entails not only reconstructing the original byte stream but delaying the forwarding of segments received out of order until the "hole" is filled. For example, if the NAT were to receive segment 2000-2999 out of order, forward it, and then upon receiving a prior segment 1000-1999 only then discover that it needs to insert 15 bytes somewhere in that segment, it's screwed because segment 2000-2999 should have had its sequence numbers translated to 2015-3014. Thus such a NAT _MUST_ forward the segments in order in the off chance that it might have to translate the sequence numbers - even if on most streams its "trigger condition" for doing such a modification will never be activated. NATs have been doing this to FTP control connections for years, and we've all heard of the evil ISPs that recently started deploying boxes that insert ads into the HTML payloads of HTTP connections their customers make to random web sites. I'm not in any way trying to defend such practices in general, and we might come up with reasons to hope that such TCP stream modification capabilities would never actually be triggered on an MPTCP stream masquerading as a TCP stream - but just the fact that many middleboxes have such capabilities means that they HAVE TO have been designed with the ability to buffer out-of-order packets and wait for holes to be filled before forwarding data, and it's anyone's guess under what (perhaps even unnecessary) conditions this capability in a particular NAT vendor's box might be in effect on random TCP connections, causing hole-filled MPTCP streams to grind suddenly to a halt. I'm not claiming to know whether this is really a big, small, or nonexistent problem in practice: I'm just saying that it's a concern that seams realistic, we don't yet have the data to answer that question, and it seems like we would need to before we should commit to an MPTCP design that fills TCP connections with holes. Bryan From iljitsch@muada.com Sat May 16 13:53:36 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 3CFD43A6C30 for ; Sat, 16 May 2009 13:53:36 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Z8RfmZ2npuyr for ; Sat, 16 May 2009 13:53:35 -0700 (PDT) Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by core3.amsl.com (Postfix) with ESMTP id 2D01D3A67D9 for ; Sat, 16 May 2009 13:53:35 -0700 (PDT) Received: from [192.168.2.4] (static-167-138-7-89.ipcom.comunitel.net [89.7.138.167] (may be forged)) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n4GKsKNo093355 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Sat, 16 May 2009 22:54:37 +0200 (CEST) (envelope-from iljitsch@muada.com) Message-Id: <312FDCC4-772D-4D11-912D-31B77BCB2094@muada.com> From: Iljitsch van Beijnum To: Bryan Ford In-Reply-To: <8FA18689-5954-4813-AEF5-480FC68A9E76@mpi-sws.org> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v935.3) Date: Sat, 16 May 2009 22:54:27 +0200 References: <4A0CA1D0.4040802@fe.up.pt> <61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <8FA18689-5954-4813-AEF5-480FC68A9E76@mpi-sws.org> X-Mailer: Apple Mail (2.935.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 May 2009 20:53:36 -0000 On 16 mei 2009, at 21:23, Bryan Ford wrote: > But some middleboxes do need to reconstruct the original byte > stream: e.g., firewalls that parse HTTP or other application-level > headers and filter traffic based on them; virus scanners that scan > for virus signatures within payloads, etc. Ugh! However, at this stage we're beyond regular NAT and even typical firewall behavior. The trouble is that we would have to detect this kind of behavior to disable the affected subflows because otherwise the integrity of the data stream would be compromised. (Why is it "compromised" but "analyzed"? English is so infuriatingly inconsistent.) I guess a good way to do this would be with some kind of HMAC protection of the data. (There goes the option space...) [...] > NATs have been doing this to FTP control connections for years, Right, and with stuff like SIP and RTSP. However, this is all low- volume control stuff, so if we can detect this we can disable multipath for it without ill effects. > and we've all heard of the evil ISPs that recently started deploying > boxes that insert ads into the HTML payloads of HTTP connections > their customers make to random web sites. That's nature's way of telling people that they should use encryption... [...] > causing hole-filled MPTCP streams to grind suddenly to a halt. Perhaps this is a good opportunity for Costin to remind us why it was a bad idea to send a packet with the seqnum that would fill the hole, but now with different data? It seems that that would solve this particular issue. From c.raiciu@cs.ucl.ac.uk Sat May 16 15:01:15 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id C286F3A696B for ; Sat, 16 May 2009 15:01:15 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 53L2VZXG2+CE for ; Sat, 16 May 2009 15:01:14 -0700 (PDT) Received: from bells2.cs.ucl.ac.uk (bells2.cs.ucl.ac.uk [128.16.5.33]) by core3.amsl.com (Postfix) with ESMTP id 5E9583A6A46 for ; Sat, 16 May 2009 15:01:07 -0700 (PDT) Received: from [82.45.130.212] (helo=[172.24.121.151]) by bells2.cs.ucl.ac.uk with esmtpsa (TLSv1:AES128-SHA:128) (C.Raiciu authenticated) (Exim 4.54) id 1M5SS4-0009Pf-VO for multipathtcp@ietf.org; Sat, 16 May 2009 23:33:57 +0100 Mime-Version: 1.0 (Apple Message framework v753.1) In-Reply-To: <312FDCC4-772D-4D11-912D-31B77BCB2094@muada.com> References: <4A0CA1D0.4040802@fe.up.pt> <61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <8FA18689-5954-4813-AEF5-480FC68A9E76@mpi-sws.org> <312FDCC4-772D-4D11-912D-31B77BCB2094@muada.com> Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <97A4DC0A-620A-41A1-A0ED-A1B49BEBFDA0@cs.ucl.ac.uk> Content-Transfer-Encoding: 7bit From: Costin Raiciu Date: Sat, 16 May 2009 23:02:36 +0100 To: multipathtcp@ietf.org X-Mailer: Apple Mail (2.753.1) Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 May 2009 22:01:15 -0000 I agree with Bryan that there are many failure modes, and one of the reasons to not retransmit different data with the same seq no was the fact that traffic normalizers cache seqno-data mappings and replay the original (cached) data instead of the new data when the same seqno arrives - if the packet was lost after the normalizer, than this corrupts application data. Also, when you get an ack for that data you don't know whether the original data arrived (that maybe got lost) or the new one (remember, there is no data ack). The resync packet is just an optimization to avoid resending useless packets, but and it could be scrapped it for compliance. I admit had not thought about the fragmentation/reassembly scenario before. Iljitsch's proposal to do in band signalling doesn't help there either; when you fragment a packet whose data begins with a data seq num, the second packet will not have the proper seq num. I can think of a few heuristics to detect when such fragmentation reassembly happens, but am not sure they cover all cases. To be able to deterministically tell whether the packet has been fragmented one fix would be to include the data size as yet another option. That seems like an overkill :(. Cheers, Costin On 16 May 2009, at 21:54, Iljitsch van Beijnum wrote: > On 16 mei 2009, at 21:23, Bryan Ford wrote: > >> But some middleboxes do need to reconstruct the original byte >> stream: e.g., firewalls that parse HTTP or other application-level >> headers and filter traffic based on them; virus scanners that scan >> for virus signatures within payloads, etc. > > Ugh! > > However, at this stage we're beyond regular NAT and even typical > firewall behavior. > > The trouble is that we would have to detect this kind of behavior > to disable the affected subflows because otherwise the integrity of > the data stream would be compromised. (Why is it "compromised" but > "analyzed"? English is so infuriatingly inconsistent.) > > I guess a good way to do this would be with some kind of HMAC > protection of the data. (There goes the option space...) > > [...] > >> NATs have been doing this to FTP control connections for years, > > Right, and with stuff like SIP and RTSP. However, this is all low- > volume control stuff, so if we can detect this we can disable > multipath for it without ill effects. > >> and we've all heard of the evil ISPs that recently started >> deploying boxes that insert ads into the HTML payloads of HTTP >> connections their customers make to random web sites. > > That's nature's way of telling people that they should use > encryption... > > [...] > >> causing hole-filled MPTCP streams to grind suddenly to a halt. > > Perhaps this is a good opportunity for Costin to remind us why it > was a bad idea to send a packet with the seqnum that would fill the > hole, but now with different data? It seems that that would solve > this particular issue. > _______________________________________________ > multipathtcp mailing list > multipathtcp@ietf.org > https://www.ietf.org/mailman/listinfo/multipathtcp From Michael.Tuexen@lurchi.franken.de Sun May 17 02:58:30 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 97AF93A6A55 for ; Sun, 17 May 2009 02:58:30 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.354 X-Spam-Level: X-Spam-Status: No, score=-0.354 tagged_above=-999 required=5 tests=[AWL=1.595, BAYES_00=-2.599, HELO_EQ_DE=0.35, MIME_8BIT_HEADER=0.3] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pFLOOM9sEn54 for ; Sun, 17 May 2009 02:58:29 -0700 (PDT) Received: from mail-n.franken.de (mail-n.franken.de [193.175.24.27]) by core3.amsl.com (Postfix) with ESMTP id 934993A6C97 for ; Sun, 17 May 2009 02:58:28 -0700 (PDT) Received: from [192.168.1.194] (p508FD7DD.dip.t-dialin.net [80.143.215.221]) by mail-n.franken.de (Postfix) with ESMTP id 4FC271C0C0BD8; Sun, 17 May 2009 11:59:31 +0200 (CEST) Message-Id: From: =?ISO-8859-1?Q?Michael_T=FCxen?= To: Bryan Ford In-Reply-To: <61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed; delsp=yes Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Apple Message framework v935.3) Date: Sun, 17 May 2009 11:59:30 +0200 References: <4A0CA1D0.4040802@fe.up.pt> <61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> X-Mailer: Apple Mail (2.935.3) Cc: "multipathtcp@ietf.org" , "j.araujo@ucl.ac.uk" Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 May 2009 09:58:30 -0000 On May 15, 2009, at 8:22 AM, Bryan Ford wrote: > On May 15, 2009, at 12:57 AM, Jo=E3o Taveira Ara=FAjo =20= > wrote: > >> Iljitsch van Beijnum wrote: >>> So if we have a shared architecture, does it make sense to =20 >>> implement the multipath congestion control up to five separate =20 >>> times? >>> >>> Wouldn't it make more sense to define an abstraction layer between =20= >>> the transport protocol and the congestion control algorithm? >> This makes sense and reminds me of Bryan's case for decoupling CC =20 >> and transport semantics in "Breaking up the transport logjam". But =20= >> are we drifting away from the original motivation for extending TCP =20= >> for multipath, namely for deployment purposes. Is the purpose of =20 >> the eventual WG to work on multipath as a general concept and =20 >> attempt to retrofit these results to transport protocols, or to =20 >> take into account the limitations TCP and retain backwards =20 >> compatibility whilst enabling multipath? > > If the latter is the case, then I think that the scope of this WG =20 > should include ONLY single-ended TCP, because that's the only =20 > approach we know of so far that preserves any kind of backward =20 > compatibility. A two-ended MPTCP will encounter all the same =20 > deployment challenges that, say, CMT SCTP does, including the huge =20 > practical problem of legacy NATs in the path. Hi Bryan, just a note: The "problem" is not with CMT SCTP and NATs, but with =20 multihoming an NATs. Please note that this problem can be solved by using a NAT as =20 described in http://www.ietf.org/internet-drafts/draft-ietf-behave-sctpnat-01.txt or one can use UDP encapsulation. Both mechanisms are implemented in =20 FreeBSD. (which also has an implementation of CMT). > > > Bryan > >> >> >> Cheers, >> Joao. >> >> >> >> _______________________________________________ >> multipathtcp mailing list >> multipathtcp@ietf.org >> https://www.ietf.org/mailman/listinfo/multipathtcp > _______________________________________________ > multipathtcp mailing list > multipathtcp@ietf.org > https://www.ietf.org/mailman/listinfo/multipathtcp From Michael.Tuexen@lurchi.franken.de Sun May 17 03:03:48 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 12E793A6C97 for ; Sun, 17 May 2009 03:03:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.62 X-Spam-Level: X-Spam-Status: No, score=-0.62 tagged_above=-999 required=5 tests=[AWL=1.329, BAYES_00=-2.599, HELO_EQ_DE=0.35, MIME_8BIT_HEADER=0.3] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id c2eQTlsxzxxI for ; Sun, 17 May 2009 03:03:40 -0700 (PDT) Received: from mail-n.franken.de (mail-n.franken.de [193.175.24.27]) by core3.amsl.com (Postfix) with ESMTP id 5339D3A6C84 for ; Sun, 17 May 2009 03:03:39 -0700 (PDT) Received: from [192.168.1.194] (p508FD7DD.dip.t-dialin.net [80.143.215.221]) by mail-n.franken.de (Postfix) with ESMTP id 8A7721C0C0BD8; Sun, 17 May 2009 12:04:43 +0200 (CEST) Message-Id: From: =?ISO-8859-1?Q?Michael_T=FCxen?= To: Lloyd Wood In-Reply-To: <1566C013-B291-48E7-A826-7667C1B0502C@surrey.ac.uk> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v935.3) Date: Sun, 17 May 2009 12:04:42 +0200 References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es><4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><20090507230452.GC35382@cisco.com><1241789497.4055.8.camel@nhuynhth-laptop><001e01c9cfff$bde8f1e0$103947ab@cisco.com><97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com><001f01c9d005$0c583110$103947ab@cisco.com><1241818843.26211.67.camel@nhuynhth-laptop> <0D6E7BC4-17AF-4F5C-9107-ABA2B729683F@muada.com> <000b01c9d250$a9c311c0$103947ab@cisco.com> <1566C013-B291-48E7-A826-7667C1B0502C@surrey .ac.uk> X-Mailer: Apple Mail (2.935.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] TCP/SCTP and BoF description, was: Re: TCP modification vs switching to SCTP X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 May 2009 10:03:48 -0000 On May 11, 2009, at 6:20 PM, Lloyd Wood wrote: > CRC32c isn't intensive compared to e.g. message digests. Hi Lloyd, CRC32C offloading is now available in 3rd generation Intel NICs and supported by the next release of FreeBSD. I assume that other NICs will also support it in the future and therefore the impact of CRC32C becomes less... Best regards Michael > > > The choice of a better checksum mechanism for SCTP was predicated on > the fact that the ones-complement checksum used in TCP/UDP is very > weak, and doesn't catch swapped words - which is how echo servers > worked way back when. Swap the addresses, the ones-complement > checksum doesn't even notice. > > See RFC3309 for some reasons for choosing CRC32c over the previous > Adler32 (which is still better than the UDP/TCP checksums). > > The size and complexity of the SCTP state machine and whether its > features are desirable or necessary are better embedded arguments > against SCTP than its choice of checksum to protect itself. > > > On 11 May 2009, at 16:53, Preethi Natarajan wrote: >>> Another difference is that TCP is extremely simple, while >>> SCTP has a more complex structure with its chunks. This and >>> the CRC32 checksum make SCTP processing more CPU intensive >>> than TCP processing. I don't know how big the difference is, >>> but I'd say that SCTP isn't appropriate in very constrained >>> environments >> >> 1. CRC32c might be an intensive operation. Nonetheless CRC32c was >> chosen for >> its superior error detection mechanisms. >> 2. As you mention, the notion of "very constrained envrionments" >> has been a >> moving target -- what we think is "very constrained" today might >> not be in >> the future. > > DTN work: http://info.ee.surrey.ac.uk/Personal/L.Wood/saratoga/ > > > > > > > > _______________________________________________ > multipathtcp mailing list > multipathtcp@ietf.org > https://www.ietf.org/mailman/listinfo/multipathtcp > From Michael.Tuexen@lurchi.franken.de Sun May 17 03:05:26 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 64B4F3A6C9C for ; Sun, 17 May 2009 03:05:26 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.81 X-Spam-Level: X-Spam-Status: No, score=-0.81 tagged_above=-999 required=5 tests=[AWL=1.139, BAYES_00=-2.599, HELO_EQ_DE=0.35, MIME_8BIT_HEADER=0.3] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GpKimpqBU33X for ; Sun, 17 May 2009 03:05:25 -0700 (PDT) Received: from mail-n.franken.de (mail-n.franken.de [193.175.24.27]) by core3.amsl.com (Postfix) with ESMTP id F24463A6C84 for ; Sun, 17 May 2009 03:05:24 -0700 (PDT) Received: from [192.168.1.194] (p508FD7DD.dip.t-dialin.net [80.143.215.221]) by mail-n.franken.de (Postfix) with ESMTP id 841E01C0C0BD8; Sun, 17 May 2009 12:06:28 +0200 (CEST) Message-Id: <2B82EE72-A40B-48AF-B963-16A052DF542E@lurchi.franken.de> From: =?ISO-8859-1?Q?Michael_T=FCxen?= To: Iljitsch van Beijnum In-Reply-To: Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v935.3) Date: Sun, 17 May 2009 12:06:27 +0200 References: <4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><4A03635F.4010107@it.uc3m.es> <4A02FC62.8080503@it.uc3m.es><4A031B0F.4030606@employees.org> <4A031E1E.5050900@it.uc3m.es><20090507211619.GA35382@cisco.com><5E59C0C6-4A97-4253-954D-05139E1F684C@muada.com><20090507230452.GC35382@cisco.com><1241789497.4055.8.camel@nhuynhth-laptop><001e01c9cfff$bde8f1e0$103947ab@cisco.com><97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com><001f01c9d005$0c583110$103947ab@cisco.com><1241818843.26211.67.camel@nhuynhth-laptop> <0D6E7BC4-17AF-4F5C-9107-ABA2B729683F@muada.com> <000b01c9d250$a9c311c0$103947ab@cisco.com> X-Mailer: Apple Mail (2.935.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] TCP/SCTP and BoF description, was: Re: TCP modification vs switching to SCTP X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 May 2009 10:05:26 -0000 On May 11, 2009, at 9:32 PM, Iljitsch van Beijnum wrote: > On 11 mei 2009, at 17:53, Preethi Natarajan wrote: > >> 1. CRC32c might be an intensive operation. Nonetheless CRC32c was >> chosen for >> its superior error detection mechanisms. > > I understand. However, there are times when spending CPU cycles (I'm > assuming the SCTP checksum calculation can't be offloaded to > hardware) needed to do this are not worth the benefit it provides. It can be using 3rd gen. Intel NICs. Support for it has recently been added to FreeBSD. > > >> My larger point -- why are _we_ trying to decide which transport >> would be >> best for today's/tomorrow's requirements? Instead, I would prefer >> that the >> consumers (app developers) make that decision. > > Certainly, I agree. > >> I was under the impression that nobody has even tried to do this with >> SCTP... there are a few upcoming proposals but thats about it? > > Like I said, SRV records only have a port number, not a protocol > number. > >>> So if this BoF results in a wg, and that wg would take on >>> this work for both TCP and SCTP, how would that work? Would >>> we have different documents for TCP and for SCTP, or one >>> document? Is anyone volunteering to spend time on this for SCTP? > >> I know that there are volunteers for SCTP, and am sure there will be >> volunteers for other transports as well. > > Ok, let's think about this. > > With regard to the single ended case: in addition to the multipath > congestion control that is common to all types of multipath with > "normal" congestion control, the tricky parts here are how to handle > fast retransmit, how to handle stalls because of receive buffer > depletion and correlating incoming ACKs with the path the ACKed > packets were sent over. > > Without having studied SCTP in detail, it seems to me that the one- > ended multipath TCP draft could easily be applied to SCTP with minor > changes. This could be done during the TCP work or immediately after > so a single document can apply to both if there are no > incompatibilities. If the SCTP stuff requires a lot of extra text, > it probably makes more sense to make a separate document. > > AS for the two-ended multiaddress multipath: SCTP already does all > the protocol work, the only thing that's needed is apply the > multipath congestion control. > > Iljitsch > _______________________________________________ > multipathtcp mailing list > multipathtcp@ietf.org > https://www.ietf.org/mailman/listinfo/multipathtcp > From baford@mpi-sws.org Sun May 17 10:19:09 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id E22743A6DE5 for ; Sun, 17 May 2009 10:19:09 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.249 X-Spam-Level: X-Spam-Status: No, score=-6.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4iDEyJAAoIFp for ; Sun, 17 May 2009 10:19:03 -0700 (PDT) Received: from hera.mpi-sb.mpg.de (hera.mpi-sb.mpg.de [139.19.1.49]) by core3.amsl.com (Postfix) with ESMTP id 6182A28C1D7 for ; Sun, 17 May 2009 10:19:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mpi-sb.mpg.de; s=mail200803; h=Cc:Message-Id:From:To: In-Reply-To:Content-Type:Content-Transfer-Encoding:Mime-Version: Subject:Date:References; bh=18HAmrC4AfVAblrsTS3xuSGycaOuhrPLLN8C UUaWwxs=; b=Z4FmK67w4l9qaO5sHvYcfZipMhAgOLr13Yf9OrThXAZOvotFjvvi wIpU6Pyn5h0MChce+1A77fIRFFeq8/ZkdQgWF9xSol8iK2yMAtuyTP+eR67D5R82 BK4dzIt/I6//u+aV3tBIbzy4bvhBveKDKUa0K9ENN0DUbyA06N9b+fc= Received: from infao0525.mpi-sb.mpg.de ([139.19.1.26]:39199 helo=newmaniac.mpi-sb.mpg.de) by hera.mpi-sb.mpg.de (envelope-from ) with esmtp (Exim 4.69) id 1M5k2I-00012D-G2; Sun, 17 May 2009 19:20:33 +0200 Received: from [84.226.232.123] (port=62545 helo=[192.168.1.102]) by newmaniac.mpi-sb.mpg.de with esmtpsa (TLS-1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.63) (envelope-from ) id 1M5k2G-0001WA-Re; Sun, 17 May 2009 19:20:30 +0200 Message-Id: From: Bryan Ford To: Iljitsch van Beijnum In-Reply-To: Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Sun, 17 May 2009 19:20:27 +0200 References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> X-Mailer: Apple Mail (2.930.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 May 2009 17:19:10 -0000 On May 16, 2009, at 3:22 PM, Iljitsch van Beijnum wrote: > On 16 mei 2009, at 6:50, Bryan Ford wrote: > >> That still leaves all the issues with the new TCP options. > > A discussion that a bunch of us had offline: would it be useful to > exchange signaling information in-band at the beginning of a subflow? > > I.e., I set up something that looks like a new TCP session to > middleboxes, but in reality it's an additional subflow for an > existing session. Rather than negotiate all the housekeeping in > options, I simply do that in-band. When the negotiation is finished, > I start exchanging data. > > In fact, I could do that for all packets. The first n bytes > following the TCP header could be the second sequence number. > > Obviously deep packet inspection boxes wouldn't like that, but if we > can't do options, we must maintain the sequence numbering and we > can't touch the data then maybe using that path for multipath isn't > such a good idea after all. This approach sounds more promising to me in general. Then MPTCP starts to take more and more of the flavor of a new transport protocol layered atop an existing protocol (namely legacy single-path TCP), taking advantage of legacy TCP to provide per-flow congestion control and get through middleboxes. From another angle, in the architectural view Jana and I proposed in our HotNets paper, it sounds like MPTCP is effectively using legacy TCP as a Flow Layer protocol, providing per- flow congestion control (and middlebox compatibility) but implementing a new inter-flow level of sequencing and retransmission to provide the new, "real" transport protocol. If MPTCP builds atop legacy TCP mostly as-is, using TCP as a Flow Layer (perhaps with enhancements to legacy TCP's congestion control to address the fairness/aggressiveness/resource pooling issues), and if MPTCP doesn't play too fast and loose with legacy TCP's options or sequencing, then individual MPTCP flows should even be able to get through fancier middleboxes like Performance Enhancing Proxies (PEPs). This approach would also have the nice benefit of effectively restoring the long-lost "end-to-endness" of TCP's originally intended semantics, from the perspective of the application, even when many or all of the individual legacy TCP flows atop which an MPTCP connection is built are being split in the middle by stateful firewalls, NATs, and/or PEPs. For example, if a stateful middlebox crashes and loses its legacy TCP state, the legacy TCP connection is hosed, but the MPTCP connection might well be able to notice the TCP flow's connection reset and just retry establishing that flow, while shifting traffic to other flows in the meantime. Taking this approach further, have you thought about how MPTCP should/ might interact with IPSEC? Of course IPSEC can always be deployed above IP and below legacy TCP, as usual, but we know how well IPSEC tends to get along with middleboxes. Combining traditional IPSEC with MPTCP would effectively defeat all the care MPTCP went to to provide backward compatibility by "looking like" legacy TCP. Even with IPSEC's recent UDP encapsulation enhancements to get through NATs and firewalls, it remains incompatible with PEPs or TCP proxies that need to split or monitor TCP's congestion control loop, not to mention middleboxes that let through only TCP and not UDP. But taking the view of legacy TCP as a Flow Layer when used by MPTCP, the right place to deploy IPSEC is actually _above_ legacy TCP but below MPTCP's inter- flow transport layer. This way, "IPSEC-secured MPTCP" would have all the same backward compatibility advantages of unsecured MPTCP because the Flow Layer part looks just like ordinary TCP, but the IPSEC layer in the middle would then be able to enforce the true "end-to-endness" of the MPSEC transport layer by preventing middleboxes from interfering with or even seeing the MPSEC-level data transmission loop. >> But given all the innumerable types of weird behavior we know >> middleboxes to exhibit, this strikes me as wishful thinking. > > The good thing here is that we don't really care that we can't get > through a given middlebox. Obviously we want to get through as many > as we can in order to be able to use as many paths as we can, but we > always have the default path so losing additional paths is something > that we can live with. The only issue would be middleboxes that do > stuff that makes MPTCP work incorrectly. This could happen when the > options and sequence numbers get out of alignment like you describe, > so this situation may need to be detected. I agree that it's not disaster if MPTCP doesn't manage to run multipath through certain types of middleboxes - although if backward compatibility is one of MPTCP's primary design goals, it's obviously best if it can run multipath through as wide a variety of middleboxes as possible, and I believe the above approach would achieve that. Cheers, Bryan From baford@mpi-sws.org Sun May 17 10:32:28 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 84B483A68F4 for ; Sun, 17 May 2009 10:32:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.249 X-Spam-Level: X-Spam-Status: No, score=-6.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id l3gIcRizFHvn for ; Sun, 17 May 2009 10:32:22 -0700 (PDT) Received: from hera.mpi-sb.mpg.de (infao0809.mpi-sb.mpg.de [139.19.1.49]) by core3.amsl.com (Postfix) with ESMTP id 106813A68C1 for ; Sun, 17 May 2009 10:32:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mpi-sb.mpg.de; s=mail200803; h=Cc:Message-Id:From:To: In-Reply-To:Content-Type:Content-Transfer-Encoding:Mime-Version: Subject:Date:References; bh=akIXbT4o3jWUsC9xNNGDUmjJbCuD0KZN7LUx go/zUfY=; b=d2ZPcc+zyGOof0b67xSYO28V8At/fOuOjAl5XMF0W86SAOKFHHaI aEus0K6uBl+4iiGgAh8+X6pxZcWYMa3fOQMGMartWw+zF7SnAl85p5w1MB75JzhI uCcIbdIZ/280WVLHRtLzXaO4KQh0ixIyzUUpP9dAEs446nWsuedysKY= Received: from infao0525.mpi-sb.mpg.de ([139.19.1.26]:59294 helo=newmaniac.mpi-sb.mpg.de) by hera.mpi-sb.mpg.de (envelope-from ) with esmtp (Exim 4.69) id 1M5kFE-0004RH-8P; Sun, 17 May 2009 19:33:55 +0200 Received: from [84.226.232.123] (port=63213 helo=[192.168.1.102]) by newmaniac.mpi-sb.mpg.de with esmtpsa (TLS-1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.63) (envelope-from ) id 1M5kFD-0001ZR-JI; Sun, 17 May 2009 19:33:51 +0200 Message-Id: <27A3D7CE-F621-4FCE-B378-CF625D44337E@mpi-sws.org> From: Bryan Ford To: John Leslie In-Reply-To: <20090516134422.GE70521@verdi> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Sun, 17 May 2009 19:33:50 +0200 References: <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> <20090516134422.GE70521@verdi> X-Mailer: Apple Mail (2.930.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 May 2009 17:32:28 -0000 On May 16, 2009, at 3:44 PM, John Leslie wrote: > Bryan Ford wrote: >> >> On May 15, 2009, at 3:00 PM, Ford, Alan wrote: >>> >>> In draft-ford-mptcp-multiaddressed we describe using two sequence >>> spaces, so that each individual TCP connection that makes up a >>> multipath connection appears to be a complete and independent TCP >>> connection to a legacy NAT. The "implicit path management" as >>> proposed in that draft allows new connections to be added without >>> the >>> need for source address knowledge. >> >> If you care about getting through DPI firewalls/NATs semi-reliably, >> though, I strongly suspect you'll have to scrap the resync packet >> mechanism described in section 4.4.3, as it seems extremely likely >> that many firewalls/NATs will balk in some way if you try to leave >> what looks to them like a hole in the TCP sequence space (which is >> what the resync mechanism will do)... > > We're getting far beyond the usual BoF stage here -- speculating > on what (future?) NATs might do. (I don't believe checking for holes > in TCP sequence space is common today -- how would you ever recover > synchronization if the NAT starts diddling?) We're not talking about future NATs here: almost all NATs already have the capability to reconstruct the byte stream and diddle with sequence numbers (e.g., for FTP control connections, which NATs have been doing for years). This capability might currently only get activated under certain very specialized conditions that we might hope would never include MPTCP flows, but we just don't know until we start testing. And as I said earlier, the problem here is that the price for getting this decision wrong even only once in a very rare while can be extremely high: e.g., silent nondeterministic corruption of application data. >> I don't have any concrete data on this, but since many middleboxes >> fully reassemble the original TCP byte stream in order to do reliable >> DPI, > > It seems unlikely that all that "many" middleboxes do deep packet > inspection. And I most sincerely hope that we don't include waging > an arms-escalation war with deep-packet-inspection in our charter. > Deep-packet-inspection is something governments do, and they're > likely to take offense if we try to play in that playground. :^( Deep packet inspection is something practically all state-of-the-art corporate firewalls do. Take a look at the marketing literature for McAfee Sidewinder just as one example. In fact I suspect if you asked a firewall marketing person whether a non-DPI firewall could be marketed today, I expect they'd say no. >> it seems very likely that they may treat the TCP stream's original >> segment boundaries as advisory at best. For example, what if a >> middlebox receives a 2000-byte segment on its input side and >> fragments >> it into two 1000-byte segments for transmission on the smaller-MTU >> output side link, in the process simply duplicating any unkown TCP >> options? > > I can barely imagine how many things that would break! Why would it break anything? It would just be doing the same things the original TCP sender would do, while remaining perfectly consistent with TCP's byte stream semantics. (Of course it breaks TCP's end-to- endness, but all stateful middleboxes - i.e., practically all middleboxes - already do that, and we don't like that but have learned to live with it.) >> ... It might arguably be reasonable for us to hope (and pray!!!) >> that all or practically all deployed middleboxes either (a) drop >> all unknown TCP options, which castrates MPTCP by falling back on >> single-path mode but at least still works, or (b) pass through >> unknown TCP options but never modify the sequence numbering. > > NAT traversal is being considered in many IETF WGs. I haven't > heard this level of speculation in other WGs. You are describing > things that do serious violence to the very _basis_ of TCP; and > you seem to be suggesting that we should avoid any changes to TCP > that might be broken by broken middleboxes. I dissent. If by "the very _basis_ of TCP" you refer to TCP's end-to-end principle, then middleboxes already do serious violence to that. We can define DPI middleboxes as broken, but that's just denial of reality: they're here, they're widely deployed, and we have to live with them. >> ... But given all the innumerable types of weird behavior we know >> middleboxes to exhibit, this strikes me as wishful thinking. At the >> very least, someone would need to do a fairly large-scale test of the >> behavior of TCP with unknown options over a wide variety of deployed >> NATs, before we could have much confidence that this approach to an >> MPTCP would be reliable. > > ... which I believe is the intent of our charter: making an > Experimental protocol. Granted. Bryan From janardhan.iyengar@fandm.edu Sun May 17 11:38:07 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 00EA928C1DA for ; Sun, 17 May 2009 11:38:07 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.855 X-Spam-Level: X-Spam-Status: No, score=-1.855 tagged_above=-999 required=5 tests=[AWL=-0.745, BAYES_05=-1.11] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tSMtTEquQjE4 for ; Sun, 17 May 2009 11:37:59 -0700 (PDT) Received: from zimfe2.fandm.edu (zimfe2.fandm.edu [155.68.1.75]) by core3.amsl.com (Postfix) with ESMTP id 23EC128C0EC for ; Sun, 17 May 2009 11:37:56 -0700 (PDT) Received: from surutti.fandm.edu (pool-71-173-164-155.hrbgpa.east.verizon.net [71.173.164.155]) by zimfe2.fandm.edu (Postfix) with ESMTP id 07D24116801B; Sun, 17 May 2009 14:39:30 -0400 (EDT) Message-ID: <4A1059E2.6090403@fandm.edu> Date: Sun, 17 May 2009 14:39:30 -0400 From: Janardhan Iyengar Organization: Franklin & Marshall College User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: John Leslie References: <001e01c9cfff$bde8f1e0$103947ab@cisco.com> <97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com> <001f01c9d005$0c583110$103947ab@cisco.com> <1241818843.26211.67.camel@nhuynhth-laptop> <0D6E7BC4-17AF-4F5C-9107-ABA2B729683F@muada.com> <20090509184241.GT32848@verdi> <4A0C8E89.9080000@fandm.edu> <20090514231745.GB70521@verdi> In-Reply-To: <20090514231745.GB70521@verdi> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] TCP/SCTP and BoF description, was: Re: TCP modification vs switching to SCTP X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: janardhan.iyengar@fandm.edu List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 May 2009 18:38:07 -0000 Hi John, >> Again, I think that this BoF should be broadly scoped to allow for >> multiple parallel/competing architectures. We should be cautious >> not to scope the group too narrowly yet. > > Is that first Charter item too narrow? No, but the second one is: "In particular, the proposed work is to extend TCP to support simultaneous data exchange through multiple paths." I think the charter at this time should be about multipath transport, and while while TCP can be one of the transports, it should be just that. TCP should not be the end-all for this group. >> I sincerely think that there are shared questions here that cut across >> transports, and there will be significant differences. Architecturally, >> this conversation is exactly where we should discuss where multipath >> belongs on the stack. Why not have a common flow layer for *all* >> transports that does multipath? > > I don't follow how we'd get there. Implementations already pretty > thoroughly obscure "layer" boundaries. > > While I am very sympathetic to the idea of Transport layers that know > how to do multipath, I am not convinced that "one true way" is even a > desirable goal. I am talking here about transport layers shipping multipath out of the transport, and leaving it to a different entity; let's call this entity the "flow" layer. But as I've said, we don't have to agree yet to doing this; I think it is a powerful idea, and I think the group should allow space for discussion of it. > (Seriously, I don't think we're any more likely to converge on "one > true way" that other groups are to converge on a single "congestion > control" algorithm.) There may be properties that an application desires from its transport such as low jitter or high throughput, and these properties are important when determining a congestion control algorithm. Similarly, we will need to figure out how different multipath techniques can work given the different demands that applications are likely to have. Why shouldn't we discuss these needs, and work on multipath techniques for protocols such as DCCP that has non-Reno congestion control, if there is interest in it? We don't have to arrive at the one "true" way of doing multipath for all applications---I agree with you that there is probably no one way---but we should surely be discussing these different application needs, and ways of doing (or not doing) multipath for them. My larger point is that we need to discuss other transports and not only TCP. >> Incidentally, this is how I remember the "Resource-Pooling" paper >> as being scoped. > > We're talking "engineering" here, not academic papers. We tend to > avoid areas where we need to debate what would be "best." This paper is a reference from the BoF description: http://trac.tools.ietf.org/area/tsv/trac/wiki/MpTcpBofDescription I could be wrong, but my understanding is that that paper has a strong influence on this BoF and the two drafts that were circulated. Engineering doesn't mean that we stop debating and simply code the first draft that is circulated. If this discussion means that this project belongs in the IRTF and not in the IETF yet, so be it. Better that than to have a solution where coding can happen immediately but without sufficient discussion. > I'm very happy to see interest by academic types in this subject. > But unless there's a place for engineers to dive in and code something, > I fear the project would drag on forever. :^( I won't blame academic-types for that. Academics aren't the ones causing heartburn on all dragged-out projects in the IETF. - jana -- Janardhan Iyengar Assistant Professor, Computer Science Franklin & Marshall College http://www.fandm.edu/jiyengar From janardhan.iyengar@fandm.edu Sun May 17 13:04:41 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 352243A6A9D for ; Sun, 17 May 2009 13:04:41 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.45 X-Spam-Level: X-Spam-Status: No, score=-2.45 tagged_above=-999 required=5 tests=[AWL=0.149, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OsPxj8Elf8Ci for ; Sun, 17 May 2009 13:04:40 -0700 (PDT) Received: from zimfe2.fandm.edu (zimfe2.fandm.edu [155.68.1.75]) by core3.amsl.com (Postfix) with ESMTP id 1B18F3A68DA for ; Sun, 17 May 2009 13:04:39 -0700 (PDT) Received: from surutti.fandm.edu (pool-71-173-164-155.hrbgpa.east.verizon.net [71.173.164.155]) by zimfe2.fandm.edu (Postfix) with ESMTP id 71E4B116801B; Sun, 17 May 2009 16:06:13 -0400 (EDT) Message-ID: <4A106E2F.2070200@fandm.edu> Date: Sun, 17 May 2009 16:06:07 -0400 From: Janardhan Iyengar Organization: Franklin & Marshall College User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Bryan Ford References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> In-Reply-To: <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: janardhan.iyengar@fandm.edu List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 May 2009 20:04:41 -0000 Bryan, > OK, I see where you're going. If you care about getting through DPI > firewalls/NATs semi-reliably, though, I strongly suspect you'll have to > scrap the resync packet mechanism described in section 4.4.3, as it > seems extremely likely that many firewalls/NATs will balk in some way if > you try to leave what looks to them like a hole in the TCP sequence > space (which is what the resync mechanism will do). Instead I see no > reliable alternative but to make each subflow individually faithful to > TCP semantics, including retransmitting any data that was previously > transmitted on that subflow and lost - i.e., retransmitting the data _on > the same subflow_ even if the data has since been successfully > transmitted and ACKed on a different subflow. Thus, shifting previously > transmitted data among subflows would imply some semantically > unnecessary data retransmissions, but this shouldn't be a performance > problem as long as it doesn't happen too often, and it presumably > shouldn't need to happen often if the transmit scheduler is designed > with a bit of care. There is another way to deal with holes in the sequence space---divorce the data that is being sent from the associated sub-flow sequence number. Data needs to be associated with only Data-Sequence-Numbers. Thus what appears to be a retransmission at the flow level can in reality be a new data transmission, and the actual retransmission could have been transmitted over a different subflow with a new subflow sequence number. This does need more changes to the TCP state machine, but it is an alternative that has been explored before in pTCP (http://www.ece.gatech.edu/research/GNAN/work/ptcp/ptcp.html) - jana -- Janardhan Iyengar Assistant Professor, Computer Science Franklin & Marshall College http://www.fandm.edu/jiyengar From janardhan.iyengar@fandm.edu Sun May 17 13:16:37 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 7F3B53A6EFE for ; Sun, 17 May 2009 13:16:37 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.475 X-Spam-Level: X-Spam-Status: No, score=-2.475 tagged_above=-999 required=5 tests=[AWL=0.124, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ksBnEcqAHq2F for ; Sun, 17 May 2009 13:16:31 -0700 (PDT) Received: from zimfe2.fandm.edu (zimfe2.fandm.edu [155.68.1.75]) by core3.amsl.com (Postfix) with ESMTP id 45CD63A6EE2 for ; Sun, 17 May 2009 13:16:31 -0700 (PDT) Received: from surutti.fandm.edu (pool-71-173-164-155.hrbgpa.east.verizon.net [71.173.164.155]) by zimfe2.fandm.edu (Postfix) with ESMTP id E824C116801B; Sun, 17 May 2009 16:18:05 -0400 (EDT) Message-ID: <4A1070FC.8040909@fandm.edu> Date: Sun, 17 May 2009 16:18:04 -0400 From: Janardhan Iyengar Organization: Franklin & Marshall College User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Costin Raiciu References: <4A0CA1D0.4040802@fe.up.pt> <61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <5C4FB334-B163-49C4-BC94-775247962B34@cs.ucl.ac.uk> In-Reply-To: <5C4FB334-B163-49C4-BC94-775247962B34@cs.ucl.ac.uk> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: janardhan.iyengar@fandm.edu List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 May 2009 20:16:37 -0000 > 3. sequence numbers > The two-ended multipath has subflows which look over the wire as normal > tcp connections, i.e. with contiguous sequence spaces, so they will pass > through boxes like normalizers. The single ended one or CMT wont. Its > unclear how many normalizers are deployed; Mark had understood from > discussions with guys from the industry that many people use them (how > many, don't know). This is true: CMT will not appear contiguous in sequence space to middleboxes. Are "normalizers" middleboxes that reconstruct flows? > Finally, the one-ended one, and cmt for that matter, have the following > issue: when a packet is lost and retransmitted on a different path, > there is no way to know which of the paths delivered the packet unless > one adds "options" to the packets. This is a more serious problem than > not knowing whether the original or retransmitted packet got there in > traditional tcp, as it places the host in the impossibility of deciding > whether the path that carried the retransmitted packet has lost or > delivered the packet. I don't understand why this retransmission ambiguity is any worse with CMT than with regular TCP. TCP also tries to determine whether "the path that carried the retransmitted packet has lost or delivered the packet", TCP just happens to use only one path on which it tries to determine if a loss occurred. This is no different in CMT, and the same disambiguation methods that are employed for TCP can be used for CMT. - jana -- Janardhan Iyengar Assistant Professor, Computer Science Franklin & Marshall College http://www.fandm.edu/jiyengar From iljitsch@muada.com Sun May 17 13:53:19 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id E90CF3A6D3A for ; Sun, 17 May 2009 13:53:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2uU7GnuO8qNH for ; Sun, 17 May 2009 13:53:13 -0700 (PDT) Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by core3.amsl.com (Postfix) with ESMTP id BE1FE28C1D3 for ; Sun, 17 May 2009 13:53:12 -0700 (PDT) Received: from [192.168.2.4] (static-167-138-7-89.ipcom.comunitel.net [89.7.138.167] (may be forged)) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n4HKs7dI000235 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Sun, 17 May 2009 22:54:08 +0200 (CEST) (envelope-from iljitsch@muada.com) Message-Id: <87568DCC-90C4-4AF3-9550-AF8742F6E48B@muada.com> From: Iljitsch van Beijnum To: janardhan.iyengar@fandm.edu In-Reply-To: <4A1059E2.6090403@fandm.edu> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v935.3) Date: Sun, 17 May 2009 22:54:15 +0200 References: <001e01c9cfff$bde8f1e0$103947ab@cisco.com> <97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com> <001f01c9d005$0c583110$103947ab@cisco.com> <1241818843.26211.67.camel@nhuynhth-laptop> <0D6E7BC4-17AF-4F5C-9107-ABA2B729683F@muada.com> <20090509184241.GT32848@verdi> <4A0C8E89.9080000@fandm.edu> <20090514231745.GB70521@verdi> <4A1059E2.6090403@fandm.edu> X-Mailer: Apple Mail (2.935.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] TCP/SCTP and BoF description, was: Re: TCP modification vs switching to SCTP X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 May 2009 20:53:20 -0000 On 17 mei 2009, at 20:39, Janardhan Iyengar wrote: > I think the charter at this time should be about multipath > transport, and while while TCP can be one of the transports, it > should be just that. TCP should not be the end-all for this group. So how would that work? Do we, for each TCP mechanism, specify the corresponding SCTP mechanism? And then the same for DCCP and RTP? Or do we stop just shy of specifying specific implementation details? > I think the group should allow space for discussion of it. I don't think anyone is trying to stop any discussions here. However, there is a difference between having a discussion and a work item in the charter. > My larger point is that we need to discuss other transports and not > only TCP. What is it that you want to discuss? Nobody is stopping you. :-) From janardhan.iyengar@fandm.edu Sun May 17 14:27:26 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id E37993A6C23 for ; Sun, 17 May 2009 14:27:26 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.493 X-Spam-Level: X-Spam-Status: No, score=-2.493 tagged_above=-999 required=5 tests=[AWL=0.106, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ElJfQFrqr9Qq for ; Sun, 17 May 2009 14:27:26 -0700 (PDT) Received: from zimfe2.fandm.edu (zimfe2.fandm.edu [155.68.1.75]) by core3.amsl.com (Postfix) with ESMTP id 0E2F73A69C9 for ; Sun, 17 May 2009 14:27:25 -0700 (PDT) Received: from surutti.fandm.edu (pool-71-173-164-155.hrbgpa.east.verizon.net [71.173.164.155]) by zimfe2.fandm.edu (Postfix) with ESMTP id 77E55116801B; Sun, 17 May 2009 17:29:00 -0400 (EDT) Message-ID: <4A10819B.2020306@fandm.edu> Date: Sun, 17 May 2009 17:28:59 -0400 From: Janardhan Iyengar Organization: Franklin & Marshall College User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Iljitsch van Beijnum References: <001e01c9cfff$bde8f1e0$103947ab@cisco.com> <97963A5D-FD5D-4D08-99F6-12455BB90FE4@muada.com> <001f01c9d005$0c583110$103947ab@cisco.com> <1241818843.26211.67.camel@nhuynhth-laptop> <0D6E7BC4-17AF-4F5C-9107-ABA2B729683F@muada.com> <20090509184241.GT32848@verdi> <4A0C8E89.9080000@fandm.edu> <20090514231745.GB70521@verdi> <4A1059E2.6090403@fandm.edu> <87568DCC-90C4-4AF3-9550-AF8742F6E48B@muada.com> In-Reply-To: <87568DCC-90C4-4AF3-9550-AF8742F6E48B@muada.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] TCP/SCTP and BoF description, was: Re: TCP modification vs switching to SCTP X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: janardhan.iyengar@fandm.edu List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 May 2009 21:27:27 -0000 > I don't think anyone is trying to stop any discussions here. However, I wasn't trying to say that... apologies if I came across that way. > there is a difference between having a discussion and a work item in the > charter. Good point. I was not making a strong point; only that if there are interested parties that want to work on SCTP and/or DCCP, then that should happen here. I heard someone (was it Pasi?) say that there may be a draft for DCCP. I will try to put together CMT experiences in a draft too. - jana -- Janardhan Iyengar Assistant Professor, Computer Science Franklin & Marshall College http://www.fandm.edu/jiyengar From c.raiciu@cs.ucl.ac.uk Sun May 17 14:38:23 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 95A113A6DF3 for ; Sun, 17 May 2009 14:38:23 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XcMHtsxE7w4m for ; Sun, 17 May 2009 14:38:16 -0700 (PDT) Received: from bells2.cs.ucl.ac.uk (bells2.cs.ucl.ac.uk [128.16.5.33]) by core3.amsl.com (Postfix) with ESMTP id AF6EC3A6BFD for ; Sun, 17 May 2009 14:38:16 -0700 (PDT) Received: from [82.45.130.212] (helo=[172.24.121.185]) by bells2.cs.ucl.ac.uk with esmtpsa (TLSv1:AES128-SHA:128) (C.Raiciu authenticated) (Exim 4.54) id 1M5oZ0-0009YJ-CL for multipathtcp@ietf.org; Sun, 17 May 2009 23:10:35 +0100 Mime-Version: 1.0 (Apple Message framework v753.1) In-Reply-To: <4A1070FC.8040909@fandm.edu> References: <4A0CA1D0.4040802@fe.up.pt> <61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <5C4FB334-B163-49C4-BC94-775247962B34@cs.ucl.ac.uk> <4A1070FC.8040909@fandm.edu> Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Costin Raiciu Date: Sun, 17 May 2009 22:39:37 +0100 To: multipathtcp@ietf.org X-Mailer: Apple Mail (2.753.1) Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 May 2009 21:38:23 -0000 On 17 May 2009, at 21:18, Janardhan Iyengar wrote: > >> 3. sequence numbers >> The two-ended multipath has subflows which look over the wire as >> normal tcp connections, i.e. with contiguous sequence spaces, so >> they will pass through boxes like normalizers. The single ended >> one or CMT wont. Its unclear how many normalizers are deployed; >> Mark had understood from discussions with guys from the industry >> that many people use them (how many, don't know). > > This is true: CMT will not appear contiguous in sequence space to > middleboxes. Are "normalizers" middleboxes that reconstruct flows? Yes. > >> Finally, the one-ended one, and cmt for that matter, have the >> following issue: when a packet is lost and retransmitted on a >> different path, there is no way to know which of the paths >> delivered the packet unless one adds "options" to the packets. >> This is a more serious problem than not knowing whether the >> original or retransmitted packet got there in traditional tcp, as >> it places the host in the impossibility of deciding whether the >> path that carried the retransmitted packet has lost or delivered >> the packet. > > I don't understand why this retransmission ambiguity is any worse > with CMT than with regular TCP. TCP also tries to determine > whether "the path that carried the retransmitted packet has lost or > delivered the packet", TCP just happens to use only one path on > which it tries to determine if a loss occurred. This is no > different in CMT, and the same disambiguation methods that are > employed for TCP can be used for CMT. It's not exactly the same thing. When you retransmit on the same path, when you get the ack you know this path delivered one the packets for sure, so its working. This is not the case in multipath: say you have a single packet, it gets lost on one path, you send it down the other, and you get an ack. Which path delivered it? It can be that the second path is dead, and the first path had a (really bad) hickup, so assuming the second path delivered the packet is not necessarily correct. Determining which path works is important to schedule the following packets, otherwise the whole connection can just grind to a halt. Its important to deterministically know which path delivered the packet. In the single seq space design, if one doesn't add extra options (or the like), the only way to achieve this is restricting retransmissions to be on the same flow. Cheers, Costin From iljitsch@muada.com Mon May 18 04:28:08 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 47BAF28C16D for ; Mon, 18 May 2009 04:28:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QqEC30dGI0Bu for ; Mon, 18 May 2009 04:28:07 -0700 (PDT) Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by core3.amsl.com (Postfix) with ESMTP id 230553A6D50 for ; Mon, 18 May 2009 04:28:06 -0700 (PDT) Received: from claw.it.uc3m.es (claw.it.uc3m.es [163.117.139.52]) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n4IBStVt004707 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Mon, 18 May 2009 13:28:58 +0200 (CEST) (envelope-from iljitsch@muada.com) Message-Id: <7BE7F3C3-2CD8-4C86-B4A1-4748205B30BE@muada.com> From: Iljitsch van Beijnum To: Bryan Ford In-Reply-To: Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v935.3) Date: Mon, 18 May 2009 13:29:03 +0200 References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> X-Mailer: Apple Mail (2.935.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 May 2009 11:28:08 -0000 On 17 mei 2009, at 19:20, Bryan Ford wrote: >> In fact, I could do that for all packets. The first n bytes >> following the TCP header could be the second sequence number. > This approach sounds more promising to me in general. Then MPTCP > starts to take more and more of the flavor of a new transport > protocol layered atop an existing protocol (namely legacy single- > path TCP), taking advantage of legacy TCP to provide per-flow > congestion control and get through middleboxes. Right. The multiaddress multipath TCP as outlined in the two-ended draft already seems a bit like TCP-over-TCP. > Even with IPSEC's recent UDP encapsulation enhancements to get > through NATs and firewalls, it remains incompatible with PEPs or TCP > proxies that need to split or monitor TCP's congestion control loop, Isn't that a feature? > not to mention middleboxes that let through only TCP and not UDP. > But taking the view of legacy TCP as a Flow Layer when used by > MPTCP, the right place to deploy IPSEC is actually _above_ legacy > TCP but below MPTCP's inter-flow transport layer. Isnt't that basically what SSL/TLS does today? The main advantage of IPsec over TLS is that IPsec also protects against attacks against TCP, IPsec-over-TCP wouldn't have that protection for the outer TCP session. However, the multipath layer would be able to recover from successful attacks so this protection wouldn't be as crucial as in the single path case (unless all paths are being attacked). So TLS may suffice. An interesting approach would be TCP over TLS over TCP where the TLS protects against the issues with middleboxes rewriting data that we talked about. To do this well, it would be helpful to send each TLS frame/packet (not sure what the right terminology is) over the same path. Iljitsch From alan.ford@roke.co.uk Mon May 18 08:31:59 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 26A963A6BA9 for ; Mon, 18 May 2009 08:31:59 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.599 X-Spam-Level: X-Spam-Status: No, score=-3.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id f7O45+GDlsA2 for ; Mon, 18 May 2009 08:31:58 -0700 (PDT) Received: from rsys001x.roke.co.uk (rsys001x.roke.co.uk [193.118.201.108]) by core3.amsl.com (Postfix) with ESMTP id 2190B28C13A for ; Mon, 18 May 2009 08:31:18 -0700 (PDT) Received: from rsys005a.comm.ad.roke.co.uk ([193.118.193.85]) by rsys001x.roke.co.uk (8.13.1/8.13.1) with ESMTP id n4IFWjOh017271; Mon, 18 May 2009 16:32:45 +0100 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Mon, 18 May 2009 16:32:44 +0100 Message-ID: <2181C5F19DD0254692452BFF3EAF1D6807BFD985@rsys005a.comm.ad.roke.co.uk> In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [multipathtcp] MPCC and different transports Thread-Index: AcnXE8/OzHLMVwFJQnqZu5dX/d6nHgAtDQSA References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> From: "Ford, Alan" To: "Bryan Ford" , "Iljitsch van Beijnum" X-roke-co-uk-MailScanner: Found to be clean X-roke-co-uk-MailScanner-SpamCheck: X-roke-co-uk-MailScanner-From: alan.ford@roke.co.uk Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 May 2009 15:31:59 -0000 Bryan Ford wrote: > On May 16, 2009, at 3:22 PM, Iljitsch van Beijnum wrote: > > On 16 mei 2009, at 6:50, Bryan Ford wrote: > > > >> That still leaves all the issues with the new TCP options. > > > > A discussion that a bunch of us had offline: would it be useful to > > exchange signaling information in-band at the beginning of a subflow? > > > > I.e., I set up something that looks like a new TCP session to > > middleboxes, but in reality it's an additional subflow for an > > existing session. Rather than negotiate all the housekeeping in > > options, I simply do that in-band. When the negotiation is finished, > > I start exchanging data. > > > > In fact, I could do that for all packets. The first n bytes > > following the TCP header could be the second sequence number. > > > > Obviously deep packet inspection boxes wouldn't like that, but if we > > can't do options, we must maintain the sequence numbering and we > > can't touch the data then maybe using that path for multipath isn't > > such a good idea after all. >=20 > This approach sounds more promising to me in general. Then MPTCP > starts to take more and more of the flavor of a new transport protocol > layered atop an existing protocol (namely legacy single-path TCP), > taking advantage of legacy TCP to provide per-flow congestion control > and get through middleboxes. =20 However, simply putting data at the start of a packet still relies on middleboxes not to mess with the segmentation. It would be nice to get a solution where it doesn't matter if this happens. Which makes me wonder whether a TLS-like solution, where all data (both application and control) is carried in the TCP stream, and data is chunked and preceded by a header of type, length, and data sequence number (if appropriate). We could then switch between control data and application data with no major issues, and we still get the benefits of data sequence numbering without TCP Options. I haven't thought this through yet, so it may be riddled with holes, so fire away...! > This approach would also have the nice benefit of effectively > restoring the long-lost "end-to-endness" of TCP's originally intended > semantics, from the perspective of the application, even when many or > all of the individual legacy TCP flows atop which an MPTCP connection > is built are being split in the middle by stateful firewalls, NATs, > and/or PEPs. =20 I'm not sure if that really adds anything over and above what we already have. Although the MPTCP layer would be (at the moment - who knows what NATs may do in the future!) end-to-end, its primary purpose is to reassemble data spread across multiple paths, and there is little else in the protocol at that level (of course, this could be a convenient place for future extensions, but that's not our concern). Data as delivered to the application would be pretty much the same as today. > For example, if a stateful middlebox crashes and loses > its legacy TCP state, the legacy TCP connection is hosed, but the > MPTCP connection might well be able to notice the TCP flow's > connection reset and just retry establishing that flow, while shifting > traffic to other flows in the meantime. Indeed, that's one particular use case for MPTCP, and should work in either of the proposals put forward so far. > Taking this approach further, have you thought about how MPTCP should/ > might interact with IPSEC? Of course IPSEC can always be deployed > above IP and below legacy TCP, as usual, but we know how well IPSEC > tends to get along with middleboxes. Combining traditional IPSEC with > MPTCP would effectively defeat all the care MPTCP went to to provide > backward compatibility by "looking like" legacy TCP. Even with > IPSEC's recent UDP encapsulation enhancements to get through NATs and > firewalls, it remains incompatible with PEPs or TCP proxies that need > to split or monitor TCP's congestion control loop, not to mention > middleboxes that let through only TCP and not UDP. But taking the > view of legacy TCP as a Flow Layer when used by MPTCP, the right place > to deploy IPSEC is actually _above_ legacy TCP but below MPTCP's inter- > flow transport layer. This way, "IPSEC-secured MPTCP" would have all > the same backward compatibility advantages of unsecured MPTCP because > the Flow Layer part looks just like ordinary TCP, but the IPSEC layer > in the middle would then be able to enforce the true "end-to-endness" > of the MPSEC transport layer by preventing middleboxes from > interfering with or even seeing the MPSEC-level data transmission loop. An interesting thought, but would IPSec-over-MPTCP be any more appropriate than using TLS? (Aside from being transparent to the application, of course). Assuming paths with no NATs and thus IPSec end-to-end, it should be feasible to run MPTCP over TCP over IPSec with no issues. But I must admit I haven't gone through this step-by-step to make sure! Cheers, Alan From janardhan.iyengar@fandm.edu Mon May 18 12:20:04 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 856C43A681A for ; Mon, 18 May 2009 12:20:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id e5wUi0iM1jHb for ; Mon, 18 May 2009 12:20:01 -0700 (PDT) Received: from zimfe2.fandm.edu (zimfe2.fandm.edu [155.68.1.75]) by core3.amsl.com (Postfix) with ESMTP id DC8A73A6CEE for ; Mon, 18 May 2009 12:20:00 -0700 (PDT) Received: from surutti.fandm.edu (dhcp-155-68-61-189.fandm.edu [155.68.61.189]) by zimfe2.fandm.edu (Postfix) with ESMTP id E6DDA116802F; Mon, 18 May 2009 15:21:34 -0400 (EDT) Message-ID: <4A11B53E.9050900@fandm.edu> Date: Mon, 18 May 2009 15:21:34 -0400 From: Janardhan Iyengar Organization: Franklin & Marshall College User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: "Ford, Alan" References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807BFD985@rsys005a.comm.ad.roke.co.uk> In-Reply-To: <2181C5F19DD0254692452BFF3EAF1D6807BFD985@rsys005a.comm.ad.roke.co.uk> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: janardhan.iyengar@fandm.edu List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 May 2009 19:20:05 -0000 > Which makes me wonder whether a TLS-like solution, where all data (both > application and control) is carried in the TCP stream, and data is > chunked and preceded by a header of type, length, and data sequence > number (if appropriate). We could then switch between control data and > application data with no major issues, and we still get the benefits of > data sequence numbering without TCP Options. I haven't thought this > through yet, so it may be riddled with holes, so fire away...! This does sound interesting... if you could define specific "control" chunks, then you could get rid of almost all of the TCP options, except for negotiation of MPTCP. It is better than having bytes at the head of segments that can be lost during segmentation in the middle. I'll note that the TCP stream now looks even more like an SCTP association (data and control chunks in SCTP are exactly chunks in the SCTP association with TLV headers and sequence numbers where appropriate.) - jana -- Janardhan Iyengar Assistant Professor, Computer Science Franklin & Marshall College http://www.fandm.edu/jiyengar From baford@mpi-sws.org Tue May 19 00:34:44 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id D29A928C18F for ; Tue, 19 May 2009 00:34:44 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.249 X-Spam-Level: X-Spam-Status: No, score=-6.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6bP8JnVXj9J9 for ; Tue, 19 May 2009 00:34:43 -0700 (PDT) Received: from hera.mpi-sb.mpg.de (hera.mpi-sb.mpg.de [139.19.1.49]) by core3.amsl.com (Postfix) with ESMTP id 5E61828C17B for ; Tue, 19 May 2009 00:34:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mpi-sb.mpg.de; s=mail200803; h=Cc:Message-Id:From:To: In-Reply-To:Content-Type:Content-Transfer-Encoding:Mime-Version: Subject:Date:References; bh=YJCTZO16VhYNoDagfStWI1qJrCdTC2EhaWr9 5hzjBG8=; b=wLJv9DYmNwUYlvhIzrA9aFNFBHTYcreZvezq2FK4WibE1S1Q7Gti Ie1UB3/RxSVCbPf4eAKM12yvDvJA+2Q2HsUzfvnWuh2ect7cxA10VneGQjRthnNV rWvyFEmszxktgciT7ISU2lpqAzRwZJOsFRP4Q3uxOUA6r2vXilgEnwY= Received: from newzak.mpi-sb.mpg.de ([139.19.1.28]:54806) by hera.mpi-sb.mpg.de (envelope-from ) with esmtp (Exim 4.69) id 1M6Jrv-0006SX-Rt; Tue, 19 May 2009 09:36:16 +0200 Received: from g226213088.adsl.alicedsl.de ([92.226.213.88]:55901) by newzak.mpi-sb.mpg.de with esmtpsa (TLS-1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.63) (envelope-from ) id 1M6Jrv-0003J4-Ad; Tue, 19 May 2009 09:36:11 +0200 Message-Id: <3628F6C3-E9F7-4AF3-91AD-44B6C82D329E@mpi-sws.org> From: Bryan Ford To: janardhan.iyengar@fandm.edu In-Reply-To: <4A106E2F.2070200@fandm.edu> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Tue, 19 May 2009 09:36:10 +0200 References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> <4A106E2F.2070200@fandm.edu> X-Mailer: Apple Mail (2.930.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 May 2009 07:34:44 -0000 On May 17, 2009, at 10:06 PM, Janardhan Iyengar wrote: > Bryan, > >> OK, I see where you're going. If you care about getting through >> DPI firewalls/NATs semi-reliably, though, I strongly suspect you'll >> have to scrap the resync packet mechanism described in section >> 4.4.3, as it seems extremely likely that many firewalls/NATs will >> balk in some way if you try to leave what looks to them like a hole >> in the TCP sequence space (which is what the resync mechanism will >> do). Instead I see no reliable alternative but to make each >> subflow individually faithful to TCP semantics, including >> retransmitting any data that was previously transmitted on that >> subflow and lost - i.e., retransmitting the data _on the same >> subflow_ even if the data has since been successfully transmitted >> and ACKed on a different subflow. Thus, shifting previously >> transmitted data among subflows would imply some semantically >> unnecessary data retransmissions, but this shouldn't be a >> performance problem as long as it doesn't happen too often, and it >> presumably shouldn't need to happen often if the transmit scheduler >> is designed with a bit of care. > > There is another way to deal with holes in the sequence space--- > divorce the data that is being sent from the associated sub-flow > sequence number. Data needs to be associated with only Data-Sequence- > Numbers. Thus what appears to be a retransmission at the flow level > can in reality be a new data transmission, and the actual > retransmission could have been transmitted over a different subflow > with a new subflow sequence number. > > This does need more changes to the TCP state machine, but it is an > alternative that has been explored before in pTCP (http://www.ece.gatech.edu/research/GNAN/work/ptcp/ptcp.html > ) A clever trick, but it seems this approach would still be subject to corruption by over-intelligent middleboxes (I guess we're calling them "normalizers"?)... Bryan > > - jana > > -- > Janardhan Iyengar > Assistant Professor, Computer Science > Franklin & Marshall College > http://www.fandm.edu/jiyengar From baford@mpi-sws.org Tue May 19 01:14:56 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 0292A28C107 for ; Tue, 19 May 2009 01:14:56 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.249 X-Spam-Level: X-Spam-Status: No, score=-6.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kg9vHw-dpLQJ for ; Tue, 19 May 2009 01:14:50 -0700 (PDT) Received: from hera.mpi-sb.mpg.de (infao0809.mpi-sb.mpg.de [139.19.1.49]) by core3.amsl.com (Postfix) with ESMTP id 064FF28C322 for ; Tue, 19 May 2009 01:14:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mpi-sb.mpg.de; s=mail200803; h=Cc:Message-Id:From:To: In-Reply-To:Content-Type:Content-Transfer-Encoding:Mime-Version: Subject:Date:References; bh=T55CBfCgNPLQPnisf9jIo5kyY9FbDTBjh/WB 3zt/Y4Q=; b=Aba47z2NWPt11SqpOUGTd8ZPn4gg3Qdc8XH+O08/LZvE4ywkeiFC Luf0VTAwSzBx1umm1OE+C94ASx7+5CKHSDb1uKJEtovCrlvqpQeqPiS8UysGe3zZ MBUO3YmzNanAk7XAmxJzu0+HkQUiScYZtECaBlSTTph4kr7MIjPix3Y= Received: from infao0710.mpi-sb.mpg.de ([139.19.1.28]:40779 helo=newzak.mpi-sb.mpg.de) by hera.mpi-sb.mpg.de (envelope-from ) with esmtp (Exim 4.69) id 1M6KUn-0000QV-5e; Tue, 19 May 2009 10:16:24 +0200 Received: from g226213088.adsl.alicedsl.de ([92.226.213.88]:55927) by newzak.mpi-sb.mpg.de with esmtpsa (TLS-1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.63) (envelope-from ) id 1M6KUm-0003ZR-Ly; Tue, 19 May 2009 10:16:20 +0200 Message-Id: <8A52D0F9-2C40-497D-84F5-13CC0575BF97@mpi-sws.org> From: Bryan Ford To: Iljitsch van Beijnum In-Reply-To: <7BE7F3C3-2CD8-4C86-B4A1-4748205B30BE@muada.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Tue, 19 May 2009 10:16:19 +0200 References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> <7BE7F3C3-2CD8-4C86-B4A1-4748205B30BE@muada.com> X-Mailer: Apple Mail (2.930.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 May 2009 08:14:56 -0000 On May 18, 2009, at 1:29 PM, Iljitsch van Beijnum wrote: > On 17 mei 2009, at 19:20, Bryan Ford wrote: > >>> In fact, I could do that for all packets. The first n bytes >>> following the TCP header could be the second sequence number. > >> This approach sounds more promising to me in general. Then MPTCP >> starts to take more and more of the flavor of a new transport >> protocol layered atop an existing protocol (namely legacy single- >> path TCP), taking advantage of legacy TCP to provide per-flow >> congestion control and get through middleboxes. > > Right. The multiaddress multipath TCP as outlined in the two-ended > draft already seems a bit like TCP-over-TCP. > >> Even with IPSEC's recent UDP encapsulation enhancements to get >> through NATs and firewalls, it remains incompatible with PEPs or >> TCP proxies that need to split or monitor TCP's congestion control >> loop, > > Isn't that a feature? You mean in that IPSEC effectively enforces the "end-to-endness" of TCP? The fact that IPSEC prevents devices in the middle of the network from interfering with TCP's end-to-end _reliability_ and data integrity is indeed a feature. The fact that IPSEC also prevents devices in the network from implementing reasonable performance optimizations as flows cross specific network technologies or specific administrative domains is not a feature. The end-to-end principle is about reliability, not performance; it has always been about reliability and not performance; the E2E paper even explicitly pushes the point that in-network mechanisms can be useful and valuable to enhance performance (they were talking about in-network retransmission in particular) but just aren't a substitute for end-to-end reliability mechanisms. I even asked Dave Clark about his perspective on this when I ran into him a few weeks ago, and he confirmed this interpretation of the E2E paper. The idea that the end-to-end principle might or should also apply to performance-related mechanisms such as congestion control came much later and IMO is a complete misapplication of the principle. There are reasonable technical arguments for (and against) end-to-end congestion control versus sectioned or hop-by-hop CC, of course, but those arguments have nothing to do with the end-to-end principle as proposed in the original E2E paper. >> not to mention middleboxes that let through only TCP and not UDP. >> But taking the view of legacy TCP as a Flow Layer when used by >> MPTCP, the right place to deploy IPSEC is actually _above_ legacy >> TCP but below MPTCP's inter-flow transport layer. > > Isnt't that basically what SSL/TLS does today? Yes, with the additional difference (unimportant prorocol-wise but very important in practical use) that IPSEC is generally deployed and administered at OS level whereas SSL is deployed and administered by individual applications. SSL also needs to be modified/redesigned for each transport (e.g. DTLS for UDP) whereas IPSEC is in theory transport-neutral (although the recent NAT traversal enhancements defeat this advantage). The general point I was trying to make was that if we want IPSEC-style, application-transparent, transport- neutral security in some form, the best place for it to happen is above the Flow Layer (congestion control, PEPs, etc.) but below whatever protocol provides the application-visible transport semantics (reliability, ordering, etc.) That said, you make an excellent point - since SSL over TCP is already ubiquitous and the convention of deploying SSL at application rather than system level is merely a convention and by no means essential to the protocol... If we wanted to provide system-level, application- transparent security for MPTCP (or CMT-SCTP or SST or...) while providing as much backward compatibility as possible, maybe the right way to do that is to layer MPTCP over SSL over TCP. That way secured MPTCP flows would look to the network (and to middleboxes in the network) indistinguishable from, say, ordinary HTTPS flows, which may be a really good thing. > The main advantage of IPsec over TLS is that IPsec also protects > against attacks against TCP, IPsec-over-TCP wouldn't have that > protection for the outer TCP session. However, the multipath layer > would be able to recover from successful attacks so this protection > wouldn't be as crucial as in the single path case (unless all paths > are being attacked). So TLS may suffice. Yes, very true. > An interesting approach would be TCP over TLS over TCP where the TLS > protects against the issues with middleboxes rewriting data that we > talked about. To do this well, it would be helpful to send each TLS > frame/packet (not sure what the right terminology is) over the same > path. I think we're thinking along the same lines here. Bryan From baford@mpi-sws.org Tue May 19 01:22:37 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id E791D3A6A58 for ; Tue, 19 May 2009 01:22:37 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.249 X-Spam-Level: X-Spam-Status: No, score=-6.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tKM-SKbxzvEv for ; Tue, 19 May 2009 01:22:36 -0700 (PDT) Received: from hera.mpi-sb.mpg.de (hera.mpi-sb.mpg.de [139.19.1.49]) by core3.amsl.com (Postfix) with ESMTP id 5F6DD3A6D6A for ; Tue, 19 May 2009 01:22:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mpi-sb.mpg.de; s=mail200803; h=Cc:Message-Id:From:To: In-Reply-To:Content-Type:Content-Transfer-Encoding:Mime-Version: Subject:Date:References; bh=O8rn+sFalCbXW3U9ssBFBl8U87nxY98/AHNL YuvBS+I=; b=aRaDUtObgLF8aQhIU3n2MiuAV4TCxTmgxTcrxqgoZPNfR0QTRjha Vzy1ppq4s5vNsMSNug12F1XV/B6jwce4jH6zRCbfuEaZnD1RO8E+Q+63Xr/+XukN lOF/2kK+K5gq9Hk5k8VrHSTxM9PiYLWS/XXvUceHI7FR/6/kSwL8Dbo= Received: from infao0710.mpi-sb.mpg.de ([139.19.1.28]:40999 helo=newzak.mpi-sb.mpg.de) by hera.mpi-sb.mpg.de (envelope-from ) with esmtp (Exim 4.69) id 1M6KcK-0001oh-3e; Tue, 19 May 2009 10:24:11 +0200 Received: from g226213088.adsl.alicedsl.de ([92.226.213.88]:55928) by newzak.mpi-sb.mpg.de with esmtpsa (TLS-1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.63) (envelope-from ) id 1M6KcJ-0003ao-Ga; Tue, 19 May 2009 10:24:07 +0200 Message-Id: <23595DC0-4A86-4605-BA1E-C0E68625A122@mpi-sws.org> From: Bryan Ford To: "Ford, Alan" In-Reply-To: <2181C5F19DD0254692452BFF3EAF1D6807BFD985@rsys005a.comm.ad.roke.co.uk> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Tue, 19 May 2009 10:24:06 +0200 References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807BFD985@rsys005a.comm.ad.roke.co.uk> X-Mailer: Apple Mail (2.930.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 May 2009 08:22:38 -0000 On May 18, 2009, at 5:32 PM, Ford, Alan wrote: > Bryan Ford wrote: >> On May 16, 2009, at 3:22 PM, Iljitsch van Beijnum wrote: >>> On 16 mei 2009, at 6:50, Bryan Ford wrote: >>> >>>> That still leaves all the issues with the new TCP options. >>> >>> A discussion that a bunch of us had offline: would it be useful to >>> exchange signaling information in-band at the beginning of a > subflow? >>> >>> I.e., I set up something that looks like a new TCP session to >>> middleboxes, but in reality it's an additional subflow for an >>> existing session. Rather than negotiate all the housekeeping in >>> options, I simply do that in-band. When the negotiation is finished, >>> I start exchanging data. >>> >>> In fact, I could do that for all packets. The first n bytes >>> following the TCP header could be the second sequence number. >>> >>> Obviously deep packet inspection boxes wouldn't like that, but if we >>> can't do options, we must maintain the sequence numbering and we >>> can't touch the data then maybe using that path for multipath isn't >>> such a good idea after all. >> >> This approach sounds more promising to me in general. Then MPTCP >> starts to take more and more of the flavor of a new transport >> protocol >> layered atop an existing protocol (namely legacy single-path TCP), >> taking advantage of legacy TCP to provide per-flow congestion control >> and get through middleboxes. > > However, simply putting data at the start of a packet still relies on > middleboxes not to mess with the segmentation. It would be nice to > get a > solution where it doesn't matter if this happens. > > Which makes me wonder whether a TLS-like solution, where all data > (both > application and control) is carried in the TCP stream, and data is > chunked and preceded by a header of type, length, and data sequence > number (if appropriate). We could then switch between control data and > application data with no major issues, and we still get the benefits > of > data sequence numbering without TCP Options. I haven't thought this > through yet, so it may be riddled with holes, so fire away...! Yes, this is exactly the type of thing I had in mind. >> This approach would also have the nice benefit of effectively >> restoring the long-lost "end-to-endness" of TCP's originally intended >> semantics, from the perspective of the application, even when many or >> all of the individual legacy TCP flows atop which an MPTCP connection >> is built are being split in the middle by stateful firewalls, NATs, >> and/or PEPs. > > I'm not sure if that really adds anything over and above what we > already > have. Although the MPTCP layer would be (at the moment - who knows > what > NATs may do in the future!) end-to-end, its primary purpose is to > reassemble data spread across multiple paths, and there is little else > in the protocol at that level (of course, this could be a convenient > place for future extensions, but that's not our concern). Data as > delivered to the application would be pretty much the same as today. > >> For example, if a stateful middlebox crashes and loses >> its legacy TCP state, the legacy TCP connection is hosed, but the >> MPTCP connection might well be able to notice the TCP flow's >> connection reset and just retry establishing that flow, while >> shifting >> traffic to other flows in the meantime. > > Indeed, that's one particular use case for MPTCP, and should work in > either of the proposals put forward so far. > >> Taking this approach further, have you thought about how MPTCP >> should/ >> might interact with IPSEC? Of course IPSEC can always be deployed >> above IP and below legacy TCP, as usual, but we know how well IPSEC >> tends to get along with middleboxes. Combining traditional IPSEC >> with >> MPTCP would effectively defeat all the care MPTCP went to to provide >> backward compatibility by "looking like" legacy TCP. Even with >> IPSEC's recent UDP encapsulation enhancements to get through NATs and >> firewalls, it remains incompatible with PEPs or TCP proxies that need >> to split or monitor TCP's congestion control loop, not to mention >> middleboxes that let through only TCP and not UDP. But taking the >> view of legacy TCP as a Flow Layer when used by MPTCP, the right >> place >> to deploy IPSEC is actually _above_ legacy TCP but below MPTCP's > inter- >> flow transport layer. This way, "IPSEC-secured MPTCP" would have all >> the same backward compatibility advantages of unsecured MPTCP because >> the Flow Layer part looks just like ordinary TCP, but the IPSEC layer >> in the middle would then be able to enforce the true "end-to-endness" >> of the MPSEC transport layer by preventing middleboxes from >> interfering with or even seeing the MPSEC-level data transmission > loop. > > An interesting thought, but would IPSec-over-MPTCP be any more > appropriate than using TLS? (Aside from being transparent to the > application, of course). Yes; see my last response to Iljitsch, speculating that MPTCP-over-TLS- over-TCP might actually provide the best combination of protocol properties (though with the slight pragmatic challenge that we'd probably need to develop OS kernel-level implementations of SSL/TLS...). Bryan > Assuming paths with no NATs and thus IPSec end-to-end, it should be > feasible to run MPTCP over TCP over IPSec with no issues. But I must > admit I haven't gone through this step-by-step to make sure! > > Cheers, > Alan > From c.raiciu@cs.ucl.ac.uk Tue May 19 02:15:15 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 67C8F3A692E for ; Tue, 19 May 2009 02:15:15 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.539 X-Spam-Level: X-Spam-Status: No, score=-2.539 tagged_above=-999 required=5 tests=[AWL=0.060, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XT6YgPCxXzss for ; Tue, 19 May 2009 02:15:14 -0700 (PDT) Received: from bells2.cs.ucl.ac.uk (bells2.cs.ucl.ac.uk [128.16.5.33]) by core3.amsl.com (Postfix) with ESMTP id E859B3A68E7 for ; Tue, 19 May 2009 02:15:13 -0700 (PDT) Received: from blackie2.cs.ucl.ac.uk ([128.16.68.58]) by bells2.cs.ucl.ac.uk with esmtpsa (TLSv1:AES128-SHA:128) (C.Raiciu authenticated) (Exim 4.54) id 1M6LuH-0009sT-Cr for multipathtcp@ietf.org; Tue, 19 May 2009 10:46:45 +0100 Mime-Version: 1.0 (Apple Message framework v753.1) In-Reply-To: <23595DC0-4A86-4605-BA1E-C0E68625A122@mpi-sws.org> References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807BFD985@rsys005a.comm.ad.roke.co.uk> <23595DC0-4A86-4605-BA1E-C0E68625A122@mpi-sws.org> Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <98ABAA38-DAD3-4852-B7D4-65AD4E171A5E@cs.ucl.ac.uk> Content-Transfer-Encoding: 7bit From: Costin Raiciu Date: Tue, 19 May 2009 10:16:46 +0100 To: multipathtcp@ietf.org X-Mailer: Apple Mail (2.753.1) Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 May 2009 09:15:15 -0000 I It seems nice to layer MTCP over TLS/TCP, but I see two issues: 1. We need at least one endpoint to have a verifiable certificate. 2. The public key operations involved seem a bt heavyweight to be used in *every* TCP connection. Perhaps a better fit is Mark's recent work on TCP crypt, which is an opportunistic encryption layer over tcp that doesn't require certificates and is a bit lighter in terms of public key ops.. That said, chunking the data and introducing the sequence numbers in the payload seems worthy of further exploration. Costin On 19 May 2009, at 09:24, Bryan Ford wrote: > On May 18, 2009, at 5:32 PM, Ford, Alan wrote: >> Bryan Ford wrote: >>> On May 16, 2009, at 3:22 PM, Iljitsch van Beijnum wrote: >>>> On 16 mei 2009, at 6:50, Bryan Ford wrote: >>>> >>>>> That still leaves all the issues with the new TCP options. >>>> >>>> A discussion that a bunch of us had offline: would it be useful to >>>> exchange signaling information in-band at the beginning of a >> subflow? >>>> >>>> I.e., I set up something that looks like a new TCP session to >>>> middleboxes, but in reality it's an additional subflow for an >>>> existing session. Rather than negotiate all the housekeeping in >>>> options, I simply do that in-band. When the negotiation is >>>> finished, >>>> I start exchanging data. >>>> >>>> In fact, I could do that for all packets. The first n bytes >>>> following the TCP header could be the second sequence number. >>>> >>>> Obviously deep packet inspection boxes wouldn't like that, but >>>> if we >>>> can't do options, we must maintain the sequence numbering and we >>>> can't touch the data then maybe using that path for multipath isn't >>>> such a good idea after all. >>> >>> This approach sounds more promising to me in general. Then MPTCP >>> starts to take more and more of the flavor of a new transport >>> protocol >>> layered atop an existing protocol (namely legacy single-path TCP), >>> taking advantage of legacy TCP to provide per-flow congestion >>> control >>> and get through middleboxes. >> >> However, simply putting data at the start of a packet still relies on >> middleboxes not to mess with the segmentation. It would be nice to >> get a >> solution where it doesn't matter if this happens. >> >> Which makes me wonder whether a TLS-like solution, where all data >> (both >> application and control) is carried in the TCP stream, and data is >> chunked and preceded by a header of type, length, and data sequence >> number (if appropriate). We could then switch between control data >> and >> application data with no major issues, and we still get the >> benefits of >> data sequence numbering without TCP Options. I haven't thought this >> through yet, so it may be riddled with holes, so fire away...! > > Yes, this is exactly the type of thing I had in mind. > >>> This approach would also have the nice benefit of effectively >>> restoring the long-lost "end-to-endness" of TCP's originally >>> intended >>> semantics, from the perspective of the application, even when >>> many or >>> all of the individual legacy TCP flows atop which an MPTCP >>> connection >>> is built are being split in the middle by stateful firewalls, NATs, >>> and/or PEPs. >> >> I'm not sure if that really adds anything over and above what we >> already >> have. Although the MPTCP layer would be (at the moment - who knows >> what >> NATs may do in the future!) end-to-end, its primary purpose is to >> reassemble data spread across multiple paths, and there is little >> else >> in the protocol at that level (of course, this could be a convenient >> place for future extensions, but that's not our concern). Data as >> delivered to the application would be pretty much the same as today. >> >>> For example, if a stateful middlebox crashes and loses >>> its legacy TCP state, the legacy TCP connection is hosed, but the >>> MPTCP connection might well be able to notice the TCP flow's >>> connection reset and just retry establishing that flow, while >>> shifting >>> traffic to other flows in the meantime. >> >> Indeed, that's one particular use case for MPTCP, and should work in >> either of the proposals put forward so far. >> >>> Taking this approach further, have you thought about how MPTCP >>> should/ >>> might interact with IPSEC? Of course IPSEC can always be deployed >>> above IP and below legacy TCP, as usual, but we know how well IPSEC >>> tends to get along with middleboxes. Combining traditional IPSEC >>> with >>> MPTCP would effectively defeat all the care MPTCP went to to provide >>> backward compatibility by "looking like" legacy TCP. Even with >>> IPSEC's recent UDP encapsulation enhancements to get through NATs >>> and >>> firewalls, it remains incompatible with PEPs or TCP proxies that >>> need >>> to split or monitor TCP's congestion control loop, not to mention >>> middleboxes that let through only TCP and not UDP. But taking the >>> view of legacy TCP as a Flow Layer when used by MPTCP, the right >>> place >>> to deploy IPSEC is actually _above_ legacy TCP but below MPTCP's >> inter- >>> flow transport layer. This way, "IPSEC-secured MPTCP" would have >>> all >>> the same backward compatibility advantages of unsecured MPTCP >>> because >>> the Flow Layer part looks just like ordinary TCP, but the IPSEC >>> layer >>> in the middle would then be able to enforce the true "end-to- >>> endness" >>> of the MPSEC transport layer by preventing middleboxes from >>> interfering with or even seeing the MPSEC-level data transmission >> loop. >> >> An interesting thought, but would IPSec-over-MPTCP be any more >> appropriate than using TLS? (Aside from being transparent to the >> application, of course). > > Yes; see my last response to Iljitsch, speculating that MPTCP-over- > TLS-over-TCP might actually provide the best combination of > protocol properties (though with the slight pragmatic challenge > that we'd probably need to develop OS kernel-level implementations > of SSL/TLS...). > > Bryan > >> Assuming paths with no NATs and thus IPSec end-to-end, it should be >> feasible to run MPTCP over TCP over IPSec with no issues. But I must >> admit I haven't gone through this step-by-step to make sure! >> >> Cheers, >> Alan >> > > _______________________________________________ > multipathtcp mailing list > multipathtcp@ietf.org > https://www.ietf.org/mailman/listinfo/multipathtcp From iljitsch@muada.com Tue May 19 03:24:44 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 6B29F28C2A7 for ; Tue, 19 May 2009 03:24:44 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8SyTzfU5t3YL for ; Tue, 19 May 2009 03:24:43 -0700 (PDT) Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by core3.amsl.com (Postfix) with ESMTP id 599F228C29D for ; Tue, 19 May 2009 03:24:43 -0700 (PDT) Received: from claw.it.uc3m.es (claw.it.uc3m.es [163.117.139.52]) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n4JAPhZ5012147 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Tue, 19 May 2009 12:25:43 +0200 (CEST) (envelope-from iljitsch@muada.com) Message-Id: From: Iljitsch van Beijnum To: Bryan Ford In-Reply-To: <8A52D0F9-2C40-497D-84F5-13CC0575BF97@mpi-sws.org> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v935.3) Date: Tue, 19 May 2009 12:25:52 +0200 References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> <7BE7F3C3-2CD8-4C86-B4A1-4748205B30BE@muada.com> <8A52D0F9-2C40-497D-84F5-13CC0575BF97@mpi-sws.org> X-Mailer: Apple Mail (2.935.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 May 2009 10:24:44 -0000 On 19 mei 2009, at 10:16, Bryan Ford wrote: >> Isn't that a feature? > You mean in that IPSEC effectively enforces the "end-to-endness" of > TCP? Yes. > The fact that IPSEC also prevents devices in the network from > implementing reasonable performance optimizations as flows cross > specific network technologies or specific administrative domains is > not a feature. I wouldn't want attackers to be able to slow my traffic down to a crawl. But can you explain a bit more about these benevolent optimizers? What is it that they do, exactly? > That said, you make an excellent point - since SSL over TCP is > already ubiquitous and the convention of deploying SSL at > application rather than system level is merely a convention and by > no means essential to the protocol... I'd say that proper layering would be to provide authentication below the TCP layer and encryption above it, as encryption benefits from TCP's ordering, and TCP benefits from IPsec's authentication. > If we wanted to provide system-level, application-transparent > security for MPTCP (or CMT-SCTP or SST or...) while providing as > much backward compatibility as possible, maybe the right way to do > that is to layer MPTCP over SSL over TCP. That way secured MPTCP > flows would look to the network (and to middleboxes in the network) > indistinguishable from, say, ordinary HTTPS flows, which may be a > really good thing. Well, it would look like that even if TLS is multiplexed over multiple subflows, but: >> An interesting approach would be TCP over TLS over TCP where the >> TLS protects against the issues with middleboxes rewriting data >> that we talked about. To do this well, it would be helpful to send >> each TLS frame/packet (not sure what the right terminology is) over >> the same path. > I think we're thinking along the same lines here. From baford@mpi-sws.org Tue May 19 05:05:26 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 397DC3A68BB for ; Tue, 19 May 2009 05:05:26 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.249 X-Spam-Level: X-Spam-Status: No, score=-6.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0oyJXEqvBpGV for ; Tue, 19 May 2009 05:05:24 -0700 (PDT) Received: from hera.mpi-sb.mpg.de (infao0809.mpi-sb.mpg.de [139.19.1.49]) by core3.amsl.com (Postfix) with ESMTP id 101943A6A97 for ; Tue, 19 May 2009 05:05:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mpi-sb.mpg.de; s=mail200803; h=Cc:Message-Id:From:To: In-Reply-To:Content-Type:Content-Transfer-Encoding:Mime-Version: Subject:Date:References; bh=7K3yFhyi5s5NM2KDnCvRre9P2YTFLZc87Zki n9xuWek=; b=G147o+x0/weLgrXV69yDs9AQhDs9vBIl6/1+c6RfipUDgzLSpesY LiVaZEw03t2SlQFLv4KX2msHXYfynQfnjUFOb59C4TXWI+imDkZlmJa4SaX8ut9a zhbzJULsCcVvLR8yPeD3QdiLp2JVhozMXNi/8/W+e+EPtph9AIrXCrU= Received: from infao0710.mpi-sb.mpg.de ([139.19.1.28]:41800 helo=newzak.mpi-sb.mpg.de) by hera.mpi-sb.mpg.de (envelope-from ) with esmtp (Exim 4.69) id 1M6O5u-0008S2-Qz; Tue, 19 May 2009 14:06:58 +0200 Received: from f051045168.adsl.alicedsl.de ([78.51.45.168]:56131) by newzak.mpi-sb.mpg.de with esmtpsa (TLS-1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.63) (envelope-from ) id 1M6O5u-0004wh-AM; Tue, 19 May 2009 14:06:54 +0200 Message-Id: <7111CACD-CF77-4881-8324-8B2EB77FB943@mpi-sws.org> From: Bryan Ford To: Costin Raiciu In-Reply-To: <98ABAA38-DAD3-4852-B7D4-65AD4E171A5E@cs.ucl.ac.uk> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Tue, 19 May 2009 14:06:53 +0200 References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807BFD985@rsys005a.comm.ad.roke.co.uk> <23595DC0-4A86-4605-BA1E-C0E68625A122@mpi-sws.org> <98ABAA38-DAD3-4852-B7D4-65AD4E171A5E@cs.ucl.ac.uk> X-Mailer: Apple Mail (2.930.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 May 2009 12:05:26 -0000 On May 19, 2009, at 11:16 AM, Costin Raiciu wrote: > I It seems nice to layer MTCP over TLS/TCP, but I see two issues: > 1. We need at least one endpoint to have a verifiable certificate. IPSEC (and any other secure protocol with strong authentication) has the same issue, no matter where in the stack it's layered. If you want strong authentication, you have to have a strong identity to authenticate. Although a self-certifying host namespace as in HIP can help by making the name effectivity double as its own verifiable certificate. I don't know if anyone has designed a HIP-like self- certifying namespace for SSL/TLS, but it could certainly be done. > 2. The public key operations involved seem a bt heavyweight to be > used in *every* TCP connection. True - but even a lot of the stuff TCP traditionally does, such as establishing a congestion control context, is "a bit heavyweight" to be done once for *every* application stream, as we discovered particularly acutely in the context of HTTP/1.0. Because TCP has traditionally been doing all this heavyweight stuff once for every application stream, applications have traditionally been forced to compensate - e.g., as HTTP/1.1 did by adding persistent/pipelined connections, even though persistent/pipelined connections introduce a bunch of problems of their own like extra compatibility and unnecessary serialization of independent transactions. The real solution is to modify the transport to stop doing all the heavyweight stuff for every application stream, instead doing the heavyweight stuff once to establish a secure, congestion controlled session between a host pair and then allowing the application to open many lightweight streams that reuse the same heavyweight communication state. That was one of the main ideas of my experimental Structured Stream Transport (http://brynosaurus.com/pub/net/sst-abs.html), but it shouldn't be difficult to integrate many of the same ideas in a "next- generation TCP" that retains a stronger emphasis on backward compatibility. > Perhaps a better fit is Mark's recent work on TCP crypt, which is an > opportunistic encryption layer over tcp that doesn't require > certificates and is a bit lighter in terms of public key ops.. This also sounds reasonable to me, but it basically just sounds like a different "control plane" key exchange scheme that could be attached to one basic "data plane" encryption/authentication protocol. There are at least three such identity/key exchange schemes that seem useful and relevant in different systems: "weak identities", ala TCP crypt; PKI-based identities, as in SSL/TLS/IPSEC; and self-certifying identities, as in HIP or UIA. Why should we necessarily have to choose one approach? They should be pluggable modules configurable/ selectable by system administrators or applications. At any rate, this is all security-land stuff, and I wouldn't really expect that to be in the scope of this BOF/WG other than to the point of considering how security layers like SSL/TLS/IPSEC should fit into a multipath- capable next-generation TCP architecture. Bryan > That said, chunking the data and introducing the sequence numbers in > the payload seems worthy of further exploration. > Costin > > On 19 May 2009, at 09:24, Bryan Ford wrote: > >> On May 18, 2009, at 5:32 PM, Ford, Alan wrote: >>> Bryan Ford wrote: >>>> On May 16, 2009, at 3:22 PM, Iljitsch van Beijnum wrote: >>>>> On 16 mei 2009, at 6:50, Bryan Ford wrote: >>>>> >>>>>> That still leaves all the issues with the new TCP options. >>>>> >>>>> A discussion that a bunch of us had offline: would it be useful to >>>>> exchange signaling information in-band at the beginning of a >>> subflow? >>>>> >>>>> I.e., I set up something that looks like a new TCP session to >>>>> middleboxes, but in reality it's an additional subflow for an >>>>> existing session. Rather than negotiate all the housekeeping in >>>>> options, I simply do that in-band. When the negotiation is >>>>> finished, >>>>> I start exchanging data. >>>>> >>>>> In fact, I could do that for all packets. The first n bytes >>>>> following the TCP header could be the second sequence number. >>>>> >>>>> Obviously deep packet inspection boxes wouldn't like that, but >>>>> if we >>>>> can't do options, we must maintain the sequence numbering and we >>>>> can't touch the data then maybe using that path for multipath >>>>> isn't >>>>> such a good idea after all. >>>> >>>> This approach sounds more promising to me in general. Then MPTCP >>>> starts to take more and more of the flavor of a new transport >>>> protocol >>>> layered atop an existing protocol (namely legacy single-path TCP), >>>> taking advantage of legacy TCP to provide per-flow congestion >>>> control >>>> and get through middleboxes. >>> >>> However, simply putting data at the start of a packet still relies >>> on >>> middleboxes not to mess with the segmentation. It would be nice to >>> get a >>> solution where it doesn't matter if this happens. >>> >>> Which makes me wonder whether a TLS-like solution, where all data >>> (both >>> application and control) is carried in the TCP stream, and data is >>> chunked and preceded by a header of type, length, and data sequence >>> number (if appropriate). We could then switch between control data >>> and >>> application data with no major issues, and we still get the >>> benefits of >>> data sequence numbering without TCP Options. I haven't thought this >>> through yet, so it may be riddled with holes, so fire away...! >> >> Yes, this is exactly the type of thing I had in mind. >> >>>> This approach would also have the nice benefit of effectively >>>> restoring the long-lost "end-to-endness" of TCP's originally >>>> intended >>>> semantics, from the perspective of the application, even when >>>> many or >>>> all of the individual legacy TCP flows atop which an MPTCP >>>> connection >>>> is built are being split in the middle by stateful firewalls, NATs, >>>> and/or PEPs. >>> >>> I'm not sure if that really adds anything over and above what we >>> already >>> have. Although the MPTCP layer would be (at the moment - who knows >>> what >>> NATs may do in the future!) end-to-end, its primary purpose is to >>> reassemble data spread across multiple paths, and there is little >>> else >>> in the protocol at that level (of course, this could be a convenient >>> place for future extensions, but that's not our concern). Data as >>> delivered to the application would be pretty much the same as today. >>> >>>> For example, if a stateful middlebox crashes and loses >>>> its legacy TCP state, the legacy TCP connection is hosed, but the >>>> MPTCP connection might well be able to notice the TCP flow's >>>> connection reset and just retry establishing that flow, while >>>> shifting >>>> traffic to other flows in the meantime. >>> >>> Indeed, that's one particular use case for MPTCP, and should work in >>> either of the proposals put forward so far. >>> >>>> Taking this approach further, have you thought about how MPTCP >>>> should/ >>>> might interact with IPSEC? Of course IPSEC can always be deployed >>>> above IP and below legacy TCP, as usual, but we know how well IPSEC >>>> tends to get along with middleboxes. Combining traditional IPSEC >>>> with >>>> MPTCP would effectively defeat all the care MPTCP went to to >>>> provide >>>> backward compatibility by "looking like" legacy TCP. Even with >>>> IPSEC's recent UDP encapsulation enhancements to get through NATs >>>> and >>>> firewalls, it remains incompatible with PEPs or TCP proxies that >>>> need >>>> to split or monitor TCP's congestion control loop, not to mention >>>> middleboxes that let through only TCP and not UDP. But taking the >>>> view of legacy TCP as a Flow Layer when used by MPTCP, the right >>>> place >>>> to deploy IPSEC is actually _above_ legacy TCP but below MPTCP's >>> inter- >>>> flow transport layer. This way, "IPSEC-secured MPTCP" would have >>>> all >>>> the same backward compatibility advantages of unsecured MPTCP >>>> because >>>> the Flow Layer part looks just like ordinary TCP, but the IPSEC >>>> layer >>>> in the middle would then be able to enforce the true "end-to- >>>> endness" >>>> of the MPSEC transport layer by preventing middleboxes from >>>> interfering with or even seeing the MPSEC-level data transmission >>> loop. >>> >>> An interesting thought, but would IPSec-over-MPTCP be any more >>> appropriate than using TLS? (Aside from being transparent to the >>> application, of course). >> >> Yes; see my last response to Iljitsch, speculating that MPTCP-over- >> TLS-over-TCP might actually provide the best combination of >> protocol properties (though with the slight pragmatic challenge >> that we'd probably need to develop OS kernel-level implementations >> of SSL/TLS...). >> >> Bryan >> >>> Assuming paths with no NATs and thus IPSec end-to-end, it should be >>> feasible to run MPTCP over TCP over IPSec with no issues. But I must >>> admit I haven't gone through this step-by-step to make sure! >>> >>> Cheers, >>> Alan >>> >> >> _______________________________________________ >> multipathtcp mailing list >> multipathtcp@ietf.org >> https://www.ietf.org/mailman/listinfo/multipathtcp > > _______________________________________________ > multipathtcp mailing list > multipathtcp@ietf.org > https://www.ietf.org/mailman/listinfo/multipathtcp From baford@mpi-sws.org Tue May 19 06:00:08 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id A801028C32F for ; Tue, 19 May 2009 06:00:08 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.249 X-Spam-Level: X-Spam-Status: No, score=-6.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yqpCz-6Tt27k for ; Tue, 19 May 2009 06:00:07 -0700 (PDT) Received: from hera.mpi-sb.mpg.de (hera.mpi-sb.mpg.de [139.19.1.49]) by core3.amsl.com (Postfix) with ESMTP id 627C128C32A for ; Tue, 19 May 2009 06:00:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mpi-sb.mpg.de; s=mail200803; h=Cc:Message-Id:From:To: In-Reply-To:Content-Type:Content-Transfer-Encoding:Mime-Version: Subject:Date:References; bh=gLSXg3Ca7FC6c5nAYGLw9uxQu6aS9JySG8mw K55SV5E=; b=DoYim1lRmmn8VJCd3UOhIQk9bcCnuQHwNZUKEEH60VlYV5e1TtGk Pf4tjKEDFx0sZTD/OrzbSxCDw06jn07Jw/iGfOO4rU221CaPO2O7VYvvd2HfN6pO Sw7WatZ87WJE65EJUScOEQnb4noJ2Swm8LQSoYLLgfnLsIhR5H0e6S0= Received: from newzak.mpi-sb.mpg.de ([139.19.1.28]:50859) by hera.mpi-sb.mpg.de (envelope-from ) with esmtp (Exim 4.69) id 1M6Owr-0000kK-If; Tue, 19 May 2009 15:01:40 +0200 Received: from f051045168.adsl.alicedsl.de ([78.51.45.168]:56221) by newzak.mpi-sb.mpg.de with esmtpsa (TLS-1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.63) (envelope-from ) id 1M6Owr-0005LM-3w; Tue, 19 May 2009 15:01:37 +0200 Message-Id: <415C2785-40E3-4F6C-8006-AA2420AE357F@mpi-sws.org> From: Bryan Ford To: Iljitsch van Beijnum In-Reply-To: Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Tue, 19 May 2009 15:01:36 +0200 References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> <7BE7F3C3-2CD8-4C86-B4A1-4748205B30BE@muada.com> <8A52D0F9-2C40-497D-84F5-13CC0575BF97@mpi-sws.org> X-Mailer: Apple Mail (2.930.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 May 2009 13:00:08 -0000 On May 19, 2009, at 12:25 PM, Iljitsch van Beijnum wrote: > On 19 mei 2009, at 10:16, Bryan Ford wrote: > >>> Isn't that a feature? > >> You mean in that IPSEC effectively enforces the "end-to-endness" of >> TCP? > > Yes. > >> The fact that IPSEC also prevents devices in the network from >> implementing reasonable performance optimizations as flows cross >> specific network technologies or specific administrative domains is >> not a feature. > > I wouldn't want attackers to be able to slow my traffic down to a > crawl. On-path attackers can by definition do anything to your traffic, including cut off connectivity completely, so I assume you're referring only to off-path attackers here. Defense against such off- path attacks against performance is indeed important, but that doesn't imply that such defenses must operate by denying _anyone_ in the network - including on-path elements - the ability to see or affect performance-related aspects of flows. That's one way to provide such a defense, but it amounts to throwing out the baby with the bathwater. Routing faces the same security issues: without appropriate security measures off-path attackers can potentially capture other hosts' traffic by injecting BGP announcements for prefixes they don't own, or dynamically impersonate other hosts by source address forgery. One conceivable "end-to-end-style" solution would be to move the Internet to authenticate source routing, where end hosts are responsible for determining the complete route a packet is to take, and embedding that route in each packet along with a series of verifiable authentication tokens - but no one is seriously considering such an approach to dealing with the Internet's routing vulnerabilities, in part because it takes too much control away from in-network entities (ISPs, corporate network admins) who have legitimate needs to control how traffic flows on their network. I think exactly the same applies to congestion control - preventing anyone in the network from having any access at all is one way of "protecting" congestion control, but it's far too heavyhanded. > But can you explain a bit more about these benevolent optimizers? > What is it that they do, exactly? They sit in the path and try to make TCP performance suck less, e.g., when a mobile network link has a nontrivial non-congestion loss rate, or when a cellular link cuts out for a short period and causes TCP's congestion control to back off to zero unnecessarily when it comes back, or when a link has a high bandwidth-delay product causing TCP take forever to ramp up... Many companies, including Cisco (look up their RBSCP documentation) already sell PEPs of various kinds; a lot of these devices are admittedly horrible hacks of one sort or another, but they clearly fill a market demand in the same way that NATs and firewalls do. RFC3515 discusses various goals and techniques such devices commonly use, as well as the problems they typically cause (including breaking the end-to-endness of TCP and being completely incompatible with IPSEC). For some relevant research papers on the principles these devices were built from, see for example Bakre's I- TCP (Indirect TCP), Kopparty's Split-TCP, Balakrishnan's TCP Snoop, Martin Swany's Logistical Session Layer (SC2004), and the various other references you'll find in section 3.1 of our "Breaking Up the Transport Logjam" paper. >> That said, you make an excellent point - since SSL over TCP is >> already ubiquitous and the convention of deploying SSL at >> application rather than system level is merely a convention and by >> no means essential to the protocol... > > I'd say that proper layering would be to provide authentication > below the TCP layer and encryption above it, as encryption benefits > from TCP's ordering, and TCP benefits from IPsec's authentication. Perhaps, as long as there remains some provision for in-path devices to get in the loop when they need to, ideally in a clean and composable fashion. Bryan >> If we wanted to provide system-level, application-transparent >> security for MPTCP (or CMT-SCTP or SST or...) while providing as >> much backward compatibility as possible, maybe the right way to do >> that is to layer MPTCP over SSL over TCP. That way secured MPTCP >> flows would look to the network (and to middleboxes in the network) >> indistinguishable from, say, ordinary HTTPS flows, which may be a >> really good thing. > > Well, it would look like that even if TLS is multiplexed over > multiple subflows, but: > >>> An interesting approach would be TCP over TLS over TCP where the >>> TLS protects against the issues with middleboxes rewriting data >>> that we talked about. To do this well, it would be helpful to send >>> each TLS frame/packet (not sure what the right terminology is) >>> over the same path. > >> I think we're thinking along the same lines here. From iljitsch@muada.com Tue May 19 06:27:01 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 259E728C34A for ; Tue, 19 May 2009 06:27:01 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MmeH26qyai2Q for ; Tue, 19 May 2009 06:26:59 -0700 (PDT) Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by core3.amsl.com (Postfix) with ESMTP id 86AB33A6C3E for ; Tue, 19 May 2009 06:26:59 -0700 (PDT) Received: from [163.117.139.52] ([163.117.139.52]) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n4JDRetu013303 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Tue, 19 May 2009 15:27:40 +0200 (CEST) (envelope-from iljitsch@muada.com) Message-Id: <09997716-CAC3-47CD-B2D1-D4A9F9352591@muada.com> From: Iljitsch van Beijnum To: Bryan Ford In-Reply-To: <415C2785-40E3-4F6C-8006-AA2420AE357F@mpi-sws.org> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v935.3) Date: Tue, 19 May 2009 15:27:49 +0200 References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> <7BE7F3C3-2CD8-4C86-B4A1-4748205B30BE@muada.com> <8A52D0F9-2C40-497D-84F5-13CC0575BF97@mpi-sws.org> <415C2785-40E3-4F6C-8006-AA2420AE357F@mpi-sws.org> X-Mailer: Apple Mail (2.935.3) Cc: multipathtcp@ietf.org Subject: [multipathtcp] TCP performance optimizing middleboxes X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 May 2009 13:27:01 -0000 On 19 mei 2009, at 15:01, Bryan Ford wrote: >> But can you explain a bit more about these benevolent optimizers? >> What is it that they do, exactly? > They sit in the path and try to make TCP performance suck less, > e.g., when a mobile network link has a nontrivial non-congestion > loss rate, And how do they do that? By magically reconstituting lost packets? A loss is a loss, congestion control is going to react to that no matter what. > or when a link has a high bandwidth-delay product causing TCP take > forever to ramp up... You mean like sending ACKs for data that hasn't been received yet? Are there still boxes doing this? It's a terrible hack, it breaks every session where a packet gets lost. > Many companies, including Cisco (look up their RBSCP documentation) > already sell PEPs of various kinds; A dubious endorsement if I ever heard one... The network should stick to dropping packets and setting the ECN bits, and not second guess TCP congestion control. That way, we'll be stuck with NewReno for the rest of eternity like NATs force us to stay with TCP and UDP. > RFC3515 discusses various goals and techniques such devices commonly > use The SIP Refer Method ? From touch@ISI.EDU Tue May 19 06:39:00 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id DDD963A6910 for ; Tue, 19 May 2009 06:39:00 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.489 X-Spam-Level: X-Spam-Status: No, score=-2.489 tagged_above=-999 required=5 tests=[AWL=0.110, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nhYRciCI8nQc for ; Tue, 19 May 2009 06:38:58 -0700 (PDT) Received: from vapor.isi.edu (vapor.isi.edu [128.9.64.64]) by core3.amsl.com (Postfix) with ESMTP id 586FE28C33A for ; Tue, 19 May 2009 06:38:58 -0700 (PDT) Received: from [192.168.1.47] (pool-71-106-119-240.lsanca.dsl-w.verizon.net [71.106.119.240]) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id n4JDe7xw008115; Tue, 19 May 2009 06:40:08 -0700 (PDT) Message-ID: <4A12B6B7.40909@isi.edu> Date: Tue, 19 May 2009 06:40:07 -0700 From: Joe Touch User-Agent: Thunderbird 2.0.0.21 (Windows/20090302) MIME-Version: 1.0 To: Iljitsch van Beijnum References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> <7BE7F3C3-2CD8-4C86-B4A1-4748205B30BE@muada.com> <8A52D0F9-2C40-497D-84F5-13CC0575BF97@mpi-sws.org> <415C2785-40E3-4F6C-8006-AA2420AE357F@mpi-sws.org> <09997716-CAC3-47CD-B2D1-D4A9F9352591@muada.com> In-Reply-To: <09997716-CAC3-47CD-B2D1-D4A9F9352591@muada.com> X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] TCP performance optimizing middleboxes X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 May 2009 13:39:01 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Iljitsch van Beijnum wrote: > On 19 mei 2009, at 15:01, Bryan Ford wrote: > >>> But can you explain a bit more about these benevolent optimizers? >>> What is it that they do, exactly? > >> They sit in the path and try to make TCP performance suck less, e.g., >> when a mobile network link has a nontrivial non-congestion loss rate, > > And how do they do that? By magically reconstituting lost packets? Generally by spoofing the endpoint, i.e., by acting like a proxy, even though the source thinks it's talking to the endpoint, e.g.: - sending spoofed ACKs to the source - buffering data and/or re-framing the segments - adjusting option values (e.g., faking a smaller RTT) On wireless links, they can reduce the non-congestion loss by doing "FEC" - either passively (sending the same segment multiple times, using true FEC mechanisms) or actively (buffering and retransmitting until received at some upstream optimizer). > A loss is a loss, congestion control is going to react to that no matter > what. Not if it doesn't see the loss anymore. But it would see, e.g., much larger RTTs, phony smaller RTTs, reordering, and/or highly variable delays. > >> or when a link has a high bandwidth-delay product causing TCP take >> forever to ramp up... > > You mean like sending ACKs for data that hasn't been received yet? > > Are there still boxes doing this? It's a terrible hack, it breaks every > session where a packet gets lost. Yes, there are. It doesn't have to break during loss if the spoofer holds the data until it gets the upstream ACK - i.e., if it emulates a proxy. It's still evil (use a nontransparent proxy if that's what you want), though. Joe -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkoStrYACgkQE5f5cImnZrteOwCfemNuw2XMVr+ij+ub+MQCbtIZ S9gAnR/68I7Cs2yBiYJT4qmTbnzPyKmB =Rp3+ -----END PGP SIGNATURE----- From touch@ISI.EDU Tue May 19 06:42:35 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 959323A70C4 for ; Tue, 19 May 2009 06:42:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.499 X-Spam-Level: X-Spam-Status: No, score=-2.499 tagged_above=-999 required=5 tests=[AWL=0.100, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IXtWPc45OB5j for ; Tue, 19 May 2009 06:42:29 -0700 (PDT) Received: from vapor.isi.edu (vapor.isi.edu [128.9.64.64]) by core3.amsl.com (Postfix) with ESMTP id 7386528C37C for ; Tue, 19 May 2009 06:41:44 -0700 (PDT) Received: from [192.168.1.47] (pool-71-106-119-240.lsanca.dsl-w.verizon.net [71.106.119.240]) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id n4JDgc3R008760; Tue, 19 May 2009 06:42:40 -0700 (PDT) Message-ID: <4A12B74E.3080308@isi.edu> Date: Tue, 19 May 2009 06:42:38 -0700 From: Joe Touch User-Agent: Thunderbird 2.0.0.21 (Windows/20090302) MIME-Version: 1.0 To: Iljitsch van Beijnum References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> <7BE7F3C3-2CD8-4C86-B4A1-4748205B30BE@muada.com> <8A52D0F9-2C40-497D-84F5-13CC0575BF97@mpi-sws.org> <415C2785-40E3-4F6C-8006-AA2420AE357F@mpi-sws.org> <09997716-CAC3-47CD-B2D1-D4A9F9352591@muada.com> In-Reply-To: <09997716-CAC3-47CD-B2D1-D4A9F9352591@muada.com> X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] TCP performance optimizing middleboxes X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 May 2009 13:42:35 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Iljitsch van Beijnum wrote: > On 19 mei 2009, at 15:01, Bryan Ford wrote: > >>> But can you explain a bit more about these benevolent optimizers? >>> What is it that they do, exactly? > >> They sit in the path and try to make TCP performance suck less, e.g., >> when a mobile network link has a nontrivial non-congestion loss rate, ... > The network should stick to dropping packets and setting the ECN bits, > and not second guess TCP congestion control. That way, we'll be stuck > with NewReno for the rest of eternity like NATs force us to stay with > TCP and UDP. One PS - a PEP that doesn't spoof ACKs can still reduce link errors by doing FEC or ARQ; that PEP is just a software link conditioner. Yes, it messes with TCP - esp. the ARQ that effectively makes the PEP look like a variable delay link - but it's not much different than what is already done at the link layer. The evil ones spoof the endpoint; the "kosher" ones aren't evil, but can impact TCP too. Joe -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkoSt04ACgkQE5f5cImnZrtLngCfeVXO1D+WiQEH5F3QJA7prokD BnwAoMMI+Y+Jt3wFJ/SUemXWeL03edI9 =o6Gi -----END PGP SIGNATURE----- From iljitsch@muada.com Tue May 19 07:06:16 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 6D5F13A68AD for ; Tue, 19 May 2009 07:06:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bzDywAJoaY4u for ; Tue, 19 May 2009 07:06:15 -0700 (PDT) Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by core3.amsl.com (Postfix) with ESMTP id 8BFDE28C36C for ; Tue, 19 May 2009 07:05:58 -0700 (PDT) Received: from [163.117.139.52] ([163.117.139.52]) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n4JE6piS013641 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Tue, 19 May 2009 16:06:51 +0200 (CEST) (envelope-from iljitsch@muada.com) Message-Id: <0F988093-BC4A-4D77-8477-07937D0F3D2A@muada.com> From: Iljitsch van Beijnum To: Joe Touch In-Reply-To: <4A12B6B7.40909@isi.edu> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v935.3) Date: Tue, 19 May 2009 16:07:00 +0200 References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> <7BE7F3C3-2CD8-4C86-B4A1-4748205B30BE@muada.com> <8A52D0F9-2C40-497D-84F5-13CC0575BF97@mpi-sws.org> <415C2785-40E3-4F6C-8006-AA2420AE357F@mpi-sws.org> <09997716-CAC3-47CD-B2D1-D4A9F9352591@muada.com> <4A12B6B7.40909@isi.edu> X-Mailer: Apple Mail (2.935.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] TCP performance optimizing middleboxes X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 May 2009 14:06:16 -0000 On 19 mei 2009, at 15:40, Joe Touch wrote: >> A loss is a loss, congestion control is going to react to that no >> matter >> what. > Not if it doesn't see the loss anymore. Well, either the packet is lost or it isn't. :-) If someone in the middle buffers then it's not really lost, just a very particular instance of duplication in the network... >> You mean like sending ACKs for data that hasn't been received yet? >> Are there still boxes doing this? It's a terrible hack, it breaks >> every >> session where a packet gets lost. > Yes, there are. It doesn't have to break during loss if the spoofer > holds the data until it gets the upstream ACK - i.e., if it emulates a > proxy. Who wants to keep that much state? Ideally, there would be a TCP proxy in the middle of the ocean or in the satellite. The "right" way to do this statelessly would be to spoof a SACK, which is renegotiable. My netstat -s tells me: 306555606 packets received 171992750 acks (for 3031050254 bytes) 7564350 duplicate acks 31162 acks for unsent data From touch@ISI.EDU Tue May 19 09:18:19 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id E81DB3A6CA8 for ; Tue, 19 May 2009 09:18:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.016 X-Spam-Level: X-Spam-Status: No, score=-2.016 tagged_above=-999 required=5 tests=[AWL=0.583, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id z9bGIGdQ66lX for ; Tue, 19 May 2009 09:18:19 -0700 (PDT) Received: from vapor.isi.edu (vapor.isi.edu [128.9.64.64]) by core3.amsl.com (Postfix) with ESMTP id AE13C3A6A66 for ; Tue, 19 May 2009 09:18:18 -0700 (PDT) Received: from [128.9.161.245] ([128.9.161.245]) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id n4JGJU0h019050; Tue, 19 May 2009 09:19:32 -0700 (PDT) Message-ID: <4A12DC12.4070206@isi.edu> Date: Tue, 19 May 2009 09:19:30 -0700 From: Joe Touch User-Agent: Thunderbird 2.0.0.21 (Windows/20090302) MIME-Version: 1.0 To: Iljitsch van Beijnum References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> <7BE7F3C3-2CD8-4C86-B4A1-4748205B30BE@muada.com> <8A52D0F9-2C40-497D-84F5-13CC0575BF97@mpi-sws.org> <415C2785-40E3-4F6C-8006-AA2420AE357F@mpi-sws.org> <09997716-CAC3-47CD-B2D1-D4A9F9352591@muada.com> <4A12B6B7.40909@isi.edu> <0F988093-BC4A-4D77-8477-07937D0F3D2A@muada.com> In-Reply-To: <0F988093-BC4A-4D77-8477-07937D0F3D2A@muada.com> X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] TCP performance optimizing middleboxes X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 May 2009 16:18:20 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Iljitsch van Beijnum wrote: > On 19 mei 2009, at 15:40, Joe Touch wrote: > >>> A loss is a loss, congestion control is going to react to that no matter >>> what. > >> Not if it doesn't see the loss anymore. > > Well, either the packet is lost or it isn't. :-) > > If someone in the middle buffers then it's not really lost, just a very > particular instance of duplication in the network... Sometimes the duplication is removed too, FWIW. >>> You mean like sending ACKs for data that hasn't been received yet? > >>> Are there still boxes doing this? It's a terrible hack, it breaks every >>> session where a packet gets lost. > >> Yes, there are. It doesn't have to break during loss if the spoofer >> holds the data until it gets the upstream ACK - i.e., if it emulates a >> proxy. > > Who wants to keep that much state? People who make money selling this sort of stuff ;-) > Ideally, there would be a TCP proxy in the middle of the ocean or in the > satellite. Rarely; more typically, the "PEP" is implemented at the endpoints of a subpath that traverse the oceanic/satellite path. Satellites and undersea cables tend to be quite dumb. > The "right" way to do this statelessly would be to spoof a SACK, which > is renegotiable. I don't think it's "right" to spoof anythhing. ;-) Joe -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkoS3BIACgkQE5f5cImnZrufEgCfSgvs5htGlYVP/o8ZFMsiaKcD BDEAn1B67/iA50HVtTFCZPhwsmmw7eDX =88Ye -----END PGP SIGNATURE----- From touch@ISI.EDU Tue May 19 11:33:49 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id D8A9128C194 for ; Tue, 19 May 2009 11:33:49 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.125 X-Spam-Level: X-Spam-Status: No, score=-2.125 tagged_above=-999 required=5 tests=[AWL=0.474, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZYnEAQ8IG5Zy for ; Tue, 19 May 2009 11:33:48 -0700 (PDT) Received: from vapor.isi.edu (vapor.isi.edu [128.9.64.64]) by core3.amsl.com (Postfix) with ESMTP id DD2CA28C180 for ; Tue, 19 May 2009 11:33:48 -0700 (PDT) Received: from [128.9.168.63] (bet.isi.edu [128.9.168.63]) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id n4JIZBnx023760; Tue, 19 May 2009 11:35:13 -0700 (PDT) Message-ID: <4A12FBDF.4020405@isi.edu> Date: Tue, 19 May 2009 11:35:11 -0700 From: Joe Touch User-Agent: Thunderbird 2.0.0.21 (Windows/20090302) MIME-Version: 1.0 To: Bryan Ford References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> <7BE7F3C3-2CD8-4C86-B4A1-4748205B30BE@muada.com> <8A52D0F9-2C40-497D-84F5-13CC0575BF97@mpi-sws.org> <415C2785-40E3-4F6C-8006-AA2420AE357F@mpi-sws.org> In-Reply-To: <415C2785-40E3-4F6C-8006-AA2420AE357F@mpi-sws.org> X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] MPCC and different transports X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 May 2009 18:33:49 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Bryan Ford wrote: ... >> I wouldn't want attackers to be able to slow my traffic down to a crawl. > > On-path attackers can by definition do anything to your traffic, > including cut off connectivity completely, so I assume you're referring > only to off-path attackers here. Defense against such off-path attacks > against performance is indeed important, but that doesn't imply that > such defenses must operate by denying _anyone_ in the network - > including on-path elements - the ability to see or affect > performance-related aspects of flows. ... Well, I agree with "see" the aspects. I disagree with allowing anyone other than endpoints to muck with the endpoint aspects - i.e., presenting fictitous RTTs, window sizes, or ACKs. IPsec and TCP MD5 (and its successor, TCP-AO) are intended to prevent exactly that interference. They are deployed where the endpoints *want* that sort of e2e protection. Sure, you can put the result inside TCP, but then your middlebox is playing with the outer ('link') TCP, not the e2e one, which is fine. >> But can you explain a bit more about these benevolent optimizers? What >> is it that they do, exactly? > > They sit in the path and try to make TCP performance suck less, e.g., > when a mobile network link has a nontrivial non-congestion loss rate, or > when a cellular link cuts out for a short period and causes TCP's > congestion control to back off to zero unnecessarily when it comes back, > or when a link has a high bandwidth-delay product causing TCP take > forever to ramp up... Many companies, including Cisco (look up their > RBSCP documentation) already sell PEPs of various kinds; a lot of these > devices are admittedly horrible hacks of one sort or another, but they > clearly fill a market demand in the same way that NATs and firewalls > do. RFC3515 discusses various goals and techniques such devices > commonly use, as well as the problems they typically cause (including > breaking the end-to-endness of TCP and being completely incompatible > with IPSEC). For some relevant research papers on the principles these > devices were built from, see for example Bakre's I-TCP (Indirect TCP), > Kopparty's Split-TCP, Balakrishnan's TCP Snoop, Martin Swany's > Logistical Session Layer (SC2004), and the various other references > you'll find in section 3.1 of our "Breaking Up the Transport Logjam" paper. PEPs are fine - they're just a virtual link layer. Devices that modify TCP headers, spoof ACKs, or otherwise wouldn't work when the segments are authenticated shouldn't work anyway, IMO. >> That said, you make an excellent point - since SSL over TCP is >>> already ubiquitous and the convention of deploying SSL at application >>> rather than system level is merely a convention and by no means >>> essential to the protocol... >> >> I'd say that proper layering would be to provide authentication below >> the TCP layer and encryption above it, as encryption benefits from >> TCP's ordering, and TCP benefits from IPsec's authentication. > > Perhaps, as long as there remains some provision for in-path devices to > get in the loop when they need to, ideally in a clean and composable > fashion. Who decides when a device "needs" to? IMO, when the endpoint authenticates or encrypts, it is already saying the middlebox neither needs nor is allowed to do this sort of thing... Joe -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFKEvvfE5f5cImnZrsRAo+rAJ40ncjLF77oJ8agt4DvtdvUWiTkLwCfUiHU 3bReI3B/bkOwsfHscsLZadI= =AD2y -----END PGP SIGNATURE----- From baford@mpi-sws.org Tue May 19 11:38:12 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 615713A6FAC for ; Tue, 19 May 2009 11:38:12 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.249 X-Spam-Level: X-Spam-Status: No, score=-6.249 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HELO_EQ_DE=0.35, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6bfepEYYnOvh for ; Tue, 19 May 2009 11:38:11 -0700 (PDT) Received: from hera.mpi-sb.mpg.de (infao0809.mpi-sb.mpg.de [139.19.1.49]) by core3.amsl.com (Postfix) with ESMTP id 0CFE23A6A93 for ; Tue, 19 May 2009 11:38:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mpi-sb.mpg.de; s=mail200803; h=Cc:Message-Id:From:To: In-Reply-To:Content-Type:Content-Transfer-Encoding:Mime-Version: Subject:Date:References; bh=DPKXqrUYUeFIdJEfM/SPK+AoDQZLhLoIygEH g+BCC60=; b=zIftnykpT/7W6zbGGsWhUEC8xvFxZxe0ZDQOalDqFNSqRHlo+iwM r5LJOhqFh3tclakQiqwREOpWRdF9WhQ1KvJN0nYMDiN8LbpSvCAn4sLkNwyjzl76 Syf76T1mvyfQ7QDOMTXw8NELKqW3Wr/DxisGYCcVkvG/9CYtjCWJ9VA= Received: from newzak.mpi-sb.mpg.de ([139.19.1.28]:46498) by hera.mpi-sb.mpg.de (envelope-from ) with esmtp (Exim 4.69) id 1M6UDw-0001tJ-Pj; Tue, 19 May 2009 20:39:40 +0200 Received: from g228086116.adsl.alicedsl.de ([92.228.86.116]:59579 helo=[192.168.1.100]) by newzak.mpi-sb.mpg.de with esmtpsa (TLS-1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.63) (envelope-from ) id 1M6UDw-0007Jw-5z; Tue, 19 May 2009 20:39:36 +0200 Message-Id: From: Bryan Ford To: Joe Touch In-Reply-To: <4A12B6B7.40909@isi.edu> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Tue, 19 May 2009 20:39:35 +0200 References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> <7BE7F3C3-2CD8-4C86-B4A1-4748205B30BE@muada.com> <8A52D0F9-2C40-497D-84F5-13CC0575BF97@mpi-sws.org> <415C2785-40E3-4F6C-8006-AA2420AE357F@mpi-sws.org> <09997716-CAC3-47CD-B2D1-D4A9F9352591@muada.com> <4A12B6B7.40909@isi.edu> X-Mailer: Apple Mail (2.930.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] TCP performance optimizing middleboxes X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 May 2009 18:38:12 -0000 On May 19, 2009, at 3:40 PM, Joe Touch wrote: > Iljitsch van Beijnum wrote: >> On 19 mei 2009, at 15:01, Bryan Ford wrote: >>> or when a link has a high bandwidth-delay product causing TCP take >>> forever to ramp up... >> >> You mean like sending ACKs for data that hasn't been received yet? >> >> Are there still boxes doing this? It's a terrible hack, it breaks >> every >> session where a packet gets lost. > > Yes, there are. It doesn't have to break during loss if the spoofer > holds the data until it gets the upstream ACK - i.e., if it emulates a > proxy. It's still evil (use a nontransparent proxy if that's what you > want), though. There are various flavors of evilness, too, not necessarily all on one continuum. For example, TCP splitting approaches (i.e., outright interposition) are evil because they break TCP's end-to-endness by putting hard state into the network: e.g., if the proxy goes down the connection dies. But at least they are more or less cleanly composable: if a communication path crosses two "interesting" links, each of which has a pair of TCP splitting PEPs on either end using the above tricks to make performance not suck, then there's a decent change the TCP connection crossing both pairs of PEPs will behave in a reasonable fashion and not suck. That's because TCP splitting effectively divides the path into several independent congestion control loops, so that the two path sections crossing the "interesting" links are isolated from each other and the PEPs around their links can do the appropriate magic independent of what's going on in the rest of the path. On the other hand, other approaches are evil for different reasons. For example, in one approach PEPs use to get the TCP sender to ramp up faster and fill up a high-bandwidth-delay-product link, known as TCP splitting, the PEP basically splits an each ACK coming down the return path for a given segment (e.g., bytes 1000-1999) into ACKs for several smaller segments (e.g., 1000-1249, 1250-499, 1500-1749, 1750-1999), in order to trick the TCP sender into increasing its cwnd several times per segment instead of just one. This trick is "not evil" in the end- to-endness sense because it doesn't do anything bad to connection reliability or fate sharing: no data is lost if anything goes wrong, and it doesn't even inherently place any hard state in the network (although ACK splitting PEPs usually maintain hard state for other reasons). On the other hand, this trick is very evil in at least two other ways. First, its behavior under composition can be nasty: for example, if a path happens to cross two high-bandwidth-delay links with ACK splitting PEPs on their endpoints, each one amplifying TCP's aggressiveness, then the combination amplifies TCP's aggressiveness by 16x. Oops. Second, the same technique can be used as an attack by malicious or greedy clients on servers, to gain an unfair share of the server's bandwidth. Hence, we see one ongoing line of research in all the benefits and wonderful ways to use ACK splitting (e.g., Jin, "SPACK: rapid recovery...", '99; Hasegawa, "Receiver-based ACK Splitting...", '07), together with a concurrent line of research on ways to shut down the use of such techniques (e.g., Savage, "TCP congestion control with a misbehaving receiver", 99; Stanojevic, "Can you fool me? ...", 08). What fun! In our "Breaking the Logjam" architecture we take the position that when evil in-network performance enhancement tricks prove necessary, they should be done via the former, cleanly-composable, interposition- based approaches - but only in the Flow Layer, where end-to-end reliability and fate sharing isn't an issue because all Flow Layer state can be reconstructed on demand by the higher-level Transport Layer. Building MPTCP or SCTP++ or whatever atop multiple legacy TCP flows would be consistent with this architecture, effectively removing the primary reason TCP-splitting PEPs are evil. Whether such PEPs are still evil for other reasons depends on one's perspective and on the specific PEP's design. :) Bryan > > Joe > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.9 (MingW32) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iEYEARECAAYFAkoStrYACgkQE5f5cImnZrteOwCfemNuw2XMVr+ij+ub+MQCbtIZ > S9gAnR/68I7Cs2yBiYJT4qmTbnzPyKmB > =Rp3+ > -----END PGP SIGNATURE----- From touch@ISI.EDU Tue May 19 11:48:13 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id F0A1B3A70CD for ; Tue, 19 May 2009 11:48:13 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.153 X-Spam-Level: X-Spam-Status: No, score=-2.153 tagged_above=-999 required=5 tests=[AWL=0.446, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id f1Y3X72VaEFP for ; Tue, 19 May 2009 11:48:12 -0700 (PDT) Received: from vapor.isi.edu (vapor.isi.edu [128.9.64.64]) by core3.amsl.com (Postfix) with ESMTP id D6EDF3A6D5F for ; Tue, 19 May 2009 11:48:12 -0700 (PDT) Received: from [128.9.168.63] (bet.isi.edu [128.9.168.63]) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id n4JInYOC027503; Tue, 19 May 2009 11:49:36 -0700 (PDT) Message-ID: <4A12FF3E.2010008@isi.edu> Date: Tue, 19 May 2009 11:49:34 -0700 From: Joe Touch User-Agent: Thunderbird 2.0.0.21 (Windows/20090302) MIME-Version: 1.0 To: Bryan Ford References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> <7BE7F3C3-2CD8-4C86-B4A1-4748205B30BE@muada.com> <8A52D0F9-2C40-497D-84F5-13CC0575BF97@mpi-sws.org> <415C2785-40E3-4F6C-8006-AA2420AE357F@mpi-sws.org> <09997716-CAC3-47CD-B2D1-D4A9F9352591@muada.com> <4A12B6B7.40909@isi.edu> In-Reply-To: X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] TCP performance optimizing middleboxes X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 May 2009 18:48:14 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Bryan Ford wrote: > On May 19, 2009, at 3:40 PM, Joe Touch wrote: >> Iljitsch van Beijnum wrote: >>> On 19 mei 2009, at 15:01, Bryan Ford wrote: >>>> or when a link has a high bandwidth-delay product causing TCP take >>>> forever to ramp up... >>> >>> You mean like sending ACKs for data that hasn't been received yet? >>> >>> Are there still boxes doing this? It's a terrible hack, it breaks every >>> session where a packet gets lost. >> >> Yes, there are. It doesn't have to break during loss if the spoofer >> holds the data until it gets the upstream ACK - i.e., if it emulates a >> proxy. It's still evil (use a nontransparent proxy if that's what you >> want), though. > > There are various flavors of evilness, too, not necessarily all on one > continuum. > > For example, TCP splitting approaches (i.e., outright interposition) are > evil because they break TCP's end-to-endness by putting hard state into > the network: e.g., if the proxy goes down the connection dies. But at > least they are more or less cleanly composable: if a communication path > crosses two "interesting" links, each of which has a pair of TCP > splitting PEPs on either end using the above tricks to make performance > not suck, then there's a decent change the TCP connection crossing both > pairs of PEPs will behave in a reasonable fashion and not suck. That's > because TCP splitting effectively divides the path into several > independent congestion control loops, so that the two path sections > crossing the "interesting" links are isolated from each other and the > PEPs around their links can do the appropriate magic independent of > what's going on in the rest of the path. As I said, a PEP that emulates a link is fine. Anything that spoofs a TCP segment or modifies a TCP header is not. To that end, making the TCP info visible to the PEP can make a PEP's life easier, though it's still possible to install a PEP over a flakey or non-congestion lossy link and have a benefit even for completely encrypted traffic. ... > In our "Breaking the Logjam" architecture we take the position that when > evil in-network performance enhancement tricks prove necessary, they > should be done via the former, cleanly-composable, interposition-based > approaches - but only in the Flow Layer, where end-to-end reliability > and fate sharing isn't an issue because all Flow Layer state can be > reconstructed on demand by the higher-level Transport Layer. If by "cleanly composeable" you mean "doesn't spoof, and doesn't modify", then I agree. I can't see how you can assert anything about a 'flow layer' that isn't already true about TCP itself - i.e., TCP is a flow, and you shouldn't be interfering with its endpoints. > Building > MPTCP or SCTP++ or whatever atop multiple legacy TCP flows would be > consistent with this architecture, effectively removing the primary > reason TCP-splitting PEPs are evil. Whether such PEPs are still evil > for other reasons depends on one's perspective and on the specific PEP's > design. :) So if the PEP doesn't interfere with your transport (MPTCP, SCTP++), it's OK. But if it interferes with an underlying transport (TCP), that's fine, right? Sounds very NIMBY to me... IMO, if you aren't prepared to have a PEP muck with your transport, you shouldn't endorse it mucking with the transport of others, period. Joe -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFKEv8+E5f5cImnZrsRAlbEAJ90F3xI4mXrQgWNJzZa22q/E5HcFwCgzEz0 jEafbv7shmtw6PZ6Pp1ygbE= =v1oQ -----END PGP SIGNATURE----- From iljitsch@muada.com Tue May 19 11:53:55 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 019D13A6CEB for ; Tue, 19 May 2009 11:53:55 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Q8+gVqUcPXbK for ; Tue, 19 May 2009 11:53:54 -0700 (PDT) Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by core3.amsl.com (Postfix) with ESMTP id E89B13A6B44 for ; Tue, 19 May 2009 11:53:53 -0700 (PDT) Received: from [192.168.2.4] (static-167-138-7-89.ipcom.comunitel.net [89.7.138.167] (may be forged)) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n4JIsxFG015228 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Tue, 19 May 2009 20:55:00 +0200 (CEST) (envelope-from iljitsch@muada.com) Message-Id: <7778620A-1519-4EBB-A524-5AA38120C70E@muada.com> From: Iljitsch van Beijnum To: Bryan Ford In-Reply-To: Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v935.3) Date: Tue, 19 May 2009 20:55:09 +0200 References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> <7BE7F3C3-2CD8-4C86-B4A1-4748205B30BE@muada.com> <8A52D0F9-2C40-497D-84F5-13CC0575BF97@mpi-sws.org> <415C2785-40E3-4F6C-8006-AA2420AE357F@mpi-sws.org> <09997716-CAC3-47CD-B2D1-D4A9F9352591@muada.com> <4A12B6B7.40909@isi.edu> X-Mailer: Apple Mail (2.935.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] TCP performance optimizing middleboxes X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 May 2009 18:53:55 -0000 On 19 mei 2009, at 20:39, Bryan Ford wrote: > in one approach PEPs use to get the TCP sender to ramp up faster and > fill up a high-bandwidth-delay-product link, known as TCP splitting, > the PEP basically splits an each ACK coming down the return path for > a given segment (e.g., bytes 1000-1999) into ACKs for several > smaller segments (e.g., 1000-1249, 1250-499, 1500-1749, 1750-1999), > in order to trick the TCP sender into increasing its cwnd several > times per segment instead of just one. Does this work across a wide variety of implementations? My understanding of RFC 2581 is that an implementation shouldn't react to the _number_ of ACKs but rather to how much data is ACKed. And in the pseudocode of my draft I basically implement a SACK fast retransmit based on the amount of out-of-order data SACKed because the number of ACKs quickly becomes meaningless in a one-ended MPTCP because there will be cross-ACKs between the subflows (one paper, forget which, actually cites this as an important performance enhancing feature) and also a lot of duplicate ACKs. I previously said that the "correct" way to trick a TCP sender into ramping up would be with SACK, but then again I have no idea what an implementation would do when it sees data following the cumulative ACK SACKed without a gap... From janardhan.iyengar@fandm.edu Tue May 19 14:58:30 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 13B7A28C387 for ; Tue, 19 May 2009 14:58:30 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Wvpyu1azPHkz for ; Tue, 19 May 2009 14:58:28 -0700 (PDT) Received: from zimfe2.fandm.edu (zimfe2.fandm.edu [155.68.1.75]) by core3.amsl.com (Postfix) with ESMTP id 4070128C389 for ; Tue, 19 May 2009 14:58:12 -0700 (PDT) Received: from surutti.fandm.edu (unknown [155.68.152.5]) by zimfe2.fandm.edu (Postfix) with ESMTP id 55EE2116802C; Tue, 19 May 2009 17:59:49 -0400 (EDT) Message-ID: <4A132BD4.4040506@fandm.edu> Date: Tue, 19 May 2009 17:59:48 -0400 From: Janardhan Iyengar Organization: Franklin & Marshall College User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Joe Touch References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> <7BE7F3C3-2CD8-4C86-B4A1-4748205B30BE@muada.com> <8A52D0F9-2C40-497D-84F5-13CC0575BF97@mpi-sws.org> <415C2785-40E3-4F6C-8006-AA2420AE357F@mpi-sws.org> <09997716-CAC3-47CD-B2D1-D4A9F9352591@muada.com> <4A12B6B7.40909@isi.edu> <4A12FF3E.2010008@isi.edu> In-Reply-To: <4A12FF3E.2010008@isi.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] TCP performance optimizing middleboxes X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: janardhan.iyengar@fandm.edu List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 May 2009 21:58:30 -0000 Joe, >> In our "Breaking the Logjam" architecture we take the position that when >> evil in-network performance enhancement tricks prove necessary, they >> should be done via the former, cleanly-composable, interposition-based >> approaches - but only in the Flow Layer, where end-to-end reliability >> and fate sharing isn't an issue because all Flow Layer state can be >> reconstructed on demand by the higher-level Transport Layer. > > If by "cleanly composeable" you mean "doesn't spoof, and doesn't > modify", then I agree. I can't see how you can assert anything about a > 'flow layer' that isn't already true about TCP itself - i.e., TCP is a > flow, and you shouldn't be interfering with its endpoints. Just to clarify: our proposal factors out the congestion control functionality out of the transport layer into a flow layer. Our idea is that PEPs do not have to spoof anything if the ends are willing to negotiate with them in the middle. Flow layer congestion control loops do not have to go end-to-end; they go end-middle-middle-...-end, with as many flow layer middleboxes as you want to have in the end-to-end loop. Keep in mind that we do not really care about what congestion control tricks any PEPs may do, we care that they muck with TCP end-to-end semantics. In our architecture, "hard" state (reliability) and stuff of application semantics (the API) are retained in the transport layer, end-to-end. The middleboxes/PEPs can do performance tuning at the flow layer, and do not have to retain any hard state. If a PEP drops off, our end-to-end TCP connection can re-sync over a new/different flow layer instance. Thinking of the multipath discussion we are having on this list, MPTCP (with some tweaks) would nicely fit our architecture's new "transport" layer, and TCP would nicely fit our new "flow" layer. Which is what Bryan was trying to say in the following: >> Building >> MPTCP or SCTP++ or whatever atop multiple legacy TCP flows would be >> consistent with this architecture, effectively removing the primary >> reason TCP-splitting PEPs are evil. Whether such PEPs are still evil >> for other reasons depends on one's perspective and on the specific PEP's >> design. :) > So if the PEP doesn't interfere with your transport (MPTCP, SCTP++), > it's OK. But if it interferes with an underlying transport (TCP), that's > fine, right? Sounds very NIMBY to me... What our architecture does is makes it so that PEPs can actively engage with the ends at the flow layer (using TCP or DCCP or whathaveyou), and the transports can remain disengaged from this conversation. Again, the PEPs care about congestion control and performance; they don't care about transport semantics. I don't see why they would want to muck in transport layer semantics (acks and reliability) if we can let them cleanly optimize for performance. - jana -- Janardhan Iyengar Assistant Professor, Computer Science Franklin & Marshall College http://www.fandm.edu/jiyengar From touch@ISI.EDU Tue May 19 15:09:28 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 8951828C1D6 for ; Tue, 19 May 2009 15:09:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.269 X-Spam-Level: X-Spam-Status: No, score=-2.269 tagged_above=-999 required=5 tests=[AWL=0.330, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MoEmEgK+2RUj for ; Tue, 19 May 2009 15:09:27 -0700 (PDT) Received: from vapor.isi.edu (vapor.isi.edu [128.9.64.64]) by core3.amsl.com (Postfix) with ESMTP id 73B573A6B30 for ; Tue, 19 May 2009 15:09:27 -0700 (PDT) Received: from [128.9.184.226] ([128.9.184.226]) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id n4JMAWbR015800; Tue, 19 May 2009 15:10:34 -0700 (PDT) Message-ID: <4A132E58.30506@isi.edu> Date: Tue, 19 May 2009 15:10:32 -0700 From: Joe Touch User-Agent: Thunderbird 2.0.0.21 (Windows/20090302) MIME-Version: 1.0 To: janardhan.iyengar@fandm.edu References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> <7BE7F3C3-2CD8-4C86-B4A1-4748205B30BE@muada.com> <8A52D0F9-2C40-497D-84F5-13CC0575BF97@mpi-sws.org> <415C2785-40E3-4F6C-8006-AA2420AE357F@mpi-sws.org> <09997716-CAC3-47CD-B2D1-D4A9F9352591@muada.com> <4A12B6B7.40909@isi.edu> <4A12FF3E.2010008@isi.edu> <4A132BD4.4040506@fandm.edu> In-Reply-To: <4A132BD4.4040506@fandm.edu> X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] TCP performance optimizing middleboxes X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 May 2009 22:09:28 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Janardhan Iyengar wrote: > Joe, ... > Just to clarify: our proposal factors out the congestion control > functionality out of the transport layer into a flow layer. TCP doesn't factor them out; they're integrated. > Our idea is > that PEPs do not have to spoof anything if the ends are willing to > negotiate with them in the middle. Yes. But TCP isn't so willing. > Flow layer congestion control loops > do not have to go end-to-end; they go end-middle-middle-...-end, with as > many flow layer middleboxes as you want to have in the end-to-end loop. Well, you're saying that hop-by-hop congestion control is sufficient, rather than E2E. You can have that debate with Clark/Reed/Saltzer. > Keep in mind that we do not really care about what congestion control > tricks any PEPs may do, we care that they muck with TCP end-to-end > semantics. I would hope that you care that they do NOT muck with TCP E2E semantics. The point is that TCP E2E semantics includes E2E flow control. > In our architecture, "hard" state (reliability) and stuff of application > semantics (the API) are retained in the transport layer, end-to-end. > The middleboxes/PEPs can do performance tuning at the flow layer, and do > not have to retain any hard state. If a PEP drops off, our end-to-end > TCP connection can re-sync over a new/different flow layer instance. It sounds like your TCP isn't our TCP. > Thinking of the multipath discussion we are having on this list, MPTCP > (with some tweaks) would nicely fit our architecture's new "transport" > layer, and TCP would nicely fit our new "flow" layer. You say "transport", I say "transport". You say "flow", I say "hop". I.e., it seems to me you're using TCP as a link layer (been talking to the DTN folks? ;-) ... >>> Building >>> MPTCP or SCTP++ or whatever atop multiple legacy TCP flows would be >>> consistent with this architecture, effectively removing the primary >>> reason TCP-splitting PEPs are evil. Whether such PEPs are still evil >>> for other reasons depends on one's perspective and on the specific PEP's >>> design. :) > >> So if the PEP doesn't interfere with your transport (MPTCP, SCTP++), >> it's OK. But if it interferes with an underlying transport (TCP), that's >> fine, right? Sounds very NIMBY to me... > > What our architecture does is makes it so that PEPs can actively engage > with the ends at the flow layer (using TCP or DCCP or whathaveyou), and > the transports can remain disengaged from this conversation. Again, the > PEPs care about congestion control and performance; they don't care > about transport semantics. I don't see why they would want to muck in > transport layer semantics (acks and reliability) if we can let them > cleanly optimize for performance. Right - but the issue is whether when you see TCP you can know it's just a hop version of the E2E TCP. If you can't tell the difference, you shouldn't be playing with it, since that TCP might just be the transport layer you said you didn't want to bother. Joe -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkoTLlgACgkQE5f5cImnZrsi5gCfXoib3xi5WvPYG+StVXoH5JU0 UBwAnR6VRc+eYH1n+DoemFQHRHSn3yQ7 =4vh4 -----END PGP SIGNATURE----- From janardhan.iyengar@fandm.edu Tue May 19 15:17:32 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 9FC0228C380 for ; Tue, 19 May 2009 15:17:32 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0s5qUC1RZoGh for ; Tue, 19 May 2009 15:17:31 -0700 (PDT) Received: from zimfe2.fandm.edu (zimfe2.fandm.edu [155.68.1.75]) by core3.amsl.com (Postfix) with ESMTP id C177728C393 for ; Tue, 19 May 2009 15:16:38 -0700 (PDT) Received: from surutti.fandm.edu (unknown [155.68.152.5]) by zimfe2.fandm.edu (Postfix) with ESMTP id DEC20116802C; Tue, 19 May 2009 18:18:14 -0400 (EDT) Message-ID: <4A133026.9030909@fandm.edu> Date: Tue, 19 May 2009 18:18:14 -0400 From: Janardhan Iyengar Organization: Franklin & Marshall College User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Iljitsch van Beijnum References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> <7BE7F3C3-2CD8-4C86-B4A1-4748205B30BE@muada.com> <8A52D0F9-2C40-497D-84F5-13CC0575BF97@mpi-sws.org> <415C2785-40E3-4F6C-8006-AA2420AE357F@mpi-sws.org> <09997716-CAC3-47CD-B2D1-D4A9F9352591@muada.com> <4A12B6B7.40909@isi.edu> <7778620A-1519-4EBB-A524-5AA38120C70E@muada.com> In-Reply-To: <7778620A-1519-4EBB-A524-5AA38120C70E@muada.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] TCP performance optimizing middleboxes X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: janardhan.iyengar@fandm.edu List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 May 2009 22:17:32 -0000 Iljitsch, > My understanding of RFC 2581 is that an implementation shouldn't react > to the _number_ of ACKs but rather to how much data is ACKed. RFC 2581 allows it (see Savage's paper that Bryan mentioned). But yes, Appropriate Byte Counting (ABC, RFC 3465) fixes it to actually count the number of bytes acked... I don't know the deployment status of ABC (I think there may have been a study that looked at several things including ABC deployment on webservers...) > And in the pseudocode of my draft I basically implement a SACK fast > retransmit based on the amount of out-of-order data SACKed because the > number of ACKs quickly becomes meaningless in a one-ended MPTCP because > there will be cross-ACKs between the subflows (one paper, forget which, > actually cites this as an important performance enhancing feature) and > also a lot of duplicate ACKs. > Absolutely. I think that is necessary to do because the MPTCP sender is *causing* the reordering (by splitting the flow). And I don't know if you can at all do MPTCP without SACK. I'll also note that yes, the number of dupacks increases as well with interesting side-effects... one simply that the number of packets on the return path will increase (delayed-acks are sent roughly once every two packets, and dupacks are sent for every packet received out of order). We use an algorithm in CMT to reduce the number of dupacks without affecting the number of dupacks required for fast-retransmissions. We do use a bit in the header however to do this. > I previously said that the "correct" way to trick a TCP sender into > ramping up would be with SACK, but then again I have no idea what an > implementation would do when it sees data following the cumulative ACK > SACKed without a gap... Which is exactly why the correct way to trick a TCP sender into ramping up would be to not do it. :-) Tricks exploit protocol "gullibilities", and as much interest and work that goes into identifying these gullibilities and exploiting them for in-loop performance benefits, that much interest and work goes into identifying and fixing these gullibilities. Interestingly, neither of these groups is trying to actively compete with the other, and yet there is this arms-race! - jana -- Janardhan Iyengar Assistant Professor, Computer Science Franklin & Marshall College http://www.fandm.edu/jiyengar From L.Wood@surrey.ac.uk Tue May 19 23:27:28 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id EF92C28C0FB for ; Tue, 19 May 2009 23:27:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.247 X-Spam-Level: X-Spam-Status: No, score=-6.247 tagged_above=-999 required=5 tests=[AWL=0.351, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id v163Uct5+NVZ for ; Tue, 19 May 2009 23:27:26 -0700 (PDT) Received: from mail72.messagelabs.com (mail72.messagelabs.com [193.109.255.147]) by core3.amsl.com (Postfix) with SMTP id F09573A70B5 for ; Tue, 19 May 2009 23:27:25 -0700 (PDT) X-VirusChecked: Checked X-Env-Sender: L.Wood@surrey.ac.uk X-Msg-Ref: server-10.tower-72.messagelabs.com!1242800940!53828303!1 X-StarScan-Version: 6.0.0; banners=-,-,- X-Originating-IP: [131.227.102.140] Received: (qmail 8113 invoked from network); 20 May 2009 06:29:00 -0000 Received: from ads40.surrey.ac.uk (HELO ads40.surrey.ac.uk) (131.227.102.140) by server-10.tower-72.messagelabs.com with SMTP; 20 May 2009 06:29:00 -0000 Received: from EVS-EC1-NODE4.surrey.ac.uk ([131.227.102.145]) by ads40.surrey.ac.uk with Microsoft SMTPSVC(6.0.3790.3959); Wed, 20 May 2009 07:29:00 +0100 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C9D914.4230964C" Date: Wed, 20 May 2009 07:28:58 +0100 Message-ID: <4835AFD53A246A40A3B8DA85D658C4BE7B102E@EVS-EC1-NODE4.surrey.ac.uk> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [multipathtcp] TCP performance optimizing middleboxes Thread-Index: AcnYzsCzX8q6B4fDRpGPOlki5kb7QAARJmEY References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> <7BE7F3C3-2CD8-4C86-B4A1-4748205B30BE@muada.com> <8A52D0F9-2C40-497D-84F5-13CC0575BF97@mpi-sws.org> <415C2785-40E3-4F6C-8006-AA2420AE357F@mpi-sws.org> <09997716-CAC3-47CD-B2D1-D4A9F9352591@muada.com> <4A12B6B7.40909@isi.edu> <4A12FF3E.2010008@isi.edu> <4A132BD4.4040506@fandm.edu> <4A132E58.30506@isi.edu> From: To: , X-OriginalArrivalTime: 20 May 2009 06:29:00.0214 (UTC) FILETIME=[42EC8560:01C9D914] Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] TCP performance optimizing middleboxes X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 May 2009 06:27:29 -0000 This is a multi-part message in MIME format. ------_=_NextPart_001_01C9D914.4230964C Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable > > Flow layer congestion control loops > > do not have to go end-to-end; they go end-middle-middle-...-end, = with as > > many flow layer middleboxes as you want to have in the end-to-end = loop.=20 > > Well, you're saying that hop-by-hop congestion control is sufficient, No, he's not. the middleboxes have a view of two different subnets. And he's still mentioning an end-to-end loop. > rather than E2E. You can have that debate with Clark/Reed/Saltzer. ...should Clark/Reed/Saltzer ever write a treatise on congestion = control. Their end-to-end papers are about reliability and performance, not about the somewhat different congestion control, fairness and competition. Rhetoric much, Joe? L. > Keep in mind that we do not really care about what congestion control > tricks any PEPs may do, we care that they muck with TCP end-to-end > semantics. I would hope that you care that they do NOT muck with TCP E2E semantics. The point is that TCP E2E semantics includes E2E flow control. > In our architecture, "hard" state (reliability) and stuff of = application > semantics (the API) are retained in the transport layer, end-to-end.=20 > The middleboxes/PEPs can do performance tuning at the flow layer, and = do > not have to retain any hard state. If a PEP drops off, our end-to-end > TCP connection can re-sync over a new/different flow layer instance. It sounds like your TCP isn't our TCP. > Thinking of the multipath discussion we are having on this list, MPTCP > (with some tweaks) would nicely fit our architecture's new "transport" > layer, and TCP would nicely fit our new "flow" layer.=20 You say "transport", I say "transport". You say "flow", I say "hop". I.e., it seems to me you're using TCP as a link layer (been talking to the DTN folks? ;-) ... >>> Building >>> MPTCP or SCTP++ or whatever atop multiple legacy TCP flows would be >>> consistent with this architecture, effectively removing the primary >>> reason TCP-splitting PEPs are evil. Whether such PEPs are still = evil >>> for other reasons depends on one's perspective and on the specific = PEP's >>> design. :) >=20 >> So if the PEP doesn't interfere with your transport (MPTCP, SCTP++), >> it's OK. But if it interferes with an underlying transport (TCP), = that's >> fine, right? Sounds very NIMBY to me... >=20 > What our architecture does is makes it so that PEPs can actively = engage > with the ends at the flow layer (using TCP or DCCP or whathaveyou), = and > the transports can remain disengaged from this conversation. Again, = the > PEPs care about congestion control and performance; they don't care > about transport semantics. I don't see why they would want to muck in > transport layer semantics (acks and reliability) if we can let them > cleanly optimize for performance. Right - but the issue is whether when you see TCP you can know it's just a hop version of the E2E TCP. If you can't tell the difference, you shouldn't be playing with it, since that TCP might just be the transport layer you said you didn't want to bother. Joe -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkoTLlgACgkQE5f5cImnZrsi5gCfXoib3xi5WvPYG+StVXoH5JU0 UBwAnR6VRc+eYH1n+DoemFQHRHSn3yQ7 =3D4vh4 -----END PGP SIGNATURE----- _______________________________________________ multipathtcp mailing list multipathtcp@ietf.org https://www.ietf.org/mailman/listinfo/multipathtcp ------_=_NextPart_001_01C9D914.4230964C Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable RE: [multipathtcp] TCP performance optimizing middleboxes

> > Flow layer congestion control loops
> > do not have to go end-to-end; they go = end-middle-middle-...-end, with as
> > many flow layer middleboxes as you want to have in the = end-to-end loop.
>
> Well, you're saying that hop-by-hop congestion control is = sufficient,

No, he's not. the middleboxes have a view of two different subnets.
And he's still mentioning an end-to-end loop.

> rather than E2E. You can have that debate with = Clark/Reed/Saltzer.

...should Clark/Reed/Saltzer ever write a treatise on congestion = control.

Their end-to-end papers are about reliability and performance,
not about the somewhat different congestion control, fairness and
competition.

Rhetoric much, Joe?

L.

<http://www.ee.surrey= .ac.uk/Personal/L.Wood/><L.Wood@surrey.ac.uk>

> Keep in mind that we do not really care about what congestion = control
> tricks any PEPs may do, we care that they muck with TCP = end-to-end
> semantics.

I would hope that you care that they do NOT muck with TCP E2E = semantics.
The point is that TCP E2E semantics includes E2E flow control.

> In our architecture, "hard" state (reliability) and stuff = of application
> semantics (the API) are retained in the transport layer, = end-to-end.
> The middleboxes/PEPs can do performance tuning at the flow layer, = and do
> not have to retain any hard state.  If a PEP drops off, our = end-to-end
> TCP connection can re-sync over a new/different flow layer = instance.

It sounds like your TCP isn't our TCP.

> Thinking of the multipath discussion we are having on this list, = MPTCP
> (with some tweaks) would nicely fit our architecture's new = "transport"
> layer, and TCP would nicely fit our new "flow" layer.

You say "transport", I say "transport".

You say "flow", I say "hop".

I.e., it seems to me you're using TCP as a link layer (been talking = to
the DTN folks? ;-)

...
>>> Building
>>> MPTCP or SCTP++ or whatever atop multiple legacy TCP flows = would be
>>> consistent with this architecture, effectively removing the = primary
>>> reason TCP-splitting PEPs are evil.  Whether such PEPs = are still evil
>>> for other reasons depends on one's perspective and on the = specific PEP's
>>> design. :)
>
>> So if the PEP doesn't interfere with your transport (MPTCP, = SCTP++),
>> it's OK. But if it interferes with an underlying transport = (TCP), that's
>> fine, right? Sounds very NIMBY to me...
>
> What our architecture does is makes it so that PEPs can actively = engage
> with the ends at the flow layer (using TCP or DCCP or whathaveyou), = and
> the transports can remain disengaged from this conversation.  = Again, the
> PEPs care about congestion control and performance; they don't = care
> about transport semantics.  I don't see why they would want to = muck in
> transport layer semantics (acks and reliability) if we can let = them
> cleanly optimize for performance.

Right - but the issue is whether when you see TCP you can know it's = just
a hop version of the E2E TCP. If you can't tell the difference, you
shouldn't be playing with it, since that TCP might just be the = transport
layer you said you didn't want to bother.

Joe
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkoTLlgACgkQE5f5cImnZrsi5gCfXoib3xi5WvPYG+StVXoH5JU0
UBwAnR6VRc+eYH1n+DoemFQHRHSn3yQ7
=3D4vh4
-----END PGP SIGNATURE-----
_______________________________________________
multipathtcp mailing list
multipathtcp@ietf.org
https://www.i= etf.org/mailman/listinfo/multipathtcp

------_=_NextPart_001_01C9D914.4230964C-- From touch@ISI.EDU Wed May 20 07:46:34 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 172A53A6822 for ; Wed, 20 May 2009 07:46:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.503 X-Spam-Level: X-Spam-Status: No, score=-2.503 tagged_above=-999 required=5 tests=[AWL=0.096, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eX7cVs-ZQt4N for ; Wed, 20 May 2009 07:46:33 -0700 (PDT) Received: from vapor.isi.edu (vapor.isi.edu [128.9.64.64]) by core3.amsl.com (Postfix) with ESMTP id 1B3363A6801 for ; Wed, 20 May 2009 07:46:33 -0700 (PDT) Received: from [192.168.1.47] (pool-71-106-119-240.lsanca.dsl-w.verizon.net [71.106.119.240]) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id n4KElSm0016633; Wed, 20 May 2009 07:47:30 -0700 (PDT) Message-ID: <4A141800.60809@isi.edu> Date: Wed, 20 May 2009 07:47:28 -0700 From: Joe Touch User-Agent: Thunderbird 2.0.0.21 (Windows/20090302) MIME-Version: 1.0 To: L.Wood@surrey.ac.uk References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> <7BE7F3C3-2CD8-4C86-B4A1-4748205B30BE@muada.com> <8A52D0F9-2C40-497D-84F5-13CC0575BF97@mpi-sws.org> <415C2785-40E3-4F6C-8006-AA2420AE357F@mpi-sws.org> <09997716-CAC3-47CD-B2D1-D4A9F9352591@muada.com> <4A12B6B7.40909@isi.edu> <4A12FF3E.2010008@isi.edu> <4A132BD4.4040506@fandm.edu> <4A132E58.30506@isi.edu> <4835AFD53A246A40A3B8DA85D658C4BE7B102E@EVS-EC1-NODE4.surrey.ac.uk> In-Reply-To: <4835AFD53A246A40A3B8DA85D658C4BE7B102E@EVS-EC1-NODE4.surrey.ac.uk> X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] TCP performance optimizing middleboxes X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 May 2009 14:46:34 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 L.Wood@surrey.ac.uk wrote: > > >> > Flow layer congestion control loops >> > do not have to go end-to-end; they go end-middle-middle-...-end, with as >> > many flow layer middleboxes as you want to have in the end-to-end loop. >> >> Well, you're saying that hop-by-hop congestion control is sufficient, > > No, he's not. the middleboxes have a view of two different subnets. > And he's still mentioning an end-to-end loop. Not for "flows"; besides, you basically agree that HBH is what he's doing below. Why are you disagreeing here? > >> rather than E2E. You can have that debate with Clark/Reed/Saltzer. > > ...should Clark/Reed/Saltzer ever write a treatise on congestion control. > > Their end-to-end papers are about reliability and performance, > not about the somewhat different congestion control, fairness and > competition. Their paper is about the composition of services. If you have an argument that some services are indeed composeable end-to-end, the please make that case in detail. Merely asserting that C/R/S doesn't apply for these services is not an argument; it is an (unsubstantiated) assertion. Joe -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkoUGAAACgkQE5f5cImnZruKGgCdEe/G6utEmQktqdpCU9BlSN6u XwUAn3IMC8h+w/dMgp626w0i4Co464Zv =HKuo -----END PGP SIGNATURE----- From janardhan.iyengar@fandm.edu Wed May 20 10:18:53 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id E9C033A6B57 for ; Wed, 20 May 2009 10:18:53 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id W1CGTQnOgeCt for ; Wed, 20 May 2009 10:18:52 -0700 (PDT) Received: from zimfe2.fandm.edu (zimfe2.fandm.edu [155.68.1.75]) by core3.amsl.com (Postfix) with ESMTP id 5EA103A6A30 for ; Wed, 20 May 2009 10:18:52 -0700 (PDT) Received: from surutti.fandm.edu (dhcp-155-68-61-189.fandm.edu [155.68.61.189]) by zimfe2.fandm.edu (Postfix) with ESMTP id 38431116802C; Wed, 20 May 2009 13:20:29 -0400 (EDT) Message-ID: <4A143BDC.60107@fandm.edu> Date: Wed, 20 May 2009 13:20:28 -0400 From: Janardhan Iyengar Organization: Franklin & Marshall College User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Joe Touch References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> <7BE7F3C3-2CD8-4C86-B4A1-4748205B30BE@muada.com> <8A52D0F9-2C40-497D-84F5-13CC0575BF97@mpi-sws.org> <415C2785-40E3-4F6C-8006-AA2420AE357F@mpi-sws.org> <09997716-CAC3-47CD-B2D1-D4A9F9352591@muada.com> <4A12B6B7.40909@isi.edu> <4A12FF3E.2010008@isi.edu> <4A132BD4.4040506@fandm.edu> <4A132E58.30506@isi.edu> In-Reply-To: <4A132E58.30506@isi.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] TCP performance optimizing middleboxes X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list Reply-To: janardhan.iyengar@fandm.edu List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 May 2009 17:18:54 -0000 Joe, >> Our idea is >> that PEPs do not have to spoof anything if the ends are willing to >> negotiate with them in the middle. > > Yes. But TCP isn't so willing. Yes, that is true, and TCP isn't so willing to do multipath either. But that is what we are trying to do here. Along the same lines, TCP isn't so willing to talk to PEPs right now, and we think that should change. To be clear, we are talking about factoring congestion control out of the transport into a "Flow Layer" that interacts with middleboxes, and leave the rest of the stuff that applications care about (relaibility, order preservation, flow control, etc.) at the ends in the end-to-end transport layer (or as we call it, the "Semantic" Layer.) So, we are saying that TCP has both of these functionalities, and we are splitting them into different layers. More details at: "Breaking Up The Transport Logjam", at http://web1.fandm.edu/jiyengar/papers/layers-hotnets2008.pdf >> Flow layer congestion control loops >> do not have to go end-to-end; they go end-middle-middle-...-end, with as >> many flow layer middleboxes as you want to have in the end-to-end loop. > > Well, you're saying that hop-by-hop congestion control is sufficient, > rather than E2E. You can have that debate with Clark/Reed/Saltzer. Saltzer/Reed/Clark do not talk about congestion control, they talk only about reliability. Noel Chiappa has a nice article called "Will The Real "End-End Principle" Please Stand Up?" (http://mercury.lcs.mit.edu/~jnc/tech/end_end.html), which you may already seen, and it should help clarify this issue. Note that we are not pushing reliability into the network, only the performance-related function of congestion control. This is in stark contrast with the PEPs that are in operation today and openly flout the e2e principle by "transparently" interposing on the e2e path. We are, proposing to pull our collective head out of the sand and see these PEPs, understand why they exist and why anyone cares to spend the thousands of dollars to make and/or buy them, and let the ends actively negotiate with them. PEPs only care about performance (hence the subject of this thread), which is not a bad thing for anyone! It becomes bad because our transports (TCP) conflate end-to-end, application-facing semantics---in particular, reliability---with the performance-related functions. And for PEPs to do their performance tweaks, they must figure out how to get around TCP semantics. We propose to take TCP semantics out of the PEP loop, into an end-to-end loop, and let the PEPs optimize away. MPTCP is very much like our end-to-end "Semantic" Layer, and the TCP that they want to use is somewhat like our "Flow Layer", with the difference that it does not actively negotiate with the PEPs. But it is designed with the idea that the PEPs will do their optimizations, while e2e reliability will happen at the MPTCP level with DSNs and other such. Let me note that the TCP in the two-sided mulipath draft is not like the TCP we know either; for one, use of the re-sync message changes completely the reliability expectations you would have from TCP. When you start doing retransmissions on paths different from that of the original transmission (which you will have to do to manage path failure), you will have to change the TCP state machine. > I would hope that you care that they do NOT muck with TCP E2E semantics. > The point is that TCP E2E semantics includes E2E flow control. Yes, but we are talking about congestion control, not flow control. Flow control continues to be at the ends. >> In our architecture, "hard" state (reliability) and stuff of application >> semantics (the API) are retained in the transport layer, end-to-end. >> The middleboxes/PEPs can do performance tuning at the flow layer, and do >> not have to retain any hard state. If a PEP drops off, our end-to-end >> TCP connection can re-sync over a new/different flow layer instance. > > It sounds like your TCP isn't our TCP. That is probably right... our "Semantic Layer" (what I referred to, admittedly ambiguously, as TCP) does not do congestion control, much like MPTCP. Our Flow Layer interacts with the middleboxes and does congestion control. > Right - but the issue is whether when you see TCP you can know it's just > a hop version of the E2E TCP. If you can't tell the difference, you > shouldn't be playing with it, since that TCP might just be the transport > layer you said you didn't want to bother. I'm not sure I understand this point. For middleboxes to actively participate, there would need to be explicit negotiation of Flow (congestion control) instances on the end-to-end path, and that is how a PEP would be able to tell the difference. Or did you mean something else? - jana -- Janardhan Iyengar Assistant Professor, Computer Science Franklin & Marshall College http://www.fandm.edu/jiyengar From touch@ISI.EDU Wed May 20 10:32:35 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id F025E28C145 for ; Wed, 20 May 2009 10:32:35 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qt6-ZGDgC48L for ; Wed, 20 May 2009 10:32:34 -0700 (PDT) Received: from vapor.isi.edu (vapor.isi.edu [128.9.64.64]) by core3.amsl.com (Postfix) with ESMTP id C5DF93A6E10 for ; Wed, 20 May 2009 10:31:47 -0700 (PDT) Received: from [75.217.36.208] (208.sub-75-217-36.myvzw.com [75.217.36.208]) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id n4KHX8Pk029332; Wed, 20 May 2009 10:33:10 -0700 (PDT) Message-ID: <4A143ED4.8090905@isi.edu> Date: Wed, 20 May 2009 10:33:08 -0700 From: Joe Touch User-Agent: Thunderbird 2.0.0.21 (Windows/20090302) MIME-Version: 1.0 To: janardhan.iyengar@fandm.edu References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> <7BE7F3C3-2CD8-4C86-B4A1-4748205B30BE@muada.com> <8A52D0F9-2C40-497D-84F5-13CC0575BF97@mpi-sws.org> <415C2785-40E3-4F6C-8006-AA2420AE357F@mpi-sws.org> <09997716-CAC3-47CD-B2D1-D4A9F9352591@muada.com> <4A12B6B7.40909@isi.edu> <4A12FF3E.2010008@isi.edu> <4A132BD4.4040506@fandm.edu> <4A132E58.30506@isi.edu> <4A143BDC.60107@fandm.edu> In-Reply-To: <4A143BDC.60107@fandm.edu> X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] TCP performance optimizing middleboxes X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 May 2009 17:32:36 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Janardhan Iyengar wrote: > Joe, > >>> Our idea is >>> that PEPs do not have to spoof anything if the ends are willing to >>> negotiate with them in the middle. >> >> Yes. But TCP isn't so willing. > > Yes, that is true, and TCP isn't so willing to do multipath either. But > that is what we are trying to do here. Along the same lines, TCP isn't > so willing to talk to PEPs right now, and we think that should change. > > To be clear, we are talking about factoring congestion control out of > the transport into a "Flow Layer" that interacts with middleboxes, and > leave the rest of the stuff that applications care about (relaibility, > order preservation, flow control, etc.) at the ends in the end-to-end > transport layer (or as we call it, the "Semantic" Layer.) So, we are > saying that TCP has both of these functionalities, and we are splitting > them into different layers. > > More details at: "Breaking Up The Transport Logjam", at > http://web1.fandm.edu/jiyengar/papers/layers-hotnets2008.pdf ... >> Right - but the issue is whether when you see TCP you can know it's just >> a hop version of the E2E TCP. If you can't tell the difference, you >> shouldn't be playing with it, since that TCP might just be the transport >> layer you said you didn't want to bother. > > I'm not sure I understand this point. For middleboxes to actively > participate, there would need to be explicit negotiation of Flow > (congestion control) instances on the end-to-end path, and that is how a > PEP would be able to tell the difference. Or did you mean something else? Let's put it this way. First show me a system that makes this work and talks to middleboxes. Then we can talk about whether it makes sense to call it TCP or not. AFAICT, the problem with middleboxes is that you want to play nice with them, but they don't want to play nice with anybody. They want to work whether you want them there or not, whether you talk to them or not, etc. That is nonsensical, and it's impossible to understand how to make a protocol that expects to interact with a party that deliberately won't play by the rules. I.e., a middlebox that wants to play nice is, IMO, called a proxy. A proxy wants you to know its there, and won't play if you don't. Proxies work fine for doing the hop-by-hop stuff you're talking about. Everything else is "making laws for the lawless", and (IMO) has been a consistent waste of energy in the IETF. Joe -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkoUPtQACgkQE5f5cImnZrsAPgCfWDb81dfL+pvTUniV0Kpc1D7u xMgAn0RHFDiDDGcLaFuYJAC9QjOb2Hkn =zYMf -----END PGP SIGNATURE----- From L.Wood@surrey.ac.uk Wed May 20 10:49:05 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id C9F803A6D65 for ; Wed, 20 May 2009 10:49:05 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.297 X-Spam-Level: X-Spam-Status: No, score=-6.297 tagged_above=-999 required=5 tests=[AWL=0.301, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0I28TROb1zQY for ; Wed, 20 May 2009 10:49:04 -0700 (PDT) Received: from mail115.messagelabs.com (mail115.messagelabs.com [195.245.231.179]) by core3.amsl.com (Postfix) with SMTP id C39213A67F0 for ; Wed, 20 May 2009 10:49:03 -0700 (PDT) X-VirusChecked: Checked X-Env-Sender: L.Wood@surrey.ac.uk X-Msg-Ref: server-7.tower-115.messagelabs.com!1242841839!23503793!1 X-StarScan-Version: 6.0.0; banners=-,-,- X-Originating-IP: [131.227.102.140] Received: (qmail 1948 invoked from network); 20 May 2009 17:50:39 -0000 Received: from ads40.surrey.ac.uk (HELO ads40.surrey.ac.uk) (131.227.102.140) by server-7.tower-115.messagelabs.com with SMTP; 20 May 2009 17:50:39 -0000 Received: from EVS-EC1-NODE4.surrey.ac.uk ([131.227.102.145]) by ads40.surrey.ac.uk with Microsoft SMTPSVC(6.0.3790.3959); Wed, 20 May 2009 18:50:39 +0100 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C9D973.7C2E7CE9" Date: Wed, 20 May 2009 18:48:59 +0100 Message-ID: <4835AFD53A246A40A3B8DA85D658C4BE7B1030@EVS-EC1-NODE4.surrey.ac.uk> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [multipathtcp] TCP performance optimizing middleboxes Thread-Index: AcnZWg1IsVj7u4evRIGhXof9o/tDZgAGTQty References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> <7BE7F3C3-2CD8-4C86-B4A1-4748205B30BE@muada.com> <8A52D0F9-2C40-497D-84F5-13CC0575BF97@mpi-sws.org> <415C2785-40E3-4F6C-8006-AA2420AE357F@mpi-sws.org> <09997716-CAC3-47CD-B2D1-D4A9F9352591@muada.com> <4A12B6B7.40909@isi.edu> <4A12FF3E.2010008@isi.edu> <4A132BD4.4040506@fandm.edu> <4A132E58.30506@isi.edu> <4835AFD53A246A40A3B8DA85D658C4BE7B102E@EVS-EC1-NODE4.surrey.ac.uk> <4A141800.60809@isi.edu> From: To: X-OriginalArrivalTime: 20 May 2009 17:50:39.0207 (UTC) FILETIME=[7C9F9770:01C9D973] Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] TCP performance optimizing middleboxes X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 May 2009 17:49:05 -0000 This is a multi-part message in MIME format. ------_=_NextPart_001_01C9D973.7C2E7CE9 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Joe Touch said: > Lloyd Wood said: >> Joe Touch said >>> rather than E2E. You can have that debate with Clark/Reed/Saltzer. >>=20 >> ...should Clark/Reed/Saltzer ever write a treatise on congestion = control. >>=20 >> Their end-to-end papers are about reliability and performance, >> not about the somewhat different congestion control, fairness and >> competition. > > Their paper is about the composition of services. ...despite, oddly, not mentioning services. L. DTN work: http://info.ee.surrey.ac.uk/Personal/L.Wood/saratoga/ ------_=_NextPart_001_01C9D973.7C2E7CE9 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable RE: [multipathtcp] TCP performance optimizing middleboxes

Joe Touch said:
> Lloyd Wood said:
>> Joe Touch said
>>> rather than E2E. You can have that debate with = Clark/Reed/Saltzer.
>>
>> ...should Clark/Reed/Saltzer ever write a treatise on = congestion control.
>>
>> Their end-to-end papers are about reliability and = performance,
>> not about the somewhat different congestion control, fairness = and
>> competition.
>
> Their paper is about the composition of services.

...despite, oddly, not mentioning services.

L.

DTN work: http://inf= o.ee.surrey.ac.uk/Personal/L.Wood/saratoga/

<http://info.ee.surr= ey.ac.uk/Personal/L.Wood/><L.Wood@surrey.ac.uk>

------_=_NextPart_001_01C9D973.7C2E7CE9-- From touch@ISI.EDU Wed May 20 11:11:49 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 1A5CA3A6EDD for ; Wed, 20 May 2009 11:11:49 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vW+qsiWQGjxy for ; Wed, 20 May 2009 11:11:48 -0700 (PDT) Received: from vapor.isi.edu (vapor.isi.edu [128.9.64.64]) by core3.amsl.com (Postfix) with ESMTP id 593CD3A68A8 for ; Wed, 20 May 2009 11:11:42 -0700 (PDT) Received: from [75.217.36.208] (208.sub-75-217-36.myvzw.com [75.217.36.208]) by vapor.isi.edu (8.13.8/8.13.8) with ESMTP id n4KICvOu009698; Wed, 20 May 2009 11:12:59 -0700 (PDT) Message-ID: <4A144829.6080808@isi.edu> Date: Wed, 20 May 2009 11:12:57 -0700 From: Joe Touch User-Agent: Thunderbird 2.0.0.21 (Windows/20090302) MIME-Version: 1.0 To: L.Wood@surrey.ac.uk References: <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> <7BE7F3C3-2CD8-4C86-B4A1-4748205B30BE@muada.com> <8A52D0F9-2C40-497D-84F5-13CC0575BF97@mpi-sws.org> <415C2785-40E3-4F6C-8006-AA2420AE357F@mpi-sws.org> <09997716-CAC3-47CD-B2D1-D4A9F9352591@muada.com> <4A12B6B7.40909@isi.edu> <4A12FF3E.2010008@isi.edu> <4A132BD4.4040506@fandm.edu> <4A132E58.30506@isi.edu> <4835AFD53A246A40A3B8DA85D658C4BE7B102E@EVS-EC1-NODE4.surrey.ac.uk> <4A141800.60809@isi.edu> <4835AFD53A246A40A3B8DA85D658C4BE7B1030@EVS-EC1-NODE4.surrey.ac.uk> In-Reply-To: <4835AFD53A246A40A3B8DA85D658C4BE7B1030@EVS-EC1-NODE4.surrey.ac.uk> X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-ISI-4-43-8-MailScanner: Found to be clean X-MailScanner-From: touch@isi.edu Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] TCP performance optimizing middleboxes X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 May 2009 18:11:49 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 L.Wood@surrey.ac.uk wrote: > Joe Touch said: >> Lloyd Wood said: >>> Joe Touch said >>>> rather than E2E. You can have that debate with Clark/Reed/Saltzer. >>> >>> ...should Clark/Reed/Saltzer ever write a treatise on congestion control. >>> >>> Their end-to-end papers are about reliability and performance, >>> not about the somewhat different congestion control, fairness and >>> competition. >> >> Their paper is about the composition of services. > > ...despite, oddly, not mentioning services. It doesn't use the word "composition" either. I guess you've dismissed Baran's papers because they don't mention packets, too? Joe -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkoUSCkACgkQE5f5cImnZrtAAACeL52RQjh3lO/MYdmJBnfM1lmK J58AoKbgoM4unZ5eXMaMJxDKahHf5Gyf =aubV -----END PGP SIGNATURE----- From iljitsch@muada.com Thu May 21 02:18:51 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id B648128C0F3 for ; Thu, 21 May 2009 02:18:51 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.599 X-Spam-Level: X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4ieCx-Vr7qg3 for ; Thu, 21 May 2009 02:18:50 -0700 (PDT) Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by core3.amsl.com (Postfix) with ESMTP id C91A628C0E1 for ; Thu, 21 May 2009 02:18:49 -0700 (PDT) Received: from claw.it.uc3m.es (claw.it.uc3m.es [163.117.139.52]) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n4L9JVBP028168 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Thu, 21 May 2009 11:19:31 +0200 (CEST) (envelope-from iljitsch@muada.com) Message-Id: From: Iljitsch van Beijnum To: janardhan.iyengar@fandm.edu In-Reply-To: <4A143BDC.60107@fandm.edu> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v935.3) Date: Thu, 21 May 2009 11:19:42 +0200 References: <4A0CA1D0.4040802@fe.up.pt><61D9B622-947A-4E99-92CC-F5212C3F07F8@mpi-sws.org> <2181C5F19DD0254692452BFF3EAF1D6807AC5298@rsys005a.comm.ad.roke.co.uk> <5B15D831-19A9-486B-8FD5-8C733305F5D7@mpi-sws.org> <7BE7F3C3-2CD8-4C86-B4A1-4748205B30BE@muada.com> <8A52D0F9-2C40-497D-84F5-13CC0575BF97@mpi-sws.org> <415C2785-40E3-4F6C-8006-AA2420AE357F@mpi-sws.org> <09997716-CAC3-47CD-B2D1-D4A9F9352591@muada.com> <4A12B6B7.40909@isi.edu> <4A12FF3E.2010008@isi.edu> <4A132BD4.4040506@fandm.edu> <4A132E58.30506@isi.edu> <4A143BDC.60107@fandm.edu> X-Mailer: Apple Mail (2.935.3) Cc: multipathtcp@ietf.org Subject: Re: [multipathtcp] TCP performance optimizing middleboxes X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 May 2009 09:18:51 -0000 On 20 mei 2009, at 19:20, Janardhan Iyengar wrote: > More details at: "Breaking Up The Transport Logjam", at http://web1.fandm.edu/jiyengar/papers/layers-hotnets2008.pdf Wow, that thing goes on forever, seemingly making the same points 50 times... Today, boxes in the middle of the network have two mechanisms to slow down flows, one of which is in wide use and the other isn't: dropping packets and ECN. What we don't have is a mechanism to make flows ramp up faster than they otherwise would. Too bad we're basically using 2 bits for ECN to provide a 1-bit function, otherwise we could have done this with ECN, too. But a better way would be to have a few more bits so a router can tell a flow how fast it may go, based on the available capacity at the router. That way, a flow reaches the optimal speed after a roundtrip. It's such a shame that the IPv4 and especially IPv6 IP option mechanisms are so clunky that they're as good as unusable to implement something like this. Ok, maybe it's time to get back to multipath now... From marcelo@it.uc3m.es Thu May 21 20:54:53 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id C88393A6B59 for ; Thu, 21 May 2009 20:54:53 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.978 X-Spam-Level: X-Spam-Status: No, score=-5.978 tagged_above=-999 required=5 tests=[AWL=0.621, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rIo1QNGXwUVw for ; Thu, 21 May 2009 20:54:52 -0700 (PDT) Received: from smtp02.uc3m.es (smtp02.uc3m.es [163.117.176.132]) by core3.amsl.com (Postfix) with ESMTP id 74BFF3A6EE5 for ; Thu, 21 May 2009 20:54:51 -0700 (PDT) Received: from r190-135-25-50.dialup.adsl.anteldata.net.uy (r190-135-25-50.dialup.adsl.anteldata.net.uy [190.135.25.50]) by smtp02.uc3m.es (Postfix) with ESMTP id 792EB6BACA1; Fri, 22 May 2009 05:56:13 +0200 (CEST) Message-ID: <4A162258.7080300@it.uc3m.es> Date: Fri, 22 May 2009 05:56:08 +0200 From: marcelo bagnulo braun User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: multipathtcp@ietf.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-TM-AS-Product-Ver: IMSS-7.0.0.3116-5.6.0.1016-16656.004 Subject: [multipathtcp] [Fwd: MPTCP BOF request for IETF 75] X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 May 2009 03:54:53 -0000 We have sent the BOF request to the TSV ADs. See email attached below... -------- Mensaje original -------- Asunto: MPTCP BOF request for IETF 75 Fecha: Thu, 21 May 2009 22:12:20 +0200 De: marcelo bagnulo braun Para: Lars Eggert , 'Magnus Westerlund' Hi, We would like to request a MPTCP BOF for the next ietf. General information, including the ml can be found: http://trac.tools.ietf.org/area/tsv/trac/wiki/MultipathTcp An updated BOF description is attached below. As prior related efforts, we had several presentations about this topic in previous ietf meetings (Iljitsch van Beijnum and Mark Handley made presentations about this in the TSV area and RRG) There has been considerable amount of discussion in the ml. There are a couple of draft (included in the wiki) as well as a couple of papers and presentation included in the wiki as well. Multipath TCP (MPTCP) BOF Recent research has suggested that by making better use of the multiple paths between endpoints we can better utilise the resources within the network and thus improve resource pooling. In particular, recent results include models of how multipath-capable end-systems might respond to congestion signals, and what the potential benefits might be (see [1] for a more details and references). If end systems could spread their load across multiple paths in the right way, with the right reaction to the right congestion signals from the network, then traffic would quickly move away from congested or failed links in favor of available uncongested links. It is fairly intuitive to see that when multipath transport is used in conjunction with congestion-dependent load-distribution, it is possible to push more traffic through the network and to achieve enhanced fault tolerance. Traffic will flow throught any available path that has spare capacity, filling unused resources, while moving away from failures and congested resources. The result is a more flexible network, capable of functioning well, even when the actual traffic matrix differs substantially from the traffic matrix envisaged when the network was designed for. Currently used transport protocols use only one path at a time for sending data. In particular TCP and DCCP use only one path and they respond to congestion reducing their sending rate. SCTP on the other hand, is aware of multiple paths, but it uses only one at the time, reserving the alternative paths as backups in case of failure. The proposed work is to define the mechanisms to enable transport protocols to use multiple paths simultaneously and to distribute the load among the subflows of each path based on congestion. In particular, the proposed work is to extend TCP to support simultaneous data exchange through multiple paths. Two complementary approaches are proposed: On one hand, to define a two-ended multipath TCP that uses multiple addresses a-la SCTP to obtain multiple paths and that requires support from both endpoints to use them. On the other hand, single ended multipath TCP, where only one of the endpoints involved in the communication is multipath aware and relies on multipath routing capabilities provided by the IP routing to deliver data towards a legacy TCP endpoint. These two approaches provide a smooth deployment model, starting with the one-ended multipath TCP, that only requires one end to deploy it and it results in immediate benefits, and then when multipath TCP support becomes pervasive, using the two-ended multipath TCP to obtain all the potential benefits. The proposed work items are: - A set of architectural guidelines for congestion dependent multipath transport protocols. Since different transport protocols can potentially benefit from this approach, this document describes the motivations and the general approach that should be followed to enable congestion dependent multipath transport. - Extensions to current TCP to support the one-ended mutipath TCP. - Extensions to current TCP to support the two-ended mutipath TCP. - A threat analysis for multipath TCP - An extended API describing how applications can use multipath transport. The API should also provide access to multipath information and enable the control of the usage.. The initial goal is that the documents produced by the WG will be either experimental or informational. However, if when the documents are ready the technology seems mature enough, the WG will consider if it is appropriate to ask the IESG to advance the document as a Proposed Standard. With respect to the work on the support for multipath in TCP, the only acceptable modifications to the TCP packet format are in the form of new TCP options. [1] D. Wischik, M. Handley, M. Bagnulo, The resource pooling principle, ACM SIGCOMM Computer Communication Review archive, October 2008 From iljitsch@muada.com Wed May 27 04:11:21 2009 Return-Path: X-Original-To: multipathtcp@core3.amsl.com Delivered-To: multipathtcp@core3.amsl.com Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 685E03A69D3 for ; Wed, 27 May 2009 04:11:21 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -6.548 X-Spam-Level: X-Spam-Status: No, score=-6.548 tagged_above=-999 required=5 tests=[AWL=0.051, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pZZ0g2DTKFig for ; Wed, 27 May 2009 04:11:18 -0700 (PDT) Received: from sequoia.muada.com (sequoia.muada.com [83.149.65.1]) by core3.amsl.com (Postfix) with ESMTP id C9A6B3A6C90 for ; Wed, 27 May 2009 04:11:10 -0700 (PDT) Received: from claw.it.uc3m.es (claw.it.uc3m.es [163.117.139.52]) (authenticated bits=0) by sequoia.muada.com (8.13.3/8.13.3) with ESMTP id n4RBBAV0064170 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO) for ; Wed, 27 May 2009 13:11:11 +0200 (CEST) (envelope-from iljitsch@muada.com) Message-Id: <5EBA85DF-FC90-4017-88F9-B59BF4BDB227@muada.com> From: Iljitsch van Beijnum To: multipathtcp@ietf.org Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v935.3) Date: Wed, 27 May 2009 13:11:26 +0200 X-Mailer: Apple Mail (2.935.3) Subject: [multipathtcp] Path selection by routers X-BeenThere: multipathtcp@ietf.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: Multi-path extensions for TCP List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 May 2009 11:11:21 -0000 I want to preface this message with the statement that I don't think this is something that should be part of the BoF or a prospective working group _at_ _this_ _time_ for fear of biting off more than we can chew. Having said that, I think it's an important issue that we need to deal with at some point if we start work on multipath transports, and perhaps thinking ahead a bit will make joining these two parts together easier when the time comes. I'm going to assume that certain routers can reach a given destination over more than one path. For instance, in OSPF you can use multiple paths if they have the same cost and in BGP you can use multiple paths if you've configured your router for this and certain path attributes are the same (this is vendor specific). In these cases, you'll see multiple entries for a given destination in the RIB and FIB and the forwarding engine has to pick one for each packet. The usual way to do this is by doing a hash over a bunch of header fields (often addresses, ports and protocol) and then sending all the packets in the same hash bucket over the same path to avoid reordering sessions [ECMP = equal cost multipath, RFC2992]. Ideally, a multipath transport would simply use a different value for one of the fields used in the ECMP hash and have a good chance that those packets will take a different path. Unfortunately, none of the fields used for this are modifyable in transit. In IPv6 there is a chance that ECMP will look at the addresses and flow label rather than port numbers, but I haven't heard of any implementations that actually do this. The flow label is supposed to stay the same for a session but I think we can deviate from that without too much trouble. In any event, just influencing ECMP is probably not sufficient for our purposes. A better solution would be to have an explicit path selector field in packets, where the path selector value directly maps to a path. First question: what if two or more routers look at the path selector? Would that be suboptimal if we use a single path selector, do we need to have multiple, possibly one for every hop? Wouldn't that make the path selector field too large, the processing too complex, the number of paths too large to probe? But the situation where one flipped bit means that 10 routers all change their path selection seems suboptimal, too. Also, what if we wrap around? I.e., path selector says path 4 but there are only 3. If only one router looks at the value it can drop the packet and the host won't try to use the path anymore after one or more additional attempts, if multiple routers look at the value then the packet can't be dropped. Another issue is where to put the path selector. The first question here is whether the path selector needs to be something that's visible end-to-end. If it's end-to-end the path selector must be unmodifyable in transit and in the IP or transport header. This means an option. Although not completely impossible, using IP options in IPv4 or IPv6 is an uphill battle, where we expect many routers to drop packets with options. Not to mention firewalls and the destination hosts themselves. (Note though that in our case that is not immediately lethal as we always have the default path so failure just means no extra paths, not loss of connectivity.) TCP options on the other hand seem much more doable, although not entirely impervious to firewall breakage. But do we want routers to have to look this deep into packets? On the other hand, having a path selector that is only valid in part of the network could be attractive. We could use the DSCP bits in this case. There are 64 combinations so that would allow for 8 traffic classes and 8 paths or some other combination, retaining the original DSCP use along with the new multipath use. It would even be possible to make incompatible changes to IP or link layers as long as those are negotiated between a host and a router or two routers that are both upgraded and the original IP headers are restored when the packet leaves the upgraded part of the network. This would even allow for true multipath routing protocols. For instance, BGP could be upgraded to send multiple paths to its neighbors and then the desired path can be selected with the path selector in packets.