Subject: [Iccrg] Modular/Pluggable TCP Congestion Control for FreeBSD
From: [email-masked] (Lawrence Stewart)
Date: Wed Dec 19 05:31:39 2007

Hi all,

We've been involved in a research project to implement and test an
emerging TCP congestion control algorithm under FreeBSD. As a part of
this, we've put together a patch for FreeBSD 7.0-BETA4 that modularises
the congestion control code in the TCP stack. It allows for new
congestion control algorithms to be developed as loadable kernel modules.

This improves FreeBSD's usefulness as a TCP research platform and makes
it easier to customise the stack for specific scenarios like high
bandwidth, long delay paths.

There is an accompanying technical report "Light-Weight Modular
TCP Congestion Control for FreeBSD 7" [1] that covers the design,
features, kernel interface and usage of the framework. Also on our 
website is
a beta release of a module that implements the H-TCP[2] congestion control
algorithm proposed by the Hamilton Institute.

We believe that modular congestion control is a worthwhile addition to
FreeBSD. We've performed significant internal testing and there are
currently no known issues or regressions with the implementation
compared to a 'vanilla' FreeBSD 7.0-BETA4 kernel. We would welcome
further review and testing from the wider community in the hope of 
getting this
patch folded into FreeBSD 8-CURRENT.

SIFTR [3], our tool for monitoring FreeBSD kernel TCP connection state, 
has also
received a minor update to v1.1.5, with the addition of 6 new, useful 
variables.

All code and documentation is available on our website[3].

Cheers,
Jim and Lawrence
http://caia.swin.edu.au

[1] http://caia.swin.edu.au/reports/071218A/CAIA-TR-071218A.pdf
[2] http://www.hamilton.ie/net/htcp3.pdf
[3] http://caia.swin.edu.au/urp/newtcp/tools.html

From: [email-masked] (Dino-Martin LOPEZ-PACHECO)
Date: Mon Dec 17 13:16:12 2007
Subject: [Iccrg] Comments about CUBIC draft
Message-ID: <[email-masked]>

Hi all,

My name is Dino and I'm a new member in this group.

I have read the draft about the CUBIC protocol and here you are some
comments that I think can be useful for future discussions.

****** Comments concerning the writing quality ******

The CUBIC draft  (draft-rhee-tcpm-cubic-00.txt) is in general clear and
understandable. But

1. The draft is missing standards language (SHOULD, MUST, etc)
2. The used parameters must be clearly documented (the C parameter needs
more explanation - why C = 0.4?).
3. In Section 2.2 (TCP-friendly region) the authors write: "If cwnd is
less than W_tcp(t), then the protocol is in the TCP friendly region...",
while in the paper "CUBIC: A New TCP-Friendly High-Speed TCP Variant",
the authors say that CUBIC is in the TCP-friendly region when W(t+RTT)
is less than W_tcp(t). Since cwnd is not equal to W(t+RTT) I think both
arguments are different and a clarification about this point is needed.

****** Comments concerning the CUBIC protocol mechanisms ******

CUBIC in an extension of the current standard TCP, to improve the
performance of TCP in wire-based large bandwidth * delay product
networks. The new mechanisms proposed by CUBIC modify only the window
increase function in the sender side, to get high link utilization in a
large variety of network conditions. Thus, CUBIC does not impact the
Internet security and reacts as standard TCP in case of congestion
collapses, routing changes, link disconnections, intermittent links or
mobility events.

Moreover, CUBIC directly impact the throughput and the fairness. On the
one hand :

1. Some experiments (http://netsrv.csc.ncsu.edu/highspeed/) show that
CUBIC flows sharing the bottleneck converge to the fairness point, but
the needed time for converging can be large. For instance, in the paper
"Experimental Evaluation of CUBIC-TCP", the authors show that 2 flows
could need up to 500s before getting fairness in a 500Mbps bottleneck
link, with an RTT of 200ms. This phenomenon can be also observed in the
experiences presented in the web address before cited.
2. Experiments about CUBIC show also that, in many cases, CUBIC flows
degrade the performance of standard TCP. The unfairness between CUBIC
and TCP increases as the RTT increases
(http://netsrv.csc.ncsu.edu/wiki/index.php/Testing_between_Korea_and_Japan,
http://netsrv.csc.ncsu.edu/highspeed/).

But on the other hand, there are some important arguments to consider
CUBIC as a congestion protocol which can be safely deployed in the Internet:

1.- It has been proved mathematically and experimentally that CUBIC is
less aggressive than High Speed TCP, and HSTCP hasn't been known
to exhibit severe issues in deployment or testing experiences (HSTCP has
already an experimental RFC status).
2.- Several experiments have shown that CUBIC is less aggressive than
BIC, and BIC is used by most Linux-based systems since year 2005.
(Currently I don't know severe problems derived from the use of BIC in
Internet...)

Even if high speed protocols, like High Speed TCP, BIC or CUBIC, can
potentially introduce serious congestion and unfairness problems due to
their aggressiveness, the throughput of such high speed protocols stay
limited by the TCP buffer size (and the link capacity in the end hosts).
Now, we must consider that in most cases, such buffer size is
initialized with a very small value (e.g. the default maximum TCP buffer
size is 128KB in most Linux-based systems). Thus, I believe the damaged
caused by CUBIC to standard TCP flows, which is less aggressive than
HSTCP and BIC, will be very limited. Therefore, I believe this protocol
can be safely deployed in the Internet.

******************

Cheers,

Dino M. LOPEZ P.

From: [email-masked] (Michael Welzl)
Date: Wed Dec  5 06:51:27 2007
Subject: [Iccrg] ctcp review: big picture issues (3 of 4)
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <[email-masked]>

> I am not sure what the RG's job is in general.  With regards to
> reviewing this for TCPM the job of the RG is, as far as I understand it,
> to evaluate the safety statement in the document.  In this document that
> statement does not say it is safe enough to play with, it says it "is
> safe to deploy on the current Internet".

The way I see it, this is the RG's main concern, and...

> I would like to think that the RG provides more than simply this review
> for TCPM and that in fact other technical comments about proposed CC
> schemes would be in bounds.

...surely such comments are in bounds.

cheers,
Michael

From: [email-masked] (Mark Allman)
Date: Tue Dec  4 23:33:05 2007
Subject: [Iccrg] ctcp review: big picture issues (3 of 4)
In-Reply-To: <[email-masked]>
Message-ID: <[email-masked]>

> If you look at our gamma auto-tuning paper there are a number of such
> experiments. 

Can you make that reference a tad more specific?

Thanks,
allman

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 185 bytes
Desc: not available
Url : http://oakham.cs.ucl.ac.uk/pipermail/iccrg/attachments/20071204/61d91bf4/attachment.bin
>From [email-masked]  Tue Dec  4 23:38:44 2007
From: [email-masked] (Mark Allman)
Date: Tue Dec  4 23:34:01 2007
Subject: [Iccrg] ctcp review: big picture issues (3 of 4)
In-Reply-To: <[email-masked]>
Message-ID: <[email-masked]>

Oh, and ...

> mark including a summary of experiments is still in progress

A summary of the experiments that are documented in a form that we can
all look at would be *much* appreciated.

allman

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 185 bytes
Desc: not available
Url : http://oakham.cs.ucl.ac.uk/pipermail/iccrg/attachments/20071204/f3ea34e5/attachment.bin
>From [email-masked]  Tue Dec  4 23:40:25 2007
From: [email-masked] (Murari Sridharan)
Date: Tue Dec  4 23:35:24 2007
Subject: [Iccrg] ctcp review: big picture issues (3 of 4)
In-Reply-To: <[email-masked]>
References: <[email-masked]>
	<[email-masked]>
Message-ID: <[email-masked]>

Sorry, that reference is to the ICCRG Wiki

http://www3.tools.ietf.org/group/irtf/trac/wiki/ICCRG_ctcp

The link is gamma parameter tuning.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: Tuesday, December 04, 2007 3:38 PM
To: Murari Sridharan
Cc: Lachlan Andrew; [email-masked]
Subject: Re: [Iccrg] ctcp review: big picture issues (3 of 4)

> If you look at our gamma auto-tuning paper there are a number of such
> experiments.

Can you make that reference a tad more specific?

Thanks,
allman

From: [email-masked] (Mark Allman)
Date: Tue Dec  4 23:18:22 2007
Subject: [Iccrg] ctcp review: big picture issues (3 of 4)
In-Reply-To: <[email-masked]>
Message-ID: <[email-masked]>

> I think that the "experiments with only random loss" are incomplete
> rather than inappropriate in themselves.  

Well, fair enough.  I should have been more careful.  If these were a
first set of experiments and there were others then OK.  Perhaps I
should have said that my opinion is that such experiments are "not
sufficient" and should not have said "inappropriate".

> I assume that they are modelling the case in which current TCP fails.
> That is the case in which a long flow is repeatedly interrupted by
> slow-start of short-lived connections, giving losses which cause the
> window to be reduced, but not hanging around long enough to give link
> utilization.  They're taking the short-cut of just forcing the
> sporadic losses.  It is no more or less valid than considering a
> "response function" which considers losses without specifying where
> the come from.  These tests are also useful show that the congestion
> control doesn't do as much harm when there is no congestion as current
> TCP does.

OK, but if these tests are designed to show that there is no problem
when there is no congestion then I can't make a determination of whether
I think a **congestion control** scheme is "safe" or not.  If you don't
test a congestion controller in the presence of congestion then it is a
non-starter in my book.  Maybe I am weird.

> 1. I didn't think our job way to say how often there will be a benefit
> in real networks.  I thought our job was to say whether they're
> "allowed" to do experiments in real networks to find out if there is a
> significant benefit.

I am not sure what the RG's job is in general.  With regards to
reviewing this for TCPM the job of the RG is, as far as I understand it,
to evaluate the safety statement in the document.  In this document that
statement does not say it is safe enough to play with, it says it "is
safe to deploy on the current Internet".

I would like to think that the RG provides more than simply this review
for TCPM and that in fact other technical comments about proposed CC
schemes would be in bounds.

> 2. CTCP isn't as sensitive to the actual queue estimate as algorithms
> like Vegas are.  Vegas sets its rate based on the estimated queue
> size, while CTCP simply detects whether or not to be aggressive based
> on whether or not it thinks the queue is empty.  For that, it isn't
> clear that we need a correlation between our own number of packets in
> flight and our precise observed queueing delay.  That means it will
> only be aggressive (but no more than existing experimental RFCs) if it
> has a   reason   to think the link is underutilized.

Perhaps you're right.  I am not sure I feel like I understand it well
enough to weigh in on that.  But, certainly if this is the case it'd be
nice to see.

I still beleve my high-order bit, however.  If you have no contention in
the network (or, at least little and an unassessed amount) and yet are
relying on that contention (in terms of queue buildup) as part of the
controller then how can that be an effective evaluation?

allman

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 185 bytes
Desc: not available
Url : http://oakham.cs.ucl.ac.uk/pipermail/iccrg/attachments/20071204/ae3b631d/attachment.bin
>From [email-masked]  Tue Dec  4 23:35:20 2007
From: [email-masked] (Murari Sridharan)
Date: Tue Dec  4 23:30:09 2007
Subject: [Iccrg] ctcp review: big picture issues (3 of 4)
In-Reply-To: <[email-masked]>
References: <[email-masked]>
	<[email-masked]>
Message-ID: <[email-masked]>

Due to internal deadlines I am pretty slow in catching up, in fact my rebuttal mail to mark including a summary of experiments is still in progress. In any case we did do a number of tests for both random losses *and* congestion both represent very useful data points. Random losses represents the showcase for all high-speed congestion control algorithms since it clearly makes the case for the need for such algorithms and how TCP fails. Having said that I agree that congestion due to queue build up is very essential to test delay based algorithms (although as Lachlan notes below the effect it has varies depending on the specific delay controller). If you look at our gamma auto-tuning paper there are a number of such experiments.

Thanks

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Mark Allman
Sent: Tuesday, December 04, 2007 3:23 PM
To: Lachlan Andrew
Cc: [email-masked]
Subject: Re: [Iccrg] ctcp review: big picture issues (3 of 4)

> I think that the "experiments with only random loss" are incomplete
> rather than inappropriate in themselves.

Well, fair enough.  I should have been more careful.  If these were a first set of experiments and there were others then OK.  Perhaps I should have said that my opinion is that such experiments are "not sufficient" and should not have said "inappropriate".

> I assume that they are modelling the case in which current TCP fails.
> That is the case in which a long flow is repeatedly interrupted by
> slow-start of short-lived connections, giving losses which cause the
> window to be reduced, but not hanging around long enough to give link
> utilization.  They're taking the short-cut of just forcing the
> sporadic losses.  It is no more or less valid than considering a
> "response function" which considers losses without specifying where
> the come from.  These tests are also useful show that the congestion
> control doesn't do as much harm when there is no congestion as current
> TCP does.

OK, but if these tests are designed to show that there is no problem when there is no congestion then I can't make a determination of whether I think a **congestion control** scheme is "safe" or not.  If you don't test a congestion controller in the presence of congestion then it is a non-starter in my book.  Maybe I am weird.

> 1. I didn't think our job way to say how often there will be a benefit
> in real networks.  I thought our job was to say whether they're
> "allowed" to do experiments in real networks to find out if there is a
> significant benefit.

I am not sure what the RG's job is in general.  With regards to reviewing this for TCPM the job of the RG is, as far as I understand it, to evaluate the safety statement in the document.  In this document that statement does not say it is safe enough to play with, it says it "is safe to deploy on the current Internet".

I would like to think that the RG provides more than simply this review for TCPM and that in fact other technical comments about proposed CC schemes would be in bounds.

> 2. CTCP isn't as sensitive to the actual queue estimate as algorithms
> like Vegas are.  Vegas sets its rate based on the estimated queue
> size, while CTCP simply detects whether or not to be aggressive based
> on whether or not it thinks the queue is empty.  For that, it isn't
> clear that we need a correlation between our own number of packets in
> flight and our precise observed queueing delay.  That means it will
> only be aggressive (but no more than existing experimental RFCs) if it
> has a   reason   to think the link is underutilized.

Perhaps you're right.  I am not sure I feel like I understand it well enough to weigh in on that.  But, certainly if this is the case it'd be nice to see.

I still beleve my high-order bit, however.  If you have no contention in the network (or, at least little and an unassessed amount) and yet are relying on that contention (in terms of queue buildup) as part of the controller then how can that be an effective evaluation?

allman

From: [email-masked] (Lachlan Andrew)
Date: Tue Dec  4 21:26:52 2007
Subject: [Iccrg] ctcp review: big picture issues (3 of 4)
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <[email-masked]>

Greetings Mark,

I think that the "experiments with only random loss" are incomplete
rather than inappropriate in themselves.  If my interpretation is
correct, it should probably be made more explicit in the draft.

I assume that they are   modelling   the case in which current TCP
fails.  That is the case in which a long flow is repeatedly
interrupted by slow-start of short-lived connections, giving losses
which cause the window to be reduced, but not hanging around long
enough to give link utilization.  They're taking the short-cut of just
forcing the sporadic losses.  It is no more or less valid than
considering a "response function" which considers losses without
specifying where the come from.  These tests are also useful show that
the congestion control doesn't do as much harm when there is no
congestion as current TCP does.

I think they chose this scenario because it is the "worst case" --
where it is most aggressive.  They show that in this case, it is no
more aggressive than an existing experimental RFC.  In other cases it
is less aggressive.

To me, that is (a model of) one interesting scenario, although many
other scenarios should also be tested eventually.

Regarding the correlation between delay and congestion:
1. I didn't think our job way to say how often there will be a benefit
in real networks.  I thought our job was to say whether they're
"allowed" to do experiments in real networks to   find out   if there
is a significant benefit.
2. CTCP isn't as sensitive to the actual queue estimate as algorithms
like Vegas are.  Vegas sets its rate based on the estimated queue
size, while CTCP simply detects whether or not to be aggressive based
on whether or not it thinks the queue is empty.  For that, it isn't
clear that we need a correlation between our own number of packets in
flight and our precise observed queueing delay.  That means it will
only be aggressive (but no more than existing experimental RFCs) if it
has a   reason   to think the link is underutilized.  Imperfect as it
may be, this is a step in the right direction compared with the
existing RFCs.

$0.02,
Lachlan

On 29/11/2007, Mark Allman <[email-masked]> wrote:
>   - As I said in another note, a summary of the experiments would be
>     useful.
>
>     But, beyond this, I skimmed through the infocom and pfldnet papers
>     (the latter which I read in detail some time back) and I have a
>     pretty big problem with the experiments.  The problem is when the
>     network is setup with dummynet to induce random losses with some
>     probability then these schemes run in a network that does not
>     include congestion.  And, I don't understand how *congestion
>     control* can be effectively evaluated in the absence of
>     *congestion*.  So,
>
>       + Loss != congestion.  Enough congestion causes loss, for sure.
>         But, loss can happen for all sorts of reasons.  And, congestion
>         causes more than loss (e.g., increased queue sizes and delays).
>
>       + It somehow seems non-sensical to not have actual congestion with
>         actual queue build-up when assessing delay-based congestion
>         control.  I am not saying that there was none in the
>         experiments, but we are at least led to believe that the queue
>         was able to absorb all the congestion.  If that is the case then
>         the contention wasn't really all that great.
>
>     For me, all this adds up to being pretty skeptical of the overall
>     results that show CTCP to be "better".
>
>     For sure, CTCP performs better in this environment, but that is to
>     be expected because it is more aggressive.  I don't feel like I have
>     an accurate sense of the implications of this aggressiveness.
>
>     Further, there is this paper that is perhaps preliminary and perhaps
>     about old data, but that shows that congestion window size and delay
>     are not correlated.  Every time I think about delay-based congestion
>     control I think about this paper.  If there is no correlation in
>     real networks (where this data is from) then how is this scheme
>     going to work?
>
>       Is the Round-trip Time Correlated with the Number of Packets in
>       Flight?
>       Saad Biaz, Auburn University
>       Nitin H. Vaidya, University of Illinois at Urbana-Champaign
>       Internet Measurement Conference, 2003

-- 
Lachlan Andrew  Dept of Computer Science, Caltech
1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA
Ph: +1 (626) 395-8820    Fax: +1 (626) 568-3603
http://netlab.caltech.edu/~lachlan

From: [email-masked] (Michael Welzl)
Date: Mon Dec  3 07:17:35 2007
Subject: [Iccrg] ctcp review: big picture issues (3 of 4)
In-Reply-To: <[email-masked]>
References: <[email-masked]> <[email-masked]> <[email-masked]>
Message-ID: <[email-masked]>

On Thu, 2007-11-29 at 14:18 -0800, Lachlan Andrew wrote:
> On 29/11/2007, [email-masked] <[email-masked]> wrote:
> > On Thu, Nov 29, 2007 at 01:13:31PM -0500, Mark Allman wrote:
> > Would it be correct to say that we think for a scheme to be called "safe for
> > production Internet use" that its parameters need to all be accompanied
> > by some particular values or guidance on a range of values?
> 
> Yep, I'd vote for that as a requirement.  The range could be specified
> as "arbitrary", provided there was a justification that "arbitrary"
> values are safe.

I'd also vote for that.

Cheers,
Michael

From: [email-masked] (Nandita Dukkipati)
Date: Sat Dec  1 09:07:44 2007
Subject: [Iccrg] Reminder: Congestion Control train-wrecks workshop at Stanford (Call for Demos)
Message-ID: <[email-masked]>

Hi.

This is a gentle reminder that the deadline for the 1-page demo submissions is
approaching. Please email your demos by the end of this weekend (2 Dec.).

-Nandita
-----------------------------------------------------------------------------------------------------------

We would like to invite you to a two-day congestion control workshop,
titled "Train-wrecks and congestion control: What happens if we do
nothing?", which we are hosting at Stanford on 31 March - 1 April, 2008.

Spurred on by a widespread belief that TCP is showing its age and
needs replacing - and a deeper understanding of the dynamics of
congestion control - the research community has brought forward many
new congestion control algorithms. There has been lots of debate about
the relative merits and demerits of the new schemes; and a
standardization effort is under way in the IETF.

But before the next congestion control mechanism is deployed, it will
need to be deployed widely in operating systems and - in some cases -
in switches and routers too. This will be a long road, requiring the
buy-in of many people: Researchers, product developers and business
leaders too. Our own experience of proposing new congestion control
algorithms has been met with the challenge: "Show me the compelling
need for a new congestion control mechanisms?", and "What will really
happen to the Internet (and my business) if we keep TCP just the way
it is?"

As a community, we need examples that are simple to understand, and
demonstrate a compelling need for change. We call them the "Train
wreck scenarios". Examples might show that distribution of video over
wireless in the home will come to a halt without new algorithms. Or
that P2P traffic will bring the whole network crashing down. Or that
huge, high-performance data-centers need new algorithms. Whatever your
favorite example, we believe that if we are collectively armed with a
handful of mutually agreed examples, it will be much easier to make a
business case for change. Or put another way, if we can't articulate
compelling examples to industry leaders, then is the cost and risk of
change worth it?

The goal of the workshop is to identify a handful of really compelling
demonstrations of the impending train-wreck. The outcome will be a set
of canonical examples that we will use to persuade industry of the
need for change.

We are deliberately inviting you many months ahead of time - to give
you time to create your compelling train-wreck demonstration. You can
choose the way you present your demonstration: You could bring
equipment and show a live-demo; you could show simulations or
animations; or you could produce a video showing a real or synthetic
demo. Whatever method you choose, the goal is to create a case that
will persuade a mildly-technical but influential business leader of
the need for change.

We will invite a panel of judges to give prizes for the most
compelling examples in two categories: (1) The Overall Most Compelling
Example, which will be judged on a combination of the technical merits
and the presentation of the scenario, and (2) The Most Technically
Compelling Example, which will be judged on its technical merit alone,
without consideration of the way it is presented.

The whole purpose of the workshop it to focus on the {\em problem},
not the solutions. We are most definitely {\em not} interested in your
favorite scheme, or ours. So we need some ground-rules.
\begin{center}
{\em No-one is allowed to mention a specific mechanism, algorithm or
proposal at any time during the workshop: Not in their talk, not in a
panel, and not in questions to the speakers. The only mechanisms that
will be allowed mention are: TCP (in its standard and deployed
flavors), and idealized alternatives for purposes of demonstration.
For example, comparing TCP with an oracle that provides instantaneous
optimal rates to each flow.}
\end{center}

Attendance to the workshop will be by invitation only - to keep the
discussion focused and lively.  We will video the entire workshop and
all the demonstrations, and make it publicly available on the
Internet. We will make any proceedings and talks available too. The
goal is to open up the demonstrations for public scrutiny and feedback
after the event.

The event is hosted by the Stanford Clean Slate Program -
http://cleanslate.stanford.edu - and local arrangements will be made
by Nick McKeown and Nandita Dukkipati. The workshop has received
offers of support and funding from Cisco Systems and Microsoft. We
hope to make a limited number of travel grants available.

We have a small Program Committee (listed below) to select the final
demonstrations. We are soliciting a 1-page description of your demo.
Please note the following dates.

Important Dates:
===============
Workshop: 31 March - 1 April, 2008
1-page demo description submission deadline: 1 Dec, 2007
Notification of acceptance: 15 Dec, 2007
Final demo submissions: 17 March, 2008
Last date for demo withdrawals: 25 March, 2008

Please email your initial 1-page submissions to:
Nandita Dukkipati <[email-masked]> and
Nick McKeown <[email-masked]>

Best wishes, and we hope to see you in Stanford!

Nandita Dukkipati
Nick McKeown

Program Committee:
==================
1. Albert Greenberg, Microsoft Research
2. Peter Key, Microsoft Research
3. Flavio Bonomi, Cisco
4. Bruce Davie, Cisco
5. Steven Low, Caltech
6. Frank Kelly, Cambridge
7. Lars Eggert, Nokia Research/IETF
8. Nick McKeown, Stanford
9. Guru Parulkar, Stanford
10. Nandita Dukkipati, Stanford/Cisco/Princeton