Subject: [Tsvwg] pr-sctp

From: Mark Allman <[email-masked]>
Date: Wed, 02 Apr 2003 12:55:17 -0500

In SF I agreed to more beating on PR-SCTP via the mailing list.
So...

In SF Reiner showed a few slides that basically asked the question:
How much "extra" (unnecessary) traffic is acceptable to send?  TCP
naturally sends some.  The coding in some of the RTP protocols
constitutes more than the minimum bits required to send.  Etc.  I
have thoughts on that subject, but that is not what this email is
about.

My point to Randall about PR-SCTP was: should we be thinking about
PR-SCTP in the context of thinking about the points that Reiner
raised?  That is, with PR-SCTP you are sending things into the
network that are then (potentially) not used at the other end.  My
question was/is: If I send some segment X into the network, then
detect that it was lost, but also then decide that I don't care if X
makes it (so, I send an F-TSN), should that count against this "junk
data" counter that Reiner was alluding to?  There are two angles...

  * Phil Conrad and David Black rightly beat me and said something
    like ... but, PR-SCTP is helping you in this case because you
    won't *keep* sending this data that is useless.  And, that is
    true.  (Although, we know we have sent useless data some number
    of times.)

  * But, another question is whether we are providing an incentive
    for apps to dump things into the network quickly because they
    can then "supercede" them if necessary.  So, the tradeoff might
    be for the app to be a bit more careful and thoughtful about
    what is sent versus being quick and careless.  Are we providing
    a mechanism for apps to be more careless about sending data into
    the network?  And, if we are, then I think that thinking about
    some govenor on PR-SCTP in the context of Reiner's thought might
    be something to consider.

I hope that is a bit more clear.  It still may be erroneous thinking
... (so, am I shooting for "clearly erroneous" ... hmmm :).

Thoughts?

allman

--
Mark Allman -- BBN/NASA GRC -- http://roland.grc.nasa.gov/~mallman/
_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

---

From: "Michael A. Ramalho" <[email-masked]>
Date: Wed, 02 Apr 2003 15:15:33 -0500

Mark,

I think you have phrased the issues correctly and presented a
cohesive story.

A comment below on the "second angle".

Michael Ramalho

>     true.  (Although, we know we have sent useless data some number

[The "useless data" might not be "useless" if it arrived in a timely manner.]

You've basically got this point, though.

But to be emphatic .... under congestion this is exactly the behavior you want.
This is because every bit of information you send that the receiver can't 
profitably
use (because it is stale) is information that is sent *before more useful 
information*.
Therefore the process of re-sending stale information is *delaying* the 
sending of
useful information (potentially making it stale also ... under extended 
congestion
episodes).

Thus the setting of the lifetime parameter of the "timed reliability"
example in the pr-SCTP draft should be tuned to the application's
definition/knowledge of "staleness".

I agree that this adds a degree of definition difficulty in metrics like
"goodput". Is it good only if it is not stale? Clearly "goodput" defined
in such a manner is dependent on the application's definition of
"staleness/goodness" and the congestion condition at the time of
measurement/simulation.

Note: The congestion condition is not simply characterized by loss .... but
additionally by duration (in time) of the congestion event (as this has an
effect on "staleness"). We need to consider the time/delay element in
the congestion episode.

>     what is sent versus being quick and careless.

One could argue that if transmission bandwidth and app processing
power is almost always plentiful, the "quick and careless" approach
is OK.

But, I'd suggest that making app designers communicate to a PR-SCTP
Service Definitions such as "timed reliability" is, in itself, making the
app designer think about such things .... and creates an environment
whereby the app designer is forced to think about how they desire the
app to perform under less-than-ideal operating conditions (like congestion
episodes, routing changes that result in much larger nominal delays or
high packet loss).

Thus, I happen to believe that we are providing an incentive for app designers
*not* be careless ... but rather to think about overload/congestion scenarios.

>Are we providing

In this thought process we might consider other PR-SCTP service definitions
that are more appropriate to the applications to consider under
overload or congestion scenarios. The creation of other pr-sctp service 
definition
to help "govern" the sending of data would be welcome. However, the F-TSN
mechanism is the only thing required "on the wire" for PR-SCTP.

>... (so, am I shooting for "clearly erroneous" ... hmmm :).

Michael A. Ramalho, Ph.D.
Office email: [email-masked]
Personal email: [email-masked]
Office: +1.941.708.4650
Cell: +1.941.544.2844

---

From: "Armando L. Caro Jr." <[email-masked]>
Date: Wed, 2 Apr 2003 18:38:45 -0500 (EST)

> useful information (potentially making it stale also ... under extended

I agree with the concept of PR-SCTP. I want to emphasize that bundling
FTSNs with data should be done whenever possible. I think that sending a
FTSN in a packet of its own is not much better than re-sending the stale
data. Both will be triggered at the same point. Both will take the same
time to reach the receiver. And if lost, both suffer the same
consequences. The only improvement is that flightsize gets reduced and new
data may be sent.

Section 3.5 F2) emphasizes that bundling should be done, but just
wondering out loud... should PR-SCTP implementations reserve room
for FTSNs in all packets. If data chunks are not fragmented properly
when queued, then SCTP cannot refragment them later when bundling
is needed. The draft says that a sender MAY wait to bundle FTSNs, but is
this wording loose enough to allow SCTP to wait for newly queued data to
bundle with. In other words, data already queued may be fragmented too
large for bundling, but newly queued data may be fragmented small enough
for bundling. ...I don't think I like this idea. Maybe simply reserving
room for some minimum FTSN size is good enough.

> effect on "staleness"). We need to consider the time/delay element in

I don't follow how this relates to FTSNs.

> mechanism is the only thing required "on the wire" for PR-SCTP.

I like that the PR-SCTP mechanism is independent of the service
definition. However, I cannot think of applications that could use
anything but timed-reliability.

~armando

0--                                                  --0
| Armando L. Caro Jr.      |  Protocol Engineering Lab |
| www.cis.udel.edu/~acaro  |    University of Delaware |
0--                                                  --0

---

From: Mark Allman <[email-masked]>
Date: Wed, 02 Apr 2003 23:35:35 -0500

Michael-

> [The "useless data" might not be "useless" if it arrived in a

(but, if it did then we'd not need to issue an f-tsn for it, either)

> But to be emphatic .... under congestion this is exactly the
> behavior you want.  This is because every bit of information you
> send that the receiver can't profitably use (because it is stale)
> is information that is sent *before more useful information*.

(assuming an endless supply of new information; but I understand
your point)

> Therefore the process of re-sending stale information is
> *delaying* the sending of useful information (potentially making
> it stale also ... under extended congestion episodes).

I agree.  (But, this is orthogonal to my overall point, I think.)

> power is almost always plentiful, the "quick and careless"
> approach is OK.

But, if resources are always plentiful then one could also argue
that there is zero need for *P*R to begin with.

> But, I'd suggest that making app designers communicate to a
> PR-SCTP Service Definitions such as "timed reliability" is, in
> itself, making the app designer think about such things .... and
> creates an environment whereby the app designer is forced to think
> about how they desire the app to perform under less-than-ideal
> operating conditions (like congestion episodes, routing changes
> that result in much larger nominal delays or high packet loss).
> 
> Thus, I happen to believe that we are providing an incentive for
> app designers *not* be careless ... but rather to think about
> overload/congestion scenarios.

And, I think that was exactly Phil and David's point -- that PR-SCTP
in some ways helps the situation because it allows us to terminate
the re-sending of things that are known to be useless.

But, as a Cleveland Brown's coach once said, "there are two sides to
every piece of bologna".  And, I think the flip side very well could
be an incentive to be careless in choosing what to send.  Think
about the app sending the same thing N times and (framed with
application layer sequence numbers or something so that the app
could deal with dups).  Using PR-SCTP there is no incentive against
this.  Using TCP there is a clear incentive (head-of-line blocking
will cause absolutely lousy performance).  Using SCTP there is less
incentive than with TCP (because we can get unordered delivery), but
there is still a cost in terms of retransmissions and cwnd
sacrifice.

I am not sure what the right answer here is.... But, I think both
sides of the piece of bologna need examined.  And, I still think
that this falls within the realm of the sorts of things that Reiner
is trying to get us to think about.

Thanks!

allman

---

From: Andreas Jungmaier <[email-masked]>
Date: Thu, 3 Apr 2003 12:03:47 +0200

> have thoughts on that subject, but that is not what this email is

Dear Mark, dear all,

it would be very nice if someone could give a brief wrap-up on
the discussions/presentations that were held in SF on this topic.
(PR-SCTP).

Like me, not all people on this list could make it to SF, and 
especially the PR-SCTP topic is very interesting to us, not only
from our implementation point of view.

Thanks,
best regards,

Andreas

- --
- ---------------------------------------------------------------------------
Dipl.-Ing. Andreas Jungmaier              |  Stop sending E-POSTCARDS !
Computer Networking Technology Group      |  Send E-MAILS --  Use PGP !
University of Essen                       |
http://www.exp-math.uni-essen.de/~ajung   |   http://www.gnupg.org
My PGP Key-ID: D3824AC0                   |   http://wwwkeys.pgp.net
- ---------------------------------------------------------------------------
My PGP Key-Fingerprint: 228C 7C2C 3381 2DD4  9998 2812 5C0B 0B04  D382 4AC0
- ---------------------------------------------------------------------------

---

From: Caitlin Bestler <[email-masked]>
Date: Thu, 3 Apr 2003 11:47:11 -0600

On 4/2/03, Mark Allman wrote:

>
> * Phil Conrad and David Black rightly beat me and said
> something like ... but, PR-SCTP is helping you in this
> case because you won't *keep* sending this data that is
> useless.  And, that is true.  (Although, we know we have
> sent useless data some number of times.)
>
> * But, another question is whether we are providing an
> incentive for apps to dump things into the network
> quickly because they can then "supercede" them if
> necessary.  So, the tradeoff might be for the app to be a
> bit more careful and thoughtful about what is sent versus
> being quick and careless.  Are we providing a mechanism
> for apps to be more careless about sending data into the
> network?  And, if we are, then I think that thinking
> about some govenor on PR-SCTP in the context of Reiner's
> thought might be something to consider.
>
The question is whether PR-SCTP aids the applications that
it is intended to support (and whether there are enough of
those applications).

Obviously, any transport protocol can be abused. Consider
the still common practice of setting up multiple TCP
connections to make a session more fault tolerant,
and use of XML for the "benefit" of bypassing firewalls
by masquerading as web traffic. And as many application
levels have proven, you can build almost anything on top of
UDP, including IPv6 networks.

The profile that PR-SCTP addresses is any application that
has a content-sensitive need to retransmit lost packets.
This is most typically related to content that must be
delivered by a certain time to be useful.

But the need could be related to limited source buffering,
rather than a need at the consumer end. A real-time data
capture application, for example, may consider only some raw
data worth retaining for retransmit when the cost is not
capturing new data.

In my mind the prime application for PR-SCTP is streaming
multimedia. The content must be delivered with time
constraints, and it has an inherent structure. Certain data
messages (directory information, directives) MUST be
recieved, other data messages (I frames) will have a major
impact if lost, and the remainder (P,B frames) can be
dropped with very little impact.

Many streaming media servers do duplicate critical content,
but that is more related to support of multicast
environments. PR-SCTP is not suitable for multicast
delivery, but then almost all streaming media deliveries are
really point-to-point anyway.

The comparison with plain SCTP or TCP apparently has already
been covered. Clearly PR-SCTP reduces network load because
it allows content-sensitive prioritization of retransmits.
Only the most important data need be retransmitted, and only
if it will arrive in time. This is classic triage.
Retransmission can be limited to when it is necessary and
when it will not be futile.

The comparison with totally unreliable delivery is more
informative. An application that was merely loss tolerant,
but had no need to ever retansmit portions, would be better
served by a unreliable transport such as UDP or DCP. The
sender can release its buffers after transmitting, and the
receiver knows that what it gets is what it gets. There are
no extra packets (or network bandwidth) used to convey this
information.

More typically, loss of some portion of the data is more
critical to proper functioning of the receiver. There are
two methods to compensate for this:

-- Redundant transmission of critical data: The concern
   being raised seems to fear that PR-SCTP will encourage
   this practice. Compared to use of an unreliable transport
   it actually *discourages* it. The applications that would
   use PR-SCTP will not use TCP or SCTP instead, they will
   use UDP or DCP.
-- Application layer NACKing to selectively request 
   retransmission of critical data. This is part of
   RTP/RTSP now.
   
For typical streaming media applications, the sender has
at least or probably better idea of what data is critical
than the receiver does. PR-SCTP allows it to make these
determinations without an application level ACK or NACK.
With RTP/RTSP there is no need for explicit ACKs, but
that also forces congestion control up to the application
layer.

Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

---

From: Qiaobing Xie <[email-masked]>
Date: Thu, 03 Apr 2003 12:29:06 -0600

Mark,

Mark Allman wrote:
...
> have thoughts on that subject, but that is not what this email is

Here I have a comment on Reiner's question (which is the base of your
concern on PR-SCTP I think): I can see the point of Reiner's comment
applying to RTP/UDP traffic, but don't see it as much an issue for TCP
or SCTP. 

For RTP/UDP, people sometimes use forward error correction (FEC) to
combat frame loss. Several RFCs have been issued for this in avt. The
idea of FEC is simply to send redundant information for some or all data
frames. Of cause, if the original frame is not lost, this redundant
information becomes "unnecessary", i.e., a waste of network bandwidth.
In the case of RTP/UDP traffic, Reiner raised a good point that the
amount of redundancy the app can add to the original traffic must be
limited since UDP below has no throttle control. For example, a selfish
RTP sender may decide to add 1000% FEC redundancy to his traffic to
minimize his own data loss, at the cost of everyone else.

Things are not the same with TCP or SCTP. The CC in them automatically
puts a limit on how much bandwidth one can get. This changes the whole
situation. It is now more like that the app is given a set amount of
bandwidth and it has to decide *wisely* what portion of this bandwidth
to be used for "unnecessary" data, at the cost of the throughput of his
own "necessary" data. If he wants to do the same 1000% redundancy, he
simply hurts himself badly.

regards,
-Qiaobing

---

From: Mark Allman <[email-masked]>
Date: Thu, 03 Apr 2003 13:42:00 -0500

> It is now more like that the app is given a set amount of
> bandwidth and it has to decide *wisely* what portion of this
> bandwidth to be used for "unnecessary" data, at the cost of the
> throughput of his own "necessary" data. If he wants to do the same
> 1000% redundancy, he simply hurts himself badly.

I don't think he just hurts himself.  Say the app runs at some rate
X bps.  But, for the sake of redundancy the sender runs at 2X bps
and sends all the data twice, which may be fine according to the CC
rate limit.  And, we might even call that "fair" because everyone is
getting a rate of 2X bps.  But, if this connection didn't send all
this needless junk into the network (i.e., only sent at rate X bps)
then there would be X bps of new capacity for others to use to
transmit useful data and get actual work done.  If enough people
send enough redundant data into the network then everyone suffers
because they could all utilize more bandwidth if all the needless
junk was not floating around.

That is not to say we need to get rid of FEC in RTP or *all*
spurious retransmission in TCP or SCTP.  I am just trying to further
motivate my concern (& disagree with your statement above).

allman

---

From: Qiaobing Xie <[email-masked]>
Date: Thu, 03 Apr 2003 15:06:14 -0600

Mark,

I can see your point. But this is a little tricky and maybe out (or on
the edge) of tsvwg scope. To some degree, you are questioning whether
some portion of the data passed by the app above is REALLY necessary :-) 

I've had numerous times watching A/V codec designers (I used to be one)
and wondering "do those few extra redundancy bytes in your frame REALLY
make a difference?" They tend to say "surely, you can hear/see the
difference". So, when the app guys REALLY think that using 2X bps gives
them a critical edge, it's hard to argue with them. I guess the best we
in tsvwg could do is just to remind them "please please only put
*necessary* extra bits and not any *unnecessary* extra bits into the
network".

regards,
-Qiaobing

---

From: "Michael A. Ramalho" <[email-masked]>
Date: Thu, 03 Apr 2003 16:32:22 -0500

Mark ... Good volley ... Only one significant point below (and
a requisite amount of cute comments) ... Michael

>(assuming an endless supply of new information; but I understand

BTW - There are a lot of applications with this property. For
example, sensor data applications for roads, factory control,
weather info gathering, temperature control systems, network
monitoring information, etc.

>that there is zero need for *P*R to begin with.

I agree with your statement; but that is not exactly what I said. I said
"*almost* always" for a reason. In virtually every data characterization
environment I have run into .... the major congestion cases have
been designed to be "rare events". But it is exactly when these
rare events occur (sometimes characterized by fractal modeling)
that you *really want and need* such congestion behavior to
work in an intelligent manner.

>this.

OK ... onto the other side of bologna ... as I partially disagree with
this point when considering other potential pr-SCTP service definitions.
Let me explain.

TCP has infinite persistence (it tries until it dies and blocks
everything following).

PR-SCTP now has a service definition that has a persistence for
a timed lifetime. That is, re-transmit until information stale ... then
forget about it. The "forgetaboutit" mechanism is the F-TSN.

I'd maintain that the app you describe above is begging for
another type of PR-SCTP service definition .... the "try N
times (persistence) service definition".

If these applications are significant ... one could define a
PR-SCTP service definition (addressable through an API that
supported the definition) that told PR-SCTP to "try to resend
this chunk up to N times, then forgetaboutit". Note the mechanism
to "forgetaboutit" *ALREADY EXISTS* in the present PR-SCTP
draft. No on the wire changes are needed. The PR-SCTP
protocol doesn't need modification to accomplish what this
application desires ... only a new API to support the new
service definition.

If such a service definition existed ... there would be *no need*
for the application to send N times via APF as you suggest.
We then let PR-SCTP (the transport ... not the application) do
the "accounting" and it will only send once (if successful) the
majority of the times .... and only sends N times when it was
actually unsuccessful for [N-1] prior times. Thus this
service definition would make PR-SCTP have a "N persistence"
property.

This, in my book, is an incentive for the app to behave
better than sending "N times" in a brute force manner.
It only needs to send a message to the transport layer
once (it may desire to be informed in the event of
unsuccessful transmission ... but that is an issue for
the service definition). This would free the app designer
to be an app designer and not become a true IETF
transport protocol weenie/expert.

We could then deprecate a lot of the UDP ALF mechanisms
previously defined (which do things like try N times
using an exponential backoff mechanism). SIP UDP ALF
comes to mind ... but I don't want to open up that can
of worms.

TCP has "infinite" persistence. UDP has "1 times" persistence.
And PR-SCTP can have "variable" (in time or re-try) persistence.
All the "persistence cases" for "reliable transport" are then
covered.

[Aside: My original suggestion for the name for pr-SCTP
was lp-SCTP (for "limited persistence") ... but was overruled
by co-authors a year ago.]

Do we have a PR-SCTP convert yet?

Probably not ... but I'll keep trying ;-).

>Using TCP there is a clear incentive (head-of-line blocking

---

From: "Michael A. Ramalho" <[email-masked]>
Date: Thu, 03 Apr 2003 16:57:36 -0500

Armando - Two short comments below - Michael Ramalho

>I don't follow how this relates to FTSNs.

It doesn't. It relates to people who like to plot "goodput" vs "packet
error rate". My PR-SCTP "goodput" description of this thread
(which you cut) suggested that the goodput was a function of
staleness ... which was related to the *time duration* of the
congestion episode.

Thus you need three dimensional surface to describe the
PR-SCTP goodput ... one axis "goodput", one axis
"packet error rate" and the remaining axis "congestion
episode duration".

>anything but timed-reliability.

See my most recent email to Allman ... SIP UDP ALF as an example.

Michael

---

From: Reiner Ludwig <[email-masked]>
Date: Fri, 04 Apr 2003 10:52:58 +0200

I'd like to add an example for people to think about ...

If you try to run VoIP over TFRC, you'll most likely be disappointed by the 
quality. The reason is that when A talks, then (usually) B is silent, and 
vice versa. I.e., a talkspurt A>B is followed by a talkspurt B>A is 
followed by a talkspurt A>B, ....

While B is silent and if the talkspurt from A is long enough then B's TFRC 
send rate will decay to zero (or at least often enough below the required 
coder rate). Andrei Gurtov did this experiment and told me how 'broken' 
such a voice conversation becomes.

The "easy fix" is for B to send dummy data (or even potentially useful data 
like pictures, ads, calendar sync data, etc.) for the sole purpose of 
maintain the TFRC rate above the required coder rate. B would be a good 
Internet citizen in the sense that it is always TCP friendly. Still, B is 
sending data it does't have to send necessarily.

Should the IETF set up rules that would limit the sending of such 
potentially useless data? Do we have hope that such rules would be obeyed 
by application and protocol designers?

I could imagine VoIP application designers that would let themselves be 
convinced to move away from uncontrolled UDP to RTP/DCCP, but probably not 
much further. I.e., their view could be: "We ensure that our flows behave 
TCP friendly, but what we do with the available bandwidth is our business."

I'm not saying that I agree to that, but I would understand that view.

///Reiner

---

From: Mark Allman <[email-masked]>
Date: Fri, 04 Apr 2003 09:28:04 -0500

> I can see your point. But this is a little tricky and maybe out
> (or on the edge) of tsvwg scope. To some degree, you are
> questioning whether some portion of the data passed by the app
> above is REALLY necessary :-)

That is absolutely not the question I was asking.  I think that
trying to figure out whether the app is passing new or redundant
data is likely an NP-hard problem.

My wondering is specifically about transports aiding the process of
sending redundant data.  For instance, early retransmit proposes
resending data with less indication that it was lost than fast
retransmit.  Therefore, ER is less robust to reordering than FR.
There are worst cases where ER really wastes network resources by
resending data that does not need to be resent.  That is not an
application->transport issue, rather it is a transport only issue.

Another class of problems is whether the transport provides an
incentive for the app to send lots of data that may accomplish no
useful work.  I am not sure if PR-SCTP does this or not.  But, thus
far, nobody has convinced me that it does not.

allman

---

From: Mark Allman <[email-masked]>
Date: Fri, 04 Apr 2003 09:35:01 -0500

> for the application to send N times via APF as you suggest.  We
> then let PR-SCTP (the transport ... not the application) do the
> "accounting" and it will only send once (if successful) the
> actually unsuccessful for [N-1] prior times. Thus this service
> definition would make PR-SCTP have a "N persistence" property.
> 
> This, in my book, is an incentive for the app to behave better
> than sending "N times" in a brute force manner.

No need, but maybe a desire.

If I can send something twice from the app and one arrives at the
receiver then by some metrics the performance may be better because
we don't have to wait for (at least) an RTT for the transport to
figure out that the packet was dropped and retransmit.  If we have
to waste our own resources (buffer space, cwnd space, etc.) to
retransmit this packet that is now useless then I think there is
*some* incentive against using such a brute force method.  If we can
say "oh, nevermind on that first segment" and clean up our own
resources then it seems like there is no incentive not to waste
network resources if one wanted lots of redundancy.

allman

---

From: "Armando L. Caro Jr." <[email-masked]>
Date: Fri, 4 Apr 2003 10:44:59 -0500 (EST)

> far, nobody has convinced me that it does not.

I understand your concern, and I cannot say that I can convince you that
PR-SCTP doesn't face this problem. However, doesn't any
less-than-fully-reliable transport protocol face the same problem?

~armando

0--                                                  --0

---

From: "Michael A. Ramalho" <[email-masked]>
Date: Fri, 04 Apr 2003 11:53:10 -0500

>less-than-fully-reliable transport protocol face the same problem?

I too see Mark's lament that the transport may allow (I think encourage
is too strong) the app to send data to it that -  perhaps - will not be
sent (or re-sent if not ACKed). But let's look at the alternative (yet
another side of bologna).

The alternative is to mandate all app designers to be as smart as Mark
[or put your favorite tsvwg member name here] in order that they re-transmit
"transport friendly" (with correct backoffs, etc.) in "limited persistence" 
cases.

Unfortunately, this is too high a burden for most app designers (although
some would be smart enough to do something reasonable).

I think some reasonable, course feedback mechanism should be built
into applications in order to guide the app on the "course rate" of information
sent when the app has significant degrees of rate freedom (most video
codecs, for example).

However, I think it is an unreasonable expectation that application
designers have detailed tsvwg knowledge ... and I think the tradeoff
of letting the application send data to the transport with "lifetimes"
or "N persistence" mechanisms I described in earlier threads is
a reasonable way to let the transport layer take care of the "fine
grained" transport dynamics in a "transport friendly" manner.

Even if the far-end application doesn't provide feedback, the near end
transport can notify the app for chunks it decided to F-TSN over; this
is at least some feedback.

Lastly, I think providing such mechanisms (via service definitions)
actually encourage app designers to  give more thought on what
they desire to transmit under overload/congestion scenarios.

Michael Ramalho

---

From: Joe Touch <[email-masked]>
Date: Fri, 04 Apr 2003 11:32:37 -0800

Reiner Ludwig wrote:
> If you try to run VoIP over TFRC, you'll most likely be disappointed by 
> the quality. The reason is that when A talks, then (usually) B is 
> silent, and vice versa. I.e., a talkspurt A>B is followed by a talkspurt 
> B>A is followed by a talkspurt A>B, ....

It seems odd to consider TFRC for this case. VoIP requires communication 
that is decidedly outside the behavior patterns of TCP; making VoIP 
TCP-friendly seems bizzare.

Padding real data with dummy data for the sole purpose of being 
TCP-friendly seems even more bizzare. This would interfere with some 
potentially nice aggregation of VoIP on trunk links.

> The "easy fix" is for B to send dummy data (or even potentially useful 
> data like pictures, ads, calendar sync data, etc.) for the sole purpose 
> of maintain the TFRC rate above the required coder rate. B would be a 
> good Internet citizen in the sense that it is always TCP friendly.

Sending dummy data to quench other TCPs from getting real work done 
isn't being a good Internet citizen; it's a denial-of-service attack, of 
sorts.

The real error here, IMO, is considering TCP-friendly to be equivalent 
to Internet-friendly. There has to be a good definition of the latter, 
and that is what we should really be striving for.

Joe

---

From: Yogesh Prem Swami <[email-masked]>
Date: 04 Apr 2003 14:16:16 -0600

Reiner,

> such a voice conversation becomes.

The conversation will be equally "broken" if you don't follow TFRC in
this case. Sending a lot of data (after waiting for a long time) does
not necessarily mean that the quality will improve--the data also need
to reach the receiver. TFRC tries its best to send only as much data as
will safely reach the sender after a given time. 

What's worse in case of not following TFRC is that it might force some
TCP connections to timeout.

BTW, if a connection has waited for such a long time that it's data rate
is so low, why can't it wait a little longer and buffer the data on the
receiver side before plying it out. This should mitigate the problem if
not solve it sompletely. (Please note rate reduction happends every RTO,
which is several times the RTT.) 

I think TFRC (along with DCCP) does a great job. 

> sending data it does't have to send necessarily.

You need to define 'necessary' here. Let's take a different example. Say
there are 10 people on a LAN which are accessing the same web page. This
is also an example of 'unnecessary' data. So should the transport layer
now worry about these things too. If I am sitting on a fiber cable, I
won't be able to tell whether the data for different web clients is
unncessary or whether the data from TFRC is unnecessary. How do we draw
the line here.

> TCP friendly, but what we do with the available bandwidth is our business."

That's fine as far as transport layer is concerned. Application
protocols desiners, who're as concerned about bandwidth as transport
layer protocol designers are, will certainly find the best approach (for
example, if the sustainable delay before playout is T = x*RTO, then the
application will be happy not to send any 'unnecessary' data until Xth
RTO. Okay that's not the right calculation, but you get the idea...)

These issues are upto the application to figure out. The transport layer
should not worry about them, IMO. 

--yogesh

---

From: "Armando L. Caro Jr." <[email-masked]>
Date: Fri, 4 Apr 2003 15:32:06 -0500 (EST)

On 4 Apr 2003, Yogesh Prem Swami wrote:

> receiver side before plying it out.

Because his traffic example was VoIP, but as Joe pointed out... VoIP is
probably not a good example.

~armando

0--                                                  --0

---

From: Yogesh Prem Swami <[email-masked]>
Date: 04 Apr 2003 14:42:47 -0600

> probably not a good example.

To the best of my knowlege, even VoIP data needs to be buffered if for
no other reason than to get rid of jitter. Of course how long it needs
to wait is a different question.

--yogesh

---

From: Spencer Dawkins <[email-masked]>
Date: Fri, 4 Apr 2003 13:29:26 -0800 (PST)

Hi, Yogesh,

> to wait is a different question.

Yeah, but - "buffering" is relative, as you point out -
buffering streaming traffic for 50-ms for a less-jittery playout
isn't the order of magnitude we talk about for end-to-end
transport roundtrip times, right? 

People start noticing things at about 100-200 ms - if you start
to talk and the other person doesn't stop talking for 400 ms
because the voice stream was buffered, most people find this
disconcerting. It's something closer to "push-to-talk" than
voice traffic we know and love...

Spencer

---

From: Spencer Dawkins <[email-masked]>
Date: Fri, 4 Apr 2003 13:36:14 -0800 (PST)

Believe it or not, I don't always post whatever enters my mind.
I was trying to think of a way to say what Joe said, much better
than any way that I'd come up with, so far:

> and that is what we should really be striving for.

I'm really curious where a definition of "internet-friendly"
would begin, if it's to be different from "TCP-friendly". I'm
assuming that reducing sending rates in the face of sustained
congestion is the floor we're not willing to go beneath - is
there a higher floor that we all agree on?

Spencer

---

From: Yogesh Prem Swami <[email-masked]>
Date: 04 Apr 2003 15:55:20 -0600

Hi Spencer,

> voice traffic we know and love...

I completely agree. The point I was trying to make is it's not upto the
transport layer to worry about these things. TFRC says: this is what the
data rate an application will get. It's upto the application to decide
the right strategy to get the data rate it wants. If that requires
sending unnecessary data or buffering them, then that's fine as far as
transport layer is concerned.

--yogesh

---

From: Joe Touch <[email-masked]>
Date: Fri, 04 Apr 2003 23:37:16 -0800

> there a higher floor that we all agree on?

One challenge is the implicit assumption in defining friendliness in 
terms of rate. For some applications, rate is meaningless. For others, 
the timescale is critical - one RTT, multiple RTTs, etc.

TCP analysis generally assumes sustained offered load, with a goal of 
generating stable sending rates over many RTTs. For VoIP, those goals 
don't apply.

Here are some initial definitions of Internet-Friendly:

	- gets 1/Nth of network capacity when there are N flows
		- not more (fair to others)
			implicitly allowing the other N-1
			flows the whole of the capacity
			when there is no data to send
		- not less (fair to self/upper layers)

	- reacts to known network conditions within 1 RTT
		the key here is the term "known" -
		it can't be expected to read minds

	- adds load in a 'gentle' fashion

This might, e.g., allow VoIP to send lots of little bursts, where, e.g., 
each burst is O(10 packets), where the average sending rate over 100's 
of RTTs is stable, but need not be over 1 RTT or even 10's of RTTs.

That wouldn't be TCP-friendly, but maybe that's because TCP expects that 
everyone who backs off does so when the congestion occurs (assuming that 
all connections are constantly active), rather than when they notice it 
(which may be equally reasonable).

Joe

---

From: Caitlin Bestler <[email-masked]>
Date: Sat, 5 Apr 2003 21:24:36 -0600

On 4/4/03, Joe Touch wrote:

>
>One challenge is the implicit assumption in defining
>friendliness in terms of rate. For some applications, rate
>is meaningless. For others, the timescale is critical -
>one RTT, multiple RTTs, etc.
>
>TCP analysis generally assumes sustained offered load,
>with a goal of generating stable sending rates over many
>RTTs. For VoIP, those goals don't apply.
>
>This might, e.g., allow VoIP to send lots of little
>bursts, where, e.g., each burst is O(10 packets), where
>the average sending rate over 100's of RTTs is stable, but
>need not be over 1 RTT or even 10's of RTTs.
>
>That wouldn't be TCP-friendly, but maybe that's because
>TCP expects that everyone who backs off does so when the
>congestion occurs (assuming that all connections are
>constantly active), rather than when they notice it (which
>may be equally reasonable).
>

ATM QoS differentiates not only the sustained rate, but a
peak rate and latency. I'm not suggesting that IP networks
are capable of dealing with ATM type controls, but that at
least we should recognize the same range of application
bandwidth requirements. As you properly point out, all
current work is horrendously biased to low-latency sustained
throughput.

Caitlin Bestler

---

From: Randall Stewart <[email-masked]>
Date: Sun, 06 Apr 2003 12:31:56 -0500

> room for some minimum FTSN size is good enough.

You have the same issue with SACK's as well... one might be wise
when implementing to set the fragmentation point SCTP uses to
be small enough for a minimal SACK OR a F-TSN..

> anything but timed-reliability.

I can, one great use would be for applications such as IP-Fix... where
I "want" to get data across on peg counts but if my router that is
reporting the information gets TOO busy I would prefer to drop some
of the peg counts .. too busy being to much memory allocated to the
"peg" counts... at the same time there are things like templates that
MUST make it across...

Note no time is involved.. it has to do with buffering capacity of
the router and busy-ness..

We will be working internally at cisco in getting a prototype up
in IOS that uses PR-SCTP shortly.. if I can ever stop traveling :-0

R

-- 
Randall R. Stewart
[email-masked] 815-342-5222 (cell phone)

---

From: Reiner Ludwig <[email-masked]>
Date: Mon, 07 Apr 2003 10:05:07 +0200

At 21:32 04.04.2003, Joe Touch wrote:
>The real error here, IMO, is considering TCP-friendly to be equivalent to 
>Internet-friendly. There has to be a good definition of the latter, and 
>that is what we should really be striving for.

I agree 100 percent, since that's exactly the point I wanted to raise when 
I presented the few slides at the last TSVWG meeting:
http://www.cs.helsinki.fi/u/gurtov/reiner/TCP_FEC.pdf
http://www.cs.helsinki.fi/u/gurtov/reiner/TCP_FEC.ppt

I just tried to paint a bit in black & white hoping that that would provoke 
some deeper thinking about this :-)

But afterall, if I recall right then VoIP was mentioned as one the 
applications for TFRC in those drafts ...

///Reiner

---

From: Spencer Dawkins <[email-masked]>
Date: Mon, 7 Apr 2003 05:12:08 -0700 (PDT)

We must have gone through Reiner's slides REALLY fast. I was a
minute-taker for TSVWG, IIRC, and they didn't sink in during the
meeting at all.

Unfortunate, because they are an excellent jumping-off-point for
the issue we're discussing on this thread.

RFC 3448 includes the following text in the abstract:

   This document specifies TCP-Friendly Rate Control (TFRC). 
TFRC is a
   congestion control mechanism for unicast flows operating in a
best-
   effort Internet environment.  It is reasonably fair when
competing
   for bandwidth with TCP flows, but has a much lower variation
of
   throughput over time compared with TCP, making it more
suitable for
   applications such as telephony or streaming media where a
relatively
   smooth sending rate is of importance.

So it looks like Reiner is remembering correctly that VoIP was a
motivator for TFRC...

Spencer