Subject: RE: AW: [NSIS] framework: proposal on congestion/flow control

From: "Attila Bader (ETH)" <[email-masked]>
Date: Mon, 23 Jun 2003 14:32:52 +0200

Hi Michael and Bob,

Please note that we have shown an example for an architecture in which internal nodes are stateless. Please see the following draft: 
http://www.ietf.org/internet-drafts/draft-westberg-proposal-for-rsvpv2-nslp-00.txt

Best regards, Attila

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]On Behalf Of
> Michael Welzl
> Sent: Monday, June 23, 2003 8:10 AM
> To: Bob Braden
> Cc: [email-masked]; [email-masked]
> Subject: Re: AW: [NSIS] framework: proposal on congestion/flow control
> 
> 
> Dear Bob,
> 
> > Michael,
> > 
> > I could be wrong, but it seems to me that to do congestion 
> control you
> > need state.  The issue is when that state is established.  
> In the case
> > of ephemeral messages that you want to send using UDP (or better,
> > DCCP), I would think that the state could be set up roughly 
> once, when
> > two neighbor nodes first discover each other.  So, it's a 
> question of
> > when, not whether, to establish state.
> > 
> > Bob Braden
> 
> I agree, but It's also a question of how much state you
> establish, or (more important) how often you need to update it.
> DCCP requires a lot of state and acks every packet, which is ok
> for any long-term data transfer, but I thought that there might
> be sporadic messages in signaling (error messages of some
> sort ... keepalives ... stuff like that) where this would be
> unnecessary overhead.
> 
> For instance, an occasional "source quench"-like message (from
> the receiver, not an intermediate router!) might suffice in
> such cases. Rate-based open-loop instead of window-based
> closed-loop congestion control, perhaps using rather old
> state (as you mention - it is indeed a question of when!).
> 
> And even without setting up any kind of state, it is reasonable
> to limit the rate of signaling traffic: if a node generally
> keeps its signaling traffic below, say, 5% of the payload it
> generates, the whole signaling traffic in the whole network
> will not exceed 5% of the whole payload traffic. This means
> that signaling traffic scales linearly with payload (which
> can, in turn, be expected to be limited through congestion
> control) ... while this is not as good as full-fledged
> congestion control, it's a start. I think that such a limit
> was also specified for RTCP, btw.
> 
> Doing this would be my recommendation for such sporadic
> messages.
> 
> Best regards,
> Michael
> 
> 
> _______________________________________________
> nsis mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/nsis

_______________________________________________
nsis mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nsis

---

From: Bob Braden <[email-masked]>
Date: Mon, 23 Jun 2003 10:56:13 -0700 (PDT)

*> 
  *> > Michael,
  *> > 
  *> > I could be wrong, but it seems to me that to do congestion control you
  *> > need state.  The issue is when that state is established.  In the case
  *> > of ephemeral messages that you want to send using UDP (or better,
  *> > DCCP), I would think that the state could be set up roughly once, when
  *> > two neighbor nodes first discover each other.  So, it's a question of
  *> > when, not whether, to establish state.
  *> > 
  *> > Bob Braden
  *> 
  *> I agree, but It's also a question of how much state you
  *> establish, or (more important) how often you need to update it.
  *> DCCP requires a lot of state and acks every packet, which is ok
  *> for any long-term data transfer, but I thought that there might
  *> be sporadic messages in signaling (error messages of some
  *> sort ... keepalives ... stuff like that) where this would be
  *> unnecessary overhead.
  *> 
  *> For instance, an occasional "source quench"-like message (from
  *> the receiver, not an intermediate router!) might suffice in
  *> such cases. Rate-based open-loop instead of window-based
  *> closed-loop congestion control, perhaps using rather old
  *> state (as you mention - it is indeed a question of when!).

Michael,

I believe that you are confusing congestion control with flow control.
The "source-quench-like" or windowing mechanisms you describe are
hop/hop flow control.  Congestion control has to do with the interaction
of the signaling traffic with other Internet traffic.  We don't
want signaling traffic to starve out other traffic, either TCP traffic
or realtime traffic.  Internet transport is supposed to be "TCP-friendly".
I have not looked at DCCP lately, but I assume it has the minimal
mechanism required for honest Internet congestion control.

  *> 
  *> And even without setting up any kind of state, it is reasonable
  *> to limit the rate of signaling traffic: if a node generally
  *> keeps its signaling traffic below, say, 5% of the payload it
  *> generates, the whole signaling traffic in the whole network
  *> will not exceed 5% of the whole payload traffic. This means
  *> that signaling traffic scales linearly with payload (which
  *> can, in turn, be expected to be limited through congestion
  *> control) ... while this is not as good as full-fledged
  *> congestion control, it's a start. I think that such a limit
  *> was also specified for RTCP, btw.

As you imply above, the notion of sporadic needs to be made more
precise, because there can be bursty behavior even when the
steady-state rates are reasonable.  Suppose that a route changes, so a
lot of reservations have to be signaled at one time.  The initiating
NSIS node and the recipient NSIS node may be perfectly happy to blast
across these reservation packets, but the result could be like a DoS
attack on the link!

Is your fixed percentage bound good enough?  I don't know.  5% FEELS
OK, because it seems negligble, and we assume there are not 10 other
fixed overheads each taking 5%.  There is only one NSIS flow (or is
there?)  And if the other traffic is low, you might want to let
signaling take a higher percentage.  And the situation might be
different in the core of the network and in the edge.  Finally,
such rate limiting is NOT congestion control, since it does not
react dynamically to congestion.  It is like an RTP media stream.

It always seemed me reasonable that NSIS (or RSVP) traffic should
be given its own scheduling class (e.g., diffserv code point), but
nobody seemed to agree.

None of this is to say that I know the answer here; it's a hard
problem.  As a practical matter, the AD (;-)) will decide what
congestion control in NSIS is good enough.  But then, if NSIS
were easy, it would have been done 5 years ago, I guess.

Bob Braden

  *> 
  *> Doing this would be my recommendation for such sporadic
  *> messages.
  *> 
  *> Best regards,
  *> Michael
  *>

---

From: Bob Braden <[email-masked]>
Date: Tue, 24 Jun 2003 08:55:39 -0700 (PDT)

*> 
  *> Quick question for you:
  *> 
  *> > I believe that you are confusing congestion control with flow control.
  *> > The "source-quench-like" or windowing mechanisms you describe are
  *> > hop/hop flow control.  Congestion control has to do with the interaction
  *> > of the signaling traffic with other Internet traffic.  We don't
  *> > want signaling traffic to starve out other traffic, either TCP traffic
  *> > or realtime traffic.  Internet transport is supposed to be 
  *> > "TCP-friendly".
  *> 
  *> Michael seems to be talking about flow-control on a hop-by-hop 
  *> basis.  I've always considered congestion control as a end-to-end
  *> mechanism.  What are your thoughts about having congestion control on 
  *> a hop-by-hop basis - to me this sounds sub-optimal.
  *> 
  *> John
  *> 

John,

Perhaps I was sloppy... I meant hop/hop in the sense of NSIS hops.
For NSIS traffic between NSIS neighbors, there is not distinction
between that and E2E; the neighbors ARE the ends.

Does this help?

Bob

---

From: "Geib, Ruediger" <[email-masked]>
Date: Thu, 26 Jun 2003 09:16:34 +0200

Robert,

This thread is limited to congestion on the control 
plane/signaling, isn't it?

|what *is* (i think) the case is that the long-term response to 
|congestion has to be to force the congestion up the protocol
|stack and/or out to the network edge - in the end, it ends 
|up with a user.

The statement is that the signaling source must reduce 
the "signaling rate". I agree.
The only problem I see is the case when congestion results 
from a network failure. In that case, signaling information 
may be urgently needed on the new path (and also from a user 
perspective). I'm not sure whether NSIS is able to optimise 
for both. My preference would be to protect the network 
and reduce signaling load.

|this leaves open the question of whose responsibility it is to 
|do congestion control between 'interior' nodes
|*) the NTLP for its retransmissions, NSLP for everything else
|*) NTLP only

By retransmissions as part of congestion control you mean control
of the sending rate of retransmissions? NTLP is able to ensure 
reliable transmission of signaling messages, i.e. each message 
will be ack'ed by the NTLP peer. If this is the case, NTLP 
should deal with local congestion.
The refresh timer values should be large enough to leave space 
for the NTLP congestion control to add value. Otherwise NSLP 
controls congestion. 

Another question coming to my mind is whether there should be 
a differentiation between network congestion and 
application/signaling processor congestion. Something similar
to ECN might be useful to protect signaling processors from 
being overloaded.

Regards, Rüdiger

---

From: "Hancock, Robert" <[email-masked]>
Date: Thu, 26 Jun 2003 09:03:26 +0100

hi rudi,

> From: Geib, Ruediger [mailto:[email-masked]]
> Sent: Thursday, June 26, 2003 08:17
> To: Hancock, Robert
> Cc: [email-masked]
> Subject: Re: [NSIS] framework: proposal on congestion/flow control
> 
> 
> Robert,
> 

well, it's limited to congestion at the IP layer and at
NTLP/NSLP layers. the draft discusses all of them together
(e.g. because many of the causes are common), but there is
some decoupling possible in the solution.

> and reduce signaling load.

i think this is unarguably true; however, the trick is to know
whether or not the signalling you are generating is hurting the
network; once you know this, a reasonable strategy is to signal
as fast as possible without hurting the network. the hard part
(or, 50% of the hard part) is detecting congestion reliably in 
the first place.

> should deal with local congestion.

agreed. i don't see any alternative.

> for the NTLP congestion control to add value. Otherwise NSLP 

i don't really see what you mean by this. what about NSLP messages
which are not refreshes, for example?

> to ECN might be useful to protect signaling processors from 

this is the other issue considered in the draft. from a framework
perspective, i think the key question is whether the NTLP provides
an environment in which NSLPs can signal and detect overload implicitly
(which in practice means the NTLP providing something like a 
flow-controlled delivery service between NSLP peers), or whether the
NSLPs have to do this for themselves (in which case ideas like ECN
could well be useful, but are then part of the NSLP design debate).

my personal view is that it is not possible to solve all the 
problems with a flow-controlled NTLP, therefore the second approach
is what we should be thinking in terms of.

---

From: "Geib, Ruediger" <[email-masked]>
Date: Thu, 26 Jun 2003 10:59:03 +0200

Hello Robert

|> The refresh timer values should be large enough to leave space 
|> for the NTLP congestion control to add value. Otherwise NSLP 
|> controls congestion. 
|
|i don't really see what you mean by this. what about NSLP messages
|which are not refreshes, for example?

What I want to say is that congestion control on different levels 
should be designed in a way that both aren't active in the same 
timescale. Usually the upper layer congestion control is slower than 
the lower layer congestion control. My assumption is that the 
refresh mechanism is designed to cope with congestion too in order 
not to eliminate state caused by lost messages. There's no point in 
retransmitting NTLP messages if there's a high probability that due 
to a timeout state was removed in the receiving NSLP already. 
If NSLP was having its own ACK mechanism and starts to retransmit 
if it doesn't receive an ACK in time, this could add congestion 
should NTLP still try to retransmit the original NSLP message in 
the same time. 

|> Another question coming to my mind is whether there should be 
|> a differentiation between network congestion and 
|> application/signaling processor congestion. Something similar
|> to ECN might be useful to protect signaling processors from 
|> being overloaded.
|
|this is the other issue considered in the draft. from a framework
|perspective, i think the key question is whether the NTLP provides
|an environment in which NSLPs can signal and detect overload implicitly
|(which in practice means the NTLP providing something like a 
|flow-controlled delivery service between NSLP peers), or whether the
|NSLPs have to do this for themselves (in which case ideas like ECN
|could well be useful, but are then part of the NSLP design debate).
|
|my personal view is that it is not possible to solve all the 
|problems with a flow-controlled NTLP, therefore the second approach
|is what we should be thinking in terms of.

Yes, agreed.

Regards, Rüdiger

---

From: "Hancock, Robert" <[email-masked]>
Date: Thu, 26 Jun 2003 14:50:56 +0100

hi rudiger,

> not to eliminate state caused by lost messages.

well, for this and many other reasons, but anyway...

> There's no point in 

quite right. this is one example of a general problem of how to cope 
when there are natural mechanisms to use in 2 layers, and they end
up duplicating rather than complementing each other. but in this case,
i think it's more a question of multiple reliability mechanisms
(explicitly at the NTLP, via refresh at the NSLP) rather than multiple
congestion control mechanisms.

cheers,

r.

---

From: "Hancock, Robert" <[email-masked]>
Date: Tue, 24 Jun 2003 23:09:19 +0100

dear all,

there are at least 4 possible meanings of the word 'end' in the context
of NSIS discussions (maybe the WG chair could decide an appropriate
penalty for using the word without qualification):

a) the data flow endpoints
b) the 'furthest apart' signalling application nodes on the path
c) any signalling application node on the path
d) any NSIS node on the path
[everything else is router, still. i wouldn't like to get in an argument
about whether a node sending an NSIS message is a host or a router, but
maybe that's a discussion that's needed at some point.]

the point roughly made in the mini-draft that accompanies this thread is
that
congestion-generating load can originate with any of (a), (b) or (c), but
not [as a consequence of the 'NTLP doesn't manage signalling application
state'
decision] with (d), unless the NTLP has retransmissions, in which case
endpoints (d)
can also be guilty. therefore, doing 'end-to-end' (presumably meaning
a/b-level) 
congestion control in NSIS signalling protocols will not be a congestion
control
solution - in fact, the most likely congestion causes are the 'ends in the
middle'
(c) because of local repair actions, IMHO.

what *is* (i think) the case is that the long-term response to congestion 
has to be to force the congestion up the protocol stack and/or out to the 
network edge - in the end, it ends up with a user.

this leaves open the question of whose responsibility it is to do congestion

control between 'interior' nodes
*) the NTLP for its retransmissions, NSLP for everything else
*) NTLP only

cheers,

r.

> From: Bob Braden [mailto:[email-masked]]
> Sent: Tuesday, June 24, 2003 17:56
> To: [email-masked]; [email-masked]; [email-masked]
> Cc: [email-masked]
> Subject: RE: AW: [NSIS] framework: proposal on congestion/flow control
> 
> 
> 
>   *>