Subject: RE: [NSIS] layer split summary? opinions?

From: [email-masked]
Date: Sun, 2 Feb 2003 16:01:11 +0200

Hi all,

> > Features such as segmentation, congestion
> > control, non duplication of signaling messages, etc.
> > are specific and needed for some NSLP type, 
> > but not for all NSLP types.
> 
> Can you identify the types that need it and the types that don't?
> 
> Are you advocating that NSLP become a full-fledged transport protocol 
> for 'some' NSLP types, essentially an application-layer TCP or SCTP?

It is out of scope to develop a full-fledged transport protocol
here. 

br,
John
_______________________________________________
nsis mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nsis

---

From: "Geib, Ruediger" <[email-masked]>
Date: Mon, 3 Feb 2003 09:07:51 +0100

John,

as a provider representative I fully support Henning's view. 

Regards, Rudiger

| I don't see how you can have a signaling protocol that does not offer 
| reliability, and on short timescales. As far as I can tell, 
| there are no deployed examples of 'unreliable' signaling protocols in 
| other domains (SS7, ATM).
| 
| Now, it is possible to rely purely on end-to-end reliability, 
| but this has serious design problems, in terms of efficiency. 
| (Among other issues, the RTT estimate is likely going to be 
| much better among NSIS peers than among some random pair of 
| endpoints that haven't met each other before.)
_______________________________________________
nsis mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nsis

---

From: "Georgios Karagiannis (ELN)" <[email-masked]>
Date: Mon, 3 Feb 2003 09:50:12 +0100

Hi John

I agree with your view!

Best Regards,
Georgios

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: zondag 2 februari 2003 14:51
To: [email-masked]; [email-masked]
Cc: [email-masked]
Subject: RE: [NSIS] layer split summary? opinions?

Hi Henning & Georgios,

> > No, if an application needs TCP or SCTP features, then these applications 
> > should probably use the TCP or SCTP protocols, e.g., below NTLP.
> 
> That's what I've been advocating. Part of the discussion problem seems 
> to be that some people say "NTLP" and mean "whatever 'transport' layer 
> NSIS invents including the transport below", others mean "whatever sits 
> on top of either IP or TCP/SCTP". We should agree on one definition.

Well, I think we need to determine if, for example, reliable transport
is needed.  If so, then I suggest that we use TCP/SCTP; if not, then
UDP will probably be enough. 

> A different way of phrasing the confusion is that some people see NTLP 
> as an interface to the higher layer (NSLP) above - it describes all of 
> the services that the NSIS-new-stuff adds plus any underlying  existing 
> transport protocol offers.
> 
> Other people, possibly guided by the 'P' at the end, see it as a 
> protocol, i.e., describe only the services that this layer adds on top 
> of whatever L3/L4 mechanism is below.

Ultimately, this is what we will need to do.
 
> The re-use of the word transport can't help but add to 
> misunderstandings, but we probably lack the language supply to fix that 
> particular problem. (We have used the term messaging layer in one 
> context, but that term has its own set of problems.)

Agreed.

> However, your statement does add a requirement to NTLP: namely, that it 
> should be able to use either 'raw' IP or an existing transport 
> mechanism, such as UDP, TCP or SCTP (or possibly one of the emerging 
> UDP-like, congestion-controlled protocols being discussed in TSVAREA).

I agree.

John

---

From: [email-masked]
Date: Tue, 4 Feb 2003 13:12:22 +0200

Hi Robert,

> 3. *Whatever* of these approaches is taken, it doesn't affect the apparent 
> consensus that an NSLP should be able to call on some combination of lower 
> layers (whatever you call them) which provide these classic transport-like 
> functions. (What I get from Georgios is that he'd like them to be somehow 
> optional to use. But I'll write a separate mail on that.)

I support this.  However, regarding Lixia's mail earlier, we still need
discussion what form of reliability is needed, for example TCP-style 
reliablity or soft-state refresh reliability.

br,
John

---

From: [email-masked]
Date: Tue, 4 Feb 2003 13:17:46 +0200

Hi Steve & Ping,

> Ping, et al,
> 
> Before we make any hasty decisions, let's reconsider why TCP (etc) was
> not used in RSVP.  First of all, TCP has a lot of overhead (reliable
> delivery, in-order delivery, congestion control) not all of which is
> really needed in RSVP.  Second, TCP does not support multicast which
> was a feature in RSVP.  Third, you don't necessarily know who your
> RSVP neighbor is, so it is not clear who to open the TCP connection
> to.
> 
> I think that having some form of reliable transport as (at least) an
> option is reasonable.  But I think we need to be clear about exactly
> what features are desired and what the tradeoffs are.  Then we can 
> make a good decision about mechanisms.

It would be great to capture some of this material in the Analysis document.
Would either one of you want to submit some text?  I'm hoping that
the Analysis doc could capture this so that it is useful when doing
the protocol work.

thanks,
John

---

From: [email-masked]
Date: Tue, 4 Feb 2003 13:20:12 +0200

Hi Georgios,

> First of all I would like to mention that I am in favor 
> of using (a migrating version of) RSVP as NTLP.
> Meaning that NTLP should mainly support the transport 
> layer features that are at the moment supported by RSVP, 
> while transport layer features that are not supported by 
> RSVP should not be supported by the NTLP.

Perhaps some comments on why NTLP would not need other transport
layer features not supported by RSVP would be helpful for us
to understand.  Have you made a determination on why others
are not needed?

John

---

From: [email-masked]
Date: Tue, 4 Feb 2003 13:25:57 +0200

Hi Georgios,

> I agree that NSIS (NTLP + NSLP) in combination with other IETF existing protocols
> e.g., TCP, SCTP, that can operate below NTLP should function robustly in all 
> possible environments. This does not mean that NTLP should reinvent all the features 
> that TCP or SCTP can provide. In my opinion additional features that have be provided by 
> NSIS (for some environments) and are not supported by TCP, SCTP or NTLP,
> should be provided by NSLP.

So, let me just make sure I understand you.  If NTLP could run over TCP, SCTP, UDP, DCCP
or IP directly, it should not duplicate features found in those protocols, correct?
If so, I agree 100%.  

Do you think it is OK if we define how NTLP could run over a well known transport protocol,
or do you have any reservations (no pun intended) about which transport protocols
could be supported?

br,
John

---

From: [email-masked]
Date: Tue, 4 Feb 2003 13:29:17 +0200

Hi Henning,

> While SIP is not a network-layer reservation protocol, I think some of 
> the lessons learned there apply here as well. (While UDP will likely 
> remain part of the protocol, its sphere of use has been shrinking 
> continuously.)
> 
> - MTU discovery and fragmentation is a pain. Even if you think that the 
> message is small enough to fit 'near' where you are, it might get bigger 
> along the way, as error indications, signatures and other things are 
> added. This makes it very different from a run-of-the-mill IP packet 
> that just gets its TTL bumped. Even if you can estimate the size of the 
> message you send, you don't always know how big the response 
> is going to be.
> 
> - Even if you can reasonably assume that normal (QOS) signaling traffic 
> is low volume, with retries after failures, this is no longer 
> the case.
> 
> - Wherever some notion of trust is involved, most systems talk to 
> relatively few other systems in practice, so that the setup overhead for 
> TCP is modest, removing one of the major TCP/SCTP problems that 
> motivated UDP in the case of application-layer signaling.
> 
> - People like TLS. Yes, you can use IPsec, but once you talk to 
> implementors, nobody wants to rely on it being available and 
> configurable, given a choice.
> 
> I'm sure there are other lessons and not all apply here, but we should, 
> I believe, look around more recent design efforts in the IETF as well.

These are all good points, Jukka, do you think you could add this kind
of text into the Analysis document?  Henning, if you have any references
to these kinds of experiences, I think that they'd be valuable to
capture.

cheers,
John

---

From: "Georgios Karagiannis (ELN)" <[email-masked]>
Date: Tue, 4 Feb 2003 15:07:16 +0100

Hi John

>> So, let me just make sure I understand you.  If NTLP could run over 
>> TCP, SCTP, UDP, DCCP or IP directly, it should not duplicate features 
>> found in those protocols, correct?
Yes!

>> or do you have any reservations (no pun intended) about which 
>> transport protocols could be supported?

I am not in favor of mandating that NTLP will have to run over
one well known transport protocol!
It will, however, be interesting to describe (and not standardize) 
how NTLP could run over TCP, SCTP, UDP, DCCP or IP directly.

Best Regards,
Georgios

---

From: Bob Braden <[email-masked]>
Date: Tue, 4 Feb 2003 20:02:32 GMT

*> 
  *> A lot is being said about 'overhead'. Can somebody quantify this beyond 
  *> just statements of opinion? My perception, based on measurements in 
  *> other, related contexts, is that it has no impact whatsoever on 
  *> signaling performance, compared to other operations. After all, we 
  *> routinely build web servers that can handle thousands of operations a 
  *> second.
  *> 
  *> 
Henning,

In support of your comment/question:

Consider QoS signaling for one flow, and assume no packet loss. 

*  A trigger message is sent from node A to neighbor node B, either in
   one NTLP message/TCP segment in one IP datagram, or in one NTLP
   message in one IP datagram.

*  Node B returns some sort of ACK message, either in as a TCP ACK
   segment in one IP datagram, or as one NTLP message in one IP datagram.

Whether or not TCP is used, this elementary signaling action
creates one IP datagram in each direction. Am I missing something here?

If TCP is not used, the NTLP header can probably be smaller; it is
not clear, but probably there is not much difference in header overhead.
In any case, this overhead is probably trivial compared to the TLV
encoding overhead in the NSLP layer. 

Of course, there is more to the story... maybe refresh messages don't
need the reliable delivery that TCP would force on you (OTOH, maybe
they do need it.)  In case of packet losses, you have to retransmit
with/without TCP, although TCP's retransmission mechanism probably
works better.  TCP gives you congestion control when you need it --
which you may well, when a route change forces a flood of signaling
updates.

OTOH, maybe what you REALLY want for signaling is not TCP but XCP
(see Kitabi paper at SIGCOMM 2002).  A major point of the 2-layer
model is to modularize functionality so you can adapt to later
developments in the transport area.  After all, the NSIS protocol
needs to be able to handle signaling applications that you have
not thought of yet, as well as today's signaling applications.

Finally, in an environment where TCP is "too high overhead", I
suspect that any general-purpose Internet signaling protocol like
NSIS is also too high overhead.  We do not have to use a
back-hoe for spading our gardens.  I think that the NSIS task
is to design the Internet signaling back-hoe.

Bob Braden

---

From: Bob Braden <[email-masked]>
Date: Tue, 4 Feb 2003 20:43:01 GMT

*> > Lixia,
  *> > 
  *> > State recovery is indeed a serious design problem.
  *> 
  *> I believe the step before worrying about recovery is how one defines
  *> failure.
  *> 
  *> (e.g. If the TCP connection gone, has the RSVP failed?)
  *> 
  *> Lixia
  *> 

Lixia,

If this is an exam question, it does not seem hard.  If the connection
is gone, you reestablish it and exchange epoch numbers.  If the
epoch numbers are unchanged, you are all done.  If they have changed,
you know something bad has happened.  Yes?

Bob

---

From: Bob Braden <[email-masked]>
Date: Tue, 4 Feb 2003 21:26:59 GMT

*> 
  *> DNS may offer another lesson. It's obviously the most important user of 
  *> UDP today, but aren't there discussions about stronger TCP support once 
  *> secure DNS becomes more prevalent?
  *> 

Henning,

I would claim that DNS really is a different design point.  Any DNS
server may have to contact any other DNS server; there is nothing
corresponding to "neighbors", long-lived associations that talk only
to each other.  Datagrams really DO make sense in the DNS case.

Bob

---

From: [email-masked]
Date: Wed, 5 Feb 2003 06:36:09 +0200

Hi Henning,

> Btw, RADIUS and DIAMETER may well offer additional lessons, given that 
> they also made the transition from UDP (RADIUS) to TCP/SCTP (DIAMETER). 
> I haven't followed that debate closely enough to distill the lessons.

Well, there are some very practical reasons, especially for accounting,
not to use UDP.  From 2975

   Since RADIUS accounting is based on UDP and timeout and retry
   parameters are not specified, implementations vary widely in their
   approach to reliability, with some implementations retrying until
   delivery or buffer exhaustion, and others losing accounting data
   after a few retries.  Since RADIUS accounting does not provide for
   application-layer acknowledgments or error messages, a RADIUS
   Accounting-Response is equivalent to a transport-layer acknowledgment
   and provides no protection against application layer malfunctions.
   Due to the lack of reliability, it is not possible to do simultaneous
   usage control based on RADIUS accounting alone.  

But, yes, in general, the transport area seems to want to move protocols
away from UDP to TCP whereever possible.

br,
John

---

From: [email-masked]
Date: Wed, 5 Feb 2003 09:30:12 +0200

Hi Lixia,

> One simple example:
> - TCP offers reliable byte stream.
> - That's not what you need.  A later signaling msg may replace an
>   earlier one, yet you cannot take back the bytes that you have
>   already passed to TCP.

You seem to be suggesting that TCP might be more than what is needed,
due to reliable byte stream.  SCTP differs slightly, as it offers
reliable message transfer, but the difference is not so big.
 
> Another example:
> - TCP has this notion of connection
> - when TCP connection broke, what assumption would you make regarding
>   the state at the other signalling party?
>   currently BGP assumes all states were gone, and it ships another
>   copy of that huge routing table of 118000 entries over again
>   (bar the recent graceful restart, that does not handle all
>   the cases anyway)
> 
> And don't assume that TCP sessions between two signalling ends never
> break--if experience with BGP serves an example, it shows 
> they do, and a lot more frequent than one would think

So, what you are suggesting is that the NTLP (NSIS transport layer protcol)
should not equate a TCP connection with a NTLP session, if TCP is used.
I think that is a very good point.

br,
John

---

From: [email-masked]
Date: Wed, 5 Feb 2003 08:40:34 +0100

Hi Lixia, John, all,

Assuming that we are talking about reservations for QoS resources, won't a
BGP route change cause a problem as well? It will cause packets to be sent
over a new (unreserved) path while leaving an unused reservation on the old
path. There is absolutely no guarantee that a reservation will be possible
on the new path. The reason for this is explained in RFC 2990: you cannot
assume that BE-routing context will always allow you to set up QoS
reservations. Would you agree that this can be a problem?

Best regards,
Sven

Lixia Zhang <[email-masked]>@ietf.org on 04/02/2003 21:28:15

Sent by:    [email-masked]

To:    <[email-masked]>
cc:    <[email-masked]>
Subject:    Re: [NSIS] layer split summary? opinions?

>> A question to you:
>>
>>> Pls consider separating two different concepts:
>>> - a soft-state protocol with ACK for each message; and
>>> - a reliable transport protocol underneath RSVP.
>>>
>>> They are very different.
>>> In a later paper (in ICNP'99 I believe) we stated that it is a good
>>> enhancement to add ACK to RSVP.
>>> But I personally consider it INfeasible to put TCP underneath.
>>>
>>> Lixia
>>>
>>> (if someone here runs BGP: how well has that been working)
>>
>> What is the feasibility of running an NSIS transport over TCP to
>> certain entities, like edge routers / gateways, while supporting
>> soft-state for routers & boxes between these edge routers / gateways?
>
> That'll need more time to explain than what I have in hand at this time
> (heading to airport in an hour)
>
> But briefly: the fundamental problem is mismatch of semantics/data
framing
> in providing reliability.
>
> One simple example:

If I may add a 3rd example: consider route changes
(at least BGP doesn't have that problem)

---

From: [email-masked]
Date: Wed, 5 Feb 2003 09:54:07 +0200

Hi Sven,

> reservations. Would you agree that this can be a problem?

I agree, this is a problem, and always has been a problem.  If you use
'normal' IP routing, you are not setting up static pipes, so there is always
some chance that something changes.

Of course, I think of this as a 2nd order problem, meaning that we will
probably need to tackle this, but my strategy has always been to work on 
the general purpose solution and then seen about applying it to some
of these corner cases.  

br,
John

---

From: "Hancock, Robert" <[email-masked]>
Date: Wed, 5 Feb 2003 23:57:16 -0000

John,

since you raise this in the context specifically of SCTP's multi-address support, it would be a lot clearer if you said what the multiple addresses were that you had in mind, and what sort of conditions you were looking for robustness against.

Obviously, sctp can be used to build a pretty robust connection between two NTLP peers (but that isn't really a "crazy idea".)

If you're referring to robustness against endpoint address changes but where the peers are the same pair of nodes, I think I agree with Xiaoming's mail (but that scenario doesn't really relate to intermediate nodes sniffing the contents). This doesn't seem like a crazy idea, but I wouldn't know how to validate the 'same peer' assumption either.

Or, a genuinely crazy idea would be to [somehow] use SCTP's addressing flexibility to re-instate e2e addressing (getting robustness in the form of automatic reaction to re-routing) while still having the advantages [add caveat here that some people think they are disadvantages] of there being a transport protocol running between peers. Unfortunately, I believe you can show that most of the advantageous features of a transport protocol are intrinsically incompatible with e2e addressing (unless maybe you are prepared to develop a basically new transport protocol). Let's discuss it on tuesday.

robert h.

> Sent: 05 February 2003 07:58
> To: [email-masked]; [email-masked]
> Cc: [email-masked]
> Hello all,
> > This is only a problem if there is a long delay in the 
> pipe. Otherwise, 
> > UDP or TCP, the message will be delivered after the same 
> time. (If you 
> > believe head-of-line blocking matters, SCTP would 
> presumably address 
> > that problem.)
> 
> A somewhat crazy idea popped into my head, based upon Henning's text.
> As most of you know, SCTP supports multihoming, allowing support for
> multiple data paths between SCTP endpoints.  Could SCTP, when used
> as an NSIS transport (below NTLP) provide more path robustness?  
> The question is, however, can we engineer something using SCTP that
> intermediate nodes 'sniff' the NTLP/SCTP packets to see if they are
> concerned about the contents?
> 
> br,
> John

---

From: "Hancock, Robert" <[email-masked]>
Date: Wed, 5 Feb 2003 23:57:18 -0000

Dear all,

On the question of whether signalling application behaviour should be tied to TCP connection semantics/operation:

Back at the start of this thread, it was argued that the NSIS transport service (however you name what sits below it) should provide only a stateless message delivery capability (not a state management capability). I'd still be interested in comments on that point.

In any case, I believe that exactly the same arguments say that tying signalling application behaviour to TCP (or any other transport protocol) connection state machine would be a massively bad idea (if indeed we haven't already said that somewhere). The transport service should get messages from A to B; what's in question is how well it should do it, and whether the costs of depending on something TCP-like are worth the benefits.

Or, to use Bob's analogy: a spade and a backhoe both dig conceptually similar things in gardens (i.e. holes). The fact that the backhoe is internally connection-oriented doesn't matter to the gardener.

Cheers,

RobertH.

> Sent: 05 February 2003 07:30
> To: [email-masked]
> Cc: [email-masked]

---

From: [email-masked]
Date: Thu, 6 Feb 2003 08:35:48 +0200

Hi Robert,

> On the question of whether signalling application behaviour 
> should be tied to TCP connection semantics/operation:
> 
> Back at the start of this thread, it was argued that the NSIS 
> transport service (however you name what sits below it) 
> should provide only a stateless message delivery capability 
> (not a state management capability). I'd still be interested 
> in comments on that point.

Anyone have opinions on this?

> In any case, I believe that exactly the same arguments say 
> that tying signalling application behaviour to TCP (or any 
> other transport protocol) connection state machine would be a 
> massively bad idea (if indeed we haven't already said that 
> somewhere). The transport service should get messages from A 
> to B; what's in question is how well it should do it, and 
> whether the costs of depending on something TCP-like are 
> worth the benefits.
> 
> Or, to use Bob's analogy: a spade and a backhoe both dig 
> conceptually similar things in gardens (i.e. holes). The fact 
> that the backhoe is internally connection-oriented doesn't 
> matter to the gardener.

Agreed.

br,
John