Subject: [Spud] FW: New Version Notification for draft-hildebrand-spud-prototype-02.txt

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Tue, 3 Mar 2015 15:59:41 +0000

New version.  Note that the packet format has changed (again).  Sorry about that, but the good news is you should just have to delete some code.  Nice HTML version here:

http://hildjj.github.io/draft-hildebrand-spud-prototype/

On 3/3/15, 4:58 PM, "[email-masked]" <[email-masked]> wrote:

>
>A new version of I-D, draft-hildebrand-spud-prototype-02.txt
>has been successfully submitted by Joe Hildebrand and posted to the
>IETF repository.
>
>Name:		draft-hildebrand-spud-prototype
>Revision:	02
>Title:		Substrate Protocol for User Datagrams (SPUD) Prototype
>Document date:	2015-03-03
>Group:		Individual Submission
>Pages:		14
>URL:            http://www.ietf.org/internet-drafts/draft-hildebrand-spud-prototype-02.txt
>Status:         https://datatracker.ietf.org/doc/draft-hildebrand-spud-prototype/
>Htmlized:       http://tools.ietf.org/html/draft-hildebrand-spud-prototype-02
>Diff:           http://www.ietf.org/rfcdiff?url2=draft-hildebrand-spud-prototype-02
>
>Abstract:
>   SPUD is a prototype for grouping UDP packets together in a "tube",
>   also allowing network devices on the path between endpoints to
>   participate explicitly in the tube outside the end-to-end context.
>
>                                                                                  
>
>
>Please note that it may take a couple of minutes from the time of submission
>until the htmlized version and diff are available at tools.ietf.org.
>
>The IETF Secretariat
>
>

-- 
Joe Hildebrand

---

From: Ted Hardie <[email-masked]>
Date: Tue, 3 Mar 2015 10:28:13 -0800

The draft currently asks whether the Tube ID should be scoped to a single
source address or not.  My preference is to stick to a single 5-tuple now,
as in the common case, the path may change with a change in source address.

There is still some chance that a changing privacy address in V6-land would
not, but let's not optimize for that just now.

Ted

On Tue, Mar 3, 2015 at 7:59 AM, Joe Hildebrand (jhildebr) <
[email-masked]> wrote:

> New version.  Note that the packet format has changed (again).  Sorry
> about that, but the good news is you should just have to delete some code.
> Nice HTML version here:
>
> >URL:
> http://www.ietf.org/internet-drafts/draft-hildebrand-spud-prototype-02.txt
> >Status:
> https://datatracker.ietf.org/doc/draft-hildebrand-spud-prototype/
> >Htmlized:
> http://tools.ietf.org/html/draft-hildebrand-spud-prototype-02
> >Diff:
> http://www.ietf.org/rfcdiff?url2=draft-hildebrand-spud-prototype-02
> >
> >Abstract:
> _______________________________________________
> Spud mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/spud
>

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Tue, 3 Mar 2015 18:30:46 +0000

On 3/3/15, 7:28 PM, "Ted Hardie" <[email-masked]> wrote:

>The draft currently asks whether the Tube ID should be scoped to a single source address or not.  My preference is to stick to a single 5-tuple now, as in the common case, the path may change with a change in source address.
>
>There is still some chance that a changing privacy address in V6-land would not, but let's not optimize for that just now.

I'm ok with that for now.  The next question then is if the Tube ID adds any value at all?  Is the source sockaddr_in[6] enough?  We could save some bytes.

-- 
Joe Hildebrand

---

From: Ted Hardie <[email-masked]>
Date: Tue, 3 Mar 2015 10:50:12 -0800

On Tue, Mar 3, 2015 at 10:30 AM, Joe Hildebrand (jhildebr) <
[email-masked]> wrote:

> >There is still some chance that a changing privacy address in V6-land
> would not, but let's not optimize for that just now.
>
> I'm ok with that for now.  The next question then is if the Tube ID adds
> any value at all?  Is the source sockaddr_in[6] enough?  We could save some
> bytes.
>
> ​I don't think it is enough, because you still need to disambiguate flows
that go between the same endpoints but are different tubes.  The explicit
identifiers allow you to make statements about their relationships more
easily (think about the ordinal ranking use case, where a video app wants
to say "this thumbnail flow is more important than the full-resolution
flow")

​These aren't the bytes to save, in my opinion.

Ted​

---

From: Brian Trammell <[email-masked]>
Date: Tue, 3 Mar 2015 19:53:18 +0100

Sent from my iPhone (but not top quoted)

> On 03 Mar 2015, at 19:30, Joe Hildebrand (jhildebr) <[email-masked]> wrote:
> 

Sub-port level multiplexing for userspace spud implementations, and inability for someone who knows the 5 tuple to spoof p2a messages. It's a 64-bit strong proof you saw the packet, which is strong enough to make brute force expensive. 32 bits isn't. And 48 bit ints are weird, ieee 802 notwithstanding.

(And if we scope tube ids to 5 tuples always, the we can't carry them across v6 privacy addresses. Otherwise, whither privacy?)

---

From: Brian Trammell <[email-masked]>
Date: Tue, 3 Mar 2015 19:57:04 +0100

> On 03 Mar 2015, at 19:50, Ted Hardie <[email-masked]> wrote:
> 
>> On Tue, Mar 3, 2015 at 10:30 AM, Joe Hildebrand (jhildebr) <[email-masked]> wrote:
> ​I don't think it is enough, because you still need to disambiguate flows that go between the same endpoints but are different tubes.  The explicit identifiers allow you to make statements about their relationships more easily (think about the ordinal ranking use case, where a video app wants to say "this thumbnail flow is more important than the full-resolution flow")

Also this; it allows some spud endpoint users to group sub flows and bind properties in interesting ways.

(I was actually wondering if 64 bits are enough to allow enough subflow points while still keeping the active tube id space small)

---

From: Salvatore Loreto <[email-masked]>
Date: Tue, 3 Mar 2015 18:59:22 +0000

I tend to agree with Ted
changing the source IP address in most of the case implies a different path
(i.e. some or all different middle-boxes in the path towards the destination)

using the source sockaddr_in would mean inserting the actual ip address in SPUD
and then making easy for the rest of the path figure it out when you are behind a NAT 

/Sal

> On 03 Mar 2015, at 20:30, Joe Hildebrand (jhildebr) <[email-masked]> wrote:
>

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Tue, 3 Mar 2015 19:00:43 +0000

On 3/3/15, 7:50 PM, "Ted Hardie" <[email-masked]> wrote:

>​I don't think it is enough, because you still need to disambiguate flows that go between the same endpoints but are different tubes.  The explicit identifiers allow you to make statements about their relationships more easily (think about the ordinal ranking
> use case, where a video app wants to say "this thumbnail flow is more important than the full-resolution flow")
>

Although in TCP we currently use different source ports for different sockets, I'm happy to agree with you until/unless we learn something new.

-- 
Joe Hildebrand

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Tue, 3 Mar 2015 19:02:36 +0000

>and then making easy for the rest of the path figure it out when you are behind a NAT

No, I was just suggesting to use the one returned from recvfrom, not to add the address to the SPUD header.

-- 
Joe Hildebrand

---

From: Salvatore Loreto <[email-masked]>
Date: Wed, 4 Mar 2015 18:20:58 +0000

Hi Joe,

extensibility is mentioned in several part of the draft 02,
however I am missing extensibility be listed in the requirements and assumptions list
in section 2.

I would like SPUD would permit extension to provide additional info, but at same time
I would also have a clear explanation on the limitations, i.e. where extensions are effective, when extension
should be ignored etc.

br
Salvatore

---

From: Szilveszter Nadas <[email-masked]>
Date: Thu, 5 Mar 2015 17:04:22 +0000

Hi,

Some minor comments/questions.

Section 5 and 6 "may contain CBOR" contradicts for me with "0 key MUST NOT be used when the adec ot pdec bit is set" of Section 4.6. Or do you mean that adec=0 pdec=0 packets may also contain declarations? 

Section 5: it is unclear for me what does "proof of implementation" mean.

Section 10: "which may always have the following keys"  - I assume that this key is optional but it is often expected to be used. Or shall it always be used when responding to a specific path declarations.

Cheers,
Szilveszter

-----Original Message-----
From: Spud [mailto:[email-masked]] On Behalf Of Joe Hildebrand (jhildebr)
Sent: Tuesday, March 03, 2015 17:00
To: [email-masked]
Subject: [Spud] FW: New Version Notification for draft-hildebrand-spud-prototype-02.txt

On 3/3/15, 4:58 PM, "[email-masked]" <[email-masked]> wrote:

>Please note that it may take a couple of minutes from the time of 
>submission until the htmlized version and diff are available at tools.ietf.org.
>
>The IETF Secretariat
>
>

--
Joe Hildebrand

---

From: Mirja Kühlewind <[email-masked]>
Date: Thu, 05 Mar 2015 21:16:37 +0100

Hi Salvatore,

in the slides we are currently preparing for the BoF we have three constraints 
listed:

- Information exposure is declarative
- All entities may trust but verify
- Information must be incrementally useful (e.g. not all nodes might support spud)

I think this narrows down the scope. I bet we will discuss this further in the 
BoF and therefore leave for now this as it is.

Mirja

---

From: Szilveszter Nadas <[email-masked]>
Date: Fri, 6 Mar 2015 09:45:48 +0000

Hi,

I do not really see how these constraints contradicts extensibility. Also I see a value in pre-discussion in mailing list. I tis not really important for me whether any of the drafts is modified.

It would be funny to ossify middlebox signaling with a protocol which is supposed to help in solving ossification for E2E communication. :) I think I accept the constrains below, it would be very hard to incrementally deploy SPUD otherwise. 

A question related to this is in the back of my mind for a while. Shall SPUD standardize the minimum amount of common/necessary signaling information elements and allow any signaling solution, even proprietary, using SPUD to reach the middleboxes? Or shall SPUD include most of the information elements needed? I am in favor of the first, because that allows innovation, though clearly interop is much more a hassle. (If you think about 5-tuple ICMP, that is in fact an example of the first.) Also the first principle would help focusing the scope of the WG more, which might be an advantage. Maybe even signaling protocol design guidelines could be created similarly to the proposed transport protocol design guidelines. Clearly the constraints below do require a different thinking about how to design signaling protocols over SPUD.

Cheers,
Szilveszter

-----Original Message-----
From: Spud [mailto:[email-masked]] On Behalf Of Mirja Kühlewind
Sent: Thursday, March 05, 2015 21:17
To: Salvatore Loreto; Joe Hildebrand (jhildebr) <jhildebr@cisco. com>
Cc: [email-masked]
Subject: Re: [Spud] Extensibility (wa New Version Notification for draft-hildebrand-spud-prototype-02.txt)

Hi Salvatore,

I think this narrows down the scope. I bet we will discuss this further in the BoF and therefore leave for now this as it is.

Mirja

On 04.03.2015 19:20, Salvatore Loreto wrote:
> Hi Joe,
>
> extensibility is mentioned in several part of the draft 02, however I 
> am missing extensibility be listed in the requirements and assumptions 
> list in section 2.
>
> I would like SPUD would permit extension to provide additional info, 
> but at same time I would also have a clear explanation on the 
> limitations, i.e. where extensions are effective, when extension should be ignored etc.
>
>
> br
> Salvatore
>
>
>
>

---

From: Patrick McManus <[email-masked]>
Date: Fri, 6 Mar 2015 15:52:22 -0500

First - Thanks Joe and Brian. I have a few margin notes to share :)

Next, I may just be confused because I have read the text less than the
authors at this point, but it seems as if unidirectional flows are off the
table.. the text doesn't really say this, but the open-ack seems to imply
it. There certainly are unidirectional udp flows but some clarification
would help.

Thirdly, is there advice given or implied for middleboxes that see data
spud packets on opening tubes? The only thing I really see about the
opening state is the diagram - but the diagram applies to the endpoint, not
to the middlebox state, right? I'm concerned the text will read like there
needs to be a full rtt handshake before data packets can flow on a spud
tube and there are counter examples to things we would like to run on spud
already.

Fourthly, there are a couple "reserved bits MUST be zero".. should that be
"senders MUST set to zero, receivers MUST ignore "?

Fifthly, on the topic of whether tube-ids need a consistent 5 tuple, I
think there are several cases where it would be nice if they didn't..  load
balancers are a good example - a request is sent to a central service that
parses it and dispatches it to the best back end server.. historically that
server needs to lie about its address somehow when sending the reply
(either via proxy or more likely via spoofing - and if spoofed you need to
figure out the ack traffic). It would be cool to have that all in one tube
with real addresses so the transport could decide how to deal with those
issues.

Sixthly, am I misreading the 8.2 MUST that SPUD enabled NATs need to share
internal addressing info on the external side of the NAT? Is that really a
MUST - some folks will be frothing at the mouth with the perceived
information disclosure issues.

Again, thanks for this.

---

From: Ken Calvert <[email-masked]>
Date: Fri, 6 Mar 2015 16:46:00 -0500

On 6 Mar 2015, at 15:52, Patrick McManus <[email-masked]> wrote:

> Next, I may just be confused because I have read the text less than the authors at this point, but it seems as if unidirectional flows are off the table.. the text doesn't really say this, but the open-ack seems to imply it. There certainly are unidirectional udp flows but some clarification would help.

Actually, the state machine diagram - which I interpreted as applying at ANY specific point along a path - doesn't (quite) imply the above, at least as I interpret it.  It seems to work fine for a middlebox that only sees the initiator->responder side (i.e., the only side of a unidirectional flow).  Although in that case it looks like the initiator doesn't get out of "resuming" state...

> ...

> Fourthly, there are a couple "reserved bits MUST be zero".. should that be "senders MUST set to zero, receivers MUST ignore "?

Along those lines, it seems the intent is that ONLY endpoints touch the "command" bits, and that middleboxes MUST NOT modify them.  (SHOULD NOT?  Are there non-nefarious use cases?)  This should probably be made explicit; I didn't see it anywhere, sorry if I missed it.

Ken Calvert

---

From: Salvatore Loreto <[email-masked]>
Date: Sat, 7 Mar 2015 08:24:34 +0000

> On 06 Mar 2015, at 22:52, Patrick McManus <[email-masked]> wrote:
> 
> 
> Thirdly, is there advice given or implied for middleboxes that see data spud packets on opening tubes? The only thing I really see about the opening state is the diagram - but the diagram applies to the endpoint, not to the middlebox state, right? I'm concerned the text will read like there needs to be a full rtt handshake before data packets can flow on a spud tube and there are counter examples to things we would like to run on spud already.

I agree on this point, I was going to send a similar comment

It should be clear in the spec that the endpoint will be able to send data right after having 
sent the SPUD packet with the "open”command, without have to wait a full rtt 
(i.e. the end point should not be forced to wait for the creation acknowledge of a tube to start sending data)

actually it should be stretched to the point to make possible for the end point to star sending data in the same message containing the
SPUD packet with the “open” command

 /Salvatore

---

From: "Pal Martinsen (palmarti)" <[email-masked]>
Date: Mon, 9 Mar 2015 08:06:07 +0000

> On 07 Mar 2015, at 09:24, Salvatore Loreto <[email-masked]> wrote:
> 
> 

Clients using ICE to open up NAT/FW pinholes can probably do some smart things with the STUN connectivity checks as well. This would allow for and endpoints to check the available network paths and choose the one that supports SPUD.

I have a feeling that ICE/STUN and SPUD will be somewhat intermingled for bi-directional UDP streams like interactive video (webRTC). Probably a bit premature to have this discussion.

.-.
Pål-Erik

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Mon, 9 Mar 2015 19:29:12 +0000

>Section 5 and 6 "may contain CBOR" contradicts for me with "0 key MUST NOT be used when the adec ot pdec bit is set" of Section 4.6. Or do you mean that adec=0 pdec=0 packets may also contain declar…

I was thinking about end-to-end things for the non-0 keys in an open packet. a0p0 packets may contain non-0 keys, but I'm not thinking of those as declarations.  We'll find out more as we prototype a real transport on top of SPUD.

I suggest we leave it intentionally vague for the moment.

>Section 5: it is unclear for me what does "proof of implementation" mean. 

e.g. websockets (RFC 6455, section 1.3).  The client provides a nonce.  The server does some calculation that proves with higher confidence than the magic number provided that it is speaking SPUD, sending the result in the ack packet.

>Section 10: "which may always have the following keys"  - I assume that this key is optional but it is often expected to be used. Or shall it always be used when responding to a specific path declar…

Probably only when responding to a specific path declaration.  I'm going to change the name to "midpoint token" to make it clear that this is not a cookie in the HTTP sense.

-- 
Joe Hildebrand

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Mon, 9 Mar 2015 19:42:13 +0000

>however I am missing extensibility be listed in the requirements and assumptions list

I just added: 

"- SPUD must enable multiple new transport semantics without requiring updates
  to SPUD implementations in middleboxes."

I think that captures the most important of the extensibility directions.  We can edit this requirement going forward to capture other axes of extensibility.

>I would also have a clear explanation on the limitations, i.e. where extensions are effective, when extension

"Must ignore (and pass along unchanged) what you don't understand" is going to be one of the most powerful agents of change we have at our disposal.  Let's make sure it's well-understood.

-- 
Joe Hildebrand

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Mon, 9 Mar 2015 21:27:09 +0000

On 3/6/15, 2:45 AM, "Szilveszter Nadas" <[email-masked]> wrote:

>A question related to this is in the back of my mind for a while. Shall SPUD standardize the minimum amount of common/necessary signaling information elements and allow any signaling solution, even proprietary, using SPUD to reach the middleboxes? Or shall SPUD include most of the information elements needed? I am in favor of the first, because that allows innovation, though clearly interop is much more a hassle. 

I mostly agree.  I want to make sure we get anything that *all* implementations need to support into the base spec.  Then we can move all of the other extensions into separate documents.

-- 
Joe Hildebrand

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Mon, 9 Mar 2015 21:37:05 +0000

On 3/6/15, 1:52 PM, "Patrick McManus" <[email-masked]> wrote:

>Next, I may just be confused because I have read the text less than the authors at this point, but it seems as if unidirectional flows are off the table.. the text doesn't really say this, but the open-ack seems to imply it. There certainly are unidirectional
> udp flows but some clarification would help.

It wasn't my intent to rule them out, but we probably need to think about how to ensure that the state transitions make that more clear.

>Thirdly, is there advice given or implied for middleboxes that see data spud packets on opening tubes? 

Not yet.

>The only thing I really see about the opening state is the diagram - but the diagram applies to the endpoint, not to the middlebox state, right? I'm
> concerned the text will read like there needs to be a full rtt handshake before data packets can flow on a spud tube and there are counter examples to things we would like to run on spud already.

Agree.  This really isn't clear yet and it needs to be.

>Fourthly, there are a couple "reserved bits MUST be zero".. should that be "senders MUST set to zero, receivers MUST ignore "?

Yes.  Willfix.

>Fifthly, on the topic of whether tube-ids need a consistent 5 tuple, I think there are several cases where it would be nice if they didn't..  load balancers are a good example - a request is sent to a central service that parses it and dispatches it to
> the best back end server.. historically that server needs to lie about its address somehow when sending the reply (either via proxy or more likely via spoofing - and if spoofed you need to figure out the ack traffic). It would be cool to have that all in one
> tube with real addresses so the transport could decide how to deal with those issues.

I like the use case.

>Sixthly, am I misreading the 8.2 MUST that SPUD enabled NATs need to share internal addressing info on the external side of the NAT? Is that really a MUST - some folks will be frothing at the mouth with the perceived information disclosure issues.

The intent was that the mapping is only shared internally.  I'll see if I can fix the wording.

-- 
Joe Hildebrand

---

From: Ted Hardie <[email-masked]>
Date: Mon, 9 Mar 2015 15:05:54 -0700

On Mon, Mar 9, 2015 at 2:37 PM, Joe Hildebrand (jhildebr) <
[email-masked]> wrote:

> parses it and dispatches it to
> > the best back end server.. historically that server needs to lie about
> its address somehow when sending the reply (either via proxy or more likely
> via spoofing - and if spoofed you need to figure out the ack traffic). It
> would be cool to have that all in one
I like the use case.

​So, I may need a whiteboard here, because the way I see this use case
sharing a tube-id seems like a loss.

What I think you're saying is Client A (from 192.0.2.1) sends traffic
toward service Z (at 203.0.113.1) .  The packet goes from A across a path
until it reaches middlebox L (at 198.51.100.1) which dispatches​ it toward
a backend server Z-15 (at 203.0.113.15) which would currently spoof
203.0.113.1.  This often forces L to maintain a map for the tuple in A's
packets to the correct backend.  A tube-id would simplify this (since L
could use it instead of a tuple).

​What I hear you saying is that you'd prefer to allow Z-15 to emit a packet
using its own IP address, rather than Z's, and have it be understood to be
in the same flow by dint of it sharing the same tube-id.    This gives the
tube-id some serious security requirements that it doesn't currently (as an
advisory bit) have.  Have I misunderstood you?   Do you want something
different here?

Ted​

---

From: Patrick McManus <[email-masked]>
Date: Tue, 10 Mar 2015 16:18:26 -0400

On Mon, Mar 9, 2015 at 6:05 PM, Ted Hardie <[email-masked]> wrote:

> So, I may need a whiteboard here, because the way I see this use case
> ​What I hear you saying is that you'd prefer to allow Z-15 to emit a
> packet using its own IP address, rather than Z's, and have it be understood
> to be in the same flow by dint of it sharing the same tube-id.    This
> gives the tube-id some serious security requirements that it doesn't
> currently (as an advisory bit) have.  Have I misunderstood you?   Do you
> want something different here?
>

"want" is such a strong word :) I'm just brainstorming.

There is a case where L doesn't want to be part of the flow after Z-15 is
selected, but obviously it needs to be in a TCP scenario if for no other
reason than to forward the ack traffic to Z-15 - the fact that it continues
to be involved is really an artifact of the protocol - L isn't a typical on
path middlebox; but spuds is still interesting here for its interaction
with something like a client side NAT. This is an ordered flow (a tube?)
with 3 end points.

I don't really know that the tube-id itself takes on security requirements
here, the transport running in the tube might have a totally self defined
security and identification model independent of the tube-id - mobility
leads you down that path naturally. (but yes - that's a hard problem :)).
Is there a reason that gets bumped up to the spuds wrapper?

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Tue, 10 Mar 2015 20:52:28 +0000

On 3/10/15, 2:18 PM, "Patrick McManus" <[email-masked]> wrote:

>There is a case where L doesn't want to be part of the flow after Z-15 is selected, but obviously it needs to be in a TCP scenario if for no other reason than to forward the ack traffic to Z-15 - the fact that it continues to be involved
> is really an artifact of the protocol - L isn't a typical on path middlebox; but spuds is still interesting here for its interaction with something like a client side NAT. This is an ordered flow (a tube?) with 3 end points.

I could imagine L sending a path declaration that allows Z-15 to take over, basically an in-flight hand-off.  We would have to get that protocol correct early on in the process so that everyone implements it, though.

-- 
Joe Hildebrand

---

From: Ted Hardie <[email-masked]>
Date: Tue, 10 Mar 2015 14:02:55 -0700

On Tue, Mar 10, 2015 at 1:52 PM, Joe Hildebrand (jhildebr) <
[email-masked]> wrote:

> to be involved
> > is really an artifact of the protocol - L isn't a typical on path
> middlebox; but spuds is still interesting here for its interaction with
> something like a client side NAT. This is an ordered flow (a tube?) with 3
> end points.
>
> I could imagine L sending a path declaration that allows Z-15 to take
> over, basically an in-flight hand-off.  We would have to get that protocol
> correct early on in the process so that everyone implements it, though.
>
>
​So, I don't think this is the right thing to do in a substrate.  You may
want to build a protocol that can do this on top of the substrate's
capabilities, but gating acceptance of the substrate on this particular use
case seems to me unwarranted fate sharing.

If you want an application-layer protocol that can handle service handoff,
information like "Z-15 has path to path information for tube-id N" as an
advisory message may be useful.    But if Z-15 has a different IP address,
as above, there is no guarantee that the path will remain the same;
anything from ECMP on up may change it.

Call me old-fashioned (or make me one, even better), but I believe it is
easier here to use anycast IP addressing for the source addresses while
allowing the tube-id to be used to dispatch to the right back end.  In an
SDN world, you might not even need that to be in L, but in the baseline
network elements matching tables.

regards,

Ted

---

From: Brian Trammell <[email-masked]>
Date: Tue, 10 Mar 2015 22:15:15 +0100

hi Patrick, all,

...in the hopes the conversation has not overtaken me, see inline below...

> On 10 Mar 2015, at 21:18, Patrick McManus <[email-masked]> wrote:
> 
> So, I may need a whiteboard here, because the way I see this use case sharing a tube-id seems like a loss.
> 
> What I think you're saying is Client A (from 192.0.2.1) sends traffic toward service Z (at 203.0.113.1) .  The packet goes from A across a path until it reaches middlebox L (at 198.51.100.1) which dispatches​ it toward a backend server Z-15 (at 203.0.113.15) which would currently spoof 203.0.113.1.  This often forces L to maintain a map for the tuple in A's packets to the correct backend.  A tube-id would simplify this (since L could use it instead of a tuple).
> 
> ​What I hear you saying is that you'd prefer to allow Z-15 to emit a packet using its own IP address, rather than Z's, and have it be understood to be in the same flow by dint of it sharing the same tube-id.    This gives the tube-id some serious security requirements that it doesn't currently (as an advisory bit) have.  Have I misunderstood you?   Do you want something different here?
> 
> There is a case where L doesn't want to be part of the flow after Z-15 is selected, but obviously it needs to be in a TCP scenario if for no other reason than to forward the ack traffic to Z-15 - the fact that it continues to be involved is really an artifact of the protocol - L isn't a typical on path middlebox; but spuds is still interesting here for its interaction with something like a client side NAT. This is an ordered flow (a tube?) with 3 end points.
> 
> I don't really know that the tube-id itself takes on security requirements here, the transport running in the tube might have a totally self defined security and identification model independent of the tube-id - mobility leads you down that path naturally. (but yes - that's a hard problem :)). Is there a reason that gets bumped up to the spuds wrapper?

If we scope a tube ID to a five tuple, then it has some very nice properties. Since we're probably multiplexing significantly less than 2**64 tubes over a five tuple, the currently-active tube ID space is extremely sparse, and if the tube ID is sufficiently random, it is hard enough to guess them that we can presume that anyone that has the ID of a given tube is one of the endpoints, is on path, or is cooperating with a device on path. This is a property we need in any case in which we're requiring middleboxes to spoof endpoints, which itself will be necessary for NAT traversal.

Further, the five tuple has really become the state lookup key in each domain of routability in which its IP addresses are valid. So if the session ID is scoped to this key, it never has to be rewritten or otherwise handled by the middleboxes *other* than to bind a group of packets together so some common properties can be assigned to them.

If you use the tube ID for mobility -- i.e., to cover the "my endpoint address changed, but my session is the same" case, which I very much wanted to use it for when kicking this idea around much earlier -- now the scope is different, and all this gets much messier. The tube ID has to be chosen by the initiating endpoint, which in a mobile-client world is often the endpoint whose address may change, so scoping to the three-tuple of the fixed-address side of the connection brings with it a coordination problem -- and negotiation / decollision of tube IDs adds complexity and roundtrips, both of which are to be avoided.

Of course you could fix that problem by scoping the tube ID to the entire Internet, but this would (1) need way more bits, (2) probably need decollision anyway, and (3) has less than optimal (read: "terrifying") privacy implications for trackability etc.

What I can see being useful here is a facility built in to the "core" spud vocabulary that says "this tube is related to some other tube, here's the ID", for a small set of values of "related". This is more a signal to the remote endpoint than the path, though, so it's not clear it needs to be exposed in SPUD. One tube ID linkage that would need to be exposed to the path is "prefer to drop packets in this tube before packets in tube ID x" or "prefer to forward packets in tube ID x before packets in this tube", which gives you a nice simple vocabulary for expressing almost everything you need to for differential QoS from a given source without allowing global marking with all its attendant mark inflation problems.

Of course, since everything along the path and everything cooperating with everything along the path knows the tube id, any tube linkage features probably have implications for spoofing we'll need to be careful about.

Cheers,

Brian

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Tue, 10 Mar 2015 21:16:21 +0000

On 3/10/15, 3:02 PM, "Ted Hardie" <[email-masked]> wrote:

>​So, I don't think this is the right thing to do in a substrate.  You may want to build a protocol that can do this on top of the substrate's capabilities, but gating acceptance of the substrate on this particular use case seems to me unwarranted fate sharing.

I'll buy that argument.  One notch simpler would be "see-other-address", with some indication of whether your initial packet had been forwarded or not.

>If you want an application-layer protocol that can handle service handoff, information like "Z-15 has path to path information for tube-id N" as an advisory message may be useful.    But if Z-15 has a different IP address, as above, there is no guarantee that
> the path will remain the same; anything from ECMP on up may change it.

Agree.

>Call me old-fashioned (or make me one, even better), but I believe it is easier here to use anycast IP addressing for the source addresses 

Did you mean the responder's (server's?) address here?  "Source" is a little confusing to me.

>while allowing the tube-id to be used to dispatch to the right back end.  In an SDN world, you might not even need that
> to be in L, but in the baseline network elements matching tables.

That doesn't sound old-fashioned to me at all. (I hope this message becomes the top Google hit for "good old-fashioned SDN", for example)

I like the potential combinations of anycast, SDN, and tube-IDs as sub-addresses, though.  The responder probably needs to note the address that was used to reach it, and use that address as the source on it's outbound packets - sendmsg(2) allows for that on some systems.

-- 
Joe Hildebrand

---

From: Patrick McManus <[email-masked]>
Date: Tue, 10 Mar 2015 17:29:11 -0400

On Tue, Mar 10, 2015 at 5:15 PM, Brian Trammell <[email-masked]> wrote:

> What I can see being useful here is a facility built in to the "core" spud
> vocabulary that says "this tube is related to some other tube, here's the
> ID", for a small set of values of "related". This is more a signal to the
> remote endpoint than the path, though, so it's not clear it needs to be
> exposed in SPUD.

I think you can extend the use case under discussion to think about this as
a second tube initiated in the opposite direction.

If I have tube (a->b) started on my home lan that goes through my nat.. and
I get back tube (c->a [related to a->b]) then the tube is implicitly asking
to be allowed through the "nat/firewally-properties" based on the fact that
a->b is alive and was initiated from the a side. That's probably got some
ugly properties - but for handoff it has some nice ones (no rendezvous or
simultaneous open tricks, etc..).. maybe worth exploring. Or maybe we've
explored it here already ;)

It just seems if we're going to be signalling middleboxes, this is one of
the roles they play.

---

From: Ted Hardie <[email-masked]>
Date: Tue, 10 Mar 2015 14:31:40 -0700

On Tue, Mar 10, 2015 at 2:16 PM, Joe Hildebrand (jhildebr) <
[email-masked]> wrote:

> easier here to use anycast IP addressing for the source addresses
>
> ​Yes, I did, sorry for the lack of clarity.​

> >while allowing the tube-id to be used to dispatch to the right back end.
> In an SDN world, you might not even need that
> That doesn't sound old-fashioned to me at all. (I hope this message
> becomes the top Google hit for "good old-fashioned SDN", for example)
>
> I like the potential combinations of anycast, SDN, and tube-IDs as
> sub-addresses, though.  The responder probably needs to note the address
> that was used to reach it, and use that address as the source on it's
> outbound packets - sendmsg(2) allows for that on some systems.
>
> ​The critical thing about using tube-IDs as sub-addresses (or dispatch
tokens, if you prefer), is that the SDN system you're using has to see them
in the bits of the address it sees.  This, it turns out, may not be
trivial, though updates to the relevant standards to make it so are
possible.

Ted

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Tue, 10 Mar 2015 21:31:44 +0000

On 3/10/15, 3:15 PM, "Brian Trammell" <[email-masked]> wrote:

>What I can see being useful here is a facility built in to the "core" spud vocabulary that says "this tube is related to some other tube, here's the ID", for a small set of values of "related". This is more a signal to the remote endpoint than the path, though, so it's not clear it needs to be exposed in SPUD. 

I think you're right that the tube relationships don't need to be made explicit to path elements.  Stitching together multiple tubes into a session-like thing seems like a job for the new layer inside SPUD.  In this case, whenever you needed a new 5tuple, you'd crank up a new tube.  Similar to mptcp, but without tying the flows together.

>One tube ID linkage that would need to be exposed to the path is "prefer to drop packets in this tube before packets in tube ID x" or "prefer to forward packets in tube ID x before packets in this tube", which gives you a nice simple vocabulary for expressing almost everything you need to for differential QoS from a given source without allowing global marking with all its attendant mark inflation problems.

That's a different kind of relationship than I was considering.  I think that would Dan Wing and Pal Martinsen the tools they need for media QoS.  

>Of course, since everything along the path and everything cooperating with everything along the path knows the tube id, any tube linkage features probably have implications for spoofing we'll need t…

There would have to be some sort of proof that the flows are from the "same" endpoint; maybe the open packets would include something signed by the same private key, for instance.

-- 
Joe Hildebrand

---

From: Ted Hardie <[email-masked]>
Date: Tue, 10 Mar 2015 14:35:01 -0700

On Tue, Mar 10, 2015 at 2:31 PM, Joe Hildebrand (jhildebr) <
[email-masked]> wrote:

>
> >One tube ID linkage that would need to be exposed to the path is "prefer
> to drop packets in this tube before packets in tube ID x" or "prefer to
> forward packets in tube ID x before packets in this tube", which gives you
> a nice simple vocabulary for expressing almost everything you need to for
> differential QoS from a given source without allowing global marking with
> all its attendant mark inflation problems.
>
> ​Yes, that's quite similar to the "ordinal priority" requirement that we
discussed in relation to WebRTC where you may want to indication which of a
related set of flows should have priority based on application semantics).

That's a different kind of relationship than I was considering.  I think
> that would Dan Wing and Pal Martinsen the tools they need for media QoS.
>
> >Of course, since everything along the path and everything cooperating
> with everything along the path knows the tube id, any tube linkage features
> probably have implications for spoofing we'll need to be careful about.
>
> There would have to be some sort of proof that the flows are from the
> "same" endpoint; maybe the open packets would include something signed by
> the same private key, for instance.
>

​Are you thinking a TOFU-like trust?  I can demonstrate that flow N and
flow N+1 are from the same source, even though I cannot tell you anything
about the source that emitted them?

Ted​

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Tue, 10 Mar 2015 21:38:19 +0000

On 3/10/15, 3:35 PM, "Ted Hardie" <[email-masked]> wrote:

>​Are you thinking a TOFU-like trust?  I can demonstrate that flow N and flow N+1 are from the same source, even though I cannot tell you anything about the source that emitted them?

I think that's the right property to shoot for, privacy-wise.  If the sender wants to share the keying information around to another box in order to enable tube-sorting, that is its business.

I guess that's TOFU-ish in that I would remember at least the public key info from the first tube if I wanted to do tube-sorting.

-- 
Joe Hildebrand

---

From: "Joe Hildebrand (jhildebr)" <[email-masked]>
Date: Tue, 10 Mar 2015 21:40:34 +0000

On 3/10/15, 3:29 PM, "Patrick McManus" <[email-masked]> wrote:

>I think you can extend the use case under discussion to think about this as a second tube initiated in the opposite direction.

That's an interesting way of thinking about it. 

>If I have tube (a->b) started on my home lan that goes through my nat.. and I get back tube (c->a [related to a->b]) then the tube is implicitly asking to be allowed through the "nat/firewally-properties" based on the fact that a->b
> is alive and was initiated from the a side. That's probably got some ugly properties - but for handoff it has some nice ones (no rendezvous or simultaneous open tricks, etc..).. maybe worth exploring. Or maybe we've explored it here already ;)
>
>It just seems if we're going to be signalling middleboxes, this is one of the roles they play.

The firewall people aren't going to be very excited about inbound open packets unless we can do a strong hand-off from one flow to the next.  I wonder if this ends up being the same math as the tube bundling bits for QoS...

-- 
Joe Hildebrand

---

From: "Philipp S. Schmidt" <[email-masked]>
Date: Wed, 11 Mar 2015 14:32:14 +0100

My first impression about SPUD was more of a session layer than of “just” a middle box signaling layer…

> On 10.03.2015, at 22:31, Joe Hildebrand (jhildebr) <[email-masked]> wrote:
> 

… therefore I would rather think of the Tube ID as some kind of session ID all flows that
belong to a session share. A little like the key in MP_CAPABLE extension of MPTCP.
This implies there can be multiple conversations on different 5-tupels sharing a Tube ID.

> That's a different kind of relationship than I was considering.  I think that would Dan Wing and Pal Martinsen the tools they need for media QoS.

In case of such a thing, I would rather try to avoid using Tube IDs.

> There would have to be some sort of proof that the flows are from the "same" endpoint; maybe the open packets would include something signed by the same private key, for instance.

The question is: do we care - or do we expect that packets related to this tube should be checked by the application to e.g. allow middle boxes to announce an alternative endpoint by referencing the tube id?

AVE!
  Philipp S. Schmidt / phils…
-- 
   {phils}--->---([email-masked])--->---(http://phils.in-panik.de)----,
      wenn w eine   aube ist dn      man au dran dre en                   |
           o     Schr        an muss     hc         h   (Kurt Schwitters) |
:wq!  <---(phone: +49-179-6737439)---<---(jabber: [email-masked])---'

---

From: Brian Trammell <[email-masked]>
Date: Wed, 11 Mar 2015 14:47:03 +0100

> On 11 Mar 2015, at 14:32, Philipp S. Schmidt <[email-masked]> wrote:
> 

It seems to me like part of the problem we're having here is that we have (in the SPUD prototype) a single tube ID, which is endpoint and path visible, and we're wanting to do things with it that have fundamentally different scoping rules.

Perhaps we can apply the question that led to the separation of TCP and IP in the first place: does the path need to see it / can the path do something universally useful with it? Put it in SPUD. Is it end-to-end? Stick it above SPUD, encrypt it, and hide the key from the path.

Put that way, session IDs binding flows together seem to me to be an end-to-end thing. But of course different situations might have different requirements.

Cheers,

Brian

---

From: Patrick McManus <[email-masked]>
Date: Wed, 11 Mar 2015 10:52:27 -0400

On Wed, Mar 11, 2015 at 9:47 AM, Brian Trammell <[email-masked]> wrote:

>
> Perhaps we can apply the question that led to the separation of TCP and IP
> in the first place: does the path need to see it / can the path do
> something universally useful with it? Put it in SPUD. Is it end-to-end?
> Stick it above SPUD, encrypt it, and hide the key from the path.
>
>
I like that as a guiding principle

> Put that way, session IDs binding flows together seem to me to be an
> end-to-end thing. But of course different situations might have different
> requirements.
>
>
I feel like the sticky part here is that middle boxes on the paths are
playing a role here where the information can be useful.. in particular
without this information they aren't able to route e2e based on just 5
tuple because of the nat/firewall translations they do.

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 17 Mar 2015 16:01:40 +1100

Catching up...

> On 12 Mar 2015, at 1:52 am, Patrick McManus <[email-masked]> wrote:
> 

+1 here too. Nice way to avoid this becoming an application services kitchen sink.

--
Mark Nottingham   https://www.mnot.net/

---

From: Brian Trammell <[email-masked]>
Date: Tue, 17 Mar 2015 09:17:50 +0100

> On 17 Mar 2015, at 06:01, Mark Nottingham <[email-masked]> wrote:
> 
> Catching up...
> 

If we really want to build a nice expansive vocabulary for end to end metadata signaling, and we want to reuse chunks of the SPUD codebase to do so (maybe so the app can take the same signals and selectively expose them to the path or not) and we have a solution to the tube MTU problem which works so we know how many bits we can burn, and we believe we can restrict it to the point that it will ever interoperable -- this latter part is where I get skeptical -- then we can do this very simply, inside the crypto veil:

 +--------------------------+
 | addr addr ttl stuff      | IP
 +--------------------------+
 | port port csum len       | UDP
 +--------------------------+
 | magic tube flags cbor    | SPUD
 +--------------------------+             pathside
 | cbor 0: DTLS             | -----------------------
 +--------------------------+              appside
 | cbor                     | SPUD-lite
 +--------------------------+
 | cbor 0: payload          | your favorite transport
 +--------------------------+

Here SPUD-lite is *just* the CBOR part of SPUD.

I'm not sure this is a good idea -- though the diagram above looks a little less scary if you think of the goal of SPUD as a facility of reduced-profile, future DTLS -- but this arrangement would give us the flexibility to define things we want to say first then to freely move them between pathside and appside contexts. On the appside, the trust situation is easier because we presume we've solved the key-and-trust-management problem in a way that makes sense for a given application.

Cheers,

Brian