Subject: [mif] Fwd: New Version Notification for draft-kline-mif-mpvd-api-reqs-00.txt

From: Erik Kline <[email-masked]>
Date: Mon, 2 Nov 2015 00:08:19 +0900

All,

Quite, quite late, but I took a stab at scrapping the presentation from
Prague and writing instead an MPvD API requirements document.

-ek

---------- Forwarded message ----------
From: <[email-masked]>
Date: 2 November 2015 at 00:03
Subject: New Version Notification for draft-kline-mif-mpvd-api-reqs-00.txt

A new version of I-D, draft-kline-mif-mpvd-api-reqs-00.txt
has been successfully submitted by Erik Kline and posted to the
IETF repository.

Name:           draft-kline-mif-mpvd-api-reqs
Revision:       00
Title:          Multiple Provisioning Domains API Requirements
Document date:  2015-11-01
Group:          Individual Submission
Pages:          15
URL:
https://www.ietf.org/internet-drafts/draft-kline-mif-mpvd-api-reqs-00.txt
Status:
https://datatracker.ietf.org/doc/draft-kline-mif-mpvd-api-reqs/
Htmlized:       https://tools.ietf.org/html/draft-kline-mif-mpvd-api-reqs-00

Abstract:
   RFC 7556 [RFC7556] provides the essential conceptual guidance an API
   designer would need to support use of PvDs.  This document aims to
   capture the requirements for an API that can be used by applications
   that would be considered "advanced", according to section 6.3 [1] of
   RFC 7556 [RFC7556].  The "basic" [2] and "intermediate" [3] API
   support levels can in principle be implemented by means of layers
   wrapping the advanced API.

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat

---

From: Stjepan Groš <[email-masked]>
Date: Mon, 2 Nov 2015 10:44:14 +0100

Hi Erik,

Thank you for the draft.

I have few comments/questions.

First, what I'm reading from the draft is that each PvD aware
application should have access to a single PvD at any single moment? If
that is so, what about mpTCP and SCTP applications that are PvD aware?
They should have access to multiple PvDs simultaneously for a fail over
reasons. So, how this draft takes into account that use case?

In section 3.4 there is a note/question about "loopback PvD". What we
were thinking/discussing is that there should be such a PvD for
backwards compatibility, i.e. for non-PvD aware applications that for
some reason expect all available connections and interfaces (PvDs) to be
present (non-PvD aware mpTCP and SCTP applications, UDP applications,
debugging applications like tcpdump/wireshark). This configuration we
called internally "root network name space".

SG

On 01.11.2015 16:08, Erik Kline wrote:
> All,
>
> Quite, quite late, but I took a stab at scrapping the presentation
> from Prague and writing instead an MPvD API requirements document.
>
> -ek
>
>
> From: <[email-masked] <mailto:[email-masked]>>
> Htmlized:     
>  https://tools.ietf.org/html/draft-kline-mif-mpvd-api-reqs-00
>
>
> Abstract:
> until the htmlized version and diff are available at tools.ietf.org
> <http://tools.ietf.org>.
>
> The IETF Secretariat
>
>

---

From: Erik Kline <[email-masked]>
Date: Mon, 2 Nov 2015 21:18:27 +0900

On 2 November 2015 at 18:44, Stjepan Groš <[email-masked]> wrote:

> First, what I'm reading from the draft is that each PvD aware application
> should have access to a single PvD at any single moment? If that is so,
> what about mpTCP and SCTP applications that are PvD aware? They should have
> access to multiple PvDs simultaneously for a fail over reasons. So, how
> this draft takes into account that use case?
>

Not quite: they should be able to specify only one PvD per function call,
per thread (for example).  An app can have multiple sockets, each one using
one PvD.  An app can have something like a single unconnected datagram
socket and call sendto() with a different PvD for each sendto() call.  That
kind of thing is more what I meant to write, but I guess was not very
clear.  (Clearly, I have a bunch of sockets-y things in my head but am
trying to write things in a more general, non-sockets-dependent way.)

The confusion may be arising in part from the resolution of which PvD to
use for a given function call if no PvD is specified.  In those cases, the
system checks for "defaults" at increasingly general levels.  So a process
could set a process default PvD and avoid having to specify the PvD for
each an every call, but still make individual socket calls or query DNS on
a different PvD whenever it wants to.

Does this make more sense?

There are necessarily going to be sequences of function calls after which
trying to change a PvD will be non-sensical.  For example, after I have a
connected TCP socket: a source address has been chosen from a PvD and the
PvD's routing table is in use for every write()/send() I call.  Attempting
to change which PvD is use is a bit non-sensical at that point, and I think
perhaps that an API should not really allow it.

In section 3.4 there is a note/question about "loopback PvD". What we were
> thinking/discussing is that there should be such a PvD for backwards
> compatibility, i.e. for non-PvD aware applications that for some reason
> expect all available connections and interfaces (PvDs) to be present
> (non-PvD aware mpTCP and SCTP applications, UDP applications, debugging
> applications like tcpdump/wireshark). This configuration we called
> internally "root network name space".
>

That sounds a bit different from what I was thinking of a loopback PvD.
What you describe sounds like how things are today in a totally
non-PvD-aware system, which has the kinds of problems we're aiming to
solve, yes?

-ek

---

From: Stjepan Groš <[email-masked]>
Date: Mon, 2 Nov 2015 14:13:21 +0100

On 02.11.2015 13:18, Erik Kline wrote:
>
> On 2 November 2015 at 18:44, Stjepan Groš <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     Hi Erik,
>
>     Thank you for the draft.
>
>     application should have access to a single PvD at any single
>     moment? If that is so, what about mpTCP and SCTP applications that
>     are PvD aware? They should have access to multiple PvDs
>     simultaneously for a fail over reasons. So, how this draft takes
>     into account that use case?
>
>
> Not quite: they should be able to specify only one PvD per function
> call, per thread (for example).  An app can have multiple sockets,
> each one using one PvD.  An app can have something like a single
> unconnected datagram socket and call sendto() with a different PvD for
> each sendto() call.  That kind of thing is more what I meant to write,
> but I guess was not very clear.  (Clearly, I have a bunch of sockets-y
> things in my head but am trying to write things in a more general,
> non-sockets-dependent way.)
>
> The confusion may be arising in part from the resolution of which PvD
> to use for a given function call if no PvD is specified.  In those
> cases, the system checks for "defaults" at increasingly general
> levels.  So a process could set a process default PvD and avoid having
> to specify the PvD for each an every call, but still make individual
> socket calls or query DNS on a different PvD whenever it wants to.
>

I was thinking on functions like sctp_bindx that explicitly expect
multiple IP addresses to be present at a point when the function is
called. mpTCP could be even bigger problem because, as I understand it,
Linux kernel autonomously decides which local IP addresses to use.

>
> There are necessarily going to be sequences of function calls after
> which trying to change a PvD will be non-sensical.  For example, after
> I have a connected TCP socket: a source address has been chosen from a
> PvD and the PvD's routing table is in use for every write()/send() I
> call.  Attempting to change which PvD is use is a bit non-sensical at
> that point, and I think perhaps that an API should not really allow it.
>
>     In section 3.4 there is a note/question about "loopback PvD". What
>     we were thinking/discussing is that there should be such a PvD for
>     backwards compatibility, i.e. for non-PvD aware applications that
>     for some reason expect all available connections and interfaces
>     (PvDs) to be present (non-PvD aware mpTCP and SCTP applications,
>     UDP applications, debugging applications like tcpdump/wireshark).
>     This configuration we called internally "root network name space".
>
>
> That sounds a bit different from what I was thinking of a loopback
> PvD.  What you describe sounds like how things are today in a totally

Probably I missed the point here, i.e. there is no point of having
something like "all addresses" when we are talking about PvD aware
applications that can find out all of them via PvD API. Yet, I believe
that something like "all addresses present" will have to exists on a
system due to the backwards compatibility with legacy applications which
will be with us for a long time. Now, I don't know if that is something
that should be covered by PvD API or not, probably depends on use-cases.

SG

P.S. When I'm discussing these issues I'm thinking about general Linux
Workstation, like Fedora, and in addition I'm also I'm in terms of
network name spaces as the way to implement PvD separation.

---

From: Erik Kline <[email-masked]>
Date: Wed, 4 Nov 2015 21:01:22 +0900

On 2 November 2015 at 22:13, Stjepan Groš <[email-masked]> wrote:

> multiple IP addresses to be present at a point when the function is called.
> mpTCP could be even bigger problem because, as I understand it, Linux
> kernel autonomously decides which local IP addresses to use.
>

Ah, thank you, that kind of call is an important consideration.  I believe
I have a solution that will work: when determining the intended PvD in the
case of existing APIs that specify source addresses (and/or maybe even
interfaces) we should use PvDs that are uniquely identified by this
information.

What to do if an address does not unique identify a PvD is another matter.
For both of these issues I'm hoping to have some discussion in Friday's mif
meeting.

> Workstation, like Fedora, and in addition I'm also I'm in terms of network
> name spaces as the way to implement PvD separation.
>
>
FWIW, for Android's mif implementation network namespaces were considered
and rejected.  Among other things, I don't think it's really possible to
write an application that can have multiple sockets with each one "bound"
to a different network namespace.

Instead, it's a bit more like implicit PvDs are used with one interface
belonging to one and only one PvD and each PvD gets its own routing table.
This model is, obviously, not as general as mif work is trying to address
(i.e. m:n interfaces to PvDs are not supported).  (but it has been
sufficient to gain some experience with how an API would look and behave)

---

From: Stjepan Groš <[email-masked]>
Date: Wed, 4 Nov 2015 16:44:44 +0100

On 04.11.2015 13:01, Erik Kline wrote:
>
>
> On 2 November 2015 at 22:13, Stjepan Groš <[email-masked]
>>         moment? If that is so, what about mpTCP and SCTP applications
>>         that are PvD aware? They should have access to multiple PvDs
>>         simultaneously for a fail over reasons. So, how this draft
>>         takes into account that use case?
>>
>>
>>     Not quite: they should be able to specify only one PvD per
>>     function call, per thread (for example).  An app can have
>>     multiple sockets, each one using one PvD.  An app can have
>>     something like a single unconnected datagram socket and call
>>     sendto() with a different PvD for each sendto() call.  That kind
>>     of thing is more what I meant to write, but I guess was not very
>>     clear.  (Clearly, I have a bunch of sockets-y things in my head
>>     but am trying to write things in a more general,
>>     The confusion may be arising in part from the resolution of which
>>     PvD to use for a given function call if no PvD is specified.  In
>>     those cases, the system checks for "defaults" at increasingly
>>     general levels.  So a process could set a process default PvD and
>>     avoid having to specify the PvD for each an every call, but still
>>     make individual socket calls or query DNS on a different PvD
>>     whenever it wants to.
>>
>     multiple IP addresses to be present at a point when the function
>     is called. mpTCP could be even bigger problem because, as I
>     understand it, Linux kernel autonomously decides which local IP
>     addresses to use.
>
>
> Ah, thank you, that kind of call is an important consideration.  I
> believe I have a solution that will work: when determining the
> intended PvD in the case of existing APIs that specify source
> addresses (and/or maybe even interfaces) we should use PvDs that are
> uniquely identified by this information.

You mean like having some separate function which is called before
sctp_bindx with the same arguments so that appropriate PvDs can be
activated?

We were thinking about some kind of a PvD activation function (e.g.
pvd_attach_pvd(pvdid *id) and possibly also pvd_detach_pvd) so that
application can attach any combination of PvDs it wants to have access
to. This, at least to me, seems to be the most flexible solution?

>
> What to do if an address does not unique identify a PvD is another
> matter.  For both of these issues I'm hoping to have some discussion
> in Friday's mif meeting.
>  
>
>     P.S. When I'm discussing these issues I'm thinking about general
>     Linux Workstation, like Fedora, and in addition I'm also I'm in
>     terms of network name spaces as the way to implement PvD separation.
>
>
> FWIW, for Android's mif implementation network namespaces were
> considered and rejected.  Among other things, I don't think it's
> really possible to write an application that can have multiple sockets
> with each one "bound" to a different network namespace.

That's interesting. Namely, I wrote a test application that does the
following:

1. creates a socket (just calls socket function) in the default namespace
2. then, it switches to some other namespace using setns. This new
namespace doesn't have any connectivity.
3. test application connects to a server and issues HTTP GET request
4. writes response to the stdout

In short, this worked which made us conclude that the socket is bound to
whatever namespace was active at the time socket was created.
Application can switch to other namespaces after that at will, even
within a single thread. This means that you can use multiple namespaces
from within the single application at will. You have to only take care
to activate namespace you need before issuing socket call.

Did you try to do something like that?

The issue we have though, with respect to namespaces, is that we might
end up in a situation in which two namespaces share the same IP address.
It is almost equivalent to having two machines on the network with the
same IP address. We have some half-baked ideas for solutions for this
issue, and nothing that works.

>
> Instead, it's a bit more like implicit PvDs are used with one
> interface belonging to one and only one PvD and each PvD gets its own
> routing table.  This model is, obviously, not as general as mif work
> is trying to address (i.e. m:n interfaces to PvDs are not supported).
>  (but it has been sufficient to gain some experience with how an API
> would look and behave)

This probably could work for Android, but this brings up two points when
we talk about some general Linux distribution:

1. Is it really necessary to support multiple interfaces with a single
PvD instance? And by single PvD instance I mean PvD with the same
parameters, i.e. same address on multiple interfaces. In that case it
seems to me that bridging is a better approach.

2. How do you intend to control which process uses which interface/PvD?

SG