Subject: RE: [rddp] STag validation semantics

From: "Mukund, Shridhar" <[email-masked]>
Date: Tue, 8 Oct 2002 18:48:53 -0700

Caitlin,

[ response to Caitlin's mail dtd Monday, September 30 ]

I am a newB to IB. However, I do not think that the Protection 
Domain model from Infiniband addresses the security issue 
surrounding STag scope here(I am refering to the DDP context)

PD is only *one of the* elements that along with other
protection elements addresses the said security issue in
the context of IB.

Other elements include:
  1. Memory Windows - allows the application to dynamically
     grant access to different remote agents and limit
     that access to only the memory needed by those remote
     agents : Different access control domains within the same PD.
  2. P_Key - Partitions allow setting up of access bounds.
  3. Q_Key - identifies remote QP; prevents improper access.
  4. M_Keys, B_Keys, CFM_Keys, RDD ... ? 

There are two distinct types of R-Keys. One associated with 
memory regions obtained through the expensive process of
memory registration, typically obtained once on start-up by
the application: Region-R-Key. The other type that *IB recommends*
for remote access is associated with Memory Windows and is 
obtained through a simple process of memory binding: Window-R-Key.
Memory Window inherits the PD of it's parent Memory Region
but *has it's own access control*.

DDP STag needs to represent both types of R-Keys. Each R-Key,
(hence STag) may represent an independent access domain(say AD). 

Before data placament, merely checking if the DDP stream and STag 
belong to the same PD is not good enough. Multiple ADs(R-Keys) may
be associated with every DDP stream. In order to close the security 
gap, we need to *explicitly allow* STags on DDP Streams. 

I am not sure, but DAFS may want to explicitly allow the 
same STag on two DDP streams. There may however be an
efficiency issue with explicitly allowing an STag on
all DDP streams, but then ...

-Shridhar Mukund

> -----Original Message-----
> From: Caitlin Bestler [mailto:[email-masked]]
> Sent: Monday, September 30, 2002 6:06 PM
> To: [email-masked]
> Subject: Re: [rddp] STag validation semantics 
> 
> 
> On 9/30/02, Mukund, Shridhar wrote:
> 
> >
> >Caitlin,
> >
> >Isn't it true that a connection can be associated with 
> >multiple protection domains at the same time? 
> >
> >Are you assuming otherwise? Why?
> >
> 
> 
> The Protection Domain model I was referring to comes from
> InfiniBand. Each QP(connection) is associated with exactly
> one Protection Domain. Each R-Key(STag) is also associated
> with exactly one Protection Domain. An R-Key with the 'wrong
> Protection Domain' for a QP is rejected.
> 
> This requires far less NIC-accessible table space, and fewer
> cycles, to validate than a generalized validating lookup on
> the QP/R-Key combination.
> 
> 
> Caitlin Bestler
> http://asomi.com/CaitlinBestler/
> _______________________________________________
> rddp mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/rddp
> 
_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: Caitlin Bestler <[email-masked]>
Date: Wed, 9 Oct 2002 22:40:48 -0500

>   4. M_Keys, B_Keys, CFM_Keys, RDD ... ? 

These are all relevant, and likely to have their equivalents on
any reliable transport that DDP/RDMAP is caried over. The goal
is to define the absolute *minimum* that must be guaranteed
to ensure interoperability.

Hence *most* STags will indeed be something *very* equivalent
to InfiniBand R-Keys. However they could be as minimal as a token
that invoked a bounds check on the virtual address, with no
further mapping of the "virtual" address implied.

A buffer advertisement consists of an STag, base address and size.
Other than the fact that this buffer actually has 'size' distinct bytes
and can be accessed using the STag / address combinations that fall
within it, there is nothing that the remote peer needs or is entitled
to know.

What is proposed is that additionally, we know that through some 
otherwise unspecified local interface that the ULP can control
exactly what local memory is to be exposed by this STag, and it
may control what set of streams it can be used upon.

Howver, it is not *any* set of streams. It is a set of streams
that belong to a ULP session, and hence assigned a common Protection
Domain. Typically, each Protection Domain would represent one
local OS 'process', but it could be more tightly drawn, such
as one user session being handled by that process or even a
single data stream. The scope is under ULP control.

P_Keys and M_Keys, or their equivalents, are part of configuring RNICs.
If they exist for this transport they are pure LLP issues. Hence, if
there is VLAN enforcement, that is handled by the LLP and/or L2 layers.
It is not visible to the RDMA/DDP layers. The ULP may choose to be
aware of it should it so choose. Generally the point of VLAN / P-Key
and similar schemes is for the applications *not* to be aware of it,
however. The goal is to create a virtual interface with a more limited
scope.

> but *has it's own access control*.

Those are two methods of providing meaning to an STag. It would
not surprise me if this exact interface is selected by many
implementations. But those are local interface details that do
not have to be specified to ensure interoperability.

Application portability is a valid concern, but I do not believe
it is a goal of this working group. There are also excellent
solutions for RDMA applications to have portable APIs via the
Direct Access Transport Collaboratie (http://www.datcollaborative.org)
and the Open Group. Those groups are attempting to define a common
user level API. DAT already has APIs published. Nothing proposed
in the current drafts is incompatible with those APIs, although
automatic tag invalidation would be a new feature not yet
supported in the APIs.

> gap, we need to *explicitly allow* STags on DDP Streams. 
The data structures to support explicit STag/stream pairing would be
far more extensive that comparing each to a Protection Domain.

These data structures must be accessible to the RNIC, or more likely
stored upon it. Making large amounts of data accessible to the RNIC
for calculation at wire speed is not a trivial matter. That is why
anyone designing an RNIC will prefer the Protection Domain option
to an any-to-any mapping.

Further the Protection Domain solution allows a group of streams to
representing a single session to share their STags. But it does not
require any wider exposure.

The wider any-to-any mapping makes for simpler language, but more
complex implementations. I cannot think of any ULP scenario that
would truly require this capability.

Allowing the same STags to be used over multiple streams in the
same ULP sessin, by contrast, is clearly desirable. It avoids
complex error-prone coding at the application layer.

It also minimizes the risk that redundant mapping resources will
be declared to support the same window for each stream (although
that *could* be solved seperately).

> all DDP streams, but then ...
I could see including language that placing all STags in
the same Protection Domain by default SHOULD NOT be the 
default.

However, there are limited resource embedded environments
where such an engineering tradeoff would be valid. The
farthest we should go is mandating that the ULP MUST have
the ability to create multiple Protection Domains. How
many is "enough" is a question best left to the market.
_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp