Subject: Re: Fw: [rddp] STag validation semantics

From: "Renato Recio" <[email-masked]>
Date: Fri, 4 Oct 2002 11:40:05 -0500

Julian,

Regarding protection:

   Protecting an STag from being used on a connection which is not allowed
to use it is a problem that was solved even before IB.  The solution
consists of providing a *protection* mechanism for associating an STag with
a connection.  Lets call that mechanism a Protection Domain (PD).  The
semantics for the PD are:

 - A Protection Domain (PD) MUST be used to associate a RDMA Stream to an
STag.  Each RDMA Stream MUST be associated with one, and only one,
Protection Domain.  Each STag MUST be associated with one, and only one,
Protection Domain.  However, the same PD can be used on multiple RDMA
Streams and on multiple STags.

- If the PD associated with an STag does not match the PD associated with
the RDMA Stream, RDMAP MUST terminate the RDMAP Stream using the
termination mechanism in section 6.8.

Under this approach the RNIC (RDMA enable NIC) resource used to perform the
memory protection and translation process, based on the incoming STag and
TO,  only has to create one resource for *all* RDMA Streams that fall under
the same PD.  A ULP can then expose different portions of a single buffer
(that is referenced by a single STag) on multiple RDMA Streams and only the
RDMA Streams which share the same PD will have access to the buffer.  RDMA
Streams that do not share the same PD cannot access the buffer.

An inferior approach would be to have:  An STag *MUST* only be valid on
one, and only one, RDMA Stream.  Why is this approach inferior?  Because it
either requires:
1.  The RNIC to have multiple memory protection and translation entries for
each buffer which a ULP wants to share across multiple RDMA Streams.
2.  The RNIC entry used to hold the memory protection and translation for
the buffer would need one additional chunk of state ("connection salt" as
you called it) for each RDMA Stream that the ULP wants to access the shared
buffer.

So in my opinion,  the PD approach I described above is "sweeter" :{).

BTW,  under the above approach,  a ULP *could* choose to make the STag
associated with a single RDMA Stream.

BTW,  You may ask,  what ULPs would want to share buffers?  Here is a list
of some of the top:
- Web servers that send the same content to many, many clients.
- File servers that provide many clients with read access to the same
files.
- Databases in a cluster or multi-tier server environment.

Regarding security:

Obviously if a rogue endpoint can inject RDMA/DDP/Transport Segments into
the fabric,  all kinds of damage would result.  I believe iSCSI faces the
same problem.

Thanks,

Renato J Recio
Distinguished Engineer
IBM I/O Architecture and Technology
Tel 512-838-3685, T/L 678-3685

                      "Julian Satran"                                                                                   
                      <Julian_Satran@il        To:       [email-masked]                                                  
                      .ibm.com>                cc:                                                                      
                      Sent by:                 Subject:  Fw: [rddp] STag validation semantics                           
                      [email-masked]                                                                                 
                      rg                                                                                                

                      10/04/2002 09:52                                                                                  
                      AM                                                                                                

Renato,

Perhaps the context is not right. But the IB design point is different from
IP.
And security is not protection. The type of attacks that have to be
considered in an IP network
are beyond protection.  The whole STag concept is weak if you have to
secure it and have no way to enforce it.

Perhaps the session is the answer but sharing the same type of security
over all connections is not painless (even to specify).
But perhaps we should consider all other alternatives.

If the ULP maintains a connection affinity (as iSCSI does) then limiting
the scope of an STag to a connection is painless even in face of recovery.
If the ULP needs a wider scope - and the RDMAP has no effective means to
police that - then it may want to give up the invalidation shortcut - but I
am afraid that even this will be frowned upon in some quarters.

But I have real trouble understanding where limiting STag to a connection
will cause pain. Many an implementer may find ways to express within the
STag a "constant part" (the core-STag) and a "connection salt" and avoid
the whole issue of securing a session.

Regards,
Julo

    Renato Recio/Austin/IBM@IBMUS                                          
    Sent by: [email-masked]           To:        [email-masked]        
                                           cc:                             
                                           Subject:        Re: [rddp] STag 
    01/10/02 16:12                 validation semantics                    

I agree with Caitlin and wanted to add another point.  In IB,  our view was
the ULP could associate the Protection Domain with a process.  For example,
the ULP could make the process id = the protection domain.  If the ULP does
that, then one process cannot access the resources owned by another
process,  because they are protected by the PD.

Thanks

Renato J Recio
Distinguished Engineer
IBM I/O Architecture and Technology
Tel 512-838-3685, T/L 678-3685

                     Caitlin Bestler

                     <[email-masked]        To:       [email-masked]

                     i.net>                   cc:

                     Sent by:                 Subject:  Re: [rddp] STag
validation semantics
                     [email-masked]

                     rg

                     09/30/2002 08:05

                     PM

On 9/30/02, Mukund, Shridhar wrote:

>
>Caitlin,
>
>Isn't it true that a connection can be associated with
>multiple protection domains at the same time?
>
>Are you assuming otherwise? Why?
>

The Protection Domain model I was referring to comes from
InfiniBand. Each QP(connection) is associated with exactly
one Protection Domain. Each R-Key(STag) is also associated
with exactly one Protection Domain. An R-Key with the 'wrong
Protection Domain' for a QP is rejected.

This requires far less NIC-accessible table space, and fewer
cycles, to validate than a generalized validating lookup on
the QP/R-Key combination.

Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: "Julian Satran" <[email-masked]>
Date: Wed, 9 Oct 2002 01:34:56 +0200

Renato,

I am sorry but I have to reiterate - protection is not security.
The PD mechanism you describe is a good approximation of the PD model found
in any undergraduate OS textbook.

Applied to communicating parties it needs a "security envelope" insuring
that connections are protected in similar ways
and whatever has to be communicated about PDs is done properly (there must
be some knowledge about PDs that traverses the network).

I am not claiming that this can't be done - only that none of the current
texts talks about it.

Julo

                      Renato                                                                                                                     
                      Recio/Austin/IBM@        To:       [email-masked]                                                                           
                      IBMUS                    cc:                                                                                               
                      Sent by:                 Subject:  Re: Fw: [rddp] STag validation semantics                                                
                      [email-masked]                                                                                                          
                      rg                                                                                                                         

                      04/10/02 19:40                                                                                                             

Julian,

Regarding protection:

Regarding security:

Thanks,

Renato J Recio
Distinguished Engineer

                      "Julian Satran"

                      .ibm.com>                cc:

                      Sent by:                 Subject:  Fw: [rddp] STag
validation semantics
                      [email-masked]

                      rg

                      10/04/2002 09:52

                      AM

Renato,

Regards,
Julo

Thanks

Renato J Recio
Distinguished Engineer

                     Caitlin Bestler

                     i.net>                   cc:

                     rg

                     09/30/2002 08:05

                     PM

Caitlin Bestler

---

From: Caitlin Bestler <[email-masked]>
Date: Wed, 9 Oct 2002 22:39:50 -0500

> 
> Renato,
> 
> I am sorry but I have to reiterate - protection is not security. The
> PD mechanism you describe is a good approximation of the PD model
> found in any undergraduate OS textbook.
> 
> Applied to communicating parties it needs a "security envelope"
> insuring that connections are protected in similar ways and whatever
> has to be communicated about PDs is done properly (there must be some
> knowledge about PDs that traverses the network).
> 
> I am not claiming that this can't be done - only that none of the
> current texts talks about it.
> 

Is there anything other than documentation that you feel needs to be
added regarding Protection Domains? If so, I'm not sure what you
are requesting be added.

STags are delivered as part of ULP buffer advertisements. The scope
of their validity must be part of that advertising. Since buffer
advertisement is a ULP issue, there is no need to coummicate
anything about Protection Domains between ULP peers. The only
requirement is that the advertiser exercise its local interface
in a mannner that is consistent with what it advertised.

What the local interface must offer is the ability to create a 
set of connections (where we are calling that set a Protection
Domain), create and delete STags that are valid for that set
(Protection Domain), and to add or drop connections to the set.

To simplify implementation, it is further proposed that any STag
belongs to at most one Protection Domain, and that each connection
belong to exactly one Proection Domain.

The Protection Domain concept is a local interface concept which
guarantees the ability of a ULP to specifry multi-stream sessions
and allow simple session-wide buffer advertisements. As opposed
to simpler language of allowing STags to be made valid for "any"
set of streams, it can be implemented with very simple RNIC
data structures.

The current draft is, in my reading, ambiguous. It can be read
to be compatible with an STag that MUST have the scope of one
connection, with the Protection Domain approach, or that the
ULP MUST be able to associate any STag with any stream. Do
you have a different preference for resolving this other than
the Protection Domain model?