Subject: Re: [rddp] STag validation semantics

From: "Renato Recio" <[email-masked]>
Date: Tue, 1 Oct 2002 09:12:59 -0500

I agree with Caitlin and wanted to add another point.  In IB,  our view was
the ULP could associate the Protection Domain with a process.  For example,
the ULP could make the process id = the protection domain.  If the ULP does
that, then one process cannot access the resources owned by another
process,  because they are protected by the PD.

Thanks

Renato J Recio
Distinguished Engineer
IBM I/O Architecture and Technology
Tel 512-838-3685, T/L 678-3685

                      Caitlin Bestler                                                                                   
                      <[email-masked]        To:       [email-masked]                                                  
                      i.net>                   cc:                                                                      
                      Sent by:                 Subject:  Re: [rddp] STag validation semantics                           
                      [email-masked]                                                                                 
                      rg                                                                                                

                      09/30/2002 08:05                                                                                  
                      PM                                                                                                

On 9/30/02, Mukund, Shridhar wrote:

>
>Caitlin,
>
>Isn't it true that a connection can be associated with
>multiple protection domains at the same time?
>
>Are you assuming otherwise? Why?
>

The Protection Domain model I was referring to comes from
InfiniBand. Each QP(connection) is associated with exactly
one Protection Domain. Each R-Key(STag) is also associated
with exactly one Protection Domain. An R-Key with the 'wrong
Protection Domain' for a QP is rejected.

This requires far less NIC-accessible table space, and fewer
cycles, to validate than a generalized validating lookup on
the QP/R-Key combination.

Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: "Renato Recio" <[email-masked]>
Date: Tue, 1 Oct 2002 09:31:56 -0500

In a previous e-mail I stated:   "the ULP could associate the Protection
Domain with a process.  For example, the ULP could make the process id =
the protection domain.  If the ULP does that, then one process cannot
access the resources owned by another process,  because they are protected
by the PD".

The ULP can also share a single Protection Domain across multiple
processes.  However, such sharing brings with it all the semantics of
sharing a single physical memory buffer between processes.  For example,
any process that is sharing the PD has access to the memory buffers that
are referenced by an STag that is associated with the shared PD.

In my view,  this sort of discussion doesn't belong in the RDMA Protocol
Specification.  It belongs in a verbs specification or something similar to
that.

However,  the RDMAP spec could state the following to clarify some of these
semantics:

 - A Protection Domain (PD) MUST be used to associate a RDMA Stream to an
STag.  Each RDMA Stream MUST be associated with one, and only one,
Protection Domain.  Each STag MUST be associated with one, and only one,
Protection Domain.  However, the same PD can be used on multiple RDMA
Streams and on multiple STags.

- If the PD associated with an STag does not match the PD associated with
the RDMA Stream, RDMAP MUST terminate the RDMAP Stream using the
termination mechanism in section 6.8.

Though I think the current RDMAP specification text is sufficient,  if
folks think the above would help,  it could be added to the RDMAP
specification.

Thanks

Renato J Recio
Distinguished Engineer

                      Bill Sommerfeld                                                                                           
                      <[email-masked]        To:       "Mukund, Shridhar" <[email-masked]>               
                      gton.ma.us>                      cc:       [email-masked]                                                  
                      Sent by:                         Subject:  Re: FW: [rddp] STag validation semantics                       
                      [email-masked]                                                                                       

                      09/30/2002 07:52 PM                                                                                       
                      Please respond to                                                                                         
                      sommerfeld                                                                                                

> multiple protection domains at the same time?

Certainly unix allows multiple processes to have an open descriptor to
the same connection.

On the other hand, an rddp-enabled connection should not allow a
communicating peer to overwrite arbitrary data within a program's
address space.

Perhaps the right model here is akin to shared memory segments -- one
segment per connection.  If connections are shared between processes,
the segment could be, too..

                                                                         -
Bill

---

From: "Renato Recio" <[email-masked]>
Date: Tue, 1 Oct 2002 11:29:56 -0500

No additional comments from me on this one.  I agree with Caitlin.

Renato J Recio
Distinguished Engineer

----- Forwarded by Renato Recio/Austin/IBM on 10/01/2002 11:28 AM -----
                                                                                                                        
                      Caitlin Bestler                                                                                   
                      <[email-masked]        To:       Julian Satran/Haifa/IBM@IBMIL, [email-masked]                   
                      i.net>                   cc:                                                                      
                      Sent by:                 Subject:  RE: [rddp] STag validation semantics                           
                      [email-masked]                                                                                 
                      rg                                                                                                

                      10/01/2002 11:09                                                                                  
                      AM                                                                                                

On 10/1/02, Julian Satran wrote:

>The whole discussion is about scope not only exposure. The
>iSCSI ITT is limited to a single command. Having an STag
>span connections will be hard to accept because the
>security enclosure has to be as wide as the scope. One way
>to make it acceptable would be ask for a coordinated
>security/STag enclosure.
>
>Julo
>
The intent is to allow an STag to have the scope of a ULP
session. A ULP session would be defined as a set of one or
more DDP/RDMAP Streams that are meaningfully connected by
the ULP (i.e., it is not defined).

An example of a ULP Stream would be the set of
communications channels in a DAFS Session.

It is extremely desirable to allow an STag to be used on any
stream *within a session* without having to remember to use
a different value for each Stream.

With InfiniBand, that goal is met by allowing multiple
connections to be placed in a single Protection Domain.
R-Keys (IB's STag) are validated within a Protection Domain.

I believe that a Protection Domain concept: increases
compatability with IB, provides a meaningful scope /
security enclosure,  and can be implemented easily.

Caitlin Bestler

---

From: Al Hartmann <[email-masked]>
Date: Wed, 9 Oct 2002 08:14:43 -0500

Julian,  Continuing this textbook discussion, security is based upon a trust
model, and trust relationships are determined at the ULP and/or application
levels.  So whatever "security envelope" mechanisms are potentially provided
by RDMAP would be controlled by ULP and/or application security policy.  Is
that accurate?

Alfred Hartmann, Ph.D. <mailto:[email-masked]> 
VIEO, Inc. <http://www.vieo.com/>  
12401 Research Blvd.
Building 1, Suite 350
Austin, TX 78759
+1.512.257.3031, x267
VIEO:  The Adaptive Application Infrastructure Management (AAIM) Company

>  -----Original Message-----
>  From: Julian Satran [mailto:[email-masked]]
>  Sent: Tuesday, October 08, 2002 6:35 PM
>  To: Renato Recio
>  Cc: [email-masked]; [email-masked]
>  Subject: Re: Fw: [rddp] STag validation semantics
>  
>  
>  
>  Renato,
>  
>  I am sorry but I have to reiterate - protection is not security.
>  The PD mechanism you describe is a good approximation of the 
>  PD model found
>  in any undergraduate OS textbook.
>  
>  Applied to communicating parties it needs a "security 
>  envelope" insuring
>  that connections are protected in similar ways
>  and whatever has to be communicated about PDs is done 
>  properly (there must
>  be some knowledge about PDs that traverses the network).
>  
>  I am not claiming that this can't be done - only that none 
>  of the current
>  texts talks about it.
>  
>  Julo
>  
>  
>                                                               
>                                                               
>                         
>                        Renato                                 
>                                                               
>                         
>                        Recio/Austin/IBM@        To:       
>  [email-masked]                                                
>                             
>                        IBMUS                    cc:           
>                                                               
>                         
>                        Sent by:                 Subject:  Re: 
>  Fw: [rddp] STag validation semantics                         
>                         
>                        [email-masked]                      
>                                                               
>                         
>                        rg                                     
>                                                               
>                         
>                                                               
>                                                               
>                         
>                                                               
>                                                               
>                         
>                        04/10/02 19:40                         
>                                                               
>                         
>                                                               
>                                                               
>                         
>  
>  
>  
>  
>  Julian,
>  
>  Regarding protection:
>  
>     Protecting an STag from being used on a connection which 
>  is not allowed
>  to use it is a problem that was solved even before IB.  The solution
>  consists of providing a *protection* mechanism for 
>  associating an STag with
>  a connection.  Lets call that mechanism a Protection Domain 
>  (PD).  The
>  semantics for the PD are:
>  
>  Under this approach the RNIC (RDMA enable NIC) resource used 
>  to perform the
>  memory protection and translation process, based on the 
>  incoming STag and
>  TO,  only has to create one resource for *all* RDMA Streams 
>  that fall under
>  the same PD.  A ULP can then expose different portions of a 
>  single buffer
>  (that is referenced by a single STag) on multiple RDMA 
>  Streams and only the
>  RDMA Streams which share the same PD will have access to the 
>  buffer.  RDMA
>  Streams that do not share the same PD cannot access the buffer.
>  
>  An inferior approach would be to have:  An STag *MUST* only 
>  be valid on
>  one, and only one, RDMA Stream.  Why is this approach 
>  inferior?  Because it
>  either requires:
>  1.  The RNIC to have multiple memory protection and 
>  translation entries for
>  each buffer which a ULP wants to share across multiple RDMA Streams.
>  2.  The RNIC entry used to hold the memory protection and 
>  translation for
>  the buffer would need one additional chunk of state 
>  ("connection salt" as
>  you called it) for each RDMA Stream that the ULP wants to 
>  access the shared
>  buffer.
>  
>  So in my opinion,  the PD approach I described above is 
>  "sweeter" :{).
>  
>  BTW,  under the above approach,  a ULP *could* choose to 
>  make the STag
>  associated with a single RDMA Stream.
>  
>  BTW,  You may ask,  what ULPs would want to share buffers?  
>  Here is a list
>  of some of the top:
>  - Web servers that send the same content to many, many clients.
>  - File servers that provide many clients with read access to the same
>  files.
>  - Databases in a cluster or multi-tier server environment.
>  
>  
>  Regarding security:
>  
>  Obviously if a rogue endpoint can inject RDMA/DDP/Transport 
>  Segments into
>  the fabric,  all kinds of damage would result.  I believe 
>  iSCSI faces the
>  same problem.
>  
>  Thanks,
>  
>  Renato J Recio
>  Distinguished Engineer
>                        "Julian Satran"
>                        <Julian_Satran@il        To:       
>                        .ibm.com>                cc:
>                        Sent by:                 Subject:  Fw: 
>  [rddp] STag
>  validation semantics
>                        10/04/2002 09:52
>  Renato,
>  Perhaps the context is not right. But the IB design point is 
>  different from
>  IP.
>  And security is not protection. The type of attacks that have to be
>  considered in an IP network
>  are beyond protection.  The whole STag concept is weak if you have to
>  secure it and have no way to enforce it.
>  
>  Perhaps the session is the answer but sharing the same type 
>  of security
>  over all connections is not painless (even to specify).
>  But perhaps we should consider all other alternatives.
>  
>  If the ULP maintains a connection affinity (as iSCSI does) 
>  then limiting
>  the scope of an STag to a connection is painless even in 
>  face of recovery.
>  If the ULP needs a wider scope - and the RDMAP has no 
>  effective means to
>  police that - then it may want to give up the invalidation 
>  shortcut - but I
>  am afraid that even this will be frowned upon in some quarters.
>  
>  But I have real trouble understanding where limiting STag to 
>  a connection
>  will cause pain. Many an implementer may find ways to 
>  express within the
>  STag a "constant part" (the core-STag) and a "connection 
>  salt" and avoid
>  the whole issue of securing a session.
>  
>  Regards,
>  Julo
>  
>  
>      Renato Recio/Austin/IBM@IBMUS
>      Sent by: [email-masked]           To:        [email-masked]
>                                             cc:
>                                             Subject:        
>  Re: [rddp] STag
>      01/10/02 16:12                 validation semantics
>  
>  
>  
>  
>  
>  
>  
>  
>                       Sent by:                 Subject:  Re: 
>  [rddp] STag
>  validation semantics

---

From: "Culley, Paul" <[email-masked]>
Date: Fri, 18 Oct 2002 09:00:36 -0500

Please forgive Renato (and the other authors) for continued use of RDMA when discussing DDP STags.  Most of us don't really believe that anyone will use DDP without RDMA and are concentrating our efforts on getting the combination of DDP/RDMA and its upper interface to have a useful set of semantics and protections.  And most initial implementations will treat DDP/RDMA as a single monolithic entity, making any interface definitions between them unnecessary.  So, for these implementations, the ULP-STags ARE THE SAME as DDP-STags!

The original RDMA/DDP split was done because a few folks seemed to strongly believe that RDMA (specifically RDMA Read) was not necessary.  To attempt some level of forward progress, we created the split hoping that we would get just such feedback as we are now getting.

We would be happy to see a discussion get started on what exact changes you believe are necessary to the DDP document to make it more acceptable.

Paul R. Culley (Sr. Fellow)
Hewlett-Packard Company
281-514-5543

-----Original Message-----
From: Mukund, Shridhar [mailto:[email-masked]]
Sent: Thursday, October 17, 2002 4:53 PM
To: 'Renato Recio'; [email-masked]
Subject: RE: Fw: [rddp] STag validation semantics

Renato,

Some thoughts on the DDP draft:

1. DDP (RDDP?) draft needs to stand on it's own without depending on 
   either RDMAP draft or the MPA draft. For instance a ULP like 
   iSCSI *may* want to ride on DDP without requiring RDMAP. For that 
   matter it should be valid to implement a DNIC(DDP NIC) instead of 
   a RNIC. I hope we agree there.

2. Please introduce the notion of DDP-STag vs. ULP-STag(RDMAP-STag). 
   This seems to be the source for much confusion. Further more, the 
   two layers DDP and RDMAC(or another ULP of DDP) associate different 
   meanings to them.

   * DDP-STag = RDMAP-STag is just an artifact of RDMAP implementation.
     Only when the ULP consumer wants to sink remote data, it needs to
     bring DDP-STag into existence. 
     For iSCSI over DDP, may be DDP-STag = ITT or TTT.

   * It is the DDP-STag that needs to be scrutinized rigorously 
     against IP-related attacks.(Granted, ULP-STag will be subject to 
     other attacks outside the scope of IPSec)

   * You cited applications such as file/web servers that need to give 
     read access to the world. Those ULP-STags will not map down to 
     DDP-STags on the wire! ULP may define broader scope for ULP-STag
     and find ways to enforce it.

   * As Caitlin/Julian alluded elsewhere, some may find innovative ways 
     of using DDP-STags including DDP-STag = foo(ULP-STag,...)to provide 
     useful functions/features. ... sure, let's keep things simple 
     for RDMAP other than recognizing the difference for documentation.

3. Since buffers(security critical resources) are modified by the DDP
   layer, the security envelope should be stretched into the DDP layer.
   (Granted, the ULP creates & controls that envelope). 
   * Most verbiage in your email refers to RDMAP instead of DDP.
   * Postponing security checks to RDMAP/ULP layer while the 
     underlying layer(DDP) modifies secure resources does not sound 
     like a good design choice.( I believe it is not intended )

4. If IPSec is used for security against malicious data placement 
   then all connections within a PD MUST be secured at the same level 
   of IPSec *connection* security( SA relationships and related language
   as the security envelope moves further from TCP connection ... ). 
   - But for the above, we give a FALSE sense of security to remote
     clients who choose to protect their connection using IPSec 
     but are still exposed to attacks from packet injection on other 
     insecure connections within the PD.  
   
   Alternatively, we could state that IPSec CANNOT be used to secure 
   against malicious data placement. I think that may not fly. Note 
   that the last sentence in section 10.2 of DDP draft actually implies 
   this.

Thanks.

-Shridhar Mukund

> From: Renato Recio [mailto:[email-masked]]
> Sent: Friday, October 04, 2002 9:40 AM
> To: [email-masked]
> a connection.  Lets call that mechanism a Protection Domain (PD).  The
> So in my opinion,  the PD approach I described above is "sweeter" :{).
> 
> BTW,  under the above approach,  a ULP *could* choose to make the STag
> [rddp] STag validation semantics                           
>                       [email-masked]                       
>                                                           
>                       rg                                      
>                                                           
>                                                               
>                                                           
>                                                               
>                                                           
>                       10/04/2002 09:52                        
>                                                           
>                       AM                                      
>                                                           
>                                                               
>                                                           
>                                                               
>                                                           
> 
> 
> 
> 
> 
> Renato,
> 
> the scope of an STag to a connection is painless even in face 
> of recovery.
> will cause pain. Many an implementer may find ways to express 
> within the

---

From: "Mukund, Shridhar" <[email-masked]>
Date: Fri, 18 Oct 2002 11:33:20 -0700

Caitlin,

> >3. Since buffers(security critical resources) are modified
> > by the DDP layer, the security envelope should be
> > stretched into the DDP layer. (Granted, the ULP creates &
> > controls that envelope). 
> > underlying layer(DDP) modifies secure resources does not
> > sound like a good design choice.( I believe it is not
> > intended )
> >
> 
> What sort of security are you looking for? A one-time key
> to enable each transfer?
> 
> 

By security check I am merely refering to the act of 
associating STag with a connection (directly or thru' PD). 
Thanks to Paul for clarifying that the intent is to do this 
in the DDP layer and not defer it to the RDMAP/ULP layer.

-Shridhar Mukund