Subject: RE: [rddp] verbs draft - STag validation error in 7.10.6.2

From: "Deleganes, Ellen M" <[email-masked]>
Date: Mon, 7 Jul 2003 07:29:59 -0700

Making Memory Windows only valid on a single stream in the verbs was a
deliberate choice. In most programming models it makes sense to bind a
window to a stream rather than to the PD. If the application writer
wants a memory entity that can be remotely accessible by multiple
streams, they can still use Memory Regions. This decision actually makes
Memory Windows more lightweight. It is a lot quicker to do the
invalidation because you only need to check incoming traffic on a single
stream, instead of across all of the streams in a Protection Domain,
which can be rather onerous.

Ellen

-----Original Message-----
From: Caitlin Bestler [mailto:[email-masked]] 
Sent: Thursday, July 03, 2003 8:40 PM
To: [email-masked]
Subject: [rddp] verbs draft - STag validation error in 7.10.6.2

Section 7.10.6.2 of the verbs draft has a mistake. In specifying the
validation that must be done for received STags it states:

>   *   The RNIC MUST check and enforce that the QP ID associated with 
>       the Memory Window is equal to the QP ID associated with the 
>       incoming remote operation that is accessing the Memory Window.

If a Memory Window has "a" QP ID associated with it then it can never
be valid on multiple QPs (DDP Streams).

But the protocol specifically states:

>   Protection Domain (PD) - A Mechanism used to associate a DDP Stream 
>       and an STag. Under this mechanism, the use of an STag is valid 
>       on a DDP Stream if the STag has the same Protection Domain 
>       Identifier (PD ID) as the DDP Stream.

An STag with the "wrong" QP ID nevertheless has the correct PD ID.
Therefore the STag is valid.

Caitlin Bestler - [email-masked] - http://asomi.com/

_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: Caitlin Bestler <[email-masked]>
Date: Mon, 7 Jul 2003 11:45:12 -0500

On Monday, July 7, 2003, [email-masked] wrote:

>deliberate choice.

But it was not the deliberate choice of the WG.

This issue was explicitly discussed. The WG rejected both limiting
STags to a single stream and allowing "any to any" matching of
STags and streams.

Instead, ULPs that wanted to have an STag be valid across multiple
streams had to place those streams in a single Protection Domain
so that the stack could perform a simple comparision rather than
being required to do a lookup.

>In most programming models it makes sense to bind a
>streams, they can still use Memory Regions.

With the verbs as proposed, they can only do so by losing the
ability to dynamically define the boundaries of that region.

Basically, you leave a very wide region open to nearly every
stream that the server uses, or you gain byte-boundary control
over a single stream. There is a very wide gap between those two.

>This decision actually makes Memory Windows more lightweight.

Only if you confuse Protection Domains (as defined in the protocol)
with the local memory protectio mechanisms.

I would submit that all of the following are "lightweight"

-- A Consumer creating a "Protection Domain" identifier.
-- A Consumer assigning that identifier to Memory Windows
-- A Consumer assigning that identifier to a DDP Stream.
-- An incoming STag is validated by comparing PD IDs.

Such an interface is fully compatible with the DDP draft,
easily implemented, has fewer security implications and
provides the ULP with complete flexibility in creating
multi-stream Sessions.

>It is a lot quicker to do the invalidation because you only
>need to check incoming traffic on a single stream, instead of
>across all of the streams in a Protection Domain, which can be
>rather onerous.
>
Which was specifically discussed on the reflector when Protection
Domains were proposed and accepted. Specifically, that an invalidate
would invalite the STag for the entire Protection Domain.

Keep in mind that multi-stream STags are created *by the ULP*
to server *ULP* purposes. So just as with single-stream STags,
we can presume that if the ULP says to invalidate an STag that
it is doing so in compliance with ULP rules.

In other words, the ULP should not create a multi-stream session
unless it has reason to trust all participants.

Even with those restrictions, I can still see many shared-data
applications where communal STags within a cluster could be
of considerable value. This is particularly true for MPI apps.