Subject: RE: [rddp] WG chair on buffer exhaustion (again)

From: "Kanevsky, Arkady" <[email-masked]>
Date: Thu, 31 Jul 2003 08:37:57 -0700

David,
I am trying to comprehand the 2 issues.

You stated that Mallikarjun's issue involves statistical (under-) 
provisioning of buffers can happen without any Shared
Receive queues.
1. What is the definition of reliable connection based on this change?
2. How does UPL knows/control what is the connection behavior:
	connection broken(termination DDP stream) if no recv buffer is
availble vs.
	Send message dropped on the floor and not delivered, but DDP
connection stays?
3. How is the Stag invalidation effected (if at all) with both proposals?

Thanks,
Arkady

Arkady Kanevsky                                email: [email-masked]
Network Appliance                              phone: 781-768-5395
375 Totten Pond Rd.                            Fax: 781-895-1195
3rd Floor                                              http: www.netapp.com
Waltham, MA 02451-2010                  general phone: 781-768-5300

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] 
> Sent: Wednesday, July 30, 2003 5:40 PM
> To: [email-masked]
> Subject: [rddp] WG chair on buffer exhaustion (again)
> 
> 
> With WG Chair hat on, let me step into this debate:
> 
> > > Section 3.2 to:
> > >     " Note, however, that the ULP needs to address flow control 
> > > issues.
> Any
> > >       ULP that runs on DDP SHOULD consider the lack of an 
> associated
> receive
> > >       buffer to place an inbound Untagged Message of the ULP as an
> exception."
> > 
> > If the local interface has allows the DDP endpoint to know 
> the valid 
> > range of MSNs on a stream (which it will for simple Receive 
> Queues at 
> > the minimum) then it definitely SHOULD NOT consider the lack of a 
> > buffer for an *invalid message* to be "an exception".
> 
> Two issues are getting conflated here, and need to be kept separate:
> - Mallikarjun's issue involves statistical (under-) 
> provisioning of buffers,
> 	so that an otherwise valid untagged message shows up 
> without a buffer
> 	available to receive it (if the implementation had done 
> worst-case
> 	provisioning, there would have been a buffer 
> available).  I believe
> 	there is WG rough consensus that this is an exceptional 
> condition
> 	that does not need to immediately cause the termination 
> of the DDP
> 	stream that sent the message, but it SHOULD not occur 
> frequently.
> 	Mallikarjun's proposed text is close enough to what's 
> needed for this.
> 	This condition could happen even on a simple Receive Queue if
> 	statistical (under-) provisioning is being used.
> - Caitlin's issue involves a peer violating a fixed 
> per-DDP-stream limit on
> 	the number of untagged messages allowed .  At the 
> moment, the exact
> 	need and nature of such limits is still an open issue 
> that needs to be
> 	resolved in the context of the security draft.  When we 
> have it resolved
> 	there, additional text will (probably) get driven into 
> the DDP draft.
> 	If the issue is resolved in the manner Caitlin advocates, then
> 	her proposed text is certainly a candidate for what the 
> DDP draft
> 	will need to say, but at the moment, the issue is open, and the
> 	DDP draft should not change what it says about this issue until
> 	it is resolved in the context of the security draft.
> 
> Also, I've seen some confusing references to iSER and 
> assumptions about local interface and resource sharing model. 
>  Shared Receive Queues have
> *not* been approved/adopted by this WG (and can't be until we 
> get the resource sharing issue [2nd bullet above] resolved).  
> To the extent that iSER requires Shared Receive Queues and 
> can't cope with any other local resource model, it's probably 
> made an overly restrictive design assumption that it's 
> designers would be well-advised to reconsider, keeping in 
> mind that the Hilland verbs draft is at best an *example* of 
> a local interface to RDDP; it should not be viewed as 
> normative functional requirements on that local interface.
> 
> With luck, that's useful clarification, but feel free to ask 
> if anything above doesn't make sense ...
> 
> Thanks,
> --David
> ----------------------------------------------------
> David L. Black, Senior Technologist
> EMC Corporation, 176 South St., Hopkinton, MA  01748
> +1 (508) 293-7953             FAX: +1 (508) 293-7786
> [email-masked]        Mobile: +1 (978) 394-7754
> ----------------------------------------------------
> 
> _______________________________________________
> rddp mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/rddp
> 

_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: Caitlin Bestler <[email-masked]>
Date: Thu, 31 Jul 2003 11:51:14 -0500

On Thursday, July 31, 2003, at 10:37 AM, Kanevsky, Arkady wrote:
>
> 2. How does ULP knows/control what is the connection behavior:
>    available vs. Send message dropped on the floor and not delivered,
>    but DDP connection stays?

The drop, if done, must be done transparently to the ULP. That means it 
is
being "dropped" temporarily, requesting that the LLP do something to 
delay
delivery. Dropping the packet at the LLP layer is one option that the
implementation might consider, but it is the *last* one that it should 
use
before simply terminating the connection/stream.

The primary difference of opinion is not on whether such efforts are
commendable or not, but when they should be applied. I believe that they
MUST NOT / SHOULD NOT be applied for invalid untagged messages, i.e. 
those
that would have been dropped by a conventional non-sharing Receive 
Queue.

The contrary opinion seems to favor capturing as many buffers as 
possible,
and leaving enforcement entirely to the ULP.

> 3. How is the Stag invalidation effected (if at all) with both 

STag invalidation occurs at the RDMAP layer, *after* delivery from DDP.
DDP delivery follows placement. So STag invalidation cannot occur unless
there is a buffer available. If the lack of a buffer is not corrected,
somehow, then the connection/stream will be terminated.

---

From: "Mallikarjun C." <[email-masked]>
Date: Thu, 31 Jul 2003 17:53:50 -0700

Arkady,

Please note the

"....MUST however ensure that the affected DDP Segment 
  is eventually recovered ....way that guarantees ULP-transparency"

language in my suggested text.  So the proposed text does not 
change anything wrt RDMAP and thus the automatic STag 
invalidation, and of course the definition of reliable connection
stays the same as it was because the delivery semantics are still
in-order.

Regards.
--
Mallikarjun

Mallikarjun Chadalapaka
Networked Storage Architecture
Network Storage Solutions
Hewlett-Packard MS 5668 
Roseville CA 95747
[email-masked]