Subject: RE: [rddp] WG chair on buffer exhaustion

From: [email-masked]
Date: Tue, 22 Jul 2003 20:52:34 -0400

Paul,

> Do I read into this that you believe that DDP implementations
> MUST implement a per stream hard limit on untagged messages 
> ("termination is a reasonable response")?

Pending further work on the security draft, I'm not certain
about either "MUST implement", or the precise definition of
"hard limit".  I would prefer to allow work on the security
draft to determine the requirements to protect shared resources
rather than preemptively decide exactly what is necessary as WG
chair now.
 
> If you do, the next question is how fast?  Is it ok to delay
> the stream termination under some conditions, until some 
> "extra" untagged messages have arrived and used buffers?  
> This might occur if the mechanism for detecting the hard 
> limit exception depended on counting completions in software, 
> for example, or if it depended on software to respond to an exception.

If this class of mechanism is needed, that sort of delay may
be ok.  The determining factors will be the level and nature
of protection of the shared untagged receive resources from
exhaustion that is necessary, and "necessary" will probably
be in the sense of "available to the ULP" (i.e., "MUST implement",
not "MUST use").

> As pointed out earlier in this thread, the mechanism that
> returns "credits" for the "hard limit" can become an 
> additional burden for implementations, depending on how that 
> is done.  A "loose" credit check and/or update can 
> significantly reduce that burden.  

Yes, and this was also clear from the discussion in Vienna.
 
> It is not clear how much guidance the DDP security specs need
> to provide on this issue, other than to say something to the 
> effect of: "Here is an issue, make sure that your 
> implementation, or upper interface and ULP can deal with it." 
>  Seems like getting too detailed may provide unnecessary 
> restrictions on implementations.

Assuming that we do have a requirement to protect shared receive
resources for untagged messages in this fashion, the suggested
text is a bit too loose - ULPs will need to rely on somewhat
tighter functional behavior promises than "make sure that your ...
can deal with it", as the resulting engineering for the worst
case will waste resources.

Thanks,
--David
 
> > -----Original Message-----
> > From: [email-masked] [mailto:[email-masked]]
> > Sent: Monday, July 21, 2003 2:03 PM
> > To: [email-masked]
> > Subject: [rddp] WG chair on buffer exhaustion
> > 
> > 
> > The heated debate on whether lack of availability of a buffer for an 
> > untagged message causes immediate termination of the DDP stream on 
> > which it arrived caused some additional email behind the scenes in 
> > addition to what's been on the list.
> > 
> > With my WG chair hat on, here is what I believe to be the correct 
> > resolution of this issue:
> > 
> > The request to do something other than immediately terminate a stream
> > when a message shows up that does not have a pre-queued untagged 
> > buffer to receive it does not require that DDP tell TCP not to ACK the 
> > message; an implementation could choose to do that based on 
> > inter-layer implementation optimizations, but had better do so very
rarely.
> > Alternatively, a DDP implementation could choose to have some 
> > "hidden" resources in its back pocket that might involve a 
> > less efficient receive path (e.g., receive the errant message 
> > to a hidden kernel buffer and then copy to an application 
> > buffer when it becomes available)
> > - again, this would need to be a rare occurrence for obvious 
> > reasons. This sort of approach is more robust than having TCP 
> > not ACK, as it avoids propagating a host problem into the 
> > network and puts the performance penalty for the 
> > implementation problem (should have had a buffer available, 
> > but didn't) squarely on the system that had the problem.
> > 
> > I would prefer to see the "hidden [kernel] buffer" approach 
> > described as an example rather than the "TCP doesn't ACK" approach. 
> > Among the other problems with "TCP doesn't ACK" is that it creates a 
> > nasty head-of-line blocking problem that affects all traffic on the 
> > connection, not just the untagged traffic that is facing the 
> > resource shortage.
> > 
> > OTOH, if the ULP has set a hard limit on the number of untagged 
> > messages that can come in from a specific stream, and that stream 
> > exceeds the limit, this is evidence that the counterparty isn't 
> > following the rules, and termination of the stream is a reasonable 
> > response.  In the particular scenario that John raises below, the 
> > hard limit would need to be set high enough to accommodate 
> > error/special/ emergency conditions, and statistical/shared buffer 
> > provisioning would probably be used to avoid having to provision 
> > buffers for simultaneous occurrence of error/special/emergency 
> > conditions on all inbound streams.
> > 
> > Thanks,
> > --David
> > ----------------------------------------------------
> > David L. Black, Senior Technologist
> > EMC Corporation, 176 South St., Hopkinton, MA  01748
> > +1 (508) 293-7953             FAX: +1 (508) 293-7786
> > [email-masked]        Mobile: +1 (978) 394-7754
> > ----------------------------------------------------
> > 
> > > >
> > > >On Wednesday, July 16, 2003, [email-masked] wrote:
> > > >
> > > >>Again, I think that Mallikarjun stated things correctly. There 
> > > >>are important applications which can only correctly predict the 
> > > >>number of buffers needed in non error/special/emergency 
> > > >>conditions.  When error/special/emergency conditions occur, the 
> > > >>number and urgency of the handling is problematical.
> > 
> > _______________________________________________
> > rddp mailing list
> > [email-masked]
> > https://www1.ietf.org/mailman/listinfo/rddp
> > 
> 

_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: Caitlin Bestler <[email-masked]>
Date: Tue, 22 Jul 2003 22:41:44 -0500

> be in the sense of "available to the ULP" (i.e., "MUST implement",

Which suggests that the obvious set of options to consider are:

- MUST support hard or soft limits. MAY support both.
- MUST support hard limits. MAY support soft.
- MUST support soft limits. MAY support hard.
- MUST support hard and soft limits (but not necessarily
   both limits on any single stream).

My initial hunch is to favor the first or the last option,
with a slight preference for the last one.

Caitlin Bestler - [email-masked] - http://asomi.com/