Subject: Another problem with abandon

From: "Jim Sermersheim" <[email-masked]>
Date: Fri, 01 Nov 2002 11:31:05 -0700

Due to abandon not having a response, I've found another little problem with message ID re-use.

The current protocol draft says that a client can re-use a messageID as long as it can be determined that the operation associated with that messageID has completed (this can be determined either by seeing a response to the earlier message, or completing a successful bind).

Here's the problem:

MessageID 1 is used for an operation, the server is processing this op.
Abandon is sent for MessageID 1
A response is sent for operation 1
The client, seing that operation 1 has returned a response, uses messageID 1 again for another operation
This next message reaches the server, and is processed before the previously sent abandon request.
The previous abandon request ends up abandoning the second operation.

I'm not sure yet how to fix this. If we say that:
- a messageID cannot be reused until a subsequent bind, we limit the number of messages between binds to maxInt.
- a messageID of an abandoned operation cannot be reused, it limits the number of abandoned messages between binds to maxInt.

I sure wish abandon had a resonse...

Jim

---

From: John McMeeking <[email-masked]>
Date: Mon, 4 Nov 2002 08:22:50 -0600

I can see the problem, but it seems like it could be handled on the server
side.  The server shouldn't abandon a request that is received after the
abandon request.  TCP/IP delivers these messages to the server in order, so
at a protocol level, it doesn't seem like this particular problem exists.
It may be worth a cautionary note to server implementors.

John  McMeeking

                      "Jim Sermersheim"                                                                                        
                      <[email-masked]>          To:       <[email-masked]>                                        
                      Sent by:                    cc:                                                                          
                      owner-ietf-ldapbis@O        Subject:  Another problem with abandon                                       
                      penLDAP.org                                                                                              

                      11/01/2002 12:31 PM                                                                                      

Due to abandon not having a response, I've found another little problem
with message ID re-use.

The current protocol draft says that a client can re-use a messageID as
long as it can be determined that the operation associated with that
messageID has completed (this can be determined either by seeing a response
to the earlier message, or completing a successful bind).

Here's the problem:

MessageID 1 is used for an operation, the server is processing this op.
Abandon is sent for MessageID 1
A response is sent for operation 1
The client, seing that operation 1 has returned a response, uses messageID
1 again for another operation
This next message reaches the server, and is processed before the
previously sent abandon request.
The previous abandon request ends up abandoning the second operation.

I'm not sure yet how to fix this. If we say that:
- a messageID cannot be reused until a subsequent bind, we limit the number
of messages between binds to maxInt.
- a messageID of an abandoned operation cannot be reused, it limits the
number of abandoned messages between binds to maxInt.

Jim

---

From: "Jim Sermersheim" <[email-masked]>
Date: Mon, 04 Nov 2002 15:53:53 -0700

If the client is multi-threading these messages, and/or the server is mutithreading its handling of these requests, how is the order of requests determined?

Also, we need to allow for LDAP to be transferred over other transports (like UDP) which don't guarantie delivery.

Jim

>>> John McMeeking <[email-masked]> 11/04/02 07:22AM >>>

John  McMeeking

                      "Jim Sermersheim"                                                                                        

                      11/01/2002 12:31 PM                                                                                      

Here's the problem:

Jim

---

From: John McMeeking <[email-masked]>
Date: Wed, 6 Nov 2002 13:54:33 -0600

First,  I better say that I agree that it would simplify things if abandon
had a response.  But I don't think we're broke without a response, and some
could argue we're broke if it is added.

> If the client is multi-threading these messages, and/or the server is
> mutithreading its handling of these requests, how is the order of
requests
> determined?

Talking TCP/IP, I don't think client multi-threading affects this.  From
the server side, it seems like this requires synchronization when reading
incoming messages and keeping some kind of timestamp for each request - at
least with respect to making sure you don't abandon a later request that
reused a message id.  The server might need to serialize more processing to
make sure that when the abandon is processed, the operation that is being
abandoned is known (that is, we don't have thread 1 getting a search
request, and thread 2 getting an abandon without thread 1 having recorded
the search request where thread 2 can find it, otherwise there could be
nothing for thread 2 to abandon), but I don't think that has anything to do
with whether abandon has a response (unless you want the client to retry
the abandon) or reuse of message ids.

> Also, we need to allow for LDAP to be transferred over other transports
(like UDP)
> which don't guarantie delivery.

Talking UDP, there's more to this: messages can be received out of order
(abandon before the abandoned request), received multiple times (search,
abandon, resent search), never received.  I don't think the LDAP over UDP
draft (or my understanding of the issues associated with UDP) has
progressed to where I'd know under what circumstances message ids could be
reused even without throwing abandon into the fray.  If the "result code
cache" concept is used, it seems clear you would never want to reuse
"recent" message ids - they'd be ignored by the server as repeats.  I think
you'd be safest using increasing message ids and wrapping at maxint.  But
the "result code cache" is only mentioned as one way, not "the" way, so who
knows.

>> John McMeeking <[email-masked]> 11/04/02 07:22AM >>
>>

John  McMeeking

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Wed, 06 Nov 2002 16:46:06 -0800

>could argue we're broke if it is added.

We should avoid that argument. 

I, personally, think we just need a simple clarification here.
In particular, I think we just need to say that the client should
not reuse any message id unless it is safe to do so, otherwise
behavior is undefined.  We can define safe as "obtained a final
response for the operation or completed a subsequently issued
bind operation."  This implies that clients should avoid reuse of
abandon and abandoned message ids, which most do by use of a
message id counter.  That's plenty good enough for most clients.
Those few where it isn't can use other mechanisms (such as never
reusing an abandon/abandoned message id... and doing a rebind if
one runs out of message ids.

>Talking UDP,

I don't need to consider UDP.

Kurt

---

From: "Jim Sermersheim" <[email-masked]>
Date: Wed, 06 Nov 2002 19:09:02 -0700

Your suggestion Kurt, is what the current draft states. I guess if servers want to avoid the problem, they need to keep track of the order in which requests are received.

Jim

>>> "Kurt D. Zeilenga" <[email-masked]> 11/06/02 05:46PM >>>

Kurt