Subject: [dhcwg] max-unacked-bndupd

From: Damien Neil <[email-masked]>
Date: Thu, 17 Aug 2006 01:53:34 -0700

draft-ietf-dhc-failover-12.txt defines a "max-unacked-bndupd" option,  
which is sent in all failover CONNECT and CONNECTACK messages.  This  
is defined as follows in section 12.14:

    The maximum number of BNDUPD message that this server is prepared to
    accept over the TCP connection without causing the TCP connection to
    block.

I'm attempting to ascertain whether max-unacked-bndupd applies to the  
number of BNDUPD messages, or the number of binding update  
transactions.  On the face of it, the above definition would appear  
to state the former.  However, sections 7.1 and 7.2 include the  
following phrase:

    The rest of this section is written as though every BNDUPD message
    contains only a single binding update transaction in order to reduce
    the complexity of the discussion.

This introduces a certain confusion between BNDUPD messages and  
binding update transactions.

Since max-unacked-bndupd is defined in section 12.14, which does not  
include the caveat from sections 7.1 and 7.2, I think that a literal  
reading of the draft will conclude that this option applies to  
messages and not transactions.  On the other hand, I think that this  
option is more useful if it is considered to apply to transactions  
rather than messages--the size of a message is variable, depending on  
the number of transactions contained within it, while the size of a  
transaction is somewhat consistent.  (I suspect that the definition  
of this option may predate the ability to pack multiple transactions  
in a single BNDUPD, but my knowledge of the history of the DHCP  
failover draft is insufficient to be certain.)

Any thoughts on how to treat this option would be appreciated.

                     - Damien

_______________________________________________
dhcwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/dhcwg

---

From: Kim Kinnear <[email-masked]>
Date: Thu, 17 Aug 2006 09:34:57 -0400

At 04:53 AM 8/17/2006, Damien Neil wrote:
>messages and not transactions. 

        Yes, this was the intent when it was written.

> On the other hand, I think that this  
>transaction is somewhat consistent. 

        Yes ... but ... if you have a pool of buffers, they need to
        be able to handle the maximum message that you expect.  And
        the maximum message size is 2048 bytes, so what you are saying
        with this is how many 2048 byte message you are prepared to
        accept.

        So, it really is about messages and not individual IP address
        transactions in a bulked message.

> (I suspect that the definition  

        Actually, it doesn't, but I can understand how
        you might think so.

        Cheers -- Kim

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 17 Aug 2006 09:33:12 -0700

> draft-ietf-dhc-failover-12.txt

What is the status of this draft?

Fred
[email-masked]

---

From: "David W. Hankins" <[email-masked]>
Date: Thu, 17 Aug 2006 10:02:47 -0700

> messages and not transactions.  On the other hand, I think that this  

There's a simpler solution:  Section 6.3 should be removed.

It's a rediculous idea; the benefits are negligable, and certainly
don't outweigh the complexity imposed.

-- 
David W. Hankins	"If you don't do it right the first time,
Software Engineer		you'll just have to do it again."
Internet Systems Consortium, Inc.	-- Jack T. Hankins
_______________________________________________
dhcwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/dhcwg

---

From: Damien Neil <[email-masked]>
Date: Thu, 17 Aug 2006 11:33:09 -0700

>         Yes, this was the intent when it was written.

Thanks for the clarification.

>         with this is how many 2048 byte message you are prepared to

Okay, that makes sense.

Looking at this option more closely, I'm wondering about this phrasing:

It seems to me that the focus on TCP connection blocking is  
unimportant.  If the purpose of the max-unacked-bndupd option is to  
ensure that CONTACT messages are transmitted and processed in a  
timely fashion (as stated in section 8.3), then what is important is  
the receiving peer's ability to read BNDUPD messages and queue them  
for processing.

For example, a failover peer that sets its TCP receive buffer size to  
1MB will be able to queue 512 2048-byte BNDUPD messages without  
blocking the connection--but if it reads only one message every  
second, it will take almost ten minutes for it to receive a CONTACT  
message when the buffer is full.

I'm not certain what a better way to phrase the definition of max- 
unacked-bndupd would be, but I don't think it should be described in  
terms of TCP connection blocking.

I'm also curious as to the reason for the focus on ensuring that  
CONTACT messages are processed in a timely fashion.  In the case  
where the failover connection is clogged with BNDUPD messages,  
there's no need to send CONTACTs--since the tSend timer will be reset  
every time a BNDUPD message is sent, and the tReceive timer will be  
reset every time a BNDUPD message is read.  CONTACT messages are only  
useful in circumstances when there is no traffic passing between the  
peers.

                         - Damien

---

From: Damien Neil <[email-masked]>
Date: Thu, 17 Aug 2006 11:35:21 -0700

> don't outweigh the complexity imposed.

I don't think 6.3 is all that complicated, but I suspect that you're  
right about the benefits being negligible.  I don't think I'd shed  
any tears if it went away.

                - Damien

---

From: "David W. Hankins" <[email-masked]>
Date: Thu, 17 Aug 2006 13:36:09 -0700

> I don't think 6.3 is all that complicated, but I suspect that you're  

I suppose it depends on the implementation.  ISC DHCP's failover was
implemented when section 6.3 was a "MAY" (I don't actually know the
history, it might go further back), and the developer at the time
chose a simple loop that digested the option space, to cut out their
values and place them in known locations for later scrutiny by whatever
processing function was to come after (wether it be a BNDUPD or a
BNDACK or etc).  A table is looked up for each option to determine
where and how to cut the option contents out.

In short, our code currently doesn't know it's a binding update until
after the option space has already been shunted into indexed fields,
by which time it's a little late to separate the message into multiple
transactions.

In such an architecture, the changes to support this feature are not
plain or simple.  Short of a complete rewrite, the feature would be
impossible to support in a production capacity (too much spaghetti
code).

I wouldn't mind rewriting it, mind you, but the motivation would not
be to make it act substantively differently than it does today.

For others, admittedly, it might not be that hard to support, but it
is still markedly 'more complex' than simply having all options
be treated the same...a concise format that leaves little if anything
open to interpretation.  Tell me, how might one interpret this:

       Non-IP Address/Non-client specific options first
       assigned-IP-address option for the first IP address
           Options pertaining to first address, including at least the
           binding-status option and others as required.
       assigned-IP-address option for the second IP address
           Options pertaining to second address, including at least the
           binding-status option and others as required.
       ...
       Trailing options (message digest).

Which option(s) are non-client-specific?  Open to interpretation.  If
an option listed in that field appeared later in another ip-address-
delimited section, we can interpret that it should probably take
precedence over the first one (but this is still interpretation).  So,
should the new value then also be applied to all subsequent ip-address-
delimited fields (eg you over-write the earlier value and use that to
the end of the packet)?  Probably not, but it's still interpretation
(and an implementation might very well do that for efficiency).  Is a
message-digest option implicitly an end-of-field marker?  What do you
do with other options that appear there, and how do you know they
don't apply to the previous transaction?  If a server processes each
binding update as it goes (rather than batching the transactions until
after this message is "parsed"), and commits the changes before, let
us imagine, an invalid option or corrupted format entry is discovered,
how should it respond?  Transmit a binding ack for the transactions it
has so far?  But the specification says you have to have all of them.
Rollback (or maybe not!) and pretend the packet was lost, then reset
the connection (or whatever you choose to do to deal with protocol
format errors)?  Again, more interpretation.

So even so, I still don't think it's worth it to complicate the
protocol in this way for at best a trivially small decrease in bytes
transferred on the wire...for a protocol that isn't even bandwidth
conscious, let alone starved!

It is better that all the message option fields be indistiguishably
processed from all other message types option fields - so that code
may be reused, and simplified.

Keep it simple, stupid.

---

From: "David W. Hankins" <[email-masked]>
Date: Thu, 17 Aug 2006 13:53:49 -0700

> reset every time a BNDUPD message is read.  CONTACT messages are only  

I believe the timers would be reset for any message, not specific
ones (don't know if you meant that to be specific or not):

   The tReceive timer is reset whenever a message is received from this
   TCP connection.  If it ever expires, the TCP connection is dropped
   and communications with this partner is considered not ok.  The
   reject reason 17: "No traffic within sufficient time" is placed in
   the DISCONNECT message sent prior to dropping the TCP connection.

   The tSend timer is reset whenever a message is sent over this connec-
   tion. When it expires, a CONTACT message MUST be sent.

So, any message, even ACKs if the dialogue is rather uni-directional.

> useful in circumstances when there is no traffic passing between the  

Well, only when no traffic is actively passing, yes.

Imagine a system has 10 buckets for update messages internally,
no matter how big the tcp buffer sizes are.  It fills all 10,
but none of them are being processed - a lock on the database
has held up processing, let's imagine, so none of them can complete.

The server isn't down, it can still be answering DHCP clients (say
from a memory cache of said database) - it just has...committment
problems.

If you queue 11 updates, then a contact, it will block (or get dropped,
or reset the connection, or who knows) and you may (incorrectly) determine
that the server is down.  Hopefully, however, it just blocks - the
server should have 11 buckets where the 11th is the place it reads
data off the TCP stream into.  Hopefully it just stops reading until
a slot opens up.

So I think the text is correct: it has more to do with the remote
system's processing (eg database) blocking than TCP blocking, but the
result, and what's trying to be avoided, is TCP blocking.

The remote system may still be unable to process the CONTACT message,
eg because it is single-threaded and stuck on that database lock,
and you'll switch to comms-interrupted most likely.  But it won't be
because of the TCP channel.

---

From: Damien Neil <[email-masked]>
Date: Thu, 17 Aug 2006 23:38:46 -0700

> ones (don't know if you meant that to be specific or not):

Right; I specified BNDUPDs, because they're the only other type of  
message that's likely to be being sent at that time.

> from a memory cache of said database) - it just has...committment

Okay.  Let's call this server A, and its peer server B.

There's no worry that server B will conclude that server A is down,  
so long as A continues to send CONTACT messages.

If the TCP connection from B to A blocks, there's still no worry, as  
long as A reads another message in a timely fashion.  The fact that B  
can't send more messages isn't a concern, since there are messages  
queued in A's send buffer--when it goes to read a message, it will  
find one, and reset its tReceive timer.

There's only a problem if A stops reading messages because it has too  
many pending BNDUPDs.  In this case, A may conclude that B is down.   
This is the scenario that the max-unacked-bndupd option is intended  
to prevent.

Imagine a different system: Peer A reads a single failover message  
every second, processes it, and continues.  600 BNDUPD messages are  
sent to it, followed by a CONTACT.  It will take it ten minutes to  
reach and process that CONTACT--but there is no chance that A will  
conclude that peer B is down, since A is processing the other  
messages and resetting tReceive on each one.

> result, and what's trying to be avoided, is TCP blocking.

I think that TCP blocking is unimportant; only the remote system  
processing (database, etc.) is important.  If the TCP connection  
blocks, then the remote peer unquestionably has pending messages  
available to reset its tReceive timer--problems only occur when the  
remote peer stops reading additional messages, regardless of whether  
the TCP channel has blocked or not.

To look at this in a different light: If a peer has a TCP receive  
buffer of infinite size, such that the TCP connection never blocks,  
should it send a very high value for max-unacked-bndupd?  Certainly  
not, if its internal processing blocks after half a dozen BNDUPDs.

                 - Damien

---

From: "David W. Hankins" <[email-masked]>
Date: Fri, 18 Aug 2006 10:10:28 -0700

> the TCP channel has blocked or not.

If the remote peer stops reading any message at all, that blocks the
TCP stream, from a certain point of view.

So it all leads back to TCP blocking, and that makes it the easiest
way to describe the problem.

> buffer of infinite size, such that the TCP connection never blocks,  

I think if the document intended that it was specifically a TCP
protocol layer blocking issue that it epxected to avoid, it would
have said "...cause the TCP window size to reach zero."  I think the
term "TCP blocking" is more generic, and covers a variety of
situations we're all familiar with.

It's possible other words would acheive the same result without
being too specific, however.