Subject: [BEEPwg] Some SEQ frame questions

From: Francis Brosnan Blazquez <[email-masked]>
Date: Thu, 05 Jan 2006 10:04:06 +0100

Hi there,

Now some technical question about BEEP and SEQ frames:

Paul [1] have reported a bug on the Vortex Library bugzilla showing
that SEQ frame are still not really supported. While making some code
to finish this area in a basic manner I've runned into the following
questions:

1) If a client peer wants to reduce/increase the window size for a
   given channel, one might deduce that it should issue a SEQ message
   with the desired new window size.

   This will produce that the local window size will be configured
   with the new window size and a SEQ frame is set to the remote
   peer. It this right? 

   But, what if the remote side doesn't agree on changing requested
   value for the new window size?

   How it is used the ackno value for the SEQ frame received?

2) Because the purpose of the SEQ frame, it is stated that they should
   have greater priority over same messages on the same channel, but,
   this will break the principle "every message send on the same
   channel is sent sequentially". 

   If there are 10 frames waiting to be send and a SEQ frame is issued
   then these messages are freezed until the SEQ frame is sent?

   Does this means that over the same channel "normal" frames are sent
   sequentially but, there another queue with higher priority to send
   SEQ frames generated?

   Does SEQ frames increase frame octec counting (seqno, size) or the
   message counting (msgno) for a channel when the are sent and
   received?

3) It is easy to understand that any change required to the window
   size for a given channel will required to produce a SEQ frame to be
   sent, but if the window size doesn't change:

   It is required to keep on sending SEQ frames notifying current
   status of the channel buffer?

4) What could happen if BEEP peer just ignore SEQ frames or don't
   generate SEQ frames?

Any comment would be appreciated!

Cheers! 

[1] http://dolphin.aspl.es/cgi-bin/bugzilla/show_bug.cgi?id=277

-- 
Francis Brosnan Blazquez <[email-masked]>
Advanced Software Production Line, S.L.

_______________________________________________
BEEPwg mailing list
[email-masked]
http://drakken.dbc.mtview.ca.us/mailman/listinfo/beepwg

---

From: Peter Hall <[email-masked]>
Date: Thu, 05 Jan 2006 09:03:51 -0500

Francis

See my comments inline

> From: Francis Brosnan Blazquez <[email-masked]>
> Organization: Advanced Software Production Line, S.L.
> Date: Thu, 05 Jan 2006 10:04:06 +0100
> To: Paul Lacy <[email-masked]>, BEEPwg <[email-masked]>
> Subject: [BEEPwg] Some SEQ frame questions
> 
> Hi there,
> 

Correct. But RFC3081 has an implementation guideline that says "For
   example, Section 4.2.2.16 of RFC1122 [5] indicates that a "receiver
   SHOULD NOT shrink the window, "
>  value for the new window size?

I cannot find an answer to this one. Read the 3 bullet points in rfc3081
3.1.2 Sending Messages
   1) segment messages
   2) delay until the window is large enough
   3) report error to application

>  How it is used the ackno value for the SEQ frame received?

There is nothing.

>  message counting (msgno) for a channel when the are sent and

No changes to the counts

>  status of the channel buffer?

No need to keep sending them

> 4) What could happen if BEEP peer just ignore SEQ frames or don't

This is what I have done. I'll accept the SEQ frame but I never send a SEQ
frame. When I send a message I'll split the message into whatever the remote
peer SEQ requested.
When I receive a segmented message I'll cache the data until I received the
final piece. Only after I've received the complete message do I hand it back
to the application.

---

From: David Blacka <[email-masked]>
Date: Thu, 5 Jan 2006 11:07:39 -0500

> to finish this area in a basic manner I've runned into the following

One thing that may not be clear here: supporting SEQ frames is  
REQUIRED if you want your BEEP implementation to interoperate with  
other BEEP implementations.  There is some flexibility in how the  
implementation actually handles them, but it must honor incoming SEQ  
frames and emit SEQ frames.

I have, in the past, worked with a BEEP implementation that didn't  
support SEQ frames, and it did work fairly well with itself (this was  
beepy, prior to its switch to using twisted), but it had no hope of  
working with any other BEEP implementation.

>    with the desired new window size.

Um, yes, but I'm not sure you are seeing what the SEQ frame is  
saying.  It isn't saying that "oh, I accept messages up to 4k", it is  
saying "I have 4k available *right now*".  That is, the SEQ frame  
isn't a policy announcement, it is a statement about what the peer is  
willing to accept at that moment.

>    with the new window size and a SEQ frame is set to the remote

I couldn't parse this question.

>    value for the new window size?

There is no room for disagreement.  It is up to the remote side to  
not send more data than the local side is willing to accept.

>    How it is used the ackno value for the SEQ frame received?

I can't remember anything specific.  In general, it is probably used  
as a sanity check.

>    channel is sent sequentially".

What is being said is that the SEQ frames should be given priority in  
the sequence.

>    then these messages are freezed until the SEQ frame is sent?

Could be.

>    sequentially but, there another queue with higher priority to send

That is one way to implement it, but not the only way.

>    status of the channel buffer?

Yes!

> 4) What could happen if BEEP peer just ignore SEQ frames or don't

When I was implementing my own BEEP implementation (the  
Net::BEEP::Lite perl module), I found SEQ to be the hardest thing to  
grasp, so I'm not particularly surprised when implementations skip  
it.  However, SEQ is not an optional part of BEEP over TCP.

What SEQ is doing, essentially, is saying: "hey! I can receive N  
octets on this channel!".  If you are a sender, it is required that  
you do not send any more than N octets on that channel, possibly  
causing the message to fragment. (note that the octet count doesn't  
include the frame header and \r\nEND bit).  If you, as the sender,  
actually send more than N octets, then you can expect the remote end  
to either discard data or to freak out and end the session.  So, this  
is why ignoring incoming SEQs isn't a good idea.  If you are  
receiving data, yet not emitting SEQs, then the remote sender will  
stop sending you data past the default SEQ of 4096 octets.   
Essentially, your channel will quickly stall.  So this is why it is a  
good idea to *send* SEQs.

Now, as I stated at the top of this message, if both sender and  
receiver are ignoring SEQs, then the protocol will work, which is why  
an implementation can get away with not implementing SEQ support:  
they self-interoperate.  However, the implementation will NOT  
interoperate with a different BEEP implementation that *does*  
implement SEQ.

--
David Blacka    <[email-masked]>
Sr. Engineer    VeriSign Applied Research

---

From: David Blacka <[email-masked]>
Date: Thu, 5 Jan 2006 11:26:32 -0500

> No need to keep sending them

Um, no.  I think you've misunderstood how SEQ works.

It is true that you do not have to send a new SEQ frame after every  
received frame (although you could).  You DO need to send one either  
before the original buffer size is exhausted, or immediately afterwards.

What the SEQ frame says is: on this channel, you may send me N  
octets.  It isn't saying "break every message in to N octet frames".

> frame. When I send a message I'll split the message into whatever  

Have you actually interoperated with, say, beepcore-j?

---

From: Peter Hall <[email-masked]>
Date: Thu, 05 Jan 2006 12:45:57 -0500

I'm not sure if I've tried against beepcore. I'm trying to findout for sure
I've personally not done it.

The way I read it was that the SEQ message says don't send me frames larger
X. is that wrong?

What I think your saying is don¹t send a message with a total payload larger
that X. in other words send me either 1 message no larger than X or send me
many continuation messages but not to exceed X.

Why send multiple SEQ messages if it's not changed? Isn't that just wasting
bandwidth?

As it happens my application a reliable syslog server/client don't need to
send more than 4K so may be I've just not had to deal with that problem.
saying that it does work against over vendors servers/clients.

The only reason I implemented what I did with the SEQ message with because
other vendors applications sent it.

Because I don¹t send a SEQ it just means I'll accept the default 4K.

Peter

> From: David Blacka <[email-masked]>
> Date: Thu, 5 Jan 2006 11:26:32 -0500
> To: Peter Hall <[email-masked]>
> Cc: Francis Brosnan Blazquez <[email-masked]>, Paul Lacy
> <[email-masked]>, BEEPwg <[email-masked]>
> Subject: Re: [BEEPwg] Some SEQ frame questions
> 
>

---

From: Francis Brosnan Blazquez <[email-masked]>
Date: Thu, 05 Jan 2006 19:45:23 +0100

El jue, 05-01-2006 a las 11:07 -0500, David Blacka escribió:

Hi David, really helpful comments!, 

As you have clearly described, it seems that the SEQ frames is a kind of
ACK datagram commonly used on many underlaying protocols that allow
peers to perform flow control, so: 

1) It is up to each peer to announce the SEQ frame for its own channel
window (maybe should be called "buffer") sizes and ...

2) Remote peer receiving SEQ frames MUST follow that information to
avoid getting annoyed (aka flooded) its partner peer.

But, what makes me get confused is that RFC3081 doesn't say something
like: "BEEP peers receiving a SEQ frame should take its ackno value to
check it with current seqno over the selected channel and limit its next
sending to the window value received, including stopping from sending
more data if it is overflow, until a next SEQ frame is received".

Then I read your comment:

---

---

Well, or SEQ frames are windows rolling on mech reporting to the remote
peer how many data could hold at a particular time until get it into the
application level or it is something else!. In case that SEQ frames are
buffer size ack mechanism, ackno value should play a very important role
on this issue.

>From your next two comments, could you be more specific?:

1)---

1)---

2)---

2)---

Great reply David,

---

From: David Kramer <[email-masked]>
Date: Thu, 5 Jan 2006 10:46:29 -0800

> The way I read it was that the SEQ message says don't send me  

Yes, that is incorrect.

I think I can make this clearer for everyone with an example.

When a channel opens, the receiving window size for both peer A and  
peer B is 4096 bytes.  If peer A sends a frame with a payload of size  
1024 to peer B, then peer B's receiving window size shrinks to 3072.   
If peer A sends another frame with payload size 1024 to peer B, then  
peer B's receiving window size shrinks further to 2048.  If peer B  
never sends a SEQ frame, and peer A continues to send frames, then  
peer B's receiving window size will eventually reach zero.  At this  
point, if peer A sends any data to peer B, this is a protocol  
violation and peer B immediately terminates the session.

What you need to understand is that the window shrinks with every  
byte received.  The SEQ doesn't say "this is the maximum per-frame  
size I'll accept whenever", it says "this is the total number of  
bytes you may send me until I send you a new SEQ frame extending the  
window."  For a window size of 4096 it could by 4096 1-byte frames,  
or 1 4096-byte frame, or some combination in between.

Note that it is required for a BEEP stack to deal with SEQ frames in  
the middle of a message.  For instance, if peer A wants to sends a  
message that is 5000 bytes long, and peer B has a receiving window  
size of of 4096, then peer A could send a frame no larger than 4096  
bytes.  At this point, peer A still has 4 bytes left to send, which  
it cannot do until it receives a new SEQ frame from peer B extending  
with window by at least 4 bytes.  Therefore peer B needs to be able  
to send SEQ frames in the middle of receiving the message, and peer A  
needs to be able to receive SEQ frames in the middle of sending the  
message.

This might be easier to understand if you consider what the reasoning  
behind the SEQ frame is.  It is a mechanism for flow control.  It is  
a way for peer B to say, I have only allocated a certain amount of  
memory for receiving frames, so don't send me more than that until I  
say it's okay.  If peer B wants to receive a message larger than the  
window size, then peer B will have to send SEQ frame, which it will  
do once it has allocated additional space to receive it.

Suppose peer A wants to send a large file to peer B, which peer B  
will save to disk.  However, peer B doesn't have very much memory  
available.  Therefore, peer B will advertise a small (4K) window, and  
every time the window gets close to being filled, it will write out  
all of the data from memory onto disk, and then grow the window back  
to 4K.  This way peer B never requires more than a 4K buffer for  
holding frame payloads but it can still receive a very large file.

Another tricky aspect of SEQ handling is that each channel of each  
peer has it's own independent receiving window size.  Thus there are  
two window sizes associated with each channel.  Each peer needs to  
know the size of the window being advertised by the other, while at  
the same time keeping track of the window it has advertised to the  
other.  The former is important so that the peer doesn't send more  
data than the other can handle, while the latter is important so that  
the peer knows when it needs to send a SEQ to the other to re-extend  
the window.

Another point which was made earlier is that the window may never be  
shrunk by a SEQ frame.  The window is only, and always, shrunk as the  
result of receiving data.  The sending peer needs to keep track of  
how many bytes it sends and subtract that from the last advertised  
window size, so that it never sends more data than can fit in the  
window.  Likewise, the receiving peer needs to keep track of how many  
bytes it receives and subtract that from the last window size it  
advertised, so that it knows when the window is full and needs to be  
re-extended.

-David Kramer

On Jan 5, 2006, at 9:45 AM, Peter Hall wrote:

> What I think your saying is don’t send a message with a total  
> payload larger
> Because I don’t send a SEQ it just means I'll accept the default 4K.
>
> Peter
>
>

---

From: Marshall Rose <[email-masked]>
Date: Thu, 5 Jan 2006 10:58:46 -0800

> This might be easier to understand if you consider what the  
> reasoning behind the SEQ frame is.  It is a mechanism for flow  
> control.  It is a way for peer B to say, I have only allocated a  
> certain amount of memory for receiving frames, so don't send me  
> more than that until I say it's okay.  If peer B wants to receive a  
> message larger than the window size, then peer B will have to send  
> SEQ frame, which it will do once it has allocated additional space  
> to receive it.

i agree with absolutely everything in your email as to how SEQ works  
and why; however, i want to add one thing to the above paragraph:  
let's answer the question: "why?".

after any tuning, if you could open only one channel for data  
exchange, then you wouldn't have SEQ frames in BEEP. they wouldn't be  
needed.

however, because you are allowed to open multiple simultaneous  
channels for data exchange, you can run into a problem commonly  
referred to as "head of line". this is where one channel monopolizes  
all of the underlying transport capacity by sending something really  
big, and the other channels get blocked behind that data.

so, what SEQ allows a BEEP peer to do is coordinate use of the  
available bandwidth by the channels being multiplexed over a single  
TCP connection. for example, the BEEP API you use may allow you to  
associate a priority with each channel (or each message sent by a  
channel), then when the implementation of that API sees that it's  
able to send a frame over the TCP connection, it can determine which  
channel should get to use the bandwidth.

/mtr

---

From: David Blacka <[email-masked]>
Date: Thu, 5 Jan 2006 14:04:18 -0500

> I can't remember anything specific.  In general, it is probably used

Yeah, I spaced for a moment.  The ackno value is, indeed, critical.   
In my head I was thinking that that field was called "seqno" and  
ackno was some other useless field.  This is what you get from not  
looking stuff up!

---

From: David Blacka <[email-masked]>
Date: Thu, 5 Jan 2006 14:16:39 -0500

> The way I read it was that the SEQ message says don't send me  

SEQ is advertising a current buffer size for a channel for that  
moment in time (or more precisely, for a particular sequence  
number).  It isn't talking about frames specifically.

> many continuation messages but not to exceed X.

It is saying that you can send up to X octets without another SEQ,  
regardless of the message/frame breakdown.

> Why send multiple SEQ messages if it's not changed? Isn't that just  

SEQ is implementing flow control, so think of it as an ACK.

> saying that it does work against over vendors servers/clients.

Well, you would be able to send *only* 4k on a given channel.  Well,  
reliably in any case.  Also keep in mind that you will be able to  
perfectly interoperate with yourself.  So if both peers are using  
your BEEP implementation, I wouldn't expect a problem.

> other vendors applications sent it.

Well, they have to.

> Because I don’t send a SEQ it just means I'll accept the default 4K.

It means you will only accept 4k total on each channel.

Here is what BEEP implementations need to do with respect to SEQ:

Senders must track on a per channel basis how many octets they are  
allowed to send.  So, initially, on channel creation, this may be set  
to 4096.  Now lets say you send 200 octets in two frames.   
Immediately, the sender updates this number to 3896, and notes that  
it has sent a total of 200 octets.  Now lets say that the sender has  
received a SEQ frame that says "SEQ 1 100 4096".  This means: "after  
receiving octet 100 on channel 1, I can receive 4096 more octets of  
payload".  So the sender gets to update its number to 3996.  It is  
now allowed to send at most 3996 octets before getting another SEQ  
frame.  Note that the ackno field is critical here.

Receivers need to have a mechanism for knowing when they must send a  
SEQ frame to prevent the channel from stalling.  In general, they do  
this by mirroring the calculation that the sender is doing (i.e.,  
subtract on receipt of data, resetting when sending a SEQ), but this  
isn't the only way to do it.

Also note that like sequence numbers, SEQ frames operate on a per- 
channel and per-direction basis.

---

From: Peter Hall <[email-masked]>
Date: Thu, 05 Jan 2006 14:32:11 -0500

I've just had confirmation that a vender has used beepcore-java and it works
fine with my implemention.

P

---

From: Peter Hall <[email-masked]>
Date: Thu, 05 Jan 2006 14:51:27 -0500

That's better, now I understand what it is trying to do. I never did it in
the first place because I could get my head around it.

I'll make the changes.

Thanks
Peter

> Date: Thu, 5 Jan 2006 14:16:39 -0500

---

From: Gabe Wachob <[email-masked]>
Date: Thu, 5 Jan 2006 11:54:57 -0800

Just a reminder that for folks implementing beep libraries/code and
would like to talk with similar folks, there is the beepbuilders email
list and sourceforge project (the sf project never did much, but the
email list is active):

http://beepbuilders.sf.net

If there is interest in developing the sf project (test cases,
harnesses, etc), please let me know - I'd be even willing to hand over
the project to someone else with a more vested interest at this point.

   -Gabe

--
Gabe Wachob / [email-masked] / http://www.wachob.com