Subject: Re: [tcpm] question about if a recent Linux patch is compliant with RFC7323 on window scaling

From: Yoshifumi Nishida <[email-masked]>
Date: Sun, 4 Sep 2016 03:55:05 -0700

Hi Cheng,

On Tue, Aug 30, 2016 at 7:16 AM, Cui, Cheng <[email-masked]> wrote:
> Six days have passed and no response yet. Can anyone in this board make a comment?
>
> Thanks,
> --Cheng Cui
> NetApp Scale Out Networking
>
>
> On 8/24/16, 5:06 PM, "Cui, Cheng" <[email-masked]> wrote:
>
>     Hello David,
>
>     I hope this email could reach you well, because I found related
>     discussions about this topic on window scaling and the case of window
>     shrinking (or retraction or loss of precision). And I try to make this
>     question simple.
>
>     There is a recent Linux patch at receiver side to round-up advertised
>     window due to precision loss of window scaling. It reaches my attention
>     because the same problem could also happen between a pair of Linux and
>     FreeBSD nodes, and I am not aware of any similar patch in FreeBSD yet.
>
>     The Linux patch is this:
>     http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=6
>     07bfbf2d55dd1cfe5368b41c2a81a8c9ccf4723
>
>     And I quote some description of the Linux patch below:
>     > If the sender uses up the entire window before it is shrunk, this can
>     >have
>     > chaotic effects on the connection. When sending ACKs,
>     >tcp_acceptable_seq()
>     > will notice that the window has been shrunk since tcp_wnd_end() is before
>     > tp->snd_nxt, which makes it choose tcp_wnd_end() as sequence number.
>     > This will fail the receivers checks in tcp_sequence() however since it
>     > is before it's tp->rcv_wup, making it respond with a dupack.
>
>
>     If the Linux sender?s behavior is right ("ACK-only packets should be sent
>     with the largest in-window sequence number that has ever been sent." ref:
>     https://www.ietf.org/mail-archive/web/tcpm/current/msg10512.html), it
>     actually chooses "tp->snd_una+tp->snd_wnd" (tcp_wnd_end()) instead of
>     tp->snd_nxt, as it thought tp->snd_nxt is out of window, in case of
>     precision loss which made the receiver?s advertise-window smaller.
>     However, the RFC7323 explicitly specifies the receiver should right shift
>     the exact scale factor, without mentioning any further round-up:
>
>     https://tools.ietf.org/html/rfc7323#page-9
>     >    o  The window field (SEG.WND) of every outgoing segment, with the
>     >       exception of <SYN> segments, MUST be right-shifted by
>     >       Rcv.Wind.Shift bits:
>     >
>     >                     SEG.WND = RCV.WND >> Rcv.Wind.Shift
>
>     And also if I am understanding correctly, RFC7323 page 10 (2.4. Addressing
>     Window Retraction) specifies it is the sender?s responsibility to handle
>     the sequence number out of window:
>     https://tools.ietf.org/html/rfc7323#page-10
>     >    4)  On first retransmission, or if the sequence number is out of
>     >        window by less than 2^Rcv.Wind.Shift, then do normal
>     >        retransmission(s) without regard to the receiver window as long
>     >        as the original segment was in window when it was sent.
>     >
>     >    5)  Subsequent retransmissions MAY only be sent if they are within
>     >        the window announced by the most recent <ACK>.
>
>     In your discussion of the link below, if I am understanding correctly, you
>     were also in favor of "Yes. It is ok to have more receive buffer space
>     available than you advertise in the window, but not ok to advertise a
>     larger window than you are willing to accept." So I think you were not in
>     favor of rounding-up the advertise-window.
>
>     https://www.ietf.org/mail-archive/web/tsvwg/current/msg06475.html
>
>
>     So my question is: Is the Linux patch (on receiver side to round-up the
>     advertise-window) RFC compliant? Or is it just the right thing (vendor
>     specific) and other systems like FreeBSD have to follow up, so that their
>     talks can have no problem?

I think the Linux patch at least follows the following principle in RFC793.

  The mechanisms provided allow a TCP to advertise a large window and to
  subsequently advertise a much smaller window without having accepted
  that much data.  This, so called "shrinking the window," is strongly
  discouraged.  The robustness principle dictates that TCPs will not
  shrink the window themselves, but will be prepared for such behavior
  on the part of other TCPs.

I personally think a real question would be how FreeBSD prepares for
shrinking window.
Linux chose one way for it, but I guess all implementations don't need
to follow the same way.

--
Yoshi

---

From: "Cui, Cheng" <[email-masked]>
Date: Tue, 6 Sep 2016 15:10:14 +0000

On 9/4/16, 6:55 AM, "Yoshifumi Nishida" <[email-masked]> wrote:
>   that much data.  This, so called ³shrinking the window,² is strongly
> Linux chose one way for it, but I guess all implementations don¹t need
> to follow the same way.
> 
> --
> Yoshi

Hi Yoshi,

Thanks a lot for the reply.

I agree that's a choice for Linux. But I think FreeBSD has to follow the same
way; otherwise, Linux has to do something at its sender side to send a correct
th_seq number.

I think this forum may not be a best place to talk about FreeBSD
implementation. But talking about FreeBSD, I did not find anything in FreeBSD
to round-up the shrinking window due to window scale precision loss, neither
did I find anything about how FreeBSD can tolerate the incoming left-shifted
th_seq ("tp->snd_una+tp->snd_wnd") in an ACK packet sent by Linux in this
particular case, as Linux had sent all the data in a previous larger window. I
guess the FreeBSD will just drop the Linux ACK packet as a DUP (see "todrop =
tp->rcv_nxt - th->th_seq;").

I also sent a question to the FreeBSD mailing lists. But so far no reply yet:
https://lists.freebsd.org/pipermail/freebsd-transport/2016-August/000141.html
https://lists.freebsd.org/pipermail/freebsd-net/2016-September/046048.html

Therefore, I think my best plan is to send separate patches to Linux and
FreeBSD. For Linux, the patch will be correct the th_seq number at sender side
if window scale precision loss is detected. For FreebSD, I think the patch has
to round-up the receive-advertise-window at receiver side.

-- Cheng