Subject: Comments regarding draft-nishida-tsvwg-sctp-failover-03.txt

From: Michael Tüxen <[email-masked]>
Date: Mon, 22 Aug 2011 21:04:41 +0200

Dear all,

as promised at the last TSVWG session I would like to bring up
some comments regarding draft-nishida-tsvwg-sctp-failover-03.txt.

Randy and myself implemented the ID during the last IETF, so
these comments are based on some implementation experience.

Editorial comments:

* I would suggest to have a section 4 which describes existing
  stuff. Basically it should contain only what is now in section 4.1
  and 4.2.
* In another section I would describe the the suggested method.
  This would cover section 4.3, 5.1 and 5.2.

This would result in having the suggested method and its discussion
in one section.

The rules given in 4.3 should be enumerated. Then one can easily refer
to them.

For the following comments, I will paste them for simpler reading
of this e-mail:

   o  The sender maintains a new tunable parameter called Potentially-
      failed.Max.Retrans (PFMR).  An association's PFMR value MUST be
      lower than the association's PMR value.  The recommended value of
      PFMR = 0.

I think "MUST be lower" is not the appropriate formulation. What you
want to say is that if PFMR >= PMR, than quick failover is basically
switched off. This is a valid choice of parameters.
In FreeBSD, we have used this as the default (PFMR = 0xffff).

   o  The sender never transmits data to a PF destination.  However,
      when all destinations are in either PF or Inactive state, the
      sender SHOULD transition a destination marked PF to the active
      state and transmit data to this destination.  The destination's
      error counter MUST NOT be cleared during this state transition.
      It is recommended that the sender transitions the PF destination
      with least error count (fewest consecutive timeouts) to the active
      state.  In case of a tie (multiple PF destinations with same error
      count), the sender MAY choose the last active destination.

I would prefer to make a weaker formulation than "never transmits data
to a PF destination" and not take destinations out of PF. This is because
I would prefer doing it similar to the handling of the unreachable state
and there taking it out would result in sending up notifications.
So choose a formulation that you want to avoid sending to PF destinations,
but do it if no non PF and reachable destinations are available.

   o  Only heartbeats MUST be sent to PF destination(s) once per RTO.
      This means the sender SHOULD ignore HB.interval for PF
      destinations.  If an heartbeat is unanswered, the sender
      increments the error counter and exponentially backs off the RTO
      value.  If error counter is less than PMR, the sender SHOULD
      transmit another heartbeat immediately after T3-timer expiration.
      An implementation MAY use protocol parameter 'PFHB.interval' for
      the interval of heartbeat transmissions.  If PFHB.interval is non-
      zero, a heartbeat packet is sent once per RTO of each destination
      address plus PFHB.interval with jittering of +/- 50% of the RTO
      value.  Use of PFHB.interval can reduce the frequency of failover,
      which might be useful where the characteristic of the paths are
      mostly equal.

I'm not sure if it is worth to have a PFHB.interval. At least in FreeBSD
we just use 0... So maybe just get rid of it.

   o  When the sender receives an heartbeat ACK from a PF destination,
      the sender clears the destination's error counter and transitions
      the PF destination back to active state.  This state transition
      MUST NOT be notified to the ULP unless it is explicitly requested.
      This destination's cwnd is set to 1 MTU (TODO: or 2?  Needs more
      text discussing rationale; can revisit later?)

We don't have defined a notification. I don't see the value
of it. So maybe you want to get rid of it?

   o  When all destinations are in the Inactive state, the sender
      transitions one of the destinations back to the Active state and
      continues data transmission to this destination.  This proposal
      recommends that the sender transitions the Inactive destination
      with least error count (fewest consecutive timeouts) to the active
      state.  In case of a tie (multiple Inactive destinations with same
      error count), the sender MAY choose the last active destination.

As said above: Moving one to the active state would require sending
up notifications. At least the FreeBSD stack does not move the
state to active, it just uses an unreachable (but confirmed) destination
address.

I think it would also be good for the application to control the thresholds
involved. So I suggest to add a socket API section.

The following describes what FreeBSD does:

<section title="Socket API Considerations">

<t>This section describes how the socket API defined in
<xref target='I-D.ietf-tsvwg-sctpsocket'/> is extended
to provide a way for the application to control the quick
failover behavior.</t>

<t>Please note that this section is informational only.</t>

<t>A socket API implementation based on
<xref target='I-D.ietf-tsvwg-sctpsocket'/>
is extended by adding a new read/write socket option for the
level IPPROTO_SCTP and the name SCTP_PEER_ADDR_THLDS
as described below.</t>

<t>Support for the SCTP_PEER_ADDR_THLDS socket option needs
also to be added to the function sctp_opt_info().</t>

<section title="Peer Address Thresholds (SCTP_PEER_ADDR_THLDS) socket option">

<t>Applications can control the quick failover behavior by getting or
setting the number of timeouts before a peer address is considered
potentially failed or unreachable.</t>

<t>The following structure is used to access and modify the thresholds:
<figure>
<artwork>
struct sctp_paddrthlds {
  sctp_assoc_t spt_assoc_id;
  struct sockaddr_storage spt_address;
  uint16_t spt_pathmaxrxt;
  uint16_t spt_pathpfthld;
};
</artwork>
</figure>
<list style='hanging'>
<t hangText="spt_assoc_id:">
This parameter is ignored for one-to-one style sockets.
For one-to-many style sockets the application may fill in an association
identifier or SCTP_FUTURE_ASSOC for this query.
It is an error to use SCTP_{CURRENT|ALL}_ASSOC in spt_assoc_id.</t>
<t hangText="spt_address:">
This specifies which peer address is of interest. If a wildcard address
is provided, this socket option applies to all current and future peer addresses.</t>
<t hangText="spt_pathmaxrxt:">
Each peer address of interest is considered unreachable,
if its path error counter exceeds spt_pathmaxrxt.</t>
<t hangText="spt_pathpfthld:">
Each peer address of interest is considered potentially failed,
if its path error counter exceeds spt_pathpfthld.</t>
</list></t>

</section>

</section>

Feel free to use this text or change it...

Best regards
Michael

---

From: Preethi Natarajan <[email-masked]>
Date: Tue, 23 Aug 2011 16:46:49 -0700

Hi Michael,

Thanks a lot for the comments/suggestions. Please find our response below.

> This would result in having the suggested method and its discussion

Agree.

> The rules given in 4.3 should be enumerated. Then one can easily refer

OK.

> In FreeBSD, we have used this as the default (PFMR = 0xffff).

OK.

> but do it if no non PF and reachable destinations are available.

Yes, agree that transition from PF to Active shouldn't result in
notifications to the ULP. We are fine with a weaker formulation for the
first sentence. However, it may be a good idea to decouple ULP notification
and destination state transition. What do you think? How about the following
to replace the first few sentences above -- "The sender SHOULD avoid data
transmission to PF destinations. When all destinations are in either PF or
Inactive state, the sender MAY move the destination from PF to active state
(and transmit data to the active destination) or the sender MAY transmit
data to a PF destination.

> we just use 0... So maybe just get rid of it.

We'll discuss and get back on this one.

> of it. So maybe you want to get rid of it?

OK. 

> state to active, it just uses an unreachable (but confirmed) destination

OK. 

> involved. So I suggest to add a socket API section.

Agree. We'll add text describing such a socket API as you suggested.

Preethi

---

From: Yoshifumi Nishida <[email-masked]>
Date: Wed, 24 Aug 2011 03:24:14 -0700

Hi Michael,

Thank you so much for the review.  Most points were addressed by Preethi.
So, I just would like to clarify one point.

> of it. So maybe you want to get rid of it?

Hmm. I might miss your point, but I believe we cannot use arbitrary value..
Shall we refer 7.2.1 in RFC4960?

--
Yoshifumi Nishida
[email-masked]

---

From: Michael Tüxen <[email-masked]>
Date: Wed, 24 Aug 2011 14:25:05 +0200

> data to a PF destination.
The new text is OK, but my reasoning was:
1. You have text describing what to do if all destinations are in PF.
   This is necessary.
2. You have text describing what to do if all destinations are unreachable.
   This is necessary.
3. Both texts are similar.
   This is good.
4. There should not be a state change required in 2., since this would
   require notifications which would give the wrong impression to
   the application.
Therefore I would also prefer no state change in 1.

---

From: Michael Tüxen <[email-masked]>
Date: Wed, 24 Aug 2011 14:27:01 +0200

> Shall we refer 7.2.1 in RFC4960?
I was not clear... I'm referring to the sentence:

This state transition MUST NOT be notified to the ULP unless it
is explicitly requested.

Currently we do not have such a notification. That is why I'm asking,
if it would be better to remove it.

Best regards
Michael

---

From: Yoshifumi Nishida <[email-masked]>
Date: Wed, 24 Aug 2011 11:47:49 -0700

> if it would be better to remove it.

Oh. I see. Make sense. Thanks for the clarification.

Regards,
--
Yoshifumi Nishida
[email-masked]