Subject: [AAA-WG]: Unofficial Summary of Transport Discussion, from AAA-WG Interim Meeting, San Diego, 07-Feb-2001

From: "Bob Kopacz" <[email-masked]>
Date: Mon, 12 Feb 2001 18:22:03 -0500

After returning from the AAA-WG Interim Meeting of 07-Feb-2001 in
San Diego, I put together a memo which summarizes my recollection of
the Transport discussion, for the purpose of filling in my
co-workers.

Someone suggested that I might pass this along to the AAA-WG
attendees.  It is based on incomplete notes and recollections, but
here it is anyway:

- - - - -

Summary of Transport discussion, from AAA-WG Interim Meeting
of 07-Feb-2001, in San Diego.  Interlink attendees were
Dave Spence and Bob Kopacz.

- Servers must support TCP & SCTP.  NASes must support TCP, may
support SCTP; SCTP may be required in the future.

  SCTP because SCTP needs to move forward; one of its big virtues is
  that it minimizes the head-of-line blocking problem.

  TCP because not all NASes have SCTP in their protocol stacks, and
  SCTP also has problems getting past firewalls.

- NASes will want to connect to multiple proxies, for load balancing
and/or faster failover (don't have to wait for new connection setup)
and/or to help ameliorate the head-of-line blocking problem.  The
static load balancing algorithm uses Pearson's hash, documented in
draft-ietf-hdc-loadb-03.txt.  A NAS connected to multiple proxies
can treat them as primary/secondary or balance load between them.

- The failover/failback algorithm needs to be designed and
documented in the base protocol.

- To help NASes do timely failover, an application-level heartbeat
message is added to Diameter.  Pat's notes call it a Watchdog
message, and apparently was part of earlier Diameter drafts.  The
Watchdog is sent every N seconds when a connection is idle.  N is 3
seconds by default; may be adaptive if RTT/RTO is available to
Diameter application.

- To prevent premature failover, the NAS will include a new maximum
wait time in the request.  If the next hop server cannot return the
reply within that time period, it must send a new Device-Status-Ind
message to the NAS with an appropriate reason code.

- A new e2e message identifier will be added to the Diameter header,
to help the home server detect duplicates.  The problem is that the
current Identifier field is necessarily changed hop-by-hop by
proxies.  Home servers have to worry about duplicate requests, since
a NAS may send a request to Proxy1, then failover & resend the
request to Proxy2; both proxies forward the request to the home AAA
server, and with different identifiers.  The Session-ID is
insufficient as it does not distinguish different messages for the
the same session.

- AAA protcols are typically "application-driven" rather than
"network-driven".  Application-driven means that the time between
protocol exhanges is larger than RTT.

- A tranport "profile" is being developed for AAA protocols.  The
transport profile describes how Diameter interacts with the
transport layer, what capabilities should be enabled/disabled so
that the transport is suitable for the AAA protocol, and what
recent features should/must be added to the transport layer.

By recent features, I mean:

  () Congestion Window Validation [RFCs 2581 & 2861]
  () Limited Transmit (TCP) [RFC 3042]
  () Congestion Manager. [draft-ietf-ecm-cm-03.txt]
  () RTO Validation.

  The recent features improve performance, but do not cause failures
  if not implemented, so they are being given as "should"s rather than
  "must"s.

- CWND validation - RFC 2861

  Problem: CWND estimate not valid for application-driven scenarios
  typical of AAA, i.e. with high inter-packet spacings, RTT
  measurements are made so infrequently that network conditions may
  change between measurements

  So don’t let CWND build as a result of (now) invalid measurements,
  decay it instead.  The result is that CWND=1 or 2 most of the time,
  AAA operates in perpetual slow start.

- Limited Transmit - RFC 3042

With low CWND, there not enough packets in flight to enable fast
re-transmit (=retransmit after 3rd duplicate ack) and fast recovery.
So enable duplicate ACKs to expand CWND by 2 segments in order to
trigger fast re-transmit

- Congestion Manager. [draft-ietf-ecm-cm-03.txt]

  The Congestion Manager (CM) is an end-system module that:

  (i) Enables an ensemble of multiple concurrent streams from a sender
  destined to the same receiver and sharing the same congestion
  properties to perform proper congestion avoidance and control, and

  (ii) Allows applications to easily adapt to network congestion.

  The CM helps integrate congestion management across all applications
  and transport protocols. The CM maintains congestion parameters
  (available aggregate and per-stream bandwidth, per-receiver
  round-trip times, etc.) and exports an API that enables applications
  to learn about network characteristics, pass information to the CM,
  share congestion information with each other, and schedule data
  transmissions.

  The CM enables the Diameter application to access transport
  parameters (RTTavg, RTTdev) via callbacks useful for
  failover/failback.

- RTO Validation - not yet documented

Problem: RTT/RTO not valid for application-driven scenarios. Result:
High RTO can persist long after the conditions that caused it have
dissipated.  Example: Link down, causing multiple re-transmissions,
exponential backoff of RTO.  How quickly can RTO readjust after the
link comes back up?

Solution If RTO > 3 seconds, after CWND is decayed, set RTO=3
seconds, then calculate new value from RTTavg, RTTdev after next
response is received

- Proxies complicate analysis of transport behavior.

NASes cannot sense Proxy-HomeServer bottlenecks, either at transport
or application level.  NAS has sense his hop-by-hop transport to the
proxy, but doesn't have sense of end-to-end transport dynamics.  The
tranport self-clocks hop-by-hop but not necessarily end-to-end.

- Application Layer Error Messages are needed, so that NAS can do
appropriate failover..

() Failures can occur at both transport and application layers
   * NAS-proxy transport connection can fail
   * Proxy-AAA server transport connection can fail
   * Proxy/AAA server can be congested/busy (Mice problem)
() NAS can only measure transport and app layer RTT to the proxy
   * Transport layer parameters only visible to the application
     if Congestion Manager is implemented
   * NAS has no visibility into transport or app layer RTT between proxy and AAA
server
() Result: NAS can failover inappropriately
   * Can move from proxy1 to proxy2 when problem is at AAA server
   * Can make the AAA server problem worse!

- The Application Layer Status Messages are:

  "Busy": Proxy/Server too busy to handle additional requests, NAS
  should immediately failover all requests to another proxy/server

  "Forwarding": Proxy has located AAA server, but timely response is
  not forthcoming; NAS should reset app layer timers, wait for final
  response

  "Can't Locate": Proxy can’t locate the AAA server for the indicated
  realm; NAS should reject access

  "Failover": Proxy has tried primary server, is failing over to
  secondary server; NAS should reset app layer timers, wait for final
  response

  "Can't Forward": Proxy has tried both primary and secondary AAA
  servers with no response; NAS should reject access

  "Processing": Server cannot provide an immediate response to this
  request; NAS should failover this request to another server, but not
  all requests

- TCP Profile: Recommended practice: Nagle algorithm enabled,
Congestion window validation, RTO validation, Limited Transmit,
Congestion Manager, Failover socket option

- SCTP Profile: Recommended practice: Nagle algorithm enabled,
Congestion window validation, RTO validation, Limited Transmit,
Congestion Manager.  Built-in support for multiple streams.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Bob Kopacz                       email: [email-masked]
Interlink Networks, Inc.         phone: (734) 821-1230
775 Technology Drive, Suite 200  fax:   (734) 821-1235
Ann Arbor, MI 48108
U.S.A.