Subject: Graceful shutdown #458

From: Martin Thomson <[email-masked]>
Date: Thu, 8 May 2014 14:55:03 -0700

The shutting down state machine currently looks something like this:

  accepting requests
    --[GOAWAY]-> not accepting requests
    --[TCP FIN]-> closed

And during the "not accepting requests" state, it may be that some
requests are lost.  You will have to check that each request made it.

I proposed (and Daniel provided pull #475) to not change this state
machine, but to exploit the nature of the first transition, and
repeating the GOAWAY frame.

Roberto would prefer that we create a new state in the flow:

  accepting requests
    --[DRAINING]-> about to stop accepting requests
    --[GOAWAY]-> not accepting requests
    --[TCP FIN]-> closed

I note that this DRAINING frame looks similar in function to ALTSVC.

If this summary is accurate, and in the absence of new information, I
think that the best plan for this is to close at the interim, once we
have more feedback from implementations.

---

From: Roberto Peon <[email-masked]>
Date: Thu, 8 May 2014 15:01:11 -0700

DRAINING differs from ALTSVC mostly in that it requires no state be kept.
Otherwise, I believe ALTSVC can be used to drain things so long as it is
worded such that an ALTSVC to the same location causes a new connection to
be created.

-=R

---

From: Martin Thomson <[email-masked]>
Date: Thu, 8 May 2014 15:13:16 -0700

> DRAINING differs from ALTSVC mostly in that it requires no state be kept.

True, equivalent to ALTSVC with ma=0, I guess.

> worded such that an ALTSVC to the same location causes a new connection to

I think that we'd definitely need that wording.  Would that work for you?

---

From: Roberto Peon <[email-masked]>
Date: Thu, 8 May 2014 15:42:32 -0700

That'd probably work. I'd be a bit worried about someone always sending
ALTSVC <policy> on every connection (since that is easy), but if policy is
self, it could cause some interesting effects.
DRAINING is probably easier to use than ALTSVC for things like that.

In the end, I'm probably fine either way, I'm sure we can get it right. I
just don't want to be afraid of adding an opcode if that achieves better
clarity/safety.

-=R

---

From: Daniel Sommermann <[email-masked]>
Date: Thu, 8 May 2014 16:01:52 -0700

I'm not very keen on using ALTSVC to indicate a connection draining 
state. Especially since client implementation of alternative services is 
rather fuzzy (not much normative language in that spec), I'd rather 
stick to a mechanism where the client behavior is well understood and 
requires minimal changes to the spec.

I have a slight preference for GOAWAY over DRAINING. With DRAINING we 
gain added explicitness, but at the cost of verbosity (the state diagram 
would grow at least). The spec already establishes that multiple GOAWAYs 
can be received (NO_ERROR followed by some other error during the 
graceful connection shutdown), so I think the work we need to do here is 
minimal. Either mechanism is fine for me, and I'm happy to resolve this 
at the interim.

---

From: Jeff Pinner <[email-masked]>
Date: Thu, 8 May 2014 16:09:27 -0700

We have a SPDY proof of concept running that will send two GOAWAY frames on
shutdown:

GOAWAY(max stream id, NO_ERROR)
(send ping and wait for pong or timeout)
GOAWAY(last good stream id, *some code*)
TCP FIN

On Thu, May 8, 2014 at 4:01 PM, Daniel Sommermann <[email-masked]> wrote:

> I'm not very keen on using ALTSVC to indicate a connection draining state.
> Especially since client implementation of alternative services is rather
> fuzzy (not much normative language in that spec), I'd rather stick to a
> mechanism where the client behavior is well understood and requires minimal
> changes to the spec.
>
> I have a slight preference for GOAWAY over DRAINING. With DRAINING we gain
> added explicitness, but at the cost of verbosity (the state diagram would
> grow at least). The spec already establishes that multiple GOAWAYs can be
> received (NO_ERROR followed by some other error during the graceful
> connection shutdown), so I think the work we need to do here is minimal.
> Either mechanism is fine for me, and I'm happy to resolve this at the
> interim.
>
>

---

From: Martin Thomson <[email-masked]>
Date: Thu, 8 May 2014 16:26:39 -0700

> GOAWAY(last good stream id, *some code*)

I would have thought that you would want to let the requests you have
received complete, or is that not the plan here?

Note that you don't send the last "good" stream, you send the last
stream that you passed on for processing.

---

From: Jeff Pinner <[email-masked]>
Date: Thu, 8 May 2014 16:32:53 -0700

> received complete, or is that not the plan here?
We do, specifically we send GOAWAY with max stream id, and then for some
period of time until we send the second GOAWY, we continue to accept
streams. Then when we send the second GOAWAY, we send it with the last
stream id we have decided to process (which could have been received after
the first GOAWAY). Then we reject new streams with REFUSED and gracefully
close.