Subject: [alto] Issue about incr-updates-sse update stream control design

From: Jensen Zhang <[email-masked]>
Date: Fri, 02 Mar 2018 15:57:19 +0000

Hi incr-update-sse authors and all,

I have a question about the current update stream control design in
incr-updates-sse.

In the last paragraph of Section 7.6.2, the document writes

"When the client uses the stream control service to stop updates for one or
more resources (Section 8
<https://tools.ietf.org/html/draft-ietf-alto-incr-update-sse-09#section-8>),
the ALTO server MUST send a control event (Section 6.3
<https://tools.ietf.org/html/draft-ietf-alto-incr-update-sse-09#section-6.3>)
whose "remove" field has the client-ids of those resources."

As section 7 defines "Update Stream Service", this paragraph means the ALTO
server MUST send a control event in the update stream to notify the client
the status of a stream control service request. Why do we want to use a
different channel to return the output? Why not return this control event
from the response of update stream control service? I feel it will
introduce the unnecessary complexity.

Can anyone of authors elaborate the reason for this design?

Thanks,
Jensen

---

From: "Y. Richard Yang" <[email-masked]>
Date: Fri, 2 Mar 2018 18:30:24 -0500

Hi Jensen,

Good question! Let's see the design space:
1. Client can send "add" or "remove"; since HTTP requests are "one-way",
this needs to use a new control channel--a new HTTP request

2. Response option 1: Server notifies client outcome in the HTTP response
by using a HTTP response code;

3. Response option 2: Server notifies client outcome in the update stream;

Note that we can have 3 options: (1) option 1 only; (2) option 2 only; (3)
both.

Option 1 only may have a concurrency issue, depending on the specific
design and semantics. Consider two semantics:

1. Server responds *only after* the last updates have been sent
successfully. This will lead to synchronization of likely two processes at
the server; hence the guaranteed serialized sequence is:

client sends remove request -> server (process 1) processes the request ->
server (may be process 2) flushes all outstanding updates to client ->
server (process 1) notifies client by returning in the http request.

The price here is that the server process structure can be more complex.

2. Server responds *immediately* (i.e., not blocked by flushing and ack of
all pending updates), and hence, the semantics is that although the client
gets a "remove" success, updates may continue to arrive; hence the client
code needs to be careful to handle the issue. It is not clear when the
client can be notified that the buffer is flushed indeed.

Option 2 does not have the preceding issue: the server HTTP response only
acknowledges that the "remove" request is received successfully and can be
processed, but not that it is cleared.

But thinking about the issue one more time, we can use option 2 only, but
then the document need to make clear that the client can handle remaining
updates gracefully---just as after TCP close but data can continue to
arrive. Is this a design that you appreciate more?

Richard

On Fri, Mar 2, 2018 at 10:57 AM, Jensen Zhang <[email-masked]>
wrote:

> "When the client uses the stream control service to stop updates for one
> or more resources (Section 8
> As section 7 defines "Update Stream Service", this paragraph means the
> ALTO server MUST send a control event in the update stream to notify the
> client the status of a stream control service request. Why do we want to
> use a different channel to return the output? Why not return this control
> event from the response of update stream control service? I feel it will

-- 
-- 
 =====================================
| Y. Richard Yang <[email-masked]>   |
| Professor of Computer Science       |
| http://www.cs.yale.edu/~yry/        |
 =====================================

---

From: Dawn Chan <[email-masked]>
Date: Sat, 3 Mar 2018 12:57:29 +0000

Hi Jensen and Richard,

Here are some of my ideas about the problem.

Actually, the solution for option 1 is already mentioned in Section 8.6 of the draft:
“If a request is successful, the server returns an HTTP ‘204 No Content’ response with no data.  If there are any errors, the server MUST return the appropriate error code, and MUST NOT add or remove any resources from the update stream.  Thus control requests are atomic: they cannot partially succeed.”

Here, the appropriate error code is returned in the HTTP response. Actually, an HTTP response has to be sent to the client whether the request is successful or not.

The approach that the server also notifies client outcome in the update stream can guarantee that the request does succeed. So I think it is better to use them both.

This way, if the request is processed successfully, the server will return with an HTTP “204” no content and a message in the update stream. If the request fails, the server will only return an error with ALTO error code in the HTTP response.

Another issue in the draft is the name of “client-id”. The name “client-id” looks more likely to distinguish ALTO clients, rather the different requests of different resources. So, I think it might be better to use another name, maybe “request-id”, to replace “client-id”. Do you have any better names?

Dawn

On 3 Mar 2018, at 7:30 AM, Y. Richard Yang <[email-masked]<mailto:[email-masked]>> wrote:

Hi Jensen,

Good question! Let's see the design space:
1. Client can send "add" or "remove"; since HTTP requests are "one-way", this needs to use a new control channel--a new HTTP request

2. Response option 1: Server notifies client outcome in the HTTP response by using a HTTP response code;

Note that we can have 3 options: (1) option 1 only; (2) option 2 only; (3)  both.

Option 1 only may have a concurrency issue, depending on the specific design and semantics. Consider two semantics:

1. Server responds *only after* the last updates have been sent successfully. This will lead to synchronization of likely two processes at the server; hence the guaranteed serialized sequence is:

client sends remove request -> server (process 1) processes the request -> server (may be process 2) flushes all outstanding updates to client -> server (process 1) notifies client by returning in the http request.

2. Server responds *immediately* (i.e., not blocked by flushing and ack of all pending updates), and hence, the semantics is that although the client gets a "remove" success, updates may continue to arrive; hence the client code needs to be careful to handle the issue. It is not clear when the client can be notified that the buffer is flushed indeed.

Option 2 does not have the preceding issue: the server HTTP response only acknowledges that the "remove" request is received successfully and can be processed, but not that it is cleared.

But thinking about the issue one more time, we can use option 2 only, but then the document need to make clear that the client can handle remaining updates gracefully---just as after TCP close but data can continue to arrive. Is this a design that you appreciate more?

Richard

On Fri, Mar 2, 2018 at 10:57 AM, Jensen Zhang <[email-masked]<mailto:[email-masked]>> wrote:
Hi incr-update-sse authors and all,

I have a question about the current update stream control design in incr-updates-sse.

"When the client uses the stream control service to stop updates for one or more resources (Section 8<https://tools.ietf.org/html/draft-ietf-alto-incr-update-sse-09#section-8>), the ALTO server MUST send a control event (Section 6.3<https://tools.ietf.org/html/draft-ietf-alto-incr-update-sse-09#section-6.3>) whose "remove" field has the client-ids of those resources."

As section 7 defines "Update Stream Service", this paragraph means the ALTO server MUST send a control event in the update stream to notify the client the status of a stream control service request. Why do we want to use a different channel to return the output? Why not return this control event from the response of update stream control service? I feel it will introduce the unnecessary complexity.

Thanks,
Jensen

--
--
 =====================================
| Y. Richard Yang <[email-masked]<mailto:[email-masked]>>   |
| Professor of Computer Science       |
| http://www.cs.yale.edu/~yry/        |
 =====================================
_______________________________________________
alto mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/alto

---

From: "Y. Richard Yang" <[email-masked]>
Date: Sun, 4 Mar 2018 21:23:49 -0500

Thanks a lot for the points, Dawn.

Please see below.

On Sat, Mar 3, 2018 at 7:57 AM, Dawn Chan <[email-masked]> wrote:

> “If a request is successful, the server returns an HTTP ‘204 No Content’
> response with no data.  If there are any errors, the server MUST return the
> appropriate error code, and MUST NOT add or remove any resources from the
> update stream.  Thus control requests are atomic: they cannot partially
> succeed.”
>
> Here, the appropriate error code is returned in the HTTP response.
> Actually, an HTTP response has to be sent to the client whether the request
> is successful or not.
>
> The approach that the server also notifies client outcome in the update
> stream can guarantee that the request does succeed. So I think it is better
> to use them both.
>
> This way, if the request is processed successfully, the server will return
> with an HTTP “204” no content and a message in the update stream. If the
> request fails, the server will only return an error with ALTO error code in
> the HTTP response.
>
>
It is clear that using both, as it is in the current design, is an option.
Make sense to others in the WG?

> Another issue in the draft is the name of “client-id”. The name
> “client-id” looks more likely to distinguish ALTO clients, rather the
> different requests of different resources. So, I think it might be better
> to use another name, maybe “request-id”, to replace “client-id”. Do you
> have any better names?
>

Since it names an update stream, how about stream-id?

Richard

>
> Dawn
>
> On 3 Mar 2018, at 7:30 AM, Y. Richard Yang <[email-masked]> wrote:
>
> Hi Jensen,
>
> all pending updates), and hence, the semantics is that although the
> client gets a "remove" success, updates may continue to arrive; hence the
> client code needs to be careful to handle the issue. It is not clear when
> the client can be notified that the buffer is flushed indeed.
>

---

From: Dawn Chan <[email-masked]>
Date: Fri, 9 Mar 2018 11:31:25 +0000

Hi Richard,

Sorry for the late reply. I do not think “stream-id” is a good name to distinguish the request for different resources in one update stream. I failed to find a better name at the moment. Do other members have a better one?

Dawn
On 5 Mar 2018, at 10:23 AM, Y. Richard Yang <[email-masked]<mailto:[email-masked]>> wrote:

Please see below.

On Sat, Mar 3, 2018 at 7:57 AM, Dawn Chan <[email-masked]<mailto:[email-masked]>> wrote:
Hi Jensen and Richard,

It is clear that using both, as it is in the current design, is an option. Make sense to others in the WG?

Richard

Dawn

Hi Jensen,

Richard

Thanks,
Jensen