Subject: Re: [alto] review of draft-ietf-alto-incr-update-sse-02

From: Jan Seedorf <[email-masked]>
Date: Fri, 2 Sep 2016 12:24:03 -0700

Dear authors of the ALTO SSE draft,

can you please look at Sebastian's comments (see below) and address them 
in a new version of the draft. Then, we can move the draft ahead (WGLC 
done, so next steps would be shepherd write-up and send out of the WG).

@Sebastian: thanks a lot for taking the time and the good review/comments!

Thanks,

Jan

On 09/08/16 14:31, Sebastian Kiesel wrote:
> Hi Wendy and Richard, all,
>
> during the Berlin meeting our valued chairman have invited me
> to do one more review of draft-ietf-alto-incr-update-sse-02,
> so here it is:
>
> 1. One general question
>
> this document tries to make distribution of updates to ALTO information
> more efficient, in particular when network/cost maps are large and
> change frequently.  It leverages two separate mechanisms to do so:
> a) an efficient encoding for smaller updates to JSON structures
> (e.g. ALTO network/cost maps) and b) server-initiated notifications as
> opposed to the client frequently asking for updates.
>
> For a), the document argues that JSON Merge Patch [RFC7386] is clearly
> better suited than other similar mechanisms such as JSON Patch
> [RFC6902]. I am not an expert in this field and just belive that this
> assessment is correct.  Furthermore, a client can de-facto disable this
> mechanism by setting "incremental-updates" to false.
>
> So far, so good.
>
> For b), the document compares "SSE over HTTP 1.1" [W3C SSE Rec.] and
> "HTTP 2 Server-Push" [RFC7540].  It argues that the second method is
> technically superior, yet still specifies usage of the first one, in
> order to allow faster deployment, as HTTP/2 libraries are not widely
> available as of today.  This seems to be a bit questionable to me, given
> that getting this document published as an RFC will need at least
> another half a year, but then it will live forever.  What is the
> timeframe we expect HTTP/2 implementations?
>
> But my main question is: could this mechanism be made optional, similar
> to JSON merge patch?  I'm thinking of a client-initiated request
> (GET/POST), something along the lines "give me the diff for the .. map
> since tag ... (if the diff is larger than the current map or the server
> has not enough history information or CPU power to create the diff, send
> the full map instead)".
> I am not really convinced that in every scenario sending a keepalive
> message every 15 seconds is more efficient than the client asking for
> updates in similar time intervals.  Maybe this client-initiated request
> with JSON merge patch can be done, maybe I overlooked something?  Adding
> some lines of discussion may be worth it.
>
>
>
> 2. Smaller issues and nits
>
>
> Section 2 (and similar wording in 7.5):
>
>      The server selects the resource set for each stream,
>      although it is recommended that the set be closed under the ALTO
>      resource dependency relationship (i.e., the "uses" relationship).
>
> This sentence is rather difficult to understand when reading the first
> time.  It made sense to me only after reading the following sentence.
>
> Maybe reword:
>
>      The server selects the resource set for each stream. It is
>      RECOMMENDED that if a resource depends on one or more other
>      resource(s) (indicated with the "uses" attribute defined in RFC7285),
>      these other resource(s) should also be part of that stream.
>
> ****
>
> General: replace references to RFC 2616 (which is now obsolete) with
> references to its successors, 723[0-5]
>
> ****
>
> Sec 4:
>
>      Messages are delimited by two new-lines (this is a slight
>      simplification; see [SSE] for details).
>
>
> The "slight" sounds dangerous to me - an invitation not to read the
> referenced document?
>
> Maybe better:
>
>      In the following illustration, messages are delimited by two
>      new-lines.  Please refer to [SSE] for the full specification
>      of message delimitation.
>
> ****
>
> Sec 7 (Paragraphs before 7.1):
>
> The abundance of "[Update] Stream [Control] (Service|resource)" is
> difficult to read. I have the feeling that sometimes words were omitted,
> which makes it even more difficult to read.
>
> Maybe. did I get this right?
>
>     An Update Stream Service returns an Update Stream, i.e., a stream
>     of SSE messages, as defined
>     in Section 6.  An Update Stream resource is used to request a new
>     Update Stream.
>
>     A server creates an Update Stream Control resource for each active
>     Update Stream.  A client uses the Update Stream Control resource to
>     request that the corresponding Update Stream should no-longer
>     convey updates for a specifc resource,
>     or to request updates for
>     additional resources.  Update Stream resources are listed in the
>     server's IRD, but Stream Control resources are not.  Instead, the
>     first event that the server sends to the client has the URI for the
>     Update Stream Control resource for that stream (see Section 6.3.
>
> ****
>
> Sec 7.3:
>
>      If ... the client MAY set the "tag" field to "tag" part of the
>      resource's version tag.
>
> This sentence is difficult to read. Do the "" around the second "tag"
> make sense?  Is a "the" missing?
>
>
>     If that version is still current, the ALTO
>     Server SHOULD omit sending a full replacement update at the start of
>     the stream (see Section 7.6.2).
>
> Could you explicitly state what it should do (send?) instead? Just nothing?
>
>     If that version is not current, the server MUST ignore the "tag" field.
>
> Could you explicitly state what it should do (send?) after ignoring?
> Just nothing? The full map?
>
> ****
>
> Sec 7.3:
>
>     ... SHOULD send the full-replacement message soon after the change ...
>
> If I remember correctly we had a discussion on this list some years
> ago whether RFC2119 language is adequate only for behavior that
> can be blackbox-tested, or also for behavior that needs a whitebox test
> to validate. I think we agreed that both is ok.
> But "SHOULD" in conjunction with "soon" seems to me like going
> a step too far.
>
> ****
>
> Sections 8.2 and 8.6
>
>     If a request is successful, the server returns an HTTP "200
>     OK" response with Content-Type "text/plain" and no data.
>
> is this good practice?  Maybe better "204 No Content" ?
>
> ****
>
> sec 12.3
>
>     Hence when sending a full network map or cost map, an ALTO
>     server SHOULD insert a new-line character periodically.
>     Approximately every 2,000 characters should be sufficient for most
>     SSE clients.
>
> an uppercase SHOULD in conjunction with a very vague lowercase should?
>
> Can this be expressed more specific?
>
>     Hence an ALTO server SHOULD NOT send lines longer than 2000 characters.
>     This can be achieved by adding new-line characters periodically.
>
> Or is this someone else's problem and has been discussed and specified
> elsewhere, so we can just add a pointer to that document?
>
>
> ****
>
> sec 13.2
>
>     An outside party which can read the update stream response can obtain
>     the control URI and use that to send a fraudulent "remove" requests,
>     thus disabling updates for the valid client.  This can be avoided by
>     encrypting the stream (see Section 15 of [RFC7285]).
>
> This remedy implies that the server must use control URIs with enough
> randomness, so an attacker cannot guess them. Should that be added in 8.1?
> Or can we find a better mechanism to actually authenticate and autorize
> requests to the control URI?
>
> ****
>
>
>    -- Sebastian
>
> _______________________________________________
> alto mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/alto

-- 
****************************
Prof. Dr. Jan Seedorf
[email-masked]
****************************
Hochschule für Technik Stuttgart
Fakultät Vermessung, Informatik und Mathematik
Schellingstr. 24
D-70174 Stuttgart
www.hft-stuttgart.de
****************************

---

From: Wendy Roome <[email-masked]>
Date: Mon, 05 Sep 2016 16:16:37 -0400

Sebastian,

Thanks reviewing the document. And thanks for keeping us honest! I will
address your major comment first, and then the minor ones in a later post.

***** SSE vs HTTP/22 Server Push:

Yes, by the time this is published, HTTP/2 support will be more common.
Although not universal. For example, Java still does not support HTTP/2 as
a client, and will not until sometime in 2017.

But comparing the two approaches, the primary advantage of HTTP/2 is that
it is officially blessed by the IETF. SSE will still be around for a long
time.

And although HTTP/2 does allow a server to push unsolicited ages to a
client, using it for ALTO updates is a bit of a kludge. HTTP/2 server push
is designed for get-mode, static documents, like common images, css files,
etc. In effect the server tells the client,

	Here is the document with url FOO.
	Add it to your cache.
	You *will* need it!!

To use that for updates, the server will have to encode in the url FOO the
information that this is an incremental update to a previously retrieved
resource. That is possible, but it is not clean, and does not fit the
HTTP/2 model. Furthermore, a client HTTP/2 library might hide the
server-push from the client by saving the pushed document in a cache and
delivering it to the client only when the client requests that url.

And in addition, HTTP/2 is a very complicated protocol, which will
discourage embedded ALTO clients from using it.

So given that, I think SSE is better, even after HTTP/2 becomes popular.

BTW, I do not see HTTP/2 replacing HTTP/1.1. I think both will co-exist.
Simple web sites will use continue to use http/1.1; only complex,
heavy-load web sites will use http/2.

***** Could this be made optional? As in, give me the diff for the .. map
since tag ...

Alas, that only works for maps with tags. Eg, that would work with Network
Maps. But it would not work for Cost Maps, or Endpoint Cost requests, or
Endpoint Property requests, etc.

***** I am not really convinced that in every scenario sending a keep
alive message every 15 seconds is more efficient than the client asking
for updates in similar time intervals.

Keep-alives are much more efficient, because they just send 10 more bytes
on an existing TCP connection. Eg, one message. A request requires setting
up a new TCP connection. That takes several messages, involved allocating
new ports, etc. It is a lot more overhead on both the client and the
server.

	- Wendy Roome

---

From: Sebastian Kiesel <[email-masked]>
Date: Tue, 6 Sep 2016 20:15:24 +0200

Hi Wendy,

please see below...

> heavy-load web sites will use http/2.

This all sounds plausible to me.  So, no need to revise your decisions,
but maybe add some words to the draft.

> Endpoint Property requests, etc.

Could we use this draft to introduce a tag to the cost map, for the
purpose of requesting diffs?

Then, the mechanism would work with Network Map and Cost Map.
These two are probably the most important targets for periodic
incremental updates.

> new ports, etc. It is a lot more overhead on both the client and the

You are right, opening a new TCP connection every 15 seconds is less
efficient than sending 10 bytes every 15 seconds over an existing
connection,  But there may be a break-even point ... What if I want the
updates only every 120 seconds?  Is one new TCP connection more
effort than 8 keep-alives? Or 16?

Maybe it is not worth the effort of changing the spec and adding some
explanation is the better approach, but the current document leaves some
feeling of strange asymmetry in me - one mechanism optional, the other
one mandatory without a good explanation why.

Sebastian

---

From: Wendy Roome <[email-masked]>
Date: Tue, 06 Sep 2016 15:37:45 -0400

Sebastian,

Comments in-line.

	- Wendy

On 09/06/2016, 14:15, "Sebastian Kiesel" <[email-masked]> wrote:

>> ...

In progress. (Great minds think alike! :-))

>These two are probably the most important targets for periodic

Tagging cost maps might be a good idea, but there are some possible
problems, and I think that needs a separate draft. Do you want to do
submit one? Or formally solicit comments?

>one mandatory without a good explanation why.

The keep-alive time is in the SSE specification:

Legacy proxy servers are known to, in certain cases, drop HTTP connections
after a short timeout. To protect against such proxy servers, authors can
include a comment line (one starting with a ':' character) every 15
seconds or so.

---

From: Sebastian Kiesel <[email-masked]>
Date: Thu, 8 Sep 2016 13:31:32 +0200

Hi Wendy,

> >On Mon, Sep 05, 2016 at 04:16:37PM -0400, Wendy Roome wrote:
... SSE ...
> submit one? Or formally solicit comments?

Do you see further use cases for such a tag?

> include a comment line (one starting with a ':' character) every 15

yes, the 15 seconds is in the SSE spec.

that means, if I need updates every X seconds, I will either recveive
X/15sec keep-alives per update period (and get the updates as the server
generates them), or I will have to establish one TCP connection per 
update period (and explicitly ask the server for updates).

With growing X, there will be a break-even point where the second
approach will become more efficient.  The question is, if we are
beyond that point, could there be cases where JSON merge patch
still makes sense?

Thanks
Sebastian

---

From: Wendy Roome <[email-masked]>
Date: Thu, 08 Sep 2016 11:30:15 -0400

Sebastian,

The alternate approach -- client sends a new ALTO request to get the
updates -- has more overhead than simply creating a new TCP stream.

First, the client must tell the server which version the client has.
That¹s easy for tagged resources, like network maps. But for cost maps
(etc) we would have to add a tag or some other version ID.

Second, the server would have to maintain a log of all changes, just in
case a client asks for updates relative to some previously returned
version. And the server would have to maintain that log even if no client
ever asks for an incremental update.

Third, with that approach, it would be very difficult to provide updates
for POST-mode requests.

Of course the server could delete the old tail of the log, and if a client
wants updates, the client could set a flag in the initial request warning
the server.

For comparison, with the SSE approach, the server does not need a change
log. Instead, for each resource, the server keeps a list of active update
streams for that resource. Whenever the resource changes, the server just
writes the update to those streams. In effect, the network TCP buffers
become the log. And there is zero extra overhead when there are no update
streams.

Incidentally, my first approach to incremental updates was very much like
that: the server returned a time-stamp for resources for which a client
could return updates, and clients polled for updates. Once I discovered
SSE, I gave up on that approach.

	- Wendy