Subject: [Netconf] activate-configured-subecription - WGLC subscribed-notifications-16

From: Martin Bjorklund <[email-masked]>
Date: Wed, 12 Sep 2018 14:01:09 +0200 (CEST)

Hi,

I forgot one thing in my review, that has been discussed (slightly
off-topic) in the thread "netconf call home connection type".

The SN draft (implicitly) assumes that for dynamic subscriptions,
after a successful "establish-subscription" operation, notification
messages can start to flow to the client.

The exact mechanism for this is transport-protocol-dependent.

For NETCONF, the netconf-notif draft will need to update RFC 6241,
since 6241 refers to 5277 which says that a <notification> message can
only be sent after a successful "create-subscription".  netconf-notif
updates this to allow <notification> messages to also be sent after a
succsessful "establish-subscription".

For RESTCONF, the restconf-notif draft defines a new mechanism for how
notifications are delivered; instead of locating the SSE resource url
from the "streams" in "ietf-restconf-monitoring", this draft now
augments a location leaf to the output of "establish-subscription".
(It probably doesn't have to update 8040, but *maybe* the mechanism in
8040 ought to be deprecated)

For configured subscriptions however, the SN draft doesn't specify how
the flow of notifications should start.  One view is that this is
transport-protocol-specific.  However, I think that for all RPC-based
transport-protocols (currently NETCONF and RESTCONF) it makes sense to
have a common way to get the notifications to be sent.  Hence I
suggest that this draft adds a new rpc operation
"activate-configured-subscription" (better name anyone?).  This rpc
would be invoked by the client after a call home to start the flow of
configured subscriptions.  If there are no such subscriptions or the
session is not a call home session and error would be returned.

We might define non-RPC-based transport protocols for configured
subscriptions in the future.  Such a protocol can obviously not use
this rpc, but OTOH it can't use establish-subscription either.  I
think this is ok.

With such an RPC, netconf-notif would udate 6241 to say that
<notification> messages can be sent after a succsessful
"establish-subscription" or "activate-configured-subscription" rpc.

And with such an RPC, restconf-notif can support configured
subscriptions over proper RESTCONF; it would augment a "location" leaf
to the output of activate-configured-subscription, which would contain
the URL to the event flow, so that it works just like dynamic
subscriptions.

If we don't add this RPC to the SN draft, each protocol draft will
have to define its own version of this operation.

/martin

---

From: "Eric Voit (evoit)" <[email-masked]>
Date: Thu, 13 Sep 2018 20:46:07 +0000

Hi Martin,

> From: Martin Bjorklund, September 12, 2018 8:01 AM
> 
> Hi,
> 
> The SN draft (implicitly) assumes that for dynamic subscriptions, after a
> successful "establish-subscription" operation, notification messages can start
> to flow to the client.
> 
> For NETCONF, the netconf-notif draft will need to update RFC 6241, since 6241
> refers to 5277 which says that a <notification> message can only be sent after a
> successful "create-subscription".  netconf-notif updates this to allow
> <notification> messages to also be sent after a succsessful "establish-
> subscription".

Thanks.  I have included this as a note to the RFC Editor within NETCONF-Notif.

> notifications are delivered; instead of locating the SSE resource url from the
> "streams" in "ietf-restconf-monitoring", this draft now augments a location leaf
> to the output of "establish-subscription".
> For configured subscriptions however, the SN draft doesn't specify how the
> flow of notifications should start.  One view is that this is transport-protocol-
> specific.

Yes, protocol specific is what has been assumed for the current draft set.  

> However, I think that for all RPC-based transport-protocols (currently
> NETCONF and RESTCONF) it makes sense to have a common way to get the
> notifications to be sent. 

Perhaps something will be needed for NETCONF.  But for configured RESTCONF subscriptions, RESTCONF-Notif has been assuming that RESTCONF not be used.  Instead raw HTTP2 with the Publisher being the HTTP client and the receiver the HTTP server seems cleaner.  And the HTTP server need not respond with an 'ok' to a "subscription-started" unless it is ready to accept event records.  So I think there are open questions on the best way to do HTTP configured transport.

> Hence I suggest that this draft adds a new rpc
> operation "activate-configured-subscription" (better name anyone?). This rpc
> would be invoked by the client after a call home to start the flow of configured
> subscriptions.  If there are no such subscriptions or the session is not a call
> home session and error would be returned.
>
> subscriptions in the future.  Such a protocol can obviously not use this rpc, but
> OTOH it can't use establish-subscription either.  I think this is ok.

Perhaps this could be useful.  But it would be great wherever possible if we can avoid receiver RPCs as a mandatory element of the establishment flow with a configured publisher.  So I believe we should investigate further on this for NETCONF configured in the coming months.   If it does turn out useful, I would like to see more than NETCONF potentially make use of the RPC before generalizing to the SN level.
 
> With such an RPC, netconf-notif would udate 6241 to say that <notification>
> messages can be sent after a succsessful "establish-subscription" or "activate-
> configured-subscription" rpc.
> 
> And with such an RPC, restconf-notif can support configured subscriptions over
> proper RESTCONF; it would augment a "location" leaf to the output of activate-
> configured-subscription, which would contain the URL to the event flow, so
> that it works just like dynamic subscriptions.
>
> If we don't add this RPC to the SN draft, each protocol draft will have to define
> its own version of this operation.

Defining something local to a protocol might be ok until we get some clarity on the generality of this issue.

Eric

> 
> /martin
> 
> _______________________________________________
> Netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 14 Sep 2018 09:11:20 +0200 (CEST)

"Eric Voit (evoit)" <[email-masked]> wrote:
> > after a
> > since 6241
> > from the
> > the
> Perhaps something will be needed for NETCONF.  But for configured
> RESTCONF subscriptions, RESTCONF-Notif has been assuming that RESTCONF
> not be used.  Instead raw HTTP2 with the Publisher being the HTTP
> client and the receiver the HTTP server seems cleaner.

This has been pointed out before - the restconf-notif draft should
define *RESTCONF* mechanisms for subscribed notifications.

If you want to define a new "notif-only" protocol that uses HTTP/2,
that should be done in a separate document.

Also, with the addition of a "activate-configured-subecription", we
can use existing RESTCONF mechanisms to support configured
subscriptions; it more or less falls out for free, as I explained.

> And the HTTP
> server need not respond with an 'ok' to a "subscription-started"
> unless it is ready to accept event records.  So I think there are open
> questions on the best way to do HTTP configured transport.

Maybe another reason to do this in a separate document.

> > configured
> > this rpc, but
> Perhaps this could be useful.  But it would be great wherever possible
> if we can avoid receiver RPCs as a mandatory element of the
> establishment flow with a configured publisher.

The idea is that the "reciever RPC" is NOT mandatory; if a transport
needs such an RPC in order to start the notif flow, the transport
document will specifiy that the "activate-configured-subscription" rpc
is used for that transport.

If a new "notif-only" protocol is defined that doesn't utilize this
rpc, the transport spec for that protocol will not use it.

> So I believe we
> should investigate further on this for NETCONF configured in the
> coming months.  If it does turn out useful, I would like to see more
> than NETCONF potentially make use of the RPC before generalizing to
> the SN level.

I was hoping we would be able to get configured subscriptions to work,
for NETCONF and RESTCONF, sooner rather than later.

/martin

> > <notification>
> > subscriptions over

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 14 Sep 2018 09:14:45 +0200

> that should be done in a separate document.

+1

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <https://www.jacobs-university.de/>

---

From: "Eric Voit (evoit)" <[email-masked]>
Date: Fri, 14 Sep 2018 12:47:19 +0000

> > > > However, I think that for all RPC-based transport-protocols
> > > > (currently NETCONF and RESTCONF) it makes sense to have a common
> > > > way to get the notifications to be sent.
> > >
> > > RESTCONF subscriptions, RESTCONF-Notif has been assuming that
> > > RESTCONF not be used.  Instead raw HTTP2 with the Publisher being
> > > the HTTP client and the receiver the HTTP server seems cleaner.
> >

You will see HTTP2 configured will be removed in the next version of the document which Reshad will be posting shortly.

Eric

---

From: "Reshad Rahman (rrahman)" <[email-masked]>
Date: Fri, 14 Sep 2018 15:35:24 +0000

Eric

<RR> https://www.ietf.org/id/draft-ietf-netconf-restconf-notif-07.txt has dynamic subscriptions only.

Regards,
Reshad.

---

From: Andy Bierman <[email-masked]>
Date: Mon, 17 Sep 2018 09:23:24 -0700

On Fri, Sep 14, 2018 at 12:14 AM, Juergen Schoenwaelder <
[email-masked]> wrote:

> (currently

So how should this be fixed?
It seems this draft could be renamed (e.g., s/RESTCONF/HTTP2/) since it
does not use RESTCONF in any way.  I don't think RESTCONF notifications
are used very much, but I don't see why the same SSE delivery of the
<notification> element
would not work (e.g., different resource to GET for establish-subscription
parameters)

The WG seems to refactor drafts a lot, only to end up with even more
unfinished work
than before.  We should learn to drop work that is not backed with lots of
active support
and reviews.

> /js

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 17 Sep 2018 19:25:53 +0200 (CEST)

> would not work (e.g., different resource to GET for establish-subscription

Actually, the most recent version is more RESTCONF specific, and it
does in fact use SSE for notification delivery.  But there are some
issues... I'll start a new thread for this.

/martin

---

From: Douglas Hubler <[email-masked]>
Date: Tue, 18 Sep 2018 14:56:53 -0400

On Mon, Sep 17, 2018 at 12:23 PM Andy Bierman <[email-masked]> wrote:

> are used very much,

I make liberal use of them, it's become one of my biggest selling point of
RESTCONF over rolling your own REST API for management.

> but I don't see why the same SSE delivery of the <notification> element

I avoided implementing SSEs over HTTP/1.1  *because* I used a lot of
notifications and separate session for each subscription obviously doesn't
scale.  I went w/websockets for now with the idea I'd reimplement when a
reasonable draft is assembled.  I had hopes for SSE over HTTP/2 because
seems rather straightforward and in theory should scale.  Obviously with
websockets, my implementation doesn't interop w/other RESTCONF/NETCONF
systems/libs but hoping to change that with this draft.

I don't understand the details surrounding using an RPC to make a dynamic
subscription.  Unless it's optional, then I can start to understand.

Douglas
FreeConf.org

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 18 Sep 2018 21:40:22 +0200 (CEST)

> systems/libs but hoping to change that with this draft.

So it seems you support my point to make this draft use SSE.  Then if
you use HTTP/2 you (the client) can simply create one HTTP/2 stream per
notification stream.  This just falls out for free with HTTP/2.

> subscription.  Unless it's optional, then I can start to understand.

The client would POST to the "rpc" establish-subscripion, and in the
reply to the POST is a "location" leaf (*) that contains a URI.  The
client then GET this URI; this is the SSE event stream.

(*) in RFC 8040 it is called "location", in this draft it is called
"uri" 

/martin

---

From: Douglas Hubler <[email-masked]>
Date: Tue, 18 Sep 2018 17:35:21 -0400

> notification stream.  This just falls out for free with HTTP/2.

yes, with caveat that I haven't attempted actual implementation yet.

> client then GET this URI; this is the SSE event stream.

I see.  But if a client reads the YANG as

 module my-module {
    ...
    notification my-event {
    }
}

and client knew base stream URI was:

   /restconf/my-streams

Couldn't the client then deduce SSE URL as:

 GET  /restconf/my-streams/my-module:my-event?...

Thereby saving a trip to server?  Would it be wrong for a client to make
any assumptions?  I realize this ASSUMEs a transport protocol, but
could/would there be a one-time, capability check the client could make to
test for available transports?

I really appreciate your time explaining, I have been following the emails
best I could but I think some basic things are still eluding me.

---

From: "Eric Voit (evoit)" <[email-masked]>
Date: Tue, 18 Sep 2018 22:43:39 +0000

Hi Douglas,

From: Douglas Hubler, September 18, 2018 5:35 PM

On Tue, Sep 18, 2018 at 3:40 PM Martin Bjorklund <[email-masked]<mailto:[email-masked]>> wrote:
So it seems you support my point to make this draft use SSE.  Then if
you use HTTP/2 you (the client) can simply create one HTTP/2 stream per
notification stream.  This just falls out for free with HTTP/2.

<Eric> Agree that SSE over HTTP2 should work.  However to me using SSE here seems redundant as HTTP2 provides the necessary mechanisms.  But I won’t get in the way if people really do want to go in this direction.

 module my-module {
    ...
    notification my-event {
    }
}

   /restconf/my-streams

Thereby saving a trip to server?

<Eric> Yes, it would be possible to set something up to deduce the URL.  draft-ietf-netconf-restconf-notif doesn’t currently attempt this as being able to deduce the URL doesn’t stop the extra trip to the server.

The reason it doesn’t get rid of the extra round trip has to do with HTTP2 streams.  If we are going to use HTTP2, there is still a need to initiate the GET (or POST) in a separate stream other than the one being used for RESTCONF signaling.

Someone could write a specification which gets around this by forcing every RESTCONF transaction on its own HTTP2 stream.  But nobody has proposed such a specification.  Plus there are QoS scenarios where you might want multiple subscriptions on the same HTTP2 stream.  So if someone wanted this, we would still need to refine draft-ietf-netconf-restconf-notif requirements with the ability to modify or delete an existing subscription on a different HTTP2 stream.  This could be done of course, but someone would need to drive the definition.

Eric

Would it be wrong for a client to make any assumptions?  I realize this ASSUMEs a transport protocol, but could/would there be a one-time, capability check the client could make to test for available transports?

I really appreciate your time explaining, I have been following the emails best I could but I think some basic things are still eluding me.

---

From: Qin Wu <[email-masked]>
Date: Wed, 19 Sep 2018 03:06:45 +0000

发件人: Netconf [mailto:[email-masked]] 代表 Eric Voit (evoit)
发送时间: 2018年9月19日 6:44
收件人: Douglas Hubler; Martin Bjorklund
抄送: [email-masked]
主题: Re: [Netconf] activate-configured-subecription - WGLC subscribed-notifications-16

Hi Douglas,

[Qin]: HTTP2 provides Server Push mechanism however there are lots of drawbacks, typical one is Server Push may be blocked, therefore HTTP1.1+ SSE is favored.

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 19 Sep 2018 08:38:05 +0200 (CEST)

Hi,

"Eric Voit (evoit)" <[email-masked]> wrote:
> On Tue, Sep 18, 2018 at 3:40 PM Martin Bjorklund
> <[email-masked]<mailto:[email-masked]>> wrote:
> <Eric> Agree that SSE over HTTP2 should work.  However to me using SSE
> here seems redundant as HTTP2 provides the necessary mechanisms.

Eric, can you explain how this would work w/o SSE?  Which mechanism in
HTTP/2 are you referring to?

> But
> I won’t get in the way if people really do want to go in this
> direction.
> 

No.

With RFC 8040, the SSE URL is "stable", and if the client has read it
onve, it doesn't have to read it again (modulo some definition of
"stable").  The client can pass a filter in the GET request for the
SSE stream.

With the mechanism in this draft, the SSE URL will vary for each
request that has a different filter (at a minimum), since the filter
is passed a parameter to "establish-subscription" and the SSE URL is
sent back in the reply to "establish-subscription".  With the SSE URL,
the server may hide some state that corresponds to the filter params
etc.  (NOTE the word *may* in the previous sentence.  A server could
as well send back a SSE URL with the filter parameters embedded in the
URL...)

So in general, a client cannot assume that it can re-use a SSE URL
that was returned from a call to "establish-subscription".

> <Eric> Yes, it would be possible to set something up to deduce the
> URL.  draft-ietf-netconf-restconf-notif doesn’t currently attempt this
> as being able to deduce the URL doesn’t stop the extra trip to the
> server.
> 
> The reason it doesn’t get rid of the extra round trip has to do with
> HTTP2 streams.  If we are going to use HTTP2, there is still a need to
> initiate the GET (or POST) in a separate stream other than the one
> being used for RESTCONF signaling.

I disagree.  If the client wants to have a separate HTTP/2 stream for
the notifs, then it should start such a stream.  If the client doesn't
want a separate HTTP/2 stream then that should be ok too.

Compare to NETCONF.  HTTP/2 streams are similar to SSH channels.  A
NETCONF client can open a separate SSH channel for each subscription,
or it can use a single channel.  Up to the client.

> Someone could write a specification which gets around this by forcing
> every RESTCONF transaction on its own HTTP2 stream.  But nobody has
> proposed such a specification.  Plus there are QoS scenarios where you
> might want multiple subscriptions on the same HTTP2 stream.

How would this work?  Once you start SSE on a stream, that stream is
"blocked", right?

> So if
> someone wanted this, we would still need to refine
> draft-ietf-netconf-restconf-notif requirements with the ability to
> modify or delete an existing subscription on a different HTTP2 stream.

Isn't this how it is supposed to work?  The draft is silent on how to
delete a subscription, and also on how modify-subscription is supposed
to work.

The draft is also silent on the lifetime of the result of
"establish-subscription".  With NETCONF, the subscription ends if the
session ends, but in RESTCONF there are no sessions, so how long does
the subscription live after it has been created?

Note that the approach in RFC 8040 is doesn't have this issue.

/martin

> This could be done of course, but someone would need to drive the
> definition.
> 
> Eric
> 
> 
> Would it be wrong for a client to make any assumptions?  I realize
> this ASSUMEs a transport protocol, but could/would there be a
> one-time, capability check the client could make to test for available
> transports?
> 
> I really appreciate your time explaining, I have been following the
> emails best I could but I think some basic things are still eluding
> me.
>

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 19 Sep 2018 09:25:22 +0200 (CEST)

Qin Wu <[email-masked]> wrote:
> here seems redundant as HTTP2 provides the necessary mechanisms.  But
> [Qin]: HTTP2 provides Server Push mechanism however there are lots of
> drawbacks, typical one is Server Push may be blocked, therefore
> HTTP1.1+ SSE is favored.

But HTTP/2 server push doesn't even address the same problem; how to
get a stream of notifs sent from the server to the client.  If I'm not
mistaken, server push could be done w/ the GET of the SSE stream,
so maybe server push could be used to avoid the round trip:

    client                              server
    ------                              ------
      |      establish-subscription    
      |      ------------------------>    |
      |                                   |
      |         OK, location=SSE URL      |
      |      <------------------------    |
      |         PUSH_PROMISE,             |
      |           new stream, SSE URL     |
      |      <------------------------    |
      |           SSE event (notif)       |
      |      <------------------------    |
      |           SSE event (notif)       |
      |      <------------------------    |
      |                                   |

/martin

---

From: Qin Wu <[email-masked]>
Date: Wed, 19 Sep 2018 08:36:46 +0000

-----邮件原件-----
发件人: Martin Bjorklund [mailto:[email-masked]] 
发送时间: 2018年9月19日 15:25
收件人: Qin Wu
抄送: evoit=[email-masked]; [email-masked]; [email-masked]
主题: Re: [Netconf] activate-configured-subecription - WGLC subscribed-notifications-16

Qin Wu <[email-masked]> wrote:
> you use HTTP/2 you (the client) can simply create one HTTP/2 stream 
> per notification stream.  This just falls out for free with HTTP/2.
> 

But HTTP/2 server push doesn't even address the same problem; how to get a stream of notifs sent from the server to the client.  If I'm not mistaken, server push could be done w/ the GET of the SSE stream, so maybe server push could be used to avoid the round trip:

    client                              server
    ------                              ------
      |      establish-subscription    
            
[Qin]: Yes, HTTP/2 server push can save round trip, but if HTTP/2 server push is used, two channels need to be established, one for establish-subscription,
One for HTTP/2 Server Push, not sure the sequence of work flow here is correct, first how does establish-subscription RPC indicate server push support, how does server know server push should be used for response.
Second, SSE event and OK, PUSH-PROMISE should be established in another channel or HTTP connection?

HTTP server push has two typical drawbacks:
1. HTTP/2 does not guarantee that pushed resources are delivered
   to the client in the order they were sent by the client
2.Server Push is optional and can be disabled by any proxy between the client and the server.
Therefore HTTP/2 Server Push is not desirable comparing with HTTP1.1+SSE

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 19 Sep 2018 11:05:41 +0200 (CEST)

Qin Wu <[email-masked]> wrote:
> mistaken, server push could be done w/ the GET of the SSE stream, so
> maybe server push could be used to avoid the round trip:
> 
>     client                              server
>     ------                              ------
>       |      establish-subscription    
> [Qin]: Yes, HTTP/2 server push can save round trip, but if HTTP/2
> server push is used, two channels need to be established, one for
> establish-subscription,
> One for HTTP/2 Server Push, not sure the sequence of work flow here is
> correct

In the diagram I wrote "PUSH_PROMISE new stream" which was supposed to
indicate that the PUSH (and subsequent data) are sent in a new stream.

> , first how does establish-subscription RPC indicate server
> push support, how does server know server push should be used for
> response.

This is not specific to this draft, it is a generic HTTP/2 question,
right?  If I'm not mistaken this is taken care of by the HTTP/2 spec.

> another channel or HTTP connection?

Yes, another stream, see above.

> 2.Server Push is optional and can be disabled by any proxy between the

Again, this is a generic HTTP/2 PUSH issue.  If the client doesn't
recveive the PUSH_PROMISE, it would have to send a GET as usual.

> Therefore HTTP/2 Server Push is not desirable comparing with

Note that the diagram above assumes HTTP/2 + SSE.  It just shows how
PUSH_PROMISE could be used with HTTP/2 + SSE.

/martin

---

From: Qin Wu <[email-masked]>
Date: Wed, 19 Sep 2018 09:15:32 +0000

> Therefore HTTP/2 Server Push is not desirable comparing with 

Note that the diagram above assumes HTTP/2 + SSE.  It just shows how PUSH_PROMISE could be used with HTTP/2 + SSE.

[Qin]: Maybe this is implementation specific issue, but how does the server know when to use Server Push for PUSH_PROMISE and when to use SSE.
I just SSE and Server PUSH are very similar. Enabling both mechanism coulds confuse the server.

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 19 Sep 2018 11:21:10 +0200

> I just SSE and Server PUSH are very similar. Enabling both mechanism coulds confuse the server.

HTTP/2 push is a mechanism for a server to push data into a cache in
the hope that something using the cache may find the data useful. For
streaming event notifications, you can't rely on HTTP/2 push. I think
Martin is right.

/js

---

From: Qin Wu <[email-masked]>
Date: Wed, 19 Sep 2018 09:52:53 +0000

Good, thanks for clarification from both.

-Qin
-----邮件原件-----
发件人: Juergen Schoenwaelder [mailto:[email-masked]] 
发送时间: 2018年9月19日 17:21
收件人: Qin Wu
抄送: Martin Bjorklund; evoit=[email-masked]; [email-masked]
主题: Re: [Netconf] activate-configured-subecription - WGLC subscribed-notifications-16

> I just SSE and Server PUSH are very similar. Enabling both mechanism coulds confuse the server.

HTTP/2 push is a mechanism for a server to push data into a cache in the hope that something using the cache may find the data useful. For streaming event notifications, you can't rely on HTTP/2 push. I think Martin is right.

/js

---

From: "Eric Voit (evoit)" <[email-masked]>
Date: Wed, 19 Sep 2018 22:59:09 +0000

> From: Martin Bjorklund, September 19, 2018 2:38 AM
> 
> Hi,
> 

After an HTTP2 OK, the HTTP2 server has the ability to forward an unbounded number of HTTP2 Data Frames on the HTTP2 stream.  The stream is not considered complete until an END_STREAM is passed.   (Note this is independent of anything regarding HTTP2's PUSH_PROMISE.)

You can see an example flow in Figure 1 of draft-ietf-netconf-restconf-notif.  At IETF 97 we asked for some Feedback on this proposal. 
http://www.voit.org/IETF/Subscriptions-NETCONF-IETF97.ppt (see slide 14).
But I am not sure how much effort the WG should place here.   The work here is several years old, and the market has moved substantially since then.

> > > I don't understand the details surrounding using an RPC to make a
> > > dynamic subscription.  Unless it's optional, then I can start to
> > > understand.
> >
> With RFC 8040, the SSE URL is "stable", and if the client has read it onve, it
> doesn't have to read it again (modulo some definition of "stable").  The client
> can pass a filter in the GET request for the SSE stream.
> 
> With the mechanism in this draft, the SSE URL will vary for each request that
> has a different filter (at a minimum), since the filter is passed a parameter to
> "establish-subscription" and the SSE URL is sent back in the reply to "establish-
> subscription".  With the SSE URL, the server may hide some state that
> corresponds to the filter params etc.  (NOTE the word *may* in the previous
> sentence.  A server could as well send back a SSE URL with the filter
> parameters embedded in the
> URL...)
> 
> So in general, a client cannot assume that it can re-use a SSE URL that was
> returned from a call to "establish-subscription".
> 
> I disagree.  If the client wants to have a separate HTTP/2 stream for the notifs,
> then it should start such a stream.  If the client doesn't want a separate HTTP/2
> stream then that should be ok too.

Per your below, once SSE is applied a stream is blocked.  For that subscription, you still need to support 'modify-subscription' and 'delete-subscription' rpcs.

Beyond that, one thing to keep an eye on is the potential for bundling of event records from different subscriptions into a single transmittable notification message.  (Something like in draft-ietf-netconf-notification-messages.)   Anywhere such bundling occurs, the publisher will have to use a consistent HTTP2 stream across a set of subscriptions in order to preserve event record ordering.  In addition, all event records bundled will have to consistently use the same DSCP.

Note: this is another reason why it is a good idea to use separate HTTP2 streams for RESTCONF subscription RPC interactions, and for the streaming of event records.

> NETCONF client can open a separate SSH channel for each subscription, or it
> can use a single channel.  Up to the client.

HTTP2 streams have far less overhead, lower setup latency, and more scalability than separate SSH channels.
 
> How would this work?  Once you start SSE on a stream, that stream is

And per above, this is a good reason to separate RESTCONF subscription RPCs from the HTTP2 stream used for events.   (The way it could be done is have modify-subscription and delete-subscriptions come in on a different stream, impacting the HTTP2 stream where establish-subscription started things off.  Again I am not recommending this for multiple reasons.)
 
> Isn't this how it is supposed to work? 

Yes.  My point was that nobody has proposed refining the spec to forcing of each RESTCONF transaction on different HTTP2 streams.

> The draft is silent on how to delete a
> subscription, and also on how modify-subscription is supposed to work.

Certainly these can come in on a different HTTP2 stream.
 
> The draft is also silent on the lifetime of the result of "establish-subscription".
> With NETCONF, the subscription ends if the session ends, but in RESTCONF
> there are no sessions, so how long does the subscription live after it has been
> created?

There are mechanisms defined.  See 
draft-ietf-netconf-restconf-notif, Section 3.1. 
and
draft-ietf-netconf-subscribed-notifications, sections 1.3 & 5.3.

Eric

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 20 Sep 2018 09:20:11 +0200 (CEST)

Hi,

[subject change; this thread is now about SSE and HTTP/2]

"Eric Voit (evoit)" <[email-masked]> wrote:
> After an HTTP2 OK, the HTTP2 server has the ability to forward an
> unbounded number of HTTP2 Data Frames on the HTTP2 stream.

But it must still adhere to the normal HTTP semantics.  If there was a
request, it needs to send the response.  It can split the response
into several DATA frames of course.  But it cannot start to send
"other stuff" instead of the response, which figure 1 in
restconf-notif-07 possibly implies.

> The stream
> is not considered complete until an END_STREAM is passed.  (Note this
> is independent of anything regarding HTTP2's PUSH_PROMISE.)
> 
> You can see an example flow in Figure 1 of
> draft-ietf-netconf-restconf-notif.  At IETF 97 we asked for some
> Feedback on this proposal.
> But I am not sure how much effort the WG should place here.  The work
> here is several years old, and the market has moved substantially
> since then.

So do we agree that the restconf-notif draft will be updated to use
SSE regardless of HTTP version?  If there is any text at all about
HTTP/2, it will be an "implementation note" where it is recommended to
use a separate stream per subscription?

> > onve, it
> > request that
> > that was
> > the notifs,
> Per your below, once SSE is applied a stream is blocked.  For that
> subscription, you still need to support 'modify-subscription' and
> 'delete-subscription' rpcs.

But the solution cannot depend on the subscriber being able to send
"delete-subscription" on the same transport session (BTW, this
requirement should be clarified in the YANG definition for
"delete-subscription" in SN), since this would rule out HTTP/1.1.

"modify-subscription" can be sent on another session, e.g., another
HTTP/1.1 session that is not doing the SSEs.

This makes me wonder... is "delete-subscription" really necessary, or
is "kill-subscription" sufficient?

> Beyond that, one thing to keep an eye on is the potential for bundling
> of event records from different subscriptions into a single
> transmittable notification message.

How would an HTTP subscriber even request this?

The subscriber sends an establish-subscription and get an URL for that
subscription back.

Then it sends another establish-subscription and gets another URL
back.

How can a bundled message be sent when there are two different
subscription resources?

> (Something like in
> draft-ietf-netconf-notification-messages.)  Anywhere such bundling
> occurs, the publisher will have to use a consistent HTTP2 stream
> across a set of subscriptions in order to preserve event record
> ordering.  In addition, all event records bundled will have to
> consistently use the same DSCP.
> 
> Note: this is another reason why it is a good idea to use separate
> HTTP2 streams for RESTCONF subscription RPC interactions, and for the
> streaming of event records.
> 
> > or it

Really?  An SSH channel is super cheap.  Why do you think that HTTP/2
streams have "far less" overhead?  There can be 2^32 channels in an
SSH session, and 2^31 streams in a HTTP/2 session.

(For reference, in case it wasn't clear: RFC 4254, section 5)

> And per above, this is a good reason to separate RESTCONF subscription
> RPCs from the HTTP2 stream used for events.  (The way it could be done
> is have modify-subscription and delete-subscriptions come in on a
> different stream, impacting the HTTP2 stream where
> establish-subscription started things off.  Again I am not
> recommending this for multiple reasons.)
>  
> > > So if
> Yes.  My point was that nobody has proposed refining the spec to
> forcing of each RESTCONF transaction on different HTTP2 streams.
> 
> > "establish-subscription".

This section says:

   Where quick
   recognition of the loss of a publisher is required, a subscriber
   SHOULD connect over TLS [RFC5246], and use a TLS heartbeat [RFC6520]
   to track HTTP session continuity.  In the case where a TLS heartbeat
   is included, it should be sent just from receiver to publisher.  Loss
   of the heartbeat MUST result in any subscription related TCP sessions
   between those endpoints being torn down.  A subscriber can then
   attempt to re-establish.

This is not clear to me.

First it says that TLS SHOULD be used.  But with RESTCONF, TLS is
already a MUST.

Second, it says that in the case of missed TLS heartbeats, "related
TCP sessions" are torn down.  What are these sessions (plural), and
what exactly does "related" mean?

Third, it says that a subscriber can re-establish.  What can it
re-establish?  Re-send the establish-subscription?  Just resend the
POST to the subscription URL?

The flow is:

    Subscriber                                   Publisher
    ----------                                   ---------

       | RESTCONF POST (RPC:establish-subscription)   |
       |--------------------------------------------->|
       |                          HTTP 200 OK (ID,URI)|
       |<---------------------------------------------|
       |                                              
     (***)
       |   HTTP POST (URI)
       |    |--------------------------------------------->|
       |    |                                   HTTP 200 OK|

After the publisher has sent a URI for the subscription to the
subscriber, there is a delay (***) until the subscriber has sent the
new POST.  The resource that the URI points to is subscriber-specific,
which means that the publisher may have created some associated
state.  This state will eventually have to be destroyed.  When can the
publisher safely destroy this state?

For example, what happens if the new POST never arrives?  If the
subscriber loses its transport session, can it send a new POST to the
same URL?  Can it send multiple POSTs to the same URL?

> draft-ietf-netconf-subscribed-notifications, sections 1.3 & 5.3.

I didn't find the answers to the questions above in these sections.

/martin

---

From: "Eric Voit (evoit)" <[email-masked]>
Date: Thu, 20 Sep 2018 20:06:14 +0000

Hi,

> From: Martin Bjorklund, September 20, 2018 3:20 AM
> 
> Hi,
> 
> > > > So it seems you support my point to make this draft use SSE.  Then
> > > > if you use HTTP/2 you (the client) can simply create one HTTP/2
> > > > stream per notification stream.  This just falls out for free with HTTP/2.
> > > >
> > > > <Eric> Agree that SSE over HTTP2 should work.  However to me using
> > > > SSE here seems redundant as HTTP2 provides the necessary mechanisms.
> > >
> But it must still adhere to the normal HTTP semantics.  If there was a request, it
> needs to send the response.  It can split the response into several DATA frames
> of course.  But it cannot start to send "other stuff" instead of the response,
> which figure 1 in

Agree.   Not sure where you see that in Figure 1.   The HTTP2 OK doesn't mean the end of the response. (RFC-7950, page 57.)      Any suggestions on how to make this clearer?
 
> So do we agree that the restconf-notif draft will be updated to use SSE
> regardless of HTTP version?  If there is any text at all about HTTP/2, it will be an
> "implementation note" where it is recommended to use a separate stream per
> subscription?

I will leave that up to Reshad to frame.  I won't fight against it, although there is interplay here with subscription bundling (per below).
 
> > > > > I don't understand the details surrounding using an RPC to make
> > > > > a dynamic subscription.  Unless it's optional, then I can start
> > > > > to understand.
> > > >
> > > > The client would POST to the "rpc" establish-subscripion, and in
> > > > the reply to the POST is a "location" leaf (*) that contains a
> > > > URI.  The client then GET this URI; this is the SSE event stream.
> > > >
> > > With RFC 8040, the SSE URL is "stable", and if the client has read
> > > it onve, it doesn't have to read it again (modulo some definition of
> > > "stable").
> > > The client
> > > sent back in the reply to
> > > "establish-
> > > subscription".  With the SSE URL, the server may hide some state
> > > that corresponds to the filter params etc.  (NOTE the word *may* in
> > > the previous sentence.  A server could as well send back a SSE URL
> > > with the filter parameters embedded in the
> > > URL...)
> > >
> > > > URL.  draft-ietf-netconf-restconf-notif doesn’t currently attempt
> > > > this as being able to deduce the URL doesn’t stop the extra trip
> > > > to the server.
> > > >
> > > > HTTP2 streams.  If we are going to use HTTP2, there is still a
> > > > need to initiate the GET (or POST) in a separate stream other than
> > > > the one being used for RESTCONF signaling.
> > >
> > > I disagree.  If the client wants to have a separate HTTP/2 stream
> > > for the notifs, then it should start such a stream.  If the client
> > > doesn't want a separate HTTP/2 stream then that should be ok too.
> >
> But the solution cannot depend on the subscriber being able to send "delete-
> subscription" on the same transport session (BTW, this requirement should be
> clarified in the YANG definition for "delete-subscription" in SN), since this
> would rule out HTTP/1.1.

Tweaked it to "using a transport session connecting to the subscriber."

> This makes me wonder... is "delete-subscription" really necessary, or is "kill-
> subscription" sufficient?
>

It wouldn't be determined by the subscriber at all.  It would be up to the publisher to determine what gets bundled or not.   If someone wants to create signaling to manipulate what could be bundled, it would require augmentations to SN's establish-subscription RPC parameters.

> Then it sends another establish-subscription and gets another URL back.
> 
> How can a bundled message be sent when there are two different subscription
> resources?

If bundling is supported, you initially send a URL to a shared bundling resource.  Streams can be added and removed from that resource.   These possibilities are the kinds of reasons why a URI is sent back as part of the establish-subscription response (rather than it being well-known).  Further clarifications in this space will be needed as we get deeper into bundling.

> > > NETCONF client can open a separate SSH channel for each
> > > subscription, or it can use a single channel.  Up to the client.
> >
> Really?  An SSH channel is super cheap.  Why do you think that HTTP/2 streams
> have "far less" overhead?  There can be 2^32 channels in an SSH session, and
> 2^31 streams in a HTTP/2 session.
> 
> > > > Someone could write a specification which gets around this by
> > > > forcing every RESTCONF transaction on its own HTTP2 stream.  But
> > > > nobody has proposed such a specification.  Plus there are QoS
> > > > scenarios where you might want multiple subscriptions on the same
> HTTP2 stream.
> > >
> > > The draft is silent on how to delete a subscription, and also on how
> > > modify-subscription is supposed to work.
> >
> > > With NETCONF, the subscription ends if the session ends, but in
> > > RESTCONF there are no sessions, so how long does the subscription
> > > live after it has been created?
> >
> First it says that TLS SHOULD be used.  But with RESTCONF, TLS is already a
> MUST.

The section previously was supporting HTTP2 only configured subscriptions.   Updates are needed for points like this.

> Second, it says that in the case of missed TLS heartbeats, "related TCP sessions"
> are torn down.  What are these sessions (plural), and what exactly does
> "related" mean?

This should be singular.  Related means transiting.
 
> Third, it says that a subscriber can re-establish.  What can it re-establish?  Re-
> send the establish-subscription?  Just resend the POST to the subscription URL?

Text needs to be updated to say "establish-subscription". 

> After the publisher has sent a URI for the subscription to the subscriber, there
> is a delay (***) until the subscriber has sent the new POST.  The resource that
> the URI points to is subscriber-specific, which means that the publisher may
> have created some associated state.  This state will eventually have to be
> destroyed.  When can the publisher safely destroy this state?

Cases include:
- delete-subscription rpc
- kill-subscription rpc
- loss of TLS heartbeat
- inability to deliver sequential sets of notification messages
     "For connectionless or stateless transports like
      HTTP, a lack of receipt acknowledgment of a sequential set of
      notification messages and/or keep-alives can be used to trigger a
      termination of a dynamic subscription. "   (per SN 1.3)
- " A publisher MAY terminate a dynamic subscription at any time."   (per SN 1.3)

> For example, what happens if the new POST never arrives?

Then it will fail to deliver a sequential set of notification messages

> If the subscriber
> loses its transport session, can it send a new POST to the same URL?  Can it
> send multiple POSTs to the same URL?

Agree the text can be enhanced to cover your points above.  I had been assuming this would not be the case.  But I haven't thought deeply about it.
 
Eric

> > > > one-time, capability check the client could make to test for
> > > > available transports?
> > > >
> > > > I really appreciate your time explaining, I have been following
> > > > the emails best I could but I think some basic things are still
> > > > eluding me.
> > > >

---

From: "Reshad Rahman (rrahman)" <[email-masked]>
Date: Fri, 21 Sep 2018 19:11:30 +0000

Hi,

On 2018-09-20, 4:06 PM, "Eric Voit (evoit)" <[email-masked]> wrote:

    Hi,

     

<RR> I'm ok with that.  

Regards,
Reshad.

    

    

    

    

    Cases include:

    

    Eric