Subject: [netconf] HTTPS notifications (draft-ietf-netconf-https-notif)

From: Balázs Lengyel <[email-masked]>
Date: Mon, 23 Sep 2019 22:26:20 +0000

Hello,

The draft should discuss how long the HTTPS connection is held open for the
next notification. Or is it open/closed for each notification?

The draft should include a sequence diagram showing the message flow
something like:

Server                         Client

<------- establish TCP

<------- establish SSH (or TLS)

<------- establish Netconf session

<------- <edit-config>   configure subscription over Netconf

<-------- <close-session>

<-------- tear down Netconf/SSH/TCP

Later

------> establish TCP

------> establish TLS

------> Send HTTPS POST message with YANG defined notification

<-----Send OK

The last 2 or 4 steps repeat until the subscription is removed by another
<edit-config> 

Regards Balazs

-- 

Balazs Lengyel                    Senior Specialist
Ericsson Hungary Ltd. 

Mobile: +36-70-330-7909              email: [email-masked]

---

From: Kent Watsen <[email-masked]>
Date: Tue, 24 Sep 2019 00:28:24 +0000

Hi Balazs,

> The draft should discuss how long the HTTPS connection is held open for the next notification. Or is it open/closed for each notification?

Should it?  Perhaps it should say "Implementations must determine how long the HTTPS connection is held open for subsequent notifications, or if it is open/closed for each notification."

>  The draft should include a sequence diagram showing the message flow something like:
> Server                         Client
> <------- establish TCP

I don't think the draft needs to include this diagram, as it's standard NETCONF.

> The last 2 or 4 steps repeat until the subscription is removed by another <edit-config> 

Assuming the TLS session is help open, it might look like this?

> <-----Send 204 (No Content)
> <-----Send 204 (No Content)

> <-----Send 204 (No Content)

From the email I sent Martin on Sep 9th, each POST MAY contain more than one notification:

    POST /some/path HTTP/1.1
    Host: my-receiver.my-domain.com <http://my-receiver.my-domain.com/>
    Content-Type: application/yang-data+xml

    <notification
      xmlns="urn:ietf:params:xml:ns:netconf:notification:1.0">
      <eventTime>2019-03-22T12:35:00Z</eventTime>
      <foo xmlns="https://example.com <https://example.com/>/my-foobar-module">
        ...
      </foo>
    </notification>

    <notification
      xmlns="urn:ietf:params:xml:ns:netconf:notification:1.0">
      <eventTime>2019-03-22T12:35:00Z</eventTime>
      <bar xmlns="https://example.com <https://example.com/>/my-foobar-module">
        ...
      </bar>
    </notification>

    <notification
      xmlns="urn:ietf:params:xml:ns:netconf:notification:1.0">
      <eventTime>2019-03-22T12:35:00Z</eventTime>
      <baz xmlns="https://example.com <https://example.com/>/my-foobar-module">
        ...
      </baz>
    </notification>

With response:

      HTTP/1.1 204 No Content
      Date: Thu, 26 Jan 2017 20:56:30 GMT
      Server: my-receiver.my-domain.com <http://my-receiver.my-domain.com/>

Kent // co-author

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 24 Sep 2019 08:45:58 +0200 (CEST)

Hi,

Kent Watsen <[email-masked]> wrote:
> 
> Hi Balazs,
> 
> 
> > The draft should discuss how long the HTTPS connection is held open
> > for the next notification. Or is it open/closed for each notification?
> 
> Should it?  Perhaps it should say "Implementations must determine how
> long the HTTPS connection is held open for subsequent notifications,
> or if it is open/closed for each notification."

This is somewhat dependent on how notifs are streamed, see below.

> I don't think the draft needs to include this diagram, as it's

+1

> From the email I sent Martin on Sep 9th, each POST MAY contain more

... to which I replied:

  I'm not so sure about the streaming though.
  Perhaps pipelining is the right mechanism.  For "bulk" sending, the
  "bundled-message" defined in draft-ietf-netconf-notification-messages
  seems right.

To clarify: if the client just sends a stream of notifs it becomes a
variant of SSE.  The server doesn't know when the stream will end, and
thus cannot simply close the session.  You probably want to indicate
end-of-message somehow (like in SSE).  And the content type in the
example below cannot be "application/yang-data+xml", since it is not a
valid XML instance document; you'd have to invent a new media type to
indicate the streaming.

I think we should stick to simple HTTP where each notif is POSTed, as
in your diagaram above.  With HTTP pipelining you can do:

------> establish TCP
------> establish TLS
------> Send HTTPS POST message with YANG defined notification 1
------> Send HTTPS POST message with YANG defined notification 2
<-----Send 204 (No Content) for 1
<-----Send 204 (No Content) for 2
 
------> Send HTTPS POST message with YANG defined notification 3
<-----Send 204 (No Content) for 3

If the server wants to send multiple notifs at once, it can use
"bundled-message".

/martin

>       <foo xmlns="https://example.com
>       <https://example.com/>/my-foobar-module">
>         ...
>       </foo>
>     </notification>
> 
>     <notification
>       <bar xmlns="https://example.com
>       <https://example.com/>/my-foobar-module">
>         ...
>       </bar>
>     </notification>
> 
>     <notification
>       <baz xmlns="https://example.com
>       <https://example.com/>/my-foobar-module">
>         ...
>       </baz>
>     </notification>
> 
> 
> With response:
> 
>       HTTP/1.1 204 No Content

---

From: "Eric Voit (evoit)" <[email-masked]>
Date: Tue, 24 Sep 2019 13:45:22 +0000

> .... to which I replied:
variant of
> SSE.  The server doesn't know when the stream will end, and thus cannot
> simply close the session.  You probably want to indicate end-of-message
> somehow (like in SSE).  And the content type in the example below cannot
> be "application/yang-data+xml", since it is not a valid XML instance
> document; you'd have to invent a new media type to indicate the streaming.
> 
> I think we should stick to simple HTTP where each notif is POSTed, as in
your
> diagaram above.  With HTTP pipelining you can do:
> 
> ------> establish TCP
> ------> establish TLS
> ------> Send HTTPS POST message with YANG defined notification 1 Send
> ------> HTTPS POST message with YANG defined notification 2
> If the server wants to send multiple notifs at once, it can use "bundled-
> message".

This seems a reasonable approach. draft-ietf-netconf-notification-messages
has several advantages:
(1) can push multiple YANG notifications at once
(2) can include the subscription-id in notifications when subscribing to a
stream.  (Right now including an explicit subscription-id is only available
when subscribing to a datastore.)
(3) includes methods to discover lost/dropped notifications

Two things which would need to be worked:
(1) discovering receiver support for bundled notifications.  (As some form
of understanding/verifying configured receiver support over HTTP is already
needed, this is a topic which perhaps can be merged into that.)
(2) Completion of draft-ietf-netmod-yang-data-ext 

Eric

> _______________________________________________
> netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 24 Sep 2019 15:55:45 +0200 (CEST)

> needed, this is a topic which perhaps can be merged into that.)

This can be configured or auto-detected.  For auto-detection, perhaps
we can use HTTP OPTIONS with the server returning a special body or
header to indicate capabilities such as this one?

> (2) Completion of draft-ietf-netmod-yang-data-ext 

Yes!  IMO it is ready for WGLC; I'll ping the chairs again.

/martin

---

From: "Eric Voit (evoit)" <[email-masked]>
Date: Tue, 24 Sep 2019 14:57:37 +0000

> From: Martin Bjorklund, September 24, 2019 9:56 AM
> 
> > > SSE.  The server doesn't know when the stream will end, and thus
> > > cannot simply close the session.  You probably want to indicate
> > > example below cannot be "application/yang-data+xml", since it is not
> > > a valid XML instance document; you'd have to invent a new media type
> to indicate the streaming.
> > >
> > > "bundled- message".
> > (2) can include the subscription-id in notifications when subscribing
> > to a stream.  (Right now including an explicit subscription-id is only
> > available when subscribing to a datastore.)
> > (1) discovering receiver support for bundled notifications.  (As some
> > form of understanding/verifying configured receiver support over HTTP
> > is already needed, this is a topic which perhaps can be merged into
> > that.)
> 
> This can be configured or auto-detected.  For auto-detection, perhaps we
> can use HTTP OPTIONS with the server returning a special body or header to
> indicate capabilities such as this one?

The method previously proposed for HTTP configured subscriptions was an "OK"
to the POST of a <subscription-started>.  The "OK" had to be received prior
to sending YANG notifications.  If the authors want to build upon this, I
could see HTTP OPTIONS as a way to provide receiver capabilities.  Some
discussion about what capabilities would be advertisable as options would be
worthwhile.  Also worth discussion would be any intersections with
NETCONF/RESTCONF capabilities discovery/exchange.

Eric

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 24 Sep 2019 18:48:14 +0200 (CEST)

"Eric Voit (evoit)" <[email-masked]> wrote:
> to sending YANG notifications.  If the authors want to build upon
> this

This is now a WG document, so it should be discussed in the WG.

> , I
> could see HTTP OPTIONS as a way to provide receiver capabilities.

Or perhaps simply do a GET to the same URI as you POST to, and in
return get a sx:structure with the capabilities of the server.

> Some
> worthwhile.

Yes.

> Also worth discussion would be any intersections with

Sure.  But this is a new protocol, with a new (limited, hopefully) set
of capabilities.

/martin

---

From: Balázs Lengyel <[email-masked]>
Date: Thu, 26 Sep 2019 09:04:19 +0000

-----Original Message-----
From: Martin Bjorklund <[email-masked]> 
Sent: 2019. szeptember 24., kedd 15:56
To: [email-masked]
Cc: [email-masked]; Balázs Lengyel <[email-masked]>;
[email-masked]
Subject: Re: [netconf] HTTPS notifications (draft-ietf-netconf-https-notif)

"Eric Voit (evoit)" <[email-masked]> wrote:
> > a valid XML instance document; you'd have to invent a new media type to
indicate the streaming.
> is already needed, this is a topic which perhaps can be merged into 
This can be configured or auto-detected.  For auto-detection, perhaps we can
use HTTP OPTIONS with the server returning a special body or header to
indicate capabilities such as this one?

BALAZS: IMHO autodetection is much better then configuration. However we
should try to avoid extra message roundtrips to do autodetection, so IMHO
either
- cache receiver capabilities for subscriptions, so autodetection is needed
only once
- send bundled notifications immediately and fallback to simple
notifications if needed based on any error messages received 
Regards Balazs