Subject: [Dots] DOTS telemetry Issues picked up in Interop Testing

From: "Jon Shallow" <[email-masked]>
Date: Mon, 20 Apr 2020 13:06:49 +0100

Hi All,

1)

   DOTS agents MUST NOT send pre-or-ongoing-mitigation telemetry

   messages to the same peer more frequently than once every 'telemetry-

   notify-interval' (Section 6.1).

If I do a PUT /tm-setup, followed immediately by a GET /tm-setup, should the
GET fail based on the telemetry-notify-interval, or should
telemetry-notify-interval apply individually to PUT. GET and DELETE?

[I may want to do a DELETE (no tsid) followed by a GET to get suitable
max/min information and telemetry-notify-interval can easily be more than 1
second. ]

2)

Use of multiple Uri-Query:

Uri-Query: target_prefix=[1.2.3.4]

Uri-Query: target_prefix=[2.3.4.5]

I think this should be OR

Uri-Query: target_prefix=[1.2.3.4]

Uri-Query: target_port=[80]

I think this should be an AND.

Perhaps we should have operator queries

Uri-Query: target_prefix=[1.2.3.4]

Uri-Query: op=OR

Uri-Query: target_prefix=[2.3.4.5]

Uri-Query: target_prefix=[1.2.3.4]

Uri-Query: op=AND

Uri-Query: target_port=[80]

Thoughts?

3)

What happens if a Query cannot be supported.

E.g. Uri-Query: target_port=[80] and statistics on the server by a port
basis  is not supported.  Do we want to either error out with a 4.0X
responses, or should we include a (new additional) YANG body response
indicating which Uri-Query is not supported?

Regards

Jon

---

From: <[email-masked]>
Date: Mon, 20 Apr 2020 12:53:58 +0000

Hi Jon,

Thank you for sharing the comments.

Please see

De : Dots [mailto:[email-masked]] De la part de Jon Shallow
Envoyé : lundi 20 avril 2020 14:07
À : [email-masked]
Objet : [Dots] DOTS telemetry Issues picked up in Interop Testing

Hi All,

1)

If I do a PUT /tm-setup, followed immediately by a GET /tm-setup, should the GET fail based on the telemetry-notify-interval, or should telemetry-notify-interval apply individually to PUT. GET and DELETE?
[I may want to do a DELETE (no tsid) followed by a GET to get suitable max/min information and telemetry-notify-interval can easily be more than 1 second. ]

[Med] This rate limit does not apply to these messages; it applies only for notifications;

==
        leaf telemetry-notify-interval {
          type uint32 {
            range "1 .. 3600";
          }
          must '. >= ../../min-config-values/telemetry-notify-interval' {
            error-message
              "The value must be greater than or equal
               to the telemetry-notify-interval in the min-config-values";
          }
          units "seconds";
          description
            "Minimum number of seconds between successive
             telemetry notifications.";
        }
===

We can make the text clear:

OLD:

   notify-interval' (Section 6.1<https://tools.ietf.org/html/draft-ietf-dots-telemetry-07#section-6.1>).  If a telemetry notification is sent

   using a block-like transfer mechanism (e.g.,

   [I-D.bosh-core-new-block<https://tools.ietf.org/html/draft-ietf-dots-telemetry-07#ref-I-D.bosh-core-new-block>]), this rate limit policy MUST NOT consider

   these individual blocks as separate notifications, but as a single

   notification.

NEW:

   notifications to the same peer more frequently than once every 'telemetry-

   notify-interval' (Section 6.1<https://tools.ietf.org/html/draft-ietf-dots-telemetry-07#section-6.1>).  If a telemetry notification is sent

   using a block-like transfer mechanism (e.g.,

   [I-D.bosh-core-new-block<https://tools.ietf.org/html/draft-ietf-dots-telemetry-07#ref-I-D.bosh-core-new-block>]), this rate limit policy MUST NOT consider

   these individual blocks as separate notifications, but as a single

   notification.

2)

Uri-Query: target_prefix=[1.2.3.4]
Uri-Query: target_prefix=[2.3.4.5]
I think this should be OR
[Med] Yes, this is the same interpretation when we include target-prefix in the message body.

Uri-Query: target_prefix=[1.2.3.4]
Uri-Query: target_port=[80]
I think this should be an AND.
[Med] Yes, because the target-port is specifying a subset of ports bound to the target-prefix.

Uri-Query: target_prefix=[1.2.3.4]
Uri-Query: op=AND
Uri-Query: target_port=[80]
[Med] I don't think this is needed. We don't specify the operator when we include the target clauses in the message body. The same rules apply. We can clarify this in the text If you think this is needed.

Thoughts?

3)

E.g. Uri-Query: target_port=[80] and statistics on the server by a port basis  is not supported.  Do we want to either error out with a 4.0X responses, or should we include a (new additional) YANG body response indicating which Uri-Query is not supported?

[Med] Good point. The client can retrieve during telemetry setup the supported query types + server replies with 4.00 when an invalid query type is used.

Regards

Jon

---

From: "Jon Shallow" <[email-masked]>
Date: Mon, 20 Apr 2020 14:46:32 +0100

Hi Med,

See inline.

Regards

Jon

From: Dots [mailto: [email-masked]] On Behalf Of
[email-masked]
Sent: 20 April 2020 13:54
To: Jon Shallow; [email-masked]
Subject: Re: [Dots] DOTS telemetry Issues picked up in Interop Testing

Hi Jon, 

Please see

Hi All,

1)

[Med] This rate limit does not apply to these messages; it applies only for
notifications; 

==

          type uint32 {

            range "1 .. 3600";

          }

            error-message

          }

          units "seconds";

          description

        }

===

OLD: 

   DOTS agents MUST NOT send pre-or-ongoing-mitigation telemetry
   messages to the same peer more frequently than once every 'telemetry-
   notify-interval' (Section 6.1
<https://tools.ietf.org/html/draft-ietf-dots-telemetry-07#section-6.1> ).
If a telemetry notification is sent
   using a block-like transfer mechanism (e.g.,
   [I-D.bosh-core-new-block
<https://tools.ietf.org/html/draft-ietf-dots-telemetry-07#ref-I-D.bosh-core-
new-block> ]), this rate limit policy MUST NOT consider
   these individual blocks as separate notifications, but as a single
   notification.

NEW:

   DOTS agents MUST NOT send pre-or-ongoing-mitigation telemetry
   notifications to the same peer more frequently than once every
'telemetry-
   notify-interval' (Section 6.1
<https://tools.ietf.org/html/draft-ietf-dots-telemetry-07#section-6.1> ).
If a telemetry notification is sent
   using a block-like transfer mechanism (e.g.,
   [I-D.bosh-core-new-block
<https://tools.ietf.org/html/draft-ietf-dots-telemetry-07#ref-I-D.bosh-core-
new-block> ]), this rate limit policy MUST NOT consider
   these individual blocks as separate notifications, but as a single
   notification.

Jon> This change works for me.

2)

Uri-Query: target_prefix=[1.2..3.4]

Uri-Query: target_prefix=[2.3..4.5]

[Med] Yes, this is the same interpretation when we include target-prefix in
the message body.

Uri-Query: target_prefix=[1.2..3.4]

[Med] Yes, because the target-port is specifying a subset of ports bound to
the target-prefix.  

Uri-Query: target_prefix=[1.2..3.4]

Uri-Query: op=OR

Uri-Query: target_prefix=[2.3..4.5]

Uri-Query: target_prefix=[1.2..3.4]

Uri-Query: op=AND

[Med] I don’t think this is needed. We don’t specify the operator when we
include the target clauses in the message body. The same rules apply. We can
clarify this in the text If you think this is needed.  

Jon> I think that it is worth stating that the same rules apply for
clarification. 

Thoughts?

3)

[Med] Good point. The client can retrieve during telemetry setup the
supported query types + server replies with 4.00 when an invalid query type
is used. 

Jon> I think the server stating what Queries are supported is a good idea.

Regards

Jon

---

From: <[email-masked]>
Date: Mon, 20 Apr 2020 15:46:48 +0000

Re-,

FWIW, I implemented the agreed changes at: https://github.com/boucadair/draft-dots-telemetry/blob/master/draft-ietf-dots-telemetry-07.txt

Cheers,
Med

De : Jon Shallow [mailto:[email-masked]]
Envoyé : lundi 20 avril 2020 15:47
À : BOUCADAIR Mohamed TGI/OLN; [email-masked]
Objet : RE: [Dots] DOTS telemetry Issues picked up in Interop Testing

Hi Med,

See inline.

Regards

Jon

From: Dots [mailto: [email-masked]] On Behalf Of [email-masked]
Sent: 20 April 2020 13:54
To: Jon Shallow; [email-masked]
Subject: Re: [Dots] DOTS telemetry Issues picked up in Interop Testing

Hi Jon,

Please see

Hi All,

1)

==

OLD:

   notification.

NEW:

   notification.

2)

Jon> I think that it is worth stating that the same rules apply for clarification.

Thoughts?

3)

Regards

Jon

---

From: "Jon Shallow" <[email-masked]>
Date: Tue, 21 Apr 2020 09:59:14 +0100

Hi all,

A further thought on the use of Uri-Queries to clarify the AND/OR usage.

If you only allow one query per query type and put the match list in an
array, then this will be an OR of the array list (the same as we do for the
target* definitions right now.  E.G. :-

Uri-Query: target_prefix=[1.2.3.4/32,4.3.2.1/32]

Gives either 1.2.3.4 or 4.3.2.1 as a valid match.

And 

Uri-Query: target-prefix=[1.2.3.4/32,4.3.2.1/32]

Uri-Query: lower-port=[80,443]

Gives (either 1.2.3.4 or 4.3.2.1) and (either port 80 or 443)

[] should not include spaces and comma used as a separator.

Not sure about :-  If [] are not used, then it should be treated as a single
item to match.

The list of valid Uri-Queries may need to be extended.

A Uri-Query option can include the following

   parameters: target-prefix, lower-port, upper-port, target-protocol,

   target-fqdn, target-uri, alias-name.

should include mid and content (‘c’).   Do we need to be able to filter on
source attributes as well?

Regards

Jon

From: Dots [mailto: [email-masked]] On Behalf Of Jon Shallow
Sent: 20 April 2020 14:47
To: [email-masked]; [email-masked]
Subject: Re: [Dots] DOTS telemetry Issues picked up in Interop Testing

Hi Med,

See inline.

Regards

Jon

Hi Jon, 

Please see

Hi All,

1)

==

          type uint32 {

            range "1 .. 3600";

          }

            error-message

          }

          units "seconds";

          description

        }

===

OLD: 

NEW:

2)

Uri-Query: op=OR

Uri-Query: op=AND

Thoughts?

3)

Regards

Jon

---

From: <[email-masked]>
Date: Tue, 21 Apr 2020 10:04:04 +0000

Hi Jon,

Please see inline.

Cheers,
Med

De : Jon Shallow [mailto:[email-masked]]
Envoyé : mardi 21 avril 2020 10:59
À : BOUCADAIR Mohamed TGI/OLN; [email-masked]
Objet : RE: [Dots] DOTS telemetry Issues picked up in Interop Testing

Hi all,

If you only allow one query per query type and put the match list in an array, then this will be an OR of the array list (the same as we do for the target* definitions right now.  E.G. :-

And

[Med] The issue I have with this is that we will need to handle cases where both lower-port and upper-port are present. Not sure what would be the benefit of allowing multiple key values, compact uris? If that's a concern, we may consider shortened names in the query (e.g., s/target-prefix/tp, s/lower-port/lp, ..).

Not sure about :-  If [] are not used, then it should be treated as a single item to match.

should include mid and content ('c').
[Med] OK. Please note that 'c' wasn't listed because we do have the following:

  The DOTS server follows the same considerations discussed in
   Section of 4.5.3 of [I-D.ietf-dots-signal-channel] for managing DOTS
   telemetry configuration freshness and notification.  Likewise, a DOTS
   client may control the selection of configuration and non-
   configuration data nodes when sending a GET request by means of the
   'c' Uri-Query option and following the procedure specified in
   Section of 4.4.2 of [I-D.ietf-dots-signal-channel].  These
   considerations are not re-iterated in the following sections.

I added a pointer to that section.

  Do we need to be able to filter on source attributes as well?
[Med] What would be the usage? Is it the case of an administrator that is aware (using some means) that some sources are involved an attacks on other networks, then sends a request to its server to filter telemetry data bound to these source? Filtering in this case may be problematic as some attacks may be observed but are hidden by the filter on the source. Or do we want to focus on very few talkers? For example, the client would instruct the server to send data only related to the top-talker, 3 first top-talkers, etc.?

Regards

Jon

Hi Med,

See inline.

Regards

Jon

Hi Jon,

Please see

Hi All,

1)

==

OLD:

   notification.

NEW:

   notification.

2)

Thoughts?

3)

Regards

Jon

---

From: "Jon Shallow" <[email-masked]>
Date: Tue, 21 Apr 2020 11:56:21 +0100

Please see inline.

Regards

Jon

From: Dots [mailto: [email-masked]] On Behalf Of
[email-masked]
Sent: 21 April 2020 11:04
To: Jon Shallow; [email-masked]
Subject: Re: [Dots] DOTS telemetry Issues picked up in Interop Testing

Hi Jon,

Please see inline.

Cheers,

Med

Hi all,

And 

[Med] The issue I have with this is that we will need to handle cases where
both lower-port and upper-port are present. Not sure what would be the
benefit of allowing multiple key values, compact uris? If that’s a concern,
we may consider shortened names in the query (e.g., s/target-prefix/tp,
s/lower-port/lp, ..).

Jon> fair point about lower and upper ports.  Uri-Query:
target-port[80-85,443] works for me and covers both ranges and individual
ports.

Jon> As this would be options on a GET request that has no body data, I
don’t think that I am too worried about using shortened names at this point.

should include mid and content (‘c’). 

[Med] OK. Please note that ‘c’ wasn’t listed because we do have the
following:

[Med] What would be the usage? Is it the case of an administrator that is
aware (using some means) that some sources are involved an attacks on other
networks, then sends a request to its server to filter telemetry data bound
to these source? Filtering in this case may be problematic as some attacks
may be observed but are hidden by the filter on the source. Or do we want to
focus on very few talkers? For example, the client would instruct the server
to send data only related to the top-talker, 3 first top-talkers, etc.? 

Jon> Source-port= could be useful – especially when being subjected to a
reflected type attach where all the traffic is coming from, say, source port
53

Jon> Otherwise, it may help to filter on top-talkers, vendor specific attack
types etc.

Jon> Which then leads me on to something else.  If we asking for telemetry
for 2 (or more) different target IPs and the server is returning vendor
specific attack details.  For attack-detail, keyed by attack-id, if the same
attack is hitting more than 1 target IP, can I display this information per
IP or does it have to be aggregated?

~Jon

Regards

Jon

Hi Med,

See inline.

Regards

Jon

Hi Jon, 

Please see

Hi All,

1)

==

          type uint32 {

            range "1 .. 3600";

          }

            error-message

          }

          units "seconds";

          description

        }

===

OLD: 

   [I-D..bosh-core-new-block

NEW:

   [I-D..bosh-core-new-block

2)

Uri-Query: target_prefix=[1.2...3.4]

Uri-Query: target_prefix=[2.3...4.5]

Uri-Query: target_prefix=[1.2...3.4]

Uri-Query: target_prefix=[1.2...3.4]

Uri-Query: op=OR

Uri-Query: target_prefix=[2.3...4.5]

Uri-Query: target_prefix=[1.2...3.4]

Uri-Query: op=AND

Thoughts?

3)

Regards

Jon