Subject: [Ecrit] Planned-changed: notify vs polling vs TTL only

From: "Randall Gellens" <[email-masked]>
Date: Wed, 22 Sep 2021 10:11:33 -0700

One of the open issues with the planned-changes draft is the question of 
how LoST validation clients (such as LISes) know when a change will 
occur or has occurred, and which of their locations need to be 
revalidated. Three methods have been discussed:

Notification (currently in the draft): A LoST query can optionally 
contain a URI.  If it does, the LoST server stores the URI and returns 
an ID.  If a change will occur that could affect the validity of the 
location, the server sends a notification to the URI.  The notification 
includes the UD.

TTL only: LoST queries return a TTL.  The client should revalidate 
before the TTL expires.  As a planned change draws near, the server 
progressively reduces the TTL.

Polling: Clients may poll for changes.

The Notification mechanism is described in the draft currently. There is 
an issue remaining to be worked out, which is the mechanism by which the 
server verifies that the target of the URI wants the notification.

The TTL only mechanism is well understood as other systems use it (e.g., 
DNS).

The Polling mechanism has been discussed in email and during some time 
on last week's NENA i3 call, but I have not seen a detailed, specific 
proposal.  For example, is it a new LoST query or does it use a 
different interface?  What is the interface, the query, the response?  
If it is not a LoST query, how does a LoST validation client discover 
the server?  I'm requesting that those advocating for the query approach 
submit a detailed specification so it can be seen and discussed.

--Randall

---

From: "Caron, Guy" <[email-masked]>
Date: Wed, 22 Sep 2021 17:57:04 +0000

I don't see how TTL-only solution would work. The Server can only reduce the TTL in responses to validation requests and has no opportunity to update it if a planned change comes in before the TTL expires. That brings us back to blind polling.

I can't comment on the polling proposal since nothing firm has been proposed yet.

Thanks,

Guy

-----Message d'origine-----
De : Ecrit <[email-masked]> De la part de Randall Gellens
Envoyé : 22 septembre 2021 13:12
À : [email-masked]
Objet : [EXT][Ecrit] Planned-changed: notify vs polling vs TTL only

One of the open issues with the planned-changes draft is the question of how LoST validation clients (such as LISes) know when a change will occur or has occurred, and which of their locations need to be revalidated. Three methods have been discussed:

Notification (currently in the draft): A LoST query can optionally contain a URI.  If it does, the LoST server stores the URI and returns an ID.  If a change will occur that could affect the validity of the location, the server sends a notification to the URI.  The notification includes the UD.

TTL only: LoST queries return a TTL.  The client should revalidate before the TTL expires.  As a planned change draws near, the server progressively reduces the TTL.

The Notification mechanism is described in the draft currently. There is an issue remaining to be worked out, which is the mechanism by which the server verifies that the target of the URI wants the notification.

The TTL only mechanism is well understood as other systems use it (e.g., DNS).

The Polling mechanism has been discussed in email and during some time on last week's NENA i3 call, but I have not seen a detailed, specific proposal.  For example, is it a new LoST query or does it use a different interface?  What is the interface, the query, the response?  
If it is not a LoST query, how does a LoST validation client discover the server?  I'm requesting that those advocating for the query approach submit a detailed specification so it can be seen and discussed.

--Randall

_______________________________________________
Ecrit mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ecrit
------------------------------------------------------------------------------
External Email: Please use caution when opening links and attachments / Courriel externe: Soyez prudent avec les liens et documents joints

---

From: Brian Rosen <[email-masked]>
Date: Wed, 22 Sep 2021 13:57:35 -0400

Understanding that I prefer (strongly), notification, here is a specific polling proposal:
1. The only parameter in the planned changes findService request is asOf date.  There is nothing added to the response
2. A new rest/json interface is defined.  There are two entry points: plannedChanges and getChangeSet
3. plannedChanges takes a single parameter: lastChangeSeen, which can be omitted.  The response is a list of changeSetIds, which are arbitrary strings, ordered by when they were recognized as a changeSet by the server (meaning the oder doesn’t represent anything special, but never changes).  lastChangeSeen is a changeSetId the client already knows about.  If lastChangeSeen is present, the response is a list of changeSetId after the lastChangeSeen set.
4. getChangeSet takes a changeSetId and returns a changeSet.  A changeSet an an Id, an asOf date and a set of partial civic locations.  The LoST client compares each partial civic to its records.  If every element provided in the partial civic matches the fields in its record, then the record is possibly covered by the planned change.  That comparison implies that values in elements NOT in the partial civic are not checked: any value matches.
5. The client is expected to poll plannedChanges every few minutes.  If a new changeSet is found, the client fetches it with getChangeSet.  It identifies which of its records may be affected, and can plan a change.  It can pre-validate using the asOf in findService, and when the asOf date in the changeSet occurs, it updates its records accordingly

Brian

> On Sep 22, 2021, at 1:11 PM, Randall Gellens <[email-masked]> wrote:
> 
> The Polling mechanism has been discussed in email and during some time on last week's NENA i3 call, but I have not seen a detailed, specific proposal.  For example, is it a new LoST query or does it use a different interface?  What is the interface, the query, the response?  If it is not a LoST query, how does a LoST validation client discover the server?  I'm requesting that those advocating for the query approach submit a detailed specification so it can be seen and discussed.
> 
> --Randall
>

---

From: Brian Rosen <[email-masked]>
Date: Wed, 22 Sep 2021 14:02:21 -0400

BTW, I’ve implemented a polling mechanism just like this (ordered Ids, poll for ids with last id seen) four times.  NPAC also uses it.  It works well, but it’s polling, which I don’t like.

> On Sep 22, 2021, at 1:57 PM, Brian Rosen <[email-masked]> wrote:
>

---

From: "Randall Gellens" <[email-masked]>
Date: Wed, 22 Sep 2021 13:29:59 -0700

How does a LoST client discover the new interface?  Is a URI for it 
added to a findServiceResponse?  Is there a new SRV service for it?  Is 
UNAPTR used?

If I understand the proposals, in the notification mechanism, LoST 
clients receive an ID for each location they validate.  Multiple 
locations can have the same ID.  I understand we don't specify 
implementation details, but at least in an abstract sense, when 
notified, the client uses the ID as an index to identify the records 
that need to be revalidated.  In the polling mechanism, the client 
receives sets of partial civic addresses.  How does it identify the 
affected records?  Does it (effectively) need an index for each address 
field, or does it do a brute force search, or something else?

--Randall

On 22 Sep 2021, at 10:57, Brian Rosen wrote:

> 1. The only parameter in the planned changes findService request is 
> asOf date.  There is nothing added to the response
> 3. plannedChanges takes a single parameter: lastChangeSeen, which can 
> be omitted.  The response is a list of changeSetIds, which are 
> arbitrary strings, ordered by when they were recognized as a changeSet 
> by the server (meaning the oder doesn’t represent anything special, 
> but never changes).  lastChangeSeen is a changeSetId the client 
> already knows about.  If lastChangeSeen is present, the response is a 
> list of changeSetId after the lastChangeSeen set.
> 4. getChangeSet takes a changeSetId and returns a changeSet.  A 
> changeSet an an Id, an asOf date and a set of partial civic locations. 
>  The LoST client compares each partial civic to its records.  If every 
> element provided in the partial civic matches the fields in its 
> record, then the record is possibly covered by the planned change.  
> That comparison implies that values in elements NOT in the partial 
> civic are not checked: any value matches.
> 5. The client is expected to poll plannedChanges every few minutes.  
> If a new changeSet is found, the client fetches it with getChangeSet.  
> It identifies which of its records may be affected, and can plan a 
> change.  It can pre-validate using the asOf in findService, and when 
> the asOf date in the changeSet occurs, it updates its records 
> accordingly
>
> Brian
>
>> One of the open issues with the planned-changes draft is the question 
>> of how LoST validation clients (such as LISes) know when a change 
>> will occur or has occurred, and which of their locations need to be 
>> contain a URI.  If it does, the LoST server stores the URI and 
>> returns an ID.  If a change will occur that could affect the validity 
>> of the location, the server sends a notification to the URI.  The 
>> notification includes the UD.
>>
>> The Notification mechanism is described in the draft currently. There 
>> is an issue remaining to be worked out, which is the mechanism by 
>> which the server verifies that the target of the URI wants the 
>> notification.
>>
>> The Polling mechanism has been discussed in email and during some 
>> time on last week's NENA i3 call, but I have not seen a detailed, 
>> specific proposal.  For example, is it a new LoST query or does it 
>> use a different interface?  What is the interface, the query, the 
>> response?  If it is not a LoST query, how does a LoST validation 
>> client discover the server?  I'm requesting that those advocating for 
>> the query approach submit a detailed specification so it can be seen 
>> and discussed.
>>
>> --Randall
>>

---

From: Brian Rosen <[email-masked]>
Date: Wed, 22 Sep 2021 16:34:21 -0400

UNAPTR discovery of the interface

There are no server locations IDs. Just partial civics and changeSetIds.
The search is a very simple database search for matching elements.  You can
index all the elements or just search them. In database scales, these are
small databases, full search would be fast. You only need to do that when
there is a new changeSet

Brian

---

From: Henning Schulzrinne <[email-masked]>
Date: Wed, 22 Sep 2021 16:48:31 -0400

Joining the discussion late, so I probably missed a bit of the earlier
problem scoping. Leaving push vs. pull aside, this seems like a
(relatively) standard database (or git-style) synchronization problem, with
a few location-related wrinkles. Indeed, this seems to be in the same
general ballpark as the tzid updates.

Or similar to the cache invalidation problem in HTTP. If there is
versioning, the simplest approach seems to be "I have version [random
opaque string]; is there a later version?". The updates would then
obviously have an as-of date that's separate. (In theory, you could have
several pending changes, as in "it's A as of June 1, it's A-prime as of
July 1 and A-double-prime as of August 1". Given the as-of mechanism, this
allows decoupling the update effective date from the polling or
notification date.

Would you need more than per-database versioning?

One reason I like to have both polling and notification is that this can
then serve double-duty: A backup database that awakes from slumber (or is
just re-started) can poll and say "hey, I just woke up, what's been
happening since I went to sleep?" and the more efficient
unpredictable-interval notification. It also backstops the notification
mechanism in that any loss of the notification (webhook) URLs will get
fixed at the next polling interval.

---

From: "Randall Gellens" <[email-masked]>
Date: Wed, 22 Sep 2021 15:27:43 -0700

Notification vs polling is the primary open issue.  The other issues are 
secondary, as they differ depending on which mechanism is selected.

--Randall

On 22 Sep 2021, at 13:48, Henning Schulzrinne wrote:

>>>> 2. A new rest/json interface is defined.  There are two entry 
>>>> points:
>>>> plannedChanges and getChangeSet

---

From: Brian Rosen <[email-masked]>
Date: Wed, 22 Sep 2021 16:59:48 -0400

In my proposal for poll, the lastChangeSeen is “version” in your notion of
“I have version…”

“Both” seems like too much mechanism but I will go with the flow.

Brian

On Wed, Sep 22, 2021 at 4:49 PM Henning Schulzrinne <[email-masked]>
wrote:

>> On Wed, Sep 22, 2021 at 4:30 PM Randall Gellens <
>> [email-masked]> wrote:
>>