Subject: [Tools-discuss] Event stream out of datatracker?

From: "Salz, Rich" <[email-masked]>
Date: Thu, 24 Mar 2022 11:49:54 +0000

Is there an easy way to get a structured stream of events out of the datatracker?  A couple of WG’s are using Mark’s bot to tweet out things like “draft XXX is now in WGLC”  He mentioned that WG adoption is hard to get.  Thoughts?  Time for me to start  doing PR’s again?

---

From: Robert Sparks <[email-masked]>
Date: Thu, 24 Mar 2022 06:55:42 -0500

Right now, events emit email, and update various feeds.

I _think_ wgstate is in one of the working group feeds - will look.

I think, if one were willing to poll, you could get this through the v1 api.

But a more normalized event stream would be a very reasonable thing to 
produce.

But speaking specifically to WG adoption - not all chairs use the wg 
state tracker...

RjS

On 3/24/22 6:49 AM, Salz, Rich wrote:
>
> Is there an easy way to get a structured stream of events out of the 
> datatracker?  A couple of WG’s are using Mark’s bot to tweet out 
> things like “draft XXX is now in WGLC”  He mentioned that WG adoption 
> is hard to get.  Thoughts?  Time for me to start  doing PR’s again?
>
>
> ___________________________________________________________
> Tools-discuss mailing list [email-masked]
> This list is for discussion, not for action requests or bug reports.
> * Report datatracker and mailarchive bugs to:[email-masked]
> * Report tools.ietf.org bugs to:[email-masked]
> * Report all other bugs or issues to:[email-masked]
> List info (including how to Unsubscribe):https://www.ietf.org/mailman/listinfo/tools-discuss

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 24 Mar 2022 22:57:07 +1100

Yes, I currently use the v1 api. The issue I have is that it's hard to find calls for adoption in the event feed that are associated with a particular WG.

> On 24 Mar 2022, at 10:55 pm, Robert Sparks <[email-masked]> wrote:
> 
> But a more normalized event stream would be a very reasonable thing to produce.
> 
> But speaking specifically to WG adoption - not all chairs use the wg state tracker...
> 
> RjS
> 
>> * Report 
>> tools.ietf.org bugs to: [email-masked]
>> 
>> List info (including how to Unsubscribe): 
>> https://www.ietf.org/mailman/listinfo/tools-discuss

--
Mark Nottingham   https://www.mnot.net/

---

From: Tim Wicinski <[email-masked]>
Date: Thu, 24 Mar 2022 08:00:07 -0400

I use the mnot API v1 and agree on the WG adoption being a nice addition.

And I think we should solve for the folks who do use the tracker for WG
adoption,
and not those who don't.

tim

On Thu, Mar 24, 2022 at 7:57 AM Mark Nottingham <[email-masked]> wrote:

> Yes, I currently use the v1 api. The issue I have is that it's hard to
> find calls for adoption in the event feed that are associated with a
> particular WG.
>
>
>
> datatracker?  A couple of WG’s are using Mark’s bot to tweet out things
> like “draft XXX is now in WGLC”  He mentioned that WG adoption is hard to
> get.  Thoughts?  Time for me to start  doing PR’s again?
> >>
> >>
> Tools-discuss mailing list - [email-masked]
> * Report datatracker and mailarchive bugs to: [email-masked]
> * Report tools.ietf.org bugs to: [email-masked]
> * Report all other bugs or issues to: [email-masked]

---

From: "Salz, Rich" <[email-masked]>
Date: Thu, 24 Mar 2022 12:04:56 +0000

*   And I think we should solve for the folks who do use the tracker for WG adoption,
and not those who don't.

Yup.

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 24 Mar 2022 22:57:36 +1100

For reference:
  https://github.com/ietf-github-services/datatracker-tweet

---

From: Robert Sparks <[email-masked]>
Date: Thu, 24 Mar 2022 07:27:24 -0500

All groups

https://datatracker.ietf.org/api/v1/doc/statedocevent/?format=json&state_type__slug=draft-stream-ietf&state__slug=adopt-wg&time__gte=2022-3-1

particular group

https://datatracker.ietf.org/api/v1/doc/statedocevent/?format=json&state_type__slug=draft-stream-ietf&state__slug=adopt-wg&doc__group__acronym=dance&time__gte=2022-3-1

RjS

On 3/24/22 6:57 AM, Mark Nottingham wrote:
> List info (including how to Unsubscribe): https://www.ietf.org/mailman/listinfo/tools-discuss

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 24 Mar 2022 23:30:31 +1100

Sorry, I should have been more precise.

Currently, my script collects all events with /api/v1/doc/docevent/?format=json and picks out the ones it's interested in (i.e., those that apply to the WG in question), but that feed doesn't contain enough information to pull out calls for adoption by wg.

I could make a separate call for this, but I'd rather not increase load (client and server-side) by issuing extra requests.

Cheers,

> On 24 Mar 2022, at 11:27 pm, Robert Sparks <[email-masked]> wrote:
> 
> All groups
>

---

From: Robert Sparks <[email-masked]>
Date: Thu, 24 Mar 2022 07:58:40 -0500

Ok - so, I'll take this down the path, acknowledging that we've crossed 
the bar of "not easy enough"

You'll need then to make a couple of queries ahead of time to get the 
values of some things that don't change then.

Take 
https://datatracker.ietf.org/api/v1/group/group/?format=json&state=active&type__slug=wg 
and stash a map of (acronym->id). That id I'll refer to as group_id later.

Take https://datatracker.ietf.org/api/v1/doc/statedocevent/?format=json 
and stash the ids of the states you are interested. (keep a map of 
(type,slug)->id as state_id.

Those are relatively static - you can update them on a relatively slow 
clock.

Then you can extract what want from what you pull using the large query 
below.

For that large query - I suggest, if you are not doing so already, that 
you provide a start time for your query, and that you explicitly provide 
a (large) limit so that you get what you want in one return instead of 
paging.

RjS

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 25 Mar 2022 00:01:34 +1100

That should be workable -  thanks!

> On 24 Mar 2022, at 11:58 pm, Robert Sparks <[email-masked]> wrote:
> 
> Ok - so, I'll take this down the path, acknowledging that we've crossed the bar of "not easy enough"
> 
> You'll need then to make a couple of queries ahead of time to get the values of some things that don't change then.
> 
> Take https://datatracker.ietf.org/api/v1/group/group/?format=json&state=active&type__slug=wg and stash a map of (acronym->id). That id I'll refer to as group_id later.
> 
> Take https://datatracker.ietf.org/api/v1/doc/statedocevent/?format=json and stash the ids of the states you are interested. (keep a map of (type,slug)->id as state_id.
> 
> Those are relatively static - you can update them on a relatively slow clock.
> 
> Then you can extract what want from what you pull using the large query below.
> 
> For that large query - I suggest, if you are not doing so already, that you provide a start time for your query, and that you explicitly provide a (large) limit so that you get what you want in one return instead of paging.
> 
> RjS
>

---

From: Robert Sparks <[email-masked]>
Date: Thu, 24 Mar 2022 08:06:21 -0500

btw - lurking under here are requirements on what a v2 api might look like.

I would very much like to look at moving from tastypie to the django 
rest framework. I expect that to make providing things like deeper model 
recursion in the API results, when that obviously the right thing to do, 
possible in what the api returns.

RjS

On 3/24/22 7:58 AM, Robert Sparks wrote:
> https://datatracker.ietf.org/api/v1/doc/statedocevent/?format=json and 
> stash the ids of the states you are interested. (keep a map of 
> For that large query - I suggest, if you are not doing so already, 
> that you provide a start time for your query, and that you explicitly 
> provide a (large) limit so that you get what you want in one return 
> instead of paging.
>
> RjS
>
>> Currently, my script collects all events with 
>> /api/v1/doc/docevent/?format=json and picks out the ones it's 
>> interested in (i.e., those that apply to the WG in question), but 
>> that feed doesn't contain enough information to pull out calls for 
>> adoption by wg.
>>
>> I could make a separate call for this, but I'd rather not increase 
>> load (client and server-side) by issuing extra requests.
>>
>> Cheers,
>>
>>
>>>> Yes, I currently use the v1 api. The issue I have is that it's hard 
>>>> to find calls for adoption in the event feed that are associated 
>>>> with a particular WG.
>>>>
>>>>
>>>>
>>>>>> Is there an easy way to get a structured stream of events out of 
>>>>>> the datatracker?  A couple of WG’s are using Mark’s bot to tweet 
>>>>>> out things like “draft XXX is now in WGLC”  He mentioned that WG 
>>>>>> adoption is hard to get.  Thoughts?  Time for me to start  doing 
>>>>>> PR’s again?
>>>>>>
>>>>>>