Subject: [sidr] Updates: sidr-arch-09 and roa-format-06

From: Matt Lepinski <[email-masked]>
Date: Mon, 26 Oct 2009 17:35:09 -0400

I have just submitted updates to draft-ietf-sidr-arch and 
draft-ietf-roa-format

The only substanative change to the roa-format draft was to reference 
the algorithm and key size specifications in draft-ietf-sidr-rpki-algs 
(as is currently done in the rpki-manifests document).

The only substanative change to the architecture draft was to 
incorporate a suggestion from Randy Bush that entities who are actually 
using RPKI data for routing SHOULD be fetching fresh data from the 
repositories at least once every three hours.

Additionally, I made a number of very small edits (i.e., bug fixes) to 
both documents.

Thanks to Larry Blunk, Roque Gagliano, Charlie Gardner and Randy Bush 
for helpful comments and reviews.

Both of these drafts appear to be quite stable and I know of no open 
issues in either document.

---

From: Geoff Huston <[email-masked]>
Date: Tue, 27 Oct 2009 10:24:24 +1100

WG Co-Chair Hat OFF

Hi Matt,

> entities who are actually using RPKI data for routing SHOULD be  
> fetching fresh data from the repositories at least once every three  
> hours.
>

3 hours?

At a first pass that seems very frequent.

 From a server's perspective if there are 30,000 AS's out there and  
each is running a local cache and each is a distinct relying party of  
the RPKI system, then the local hit rate at the server would be 3 per  
second, assuming that all the relying parties evenly spread their load  
(which is a pretty wild assumption - the worst case is that all 30,000  
attempt to resync at the 3 hour clock chime point) Assuming that a  
repository sweep with no updates takes 30 seconds to complete then the  
server would have an average load of some 90 concurrent sync sessions.  
If there is a local rekey then the refresh would also imply a reload  
of all the signed products at this repository publication point.  
Assuming that this would then take 3 minutes to download, then the  
rekey load per server would be of the order of 540 concurrent rsync  
sessions as an average load. These load numbers appear to me to be  
somewhat large.

 From the relying party's perspective if there are 30,000 distinct  
RPKI repository publication points, and a serial form of local  
synchronisation using a top-down tree walk then the same set of  
assumptions imply that the relying party's perspective then it needs  
to process the synchronisation with the remote cache (including  
minimally the manifest crypto calculation at a rate of 3 per second.  
Assuming that there are 200,000 distinct ROAs out there that are re- 
validated at each fetch then once more the numbers imply that a 3 hour  
refresh would infer that the relying party would need to validate  
200,000 ROAS in 10,800 seconds. That probably needs some pretty quick  
hardware.

These numbers are pretty much a toss at a dart board, and the draft's  
authors' may well be using a different scale model to justify this  
recommended time cycle. What numbers did you have in mind Matt that  
would make this "SHOULD" 3 hour refresh cycle feasible in a big-I  
Internet scenario of universal use?

Geoff

WG Co-Chair hat off

---

From: Sandra Murphy <[email-masked]>
Date: Mon, 26 Oct 2009 19:52:57 -0400 (Eastern Daylight Time)

Geoff, I thought the reason for using (and mandating) rsync was precisely 
to avoid re-load of the whole data space on each synchronization.

If so, what estimates would you use of how much of the space would be new 
(what Matt calls "fresh") data at each synchronization time point?

--Sandy

(This is a clarification query, which might be viewed as co-chair hat on 
or member hat on, or both, your choice.)

On Tue, 27 Oct 2009, Geoff Huston wrote:

> WG Co-Chair Hat OFF
>
> Hi Matt,
>
>
>
>> entities who are actually using RPKI data for routing SHOULD be fetching 
>> fresh data from the repositories at least once every three hours.
>> 
>
> 3 hours?
>
> From a server's perspective if there are 30,000 AS's out there and each is 
> running a local cache and each is a distinct relying party of the RPKI 
> system, then the local hit rate at the server would be 3 per second, assuming 
> that all the relying parties evenly spread their load (which is a pretty wild 
> assumption - the worst case is that all 30,000 attempt to resync at the 3 
> hour clock chime point) Assuming that a repository sweep with no updates 
> takes 30 seconds to complete then the server would have an average load of 
> some 90 concurrent sync sessions. If there is a local rekey then the refresh 
> would also imply a reload of all the signed products at this repository 
> publication point. Assuming that this would then take 3 minutes to download, 
> then the rekey load per server would be of the order of 540 concurrent rsync 
> sessions as an average load. These load numbers appear to me to be somewhat 
> large.
>
> From the relying party's perspective if there are 30,000 distinct RPKI 
> repository publication points, and a serial form of local synchronisation 
> using a top-down tree walk then the same set of assumptions imply that the 
> relying party's perspective then it needs to process the synchronisation with 
> the remote cache (including minimally the manifest crypto calculation at a 
> rate of 3 per second. Assuming that there are 200,000 distinct ROAs out there 
> that are re-validated at each fetch then once more the numbers imply that a 3 
> hour refresh would infer that the relying party would need to validate 
> recommended time cycle. What numbers did you have in mind Matt that would 
> make this "SHOULD" 3 hour refresh cycle feasible in a big-I Internet scenario 
> of universal use?
>
>
> Geoff
>
> WG Co-Chair hat off
>
>
>
>
>
> _______________________________________________
> sidr mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/sidr

---

From: Terry Manderson <[email-masked]>
Date: Mon, 26 Oct 2009 21:40:19 -0700

> to avoid re-load of the whole data space on each synchronization.

The same thing can be achieved with manifests and any download protocol
would suffice. (one might also hope that the download protocol provides a
reasonable protection from MiTM attacks (sorry, I digress) ;)

> (what Matt calls "fresh") data at each synchronization time point?

Wouldn't that depend on the frequency of change & publication adopted by the
issuing CA in its CPS, and indeed how much change is expected at that CA?

This, I think, is a really fuzzy area - a more frequent time looks like a
nicer 'catch-all' for aspects of CA cycles or local time-zone differences of
(say) midnight. However, I feel less than convinced that specifying an
actual frequency is warranted. My recommendation would be for the relying
party to assess it's own fetch cycle time on how it wishes for RPKI
adjustments to affect its own routing system.

Regardless of the adopted cycle time, the repositories are just going to
have to scale to meet it, be that 24hrs, 12hrs, 3hrs, or 30 mins (in the
extreme).

Cheers
Terry

---

From: Geoff Huston <[email-masked]>
Date: Tue, 27 Oct 2009 16:04:31 +1100

WG Chair Hat Off

On 27/10/2009, at 10:52 AM, Sandra Murphy wrote:

> Geoff, I thought the reason for using (and mandating) rsync was  
> precisely to avoid re-load of the whole data space on each  
> synchronization.

I thought so too.

>
> If so, what estimates would you use of how much of the space would  
> be new (what Matt calls "fresh") data at each synchronization time  
> point?
>

The numbers I guessed at below assumed rsync-like behaviour.

An alternative approach would be based on a relying party retrieving  
the manifest of the publication point and performing the comparison of  
the local store state with the repository based on the manifest data.  
At the level of approximation I was using to query this suggested  
refresh frequency I do not think that it would make very much  
difference to the load guesstimates.

Geoff

(This is a clarification response, which is not made in my role as a  
co-chair.)

> --Sandy
>
> (This is a clarification query, which might be viewed as co-chair  
> hat on or member hat on, or both, your choice.)
>
>>> fetching fresh data from the repositories at least once every  
>>> three hours.
>>
>> 3 hours?
>>
>> each is running a local cache and each is a distinct relying party  
>> of the RPKI system, then the local hit rate at the server would be  
>> 3 per second, assuming that all the relying parties evenly spread  
>> their load (which is a pretty wild assumption - the worst case is  
>> that all 30,000 attempt to resync at the 3 hour clock chime point)  
>> Assuming that a repository sweep with no updates takes 30 seconds  
>> to complete then the server would have an average load of some 90  
>> concurrent sync sessions. If there is a local rekey then the  
>> refresh would also imply a reload of all the signed products at  
>> this repository publication point. Assuming that this would then  
>> take 3 minutes to download, then the rekey load per server would be  
>> of the order of 540 concurrent rsync sessions as an average load.  
>> These load numbers appear to me to be somewhat large.
>>
>> assumptions imply that the relying party's perspective then it  
>> needs to process the synchronisation with the remote cache  
>> (including minimally the manifest crypto calculation at a rate of 3  
>> per second. Assuming that there are 200,000 distinct ROAs out there  
>> that are re-validated at each fetch then once more the numbers  
>> imply that a 3 hour refresh would infer that the relying party  
>> would need to validate 200,000 ROAS in 10,800 seconds. That  
>> probably needs some pretty quick hardware.
>>
>> These numbers are pretty much a toss at a dart board, and the  
>> draft's authors' may well be using a different scale model to  
>> justify this recommended time cycle. What numbers did you have in  
>> mind Matt that would make this "SHOULD" 3 hour refresh cycle  
>> feasible in a big-I Internet scenario of universal use?
>>
>>
>> Geoff
>>
>> WG Co-Chair hat off
>>
>>
>>
>>
>>

---

From: Robert Loomans <[email-masked]>
Date: Tue, 27 Oct 2009 16:06:21 +1000

> have to scale to meet it, be that 24hrs, 12hrs, 3hrs, or 30 mins (in the

... and there will always be jokers who put * * * * * in their crontab
and run it every minute...

However, if the suggested best practice (and the default installed by
relying party tools) is 12 hours, say, then a large percentage of
installations will be using 12 hours, because people don't bother to
change defaults.

Rob

-- 
Robert Loomans                         email:       [email-masked]
Senior Software Engineer, APNIC        sip:    [email-masked]
http://www.apnic.net/                  phone:         +61 7 3858 3100

---

From: Matt Lepinski <[email-masked]>
Date: Tue, 27 Oct 2009 16:28:57 -0400

Geoff,

I'm happy to accept that the new wording is poor, but I'm pretty sure 
the old wording was also bad, and I think this discussion is important.

The old wording could easily be interpreted to suggest that once per day 
was the correct frequency for pulling from a repository. (That is, I 
believe the previous version was making a de facto recommendation for a 
default behaivor of one pull every 24 hours ... there wasn't a RECOMMEND 
in the text, but we all know that examples tend to be normative in this 
type of document.)

1) So the first implicit question is: Should the working group be making 
a recommendation as to the frequency with which a relying party pulls 
from the repository?

Or equivalently: Is there a "wrong" frequency that people might use if 
we didn't give them any guidence?

It seems that retreiving updates "too frequently" (e.g., every 5 
minutes) strains the repository system and that retreiving updates "too 
infrequently" (e.g., monthly) means that when I inject a new ROA into 
the system, it will take "unacceptably long" for this information to 
propogate to the relying parties that make use of this information. 
Therefore, we should  have text in the document that articulates some 
middle ground that we believe is reasonable for the Internet. (I make no 
claims that the current text in the document achieves this goal.)

2) The second question is: If we make a recommendation regarding 
frequency with which relying parties should pull updates, what frequency 
should we recommend.

Here, I understand that "everyone hitting the repository system at once" 
is a bad outcome regardless of the frequency that we recommend. That is, 
regardless of whether we recommend "once per day", "once per month", or 
"eight times daily" we will likely see problems with too much server 
load at midnight. If anyone can recommend text to avoid this phenomena 
(i.e., to encourage people to spread out their queries to the repository 
system), please send text.

I agree that there are roughly 30,000 AS numbers visible in BGP, so it's 
reasonable to assume on the order of 30,000 relying parties who will be 
routinely querying the repository system. We might also assume that 
30,000 is a reasonable order of magnitude for the number of CAs in the 
RPKI (we might easily average 2 CAs per AS, but surely not 10 CAs per AS).

However, one thing that wasn't clear from reading your analysis was how 
many CAs a given repository server would be hosting. If a server run by 
a large ISP or an RIR was providing a cache of all RPKI data, then 
clients would have longer connections to this server (as they could 
retrieve much of the data they need in one place), but they would be 
unlikely to receive requests from all 30,000 relying parties (e.g. an 
ISP might provide a complete cache for their customers but for 
non-customers they would typically only serve data for which they are 
authoritative). Alternatively, if a server is only serving data for a 
small subset of the CAs in the RPKI, then it might receive requests from 
all relying parties, but those sessions would tend to be short 
(especially when nothing has changed).

In any case, I believe the way forward (with regards to server load) is 
to answer the question, "How many simultaneous connections are 
reasonable for a server that hosts publication points for X CAs?" and 
then work backwards from there to determine if a given interval of 
relying party requests is reasonable from the server standpoint. I admit 
that I haven't completely thought through re-key, but I'll try to dig up 
some rough connection-time numbers based on our relying party software, 
and do a few back-of-the envolope computations.

With regards to client load, I'm not convinced that there's any problem 
with frequent queries to the repository system. If the relying party 
queries a publication point and rsync determines that nothing has 
changed, then no changes are required to ethe relying party's local 
cache and no cryptographic calculations are required. If something has 
changed, then the relying party has to perform validation (which 
includes cryptographic signature verification) on the manifest and any 
new objects that have been added. (Additionally, there may be resulting 
changes to the client's local cache ... e.g., if a new CRL revokes a 
previously valid certificate ... but such changes don't require new 
cryptographic computations, and so I believe the bottleneck is going to 
be the one or two signature verifications per object changed [1]). Now 
the point from the relying party side is that if 5,000 manifests change 
and 10,000 signed objects are added to the repository system on a given 
day, then the relying party needs to do roughly 30,000 signature 
verifications regardless of whether it learns of all these changes at 
once, or whether it learns of them in small batches throughout the 
course of the day. Therefore, I don't see how making frequent checks for 
new data has a significant impact on the relying party's processing load.

Finally, in addition to server and relying party processing loads, one 
must also look at the benefit of frequent repository fetches. Keep in 
mind, that a relying party has no way of distinguishing the following 
two events: (A) a route advertisement is originated by an AS that is 
authorized to advertise the route, but the relying party hasn't fetched 
recently enough to obtain the new ROA; and (B) a route advertisement is 
originated by an unauthorized entity that is attempting to hijack 
address space. In this discussion, it is also important to note that 
manifests can gaurantee that the relying party received all signed 
objects that existed at the moment that the manifest was published 
(i.e., a manifest can detect malicious deletion of data from a 
repository or corruption of data in transit) but the manifest says 
nothing about data that may have been added since the manifest was 
issued. This is why there is benefit in a relying party going back to 
the publication point perioidically to see whether a new manifest has 
been issued.

In any case, it's good to know that we'll have plenty to talk about in 
Hiroshima.

- Matt Lepinski

Geoff Huston wrote:

> second, assuming that all the relying parties evenly spread their 
> load  (which is a pretty wild assumption - the worst case is that all 
> 30,000  attempt to resync at the 3 hour clock chime point) Assuming 
> that a  repository sweep with no updates takes 30 seconds to complete 
> then the  server would have an average load of some 90 concurrent sync 
> sessions.  If there is a local rekey then the refresh would also imply 
> a reload  of all the signed products at this repository publication 
> point.  Assuming that this would then take 3 minutes to download, then 
> the  rekey load per server would be of the order of 540 concurrent 
> rsync  sessions as an average load. These load numbers appear to me to 
> be  somewhat large.
>
> validated at each fetch then once more the numbers imply that a 3 
> hour  refresh would infer that the relying party would need to 
> validate  200,000 ROAS in 10,800 seconds. That probably needs some 
> pretty quick  hardware.
>

---

From: Geoff Huston <[email-masked]>
Date: Wed, 28 Oct 2009 07:59:30 +1100

WG Co-chair hat OFF

Matt,

>
> I'm happy to accept that the new wording is poor, but I'm pretty  
> sure the old wording was also bad, and I think this discussion is  
> important.

Please do not misunderstand me - I was not implicitly expressing a  
preference for the old wording when I questioned the change you've  
made to this version draft. What I was wondering about was the  
viability in terms of both server load and relying party actions in a  
fully populated system if the relying parties implemented such a  
recommendation with a three hour refresh cycle time, and I was  
wondering about the query load on the server under an idealized load  
model and the viability of the relying party to complete a sweep of a  
fully populated and fully distributed repository publication system   
within a 3 hour time period.

You have raised some good questions in your note:

1) So the first implicit question is: Should the working group be  
making a recommendation as to the frequency with which a relying party  
pulls from the repository?

2) The second question is: If we make a recommendation regarding  
frequency with which relying parties should pull updates, what  
frequency should we recommend.

And perhaps I could add a third, on a more procedural note:

3) Should such operational recommendations be made in an architecture  
document or collected together in an operational guidelines document?

The original rough guesstimates about load in my response to your  
original note to the WG were just that, and doubtless an answer to  
question 2 would benefit form some more detailed analysis of a number  
of possible deployment scenarios, as you have also noted. Whether the  
recommendation needs to encompass a worst case deployment scenario, or  
to what extent any such recommendation is affected by the nature of  
the deployed environment of certificate and signed object publication  
points probably merits some study in the WG as well.

The "currency" of the data and the "responsiveness" of the system of  
changes are to my mind important topics. Given that the repository  
publication system is passive, in that new data cannot be pushed to  
replying parties, but instead relies on the actions of replying  
parties to discover a change, then the recommendations to relying  
parties, and the associated implications of overall system load are  
all relevant considerations here.

     Geoff

   WG Co-chair hat OFF

---

From: George Michaelson <[email-masked]>
Date: Wed, 28 Oct 2009 10:44:59 +1000

Matt, If you don't mind I'd like to add some input to this discussion  
too.

On 28/10/2009, at 6:28 AM, Matt Lepinski wrote:

> The old wording could easily be interpreted to suggest that once per  
> day was the correct frequency for pulling from a repository. (That  
> is, I believe the previous version was making a de facto  
> recommendation for a default behaivor of one pull every 24 hours ...  
> there wasn't a RECOMMEND in the text, but we all know that examples  
> tend to be normative in this type of document.)
>
> making a recommendation as to the frequency with which a relying  
> party pulls from the repository?

I think we have to think about it, yes. But in what document? Why in  
an Architecture document, this close to 'closure' ?
>
> Or equivalently: Is there a "wrong" frequency that people might use  
> if we didn't give them any guidence?
>
> minutes) strains the repository system and that retreiving updates  
> "too infrequently" (e.g., monthly) means that when I inject a new  
> ROA into the system, it will take "unacceptably long" for this  
> information to propogate to the relying parties that make use of  
> this information. Therefore, we should  have text in the document  
> that articulates some middle ground that we believe is reasonable  
> for the Internet. (I make no claims that the current text in the  
> document achieves this goal.)
>

The goal set earlier on in the life of this project was to stabilize  
the system in two complete 24h work cycles of the repository system as  
a whole. And yes, this was predicated on a MINIMUM of one fetch per  
24h per RP.

At least, thats what I understood. Maybe I was wrong?

If certification-active 'producers' approach each other in the  
provisioning process at least 4 times a day, and this is suitably  
spread into 24h, I believe that we satisfy this 2x 24h cycle time  
goal, for relying parties, for the expected 'depth' maximums in the  
observed allocation hierarchy today.

I have not seen any discussion which suggests either we're changing  
from 2 x 24h relying party cycles, or from an expected depth of at  
worst 8 levels of delegation. (there might well be deeper trees. I  
expect that they lie in the margins of global routing. I also believe  
now that the tree will be far shallower, for the overwhelming majority  
of the space)

So, I don't understand why we've moved to 3 hourly cycle time for RPs.  
I don't see where this came from.

>
> Here, I understand that "everyone hitting the repository system at  
> once" is a bad outcome regardless of the frequency that we  
> recommend. That is, regardless of whether we recommend "once per  
> day", "once per month", or "eight times daily" we will likely see  
> problems with too much server load at midnight. If anyone can  
> recommend text to avoid this phenomena (i.e., to encourage people to  
> spread out their queries to the repository system), please send text.

Well, the discussed approach informally has been to use a simple  
randomization mechanism to select a cron time in the 24h window, such  
that people choose a start time at random, and then cycle in a  
reasonable sub-multiple of that time.

This has worked for other processes historically.

Here is one example, from a google search:

	http://www.moundalexis.com/archives/000076.php

Its not directly what I mean, but I am sure people can go back into  
their UUCP dial-sync days, or USENET call memory-stack, and remember  
approaches for doing this.

Or, a provider could publish a simple 'pick a token' system and advise  
RPs to go there, and be assigned a random slot for fetch to balance  
load.

I don't think we can recommend a fetch cycle period until we  
understand the dynamics of change in the repository.

So I'd pose questions at this time:

	what rate of delay against a change in the repository is acceptable  
to an RP?

	what impact does depth in the tree have on change?

	how many cycles of update are allowed to pass before the tree is  
considered 'stable' against significant change?

	is this in fact not deterministic but depends on context  which is  
out of scope?

>
> I agree that there are roughly 30,000 AS numbers visible in BGP, so  
> it's reasonable to assume on the order of 30,000 relying parties who  
> will be routinely querying the repository system. We might also  
> assume that 30,000 is a reasonable order of magnitude for the number  
> of CAs in the RPKI (we might easily average 2 CAs per AS, but surely  
> not 10 CAs per AS).

Certainly when I modelled a large tree, these numbers were of the  
order of magnitude I modelled. I think I did around 10,000 distinct CAs.

>
> However, one thing that wasn't clear from reading your analysis was  
> how many CAs a given repository server would be hosting. If a server  
> run by a large ISP or an RIR was providing a cache of all RPKI data,  
> then clients would have longer connections to this server (as they  
> could retrieve much of the data they need in one place), but they  
> would be unlikely to receive requests from all 30,000 relying  
> parties (e.g. an ISP might provide a complete cache for their  
> customers but for non-customers they would typically only serve data  
> for which they are authoritative). Alternatively, if a server is  
> only serving data for a small subset of the CAs in the RPKI, then it  
> might receive requests from all relying parties, but those sessions  
> would tend to be short (especially when nothing has changed).

RIRs have of the order 2000-5000 direct entities who they will be  
issuing CA certificates to. If they run hosted portals, its numbers of  
this magnitude which potentially could seek to use a hosted portal for  
EE certificate facing activity.

I don't believe individual entities below the RIR level face  
administering CA spaces of this complexity. I think they are more  
probably under the 100 to 1000 CA/EE level.

If people believe they want to host third party solutions, then I  
think the size lies anywhere between the 1-10 and the 30,000 level.  
"it depends"

Aggregated repositories would of course have different dynamics of  
change.

>
> In any case, I believe the way forward (with regards to server load)  
> is to answer the question, "How many simultaneous connections are  
> reasonable for a server that hosts publication points for X CAs?"  
> and then work backwards from there to determine if a given interval  
> of relying party requests is reasonable from the server standpoint.  
> I admit that I haven't completely thought through re-key, but I'll  
> try to dig up some rough connection-time numbers based on our  
> relying party software, and do a few back-of-the envolope  
> computations.

This would be good. But, I don't think it goes to an architecture  
question. I think it would be far better to do this work, and draft an  
operational guidelines document instead which puts this experience  
into a context which is more ameanable to change over time.

>
> With regards to client load, I'm not convinced that there's any  
> problem with frequent queries to the repository system. If the  
> relying party queries a publication point and rsync determines that  
> nothing has changed, then no changes are required to ethe relying  
> party's local cache and no cryptographic calculations are required.

Please bear in mind that some amount of held state exists server side  
to determine what has changed.   It might be a file-system walk and on- 
demand checks on files, for stat()
information.

it might be a simple DB lookup. But in either case, its not 'for free'  
-some level of work is being done.

And, held TCP/IP session state.

Is this of the same order of magnitude as running a web server? I  
think that it very probably is. Particularly ones which are doing  
dynamic content serve, rather than simple cached state.

Their solutions (million-chickens, reverse caches, memcached ...) are  
probably applicable here.

> If something has changed, then the relying party has to perform  
> validation (which includes cryptographic signature verification) on  
> the manifest and any new objects that have been added.  
> (Additionally, there may be resulting changes to the client's local  
> cache ... e.g., if a new CRL revokes a previously valid  
> certificate ... but such changes don't require new cryptographic  
> computations, and so I believe the bottleneck is going to be the one  
> or two signature verifications per object changed [1]). Now the  
> point from the relying party side is that if 5,000 manifests change  
> and 10,000 signed objects are added to the repository system on a  
> given day, then the relying party needs to do roughly 30,000  
> signature verifications regardless of whether it learns of all these  
> changes at once, or whether it learns of them in small batches  
> throughout the course of the day. Therefore, I don't see how making  
> frequent checks for new data has a significant impact on the relying  
> party's processing load.

I'm also unsure from this sum what the benefit of frequent checks was.  
Is it faster completion of the cycle time around re-key? Whats it  
benefiting?

>
> Finally, in addition to server and relying party processing loads,  
> one must also look at the benefit of frequent repository fetches.  
> Keep in mind, that a relying party has no way of distinguishing the  
> following two events: (A) a route advertisement is originated by an  
> AS that is authorized to advertise the route, but the relying party  
> hasn't fetched recently enough to obtain the new ROA; and (B) a  
> route advertisement is originated by an unauthorized entity that is  
> attempting to hijack address space. In this discussion, it is also  
> important to note that manifests can gaurantee that the relying  
> party received all signed objects that existed at the moment that  
> the manifest was published (i.e., a manifest can detect malicious  
> deletion of data from a repository or corruption of data in transit)  
> but the manifest says nothing about data that may have been added  
> since the manifest was issued. This is why there is benefit in a  
> relying party going back to the publication point perioidically to  
> see whether a new manifest has been issued.
>
> In any case, it's good to know that we'll have plenty to talk about  
> in Hiroshima.

Yes. I think there is a discussion here.

If we could abstract these questions to a yet-to-be-done Operations  
document, I think we could continue to talk about Architecture  
documents as plausibly 'done' and this means we can progress WGLC.

What do you think?

-George

---

From: Roque Gagliano <[email-masked]>
Date: Wed, 28 Oct 2009 11:25:57 -0200

Hi George/Matt,

On Oct 27, 2009, at 10:44 PM, George Michaelson wrote:
> the system in two complete 24h work cycles of the repository system  
> as a whole. And yes, this was predicated on a MINIMUM of one fetch  
> per 24h per RP.
>

That was my understanding too and that is the requirement we  
considered when building our system. We are populating the repository  
4 times a day.

I believe we are too early in the game to tighter operational  
practices, particularly in a document that we are about to ship out.

As Robert points out, applications will be shipped with a default  
fetching cycle independently of what a particular CPS mentions. We  
will have to live with this...

Roque.

---

From: Robert Kisteleki <[email-masked]>
Date: Wed, 28 Oct 2009 15:27:37 +0100

Hi,

> system), please send text.

Keep in mind that "midnight" is a relative term, so this is not as bad as it 
sounds.

> RPKI (we might easily average 2 CAs per AS, but surely not 10 CAs per AS).

Do you have any supporting evidence to this claim "but surely not 10 CAs per 
AS"?

> and do a few back-of-the envolope computations.

There are other inputs to this calculation, not just the number of CAs. For 
example, the number of certificates issued per CA, and in close relation, 
the frequency with which the set of issued certificates changes, can vary 
wildly (for a potential single root it's probably a few, for an RIR it's 
thousands, for an LIR/ISP it's likely only a few).

> cache and no cryptographic calculations are required. If something has 

While this in itself is true, you have to keep in mind that manifests and 
CRLs have to be regularly re-issued, so even if nothing substantial has 
changed, you do have to validate objects (and _then_ you'll realize that 
nothing really changed).

> new data has a significant impact on the relying party's processing load.

That depends on how expensive the null operation is (ie. rsync figuring out 
that nothing has changed). I have no idea, but for a popular repository, 
such as an RIR's might be, it is probably not trivial. So experiencing it 
less frequently is useful.

> In any case, it's good to know that we'll have plenty to talk about in 

Indeed!

Robert

---

From: Randy Bush <[email-masked]>
Date: Thu, 29 Oct 2009 08:32:23 +0900

folk going down this rathole might consider two things

  rpki-rtr suggests that the number of global fetchers will be radically
  lest than the number of global asns

  there might be ca chain depth of 3-6 for which a 24 hour cycle would
  mean a three day delay, making operators remember curtis unfondly

randy

---

From: George Michaelson <[email-masked]>
Date: Thu, 29 Oct 2009 09:54:49 +1000

>  mean a three day delay, making operators remember curtis unfondly

We modelled producer timing on a particular view of the world. I don't  
have a problem that the view of the world has changed, and that  
consumer-side timing issues have to be re-considered.

I *think* that even bearing them in mind, the producer side modelling  
is still probably valid.

If anyone thinks that producers need to be advised to rendezvous more  
than 4x in a 24h period, I think it would help to know. I don't think  
we're in crisis point for it going up btw, but I do think that the  
cost of processing will come into the limits on this at some point.

In any case, I think abstracting this into operational practice  
statements is better than wedging it into architecture.

There is an aspect of the times for re-publication which are published  
in certificates and manifests and CRLS defining the lower bound:  
everyone has to go fetch inside a cycle time which will respect them,  
or wear the consequences for things falling out-of-age.

But the upper bound sanity check, the 'avoid excess work' isn't so  
clear to me right now.

-G

---

From: Robert Kisteleki <[email-masked]>
Date: Thu, 29 Oct 2009 08:44:10 +0100

>   mean a three day delay, making operators remember curtis unfondly

Randy, could you elaborate in which case does this transitive property 
apply, that makes something longer to propagate on a deeper hierarchy? Is it 
rekey, re-issue, revocation, or some kind or relying party check?

Thanks,
Robert

---

From: Randy Bush <[email-masked]>
Date: Thu, 29 Oct 2009 21:30:18 +0900

> rekey, re-issue, revocation, or some kind or relying party check?

as ggm said, probably better than i can, a week or two after philly.

if we have a parent-child chain of length L and each runs as a batch at
some time interval T, then the mean time to propagate is (T/2)*(N-1)

randy

---

From: Randy Bush <[email-masked]>
Date: Thu, 29 Oct 2009 21:33:03 +0900

sorry.  late here.

> some time interval T, then the mean time to propagate is (T/2)*(N-1)

s/N/L/

randy

---

From: Robert Kisteleki <[email-masked]>
Date: Thu, 29 Oct 2009 13:38:58 +0100

> some time interval T, then the mean time to propagate is (T/2)*(N-1)

That is only true if the thing you're propagating has to travel hop by hop 
to the bottom of the hierarchy. So the question still stands: what is this 
"thing" that you think propagates slowly, and why does it have to propagate 
hop by hop?

I don't mean this as a quiz; so far the use cases I could come up with do 
not have this property, so I'm wondering if such a thing really exists.

Robert

---

From: Randy Bush <[email-masked]>
Date: Fri, 30 Oct 2009 07:15:40 +0900

> That is only true if the thing you're propagating has to travel hop by
> hop to the bottom of the hierarchy. So the question still stands: what
> is this "thing" that you think propagates slowly, and why does it have
> to propagate hop by hop?

incorrect or missing cert high in chain which needs delegation fix down
the chain so end site can get a fixed roa out there.  remember, my
routing depends on that roa.  and arin has extreme cases of depth,
though i think we will find itu-rir-<three or four> to be the more
common cases.

it's max time to repair which worries me.

randy

---

From: Danny McPherson <[email-masked]>
Date: Sat, 31 Oct 2009 08:07:54 -0600

One might suspect there are some things we can learn from
this:

http://tools.ietf.org/html/rfc4641

In particular, when *contrasting* even initial operational
practices with preceding recommendations made in theoretical
environments, a disparity usually emerges.

I share concerns about frequency of publication and fetching
functions.  In particular, this was one of the larger reasons
why folks stopped using IRR-based prefix lists in practice.  It's
a real PITA when intermediate ASes haven't updated policies for
newly announced prefixes in a stateful protocol such as BGP.  The
originator announces the route, the route is dropped by peers
because it's not permitted in the import policy, then the policy
is updated eventually after corresponding IRR objects were fetched
and the route is now permitted in policy, but stateful BGP doesn't
reannounce the route, so the route had to be "bounced" somehow
along the path, or at the origin, or a session reset to trigger
a new update once the policy has been updated.

Of course, we have BGP route refresh today, and implementations
arguably should store an "invalid" route in an Adj-RIB-In and
denote as such - but that does introduce cost (and a potential
attack vector) as well.  As Randy pointed out, and Curtis and I
have stated (and lived firsthand together :-/)  many times, the
ripple effects associated with disjoint and autonomous application
of policy derived from IRR (or RPKI) like systems are quite an
operations nightmare in practice, and should be given heavy
consideration here.

-danny

---

From: Jared Mauch <[email-masked]>
Date: Sat, 31 Oct 2009 12:22:49 -0400

> operations nightmare in practice, and should be given heavy

I certainly echo these comments and concerns.

- Jared