Subject: Re: [sidr] sidr-arch-09 refresh cycle time

From: Sandra Murphy <[email-masked]>
Date: Wed, 4 Nov 2009 15:03:59 -0400 (Eastern Daylight Time)

I can understand these concerns.

Would you rather see the wg try:

(a) concentrated redesign (to what, I have no idea)

(b) operational guidance to operators of how to manage this risk - get new 
certs out <inserttimeframe> before the route that relies on it, etc. 
(There's already been discussion of an operational use document.)

(c) create a way to carry info inband for emergencies (transitively - 
inducing work even on those who do not yet understand during initial 
deployment).

Which of these (or something else) is the right wg direction, in your 
mind?

--Sandy

On Sat, 31 Oct 2009, Jared Mauch wrote:

> On Oct 31, 2009, at 10:07 AM, Danny McPherson <[email-masked]> wrote:
>
>> 
>> One might suspect there are some things we can learn from
>> this:
>> 
>> http://tools.ietf.org/html/rfc4641
>> 
>> In particular, when *contrasting* even initial operational
>> practices with preceding recommendations made in theoretical
>> environments, a disparity usually emerges.
>> 
>> I share concerns about frequency of publication and fetching
>> functions.  In particular, this was one of the larger reasons
>> why folks stopped using IRR-based prefix lists in practice.  It's
>> a real PITA when intermediate ASes haven't updated policies for
>> newly announced prefixes in a stateful protocol such as BGP.  The
>> originator announces the route, the route is dropped by peers
>> because it's not permitted in the import policy, then the policy
>> is updated eventually after corresponding IRR objects were fetched
>> and the route is now permitted in policy, but stateful BGP doesn't
>> reannounce the route, so the route had to be "bounced" somehow
>> along the path, or at the origin, or a session reset to trigger
>> a new update once the policy has been updated.
>> 
>> Of course, we have BGP route refresh today, and implementations
>> arguably should store an "invalid" route in an Adj-RIB-In and
>> denote as such - but that does introduce cost (and a potential
>> attack vector) as well.  As Randy pointed out, and Curtis and I
>> have stated (and lived firsthand together :-/)  many times, the
>> ripple effects associated with disjoint and autonomous application
>> of policy derived from IRR (or RPKI) like systems are quite an
>> operations nightmare in practice, and should be given heavy
>> consideration here.
>
> I certainly echo these comments and concerns.
>
> - Jared _______________________________________________
> sidr mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/sidr
>

---

From: Robert Kisteleki <[email-masked]>
Date: Thu, 05 Nov 2009 09:58:22 +0100

Randy Bush wrote:
>> That is only true if the thing you're propagating has to travel hop by
>> hop to the bottom of the hierarchy. So the question still stands: what
>> is this "thing" that you think propagates slowly, and why does it have
>> to propagate hop by hop?
> 
> incorrect or missing cert high in chain which needs delegation fix down
> the chain so end site can get a fixed roa out there.  remember, my
> routing depends on that roa.  and arin has extreme cases of depth,
> though i think we will find itu-rir-<three or four> to be the more
> common cases.
> 
> it's max time to repair which worries me.
> 
> randy

I understand this concern, but how likely is it that there is an "incorrect 
or missing cert high in chain" which needs fixing within one cycle?

In the case of missing resources, it's very unlikely that someone deep down 
in the chain suddenly realises "oh, I need more resource from my 
grand-grand-grandparent NOW!". In the case of overclaiming, it's enough if 
any one in the chain shrinks their resources, the effect is visible in one 
cycle.

So I'm not convinced that multi-cycle propagation this is an issue.

Robert

---

From: Randy Bush <[email-masked]>
Date: Thu, 05 Nov 2009 03:01:29 -0600

Robert Kisteleki wrote:
> I understand this concern, but how likely is it that there is an
> "incorrect or missing cert high in chain" which needs fixing within one
> cycle?
> 
> In the case of missing resources, it's very unlikely that someone deep
> down in the chain suddenly realises "oh, I need more resource from my
> grand-grand-grandparent NOW!". In the case of overclaiming, it's enough
> if any one in the chain shrinks their resources, the effect is visible
> in one cycle.

think someone high in chain issues cert no longer covering youtube.

randy

---

From: Robert Kisteleki <[email-masked]>
Date: Thu, 05 Nov 2009 10:04:52 +0100

> think someone high in chain issues cert no longer covering youtube.

So they did cover it for a while, then the resource was dropped from their 
cert, yes? That means that cert was revoked on that level, and a new one 
issued. Local issue, does not propagate, therefore can be fixed immediately.

Robert

---

From: Matt Lepinski <[email-masked]>
Date: Thu, 05 Nov 2009 15:08:12 -0500

With regards to the frequency with which relying parties fetch data from 
the repository system (be that authoritative publication points or 
caches operated by ISPs or RIRs), there appears to be a trade-off 
between "max time to fix a problem" vs "work required by the repository 
system and the relying parties". From this thread, it seems that there 
isn't an obvious right answer with regards to how often relying parties 
should query the repository system, but that there are risks associated 
with queries that are too frequent or too infrequent.

To me, it seems that the best way to address this problem is to publish 
an "operational practices" document of some sort that documents this 
(and other) operational concerns and recommends practices to mitigate 
such risks. [That is, Sandy's (b) option].

With regards to the architecture document (which I would very much like 
to see leave the working group in near future), I believe the best way 
forward is for the architecture document to be silent on this issue. For 
example, consider the following text for Section 6 (last paragraph on 
page 15 of the arch-09):

   "Note that since relying parties will perform these operations 
   regularly, it is more efficient for the relying party to request from 
   the repository system only those objects that have changed since the 
   relying party last updated its local cache. A relying party shall 
   choose the frequency with which it downloads and validates updates 
   from the repository system."

- Matt Lepinski

Sandra Murphy wrote:

> (b) operational guidance to operators of how to manage this risk - get 
> new certs out <inserttimeframe> before the route that relies on it, 
> etc. (There's already been discussion of an operational use document.)
>
> _______________________________________________
> sidr mailing list
> [email-masked]

---

From: Terry Manderson <[email-masked]>
Date: Thu, 5 Nov 2009 15:54:45 -0800

Matt,

I think this is sensible.

Terry

---

From: Rob Austein <[email-masked]>
Date: Sat, 07 Nov 2009 07:33:30 +0900

[Catching up on back mail while in transit to Hiroshima...]

At Tue, 27 Oct 2009 16:28:57 -0400, Matt Lepinski wrote:
> 
> Here, I understand that "everyone hitting the repository system at once" 
> is a bad outcome regardless of the frequency that we recommend. That is, 
> regardless of whether we recommend "once per day", "once per month", or 
> "eight times daily" we will likely see problems with too much server 
> load at midnight. If anyone can recommend text to avoid this phenomena 
> (i.e., to encourage people to spread out their queries to the repository 
> system), please send text.

FWIW: rcynic has a "jitter" mechanism which delays startup (and thus
the rsync hit on servers) by a pseudo-random amount of time in the
range [0..n], where n defaults to ten minutes.  The important part is
that this behavior is on by default, so a user has to read the
documentation (or at least the usage message) to disable it.  How much
it helps, and whether a different value of n would be better, I dunno,
but it was trivial to implement, and one could easily extend it
slightly to log dire warnings or flat out refuse to run for too-small
values of n.