Subject: Re: [sidr] this is possibly Tim Bruijnzeels delta protocol

From: Tim Bruijnzeels <[email-masked]>
Date: Mon, 22 Dec 2014 12:55:22 +0100

Hi all,

On 14 Nov 2014, at 21:38, Tim Bruijnzeels <[email-masked]> wrote:
> ...and I hope to post it to the wg within two weeks...

A little later than I was hoping for, but I just uploaded a revised version:
http://www.ietf.org/id/draft-tbruijnzeels-sidr-delta-protocol-03.txt

I would like to ask the working group to read this version and I would like to ask the chairs for a formal call for adoption of this work.

This version of the document is quite different from previous versions that you may have read. The basic principle is unchanged. The format of the protocol messages has been simplified quite a bit and reflects proof of concept implementations that Rob Austein and I have worked on, and tested interoperability on, over the last months. The supporting text (reasoning and text on usage of the protocol etc) has also been revised. Some elements of that text are definitely still up for discussion: e.g. when talking about how *long* a notification file may be cached. We tried to outline these discussions clearly in the document.

Regards,

Tim Bruijnzeels

---

From: David Mandelberg <[email-masked]>
Date: Tue, 23 Dec 2014 17:41:57 -0500

Yes, I know I'm an author, but I'm going to review this anyway.

Section 3.2.3: "The serial attribute must be an unbounded, unsigned 
positive integer indicating the current version of the repository." On 
the relying party side, unbounded integers make me a bit nervous. If the 
serial number is terabytes long, I'd really prefer to reject the entire 
file. Can we instead restrict the serial attribute to fit in a 64-bit 
unsigned integer? Changing sessions once every 2^64-1 serials doesn't 
seem like a significant burden.

(nit) Section 3.2.4 uses the term "version" when I think it means 
"serial".

In section 3.2.5, "A new delta file MUST be generated", but "The 
[notification] file SHOULD also include available delta files for this 
and previous updates". Is there any point to generating a delta file and 
not listing it in the notification file? I'd recommend changing both to 
MUST or both to SHOULD.

Section 3.4.3: "Including the hashes in this manner allows relying 
parties to identify specific objects by their hash rather than the URI 
where they are found." If some RPs use just the URI and some use just 
the hash, it might be possible for a cooperating CA and repository to 
cause the two types of RP to see two completely different views of the 
RPKI. A paranoid RP could check both the hashes and the URIs to make 
sure that never happens, but that defeats the purpose of having both 
identifiers. Is the flexibility here worth the potential risk?

-- 
David Eric Mandelberg / dseomn
http://david.mandelberg.org/

---

From: Tim Bruijnzeels <[email-masked]>
Date: Tue, 30 Dec 2014 17:29:33 +0100

On 23 Dec 2014, at 23:41, David Mandelberg <[email-masked]> wrote:

> Section 3.2.3: "The serial attribute must be an unbounded, unsigned positive integer indicating the current version of the repository." On the relying party side, unbounded integers make me a bit nervous. If the serial number is terabytes long, I'd really prefer to reject the entire file. Can we instead restrict the serial attribute to fit in a 64-bit unsigned integer? Changing sessions once every 2^64-1 serials doesn't seem like a significant burden.

I believe we have unbounded numbers elsewhere, like manifest serial numbers. But if this makes you nervous. If there is an update every second, it takes more than 500 billion years to hit 2^64. So for all practical purposes I am fine with restricting this. I think server implementations can get away with not checking this during our lifetime though ;)

As for clients rejecting a number that’s too long. Maybe we should specify a reasonable maximum size for this file instead? HTTP 1.1 does not support an "if-size-smaller-than" header as far as I know, but careful RPs could do a head request to figure out the "Content-Length" and "Last-Modified", and then decide to proceed or reject.

> (nit) Section 3.2.4 uses the term "version" when I think it means "serial".

ack

> 
> In section 3.2.5, "A new delta file MUST be generated", but "The [notification] file SHOULD also include available delta files for this and previous updates". Is there any point to generating a delta file and not listing it in the notification file? I'd recommend changing both to MUST or both to SHOULD.

sorry, significant typo here.. the first sentence should be: "A new *snapshot* file MUST be generated."

In theory this protocol can be used with only snapshots. Long term this may not be so useful, so if the WG would want to change this to "MUST support deltas" we could of course. Shorter term I think a "SHOULD" allows for a more incremental uptake of this.

I can imagine a poor man's implementation where a CA just writes its stuff to disk (like it might do today for rsync), and kicks of a process that scans the subfolders to construct a single snapshot file, without keeping track of what actually changed between versions. This would force RPs to retrieve the full snapshot file whenever an update happens. Then again.. even with this scenario comparing your snapshot files server side to figure out what the delta is after the fact should not be too difficult either.

Oh and the first serial would typically not have a delta. There is no previous serial to update from. We could insist that a delta is included that's effectively the same as the first snapshot, but that seems a bit artificial to me.

> Section 3.4.3: "Including the hashes in this manner allows relying parties to identify specific objects by their hash rather than the URI where they are found." If some RPs use just the URI and some use just the hash, it might be possible for a cooperating CA and repository to cause the two types of RP to see two completely different views of the RPKI. A paranoid RP could check both the hashes and the URIs to make sure that never happens, but that defeats the purpose of having both identifiers. Is the flexibility here worth the potential risk?

Note that we have similar trust problems with rsync.

Going forward section 4.2 has text about treating "object updates and withdraws with some skepticism". I think it may be useful to include URIs and hashes in withdraw messages and updates (publish with hash attribute for old), as hints, or in case of deltas because it's just convenient to include all publish and withdraws received in a single update from a CA in a single delta. But RPs should not blindly trust them. I don't think there is only one clear strategy to achieve this though, which is why the remaining text of 4.2 is rather informal.

We can really benefit from RP implementation experience here. My idea would actually be to keep a map of all relevant objects by their hash, ignore withdraws completely - only use the protocol to learn about new objects. URIs are just hints as well. I would rely fully on hashes in validated manifests to retrieve objects. I would delete these objects only because of space and efficiency concerns, and only if they have expired, or I see a new *valid* manifest where the object is removed. I am not convinced yet though about all the details - so I would like to have an implementation first. And even then I am not convinced that this is the only way to do it.. 

In short: I think we can find a way to implement this securely in the RIPE NCC validator. I would be happy to document the exact details in an informative draft for scrutiny and reference of that implementation. But I don't think we can mandate how to do this in general, and even if we could this delta document is probably not the place to do it.

Tim

> _______________________________________________
> sidr mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/sidr