Subject: [sidr] Probable bug in rsync and non-normative file names in repo doc

From: Tim Bruijnzeels <[email-masked]>
Date: Tue, 21 Dec 2010 15:13:07 +0100

Hi all,

we noticed a possible problem when using the, non-normative, file name 
strategy suggested in the repos-structure document section 2.2. I don't 
think the suggestion should be changed, but operationally the following 
may be of interest...

There seems to be bug in rsync when you try retrieve a single file that 
start with a '-' (minus) character ('-' is part of the file system safe 
base64 set). It seems this is related to the rsync daemon 
implementation, but we have not yet followed up with the rsync 
mailinglist (could not find anything in bugtracker):

rsync: on remote machine: -SndBLVaDchzm60h4WPrfNiVu5k.cer: invalid 
numeric value
rsync error: requested action not supported (code 4) at 
clientserver.c(841) [sender=3.0.7]
rsync: read error: Connection reset by peer (54)
rsync error: error in rsync protocol data stream (code 12) at 
/SourceCache/rsync/rsync-40/rsync/io.c(603) [receiver=2.6.9]

Note that this probably does not affect the validation implementation 
for BBN and rcynic. Recursive downloads are not affected. Our validator 
supports bottom-up validation as well and tries to fetch all objects in 
the chain individually, so this functionality is affected.

 From our side we are putting a work-around in place, for the time 
being, that will prepend any filename that would start with '-' with a 
'1' character. If you are implementing software that produces 
repositories you may want to prevent having file names starting with '-' 
as well.

In general one can never be sure that RPs will not try to fetch files 
individually and if this is a bug in rsync (as it seems to be) than even 
a fix to rsync may not be good enough as people out there may use old 
versions (impact also depends on whether the problem is server or client 
side..)

If anyone has any other ideas (or experience) on this, much appreciated. 
In the meantime we will try to follow with the rsync developers. Is 
there anyone on this list who could quickly verify this with them? I 
don't mind using the regular bug-tracker but if someone knows these 
people that might speed up matters. If we have more news on this we'll 
let the list know..

Regards
Tim Bruijnzeels

---

From: Tim Bruijnzeels <[email-masked]>
Date: Wed, 22 Dec 2010 16:12:34 +0100

> mailinglist (could not find anything in bugtracker):

We did some more testing and as it turns out this problem only occurs 
when a file starting with '-' is in the top of the repository:

/Users/tim $ rsync --list-only 'rsync://xxxx/repository/-fails'
rsync: connection unexpectedly closed (0 bytes received so far) [receiver]
rsync error: error in rsync protocol data stream (code 12) at 
/SourceCache/rsync/rsync-40/rsync/io.c(452) [receiver=2.6.9]

/Users/tim $ rsync --list-only 'rsync://xxxx/repository/rsync-bug/-works'
-rw-r--r--         218 2010/12/22 10:11:12 -works

I raised the issue on the rsync list. I can let this list know if it 
gets resolved..

In the meantime you may still want to avoid having '-' as a starting 
character, especially for your TA since everyone will try to download 
that direct. Other, probably better, work-around is to make sure that 
you don't publish to the actual top of the repository as configured in 
rsyncd.conf, but one directory below. This is better as discarding one 
in 64 keys essentially degrades security.. it's not really 2048 bit that 
way..

Cheers
Tim

---

From: Tim Bruijnzeels <[email-masked]>
Date: Wed, 29 Dec 2010 11:31:31 +0100

Hi,

update as promised..

> I raised the issue on the rsync list. I can let this list know if it

it's not 100% clear to me whether the problem is client side or server 
side, but it's _somewhere_ in the communication between the two. Word on 
the wire is that it may be fixed, but there is no ETA.

Having said that. In the RPKI one cannot really be sure that everyone 
will be using the latest rsync versions for their validators / 
repositories. So I am not sure if a fix will really help operations..

There is however another work-around that will work: just prefix ./ in 
front of any file starting with '-':

rsync://host/repository/-file
becomes: rsync://host/repository/./-file

and then it does work. It's not a problem do the same for deeper levels:

rsync://host/repository/sub/-file
becomes: rsync://host/repository/sub/./-file

since you can't be sure in general as RP whether /repository or 
/repository/sub is the actual top of the repo.

We will update our validator to do this. Other validators that fetch 
individual files (eg a trust anchor referred to in a tal) may want to do 
something similar.

Cheers
Tim