Subject: Re: Fwd: ISSUE: Checkgroups control messages

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 22 Sep 2008 10:07:36 +0100

On Fri, 19 Sep 2008 21:32:33 +0100, Julien ÉLIE <[email-masked]>  
wrote:

> Hi Charles,
>
> Your address at clw.cs.man.ac.uk does not work for me:
>     Remote host said: 550 relay not permitted

That address has not worked for many years past.
>
>>> * If I have a hierarchy like this one:
>>>    fr.bienvenue
>>>    fr.test
>>> and if another administrator is in charge of the sub-hierarchy
>>> fr.autres, with for instance:
>>>    fr.autres
>>>    fr.autres.fiction
>>>    fr.autres.histoires
>>> how can he specify in the Control: header that his checkgroups
>>> contains fr.autres?
>>>    Control: checkgroups fr.autres #1
>>> means fr.autres.* and does not create fr.autres (which I do not
>>> want in my own hierarchy).
>>
>> That is correct. I think the term "sub-hiererchy" is defined as a set of
>> groups with a common prefix (ending in '.'), and so does not include its
>> own root. So it would be up to the administrator of fr.* to include
>> "fr.autres" in HIS checkgroups.
>
> I think USEPRO should not do so something which leads sub-hierarchies
> not to be self-managed:  they should not lay on the checkgroups of the
> main hierarchy in case the root of the sub-hierarchy is a valid newsgroup
> (which is not a root for the super-hierarchy).

Neither the fr.autres.* subhierarchy nor the group fr.autres can exist at  
all without the agreement of the administrators of fr.*, so I see no  
problem in agreeing with the fr.* administrator to handle the rare cases  
in which fr.autres needs some amendment. Too much copmplication in the  
chackgroups rules. So I do not want to change it (if any other USEPRO  
member wants it changed, then he will have to speak up).
>
>
>> The chkscope parameter was invented for the benefit of the de.alt.*
>> hierachy (which is used as an example in our draft). I don't think the
>> individual group de.alt actually exists, but the wording surrounding  
>> that
>> example seems to confirm my interpretation.
>
> I understand that the <chkscope> parameter was invented for de.alt.*
> but there is also cn.bbs.* in the same case.  (fido.* groups are
> almost defunct.)

Yes, I was aware of the similar cn.bbs.* case.

> And what strikes me most is that USEPRO sticks to the behaviour of
> de.alt.* (where de.alt is not a newsgroup) and do not try to be
> more general.
>
>
>>>   The <chksernr> argument may be any positive integer.
>>> Could it be better specified what should look like this integer?
>>> Especially, why not suggest something like DNS serial numbers
>>> #2008090701?  Why not limit the size of this integer (32 bits?)
>>> Suppose that I put a 448-digit number, I am not sure news servers
>>> will manage to save it and compare it with the next one.

>>
>> I wouldn't want to specify its format any further, but we might insist
>> that the integer should be less that 32768, or somesuch.
>
> I do not think it wise to limit this number to 32768 because one
> could not use DNS serial numbers (which are practical).
> 2^31=2147483648 would be better, at least.
> It could be said that implementations should except <chksernr> arguments
> to be a 32-bit integer.  And that the (unsigned) integer should be less
> than 2^32.
>

Yes, I agree that any number expressed in 32 bit would be better. Can Russ  
quietly slip it in?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131                       
   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: Russ Allbery <[email-masked]>
Date: Mon, 22 Sep 2008 02:33:38 -0700

"Charles Lindsey" <[email-masked]> writes:
> Julien ÉLIE <[email-masked]> wrote:

>> I do not think it wise to limit this number to 32768 because one could
>> not use DNS serial numbers (which are practical).  2^31=2147483648
>> would be better, at least.  It could be said that implementations
>> should except <chksernr> arguments to be a 32-bit integer.  And that
>> the (unsigned) integer should be less than 2^32.

> Yes, I agree that any number expressed in 32 bit would be better. Can
> Russ quietly slip it in?

It seems like a reasonable change to me, but I'd like to get more opinions
from the working group just to be sure.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

---

From: Thorfinn <[email-masked]>
Date: Tue, 23 Sep 2008 11:46:58 +1000

On 22 Sep 2008, at 19:33, Russ Allbery wrote:

>>> could

640kb should be enough for anyone. Um. :-)

That is, yes, 32 bit unsigned integer is better, not 15 bit.

-- 
<a href="http://tertius.net.au/~thorfinn/">[email-masked]</a>
The world is coming to an end.  Please log off.  -- BSD fortune file

---

From: Russ Allbery <[email-masked]>
Date: Mon, 22 Sep 2008 18:49:35 -0700

> That is, yes, 32 bit unsigned integer is better, not 15 bit.

Just to clarify, the current draft has no limit at all.  I think it's safe
and reasonable to add a limit of 2^32 - 1, but I'm always nervous about
adding limits on values because you never know when you might want them to
be larger later.

On the other hand, Julien has a good point that implementers aren't going
to add a bigint implementation just to check sequence numbers.

---

From: Thorfinn <[email-masked]>
Date: Tue, 23 Sep 2008 12:15:26 +1000

> to add a bigint implementation just to check sequence numbers.

*nod* 64 bit is becoming fairly prevalent, and that should certainly  
be enough.

That said - the DNS world is pretty used to using a 32 bit integer  
circle for comparison.

Looking at the current USEPRO text... *ponder* Serial number MUST  
increase.  That is definitely a problem - you certainly can run out of  
32 bit integer.  It's not all that likely, but it's certainly possible.

You're not likely to run out of 64 bit integer, provided you aren't  
silly and start somewhere near the very top of the integer space.

I'd be uncomfortable putting in a 32 bit limit unless we also change  
the text to deal with serial comparison in the same way that DNS  
serial numbers are compared.

 From RFC 1035:
SERIAL The unsigned 32 bit version number of the original copy of the  
zone. Zone transfers preserve this value. This value wraps and should  
be compared using sequence space arithmetic.

-- 
<a href="http://tertius.net.au/~thorfinn/">[email-masked]</a>
~/ For those who've come across the seas, ~/ We've boundless plains to  
share, ~/
~/   With courage let us all combine,     ~/    To Advance Australia  
Fair.    ~/
      -- The end of the second verse of the Australian National  
Anthem. --

---

From: Julien ÉLIE <[email-masked]>
Date: Tue, 23 Sep 2008 08:20:25 +0200

Hi Thorfinn,

>>> 640kb should be enough for anyone. Um. :-)

"I have to say in 1981 making those decisions I felt like I was providing
enough freedom for ten years, that is the move from 64k to 640k felt like
something that would last a great deal of time." (said in 1989 by Bill Gates)

"It [640K] was ten times what we had before.  But to my surprise, we ran out
of that address base for applications within - oh five or six years people
were complaining." (said in 1993 by Bill Gates)

> Looking at the current USEPRO text... *ponder* Serial number MUST  increase.  That is definitely a problem - you certainly can run 
> out of  32 bit integer.  It's not all that likely, but it's certainly possible.

That is why I also suggested a mechanism (yet to be defined) to *reset*
the counter.  But the main problem is that it will cause problems to
a news server which does not receive the message of reset...
So it is maybe a bad idea.

> I'd be uncomfortable putting in a 32 bit limit unless we also change  the text to deal with serial comparison in the same way that 
> DNS  serial numbers are compared.
>
> From RFC 1035:
> SERIAL The unsigned 32 bit version number of the original copy of the  zone. Zone transfers preserve this value. This value wraps 
> and should  be compared using sequence space arithmetic.

I reckon it is the best solution to adopt.
Say that there is a 32 bit limit and that serial numbers SHOULD be used
in case the checkgroups sender does not want to run out of available numbers.
It is up to him if he does not want to use serial numbers.

-- 
Julien ÉLIE

« -- C'est une bonne situation ça, scribe ?
  -- Oh, c'est une situation assise. » (Astérix)

---

From: Thorfinn <[email-masked]>
Date: Tue, 23 Sep 2008 17:02:51 +1000

On 23 Sep 2008, at 16:20, Julien ÉLIE wrote:
>> increase.  That is definitely a problem - you certainly can run out  
>> of  32 bit integer.  It's not all that likely, but it's certainly  
>> possible.
>

Yep - reset doesn't work, for exactly that reason.  We know that  
relying on a single message to propagate for future correctness is  
poor behaviour, especially since we know that message propagation  
isn't even guaranteed to be in sequence, let alone delivered at all.

>> I'd be uncomfortable putting in a 32 bit limit unless we also  
>> change  the text to deal with serial comparison in the same way  
>> that DNS  serial numbers are compared.
>>
>> From RFC 1035:
>> SERIAL The unsigned 32 bit version number of the original copy of  
>> the  zone. Zone transfers preserve this value. This value wraps and  
>> should  be compared using sequence space arithmetic.
>

DNS solves the problem by saying "sequence space arithmetic"...  
meaning if you are within plus half-the-number-range, you can consider  
the number "greater than", but if you are within minus half-the-number- 
range, then you consider it less than.

e.g. for a sequence space of decimal 1000:

if your number is 250, then anything up to 750 is considered "greater  
than", but from 751 - 249 is considered "less than".  Or, if your  
number is 999, then anything up to 499 is considered "greater than",  
and 500-998 is considered "less than".  (I might be off-by-one, but it  
sort of doesn't matter - close enough is good enough for this purpose.)

So there's "effectively" a reset - because if you keep incrementing by  
small amounts, you eventually come around to the other side of the  
sequence space, and that "resets" it for you.

I note that a few DNS implementations do actually fail to implement  
this correctly... but at least the standard says something useful, and  
most DNS servers do implement handling of SERIAL changes correctly.

So I'd either go with:

1. the current text: "unsigned integer there is no limit" - in which  
case implementers can actually trim leading zeros and do a straight  
asciibetical comparison, no bigint library required.

2. unsigned 32 bit number, wrapping and compared using sequence space  
arithmetic.

-- 
<a href="http://tertius.net.au/~thorfinn/">[email-masked]</a>
To err is human, to moo bovine.  -- BSD fortune file

---

From: Thorfinn <[email-masked]>
Date: Tue, 23 Sep 2008 18:19:30 +1000

> asciibetical comparison, no bigint library required.

I lied - string length *then* asciibetical comparison for strings of  
same length.  :-)

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 24 Sep 2008 16:33:48 GMT

>32 bit integer.  It's not all that likely, but it's certainly possible.

Well, it has taken 30 years for a few heavily populated groups to come
within sight of overflowing the 32-bit article numbers, and the number of
checkgroups messages sent is many orders of magnitude less than what such
groups have seen. So I think 32 bits is fine. People still regard 32 bits
as the 'normal' length of an integer, even though modern hardware is able
to cope with 64bit, especially for addresses.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 24 Sep 2008 16:36:20 GMT

>range, then you consider it less than.

Ah! That might well be an acceptable solution to the run-around of article
numbers problem, which has just reared its head on the NNTP list.

---

From: Thorfinn <[email-masked]>
Date: Thu, 25 Sep 2008 13:06:29 +1000

On 25 Sep 2008, at 02:33, Charles Lindsey wrote:
> In <[email-masked]> Thorfinn <[email-masked] 
> > writes:
>> of

*nod* I'm pretty comfortable with no numerical limit on checkgroups  
serial number (i.e., stick with the current text).  There's no good  
reason to treat the serial number as an binary integer at all... it's  
perfectly orderable using trim leading zeros, then compare with string  
length followed by ascii ordering after that.  It's not exactly a  
comparison you have to do very often.

Offtopic:

NNTP article numbers, different story - there are, after all, rather a  
lot more of them on a busy news server, and they are potentially  
needed as index entries and such.

Sequence space arithmetic is definitely good if you're playing with a  
limited number space - the DNS folks had it right way back then. :-)

Meep,

  Thorf