Subject: [dtn] CRC options

From: "Burleigh, Scott C (312B)" <[email-masked]>
Date: Mon, 18 Jan 2016 17:44:44 +0000

Hi.  As discussed in this morning's virtual meeting, I think we are down to just two open technical issues in the new Bundle Protocol specification ("bpbis").  One of those issues is the question of providing a bundle manifest (a.k.a. "inventory") indicating what blocks were originally present in the bundle; I'll start a separate email thread on that topic in a few minutes.  The other issue is, I think, simpler: which specific mechanisms for constructing and validating the optional CRC in the primary block should the specification identify as valid?  CRC-16-ARINC, CRC-16-CCITT, CRC-16-CDMA2000, CRC-16-DECT, etc.?

Your opinion is welcome.

Scott

---

From: <[email-masked]>
Date: Tue, 19 Jan 2016 08:09:41 +0000

Hi Scott,

I see you are mentioning only 16-bits CRC: does it mean that we should exclude 32-bits CRC? Since its use is quite widespread, I would not drop it, unless there are strong arguments against it. Alternatively, I was wondering whether we could support different  CRC-lengths and provide an extra bit (or a mask) to indicate which CRC class is being used.

Regards,

Tomaso

------------------------ 
Deutsches Zentrum für Luft- und Raumfahrt (DLR) 
German Aerospace Center 
Institute of Communications and Navigation | Satellite Networks | Oberpfaffenhofen | 82234 Wessling | Germany 
Tomaso de Cola, Ph.D. 
Telefon +49 8153 28-2156 | Telefax  +49 8153 28-2844 | [email-masked] 
http://www.dlr.de/kn/institut/abteilungen/san 

> -----Original Message-----
> From: dtn [mailto:[email-masked]] On Behalf Of Burleigh, Scott C (312B)
> Sent: Monday, January 18, 2016 18:45
> To: [email-masked]
> Subject: [dtn] CRC options
> 
> Hi.  As discussed in this morning's virtual meeting, I think we are down to just
> two open technical issues in the new Bundle Protocol specification ("bpbis").
> One of those issues is the question of providing a bundle manifest (a.k.a.
> "inventory") indicating what blocks were originally present in the bundle; I'll
> start a separate email thread on that topic in a few minutes.  The other issue is,
> I think, simpler: which specific mechanisms for constructing and validating the
> optional CRC in the primary block should the specification identify as valid?
> CRC-16-ARINC, CRC-16-CCITT, CRC-16-CDMA2000, CRC-16-DECT, etc.?
> 
> Your opinion is welcome.
> 
> Scott
> 
> _______________________________________________
> dtn mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dtn

---

From: <[email-masked]>
Date: Tue, 19 Jan 2016 09:56:37 +0000

Hey everyone,
I would strongly suggest allowing CRC-32, as well as CRC-16. Hardware support for CRC-32 is fairly widespread, and (as Tomaso said) used widely within industry.  As for which algorithms are utilized, I think we should keep it limited (as much as is reasonable) to a small set of strong, well-used, and frequently implemented CRC's, such as CRC-16 CCITT, CRC-16 ANSI, and CRC-32(C). Keeping the number small obviously simplifies implementation, and also allows (for example) a single byte of mask data for checksumming ID information.

Thanks,
Jeremy
-----Original Message-----
From: dtn [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Dienstag, 19. Januar 2016 09:10
To: [email-masked]
Cc: [email-masked]
Subject: Re: [dtn] CRC options

Hi Scott,

Regards,

Tomaso

------------------------
Deutsches Zentrum für Luft- und Raumfahrt (DLR) German Aerospace Center Institute of Communications and Navigation | Satellite Networks | Oberpfaffenhofen | 82234 Wessling | Germany Tomaso de Cola, Ph.D. 
Telefon +49 8153 28-2156 | Telefax  +49 8153 28-2844 | [email-masked] http://www.dlr.de/kn/institut/abteilungen/san 

> Hi.  As discussed in this morning's virtual meeting, I think we are 
> down to just two open technical issues in the new Bundle Protocol specification ("bpbis").
> "inventory") indicating what blocks were originally present in the 
> bundle; I'll start a separate email thread on that topic in a few 
> minutes.  The other issue is, I think, simpler: which specific 
> mechanisms for constructing and validating the optional CRC in the primary block should the specification identify as valid?

---

From: "Burleigh, Scott C (312B)" <[email-masked]>
Date: Tue, 19 Jan 2016 15:37:41 +0000

Sure, I'm fine with including 32-bit CRCs as well, let's just pick exactly which ones we want to support.  A 32-bit CRC on the primary block seems a little like overkill to me, but as long as 16-bit CRCs are supported as well I have no objection.

Scott

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Tuesday, January 19, 2016 1:57 AM
To: [email-masked]; Burleigh, Scott C (312B) <[email-masked]>
Cc: [email-masked]
Subject: RE: CRC options

Hey everyone,

Thanks,
Jeremy
-----Original Message-----
From: dtn [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Dienstag, 19. Januar 2016 09:10
To: [email-masked]
Cc: [email-masked]
Subject: Re: [dtn] CRC options

Hi Scott,

Regards,

Tomaso

------------------------

---

From: Carlo Caini <[email-masked]>
Date: Wed, 20 Jan 2016 12:53:33 +0100

Dear Scott,
         I am in favor of allowing CRC-32, as previously suggested by 
Jeremy and Tomaso. By the way, after having given a glance to the 
recently posted bpbis-02, I have noticed that in the "Significant 
Changes from RFC5050" the fifth bullet in the list is "Add optional 
CRCs to non-primary blocks", while I have not found any reference to 
it in the text. Why is this feature not included in the current 
draft? Is this bullet in error or was this feature inadvertently 
omitted? By the way, I am strongly in favor of this feature too.
I have a couple of other minor comments on the draft, but I will post 
them under a different topic as they do not concern CRCs.
Yours,
      Carlo

At 10:56 19/01/2016, [email-masked] wrote:
>Hey everyone,
>I would strongly suggest allowing CRC-32, as well as CRC-16. 
>Hardware support for CRC-32 is fairly widespread, and (as Tomaso 
>said) used widely within industry.  As for which algorithms are 
>utilized, I think we should keep it limited (as much as is 
>reasonable) to a small set of strong, well-used, and frequently 
>implemented CRC's, such as CRC-16 CCITT, CRC-16 ANSI, and CRC-32(C). 
>Keeping the number small obviously simplifies implementation, and 
>also allows (for example) a single byte of mask data for 
>checksumming ID information.
>
>Thanks,
>Jeremy
>I see you are mentioning only 16-bits CRC: does it mean that we 
>should exclude 32-bits CRC? Since its use is quite widespread, I 
>would not drop it, unless there are strong arguments against it. 
>Alternatively, I was wondering whether we could support 
>different  CRC-lengths and provide an extra bit (or a mask) to 
>indicate which CRC class is being used.
>
>Regards,
>
>Tomaso
>
>------------------------
>Deutsches Zentrum für Luft- und Raumfahrt (DLR) German Aerospace 
>Center Institute of Communications and Navigation | Satellite 
>Networks | Oberpfaffenhofen | 82234 Wessling | Germany Tomaso de Cola, Ph.D.
>Telefon +49 8153 28-2156 | Telefax  +49 8153 28-2844 | 
>[email-masked] http://www.dlr.de/kn/institut/abteilungen/san
>
> > mechanisms for constructing and validating the optional CRC in 
> the primary block should the specification identify as valid?

---

From: "Burleigh, Scott C (312B)" <[email-masked]>
Date: Wed, 20 Jan 2016 16:40:04 +0000

Hi, Carlo.  I have looked back through the minutes of the DTN working group meetings and haven't found any discussion of that feature at all.  I think the story here is:
-	The pre-WG draft I posted in June of 2014 had a CRC in the primary block and also had a "payload CRC" extension block.
-	In subsequent discussions we at some point informally agreed that the optional payload CRC was useful but a more general approach -- adding it to the header of the "canonical" block structure, so that extension blocks could be protected as well -- would be better.
-	The sense that this was a good idea was retained in that "Significant Changes" list, but we somehow never actually inserted it into the text of the specification.

If we have indeed got consensus in the working group that this is the right thing to do then I'll make sure it's included in the next draft.  The mechanism would be the same as in the primary block: a CRC type would always be included in the header of every block but the CRC itself would be present only if the CRC type was non-zero (in which case the length and nature of the CRC would be as indicated by the CRC type).

Scott

-----Original Message-----
From: Carlo Caini [mailto:[email-masked]] 
Sent: Wednesday, January 20, 2016 3:54 AM
To: Burleigh, Scott C (312B) <[email-masked]>
Cc: [email-masked]
Subject: Re: [dtn] CRC options

Dear Scott,
         I am in favor of allowing CRC-32, as previously suggested by Jeremy and Tomaso. By the way, after having given a glance to the recently posted bpbis-02, I have noticed that in the "Significant Changes from RFC5050" the fifth bullet in the list is "Add optional CRCs to non-primary blocks", while I have not found any reference to it in the text. Why is this feature not included in the current draft? Is this bullet in error or was this feature inadvertently omitted? By the way, I am strongly in favor of this feature too.
I have a couple of other minor comments on the draft, but I will post them under a different topic as they do not concern CRCs.
Yours,
      Carlo

At 10:56 19/01/2016, [email-masked] wrote:
>Keeping the number small obviously simplifies implementation, and also 
>allows (for example) a single byte of mask data for checksumming ID 
>information.
>
>Thanks,
>Jeremy
>I see you are mentioning only 16-bits CRC: does it mean that we should 
>exclude 32-bits CRC? Since its use is quite widespread, I would not 
>drop it, unless there are strong arguments against it.
>Alternatively, I was wondering whether we could support different  
>CRC-lengths and provide an extra bit (or a mask) to indicate which CRC 
>class is being used.
>
>Regards,
>
>Tomaso
>
>------------------------
>Deutsches Zentrum für Luft- und Raumfahrt (DLR) German Aerospace Center 
>Institute of Communications and Navigation | Satellite Networks | 
>Oberpfaffenhofen | 82234 Wessling | Germany Tomaso de Cola, Ph.D.
> > From: dtn [mailto:[email-masked]] On Behalf Of Burleigh, Scott 
> > C
> > (312B)

---

From: Carlo Caini <[email-masked]>
Date: Wed, 20 Jan 2016 19:31:16 +0000

Dear Scott,
      thanks for having taken into consideration my concerns. I fully agree with you on your proposal. To have a consistent mechanism for all blocks seems the most effective and elegant  solution.
Yours,
   Carlo
________________________________________
Da: Burleigh, Scott C (312B) [[email-masked]]
Inviato: mercoledì 20 gennaio 2016 17.40
A: Carlo Caini
Cc: [email-masked]
Oggetto: RE: [dtn] CRC options

Scott

-----Original Message-----
From: Carlo Caini [mailto:[email-masked]]
Sent: Wednesday, January 20, 2016 3:54 AM
To: Burleigh, Scott C (312B) <[email-masked]>
Cc: [email-masked]
Subject: Re: [dtn] CRC options

Dear Scott,

---

From: Sebastian Schildt <[email-masked]>
Date: Sat, 23 Jan 2016 10:57:52 +0100

Hello,

I haven't followed the CRC discussion (f there as any) closely in the past. Therefore, regarding the specific CRC or the length I am fine with everything.

However, if this has not been discussed before, I would strongly suggest specifying only ONE CRC valid for the BP: As a highler layer protocol a CRC in the BP is only a sort of "catch-all" indicating whether something went wrong during transmission/storage or processing, similarly in scope to for example the relatively braindead TCP checksum. Therefore, there is no way to identify a CRC checksum with  the "perfect properties" for the BP. Only in the protocols employed by convergence layers  can error detection and correction be tailored to the specific challengs of that concrete channel.

Therefore, to ease implementation and interoperability hurdles, I strongly suggest specifying only one kind of acceptable CRC for the BP (maybe that was already the plan, but reading this thread it seems people are converging towards a "suite" of CRCs)

Sebastian

> On 19 Jan 2016, at 10:56, [email-masked] wrote:
> 
> Hey everyone,

---

From: "Burleigh, Scott C (312B)" <[email-masked]>
Date: Sat, 23 Jan 2016 18:12:12 +0000

Hi, Sebastian.  I agree with you in concept, but I would suggest that we specify two supported CRCs for BP rather than one.  You are right that there is no way to identify a perfect CRC for BP because of the very broad range of possible variations in the underlying convergence layer stack.  But the CRCs we are talking about would be applied to individual blocks of the bundle rather than to the bundle as a whole, and block size is the one relevant parameter that will always be exposed to the BP implementation at the time a CRC is to be applied.  It makes sense to me to specify one CRC type for very small blocks (the primary block, most or all extension blocks, and small payloads) and one other for large payload blocks.

Scott

-----Original Message-----
From: Sebastian Schildt [mailto:[email-masked]] 
Sent: Saturday, January 23, 2016 1:58 AM
To: [email-masked]
Cc: [email-masked]; Burleigh, Scott C (312B); [email-masked]
Subject: Re: [dtn] CRC options

Hello,

Sebastian

---

From: Sebastian Schildt <[email-masked]>
Date: Sat, 23 Jan 2016 21:58:53 +0100

Hi Scott,

that is valid point, and I know you are always eager to shave off a few bits of overhead at the lower end of what the BP can do :) .

While I am not sure it is really neccessary to make this difference, I could accept two CRCs. However, my feeling is, that most blocks will be rather small, so maybe even a 16bit CRC is enough. As for  payload blocks, they might conceivably be so large (think multiple TB for extreme cases…), that even a „large“ CRC to secure it all would not be adequate (and a small 16 bit CRC would still catch a flipped bit)

So for the record: In my opinion 16 bit or 32 bit (as some have argued for in this thread), or two specific CRCs with a size threshold would all be ok for me (although I would prefer a single CRC). Only if we are planning to use multiple different CRC families and lengths I would strongly oppose.

Sebastian

> On 23 Jan 2016, at 19:12, Burleigh, Scott C (312B) <[email-masked]> wrote:
>

---

From: Carlo Caini <[email-masked]>
Date: Sat, 23 Jan 2016 21:49:32 +0000

Dear Sebastian,
     I agree with you on the fact that CRC varieties should not be enlarged without real necessity. Let me just to point out that the peculiar value of CRC at BP is to protect the bunle data against possible flipped bits in the processing or storing inside a machine, where they cannot be protected by lower layers integrity mechanisms. Of course these BP CRCs offers also an additional protection against bits that are occasionally flipped in the actual Tx from a node to another, i.e. when their integrity may be protected by lower layers CRC. If this integrity is protected mduring Tx, an additional protection is not bad as checksums, as CRC, are not infallible. If not, the BP protection becomes essential. This is for instance the case of LTP green-part, where losses of LTP segments do not prevent the delivery of corrupted bundles. This could be also the case of UDP with optional checksum switched off. 
More generally, BP source and destination cannot have control of all DTN hops between them, therefore they cannot take for granted that in all hops in between integrity mechanisms of lower layers are applied, or that intermediate nodes' hardware is error free. 
This is why I am in favor of CRC-32 (however, I suppose that one 32 bit option be enough) as an option for all blocks. It is just better than nothing or CRC-16.
Yours,
   Carlo

________________________________________
Da: dtn [[email-masked]] per conto di Sebastian Schildt [[email-masked]]
Inviato: sabato 23 gennaio 2016 21.58
A: Burleigh, Scott C (312B)
Cc: [email-masked]; [email-masked]; [email-masked]
Oggetto: Re: [dtn] CRC options

Hi Scott,

Sebastian

---

From: <[email-masked]>
Date: Sun, 31 Jan 2016 06:34:21 +0000 (UTC)

Sebastian,

Discussion of CRCs for the bundle protocol began in 2007
with considerable discussion on the dtnrg mailing list and
a number of internet-drafts, e.g.

http://personal.ee.surrey.ac.uk/Personal/L.Wood/publications/internet-drafts/draft-eddy-dtnrg-checksum/

See the dtnrg mailing list archives.

This has been discussed extensively before - but participants
rarely were aware of CRC properties or computation, and for
many years denied that errorchecking was needed. So we
published our 'A bundle of problems' paper in 2009.

This issue is probably good for another ten or so years.

I'm sure someone can fill you in on previous CRC discussion.
(Remember that when history repeats, the first time
is tragedy, and the second time is farce.)

Lloyd Wood
http://sat-net.com/L.Wood/dtn/bundle.html

----- Original Message -----
From: Sebastian Schildt <[email-masked]>
To: [email-masked]
Cc: [email-masked]; [email-masked]; [email-masked]
Sent: Saturday, 23 January 2016, 20:57
Subject: Re: [dtn] CRC options

Hello,

Sebastian