Subject: Re: IP checksum

From: Joe Touch <[email-masked]>
Date: Wed, 18 Aug 1999 10:18:54 -0700 (PDT)

> From [email-masked]  Wed Aug 18 10:07:02 1999
> Delivery-Date: Wed, 18 Aug 1999 10:07:02 -0700
> Return-Path: [email-masked]
> Date: Wed, 18 Aug 1999 17:39:54 +0200
> To: [email-masked]
> From: Mikael Degermark <[email-masked]>
> Subject: Re: IP checksum
> Cc: PILC mailing list <[email-masked]>
> 
> We have been working with these checksumming issues for a while.
> 
> For some applications it is better to pass damaged packets to the 
> application than to drop them. 

If you're talking about the IP checksum, it implies
a problem with the header. If the header is corrupted, 
(as others have pointed out), to which application, 
on which machine, would you like to deliver them?

> This is particularly true for 
> applications such as voice and video over error-prone wireless 
> links with high delays.

Good reasons to pass packets with failed UDP checksums, but
not failed IP checksums. Hoever, as Partridge et al pointed
out in Sigcomm 95, even UDP or TCP checksums are less likely to find
errors than some (most? many?) link checksums/CRCs. And if the link
thinks a packet is damaged, might not be clear where 
or whether there's an IP packet at all.

> The sad story of checksums, if you have an application that wants damaged
> packets to be delivered can be summed up as follows:
> 
> 1. IPv4 has a header checksum which many routers don't verify, they 
> just update it (clear violation of Router Requirements). The UDP checksum
> which covers the pseudoheader plus the payload can be disabled by the sender.
> 
> 2. IPv6 does not have a header checksum. The UDP checksum MUST NOT be 
> disabled. 
> 
> 3. So, for IPv6 there is, in practice, NO WAY for an application to have
> damaged packets delivered unless a new transport protocol is deployed. 
>
> With IPv4, where the UDP checksum can be disabled, it would be better to
> have the UDP checksum cover at least the demultiplexing information in the
> UDP header than to have it completely disabled. 

The checksum is nearly as big as the demuxing info. 
And if the demuxing info is bad, again, which application gets it?

Sending up info on packet corruption to the app sounds nice
in principle, but can it be achieved without some out-of-band
info? I.e., it seems like you need a filter near the source to
track the packets, and one at the destination (i.e., to model
the human reading the tcpdump output that indicates an error).
And the assumption that their channel is more reliable than the
underlying data channel. 

(PS - this doesn't count the use of FEC or stronger error recovery
mechanisms, but they need to be in the link level, to recover the
IP header, UDP demux, etc. You can't layer strong on weak on strong.)

Joe

---

From: Frank Kastenholz <[email-masked]>
Date: Wed, 18 Aug 1999 15:18:52 -0400

At 10:18 AM 8/18/99 -0700, Joe Touch wrote:

>info? 

actually, if all you are interested in is that you are experiencing
packet-loss due to corruption vs due to queue length, the experimental 
ecn work might be useful. if you are losing packets and the ecn bit
is not set, then you could assume that the loss is due to corruption.
if the ecn bit is not set, you are not as sure -- there could be
congestion in one place, but your packets might be going astray
due to corruption. (handwave handwave assume that ecn is deployed
everywhere, etc, etc)

you can't have the intermediate routers/etcs monitor traffic,
detect corruption, and then send a 'corruption experienced'
signal to the destination since you can't tell who the destination
of the corrupt packet is. and you can't send it to 'everyone' since
that would imply flow-monitoring/remembering, etc, etc. yuck.

---

From: Chip Sharp <[email-masked]>
Date: Wed, 18 Aug 1999 16:00:29 -0400

Not to mention that if you are getting enough TCP/UDP checksum errors (not 
dropped by link CRC) to really affect the application then you really 
better check your network cuz something ain't right.  How this bit of 
information gets from your host to the network mgr is another matter.

Chip

>IP header, UDP demux, etc. You can't layer strong on weak on strong.)

--------------------------------------------------
Chip Sharp                 Consulting Engineering
Cisco Systems              Telco Bio-region
Reality - Love it or Leave it.			
--------------------------------------------------

---

From: Ted Faber <[email-masked]>
Date: Wed, 18 Aug 1999 15:35:58 -0700

Chip Sharp wrote:
>better check your network cuz something ain't right.

In all likelyhood, you better check your routers and hosts (right down
to the NICs), because that's where something ain't right.  If the
wires (antennae etc) had broken it, the link CRC would (with high
probability) catch it.

---

From: Jonathan Stone <[email-masked]>
Date: Wed, 18 Aug 1999 15:51:11 -0700

In message <[email-masked]>,
Joe Touch writes:

[Mikael Degermark <[email-masked]> writes:}

>And the assumption that their channel is more reliable than the

Not if there's a tacit assumption of two checksums per level: one
checksum over the demux info (header) and another over the payload
data.  What to do about existing link-level sums is beyond me.

The Larzon and Degermark I-D proposes instead a length-limit on the
UDP checksum field; the UDP-Lite checksum covers only a prefix of the
packet, with the prefix length given by a `checksum coverage field'.
The authors recognize the problem of link-level checksums, but are
silent on what to do about it -- other than replicate the
prefix-cksum feature at the link level.

Sending data over known-lossy links, without any checksum coverage
at all, does seem rather courageous.

Has anyone considered two distinct checksums at each level?  That
might not be as flexible for demuxing above UDP (application-level
demuxing); aside from that it seems more useful.

>IP header, UDP demux, etc. You can't layer strong on weak on strong.)

Because once the underlying `weak' layer fails, you can't find your
way up to the higher `strong' (or error-tolerant) layer, right?  I
think the Larzon and Degermark I-D can layer strong on top of weak, by
doing the (weak) checksum only over that prefix of a packet necessary
to demux to the strong (or error-tolerant) application layer. But if
error rates are high, what fraction of packets will still get errors
in the weak-checksum-protetected demux info? How big is the
IP+UDP+(voice application) header relative to the codec samples?

If headers are, say, 1/3rd of a packet, and error bursts are uniformly
distributed amongst all bits of a packet, then UDP-Lite will still
fail to deliver 1/3rd of all packets which incur errors. Is that
right? Doesn't seem very compelling, even without considering the
link-level checksum issues.

---

From: Mikael Degermark <[email-masked]>
Date: Thu, 19 Aug 1999 06:59:11 +0200

>wires (antennae etc) had broken it, the link CRC would (with high

The links used by mobile users using wireless cellular, will 
cause a significant error rate *if* the system is designed for maximal
spectrum efficiency. If one wants to provide voice communication to the 
maximum number of users for the least amount of money, there will sometimes
be significant error rates for some users. That does not necessarily mean
that the network is broken. And those users will have a better service if 
the damaged packets are delivered to the codec.

Mikael Degermark

http://www.cdt.luth.se/~micke

---

From: Mikael Degermark <[email-masked]>
Date: Thu, 19 Aug 1999 06:40:25 +0200

>information gets from your host to the network mgr is another matter.

There are links, specifically wireless cellular links, where fading
and interference between many users cause the links to have *significant*
error rates, even after FEC. So there isn't necessarily something wrong
with your network. It can be the most economical way of running it to have
a certain residual error rate. 

Micke

---

From: Mikael Degermark <[email-masked]>
Date: Thu, 19 Aug 1999 07:09:42 +0200

>prefix-cksum feature at the link level.

It is of course futile to have a weak checksum such as the Internet 
checksum if there isn't strong enough protection on lower levels over
that portion of the packet.

>at all, does seem rather courageous.

Indeed. If the data needs protection. Not all data needs that. Some 
applications, such as voice with the proper codecs, work better
if (some) damaged data is delivered. 

>demuxing); aside from that it seems more useful.

It wastes to many bytes. 

>But if

Compressed headers would be 2 bytes. Payload 15-30 bytes.

---

From: Mikael Degermark <[email-masked]>
Date: Thu, 19 Aug 1999 06:30:20 +0200

>on which machine, would you like to deliver them?

It should be obvious to anyone that I was NOT talking about the 
IP checksum. If the IP header, or other header information such as the 
transport header, is damaged, it is impossible to deliver the packet
to the right destination and it must of course be discarded.

>or whether there's an IP packet at all.

I agree that most framing protocols in use today is not well suited to 
the kind of application we've been considering. Fortunately, 
the framing protocols most of the 1000 million users would use have not
yet been standardized.

>And if the demuxing info is bad, again, which application gets it?

I'm not sure what you mean with the size comment. We're proposing to make it 
possible for the UDP Lite sender to have a *partial* checksum which 
would cover at least the IP and UDP headers. 

>in principle

We are proposing to send up the damaged packets, not just a corruption 
signal. You are perhaps confusing my position with that of the originator
of this thread?

Micke

---

From: Lars-Åke Larzon <[email-masked]>
Date: Thu, 19 Aug 1999 06:50:27 +0200

> prefix-cksum feature at the link level.

We belive this is the best solution since it would be most flexible.
Another alternative is to simply ignore link layer errors for frames
carrying UDP-Lite packets. This is a somewhat uglier solution since it
requires layer violation, but it works. We have mentioned these different
alternatives in a paper published at the QoS miniconference of ICC'99, but
intentionally left them out of the draft.

> at all, does seem rather courageous.

In UDP Lite, the least acceptable checksum coverage is 8 in order to
ensure that the headers are always verified by at least the UDP Lite
checksum.

> IP+UDP+(voice application) header relative to the codec samples?

For G.723.1 encoding, the sound clips is 16 or 24 bytes if I am not
mistaking. (IP psuedo header)+UDP+RTP headers are 32 bytes, but if header
compression is used, the same headers can be as small as 2 bytes.

    /Lars-Åke Larzon

---

From: Jonathan Stone <[email-masked]>
Date: Thu, 19 Aug 1999 00:13:29 -0700

In message <[email-masked]>,

>if (some) damaged data is delivered. 

Are you assuming the wireless link is the only source of errors?  I'm
concerned that this scheme would fail to catch errors which replace
the non-checksumemd UDP-Lite traffic with completely different data,
possibly from another flow altogether.  How will that sound to the
codecs?

>>demuxing); aside from that it seems more useful.

Excuse me? You use two octets for UDP-LIte's sum-limit, but
adding two more to checksum the remaining data is too much? 
>Compressed headers would be 2 bytes. Payload 15-30 bytes.

And you cannot afford two more?!

The compressed headers cannot be two bytes, since the checksum is
two bytes, and you cannot compress it.. Call it four bytes.
So. you want to change UDP to reduce the drop rate over wireless voice
links due to checksum errors by, roughly, a factor of 4/19 to 4/34 -
cal it 12% to 21%.

Yet if the drop rate (sans UDP-Lite) was high enough that the 12% to
21% reduction is non-negligible, it's not even the right order of
magnitude to run TCP well over the link.  Forget web-based services
lke directories on these "phones", or using them as "modems".

Which seems to invite a couple of hard questions, like why design a
network with post-FEC error rates like this in the first place, and
why bother using IP on it once you have?

---

From: Mikael Degermark <[email-masked]>
Date: Thu, 19 Aug 1999 11:11:27 +0200

>possibly from another flow altogether.  How will that sound to the

I do assume that the wireless link (or previous wireless links) will be
the dominating source of damage to the payload. Packets could of course
be dropped along the path due to congestion. I also expect that all links
in the wired Internet will indeed have strong checksums covering all
packets, i.e., I do not expect that current framing protocols in the 
wired Internet will be replaced. 

I don't believe that occasional packets with completely garbled data will
affect codec performance so that you could hear it.

>adding two more to checksum the remaining data is too much? 

There seems to be a slight misunderstanding here. 

We do NOT add anything. We redefine the semantics of the UPD Length 
and UDP Checksum fields so that the UDP Length field (which is renamed
to Coverage in UDP Lite) tells the receiver how much the UDP Checksum 
covers. The UDP Length field is completely redundant with the IP Length 
field and I personally believe that the only reason it is there is to make
the UDP header a multiple of 4 octets. Nobody that I've discussed this
with has been able to give a good reason to have it there.

>And you cannot afford two more?!

Not after all the pains I went through to reduce the compressed header 
to 2 bytes. 

>links due to checksum errors by, roughly, a factor of 4/19 to 4/34 -

The interesting number is not the reduction in drop rate, it is the
improvement in spectrum efficiency for comparable voice quality. And 
since radio spectrum is rather expensive, that translates into 
many million dollars. Wouldn't you like to pay less for your mobile 
phone calls? I would. 

We don't believe it is possible to change UDP for IPv4. Deployment 
would be a nightmare. For IPv4, UDP Lite must be a separate protocol.

For IPv6, the situation is different since the installed IPv6 base is still 
rather small. Moreover, 1000 million users means IPv6. This could
actually be the killer application that makes IPv6 happen. In the US,
few people use mobile phones, but in Scandinavia the penetration is now
around 50% of the entire population.

>lke directories on these "phones", or using them as "modems".

I HAVE NOT MENTIONED TCP. It is obvious to us all that these links
must be fixed to run TCP well. That is typically done with a link-level
ARQ scheme. So the packet classes need to be separated. I have yet to 
meet a radio person who doesn't agree to this. 

>why bother using IP on it once you have?

Not hard. 

1. Because it is cheaper (we're talking many millions of dollars). 
2. Because it would make the entire system as well as the end-systems 
   less complex if they didn't have to deal with a bunch of application-
   specific network architectures. Why not use IP for it all?
   I agree with Vint Cerf: IP on everything!

Mikael Degermark

---

From: Christian Huitema <[email-masked]>
Date: Thu, 19 Aug 1999 11:26:44 -0400

Michael,

Information theory tells us that the right strategy to maximize spectrum
efficiency is to first compress the data, then add explicit redondancy (FEC
or ARQ) to cope with errors.  What you are proposing is an intermediate
scheme were the redondancy is in fact obtained by not compressing the data
as much as you could, so that the decoding scheme somehow resists errors, a
combination that is explicitly proven inferior by the information theory.
Your scheme constrains the encoding (would definitely not work well with
differential compression) and essentially prevents end-to-end encryption.

Another approach is to devise an adaptation layer that provides an adequate
error rate, using either local repetition if the wireless delay is short
enough, or some form of link-level FEC, either at the physical/bit layer,
or at the frame layer (explicit redondancy in data packets), or at the
packet layer (redondancy packets in the data stream.) That approach is in
fact needed if you want to run TCP, or if you want end-to-end encryption of
your data.

I have some doubts on the market acceptance of a system that would not
support TCP or encryption...
Christian Huitema

---

From: Mark Handley <[email-masked]>
Date: Thu, 19 Aug 1999 11:33:07 -0700

>differential compression) and essentially prevents end-to-end encryption.

Even given that the data itself may not be useful to the end
application, it might be useful to know that the packet was not
dropped due to congestion, and so a congestion control backoff is not
required.  You can't do this with the current UDP because the
receiving host can't be sure if the corruption was in the ports or in
the payload.

Cheers,
	Mark

---

From: Jonathan Stone <[email-masked]>
Date: Thu, 19 Aug 1999 12:09:28 -0700

>receiving host can't be sure if the corruption was in the ports or in

Yes, but the UDP-Lite discussion has conflated several issues.  One
issue is sending data over analog-voice-quality link-levels with the
specific engineering tradeoffs Christian Huitema has discussed.

I think you're talking about the more general issue of delay-sensitive
applications which deliver better performance if damaged packets are
delivered to them, rather than dropped.

In that domain, in the wider Internet, UDP-Lite works only if either
(a) the errrors occur outside link-levels: on router busses, in router
DRAM, on CRC-less links like SLIPI; or (b) you disable those
link-level checksums over at least part of the packet.

Who really want to change Ethernet, PPP, SONET, ... adding a pointer
to the frame, partitioning the frame into a `head' and a `tail', and
change the CRC/checksum so that CRCs are computed and verified only
only on the `head'? How about disabling link-level checksums
alogtether?

If something like UDP-Lite gets deployed, I *really* hope we use two
checksums/CRCs, rather than no CRC and no checksum at all over packet
payloads.

---

From: Mikael Degermark <[email-masked]>
Date: Sat, 21 Aug 1999 11:13:36 +0200

>specific engineering tradeoffs Christian Huitema has discussed.

I believe that this has been answered in my reply to Christian.

>only on the `head'? How about disabling link-level checksums

With the risk of repeating myself: 

UDP Lite is flexible. The sender of a packet can decide the length of 
the packet prefix which is to be protected by the transport-layer checksum.
The most common length would be to protect the entire packet, which means
that the UDP Lite packet is indistinguishable from a classic UDP packet. 

Lower layers must ensure that packets with errors in the protected prefix
are not delivered or that the errors are such that they can be caught by
the UDP Lite checksum. This typically means strong checksums extending over
at least the prefix. 

Note that lower layers CAN protect more than the prefix, specifically 
the entire packet. It is not realistic to expect that current link layers
will be changed. Where error rates are significant, we expect that the 
link layers will be. 

>checksums/CRCs, rather than no CRC and no checksum at all over packet

There are applications where one do not want another checksum over the 
payload; it would serve no purpose and would in fact worsen the 
performance for these applications. 

If the sender requests that a (true) prefix should be protected,
why should the network protect the remainder with another checksum? 
I fail to see a good reason for doing that. If you have one, please
inform me.

Please note that we do not advocate that the payload should be unprotected
UNLESS the application has specifically asked for it by having the 
UDP Coverage field be less than the IP Length.

Mikael Degermark

---

From: Mikael Degermark <[email-masked]>
Date: Sat, 21 Aug 1999 12:58:09 +0200

>combination that is explicitly proven inferior by the information theory.

I believe that those proofs do not consider the delay constraints. 

For applications like interactive voice, the delay budget must be met. 
The best approach for doing that seems to be to use as much interleaving
as possible, and then FEC on top of that. The residual error rate is 
then dealt with by robust codecs. (no time for anything else)

>differential compression) and essentially prevents end-to-end encryption.

Packet can get dropped anywhere in the Internet, so I don't really see why 
this scheme is worse for differential encoding...

The scheme which REALLY prevents end-to-end encryption is to have
transcoders before the wireless links. I would like to make it possible
to avoid having those while still doing as well as possible over the 
wireless link.

(Btw, I believe that such encryption should be on the payload only, and 
performed by the application. IPSEC does not seem right.) 

>enough, or some form of link-level FEC, either at the physical/bit layer,

There will be lots of FEC on the link layer. The residual error rate 
can be significant even after this FEC. 

>packet layer (redondancy packets in the data stream.) 

These last two will not be (radio) bandwidth efficient. 

>That approach is in

I believe there will be an ARQ scheme used for TCP and other packets that 
cannot be determined to be robust.

>support TCP or encryption...

No kidding? :-)

Mikael Degermark

---

From: Lloyd Wood <[email-masked]>
Date: Sat, 21 Aug 1999 18:20:15 +0100 (BST)

> For applications like interactive voice, the delay budget must be met. 

...SHOULD be met. No must about it.

L.

amazing what the human ear will tolerate.

<[email-masked]>PGP<http://www.ee.surrey.ac.uk/Personal/L.Wood/>