Subject: Re: IP checksum

From: Lloyd Wood <[email-masked]>
Date: Tue, 17 Aug 1999 20:48:04 +0100 (BST)

On Tue, 17 Aug 1999, Matthias Jung wrote:

> imagine a scenario where two TCP hosts communicate via the internet when
> one or more wireless or satellite links must be passed. in this case,
> transmission errors may cause loss of packets which will be interpreted
> as congestion by the sending hosts. one way to solve this problem is to
> implement protocol boosters or some kind of link layer error control
> (likely FEC) in the critical points. a more end2end compatible solution
> could be to implement a signaling protocol between the two hosts, which
> would allow the sender to estimate the transmission error rate and use
> packet level FEC to cope with or even allow the receiver to indicate if
> a packet is lost due to congestion or transmission errors.

adaptive FEC can be done, yes. It can be done on a per-link basis
below the packet level and network layer, though, without requiring
any changes to IP. This is the preferable solution, since
local transmission errors are a link-specific problem.

These issues get discussed in the IETF pilc working group, which is a
better place for this. Followups set.

> such a
> signaling protocol would only be possible if the IP checksum mechanism
> that drops corrupted packets could be disabled.

the IP checksum only flags up corrupted IP headers for packets that
can't be reliably routed to their destination; IP is not a reliable
datagram service. UDP and TCP have their own checksums for payload
contents.

> that way the receiving
> host itself could identify corrupted packets. 

And detecting corrupted payloads is what TCP and UDP checksums are
for.

> am i right with my keen
> statement that IP header checksumming is kind of violating the
> end-to-end paradigma?

Sorry, no; and without an IP header checksum, how can you be sure the
corrupted packets will reach the receiving host?

L.

packet-level FEC: about as useful as data-link security mechanisms,
but with rather more overhead.

> cheers
>   matthias
> 
> --
> Matthias Jung
> Institut Eurecom
> 2229 Route des Cretes; BP 193
> F-06904 Sophia Antipolis Cedex
> 
> Tel: +33 4.93.00.26.87
> E-mail: mailto:[email-masked]
> http://www.eurecom.fr/~jung/

<[email-masked]>PGP<http://www.ee.surrey.ac.uk/Personal/L.Wood/>

---

From: Christian Huitema <[email-masked]>
Date: Tue, 17 Aug 1999 17:16:32 -0400 (EDT)

> but with rather more overhead.

Are we feeling dogmatic, today?...

Packet level FEC does indeed imply overhead. In the case of a TCP
connection, packet level FEC implies more overhead than a SACK mechanism
-- one repeats preventively some fraction of the packets, the other
repeats exactly what needs to be repeated. On the other hand, packet level
FEC can be used to obtain stable transmission delays (no need for
end-to-end repetition); there are some applications that would happily
trade transmission overhead for delay. Packet level FEC can also be used
to reduce the overall memory requirement of the system, suppressing the
need for 2*RTT of sequencing buffer implied by SACK.

Packet level FEC can also be used in multicast configurations, where its
overhead is less than that of a centralized ACK/repeat mechanism.

-- 
Christian Huitema
------------------------------
Please note my new address: [email-masked]
http://www.telcordia.com/

---

From: John Leong <[email-masked]>
Date: Tue, 17 Aug 1999 15:59:37 -0700

> On the other hand, packet level
> end-to-end repetition)

If the application requires a reliable transport, then while FEC will
reduce the number of retransmissions, it will not be a substitue for
retransmission in practice.

Regards,
John Leong
--
---------------------------------------------------------
Bell Labs Research SV                [email-masked]
3180 Porter Drive                       Tel: 650-565-7603
Palo Alto, CA 94304                     Fax: 650-565-7676

---

From: Bora Akyol <[email-masked]>
Date: Tue, 17 Aug 1999 21:34:00 -0700

Moreover,
on wireless links, since the link quality is constantly changing (Fading,
etc), it is very difficult to optimize the amount of FEC overhead vs data
to be sent. On links where the RTT is short, ARQ based procedures can be
more affective and efficient than FECs. OTOH, in long RTT links such as
satellite, space mission FEC is definitely required.

Bora Akyol
Pluris

---

From: Mikael Degermark <[email-masked]>
Date: Wed, 18 Aug 1999 17:39:54 +0200

We have been working with these checksumming issues for a while.

For some applications it is better to pass damaged packets to the 
application than to drop them. This is particularly true for 
applications such as voice and video over error-prone wireless 
links with high delays.

The system cost is significantly lower, for comparable quality,
if codecs are explicitly designed to cope with damaged packets.
Given that there will be more than 1 billion users behind such links
in five years (CDMA, WCDMA, GSM, UMTS, etc), it seems like a good 
idea to get this right now, rather than in five years.

The sad story of checksums, if you have an application that wants damaged
packets to be delivered can be summed up as follows:

1. IPv4 has a header checksum which many routers don't verify, they 
just update it (clear violation of Router Requirements). The UDP checksum
which covers the pseudoheader plus the payload can be disabled by the sender.

2. IPv6 does not have a header checksum. The UDP checksum MUST NOT be 
disabled. 

3. So, for IPv6 there is, in practice, NO WAY for an application to have
damaged packets delivered unless a new transport protocol is deployed. 

With IPv4, where the UDP checksum can be disabled, it would be better to
have the UDP checksum cover at least the demultiplexing information in the
UDP header than to have it completely disabled. 

Fortunately, it is rather simple to fix UDP to be an excellent transport
protocol for these applications. We have outlined how to do that in 
a recent Internet-Draft, 

	http://search.ietf.org/internet-drafts/draft-larzon-udplite-01.txt

Thanks, 

Mikael Degermark

>statement that IP header checksumming is kind of violating the

No you are not right. You would be closer to the truth if you said
that the UDP checksum is violating the end-to-end paradigm.

>  matthias

http://www.cdt.luth.se/~micke

---

From: Darren Kerr <[email-masked]>
Date: Wed, 18 Aug 1999 09:29:38 -0700 (PDT)

> just update it (clear violation of Router Requirements).

Correction.  The vast majority of internet routers *always*
verify ipv4 checksums and drop offending packets.

/Darren

---

From: John Leong <[email-masked]>
Date: Wed, 18 Aug 1999 11:24:21 -0700

> protocol for these applications.

Slightly aside from your IETF draft of UDP lite in doing partial
checksum, I believe UDP in general needs work.  As it stands, it
requires various applications to implement their own flow control
protocols which increases the chances of people doing the wrong things
(or not at all) to the detriment of well behaved TCP traffics and
Internet stability.

It would be nice to see an UDP-LT (Lite) which is the same as UDP today
(with or without your partial checksum suggestion) and a UDP-BT (bulk
trnasport) which has flow control but not jitter control (for
simplicity) or error recovery (use TCP).  The LT version is to be used
for transaction type traffic such as DNS while the BT version for
multimedia stuff.   Having a standard UDP-BT will reduces the
proliferation of various quesionable or work-in-progress flow control
schemes.

Anyone else thought of doing similar thing?

Regards,
John Leong

---

From: Christian Huitema <[email-masked]>
Date: Wed, 18 Aug 1999 16:22:35 -0400

>applications such as voice and video over error-prone wireless 

Isn't the whole point moot if you start encrypting the packets? In that
case, one bit of transmission error will quickly spread into a whole raft
of decryption errors, which makes the packet essentially unusable.

Also, the more compression you use, the worse the impact of any error. An
error in an MPEG or JPEG encoding translates in an 8x8 or 16x16 artifact on
the screen.  In addition, many of the voice and video codecs are
differential, which means that a tiny error will pollute the decoded
results for a long time.

I don't believe that we should spend a lot of effort to make sure that we
can process bad data.  Erroneous deliveries should be rare enough in the
first place...

---

From: Joe Macker <[email-masked]>
Date: Wed, 18 Aug 1999 19:01:05 -0400

>overhead is less than that of a centralized ACK/repeat mechanism.

In addition, applying packet level FEC to multicast protocols in a reactive
manner (only for parity repairing or ARQ), has potential benefits with less
or equal overhead of explicit retransmission (assuming you use a good
erasure coding methods).   In PILC scenarios, perhaps useful for wireless
operation where uncorrelated group packet loss (even within local
broadcast) may be further increased by receiver-localized characteristics.
This can also be an additional motivator for proactive application, as
discussed, or hybrid application (a mixture of proactive and reactive).  Of
course, effective link layer coding considerations are essential to a
degree...just don't want to overdo it too much here.

-Joe

---

From: Mikael Degermark <[email-masked]>
Date: Thu, 19 Aug 1999 06:49:14 +0200

>of decryption errors, which makes the packet essentially unusable.

With current IPSEC encryption, yes. But there are encryption techniques 
where errors don't spread that much. GSM, for instance, encrypts at the
link layer and the GSM codecs are specifically can deal with errors.

>differential, which means that a tiny error will pollute the decoded

Still, voice codecs for cellular voice can use at least some of the 
damaged frames. One gets higher voice quality for less money if the
system is designed that way. Should we make the Internet more expensive
that it needs to be?

>can process bad data.  Erroneous deliveries should be rare enough in the

Nope. Not on the links that the 1000 million mobile "phone" users 
will use to attach themselves to the Internet with.

Mikael Degermark

---

From: Sampo A Syreeni <[email-masked]>
Date: Thu, 19 Aug 1999 10:34:27 +0300 (EETDST)

>link layer and the GSM codecs are specifically can deal with errors.

I believe only nonchained block ciphers and those stream ciphers which rely
on synchronized pseudorandom bit strings survive errors. GSM is of the
latter variety and isn't considered especially safe.

>system is designed that way.

That's because GSM codecs rely on fixed length frames and entropy coding is
not used. That leads to the data being larger than it could be. Moreover,
one may want to use something else than CELP based coders on the Net, since
speech is not the only audio signal to be transmitted. Generally speaking,
very low bitrate non-speech specific coders employ lots of long term
prediction and statistical compression - they do not react well to bit
errors.

>will use to attach themselves to the Internet with.

A better place to attack most of those errors is at the link layer - with 3G
mobiles we have a pretty good picture of common error patterns and can
deal with most of them by FEC. Beyond that, I agree. The air interface is
the most likely place to introduce errors in mobiles. If a packet gets as
far as to reach the mobile, it should not be discarded.

Sampo Syreeni <[email-masked]>, aka decoy, student/math/Helsinki university

---

From: Lloyd Wood <[email-masked]>
Date: Thu, 19 Aug 1999 16:04:25 +0100 (BST)

> latter variety and isn't considered especially safe.

And how well GSM link-level encryption survives errors is a moot point
when carrying IPSec packets.

L.

---

From: Steve Deering <[email-masked]>
Date: Thu, 19 Aug 1999 08:35:43 -0700

At 6:49 AM +0200 8/19/99, Mikael Degermark wrote:
>Should we make the Internet more expensive that it needs to be?

Sometimes yes.  Cost (efficiency or monetary expense) is only one of
the factors to weigh when making all the tradeoffs required to design
a workable system.  Sometimes issues of simplicity, manageability,
robustness, etc. should be given equal or greater weight than cost,
and often one dimension of cost (e.g., link bandwidth overhead) must
be traded off against other dimensions of cost (e.g., routing state).
If all we cared about was bandwidth cost, we would have converged on
X.25 instead of IP, since IP incurs the cost of those bigger headers
on every packet (which provide for robustness and reduced routing state);
we wouldn't have relied on you coming along and doing such fine work
in header compression. :-)

(I'm not taking a position for or against your specific proposal
regarding checksums, just against your argument that cost reduction
is paramount.)

Steve

---

From: Keith Hogie <[email-masked]>
Date: Thu, 19 Aug 1999 14:21:15 -0400

> prediction and statistical compression - they do not react well to bit

Many NASA satellites use FEC like Reed/Solomon.  However, all the modes
we use
have fixed length data blocks with R/S check symbols at the end.  These
do not
work for any sort of FEC on individual packets since the packets are variable
length and the R/S block is not.  

I much prefer using an FEC as simply a mechanism for cleaning up the
data link
and being completely independent of any upper layer data structures.  I
have seen
some NASA satellite where FEC has been used to treat the data as a
bitstream and
others which synchronize the data frames with the R/S frame.  Bitstream
FEC 
provides a very simple, flexible and transparent link enhancemnt.  Synchronizing
FEC with frames creates lot more custom configuration, management and 
data processing problems. 

Most importantly, from processing lots of satellite data, if a data link frame
fails R/S, you really don't want to see it because it is real garbage. 
I'm 
not aware of all the possible FEC options that might support variable length
FEC but I still feel FEC is best done at the data link layer.  Otherwise we
start moving toward putting it at the transport layer too. But then the 
application folks can always come along and put FEC on the data before
they 
shove it in the socket too.  Everyone can start compensating for what they
see a deficiencies in other layers.

A big strength of the Internet is in hiding the link level details from the
upper layers.  Passing status between layers and having one layer start 
taking on functions of other layers just seems to start breaking a protocol
model that while not 100% perfect, has proven to be very flexible and robust.

> Sampo Syreeni <[email-masked]>, aka decoy, student/math/Helsinki university

At NASA/GSFC we are very interested in things like IP header compression and
would like to start working with people to get them on a satellite in
the next 2 or 3 years.

----------------------------------------------------------------------
 Keith Hogie                   e-mail: [email-masked]
 Computer Sciences Corp.       office: 301-794-2999  fax: 301-794-8920
 7700 Hubble Dr.              
 Lanham-Seabrook, MD 20706  USA        301-286-3203 @ NASA/Goddard
----------------------------------------------------------------------