Subject: Re: [idn] Re: 7 bits forever!

From: "D. J. Bernstein" <[email-masked]>
Date: 1 Apr 2002 08:35:14 -0000

Claus Faerber writes:
> D. J. Bernstein <[email-masked]> schrieb/wrote:
> > I'm not saying that Quoted-Printable had no short-term benefits for its
> > short-term costs. I'm saying that, viewed from our long-term perspective
> > eleven years later, the failure to require 8-bit transparency was an
> > amazingly stupid decision.
> From our present perspective, that's true. Back then, it might
> have been the best solution.

No. The failure to require 8-bit transparency was inexcusable. Every
approach that failed to require 8-bit transparency could have been
dramatically improved. Consider, for example, these three approaches:

   (1) Quoted-Printable;
   (2) Quoted-Printable plus ``you must handle unencoded 8-bit data too'';
   (3) pure 8-bit without this 7-bit garbage.

Whether or not you understand that #3 would have been better than #2,
surely you understand that #2 would have been vastly better than #1.

> Further, remember that the first MIME standards date back to 1992.
> Back then, Unicode was brand-new and UTF-8 only came with the 2.0
> version in 1996. Without UTF-8, you just could not even think
> about using Unicode in message headers; and without Unicode, you
> could not solve the charset-labelling problem.

Get your facts straight. First, UTF-8 was introduced years before 1996,
although under another name. Second, even without knowing about UTF-8,
people _were_ thinking about Unicode in headers, and proposed several
workable approaches. Third, even without Unicode, there were several
solutions to the character-set labelling problem.

Anyway, none of this is relevant to IETF's acceptance of 7-bit garbage
in 1991, and none of it is relevant to IETF's acceptance of 7-bit
garbage today.

> The movement towards UTF-8 everywhere is quite new.

Again, get your facts straight. The ``movement'' started with X-Open,
Rob Pike, and Ken Thompson a decade ago. RFC 2277, requiring UTF-8
support for all text on the Internet, is four years old.

---D. J. Bernstein, Associate Professor, Department of Mathematics,
Statistics, and Computer Science, University of Illinois at Chicago

---

From: Einar Stefferud <[email-masked]>
Date: Mon, 1 Apr 2002 08:59:19 -0800

Not to pick on Dan's particular message.
It was just the last one to provide me with a handy response point.

Looking back, the original 1990-1991 argument was between "Just send 8 bits"
with no tagging of the content, vs "tagging and bagging" to deal with 
the fact that 8-bit at that time would break too many systems, and 
the problem of needing to tag the types of text characters that were 
contained in RFC822 message bodies.

But, as there were two problems to solve (8 bit SMTP ) and (tagging 
RFC822 content), the decision was made (after a very long and very 
heated argument) to split that problem according to the boundaries of 
SMTP and RFC822.

In fact, the two problems were and still are totally distinct.

The RFC822 solution group produced MIME with Quoted-Printable as its 
charter specified, to assure interworkability as soon as possible.

The 8-bit folks did not do anything then or since to require 8-bit carriage.

So, it is time for a working group to solve this problem.

It is not going to help anyone to argue about the past, as in fact, 
it was very rational at the time, and one side of the problem was 
ignored as soon as quoted printable relieved the immediate pressure 
for an SMTP solution.

So, now is a good time for some constructive discussion of ways to 
solve the charset problems, hopefully with some useful charset 
tagging tools for the non-RFC-822 parts of the SMTP envelope.  (But I 
do not mean to specify either the work or the results at this point.)

It is indeed unfortunate the the DRUMS WG did not find a way to solve 
the 8-bit SMTP problem, but now is the time to get to it.  There 
should not be too much 7-bit SMTP code to fix by now.

Cheers...\Stef

---

From: [email-masked]
Date: Mon, 01 Apr 2002 12:31:59 -0500

> should not be too much 7-bit SMTP code to fix by now.

One hidden gotcha is that although most MTA's are probably either 8-bit-clean
or fairly easily rendered so, there is *NOT* a widespread deployment of the
RFC3030 ESMTP extensions.  As a result, you still get bit by the "need a
CRLF pair every 998 characters" problem.
-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech

---

From: [email-masked] (Claus Färber)
Date: 11 Apr 2002 02:11:00 +0200

> Whether or not you understand that #3 would have been better than #2,

... which is irrelevant today as quoted-printable is widely  
supported ...

> surely you understand that #2 would have been vastly better than #1.

You're implying that we've had solution number (1). This is wrong,  
it is more like:
   (4) Quoted-Printable plus 'you may handle unencoded 8bit data'.

As of today, most systems *do* handle 8bit data without problems.  
So there's no point in arguing that a "must handle" would have  
improved the situation.

You can argue, of course, whether the autoconversion from 8bit to  
qp is a good thing. On one hand, it does preserve compatibility  
with old 8th bit murderer systems, on the other hand, it is often  
unnecessary a imposes a risk of corruption of these message.

> workable approaches.

Yeah, me might have "UTF-1 garbage", or even a more obsolete  
Unicode Transformation format.

> Third, even without Unicode, there were several solutions to the
> character-set labelling problem.

Any of these would have to been just as ugly as RFC 2047 is if the  
UA does not understand the label. So what?

> support for all text on the Internet, is four years old.

You have to look at the software that actually supports Unicode,  
not about programmatic decisions for future standards. One or two  
years ago, UTF-8 support in actual software was an exception. Most  
software has only recently been updated to support UTF-8. Even  
today, a lot of software does not support or is not configured for  
UTF-8 by default.

Claus
-- 
------------------------ http://www.faerber.muc.de/ ------------------------
OpenPGP: DSS 1024/639680F0 E7A8 AADB 6C8A 2450 67EA AF68 48A5 0E63 6396 80F0