Subject: [Ietf-calsify] Character set restriction in section 4.3.11 Text

From: ned.freed at mrochek.com (Ned Freed)
Date: Mon Sep 11 09:20:46 2006

> Mark Crispin wrote:

> > On Mon, 21 Aug 2006, Alexey Melnikov wrote:
> >
> >>>   If any revision is made, it should be to delete US-ASCII and make
> >>> UTF-8 mandatory.
> >>
> >> However we can't do this, as absence of the charset parameter means
> >> US-ASCII.
> >> So any iCalendar object in UTF-8 MUST have charset=utf-8.
> >
> > That's unfortunate.  I admit that I am not up to speed on this
> > specification; is there really no way to fix this problem?

> To clarify, the default charset=US-ASCII comes from MIME specs.
> I am not sure if text/calendar can change the default character set
> used. Somebody with better MIME expertise should clarify this.

This is a tricky point in MIME. The idea behind the text top-level type was in
part to support fallback display of textual material even when the specific
type isn't understood. To this end, text types are required to use the charset
parameter in a manner that's compatible with text/plain. text/plain uses the
charset parameter to specify the charset (duh) and has a default of US-ASCII.
See RFC 2046 section 4.1.2 for specifics.

In any case, RFC 2046 recommands but does not actually require that all text
types have the same default of US-ASCII. And pretty much the first other
text subtype, text/html, opted for iso-8859-1 as it's default. In retrospect
UTF-8 would have been a better choice for both text/plain and text/html, but
please remember that UTF-8 didn't even exist at the time this stuff was
defined.

So what's the right course of action? I think the right thing to do is
make the default UTF-8 but strongly recommend that the charset parameter
be present.

> > If there is some way of defining the charset as always being UTF-8,
> > then charset parameters can be removed from the specification as
> > extraneous.

> Yes, that would be nice.

Nice but not really possible. See above.

				Ned

---

From: bernard.desruisseaux at oracle.com (Bernard Desruisseaux)
Date: Thu Sep 21 11:13:26 2006

Folks, I would like to close issue 31 with a new proposal:

- UTF-8 and US-ASCII are the only character set supported by iCalendar.
- The use of other character set is not forbidden.
- Compliant applications are REQUIRED to accept UTF-8 and US-ASCII.
- Compliant applications SHOULD generate iCalendar objects in UTF-8.

Here's the proposed text changes:

Section 3.2 Parameters

OLD TEXT:

> The "charset" parameter is defined in [RFC 2046] for other body
> parts. It is used to identify the default character set used within
> the body part.

NEW TEXT:

> The "charset" parameter is defined in [RFC 2046] for subtypes of
> the "text" media type. It is used to indicate the character set
> used in the body part. The character sets supported by this revision
> of iCalendar are UTF-8 and US-ASCII.

Section 4.1.4 Character Set

OLD TEXT:

> There is not a property parameter to declare the character set used
> in a property value. The default character set for an iCalendar
> object is UTF-8 as defined in [RFC 2279].
>
> The "charset" Content-Type parameter can be used in MIME transports
> to specify any other IANA registered character set.

NEW TEXT:

> There is not a property parameter to declare the character set used
> in a property value. The default character set for an iCalendar
> object is UTF-8 as defined in [RFC 3629].
>
> The character sets supported by this revision of iCalendar are UTF-8
> and US-ASCII. Compliant applications MUST accept iCalendar objects
> using the UTF-8 or US-ASCII character set and SHOULD generate
> iCalendar object using the UTF-8 character set.
>
> The "charset" Content-Type parameter can be used in MIME transports
> to specify the character set being used.

Section 4.3.11 Text

OLD TEXT:

> Formal Definition: The character sets supported by this revision of
> iCalendar are UTF-8 and US ASCII thereof. The applicability to other
> character sets is for future work. The value type is defined by the
> following notation.

NEW TEXT:

> Formal Definition: The value type is defined by the following notation:

Cheers,
Bernard

Bernard Desruisseaux wrote:
> Hi Mark,
> 
> I don't feel that it is important for new work to support any
> character set other than UTF-8, but I'm not sure whether rfc2445bis
> should be considered as new work though!
> 
> In my proposal I only quoted section 4.3.11 which makes clear that
> UTF-8 and US-ASCII are the only character sets supported by iCalendar.
> I should probably have quoted the other sections where character sets
> are discussed, that is, sections 2.3, 3.2, and more importantly 4.1.4:
> 
> Section 4.1.4 Character Set
> 
>  > The "charset" Content-Type parameter can be used in MIME transports
>  > to specify any other IANA registered character set.
> 
> Section 4.3.11 says that only UTF-8 and US-ASCII are supported.
> While section 4.1.4 says that you can specify any IANA registered
> character set in the "charset" parameter...
> 
> The result is that some iCalendar applications currently output
> iCalendar objects in ISO-8859-1.
> 
> I see two options:
> 
> a) We modify section 4.3.11 as I proposed. But now I realize
>    that we would also need to modify all the ABNF rules that
>    makes reference to the numerical value of specific characters
>    (e.g., LF = %x0A in US-ASCII but %x25 in EBCDIC). Yuk!
> 
> b) We modify sections 3.2, 4.1.4 and 4.3.11 to be in sync and
>    clearly specify which character sets are supported by iCalendar:
> 
>     b.1) UTF-8 only.
>     b.2) UTF-8 and US-ASCII only.
>     b.3) UTF-8, US-ASCII, and ISO-8859-1 only.
> 
> At this point I favor option (b.3). It looks to me as the easiest
> way to clarify the specification without breaking most (if not all)
> existing iCalendar applications.
> 
> Cheers,
> Bernard
> 
> Mark Crispin wrote:
>> On Mon, 21 Aug 2006, Bernard Desruisseaux wrote:
>>> I don't see any reason why it states that only UTF-8 and US-ASCII
>>> are supported by this revision. Section "3.2 Parameter" doesn't
>>> specify any restriction on the values allowed for the "charset"
>>> parameter.
>>> I propose to remove this statement from section 4.3.11.
>>
>> Can you explain why you feel that it is important for new work to 
>> support any character set other than UTF-8?  If any revision is made, 
>> it should be to delete US-ASCII and make UTF-8 mandatory.
>>
>> -- Mark --
>>
>> http://panda.com/mrc
>> Democracy is two wolves and a sheep deciding what to eat for lunch.
>> Liberty is a well-armed sheep contesting the vote.
> 
> 
> _______________________________________________
> Ietf-calsify mailing list
> [email-masked]
> http://lists.osafoundation.org/mailman/listinfo/ietf-calsify

---

From: cyrus at daboo.name (Cyrus Daboo)
Date: Thu Sep 21 11:22:09 2006

Hi Bernard,

--On September 21, 2006 2:11:49 PM -0400 Bernard Desruisseaux 
<[email-masked]> wrote:

+1 to your new text on charset.

-- 
Cyrus Daboo

---

From: mrc at CAC.Washington.EDU (Mark Crispin)
Date: Thu Sep 21 11:33:02 2006

> - The use of other character set is not forbidden.

Can it be deprecated though?

US-ASCII, UTF-8, etc. are charsets, not character sets.  This is 
particularly important with UTF-8; the character set is Unicode, not 
UTF-8.

Finally, since you already state that UTF-8 SHOULD be generated, there 
is no reason that US-ASCII be listed as a supported charset.  It 
sufficies to say, as you do in 4.1.4, that US-ASCII must be accepted.

Suggested wording:

 	The "charset" parameter is defined in [RFC 2046] for subtypes of
 	the "text" media type. It is used to indicate the character set
 	used in the body part. The charset supported by this revision of
 	iCalendar is UTF-8. The use of any other charset is deprecated by
 	this revision of iCalendar; however note that this revision
 	requires that compliant applications MUST accept iCalendar objects
 	using either the UTF-8 or US-ASCII charset.

Also change other occurances of "character set" to "charset".

-- Mark --

http://panda.com/mrc

---

From: alexey.melnikov at isode.com (Alexey Melnikov)
Date: Thu Sep 21 12:00:17 2006

>> used in the body part. The character sets supported by this revision
As per Mark's comment: replace "character set" with "charset"

>> There is not a property parameter to declare the character set used
s/character set/charset

>> in a property value. The default character set for an iCalendar
s/character set/charset

>> The character sets supported by this revision of iCalendar are UTF-8
s/character sets/charsets

>> to specify the character set being used.
I would change the last sentence to become normative:

The "charset" Content-Type parameter MUST be used in MIME transports
to specify the charset being used.

(this MUST is irrelevant for US-ASCII case, but I can't think of any 
better way of saying that)

>> Formal Definition: The value type is defined by the following notation:
Otherwise I am fine with this proposal.

---

From: alexey.melnikov at isode.com (Alexey Melnikov)
Date: Thu Sep 21 12:02:07 2006

>     using either the UTF-8 or US-ASCII charset.

I can live with this alternative text as well.

---

From: lear at cisco.com (Eliot Lear)
Date: Thu Sep 21 12:44:30 2006

Are there objections to Mark's proposed wording?

Eliot