Subject: Re: [Ietf-message-headers] Re: Jabber-ID header field

From: Bruce Lilly <[email-masked]>
Date: Sun, 1 Oct 2006 23:08:08 -0400

On Thu September 28 2006 18:54, Frank Ellermann wrote:
> Bruce Lilly wrote:

> > some proposed syntax conflicts with RFC 2822 (e.g. ';' is
> > an RFC 2822 "special").
> 
> Grmbl, yes.  Do you have a tool for this ?

None needed; the list of RFC 822/2822 specials is small and easily
remembered -- when I see [RFC 4622]

     pathxmpp  = [ nodeid "@" ] host [ "/" resid ]
     nodeid    = *( unreserved / pct-encoded / nodeallow )
     nodeallow = "!" / "$" / "(" / ")" / "*" / "+" / "," / ";" /
                 "=" / "[" / "\" / "]" / "^" / "`" / "{" / "|" /
                 "}"
     resid     = *( unreserved / pct-encoded / resallow )
     resallow   = "!" / DQUOTE / "$" / "&" / "'" / "(" / ")" /
                  "*" / "+" / "," / ":" / ";" / "<" / "=" / ">" /
                  "[" / "\" / "]" / "^" / "`" / "{" / "|" / "}"

I can immediately see (in "resallow") conflicts DQUOTE, "(", ")",
":", ";", "<", ">", "[", "]", and ",", and of course "@" in
pathxmpp itself.  The "nodeallow" construct adds "\".  Indeed "." seems to be
the only 822/2822 special not directly in conflict with 4622 [and that is
pulled in via 3986 "unreserved"].  Some of these ("@", ".",) are unlikely to
cause trouble; others are likely to cause varying degrees of trouble
depending on parser construction.  N.B. [RFC 2822]:

   Each of the characters in specials can be used to indicate
   a tokenization point in lexical analysis.

Unbalanced use of specials which (in 2822 fields) always appear in properly
nested and balanced pairs, but which 4622 does not require to be balanced or
properly nested are highly likely to cause trouble except for the most robust
of parsers. That includes DQUOTE, parentheses, colon/semicolon, square
brackets, and angle brackets (N.B. all but dot, comma, and backslash).
Backslash is also highly likely to cause trouble as it is used for quoting
and has some rather baroque rules about where it may legally be used for
quoting (presumably just a plain character elsewhere) -- and many
implementations run afoul of those rules.

Now percent-encoding can be used to avoid troublesome direct use of certain
characters, e.g. as implied by RFC 2369.  However, there remain several sticky
problems:
1. obviously, the particular troublesome characters should be explicitly
   listed in the ABNF and normative text
2. from a practical point of view, to make any use of the content, encoding
   and decoding functions need to be available.  These will have to be
   defined or reference made to an existing definition.  But RFC 2369
   references RFC 1738 whereas RFC 4622 references RFC 3986, and the lists of
   "reserved" / "unreserved" characters are markedly different between 1738
   and 3986.  Given the utility of the RFC 2369 (and 2919) fields in a mail
   environment, and the corresponding near-complete separation of mail and
   chat applications (except possibly in one old implementation of one client
   line) means that the 1738 characters are going to continue to be used [for
   backward compatibility], no matter what RFC 3986 says.
3. The combination of the necessity of encoding/decoding and the separation
   of client functionality means that naive copy-and-paste (between MUA and
   chat applications) is unlikely to work well.  That is one reason why a
   transparent (i.e. not requiring specialized encoding) mechanism such as a
   media type is infinitely preferable to a highly specialized encoding of a
   (probably undisplayed) message header field; a media type whose content is
   unencoded CAN be copied-and-pasted even with naive client implementations,
   and can be easily saved to a file for export/import in cases where
   copy-and-paste is not provided among applications.

> > I believe that the peculiarity of use of an obs- construct
> > in a proposed new field has already been noted.
> 
> That's just the known issue of "no folding in a FWS directly
> before a CRLF" combined with the 2822 accept obs-FWS MUSTard
> in a pending erratum.

With one exception, the obs- constructs in 2822 exist to accommodate legal
822 and earlier syntax generation of legacy fields.  There is no reason for a
newly defined field to incorporate any obs- construct; 2822 rules prohibit
new generation of obs- constructs in any case, and precisely because the
field is new, there are no 822 or pre-822 generators of the field to worry
about.

> > exactly which mailbox or set of mailboxes corresponds to the
> > draft use of the word "sender"?
> >         Resent-Sender: [email-masked]
> >         Resent-From: [email-masked], [email-masked]
> >         From: [email-masked], [email-masked]
> >         Sender: (see RFC 3192) "FAX=+33-1-88335215"@l.com
> >         Reply-To: foo list: [email-masked], [email-masked];
> 
> It's no Resent-Jabber-ID, the Resent-* are out.

So where is the prohibition against a resender adding such a field?  For
that matter, where is the specification of who may insert/modify/interpret
such a field (see RFC 4249 sect. 3.3.1)?

>   The
> I-D says that the Sender sets the Jabber-ID.

The draft does not capitalize "sender", nor does it explicitly state whether
"sender" means the mailbox in the Sender field if present; neither does it
state what to use if the optional Sender field (in the case of a sole author)
is absent.  Note also [RFC 822]

        This field contains the authenticated identity  of  the  AGENT
        (person,  system  or  process)  that sends the message.  It is
        intended for use when the sender is not the author of the mes-
        sage,  or  to  indicate  who among a group of authors actually
        sent the message.  If the contents of the "Sender" field would
        be  completely  redundant  with  the  "From"  field,  then the
        "Sender" field need not be present and its use is  discouraged
        (though  still legal).  In particular, the "Sender" field MUST
        be present if it is NOT the same as the "From" Field.

        The Sender mailbox  specification  includes  a  word  sequence
        which  must correspond to a specific agent (i.e., a human user
        or a computer program) rather than a standard  address.   This
        indicates  the  expectation  that  the field will identify the
        single AGENT (person,  system,  or  process)  responsible  for
        sending  the mail and not simply include the name of a mailbox
        from which the mail was sent.  For example in the  case  of  a
        shared login name, the name, by itself, would not be adequate.
        The local-part address unit, which refers to  this  agent,  is
        expected to be a computer system term, and not (for example) a
        generalized person reference which can  be  used  outside  the
        network text message context.

N.B. the Sender field does not necessarily reference a person's mailbox
and in general is not usable for addressing (RFC 3834 section 4 
reinforces these facts and emphasizes that the field need not be valid
for replies).

> > There appears to be no provision for line folding within an
> > ID (RFC 4622 "pathxmpp" production).
> 
> Yes, same issue as for <msg-id> and <addr-spec> in RFC 2822.

In mail (the applicable application for addr-spec in messages), the length of
an addr-spec local-part is bounded by the SMTP protocol specifications, and
(contrary to your implication) line folding in addr-spec is permitted by RFC
2822 on either side of the "@" by [CFWS] trailing local-part (via dot-atom
and quoted-string) on the left, and by leading [CFWS] on the right (dot-atom
and domain-literal).  The length of the text form of a domain name is bounded
(255 octets) by the DNS specifications; domain literals are likewise of
bounded length.  As for msg-id, the domain name or literal bounds still
apply; the left-hand-side is limited in length by the overall line length
limit of 998 octets due to the prohibition of folding within msg-id.

However, URIs and URI-derived constructs typically do not specify upper
bounds -- indeed both "nodeid" and "resid" relevant to the draft under
discussion are specified with neither lower nor upper bounds in the
referenced RFC 4622.

> What do you propose, a statement "make sure that a Jabber-ID
> isn't longer than 997 octets" ?

No, unless it is proposed that the draft contain an amendment of 4622
imposing upper bounds on "nodeid" and "resid" such that "pathxmpp" can
never be longer than 997 octets (and that would have to include any
encoding, etc.).  Otherwise, there's not much point in defining a field
that simply will not work for some subset of otherwise valid (outside of
the Internet Message Format) IDs.  Aside from the recommended method of
simply avoiding the whole sticky mess by using a media type, it might (or
might not) be possible to adapt existing mechanisms for handling long URIs
(RFCs 2017, 2231 (probably not, as its scope is MIME field parameters), and
2369).

> > What happens when an ID exceeds 998 octets in length
> 
> The nice MSA will reject the message as specified in RFC 4409.

MSAs are optional components.  Whether or not an MSA is present, the message
is likely to bounce back to the designated handler for error responses (as
specified via SMTP MAIL FROM); in the subset of cases where that is the
message initiator, the only fallback mechanism for conveying such an ID
with the message would be a suitably-tagged MIME-part which is transfer
encoded using MIME mechanisms (including the "binary" identity transformation
if that is supported by SMTP body transport).  So if a suitable media type
needs to be defined anyway (for fallback), then one might as well use that
as the primary mechanism (with all of the advantages noted in my earlier
messages), avoiding the pitfalls associated with the proposed field. [In
cases where the message initiator has delegated handling of errors, the
initiator may be unaware of a failure; note that such failure would prevent
delivery of not only the errant proposed field, but the entire message!]

> Your media type proposal strikes me as odd, the JID could then
> be also added to some existing vcard or similar "attachment"...

Possibly, if the syntax is compatible with the existing media type (esp.
encoding; N.B. 7bit and 8bit encodings still retain the 998 octet line
length limit).  If it is in fact compatible, I have no objection to a
suitable modification of that existing type [it would of course have to go
through the media type review process, and I have no idea what comments that
might evoke, though I expect that the 1738/3986 conflict would arise (RFC
2426 references 1738)].  I suspect that due not only to changes from 1738
through 2396 to 3986 but also from the earlier registration procedures to the
current RFCs 4288/4289 (BCP 13), an RFC 2426 successor would be needed, and
that is likely to be a non-trivial task (references would need to be
organized as Normative vs. Informative, boilerplate revised, etc.; if the
revision is intended as Standards Track progress (to Draft) then an
implementation report is required).  Likewise, resolving all of the problems
noted with conflicts between 2822 and the proposed field would likely be
non-trivial (and would still leave unresolved practical problems related to
copy-and-paste, etc.).  A new application media subtype seems to be the fast
track to getting the desired functionality.
 
> Nobody would use it

And your evidence to support that assertion is...? (N.B. vcards ARE in
use)

> With the 
> JID as header field you implicitly have the vcard,

No, vcards are not header fields, they are in fact conveyed as a MIME media
type.

> if you have 
> an UA suppporting to retrieve the user info of the given JID.

And the UAs which do so are...? (unlikely to exist, due to the 1738 vs.
3986 conflicts and the above noted separation of client functionality)

_______________________________________________
Ietf-message-headers mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf-message-headers

---

From: Bruce Lilly <[email-masked]>
Date: Mon, 2 Oct 2006 01:23:07 -0400

On Fri September 29 2006 05:16, Graham Klyne wrote:
> [This message is more about reviewing RFC3864 than the specific Jabber-id header]
> 
> Frank Ellermann wrote:
> >> Why not start with registration in the provisional registry
> > 
> > Permanent and RFC is better for MUA implementors.
> 
> Sure, ...
> 
> But, in general, I note that requesting a provisional registration in no way
> precludes moving later to a permanent registration.
> 
> And the barrier for provisional registration being so much lower (this the
> Jabber-ID draft would be sufficient), it's easy to get a record of its intended
> purpose published where developers can find it.  I would expect that a header
> destined for the standards track would be provisionally registered pretty much
> as soon as it is first published.

Agreed so far.

> In this case, it looks as things might move quite quickly, so going straight for
> the permanent registry saves some administration.

I'm not so sure -- there seem to be a significant number of issues that need
to be resolved; at minimum, starting with provisional would give interested
parties an opportunity to work out the bugs before moving to permanent, while
reserving the field name.

> I'd like to think it can be 
> done as soon as IESG approves the draft for standards-track publication.  Hmmm,
> checking [http://www.rfc-editor.org/rfc/rfc3864.txt]:
> 
> [[
>       The assignment policy for
>       such registration is "Specification Required", as defined by RFC
>       2434 [3], where the specification must be published in an RFC
>       (standards-track, experimental, informational or historic), or as
>       an "Open Standard" in the sense of RFC 2026, section 7 [1].
> ]]
> 
> That text is a bit limiting... maybe also add ", or has been approved for such
> publication".

As 3864's primary co-author, you are of course in a better position than I am
to interpret the intent of 3864 as written.   That said, I read it as "RFC first
(with all of the discourse and review required for that), then assignment".
Indeed, all but two of the existing Internet Message Format field assignments in
the permanent registry are for fields which have long been defined in published
RFCs.

[The converse is another story; RFC 4021 omitted a number of RFC-defined fields.
Draft-lilly-legacy-fields is (was?) an attempt to resolve some of those, but it
seems to have gotten bogged down...  Those who look at the archive or who have
been watching this list since its inception may recall that that draft was
written in April 2005 as announced here.]

The case under discussion here differs from all of those legacy fields.  It is
also somewhat of a test case, as it appears to be the first non-legacy field to
be discussed on this list [Martin Duerst made an inquiry nearly seven months ago,
but that seems to have evoked no discussion].  I think the discussion is good; a
number of relevant issues have been raised.  Compare that to the two new Internet
Message Format fields that have been registered in the permanent registry without
discussion here; neither is particularly useful and both have either syntax or
semantics problems that might have been resolved had the fields been discussed
here.

> Looking over the text I'd also be inclined to indicate more explicitly that the
> IESG also has discretion to approve registration of any entry in the permanent
> registry, to avoid potential wrangling over such small points.

"Discretion to approve" is fine, but there still needs to be an approved open
specification and a registration template.

BCP 90 (RFC 3864) also says that the registration templates should be sent to this
discussion list (section 4.3) or incorporated in a defining RFC.

Given the aforementioned issues, I'd prefer to see discussion here required even
if the registration template is (presumably subsequently) published in a defining
RFC.

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 02 Oct 2006 07:20:57 +0200

Bruce Lilly wrote:

>> Do you have a tool for this ?

> None needed; the list of RFC 822/2822 specials is small and
> easily remembered

Admittedly I often lose track with specials / tspecials / uric
and other important subsets of VCHAR.

> I can immediately see (in "resallow") conflicts DQUOTE, "(",
> ")", ":", ";", "<", ">", "[", "]", and ",", and of course
> "@" in pathxmpp itself.

For "(" and ")" we're clear without CFWS, or rather I claimed
that a Jabber-ID can't work as is with CFWS in
<http://article.gmane.org/gmane.ietf.rfc822/11777>

Some other specials might be also harmless.  But for DQUOTE and
backslash we're very far away from "like <id-left> @ <id-right>,
only different".

> the particular troublesome characters should be explicitly

Okay, a note that it's no <addr-spec>, even if it might look
like it, and also no <id-left> "@" <id-right>

> But RFC 2369 references RFC 1738 whereas RFC 4622 references
> RFC 3986, and the lists of "reserved" / "unreserved"
> characters are markedly different between 1738 and 3986.

Yes, RFC 2369 is too old, and the JID is no complete xmpp-URL.

> the 1738 characters are going to continue to be used [for

I doubt that, with RFC 1738 you'd exclude IPv6 literals.

> The combination of the necessity of encoding/decoding and
> the separation of client functionality means that naive
> copy-and-paste (between MUA and chat applications) is
> unlikely to work well.

That's why there is no tricky encoding in the JID, I'd expect
naive copy and paste to work.  UAs can support X-FACE or FACE,
they will also manage to support a JID.  The minimal support
"leave it alone" is good enough.

> There is no reason for a newly defined field to incorporate
> any obs- construct

It gets that automatically as soon as it says FWS, who knows
what old gateways or MDAs might do ?  In another discussion
you seriously argued that the order of header fields could be
rearranged by some utter dubious software.

Of course obs-FWS can't be generated under 2822 rules, but it
must be accepted if some pre-2822 entity introduces it.  For a
2822-generator the situation is clear, it can add trailing
white space, but not a bogus trailing folding.

Killing the complete trailing white space including any obs-FWS
might be an idea.  For the hidden obs-FWS in the leading FWS we
would need a "NNFWS" (no nonsense FWS) to eliminate its obs-FWS.

> So where is the prohibition against a resender adding such a
> field?

What resenders must or may do is specified in 2822, "add new
random header fields as you see fit" isn't.

> where is the specification of who may insert/modify/interpret

It's no trace header field, otherwise the draft would mention
it.  The number of _new_ trace header fields in published RFCs
in the last twenty years or so is AFAIK limited to one.

Skipping the Sender-part of your article, nobody suggested to
send mail to the address in a Sender header field.

> As for msg-id, the domain name or literal bounds still apply;
> the left-hand-side is limited in length by the overall line
> length limit of 998 octets due to the prohibition of folding
> within msg-id.

Right, that's also good enough for the JID.

> there's not much point in defining a field that simply will
> not work for some subset of otherwise valid (outside of the
> Internet Message Format) IDs.

I haven't met any JID with more than 100 characters so far,
but playing with it for less than a week that's just my first
personal impression.

> it might (or might not) be possible to adapt existing
> mechanisms for handling long URIs

We had that in USEFOR, yes.  In addition to ignoring leading and
trailing white space also ignore FWS within the field body.

> if a suitable media type needs to be defined anyway (for
> fallback), then one might as well use that as the primary
> mechanism

There probably are RFCs dealing with bounces of syntactically
invalid mails, for an error report folding the too long line
anywhere is an idea.

> In cases where the message initiator has delegated handling
> of errors, the initiator may be unaware of a failure

That's entirely the problem of the sender, the limit 1000 is
no new or suprising feature.

> I expect that the 1738/3986 conflict would arise (RFC
> 2426 references 1738)].

The im: URI scheme is based on <uric> in 2396.  If that's used
in some newer than RFC 2426 vcard-standard, I've no clue.

> A new application media subtype seems to be the fast track to
> getting the desired functionality.

> And your evidence to support that assertion is...? (N.B.
> vcards ARE in use)

Maybe, but not in any mails or news I care about.  If you want
a JID in attachments propose it, I'm not forced to use it, or
to be interested in it.

> vcards are not header fields, they are in fact conveyed as a
> MIME media type.

Yes, that's why they're not used, any multipart has excellent
chances to go to junk folders without questions asked.  Which
isn't nice from my POV, multipart/signed is decent in relation
to ugly plain text signatures.

As a header field I like the draft, as an attachment I wouldn't
care about it.

Frank

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 02 Oct 2006 07:59:23 +0200

Bruce Lilly wrote:

> Draft-lilly-legacy-fields is (was?) an attempt to resolve
> some of those, but it seems to have gotten bogged down...
[...]
> that that draft was written in April 2005 as announced here.

The 2nd of now 24 articles here:
http://article.gmane.org/gmane.ietf.message-headers:2

IIRC we discussed "Fcc" elsewhere.  What's the reason why you
don't ask for its publication ?  It's good to have such info
in the registry. 

> neither is particularly useful and both have either syntax or
> semantics problems that might have been resolved had the
> fields been discussed here.

Received-SPF was discussed elsewhere (actually it's the reason
why I read this list, but it never appeared here).  RFC 3865
was also discussed elsewhere.  My attempt to discuss the JID on
the rfc822 list failed.  Similar for Archived-At, I have that
article still on hold with date 2005-11-01 (rfc822 list).

Frank

---

From: Graham Klyne <[email-masked]>
Date: Mon, 02 Oct 2006 11:43:53 +0100

[...]

> reserving the field name.

(Maybe so:  initially, jabber-id was looking like a simple non-controversial
spec, so aiming for permanent was not an unreasonable aspiration.  But this
isn't really what matters here...)

> (with all of the discourse and review required for that), then assignment".

Well, that's how I read the text, too!  But my perspective is that the *intent*
is served by having permanent registration allowed as soon as standards-track
publication has been approved.

> The case under discussion [...]  is
> but that seems to have evoked no discussion].

Yup.  I can't recall if Martin was requesting provisional or permanent, though I
suspect it was the former, hence less scope for controversy.

> ...  I think the discussion is good; a

Ack.  I think most of us who've worked with this process over any period
recognize the value of peer review.  Sometimes, though, it can be challenging
getting peers to actually do some reviewing ;)

> if the registration template is (presumably subsequently) published in a defining

Well, yes to all of that.  My presumption here is that the IESG won't depart
from the documented procedures unless there is a compelling reason to do so.

Don't forget that registration, even permanent registration, does not of itself
constitute a standard or even any level of approval.   It is entirely
conceivable that one may wish to put an existing escaped-to-the-wild header
field name in the permanent registry with a health warning saying "DON'T use
this".  (The additional effect of preventing registration of any header with the
same protocol/name combination would seem to be mostly desirable in such a case,
if only for the avoidance of confusion.)

#g

-- 
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 06 Oct 2006 22:55:04 +0200

[...] 
> for DQUOTE and backslash we're very far away from 
> "like <id-left> @ <id-right>, only different".

Update, one 3986 author wrote that 4622 is broken, it can't
allow characters like '"', "<", ">", "[", "]" in URLs (but of
course "[" and "]" are okay in conjunction with IPv6 literals).

See also <http://permalink.gmane.org/gmane.org.w3c.uri/861> or
<http://www.w3.org/mid/[email-masked]> (this
MID style of Archived-At is clearly better than GMaNe's idea).

Frank

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 06 Oct 2006 23:10:48 +0200

> To see what happens I've now submitted Archive-At for netnews
> claiming that it already exists for mail (CC to Martin because
> he's the change controller of his I-D).

IANA told me that it has to be posted here for two weeks, here's
a copy of [IANA #27825] with the template below "additional info":

Contact Name :
Frank Ellermann

Contact Email :
[email-masked]

Type of Assignment :
Provisional message header field 'Archived-At' for netnews

Registry :
Provisional message header field (RFC 3864),
<http://www.iana.org/assignments/message-headers/prov-headers.html>

Description :
Archived-At is used on <http://gmane.org> for netnews
with exactly the same syntax and semantics as the
Archive-At header field for mail registered (but
not yet published) by Martin Du"rst 2004-11-01 in
<http://article.gmane.org/gmane.ietf.rfc822/10973>

Additional Info :
Header field name:
        Archived-At

Applicable protocol:
        netnews (RFC 1036)

Status:
        provisional

Author/Change controller:
        Martin Duerst, Aoyama Gakuin University,
        mailto:[email-masked]

Specification document(s):
       http://tools.ietf.org/html/draft-duerst-archived-at

Related information:
        It is planned to advance this registration to
        permanent by advancing the internet-draft to an RFC.
        The syntax of the header field may change.

---

From: Bruce Lilly <[email-masked]>
Date: Sat, 7 Oct 2006 09:41:38 -0400

On Fri October 6 2006 17:10, Frank Ellermann wrote:
[...]
[...]
> Archive-At header field for mail registered (but

I think you left out a 'd'.

---

From: Frank Ellermann <[email-masked]>
Date: Sat, 07 Oct 2006 16:46:24 +0200

> I think you left out a 'd'.

Yes, thanks.  If I don't forget it I'll tell them.  What
you saw is actually a copy of the confirmation mail sent
by their general registration form, "description" is the
5th field in this form.  

I've used their 6th field "additional info" for the 3864
template (header field name, applicable protocol, etc.)

Frank

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 27 Oct 2006 14:35:23 +0200

Updates about Archived-At:

- there's a -06 draft http://tools.ietf.org/html/draft-duerst-archived-at
- Martin's last call: http://permalink.gmane.org/gmane.ietf.rfc822/11817

  Usage example:  Options (1) - headers (2) - All (3) - copy (4) on its
  Archived-At URL - paste (5) here.  Five clicks to copy this URL, nice ;-)

- IANA asked what's going on with my registration attempt after two weeks,
  that was 13 days after I posted it here as recommended, therefore I've
  wrote them that this should be okay (explaining the typo found by Bruce),
  and sent them the URL of the article here (re: IANA #27825), Archived-At:
  http://permalink.gmane.org/gmane.ietf.message-headers/27

That's it for the moment, Archived-At is not yet published on the IANA page.

Frank