Subject: Re: draft-freed-sieve-in-xml status?

From: "Robert Burrell Donkin" <[email-masked]>
Date: Tue, 6 Jan 2009 23:01:57 +0000

On Mon, Dec 29, 2008 at 8:26 AM, Alexey Melnikov
<[email-masked]> wrote:
> Ned Freed wrote:
>>>>
>>>> But I'm sitll a little confused as to what you're asking for here. If
>>>> you're
>>>> asking for removal of the explicit inline XML syntax examples in favor
>>>> of a
>>>> more abstract approach, I'd be fine with that if there's a WG consensus
>>>> to make
>>>> such a change.
>>>>
>>>
>>> no - i'm very happy with the syntax examples
>>>    i would like to see the approach used in RFC 5023 (and others)
>>> adopted, adding a normative description of the XML and making the
>>> schema only informative.
>>>
>>
>> Personally, I find RFC 5023 approach, like the XOPEN object descriptions
>> it's
>> similar to, to be almost totally unreadable. Maybe it's the only
>> reasonable way
>> to do it when the element structure is quite complex, but that's not the
>> case
>> here.
>>
>
> Speaking as an individual: I tend to agree with Ned here. I prefer having
> some formal syntax for defining XML + some examples demonstrating different
> Sieve features.
> I think adding a descriptive definition of XML is a fair bit of work for
> little gain. However, I would not oppose to having some descriptive
> definition (which I think should be informative), especially if somebody
> suggests some text ;-).

(i'm unlikely to contribute to a flawed specification unless there is
some hope that it will be fixed)

using XML Schema as the formal definition means that decisions have to
be taken which are forced by the language which would not be required
by an alternative formal method.

1. a decision will be needed to either use the default namespace or
another namespace. this will mean either sacrificing user who are
still using parsers from the 1990s or sacrificing those who need
explicit namespacing.

2. a decision will also need to be taken about the mixture of domains.
mixing sieve with annotations about sieve means that the schema has to
be edited (to separate these concerns) before being used in modern
generators. unless this is supported at least implicitly by the
specification, it just means that an alternative specification will be
needed for these use cases.

ATM lack of namespace support and a failure to separate the concerns
in the design of schema means that the specification is useless to me
and many other developers using modern, namespace aware tools. by
applying methods used in other RFCs facing these issues, it should be
possible to create a specification suitable for both legacy and modern
tooling.

- robert

---

From: "Robert Burrell Donkin" <[email-masked]>
Date: Wed, 7 Jan 2009 00:57:51 +0000

On Sun, Dec 28, 2008 at 10:31 PM, Ned Freed <[email-masked]> wrote:
>
>> On Sun, Dec 14, 2008 at 9:59 PM, Ned Freed <[email-masked]> wrote:
>> >> On Sun, Dec 14, 2008 at 6:06 PM, Ned Freed <[email-masked]> wrote:
>> >> >> i note that the draft describes the infoset rather than defining it in
>> >> >> the standard way. is there a reason for this decision?
>> >> >
>> >> > I don't know what "the standard way" is you're referring to. Perhaps you
>> >> > could provide a reference to an RFC where this has been used?
>> >
>> >> AIUI XML is maintained by w3c (rather than IEFT) so is a
>> >> recommendation. http://www.w3.org/TR/xml-infoset/ is the current
>> >> document.
>> >
>> > Quite true, however, the IETF has its own specification for XML is supposed to
>> > be used in RFCs: RFC 3470. And while infosets are mentioned as one approach to
>> > specifying things about an XML format, there's no recommendation, let alone
>> > requirement, that they be used.
>> >
>> >> > This document is a little unusual in that it's defining a mapping of, if you
>> >> > will, a non-XML infoset onto XML. As such, the natural approach seemed to be to
>> >> > first discuss the structure of the language being mapped, then explain the
>> >> > mapping, and finish up with additional unique-to-XML semantics.
>> >
>> >> i agree that most of this arangement is natural. it's just jumping to
>> >> a schema seems - to me - a little premature and inflexible.
>> >
>> > First of all, the use of XML Schema is in fact too inflexible to be allowed
>> > to continue. The next revision will use Relax instead.
>
>> XML schema is flexible but the flexibility comes at the price of
>> readability. one of relax variants would be a better choice.
>
>> however (in my experience) the generative tools commonly used for XML
>> and web service binding, and editor generation tend not to offer good
>> relax support. IMO the draft should offer secondary informative XML
>> Schema or Schemata to assist developers using these tools.
>
> The problem is that the unique particle attribution limitation in XML Schema
> effectively precludes using it without some compromises. I am therefore opposed
> to continuing to include it.

adopting a standard prefix - sieve, say - is all that is required

is this really too much to ask?

>> > But I'm sitll a little confused as to what you're asking for here. If you're
>> > asking for removal of the explicit inline XML syntax examples in favor of a
>> > more abstract approach, I'd be fine with that if there's a WG consensus to make
>> > such a change.
>
> Personally, I find RFC 5023 approach, like the XOPEN object descriptions it's
> similar to, to be almost totally unreadable. Maybe it's the only reasonable way
> to do it when the element structure is quite complex, but that's not the case
> here.
>
> So, absent some fairly strong support for this from others in the group, I'm
> not going to pursue this.

is there anyone else in group - excepting you and myself - who cares
enough to contribute at all to this discussion?

>> >> sieve). there is a large and growing requirement for integration
>> >> between mail and enterprise systems (typically coding in Java and .NET
>> >> but also ruby and python). developers from enterprise backgrounds are
>> >> typically strong on web+xml but very weak on mail.
>> >
>> > Yep, I've seen a lot of this as well. And the problem emcompasses far more than
>> > Sieve: For example, a lot of people who are unfamiliar with email don't
>> > understand very basic concepts such as the separation between envelope and
>> > message content. (This particular issue actually pokes through into Sieve in
>> > the form of whether an envelope or header test is appropriate.)
>
>> i beg to differ slightly on this one
>
>> some enterprise mail processing may happen during the SMTP transaction
>> but it is more typical for the mail processing after storage. not all
>> mail stored arrives through SMTP and so it is typical for any envelope
>> information to be reduced to simple MIME headers.
>
> Robert, with all due respect, you may have substantial expertise on the XML
> front,  but your comments here are actually doing little more than illustrate
> the validity of my argument that there's a general issue with people not
> getting how email works that isn't going to get fixed by anything we do here.

ned - with all due respect - your comments illustrate a lack of
understanding of this class of mail server

> If you want this addressed the place to look is the email architecure
> specifications being worked on by Dave Crocker.
>
> And it is NOT typical for envelope information to be stored as headers.

for the class of application (enterprise mail servers is a name that's
sometimes used but quite possibly that's not familiar to others in the
group), unfortunately it is

> There are several reasons for this:
>
> (1) Envelopes only exist between the time of submission and final delivery.
>    Transport actions do record certain bits of envelope information in
>    trace header fields and final delivery is supposed to copy some additional
>    envelope information into a couple of header fields, but these are NOT
>    a message envelope and it is mistake to assume they are.
>
> (2) During the time the envelope exists it is highy mutable, often changing
>    form at every hop. This makes header storage of envelope information
>    somewhat problematic.

true

> (3) There are several SMTP extension that add to the envelope in various ways,
>    requiring negotiation of what envelope information can and cannot be
>    passed from one system to another. This tends to interact badly with
>    schemes that store envelope information as a static part of the message.

(when SMTP delivery is just the first step in mail processing, this
isn't such a problem)

> (4) The fact that protocols other than SMTP are used for various email
>    operations doesn't necessarily impact header/envelope separability.
>    Other protocols maintain this separation and at least one of them, X.400,
>    actually has a far greater degree of separation than SMTP does.

(not all protocol maintain this separation and if any mail enters
through those protocols, this information will not be available)

> (5) Because there are effectively no controls on what ends up in headers, it
>    is fairly easy for the separation between "header" headers and "envelope"
>    headers to get lost. Among other things, this can create serious
>    security vulnerabilities.

(only during SMTP delivery )

> Now, this is not to say there aren't various ad-hoc schemes in use where active
> envelope information ends up getting stuffed into the header. Such schemes date
> back to BITNET's use of X-Envelope-To: to work around the 8x8 limit and
> probably long before. But in my experience at least these things invariably
> fail to provide a full and correct mapping for all of the possible information
> that can exist in an SMTP (or X.400) envelope. And as a consequence they
> invariably cause problems because of their inability to truly express envelope
> semantics.
>
> Indeed, if you have to capture envelope information in a static form - the main
> current use-case for this is compliance archiving - in most cases you're better
> off NOT using header-based schemes. We even have a standard format defined for
> this: Batch SMTP. Although the format that's probably used the most is the one
> Exchange generates that they call "envelope journaling", which puts the
> envelope in the first text part of a MIME multipart. (On a side note, if anyone
> knows where there's a precise and complete specification of the syntax used for
> envelope journaling, I've appreciate a pointer.)

i agree that stuffing this information into headers is a bad idea (i
intended to observe not advocate above)

i disagree that this is a protocol problem with a protocol solution -
it's simply a poor choice of data representation by the designers.
more modern approaches to meta-data use namespacing and this prevents
loss of information. (this is often then compounded by confusing a
dead MIME document with a live email.)

>> most developers in
>> these mail processing environments do not need to understand the
>> difference between envelope and message content because - for them -
>> there is no difference.
>
> Yeah, that's what a lot of them think. The problem is they're quite simply
> wrong, and it is isn't a harmless thing to be wrong about. I get plenty of
> support calls from customers who got screwed by this lack of understanding.
>
> And it is NOT a minor detail when someone sets up a compliance archiving system
> that ends up in many cases not being able to determine who actually sent or
> received a given message. (I only wish I was making up this example.)

:-)

(that's why containers are now often used in the enterprise)

>> Sieve works very well as a general MIME document processing language.
>
> Actuallly that's not Sieve's purpose at all and it isn't something Sieve is
> currently good at. In fact we've only recently taken the first fairly tentative
> step down the MIME processing path with the MIME loops extension and possibly
> the convert extension. We'll see how well that turns out, but I have to say I'm
> not optimistic that it will replace existing ad-hoc MIME processing facilities
> like MIMEdefang.

AIUI these limitations only apply to multipart documents. outside
email, multipart documents are not so common and sieve works fine on
those.

>> the envelope tests are - in many ways - peculiar since the rest of the
>> specification really isn't mail specific. there are potentially some
>> very interesting applications in this area so it would be a shame - i
>> think - for the expert group to focus too strongly on SMTP at the
>> expense of other IMHO equally valid Sieve use cases.
>
> I don't object to the use of Sieve in other contexts - in principle. But the
> devil is in the details. A good example of this is the use of Sieve in an IMAP
> server as defined in draft-ietf-lemonade-imap-sieve-05.txt. This doesn't seem
> like too much of a stretch from existing usage, but when I reviewed this
> document a while back I found all sorts of semantic mismatches, some of them
> quite serious.
>
>> > But here's the dilemma: This stuff is complicated and in some cases fairly
>> > subtle. This in turn means that the reiteration of even a subset of the
>> > underlying design principles that implementors need to know takes up a lot of
>> > space and will still fall short of the mark of giving the necessary guidance.
>> > But it may lead to the belief that reading this specification (or for that
>> > matter this one and RFC 5228) is in fact sufficient to understand how to use
>> > Sieve. It quite simply isn't.
>
>> again, i beg to differ
>
>> sieve is very similar structurally to the guerrilla standards used in
>> enterprise mail system for more than 5 years now. for most mail
>> processing applications, only the container builders need to have a
>> good understanding of the protocols. application developers are
>> offered a safe environment and an OOP interface. i see no reason why
>> sieve should be any different.
>
> Understanding of the protocols isn't necessary, but I'm very much afraid  that
> there's no avoiding an understanding of email semantics if you want things to
> work properly. We may wish it were otherwise, but it just isn't.

depends on what you mean by that

if you're talking about processing as part of SMTP processing (or any
other protocol) then i agree

but often when people think they are processing email, the use case
boils down to essentially processing a dead MIME document with
meta-data that has been previously delivered through some protocol or
other. they may do some other protocol stuff - such as forwarding some
result - but that's essentially acting as an email client. this is
usually within the capability of most general developers without a
good understanding of email semantics.

- robert

---

From: Ned Freed <[email-masked]>
Date: Tue, 06 Jan 2009 16:07:36 -0800 (PST)

> some hope that it will be fixed)

First of all, there is no such thing as a perfect specification; they are all
flawed to some extent. It's the nature of the beast.

Second, and perhaps more to the point, we demand rough consensus on
specifications, nothing more. I could probably list a couple of things I'm
unhappy about or don't agree with in any specification I've participated in
developing, and for the major ones I can easily list a dozen issues or more.
Once more this is the nature of the beast. But the fact that I didn't agree
with the consensus on some points - and in some cases on many points - didn't
prevent me from contributing to those efforts.

> by an alternative formal method.

Which is now irrelevant since we're not using XML Schema any more.

> still using parsers from the 1990s or sacrificing those who need

Perhaps I don't understand what you're after here, but since the specification
now defines a namespace this also appears to me to be irrelevant.

> needed for these use cases.

I take this to mean that you want separate namespaces for annotations and sieve
proper. I'm against this because I dislike the complexity it adds, which I see
as unnecessary. And unless there's some evidence of support for your view from
other quarters I'm not going to revisit this.

> possible to create a specification suitable for both legacy and modern

I'm afraid I just don't see it. The pointers you have given so far have been to
specifications I regard as overdesigned to the point of near unreadability.

				Ned

---

From: "Alexey Melnikov" <[email-masked]>
Date: Wed, 7 Jan 2009 09:06:52 +0100

On Wed, Jan 7, 2009 at 1:07 AM, Ned Freed <[email-masked]> wrote:
 [...]

> other quarters I'm not going to revisit this.

Excuse my ignorance, but I fail to see why it would be necessary to use a
different namespace for Sieve comments. Programs that interpret Sieve from
XML representation would just ignore such elements. Programs that generate
XML representation automatically (e.g. from some rule based UI) are unlikely
to generate them anyway.
Am I missing some use cases?

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Wed, 7 Jan 2009 13:43:39 +0100

Alexey Melnikov writes:
> Programs that generate XML representation automatically (e.g. from 
> some rule based UI) are unlikely to generate them anyway.

Uhm. Programs do generate sieve comments, typically containing magic 
syntax specific to the generator.

> Am I missing some use cases?

I think the draft would do well to suggest how magic generator-specific 
syntax ought to be handled, with an example or two. Magic stuff should 
not be syntactically similar to a sieve comment.

Arnt

---

From: "Alexey Melnikov" <[email-masked]>
Date: Wed, 7 Jan 2009 15:05:13 +0100

On Wed, Jan 7, 2009 at 1:43 PM, Arnt Gulbrandsen
<[email-masked]>wrote:

> Alexey Melnikov writes:
>
>> Programs that generate XML representation automatically (e.g. from some
>> rule based UI) are unlikely to generate them anyway.
>>
> Uhm. Programs do generate sieve comments, typically containing magic syntax
> specific to the generator.
>
> syntax ought to be handled, with an example or two. Magic stuff should not
> be syntactically similar to a sieve comment.

+1.

If we want to allow for magic comments, it make sense for them to be in a
separate namespace. But then magic comments are typically implementation
specific. So I unless we want to startadize them, I don't think the document
needs another namespace.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Wed, 7 Jan 2009 15:59:33 +0100

Alexey Melnikov writes:
> If we want to allow for magic comments, it make sense for them to be 
> in a separate namespace. But then magic comments are typically 
> implementation specific. So I unless we want to startadize them, I 
> don't think the document needs another namespace.

Each generator may want to use a namespace of its own for its particular 
foo. I am not an expert in this. If that's considered good XML taste, 
then that should be illustrated in an example or two.

Arnt

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Wed, 07 Jan 2009 16:52:36 +0100

> then that should be illustrated in an example or two.

I think that it's bad to make the XML representation more expressive
than the basic Sieve language.  we can't have two types of comments in
XML Sieve, and just one type in plain Sieve.  if we want to cater for
"magic comments", we should make a mechanism for it in Sieve itself.

one way would be to add a new "action", like
   "meta" UP-TO-DATE TOKEN TEXT
where UP-TO-DATE is a boolean, TOKEN is a unique identifier (e.g.
domainname of the implementer, I wouldn't want an IANA registry), and
TEXT is opaque information.  the specification should say that the
"meta" is attached to the next non-"meta" statement, and if that
statement changes semantic meaning, UP-TO-DATE for unknown TOKENs should
be set to FALSE.  if a statement is removed, all "meta" statements
associated with it should be removed as well.

(I only spent five minutes on this "spec", so take it for what it's
worth ;-)
-- 
regards,
Kjetil T.

---

From: Ned Freed <[email-masked]>
Date: Wed, 07 Jan 2009 11:38:02 -0800 (PST)

> then that should be illustrated in an example or two.

I certainly want to allow that - indeed, I don't know of a way to disallow it.
But I don't want to require this approach.

It also occurs to me that as a practical matter, a lot of Sieve generators
produce what amounts to a list of rules. (Our implementation works this way.)
Perhaps an example of a "rule list" sieve done first with the built in
annotation capabilities and then again using the separate namespace approach
would be helpful. Thoughts?

				Ned

---

From: Cyrus Daboo <[email-masked]>
Date: Wed, 07 Jan 2009 15:11:18 -0500

Hi Ned,

> would be helpful. Thoughts?

Could this be done in an Appendix that lists some current common practice 
with sieve scripts and how that might be re-worked for XML? I don't think 
it needs to be part of the main body of the spec.

-- 
Cyrus Daboo

---

From: Ned Freed <[email-masked]>
Date: Thu, 08 Jan 2009 08:27:34 -0800 (PST)

> adopting a standard prefix - sieve, say - is all that is required

To get around the unique particle attribution restriction that prevents XML
Schema from describing many legitimate schemata? If that's truly the case a lot
of XML developers would love to hear more.

> is this really too much to ask?

I fail to see what defining a prefix has to do with the use or non-use of XML
Schema, but for about the tenth time, I have already added a namespace name
registration to the specification.

AFAIK the specific namespace prefix that gets bound to the namespace name is
not something that can or should be standardized. Rather, it is declared on a
per-document basis using an xmlns attribute. Any agent out there that supports
namespaces should support the use of xmlns to pick whatever prefix a given
document binds to the urn:ietf:params:xml:sieve namespace.

> enough to contribute at all to this discussion?

Well, quite a few people have cared enough to discuss other aspects of this
document in the past, and now that the topic of comment mapping has come
up a number are joining in again.

In other words, it seems people care about the document, just not about most of
the issues you have raised.

> group), unfortunately it is

Robert, developing enterprise and ISP class mail servers is what I do for a
living. I've been doing it for over 20 years now. So I think this is something
I might know just a little bit about.

But really, this entire discussion has gone very far afield. Like it or not,
Sieve is intended to be a language used to process email messages at or around
the time of final delivery. The fact that it can be adapted for other uses may
be interesting to you but simply is not relevant in the context of the work
this group is chartered to do. I am therefore not going to continue to respond
further to this discussion of what applications developers do or do not need to
know about email and the specifics of header and envelpe handling. In fact I
regret having continued the discussion to this point.

				Ned

---

From: "Robert Burrell Donkin" <[email-masked]>
Date: Sat, 10 Jan 2009 17:52:13 +0000

> prevent me from contributing to those efforts.

thanks for the explaination

we're currently adding xml serialisation for jsieve over at apache.
there are number of concrete use cases that this serialisation needs
to satisfy. the current draft is currently unsatisfactory so we'll
probably just need to go ahead and create a competing standard. i
think this would be a shame.

my experience on this group hasn't exactly been pleasent for me
(thankfully most of the name calling has been in private). there don't
seem to be very many other implementators on this list with my
background so i'm consistently in a monority of one here. i'm unlikely
to waste energy contributing to a competing standard.

> now defines a namespace this also appears to me to be irrelevant.

to maintain compatibility between namespace aware and unaware parsers
this is necessary but not sufficient. also required is a standard
prefix.

> other quarters I'm not going to revisit this.

without correct separation of concerns, the specification cannot be
used for generation. for developers working in modern langauges using
modern toolsets, this is a major limitation. it just means that
another standard will be required to support these use cases.

making support for annotations optional would be good enough for me.
ideally, i would like to see any mixture of XML allowed by the
specification. this would allow editors based on different principles
to claim support without having to adopt the recommended system of
annotations.

> specifications I regard as overdesigned to the point of near unreadability.

in the xml community, these specifications are usually held in high
regard. it's just surprisingly hard to explain that it works just how
you'd hope it would.

- robert

---

From: "Robert Burrell Donkin" <[email-masked]>
Date: Sat, 10 Jan 2009 18:00:23 +0000

> XML representation automatically (e.g. from some rule based UI) are unlikely

for generative web services, i agree that this is mainly annoying and
inelegant. however, a library which claims compatibility would need to
deal with them if they are mandatory which would is a PITA.

> Am I missing some use

using eclipse (for example) it's possible to generate an editor from a
schema. mixing the domains means that the annotations will appear as
editable to the user.

- robert

---

From: "Robert Burrell Donkin" <[email-masked]>
Date: Sat, 10 Jan 2009 18:03:03 +0000

> But I don't want to require this approach.

AFACT the current draft does not allow mixing of namespaces

- robert

---

From: "Robert Burrell Donkin" <[email-masked]>
Date: Sat, 10 Jan 2009 18:20:48 +0000

> document binds to the urn:ietf:params:xml:sieve namespace.

this choice of namespace prefix is the root cause of incompatibility
between namespace aware and unaware parsers

for example, 'sieve:abc' is just a name to unaware parsers while aware
parsers understand this as a namespace prefix and a local part. the
namespace prefix is resolved to a URI and that is used as the basis of
comparison. specifing a standard prefix (for example, sieve) ensures
that in both cases equality works as expected.

<snip>

> I might know just a little bit about.

i am not unaware of your background but i fail to see why i should
respect your arguments when you are so clearly uninformed about a
class of mail server which are almost a decade old now

> the time of final delivery.
> The fact that it can be adapted for other uses may
> this group is chartered to do.

is this http://www.ietf.org/html.charters/sieve-charter.html the charter?

- robert

---

From: Ned Freed <[email-masked]>
Date: Sat, 10 Jan 2009 17:18:40 -0800 (PST)

> AFACT the current draft does not allow mixing of namespaces

Say what? All the current draft does is define a way to represent Sieve scripts
in XML. Nowhere does it specify how the resulting representation is to be used.
It could be used as a standalone thing where a given XML object only contains
Sieve content and nothing else. Or it could be used to embed Sieve material
inside of some more general sort of XML content (of course using namespaces to
keep things clear) - which in fact is exactly one of the ways we're using it.

You appear to be confusing this specification with others the IETF has done
that use XML as part of defining a protocol, e.g., XMPP. In such cases where
the goal is specifically for clients and servers written by different people to
be able to interoperate and that means there have to be tight constraints on
what can appear on the wire.

Sieve-in-XML is not a protocol or service. It is at most a tool someone can use
in building a protocol or service that needs to operate on Sieve content.
Exactly how it would be used to, say, have a Javascript-based Sieve editor
fetch and return Sieve content from a central server would require considerable
elaboration beyond what's already there. (This would be true even if XML were
not involved - I would expect in many cases for applications to require certain
Sieve extensions be supported and possibly be unable to deal with some others.)

Here's a concrete example of how this might play out. Suppose we decide to
define an extension to the managesieve protocol to allow clients to fetch and
store sieves using the XML representaation. In addition to the usual rigamarole
of having to specify the extension name, new commands or arguments, and so on,
such an extension would have to specify exactly what sort of XML would be used.
Are namespaces  allowed? Required? Can other content be included in the XML or
is it forbidden? 

I actually toyed with the idea of including a managesieve extension in this
specification for fetching and returning XML. I eventually decided against it
because I was concerned that we might run into the problems that have plagued
similar specifications in the past, most notably the abortive attempt to define
XCal, an XML representation of iCal. To the extent such specificastions look
like they are attempting to replace an original format with an XML-based one,
there has been opposition from people who are not, shall we say, believers in
the XML religion. This is also the reason I avoided coining any sort of cute
name for this like XSieve.

				Ned

---

From: "Robert Burrell Donkin" <[email-masked]>
Date: Sun, 11 Jan 2009 10:13:33 +0000

> what can appear on the wire.

no, not at all

i'll explain at more length what i meant

the draft does not allow xml from other namespaces within sieve
fragments. for example (assuming sieve is the default namespace and
mix is some other namespace), the following is excluded by the current
draft schema:

     <control name="if">
       <mix:editor-specific foo='bar'></mix:editor-specific>
       <test name="header">
         <tag>is</tag>
         <str>Sender</str>
         <str>[email-masked]</str>
       </test>
       <action name="fileinto">
         <str>filter</str>
       </action> <comment>move to "filter" mailbox</comment>
     </control>

does my comment make sense now?

- robert

---

From: Ned Freed <[email-masked]>
Date: Sun, 11 Jan 2009 07:48:27 -0800 (PST)

> does my comment make sense now?

Nope. First of all, it simply isn't true: The <displaydata> element is defined
as an xsd:any without a namespace attribute (which defaults to ##any, meaning
any namespace is allowed) and with a processContents attribute set to skip,
meaning no schema is required for anything inside. So your example can be
written:

      <control name="if">
        <displaydata>
        </displaydata>

and be schema-valid. (Actually, I cheated a little here - the schema in the
current draft has a bug where it is missing displaydata and displayblock in the
first choice in the command type definition. This has to be fixed for the
above to validate. But this has nothing to do with the use or non-use
of other namespaces.)

Now, perhaps your objection is that you have to use displaydata containers in
order to embed material from other namespaces and you don't want to have to do
that. If so, that's easily remedied by adding some

          <xsd:any namespace="##other" processContents="lax"/>

directives. The equivalent in Relax is something like:

   any = element * { mixed { ( any | attribute * { text } )* } }
   extelements = element * - ( local:* | sieve:* ) { any }*
   extattributes = attribute * - ( local:* | sieve:* ) { text }*
   extnodes = ( extattributes extelements)

So I guess the question for the group here is what extensibility model do we
really want here? Do we want to require enclosure of material from other XML
sources in displaydata blocks, or do we want to allow other stuff to appear
without such blocks as long as it's in another namespace?

				Ned

---

From: Aaron Stone <[email-masked]>
Date: Sun, 11 Jan 2009 10:57:43 -0800

> without such blocks as long as it's in another namespace?

I plead ignorance to the finer details of the XML Schema stuff described
above, but off the cuff, not requiring displaydata seems cleaner to me.

Is the reason for the displaydata tag to tell an otherwise ignorant
Sieve-XML-only program that the contents of the displaydata should be
handled as verbatim data?

Aaron

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Sun, 11 Jan 2009 22:13:57 +0100

Ned Freed writes:
> So I guess the question for the group here is what extensibility model 
> do we really want here? Do we want to require enclosure of material 
> from other XML sources in displaydata blocks, or do we want to allow 
> other stuff to appear without such blocks as long as it's in another 
> namespace?

Using displaydata for that purpose sounds like a hack, or do I 
misunderstand? "Displaydata" doesn't sound like a natural name for 
importing data from another schema.

The uses cases I've thought of involve sieve stuff embedded in a greater 
context, not other stuff included in the sieve blah. So I don't care 
very much about how importing things into sieve... but if that is to be 
allowed, allowing it properly is better than hackily.

Arnt

---

From: Ned Freed <[email-masked]>
Date: Sun, 11 Jan 2009 10:21:25 -0800 (PST)

> that in both cases equality works as expected.

This is finally something I can understand as a rationale for wanting
to define a standard namespace prefix.

This doesn't appear to be specific to Sieve-in-XML either - if this makes sense
it should be something we're doing in similar XML specifications. So the first
thing I did was check out a bunch of other RFCs - not an exhaustive search but
I did check a dozen or so - to see if this has been done and if so, how. But
AFAICT this has not been done elsewhere. The closest thing to it are statements
of the general form "this document uses the prefix foo to identify the bar
namespace throughout". This falls far short of a "MUST use foo: as the
namespace prefix" or even a SHOULD. In fact some RFCs go out of their way to
make it clear the use of a particular prefix is NOT being standardized.

For example, RFC 5023 (ATOM) has this to say:

 This specification uses the prefix "app:" for the namespace name. The prefix
 "atom:" is used for "http://www.w3.org/2005/Atom", the namespace name of the
 Atom Syndication Format [RFC4287]. These namespace prefixes are not
 semantically significant.

Note that last sentence and the lack of any compliance language.

In other words, this is apparently not current practice for IETF use of XML.
But should it be? I decided the answer to this general question is a little
above my pay grade. Fortunately, the IETF has this thing called the XML
directorate, where people with XML issues can reach out to folks with real
expertise.

I sent a query to the directorate yesterday, which included your statement
verbatim, and received two responses almost immediately, from Tim Bray and Lisa
Dusseault. (Tim you probably know - he's one of the coauthors of the XML
namespace specification, among other stuff. And Lisa is the area director
responsible for this WG.) I've asked for permission to resend their responses
to the Sieve WG, but in the meantime what I can say is that both were opposed
to doing this on the grounds that prefixes cannot be tied down this way because
of the potential for conflicts.

Given the apparently lack of stated support from others in the group and the
lack of support for this from the directorate, I now regard this issue
as closed.

> class of mail server which are almost a decade old now

And by the same token, why should I respect any of your arguments when you
clearly have minimal knoeledge of overall email architecture, Sieve semanticss,
or IETF processes?

> is this http://www.ietf.org/html.charters/sieve-charter.html the charter?

Yes.

				Ned

---

From: "Robert Burrell Donkin" <[email-masked]>
Date: Sun, 11 Jan 2009 21:57:51 +0000

On Sun, Jan 11, 2009 at 6:21 PM, Ned Freed <[email-masked]> wrote:

<snip>

> clearly have minimal knoeledge of overall email architecture, Sieve semanticss,

ad hominem are widely acknowledged to be the best way to win
discussions on the internet and play a vital role in the creation of
high standard specifications

i think i understand why the IEFT is now held in such low regard by
most modern developers and why i was warned against participating in
this group

- robert

---

From: "Robert Burrell Donkin" <[email-masked]>
Date: Sun, 11 Jan 2009 22:01:34 +0000

<snip>

> Yes

could anyone kindly point me to the sections which indicate that only
mail servers of a particular class of architecture are in scope?

- robert

---

From: "Robert Burrell Donkin" <[email-masked]>
Date: Sun, 11 Jan 2009 22:05:18 +0000

> lack of support for this from the directorate, I now regard this issue

so, just to ensure i understand this issue - you've decided to support
namespaces fully and drop support for namespace unaware parsers?

(FWIW this is fine by me)

- robert

---

From: Philip Guenther <[email-masked]>
Date: Sun, 11 Jan 2009 18:05:55 -0800

On Sat, 10 Jan 2009, Robert Burrell Donkin wrote:
...
> there are number of concrete use cases that this serialisation needs to 
> satisfy.

This entire discussion might have been less painful if the concrete use 
cases the jsieve developers have had been brought up at the beginning.  
The discussion that has progressed has been almost completely abstract; 
the one time someone actually said "for example, such-and-such doesn't 
work" the subthread seemed to resolve itself almost directly.  As is, the 
"does it support namespaces" thread is completely confused in my mind 
because I have been unable to determine what you consider necessary and 
sufficient, Robert.

(Fortunately, I'm just a bystander on this doc, so I'm not sweating my 
failure here.)

Philip Guenther

---

From: Ned Freed <[email-masked]>
Date: Sun, 11 Jan 2009 18:05:01 -0800 (PST)

> mail servers of a particular class of architecture are in scope?

I've already done so in previous messages. It's the very first thing RFC 5228,
the Sieve base specification, says:

   This document describes a language for filtering email messages at
   time of final delivery.

Final delivery is a formal term used in many IETF email standards. It refers to
the point at which the message exists the SMTP transport infrastructure and
enters the message store.

As its charter makes clear, the current Sieve group is chartered to work on a
very specific set of Sieve extensions. There is nothing in there about
revisiting the scope of applicability of Sieve.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Sun, 11 Jan 2009 18:30:28 -0800 (PST)

> (FWIW this is fine by me)

Nope. I've decided not to specify any sort of "standard namespace prefix"
in the document. That's all.

I have no idea what you mean by "support namespaces fully". But given the tone
of your most recent responses I'm no longer willing to spend time in a
give-and-take trying to figure what you mean by this.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Sun, 11 Jan 2009 18:45:16 -0800 (PST)

> above, but off the cuff, not requiring displaydata seems cleaner to me.

Unfortunately, it's not quite that simple. The question is whether or not to
require the use of namespaces in order to embed stuff inside of a Sieve script
expressed in XML.

Both XML Schema and Relax let you say "allow any additional elements you want
here as long as it's in a different namespace". Alternately, you can say "allow
absolutely anything here", but if you do that you have to put  it inside of a
container element of some sort, which is exactly what
<displaydata> is.

What you cannot do is simply allow arbitrary content in the same namespace at
the same level as other elements that are schema validated. (Actually, you can
specify something like this in Relax, but the result is that you're validating
almost nothing.)

> handled as verbatim data?

Yes, more or less. There has to be some indication that this content lies
outside the Sieve schema. That can done either with a namespace label or
with a container element.

So if we want to rely on namespaces, which IMO is a fine thing to do, we can
simply loosen the schema to allow material from other namespaces in and drop
displaydata entirely. (Displayblock, OTOH, still serves a useful purpose - you
need something like it to provide a higher level grouping construct.)

The other issue here is how to map this stuff to regular Sieve. The stylesheet
given in the appendix maps displaydata and displayblock material into
structured comments. This can easily be extended/changed to cover the handling
of material in other namespaces. But do we want to formalize the structured
comment convention?

Finally, there's the issue of how to handle other comments, which we've
discussed in the past but may be worth revisiting now. Currently regular Sieve
comments are mapped to <comment></comment> blocks rather than to proper XML
comments. The reason for this is that not all XML processors provide access to
stuff inside of <!-- foo --> constructs, so comments may get lost when
converting from Sieve-in-XML back to regular Sieve. Do we want to continue
to do things this way?

In any case, here's a concrete proposal for how to move forward:

(1) Add the necessary stuff to allow use of XML in other namespaces
    whereever displaydata is currently allowed.

(2) Drop displaydata, making the use of namespaces required if you want
    to embed other stuff inside of a Sieve-in-XML.

(3) Add a section defining the structured comment convention for
    representing this other stuff in regular Sieve format.

(4) Change displayblock to just block, making it clear it can be used for
    other sorts of groupings.

(5) Leave the current mapping of unstructured Sieve comments to
    <comment></comment> blocks alone.

Comments?

				Ned

---

From: Aaron Stone <[email-masked]>
Date: Sun, 11 Jan 2009 19:47:47 -0800

> give-and-take trying to figure what you mean by this.

Let's try to keep this conversation on the path of concrete examples. I
had no idea what this entire thread was about until Robert gave a
specific XML snippet to illustrate what he wanted to do, and you gave a
different one in response to illustrate the document state.

My current understanding from trying to follow this thread is that this
snippet (from Robert) is not valid according to the document:

     <control name="if">

This is syntactically valid XML, but semantically the element from the
'mix' namespace isn't going to make sense to a naive Sieve-XML parser.
By some magic of XML specification buzzwords, I gather that the above
snippet can be declared valid or invalid by the schema definition.

How are mixed-namespace documents like this handled by XML parsers /
editors / UIs / etc? Is it reasonable to disallow a mixed namespace like
this? Is there a reason not to allow such mixing?

What if someone wants to use the Sieve-XML schema within a larger
document? For example:

    <foomail:config>
      <foomail:service service="smtp">
        <foomail:port>25</foomail:port>
        <foomail:listen>127.0.0.1</foomail:listen>
      </foomail:service>
      <foomail:user id="1">
        <foomail:name>Username</foomail:name>
        <foomail:sieves>
          <foomail:sieve>
            <foomail:name>On vacation</foomail:name>
            <foomail:script>
              <sieve:action name="vacation">
                <sieve:str>I'm away on vacation.</sieve:str>
              </sieve:action>
            </foomail:script>
          </foomail:sieve>
        </foomail:sieves>
      </foomail:user>
    </foomail:config>

Aaron

---

From: "Robert Burrell Donkin" <[email-masked]>
Date: Mon, 12 Jan 2009 07:40:47 +0000

> revisiting the scope of applicability of Sieve.

Thank you for taking the time to explain.

I undestand now that this working group has been intentionally
chartered to exclude mail servers with a spooling architecture. Can
anybody explain why the IETF decided to exclude this class of mail
server from it's specification process?

Robert

---

From: "Robert Burrell Donkin" <[email-masked]>
Date: Mon, 12 Jan 2009 07:59:13 +0000

> give-and-take trying to figure what you mean by this.

This point arose from the discussion on namespace support. Since then
you have moved from a refusal to support namespacing to effectively
dropping your requirement to support compatibility with legacy
namespace unaware parsers. Fine by me.

Robert

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Mon, 12 Jan 2009 11:38:33 +0100

> server from it's specification process?

The IETF limits itself to things used over the public internet. How to 
spool mail on disk is out of scope, and now to operate on the spooled 
mail is, too.

There are some borderline cases. Sieve is one. The text format for IPv4 
and IPv6 addresses are another. I guess ARP is a third, come to think 
of it. I suppose RFCs 2852 and/or 5321 might contain words about spools 
and timeouts. But in general, the IETF is about the public internet.

Maybe the IETF should have a wider scope. I don't want to discuss that 
(not here and now, perhaps even never).

Arnt

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Mon, 12 Jan 2009 12:25:09 +0100

Ned Freed writes:
> The other issue here is how to map this stuff to regular Sieve. The 
> stylesheet given in the appendix maps displaydata and displayblock 
> material into structured comments. This can easily be 
> extended/changed to cover the handling of material in other 
> namespaces. But do we want to formalize the structured comment 
> convention?

My desire for structured comments is vanishingly small. My two cents.

> Finally, there's the issue of how to handle other comments, which 
> we've discussed in the past but may be worth revisiting now. 
> Currently regular Sieve comments are mapped to <comment></comment> 
> blocks rather than to proper XML comments. The reason for this is 
> that not all XML processors provide access to stuff inside of <!-- 
> foo --> constructs, so comments may get lost when converting from 
> Sieve-in-XML back to regular Sieve. Do we want to continue to do 
> things this way?

It's a decent reason at least, but it doesn't make me very happy.

>     <comment></comment> blocks alone.

OK.

I wish I'd kept my mouth shut. So many of these things are just better 
than some even worse alternative.

<comment> exists so that even bad XML processors must preserve comments 
in a round-trip conversion, which seems weak justification. Not so weak 
that I actively disagree, but weak enough to make me unhappy.

Arnt

---

From: "Robert Burrell Donkin" <[email-masked]>
Date: Mon, 12 Jan 2009 13:17:16 +0000

On Mon, Jan 12, 2009 at 10:38 AM, Arnt Gulbrandsen
<[email-masked]> wrote:
>> I undestand now that this working group has been intentionally chartered
>> to exclude mail servers with a spooling architecture. Can anybody explain
>> why the IETF decided to exclude this class of mail server from it's
>> specification process?
>
> The IETF limits itself to things used over the public internet.

agreed

> How to spool
> mail on disk is out of scope, and now to operate on the spooled mail is,
> too.

AIUI spooling does not imply that it's stored to disc, just that the
SMTP transaction has completed and that the mail has been committed to
permanent storage.

however, the limit of the end of the SMTP transaction is an arbitrary
restriction based on a particular mail server architecture. there are
many ways that mail can enter a system over the public internet before
final delivery. the working group documentation is anything but clear
on the point that this working group is limited to the consideration
of filter a single protocol (SMTP).

- robert

---

From: "Robert Burrell Donkin" <[email-masked]>
Date: Mon, 12 Jan 2009 13:50:14 +0000

> different one in response to illustrate the document state.

this isn't what i wanted to do but it was intended to illustrate a
point. i will try to remember to include more examples in future,
though.

> 'mix' namespace isn't going to make sense to a naive Sieve-XML parser.

yes - i think this goes to the heart of the discussion

> editors / UIs / etc?

it depends on the approach

generative approaches generally handle this poorly. hand crafted
approaches handle this fine. these days, using SAX directly is rare
and DOM is out of fashion (too slow) so generative approaches tend to
dominate (at least in the java and .NET).

generative editors struggle with semantically mixed domains: it needs
some clue about which are annotations and which are sieve. this can be
fixed by editing the schema to remove the annotations.

> Is it reasonable to disallow a mixed namespace like

disallowing mixed means that the schema can be used directly to
generate a good domain model which maps easily to the nodes in a sieve
processor.

allowing mixing means that existing editors which use their own
annotations can be used reasonably directly. mixed content makes
binding more fiddly.

most likely, the XML binding would either need to be done by hand,
some kind of DOM used or a SAX filter with conventional XML binding
framework upstream. not a major issue when loading the XML directly
from a stream, quite possibly more difficult if you want to use a
framework of some kind (eg web services).

a secondary question is what library creators (as opposed to
application creators) should do when they encounter mixed markup. i
think it's reasonable just to ignore it and concentrate just on the
elements which have semantic meaning in sieve (but others may
disagree).

>                <sieve:str>I'm away on vacation.</sieve:str>

this isn't usually such a problem since the foomail schema can either
specify any element, or just include sieve.

(this is very similar to SOAP so i'll include some general comments
about web services)

in practice, though, most generative approaches to web services don't
work that well with schema that include arbitrary elements. so, the
main issue would be whether the particular web service framework would
work without editing the schema.

- robert

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Mon, 12 Jan 2009 16:01:54 +0100

Robert Burrell Donkin writes:
>> How to spool mail on disk is out of scope, and now to operate on the 
>> spooled mail is, too.
>
> SMTP transaction has completed and that the mail has been committed 
> to permanent storage.

Careless of me. Please allow me to rephrase my sentence: «How to spool 
mail is out of scope, and now to operate on the spooled mail is, too».

> many ways that mail can enter a system over the public internet 
> before final delivery. the working group documentation is anything 
> but clear on the point that this working group is limited to the 
> consideration of filter a single protocol (SMTP).

It isn't limited to that. Borderline issues are in, only issues which do 
not apply to delivery over the internet are out.

Is there any such issue in Ned's draft?

Arnt

---

From: "Robert Burrell Donkin" <[email-masked]>
Date: Mon, 12 Jan 2009 17:11:15 +0000

> mail is out of scope, and now to operate on the spooled mail is, too».

Ok

> not apply to delivery over the internet are out.

I think that makes sense to me now

So, providing a use case could be equally applied to a synchronously
processed mail, it doesn't matter that the known examples are in
servers using asynchronous processing, right?

> Is there any such issue in Ned's draft?

This issue arose from disputes about whether some use cases were in
scope. I should have been a bit careful not to let it spiral out of
control but some of Ned's at hominem comments got to me a little more
than they deserved.

Thanks

Robert

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Tue, 13 Jan 2009 11:53:17 +0100

Robert Burrell Donkin writes:
>> It isn't limited to that. Borderline issues are in, only issues which 
>> do not apply to delivery over the internet are out.
>

("borderline" -> "border-straddling")

> servers using asynchronous processing, right?

Right. Provided that people think the known examples have plausible 
in-scope parallels.

Arguing ONLY from out-of-scope use cases has bad karma.

Arnt

---

From: "Robert Burrell Donkin" <[email-masked]>
Date: Wed, 14 Jan 2009 16:30:57 +0000

On Thu, Dec 25, 2008 at 12:06 PM, Alexey Melnikov
<[email-masked]> wrote:
> Robert Burrell Donkin wrote:
>>>>
>>>> IMO it is a method (but not the only one) of producing reliable
>>>> relatively bug free software. i think this is a reasonable
>>>> pre-requisite for good interoperability. a good suite should aim to
>>>> reduce the numbers of poor implementations which claim compatibility
>>>> rather than try to ensure that good implementations interoperate
>>>> perfectly.
>>>>
>>>
>>> I'm always cautious about extrapolating from limited data - and we
>>> haven't seen
>>> all that much use of scripts being moved from one implementation to
>>> another.
>>> But to the extent we have, the problems that have shown up have been
>>> interop
>>> issues. Unfortunately there are several Sieve implementations out there
>>> that
>>> have chosen to ignore the extensions we've defined and roll their own to
>>> do
>>> things the base specification does not cover.
>>>
>>
>> is there any (lightweight) way for implementors to let this group know
>> about the extensions they've rolled? (other than showing on this list)
>>
>
> I think subscribing to the mailing list is a low enough bar, but if people
> think that that is too heavyweight, then they may contact me directly.

i did ask a few people a few months ago but they declined. my
experiences in this group have not been positive enough for me to be
to be able to offer them any encouragement to join.

i have some private correspondence which i may be able to get
permission to forward to you.

> But note that such contacts are going to be purely informal in their nature
> and have nothing to do with me be the Sieve WG chair. I just happen to know
> authors of various Sieve extensions ;-) and also happen own sieve.info
> domain.

ok

- robert

---

From: "Robert Burrell Donkin" <[email-masked]>
Date: Wed, 14 Jan 2009 16:46:33 +0000

On Mon, Jan 12, 2009 at 11:25 AM, Arnt Gulbrandsen
<[email-masked]> wrote:
>> stylesheet given in the appendix maps displaydata and displayblock material
>> into structured comments. This can easily be extended/changed to cover the
>> handling of material in other namespaces. But do we want to formalize the
>> structured comment convention?
>
>> discussed in the past but may be worth revisiting now. Currently regular
>> Sieve comments are mapped to <comment></comment> blocks rather than to
>> proper XML comments. The reason for this is that not all XML processors
>> provide access to stuff inside of <!-- foo --> constructs, so comments may
>> get lost when converting from Sieve-in-XML back to regular Sieve. Do we want
>> to continue to do things this way?
>

modelling sieve comments as elements (as opposed to XML comments) is a
very natural and obvious way to map these domains and separates the
concerns of commenting on the XML serialization from comments in the
sieve script.

> I wish I'd kept my mouth shut. So many of these things are just better than
> some even worse alternative.
>
> <comment> exists so that even bad XML processors must preserve comments in a
> round-trip conversion, which seems weak justification. Not so weak that I
> actively disagree, but weak enough to make me unhappy.

XML comments tend to be difficult to preserve when transforming
documents and so on.  XML binders are unlikely to support semantic
content in comments, so it's not just bad XML processors which are
going to have problems.

using an element to model the comment part of the grammar is very
natural and makes the relationship between the XML structure and the
sieve structure clear and obvious.

if a parser is generally expected to map the data block to sieve
comments, replacing both separate block structures with support for
structure comments would not only make this clear and obvious but
would mean that the XML structure would have a simple and obvious
isomorphism to the Sieve structure.

- robert

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Wed, 14 Jan 2009 19:41:29 +0100

Robert Burrell Donkin writes:
> comments, [...]

Well, is it? That's the big question, isn't it?

Arnt

---

From: Ned Freed <[email-masked]>
Date: Wed, 14 Jan 2009 15:05:14 -0800 (PST)

First of all, I've been holding off on responding to this thread because
I wanted to get a new version pushed out. I have now done so. The
new version contains a fair number of changes, including:

(1) Definition of a URN for our namespace and registration of said URN with
    IANA.
(2) Allow material from other namespaces in Sieve XML.
(3) Specify the structured comment convention (this was implicit in the
    stylesheet example but needed to become explicit).
(4) Addition of a compact Relax NG grammar.
(5) Made the handling of comments consistent.
(5) Cleaned up a number of bugs in the schema and stylesheet.

And now back to the discussion.

> Let's try to keep this conversation on the path of concrete examples.

I agree and will try to do so.

> I had no idea what this entire thread was about until Robert gave a

>        </action> <comment>move to "filter" mailbox</comment>

This is now valid according to both schema in the document.

> snippet can be declared valid or invalid by the schema definition.

Yes. As I explained previously, a schema gets to control whether or not
elements in other namespaces are allowed, and where. I have allowed them in
most of so-called "complex" elements, i.e., elements that in turn contain other
elements.

> this?

That reallly depends on the application. Keep in mind that this is a building
block, not a finished protocol or service. Even asssuming every application out
there ends up embedding elements from other namespaces, in a typical setup
you're going to use additional schemata to constrain those elements in various
ways. This is entirely reasonable - XML fully support the idea of having
multiple schemata in play for different namespaces.

> Is there a reason not to allow such mixing?

IMO no.

>                 <sieve:str>I'm away on vacation.</sieve:str>

This is also fine, modulo a couple of details - you're missing the
necessary xmlns attributes to identify the namespaces and the <sieve:sieve>
wrapper around the entire script. Fixing these, you'd end up with:

    <foomail:config xmlns:foomail="http://exammple.com/foomail"
                    xmlns:sieve="urn:ietf:params:xml:ns:sieve">
      <foomail:service service="smtp">
        <foomail:port>25</foomail:port>
        <foomail:listen>127.0.0.1</foomail:listen>
      </foomail:service>
      <foomail:user id="1">
        <foomail:name>Username</foomail:name>
        <foomail:sieves>
          <foomail:sieve>
            <foomail:name>On vacation</foomail:name>
            <sieve:sieve>
	      <sieve:control name="require">
		<sieve:str>vacation</sieve:str>
	      </sieve:control>
              <sieve:action name="vacation">
                <sieve:str>I'm away on vacation.</sieve:str>
              </sieve:action>
            </sieve:sieve>
          </foomail:sieve>
        </foomail:sieves>
      </foomail:user>
    </foomail:config>

Of course this is just one possible variant out of many - you could use
different prefixes, or use no prefix for one of the two namespaces, you
could have an additional set of wrapper elemenets or even switch some
foomail elements to attributes and use fewer, etc. etc.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Wed, 14 Jan 2009 15:23:12 -0800 (PST)

> My desire for structured comments is vanishingly small. My two cents.

I don't think we have a choice. The problem is the XML representation is richer
and since it isn't intended as a storage format there  has to be a way to map
the additional information to something in regular sieve. Unless we want to do
something along the lines of the meta extension Kjetil proposed - which I don't
think we want to do, pretty much the only place for it is in comments.

> It's a decent reason at least, but it doesn't make me very happy.

I apologize for not being clearer here - I made this sound like it's 
a processor quality thing, which it isn't.

The problem is more fundamental: In XML comments are not considered to be part
of the infoset. This means conforming XML processors are explicitly allowed to
drop them. Which in turn means you cannot count on them being available
for processing.

> >     whereever displaydata is currently allowed.

This has been done in the new revision of the draft.

> >     to embed other stuff inside of a Sieve-in-XML.

This has not. I don;t think we have consensus on this yet, and it is much
easier to remove something than to put it back in once it is gone.

> >     representing this other stuff in regular Sieve format.

Done.

> >     other sorts of groupings.

Not done yet, as it kinda depends on the removal of displaydata to make
sense.

> >     <comment></comment> blocks alone.

This was actually inconsistent in places and required some changes.

> than some even worse alternative.

Yeah, that's the nature of the beast, I'm afraid.

> that I actively disagree, but weak enough to make me unhappy.

Mostly agreed, except that such processors are in fact fully compliant and
therefore not considered "bad", at least in any sense that matters.

				Ned

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Thu, 15 Jan 2009 09:54:40 +0100

Ned Freed writes:
> I don't think we have a choice. The problem is the XML representation 
> is richer and since it isn't intended as a storage format there  has 
> to be a way to map the additional information to something in regular 
> sieve. Unless we want to do something along the lines of the meta 
> extension Kjetil proposed - which I don't think we want to do, pretty 
> much the only place for it is in comments.

I didn't reply... but at least I would have preferred such a meta thingy 
at 302 time. Now it may be too late.

> I apologize for not being clearer here - I made this sound like it's a 
> processor quality thing, which it isn't.

You were clear enough for me at least.

> The problem is more fundamental: In XML comments are not considered to 
> be part of the infoset. This means conforming XML processors are 
> explicitly allowed to drop them. Which in turn means you cannot count 
> on them being available for processing.

Sure. The question is whether that's a problem.

If you write a script in xmlsieve, convert it and then convert it back, 
then any comments you added may have been deleted. AFAICT this is not a 
problem.

If you write it in plain old sieve, convert it and then convert it back, 
then comments are preserved due to <comment>. AFAICT this is neat, but 
it's not clear to me that it's necessary. (Btw. Now that I think about 
it. Is whitespace preserved in arguments?)

>> I wish I'd kept my mouth shut. So many of these things are just 
>> better than some even worse alternative.
>

Yes.

Arnt

---

From: Ned Freed <[email-masked]>
Date: Thu, 15 Jan 2009 19:40:26 -0800 (PST)

> at 302 time. Now it may be too late.

If it was in the base specificaiton, maybe. But having to support a brand new
Sieve extension in order to handle preservation of XML content while round
tripping scripts sounds like a bad idea to me.

> then any comments you added may have been deleted. AFAICT this is not a

Well, I don't know about you, but if I have a complex script that I have
carefully commented, and when I use a new GUI editor on it that transforms it
to XML and back again and loses the comments in the process, I'm not going to
be  a happy camper.

Sure, there are lots of cases where it doesn't matter and comments can be
dropped, assuming there are any there in the first place. But there are
going to be cases where comments need to be preserved and any mapping
needs to support that.

Really, I think this is just lack of familiarity with how things play out in
the XML world more than anything. Everywhere you look in XML-based
languages you'll find <comment>, <annotation>, and similar constructs.

For better or worse, XML just doesn't worry too much about this stuff.

> it. Is whitespace preserved in arguments?)

Whitespace in string constants darned well better be preserved. (And come to
think of it, I need to put in something about how to map text: constructs.)
Whitespace in comments can also be preserved for the most part - there may be
some edges where it's a bit difficult. But whitespace between lexical tokens in
either representation isn't going to be preserved. The solution I adopted in
the sample stylesheet was to output stuff using a simple indentation scheme.

				Ned

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 16 Jan 2009 08:54:28 +0100

Ned Freed writes:
> the XML world more than anything.

I am indeed not familiar with the XML world.

> Everywhere you look in XML-based languages you'll find <comment>, 
> <annotation>, and similar constructs.

Fine then. When in Rome, etc.

Other stuff later; I'm squeezed now.

Arnt