Subject: Re: Treat as a WGLC: draft-martin-managesieve-10.txt

From: Stephan Bosch <[email-masked]>
Date: Mon, 07 Jul 2008 10:21:36 +0200

Hi Alexey,

Alexey Melnikov wrote:
>> I thought the idea was to simply allow and ignore the '+' for legacy 
>> implementations.
>>
> ManageSieve was largely documenting Cyrus timsieved and timsieved 
> never emitted "+" to clients.
> If you know any server that emits non-synchronizing literals to 
> clients, please let me know. But absent any evidence that this would 
> break deployed servers, I would prefer to keep consistency with IMAP 
> and timsieved.
Ok, if I encounter any in the near future I will notify you.
>> Strictly requiring it from clients seems cumbersome and will, to my 
>> opinion,
>> likely introduce even more incompatibilities. 
> As far as I know ManageSieve never allowed clients to send 
> synchronizing literals (literals without +). timsieved allows for that 
> (because it reuses IMAP parser), but I don't think it ever emits IMAP 
> style "+ go ahead" back to clients. So it is not consistent with IMAP 
> either.
>
> If people think that support for synchronizing literals in the 
> direction from the client to the server is needed, I can add that. But 
> I am not convinced that this is a problem either.
I am not so much interested in the (IMAP) synchronizing semantics of the 
'+' character. With version 08 of the managesieve specification the '+' 
character was indicated as 'only allowed from clients' in the comment 
somewhere in the ABNF. I am worried that some client implementors have 
interpreted this as being optional and decided to drop the '+' 
altogether. Now it is explicitly required from the clients, making such 
clients fail (if they indeed exist, let's hope not).
>> I do like the new extensions you introduced. :) I'll thoroughly read the
>> new document in a few days to update my ManageSieve implementation.
> Sounds good to me :-).
Ok, I did some preliminary implementation of the new commands. Regarding 
the NOOP command I have only one (nitpicking) remark. The managesieve 
specification explicitly lists which commands are valid before 
authentication. However, by introducing extensions this becomes a little 
more sketchy. The RENAMESCRIPT command is clearly not useful before 
authentication. However, implementors with IMAP experience might think 
of allowing NOOP before authentication, because in IMAP the NOOP 
command  may be issued before authentication. I think it cannot hurt to 
make explicit whether these new commands may be issued before 
authentication.

Other than that, I found a typo in section 1.3 in the TRYLATER 
explanation: 'This response code only make sense when returned in a 
NO/BYE response.'. You should add an 's' there somewhere.

Regards,

--
Stephan Bosch
[email-masked]

---

From: "Robert Burrell Donkin" <[email-masked]>
Date: Mon, 7 Jul 2008 13:05:37 +0100

On Mon, Jul 7, 2008 at 9:21 AM, Stephan Bosch <[email-masked]> wrote:

<snip>

> Ok, I did some preliminary implementation of the new commands. Regarding the
> NOOP command I have only one (nitpicking) remark. The managesieve
> authentication. However, implementors with IMAP experience might think of
> allowing NOOP before authentication, because in IMAP the NOOP command  may
> be issued before authentication.

failing to allow NOOP as may be expected violates the principle of
least surprise.

is there any reason for this?

in general, this protocol seems more than a little peculiar. it's
close in syntax to IMAP but has enough differences for IMAP
implementors to be surprised and confused by the differences.

is there any reason for this choice?

- robert

---

From: Alexey Melnikov <[email-masked]>
Date: Mon, 07 Jul 2008 13:18:46 +0100

>is there any reason for this choice?
The protocol is trying to stay backward compatible with CMU 
implementation (and there are many other client and server 
implementations that mimic it).
If I were to design a new protocol from scratch, I would have made it as 
similar to IMAP as possible. But due to existing deployments, I don't 
think this would be a good option.

---

From: "Robert Burrell Donkin" <[email-masked]>
Date: Mon, 7 Jul 2008 20:27:59 +0100

On Mon, Jul 7, 2008 at 1:18 PM, Alexey Melnikov
<[email-masked]> wrote:
>>> the
> The protocol is trying to stay backward compatible with CMU implementation
> (and there are many other client and server implementations that mimic it).
> similar to IMAP as possible. But due to existing deployments, I don't think
> this would be a good option.

i can see that's a tough choice: codification of existing (possibly
dubious but at least well understood) practice verses the creation of
a better protocol

it seems unfortunate that this means that a separate port is required
for sieve management. a compatible extension to IMAP would allow sieve
management using the same URI.

- robert

---

From: Aaron Stone <[email-masked]>
Date: Mon, 7 Jul 2008 13:20:03 -0700

On Jul 7, 2008, at 12:27 PM, Robert Burrell Donkin wrote:

>>>> allowing NOOP before authentication, because in IMAP the NOOP  
>>>> command
>>>> may
>> implementation

Documenting what exists is the limit of what we can do with this  
document. The protocol itself is already implemented by several  
servers and clients and has good interoperability*. We need to freeze  
the current state and call it "version 1" before we start working on  
version 2.

* actually, I need to compare my implementation with the document  
that's been posted, because I recall at least two things that I  
followed drafts on but clients did not like, one was about response  
codes and another was about {number+} vs. {number}, but I don't  
remember exactly when and what those interop problems were. I do know  
that the clients in question changed to accommodate my server and not  
vice versa, because I was following the draft spec and they agreed.

Other server implementations I know about:

managesieve in dbmail (I wrote this one):
https://svn.ic-s.nl/svn/dbmail/trunk/dbmail/src/timsieve.c

managesieve for dovecot:
http://sinas.rename-it.nl/~sirius/

managesieve in citadel:
www.citadel.org

> management using the same URI.

It's entirely possible that this is the future of sieve script  
management. I, for one, would really like to see sieve scripts exposed  
through some namespace within folder annotations. There are problems  
with that, though, and it's not going to be implemented everywhere  
tomorrow, if at all. So we need to finish documenting what we have  
that works for now.

Aaron

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Mon, 07 Jul 2008 16:23:01 -0400

> dubious but at least well understood) practice verses the creation of

The situation here is that we are standardizing a protocol which was 
already widely deployed.  In such cases, it is usually better to avoid 
breaking interoperability with the deployed base, especially if the 
deployment experience largely shows that the existing protocol works. 
Sometimes, there is no way to avoid making a backward-incompatible change 
to correct a problem, but I don't think this is such a case.

> management using the same URI.

That makes the assumption that sieve scripts live only in IMAP servers, 
which I don't think we want to do.

-- Jeff

---

From: Stephan Bosch <[email-masked]>
Date: Mon, 07 Jul 2008 22:25:34 +0200

> http://sinas.rename-it.nl/~sirius/
Off topic: that url is old. :) The new url is:

http://www.rename-it.nl/dovecot

Regards,

--
Stephan Bosch
[email-masked]

---

From: Aaron Stone <[email-masked]>
Date: Mon, 7 Jul 2008 13:31:03 -0700

Hey, that's the same domain as in your email address! ;-)

Is your implementation now consistent with the document in question?

---

From: Stephan Bosch <[email-masked]>
Date: Mon, 07 Jul 2008 22:36:47 +0200

> Is your implementation now consistent with the document in question?
Not quite, as I did not release any changes as of yet. My latest changes 
should bring it pretty close though. However, I still don't have a 
proper quota implementation (which i personally see as a requirement).

Regards,

Stephan.

---

From: "Robert Burrell Donkin" <[email-masked]>
Date: Mon, 7 Jul 2008 21:40:11 +0100

<snip>

> which I don't think we want to do.

not at all :-)

the function contained in this protocol is really very trivial. i
doubt that any implementator using a storage mechanism other than IMAP
would bother creating an implementation rather than just reusing their
preferred protocol at the application level. for example, HTTP is a
well known protocol whose secruity characterics are know well
understood. sieve maintainance using RESTful HTTP would be much
simpler than creating an implementations of this novel protocol.

- robert

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Mon, 07 Jul 2008 16:46:53 -0400

> simpler than creating an implementations of this novel protocol.

Only if you have a web server and a bunch of related infrastructure lying 
around.  If you're implementing sieve support in an MTA, such that admins 
or even user can provide sieve scripts to be run as incoming mail is being 
accepted (rather than waiting until it hits the mail store), then you may 
not have the luxury of requiring people who want to use the new feature to 
run web servers on their inbound MX's.

Now, I don't claim to be such an implementor, but I don't believe that HTTP 
is the answer to all protocol problems any more than HTML is the answer to 
all user interface problems.

-- Jeff

---

From: "Robert Burrell Donkin" <[email-masked]>
Date: Mon, 7 Jul 2008 22:00:42 +0100

On Mon, Jul 7, 2008 at 9:46 PM, Jeffrey Hutzelman <[email-masked]> wrote:
> around.

HTTP serving libraries of reasonable quality are common

> If you're implementing sieve support in an MTA, such that admins or
> even user can provide sieve scripts to be run as incoming mail is being

opening a new port with a new protocol with unknown and untested
security characteristics sounds unlikely to fly to me. system admins
are much more likely to want to use FTP or HTTP for out-of-protocol
communication (but this is drifting OT)

> all user interface problems.

IMO there are very good reasons why HTTP as a protocol has become
dominant (but that's drifting OT)

- robert

---

From: Aaron Stone <[email-masked]>
Date: Mon, 7 Jul 2008 14:06:13 -0700

> simpler than creating an implementations of this novel protocol.

There are at least a dozen client and server implementations of this  
protocol that exist and interoperate and have been doing so for  
several years, so on that issue your point is negated by fact.

Further, HTTP is dramatically more complicated than this protocol, and  
I challenge the notion that its security is actually well understood  
enough to be applied to other applications without some seriously  
thorough and careful thinking. The only thing HTTP really buys you is  
the ability to re-use existing HTTP server frameworks. This is  
generally not a benefit to authors of email software, since most of us  
have already written our own server frameworks for IMAP and SMTP and  
the like.

Aaron

---

From: "Robert Burrell Donkin" <[email-masked]>
Date: Mon, 7 Jul 2008 22:18:35 +0100

On Mon, Jul 7, 2008 at 10:06 PM, Aaron Stone <[email-masked]> wrote:
> protocol that exist and interoperate and have been doing so for several
> years, so on that issue your point is negated by fact.

how many implementations are not IMAP servers?

> Further, HTTP is dramatically more complicated than this protocol, and I
> challenge the notion that its security is actually well understood enough to
> be applied to other applications without some seriously thorough and careful
> thinking. The only thing HTTP really buys you is the ability to re-use
> existing HTTP server frameworks. This is generally not a benefit to authors
> of email software, since most of us have already written our own server
> frameworks for IMAP and SMTP and the like.

yes but the point was predicated on the apparent requirement that this
protocol should be usable beyond the community of email server
creators. yes, anyone who's spent time writing email servers will be
comfortable but that community is small. other users are likely to
find it much easier to reuse an existing protocol.

- robert

---

From: Aaron Stone <[email-masked]>
Date: Mon, 7 Jul 2008 14:54:03 -0700

Wait, top-post here says, "is there a point to the discussion going on  
here?"

Are you suggesting that we not bring managesieve to publication as an  
RFC?
Are you suggested what we can do for a next-generation sieve  
management system?
Some of both?

Aaron

On Jul 7, 2008, at 2:18 PM, Robert Burrell Donkin wrote:

>> several

---

From: "Mark E. Mallett" <[email-masked]>
Date: Mon, 7 Jul 2008 18:28:59 -0400

I haven't done any managesieve work (other than read some earlier
documents) but gave this a quick once-over for anything that might jump
out anyway.  Nothing really did, other than some stuff calling for picky
comments, so I'll make some :)

Also, I'm a bit curious how there can be a last call (-like) when
there are editorial questions embedded (that is, the things in '[[...]]').
But this isn't a real last call, so I suppose that's the answer.

I found some of the punctuation a little odd but I won't really go
into that (much).

With that..

==========
1.3.  Response Codes:

   If this response code is returned in the OK response, it can mean
   that the user is near its quota or that the user exceeded its quota,
   but the server supports soft quotas.

Two things:  Here is one place where I think the punctuation affects
the meaning, in that "but the server supports soft quotas" seems to go
with both "near quota" and "exceeded quota" whereas it probably really
goes with the latter.  And, "soft quota" can be taken to mean something
specific, whereas I think really what is intended here is that the quota
has been exceeded but this is permitted.  (and are we calling users
"it"?)  Simply removing the comma might help, as would an alternative
such as:

   If this response code is returned in the OK response, it can mean
   that the storage is near its quota, or it can mean that the account
   exceeded its quota but that that condition is being allowed by the
   server.

==========
2.1 ...

   "Note that a failed NO response to the AUTHENTICATE command may
   contain one of the following response codes:" ...

a "failed NO response" strikes me as being a double negative.  Probably
just "a NO response" or one of the words in parens -- "a NO (failed)
response" or "a failed (NO) response".

==========
later in 2.1 ...

   "Another example demostrating use of SASL PLAIN mechanism under TLS."

spelling of "demonstrating"

==========
later in 2.1 ...
    "The following example demonstrate use of SASL "initial response"."

"demonstrates"

Yours,
mm

---

From: Aaron Stone <[email-masked]>
Date: Mon, 7 Jul 2008 15:45:54 -0700

On Jul 7, 2008, at 3:29 PM, Robert Burrell Donkin wrote:

>
> On Mon, Jul 7, 2008 at 10:54 PM, Aaron Stone <[email-masked]>  
> wrote:
> i'm confused: which top post? this top post you're posting now? (or
> another that gmail has helpfully hidden)

My top post, i.e. this sub-thread.

>
>> Are you suggesting that we not bring managesieve to publication as  
>> an RFC?
>
> if the aim is codification of existing practice then since i am not an
> existing practioner of this particular protocol, how can i object?
>
> it's been made quite clear to me that this is the priority of this RFC
> is not quality but compatibility

Ok, cool.

>> management
>> system?
> Alexey Melnikov's comments about a next generation protocol
> compatiable with IMAP make a lot more sense to me. it seems to me that
> an independent protocol would be much more widely useful if it did not
> adopt the IMAP style.

Yeah, I think so, too. Unless we exposed the protocol through IMAP  
somehow, which I'm still a fan of doing, if possible.

Thanks for clearing up the angle you're approaching this from.

Aaron

---

From: "Robert Burrell Donkin" <[email-masked]>
Date: Mon, 7 Jul 2008 23:29:01 +0100

> Are you suggesting that we not bring managesieve to publication as an RFC?

> Are you suggested what we can do for a next-generation sieve management
> system?

- robert

---

From: Alexey Melnikov <[email-masked]>
Date: Tue, 08 Jul 2008 11:30:56 +0100

> Hi Alexey,

Hi Stephan,

>> synchronizing literals (literals without +). timsieved allows for 
>> that (because it reuses IMAP parser), but I don't think it ever emits 
>> IMAP style "+ go ahead" back to clients. So it is not consistent with 
>> IMAP either.
>>
>> direction from the client to the server is needed, I can add that. 
>> But I am not convinced that this is a problem either.
>
> I am not so much interested in the (IMAP) synchronizing semantics of 
> the '+' character. With version 08 of the managesieve specification 
> the '+' character was indicated as 'only allowed from clients' in the 
> comment somewhere in the ABNF. I am worried that some client 
> implementors have interpreted this as being optional and decided to 
> drop the '+' altogether.

Right. Description in -08 was an error on my part. As far as I am aware 
the ManageSieve protocol never allowed + to be optional.

> Now it is explicitly required from the clients, making such clients 
> fail (if they indeed exist, let's hope not).

If such broken clients exist, I hope that they would be found and fixed.

> Ok, I did some preliminary implementation of the new commands. 
> Regarding the NOOP command I have only one (nitpicking) remark. The 
> managesieve specification explicitly lists which commands are valid 
> before authentication. However, by introducing extensions this becomes 
> a little more sketchy. The RENAMESCRIPT command is clearly not useful 
> before authentication. However, implementors with IMAP experience 
> might think of allowing NOOP before authentication, because in IMAP 
> the NOOP command  may be issued before authentication. I think it 
> cannot hurt to make explicit whether these new commands may be issued 
> before authentication.

Good point. I will clarify that.

> NO/BYE response.'. You should add an 's' there somewhere.

Fixed, thanks.

---

From: Alexey Melnikov <[email-masked]>
Date: Tue, 08 Jul 2008 13:12:08 +0100

Jeffrey Hutzelman wrote:

> Only if you have a web server and a bunch of related infrastructure 
> lying around.  If you're implementing sieve support in an MTA, such 
> that admins or even user can provide sieve scripts to be run as 
> incoming mail is being accepted (rather than waiting until it hits the 
> mail store), then you may not have the luxury of requiring people who 
> want to use the new feature to run web servers on their inbound MX's.

Sun has implemented Sieve in SMTP server and are in the process of 
implementing ManageSieve protocol.

---

From: "Robert Burrell Donkin" <[email-masked]>
Date: Wed, 9 Jul 2008 07:50:09 +0100

> implementing ManageSieve protocol.

Still a classic mail server, though. Nothing more exotic.
Robert

---

From: Alexey Melnikov <[email-masked]>
Date: Thu, 10 Jul 2008 22:54:21 +0100

>But this isn't a real last call, so I suppose that's the answer.
This was primarily to motivate reviewers to express their opinion :-).

>I found some of the punctuation a little odd but I won't really go
All fixed, thanks!

---

From: "Robert Burrell Donkin" <[email-masked]>
Date: Thu, 17 Jul 2008 21:21:00 +0100

> My top post, i.e. this sub-thread.

thanks

>> is not quality but compatibility

in future, i'll save my criticisms for other forums and i'll try to
refrain from replying to flames

i did turn up some specific points when i looked through the draft.
FWIW here are some of them

> 1.2 Syntax
...
>   The BYE response may be used if the server wishes to close the
>   connection.  A server may wish to do this because the client was idle
>   for too long or there were too many failed authentication attempts.
>   This response can be issued at any time and should be immediately
>   followed by a server hang-up of the connection.  If a server has a
>   inactivity timeout resulting in client autologout it MUST be no less
>   than 30 minutes.

1. too verbose: the justification in secnd paragraph is not needed;

2. unclear: using 'MAY' makes the intent of the protocol unclear.
changing this to SHOULD would allow the client to infer abnormal
shutdown from it's absence.

3. the autologout timeout requirement is evil. this requirement gains
little from a client perspective (client needs to be able to cope
robustly with connection failures) at a high cost from the server
perspective (holding connections on the server limits scalability and
complicates implementation).

client agent identity and capabilities
----------------------------------------------------------

the draft lacks a mechanism which allows a client libraries to
identify themselves or their capabilities. if it were possible for
servers to identify clients then workarounds could be applied just for
particular clients with known bugs. as this draft stands, servers are
forced to apply workarounds for all clients. this (in turn) means the
de juru standard tends to drift from the de facto one.

BAD input
----------------

the draft does not indicate the right way to deal with input that
cannot be parsed. perhaps server SHOULD deal with malformed input by
issuing a BYE followed by an appropriate response code.

"newlines"
----------------

the specification mentions "newlines" in a couple of places. it would
be clearer to specify what's meant by this eg CRLF (if that's what's
meant).

> Yeah, I think so, too. Unless we exposed the protocol through IMAP somehow,
> which I'm still a fan of doing, if possible.

+1

i'm interested in this

> Thanks for clearing up the angle you're approaching this from.

i plan to use RESTful HTTP. i believe that approach has many
advantages over this draft and may encourage easier interoperability
amongst the general development community.

- robert