Subject: [lemonade] CATENATE WGLC - selected or all

From: Eric Burger <[email-masked]>
Date: Thu, 10 Feb 2005 14:11:58 -0500

Besides some editorial comments, we did have the following substantial point
of discussion: does CATENATE work only in the selected state or should we
extend it now to work for any mailbox on the server, knowing that in the
future if we say selected state only we will end up extending it to work for
any mailbox on the server in the future?

I'm hearing a SLIGHT consensus towards it working for any mailbox, but I'd
like to re-poll the server implementers on this issue again.

>>>Ken Murchison wrote:
>>>Point of clarification: Are we saying that CATENATE is *only* valid in
the selected state (where the URL MUST NOT refer to a different mailbox) or
is it also allowed in authenticated state? A note in the text says that "the
CATENATE command is valid in the selected state", but doesn't say "only". 

>>Alexey Melnikov wrote:
>>Currently it is only allowed in the selected state. However, my gut
feeling is that CATENATE will be extended in the future, so it will be
allowed in other states. My preference is to leave the wording as is. 
>>
>>Why not just open it up now? Are we worried about implementation problems?
Even in selected state, wouldn't it be desirable to allow parts of messages
from another mailbox to be used? 

>Ken Murchison wrote:
>I think we had this argument before. Restricting CATENATE to the currently
selected mailbox makes server implementations easier.
Of course, if we allow for something evil like http URLs, there is no need
to have any mailbox selected :-).
>
>I personally prefer no restriction, even though I agree that this will need
more work for server implementors.

Alexey

_______________________________________________
lemonade mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/lemonade

---

From: Mark Crispin <[email-masked]>
Date: Thu, 10 Feb 2005 11:50:55 -0800 (PST)

> like to re-poll the server implementers on this issue again.

The only benefit to making CATENATE work only in selected state is if we 
also restrict CATENATE's arguments to something like:
  catenate-att   = string / "RFC822" [".HEADER" / ".TEXT"] /
                   "BODY" section ["<" number "." nz-number ">"] /
                   "BODY.PEEK" section ["<" number "." nz-number ">"]
This was more or less Pete's original proposal.

I believe that it is pointless to accept a URI as a CATENATE argument 
without allowing (at least) other mailboxes on the same server; and in 
that case it is equally pointless to restrict CATENATE to selected state.

Note that APPEND, the command which CATENATE most closely resembles, is 
allowed in both selected and authenticated state.

On another point, we can make CATENATE support MULTIAPPEND by wrapping the
attributes with paren.  In that case, we don't need a CATENATE command at 
all, just the CATENATE extension!

That is, CATENATE would become a new form of APPEND that replaces 
the literal argument with
 	"(" catenate-att *(SP catenate-att) ")"

I like this idea.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

---

From: Ken Murchison <[email-masked]>
Date: Thu, 10 Feb 2005 15:56:24 -0500

> that case it is equally pointless to restrict CATENATE to selected state.

I agree.  Having just written some proto-type CATENATE code for the 
Cyrus server, it wouldn't be that difficult to extend it to work in 
authenticated (non-selected) state.

> attributes with paren.  In that case, we don't need a CATENATE command 
> at all, just the CATENATE extension!
> 
> That is, CATENATE would become a new form of APPEND that replaces the 
> literal argument with

This would work for me.

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

---

From: Mark Crispin <[email-masked]>
Date: Thu, 10 Feb 2005 13:23:56 -0800 (Pacific Standard Time)

>>     "(" catenate-att *(SP catenate-att) ")"

Damn!  I wish that I had had this idea sooner.

It avoids adding Yet Another Stupid Command to IMAP.  It extends APPEND in 
an obvious way.

Unfortunately, due to APPEND's stupid syntax, we have to modify this 
proposal a bit to make the APPEND flag-list argument be mandatory. 
Something like:

append          = "APPEND" SP mailbox 1*append-message
                   ;;; only a single append-message if MULTIAPPEND
                   ;;; capability not present

append-message  = append-simple / append-complex

append-simple   = [SP flag-list] [SP date-time] SP literal

append-complex  = SP flag-list [SP date-time] SP
                   "(" catenate-att [1*(SP catenate-att)] ")"

I still think that this is better than a separate CATENATE command with 
non-overlapping functionality.

-- Mark --

---

From: Pete Resnick <[email-masked]>
Date: Sat, 12 Feb 2005 10:06:24 -0600

On 2/10/05 at 1:23 PM -0800, Mark Crispin wrote:

>>>attributes with paren.  In that case, we don't need a CATENATE 
>>>command at all, just the CATENATE extension!

Ooooooo.....that's cool! So long as everyone's on board with this, I 
can make the change before the draft deadline.

So, do you want the CATENATE extension to have separate syntax for 
what happens in the presence of MULTIAPPEND, or do you want CATENATE 
to always imply MULTIAPPEND?

>Damn!  I wish that I had had this idea sooner.

Me too. :)

>proposal a bit to make the APPEND flag-list argument be mandatory.

I'm not clear on why the flag-list has to be mandatory or why we need 
parens around the message parts. If we just incorporate the CATENATE 
syntax we get:

append-complex  = SP parameters 1*(SP (text-literal / url))

parameters = ("(" parameter *(SP parameter) ")") / "NIL"

parameter = ("FLAGS" SP flag-list) / ("DATE" SP date-time)

text-literal = "TEXT" SP literal

url = "URL" SP astring

(I'm no longer sure why the "parameters" above needs parens around it 
either. You can tell what's coming by seeing the NIL or FLAGS or DATE 
or TEXT or URL. But maybe that's being a little too magical.)

Let's nail this down and I'll start writing.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
QUALCOMM Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

---

From: Ken Murchison <[email-masked]>
Date: Sat, 12 Feb 2005 13:20:29 -0500

Pete Resnick wrote:
> either. You can tell what's coming by seeing the NIL or FLAGS or DATE or 
> TEXT or URL. But maybe that's being a little too magical.)
> 

If we're going to make this part of APPEND, why not just use the 
existing syntax for flag-list and date-time for the CATENATE case, so we 
can reuse as much of the existing parser code as possible?  I don't see 
much point in trying to clean up any distasteful syntax for only one 
variant of a command.

append-message  = [SP flag-list] [SP date-time]
                   append-simple / append-complex

append-simple   = SP literal

append-complex  = "(" 1*(SP (text-literal / url)) ")"

url = "URL" SP astring

(I think I'd prefer putting the parens around the message parts since it 
makes detecting the end of the catenate a little easier.  Otherwise we 
have to read ahead past a SP to see if we have a text-literal/url or a 
flag-list/date-time in order to differentiate between the continuation 
of an append-complex vs. another append-message.)

---

From: Mark Crispin <[email-masked]>
Date: Sat, 12 Feb 2005 11:35:01 -0800

On Sat, 12 Feb 2005, Pete Resnick wrote:
> Ooooooo.....that's cool! So long as everyone's on board with this, I can make 
> the change before the draft deadline.

Sounds good to me.

> So, do you want the CATENATE extension to have separate syntax for what 
> happens in the presence of MULTIAPPEND, or do you want CATENATE to always 
> imply MULTIAPPEND?

I think that CATENATE should not imply MULTIAPPEND, but needs to reference 
what it does in the MULTIAPPEND case.

> I'm not clear on why the flag-list has to be mandatory or why we need parens 
> around the message parts.

The problem with proposal to wrap the FLAGS and DATE arguments with parens 
is that you get a syntax ambiguity with the simple case.  The simple case 
depends upon parens to determine whether there is a flag-list argument. 
It's still possible to distinguish it, but you have to get much deeper in 
the parse before you know whether you've taken a wrong turn.  Note that 
FLAGS and DATE can be keyword flags, so it isn't until *after* that point 
that we can determine the difference.

I agree that this is how it should have been done, but it's more than 10 
years too late to fix it now.

> (I'm no longer sure why the "parameters" above needs parens around it either. 
> You can tell what's coming by seeing the NIL or FLAGS or DATE or TEXT or URL. 
> But maybe that's being a little too magical.)

Yes, magic is the problem.  Anything that involves having to back up a 
parse past multiple tokens is a problem.

-- Mark --

---

From: Mark Crispin <[email-masked]>
Date: Sat, 12 Feb 2005 11:38:17 -0800

I agree with Ken, but I worry about the syntax ambiguity between 
append-complex and flag-list in his proposal.  By making flags (don't you 
want to set \Draft??) mandatory this is avoided.

On Sat, 12 Feb 2005, Ken Murchison wrote:
> If we're going to make this part of APPEND, why not just use the existing 
> syntax for flag-list and date-time for the CATENATE case, so we can reuse as 
> much of the existing parser code as possible?  I don't see much point in 
> trying to clean up any distasteful syntax for only one variant of a command.
>
>
> makes detecting the end of the catenate a little easier.  Otherwise we have 
> to read ahead past a SP to see if we have a text-literal/url or a 
> flag-list/date-time in order to differentiate between the continuation of an 
> append-complex vs. another append-message.)

-- Mark --

---

From: Alexey Melnikov <[email-masked]>
Date: Sat, 12 Feb 2005 23:04:54 +0000

Mark Crispin wrote:

> append-complex and flag-list in his proposal.  By making flags (don't 
> you want to set \Draft??) mandatory this is avoided.

And specifying "()" is not a big deal.

So, yes, I agree with Ken's proposal as modified by Mark.

>> existing syntax for flag-list and date-time for the CATENATE case, so 
>> we can reuse as much of the existing parser code as possible?  I 
>> don't see much point in trying to clean up any distasteful syntax for 
>> only one variant of a command.
>>
>>
>> (I think I'd prefer putting the parens around the message parts since 
>> it makes detecting the end of the catenate a little easier.  
>> Otherwise we have to read ahead past a SP to see if we have a 
>> text-literal/url or a flag-list/date-time in order to differentiate 
>> between the continuation of an append-complex vs. another 
>> append-message.)
>

---

From: Pete Resnick <[email-masked]>
Date: Sun, 13 Feb 2005 13:28:34 -0600

On 2/12/05 at 11:04 PM +0000, Alexey Melnikov wrote:

>>append-complex and flag-list in his proposal.  By making flags 
>>(don't you want to set \Draft??) mandatory this is avoided.
>

I think that gets us back to Mark's original proposal, doesn't it? 
Here's what I've got, neatened up a bit:

append = "APPEND" SP mailbox 1*append-message
             ; only a single append-message if
             ; MULTIAPPEND capability is not present
                    
append-message = append-simple / append-catenate

append-catenate = SP flag-list [SP date-time] SP catenate-parts
                    
catenate-parts = "(" catenate-att *(SP catenate-att) ")"
                    
catenate-att = text-literal / url

url = "URL" SP astring

That work for folks?

pr

---

From: Ken Murchison <[email-masked]>
Date: Sun, 13 Feb 2005 14:52:49 -0500

>                    text-literal = "TEXT" SP literal

Works for me.

---

From: Cyrus Daboo <[email-masked]>
Date: Sun, 13 Feb 2005 15:52:29 -0500

Hi Pete,

--On February 13, 2005 1:28:34 PM -0600 Pete Resnick 
<[email-masked]> wrote:

> I think that gets us back to Mark's original proposal, doesn't it? Here's
> what I've got, neatened up a bit:
>

Note that ANNOTATE (in IMAPext WGLC) now has a dependency on CATENATE. If 
you change CATENATE's syntax now, that will impact ANNOTATE, however I 
think the proposed change above is worth doing, and we will have to live 
with tweaking ANNOTATE for the new CATENATE.

I think the current syntax is somewhat awkward in requiring the flags list 
for catenate. An alternative is the following:

   append          = "APPEND" SP mailbox 1*append-message
                     ; only a single append-message if
                     ; MULTIAPPEND capability is not present
   append-message  = append-opts SP append-data
   append-opts     = [SP flag-list] [SP date-time]
   append-data     = literal / append-catenate
   append-catenate = "CATENATE" SP catenate-parts
   catenate-parts  = "(" catenate-att *(SP catenate-att) ")"
   catenate-att    = text-literal / url
   text-literal    = "TEXT" SP literal
   url             = "URL" SP astring

Note the use of an atom in place of the literal - this helps disambiguate 
the flag-list which can remain optional. The main benefit for this is that 
ANNOTATE only needs to redefine one element to 'insert' itself into APPEND:

   append-opts     = [SP flag-list] [SP date-time] [SP att-annotate]

With your current syntax ANNOTATE would have to redefine both append-simple 
or append-catenate items.

-- 
Cyrus Daboo

---

From: Alexey Melnikov <[email-masked]>
Date: Sun, 13 Feb 2005 00:14:37 +0000

Mark Crispin wrote:

> I think that CATENATE should not imply MULTIAPPEND, but needs to 
> reference what it does in the MULTIAPPEND case.

I agree.

---

From: Pete Resnick <[email-masked]>
Date: Sun, 13 Feb 2005 16:55:14 -0600

On 2/13/05 at 3:52 PM -0500, Cyrus Daboo wrote:

>Note the use of an atom in place of the literal - this helps 
>disambiguate the flag-list which can remain optional.

I do like this a bit better. If everyone is OK with it, I'll use 
something like this form.

>The main benefit for this is that ANNOTATE only needs to redefine 
>one element to 'insert' itself into APPEND:
>

Which you can put into your document as:

    append-opts /= [SP att-annotate]

I do like that "or-equals" operator.

pr
QUALCOMM Incorporated

---

From: Cyrus Daboo <[email-masked]>
Date: Sun, 13 Feb 2005 19:21:15 -0500

Hi Pete,

> I do like that "or-equals" operator.

No - that does not work. That would give:

    append-opts     = [SP flag-list] [SP date-time] / [SP att-annotate]

That is wrong as att-annotate is not an alternative but an addition. This 
is a problem with the 3501 definition - it uses positional arguments there. 
What we need is a new 'incremental concatenation' operator which ABNF 
lacks. Something like:

    append-opts     += [SP att-annotate]

The other thing that ABNF lacks is an unordered list option - that is 
something that would be very useful for iCalendar syntax.

What we might want to do here, if we are anticipating any possible 
additional APPEND extensions, is to clean up the base syntax. Here is one 
solution:

    ;New 3501bis syntax
    append          = "APPEND" SP mailbox append-message
    append-message  = append-opts SP append-data
    append-opts     = [SP flag-list] [SP date-time] *(SP append-ext)
    append-ext      = ; is arbitrary bounded syntax to be defined
                      ; by extensions. This MUST start with an
                      ; an atom.
    append-data     = literal

Then MULTIAPPEND can just do the following:

    append-message  +=  *append-message
                        ; allows multi-message append

Then CATENATE can do:

    append-data     /= append-catenate

Then ANNOTATE can just do:

    append-ext     /= att-annotate

With this, none of the extensions MULTIAPPEND, CATENATE or ANNOTATE need 
reference any of the others as the new syntax allows for each to extend the 
base in a compatible way. Each extension will work in the presence of any 
of the others.

The only awkward bit is the open-ended definition of append-ext...

-- 
Cyrus Daboo