Subject: Re: [sieve] Question about draft-ietf-sieve-external-lists-02

From: Barry Leiba <[email-masked]>
Date: Wed, 13 Oct 2010 11:04:48 -0400

I'm about to submit an -03 version of external-lists, which we would
presumably do WGLC on.  This version will resolve PSA's issues, and
also Kristin Hubner has provided me with a couple more examples
off-list (thanks, Kristin!), which i've included.

Before I post it, I need to know what the result of the discussion
below (about comparators) is, in relation to updates to the document.
What, if anything, needs to be added to the document about
comparators?

For reference, I'll attach the impending -03 version.

Barry

On Wed, Aug 4, 2010 at 7:00 PM,  <[email-masked]> wrote:
>> I personally think that comparators should be allowed with :list, but I
>> don't have a strong opinion.
>
> Given the points that have been raised, I'm starting to think this is the
> right
> answer. In the rare cases where a compprator does make sense to use with an
> external list, it can be encoded into the list name.
>
>> I also think that making Sieve scripts fail because an implementation
>> decides that a particular comparator can't be used with :list would be a
>> bad idea. If we want to go this route, then we need to define how a
>> particular comparator works for a particular external list URI type.
>
> Yeah, but the problem there is what to do about tag: URLs. I don't know what
> other implementations have planned, but we're not exactly keen on letting
> users
> specify arbitrary LDAP URLs or anything similar. Instead, the mail admin
> sets
> up whatever tag: URLs (incuding families of them) they want, and these are
> then
> mapped to LDAP, database queries, code callouts, whatever.
>
> As it happens we have no exiting or planned external list mechanisms where a
> comparator makes sense, so right now we can simply reject the combination
> outright. But suppose this were to change. Given the mapping involved we'd
> basically be counting on the admin to specify the handling of comparators,
> and
> this means the admin is going to be responsible for setting it up so an
> unused
> comparator results in an error. Of course we can arrange the defaults so
> that
> the "right" think happens most of the time, but even so this doesn't sound
> like all that great of an idea...
>
>                                Ned
> _______________________________________________
> sieve mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/sieve
>

---

From: Alexey Melnikov <[email-masked]>
Date: Fri, 15 Oct 2010 09:59:18 +0100

>What, if anything, needs to be added to the document about
Right. I am not entirely clear where we stand on this issue.

>For reference, I'll attach the impending -03 version.
This looks good to me. The only other outstanding issues are:

1). ManageSieve capability for discovering support for different types 
of external lists. I will suggest some text.

2). Add a tag: personal addressbook as mandatory to implement (as per 
discussion in Spring 2010). Are people happy with using "tag:pab"? This 
is going to be a bit of a special case, because it is not going to use 
the recommended prefix, but I think we should use an exception in this case.

---

From: Alexey Melnikov <[email-masked]>
Date: Fri, 15 Oct 2010 10:16:25 +0100

> of external lists. I will suggest some text.

Suggested changes:

   EXTLISTS - A space separated list of URI schema parts [URI] for supported
   externally stored list types.  This capability MUST be specified if 
the Sieve
   implementation supports the "extlists" extension defined in this 
document.

And the updated ABNF to ManageSieve:

    single-capability     =/ DQUOTE "EXTLISTS" DQUOTE SP ext-list-types CRLF

    ext-list-types = string
                 ; space separated list of URI schema parts
                 ; for supported externally stored list types.
                 ; MUST NOT be empty.

Add to the IANA Registration

This document requests IANA to register a new ManageSieve Capability 
according to the IANA registration template specified in [MANAGESIEVE]:

      Capability name: EXTLISTS

      Description: This capability is returned if server supports
      'extlists' [RFCXXXX] Sieve extension.

      Relevant publications: this RFC, Section <TBD>>

      Person & email address to contact for further information: Alexey
      Melnikov <[email-masked]>

      Author/Change controller: IESG.

---

From: Barry Leiba <[email-masked]>
Date: Sun, 17 Oct 2010 09:46:19 -0400

The changes below look good to me.  I'll stick 'em into my working
copy of the document, pending other comments.

Barry

On Fri, Oct 15, 2010 at 05:16, Alexey Melnikov
<[email-masked]> wrote:
> Alexey Melnikov wrote:
>
>> 1). ManageSieve capability for discovering support for different types of
>> external lists. I will suggest some text.
>
> Suggested changes:
>
>  externally stored list types.  This capability MUST be specified if the
> Sieve
>  implementation supports the "extlists" extension defined in this document.
>

---

From: [email-masked]
Date: Sun, 17 Oct 2010 09:43:36 -0700 (PDT)

> of external lists. I will suggest some text.

In regards to discovering what external lists are available, maybe I'm
missing something, but I'm not seeing how this is substantially different
from the problem of knowing what notification methods are availabe to
a script.

If this is an apt comparison, then it would argue for some sort of 
valid_external_list test, wouldn't it?

> discussion in Spring 2010). Are people happy with using "tag:pab"?

I agree that having a standard name for the user's own personal address book is
a good idea. But the problem with tag:pab is it's not a valid tag: URL. The
authority and date fields are required in tag: URLs; you cannot omit them.

If I'm understanding the theory of tag: URLs correctly, the proper way to do
this would be to "mint" this in a namespace the IETF controls. So you'd end up
with something like tag:ietf.org,2010:pab. (Note that with tag: URLs the
"authority" associated with the authority name refers to control over that part
of the namespace. It doesn't imply only that authority can resolve that tag:
URL.)

But ever since they were first suggested my problem with tag: URLs has been
that they are butt-UGLY. Having to remember a specific date in order to
construct the right URL? Please! Nobody wants to do that, even if you cut it
down to the minimum required fields.

So I think it's quite telling that the minute we want to mint a specific tag:
URL ourselves for use in the specification, we immediately wanted to toss those
pesky syntax rules out the window. And if we're going to do that, I can
assure you that others are going to feel the same way.

So, in the specific case of identifiers for well known list constructs like
pab, I think the right thing to do is use simple alphanumeric identifiers and
set up an IANA registry for them. The initial contents of the registry would
be "pab" for personal address book access. The rule would then be that
a registered name SHOULD be used preferentially over tag: URLs if the
semantics fit.

I wish I had a suggestion for replacing tag: URLs for general use, but I don't.
They're a poor fit to this application, but other URL schemes are poorer. What
we actually want is something that looks like cid: or mid: syntactically, but
with tag: URL semantics.

> This

If by "prefix" you mean the taggingEntity stuff, it's required, not
recommended.

My one other issue with external lists is the inability to return properties
associated with the match. I've suggested previously that we reuse the paradigm
of :matches and variables for this - after a match ${0} is set to the thing
that matched, and ${1} and so on get set to properties in a list-specific
fashion. An implementation that doesn't want to support lists with properties
is free to see ${0} and nothing else.

				Ned

---

From: Alexey Melnikov <[email-masked]>
Date: Thu, 21 Oct 2010 09:50:07 +0100

Hi Ned,

> valid_external_list test, wouldn't it?

I thought there was an agreement (from a f-2-f meeting) not to add such 
test and possible handle unsupported URI types with yet-to-be-written 
exception handling mechanism in Sieve, or possibly use/extend ihave. But 
of course this decision can be revisited.

> authority and date fields are required in tag: URLs; you cannot omit 

Good point, I haven't checked the syntax when I wrote my proposal.

> of the namespace. It doesn't imply only that authority can resolve 

Ok.

> a registered name SHOULD be used preferentially over tag: URLs if the

Hmm. We can do that, but we already had this argument before and I 
feeling we are going in circles.
What do other people think?

> If by "prefix" you mean the taggingEntity stuff, it's required, not

I was talking about omitting <domain>,<date> part. You already pointed 
out that that is incorrect, so ignore this comment.

> is free to see ${0} and nothing else.

As long as the behavior you described in the last sentence is allowed, 
that would be Ok with me.

---

From: Derek Diget <[email-masked]>
Date: Thu, 21 Oct 2010 11:59:51 -0400 (EDT)

On Oct 15, 2010 at 09:59 +0100, Alexey Melnikov wrote:
=>Barry Leiba wrote:
=>> I'm about to submit an -03 version of external-lists, which we would
=>> presumably do WGLC on.  This version will resolve PSA's issues, and
=>> also Kristin Hubner has provided me with a couple more examples
=>> off-list (thanks, Kristin!), which i've included.

<snip>

=>> For reference, I'll attach the impending -03 version.
=>>  
=>This looks good to me. The only other outstanding issues are:
=>
=>1). ManageSieve capability for discovering support for different types of
=>external lists. I will suggest some text.
=>
=>2). Add a tag: personal addressbook as mandatory to implement (as per
=>discussion in Spring 2010). Are people happy with using "tag:pab"? This is
=>going to be a bit of a special case, because it is not going to use the
=>recommended prefix, but I think we should use an exception in this case.

Disregarding how "pab" is to be defined, as an operator, the "mandatory" 
is very welcomed.

I was not able to find the "Spring 2010" discussion in the archives to 
see if the following was discussed....

Another filtering feature we get requests for is for comparing address 
in the global/corporate address book.  In our case we have collapsed our 
e-mail down to one domain and therefore matching a domain substring 
(:contains) works 99% of the time, but I could see were sites might have 
multiple domains in use and having a gab to compare to would be helpful.  
I understand that implementations are free to add a "gab" list.  I just 
wonder if pab is a MUST; is having a gab as a MUST or SHOULD asking too 
much?

(Note in our deployment, we don't accept messages that try to use a 
RFC5321.From address that is not defined in our authoritative address 
database (LDAP), but that still allows invalid local RFC5322.From 
addresses to get through.  Having a mechanism to make sure that 
a local RFC5322.From was valid would be a win for us.)

-- 
***********************************************************************
Derek Diget                            Office of Information Technology
Western Michigan University - Kalamazoo  Michigan  USA - www.wmich.edu/
***********************************************************************

---

From: [email-masked]
Date: Thu, 21 Oct 2010 09:56:05 -0700 (PDT)

> of course this decision can be revisited.

I think it needs to be in light of the fact that notify used this approach.

> What do other people think?

Another alternative, sugggested by Chris Newman, would be to define a special
"pab" URL type. THis way you can say pab:<foo>, where <foo> is the group
in the user's address book you want to look at.

> that would be Ok with me.

It has to be allowed; there are plenty of lists that don't support any
kind of property metadata.

				Ned

---

From: Barry Leiba <[email-masked]>
Date: Thu, 21 Oct 2010 15:06:12 -0400

> Hmm. We can do that, but we already had this argument before and I feeling
> we are going in circles.

I was one who was very much against using opaque strings, and pushed
for URLs, for interoperability reasons.  Note that my objection was to
*opaque* or implementation-defined strings.  I have no problem at all
with Ned's suggestion of a registry of *well defined* strings that can
be used in addition to URLs.  In fact I think a list of well defined
strings is the *best* answer to the whole list issue for probably 90%
of the use cases (many of which involve the PAB).

Barry

---

From: [email-masked]
Date: Fri, 22 Oct 2010 20:28:15 -0700 (PDT)

> for URLs, for interoperability reasons.

But if interoperabiity/portability is your goal, I'm not sure tag: URLs
buy you much over opaque strings.

> Note that my objection was to

So the question becomes, should we (a) Set up a registry for these well defined
strings, or (b) Define a pab: URL type. The former is a little simpler and
generalizes to non-pab applications, the latter gets us  support for checking
memoership in pab groups.

I think (b) is the better bet, mostly because I agree that pab is the main
use case.

I've also been thinking about the issue how to test to see if a given
list is available. I know of three proposals: (a) environment, (b) ihave,
or (c) a new test along the lines of valid_notify_method.

Using environment for this is problematic. Here's why:

    require ["environment", "extlists", "variables"];
    if environment :matches "extlists" "*foo*" { ... }

Now suppose you have an MUA implementation that allows lists to be defined
using arbitrary LDAP URLs. How could such an implementation possibly
implement the above test? And what value would ${0} be set to on a successful
test result?

The problem here is fundamental: Environment, by it's nature, depends on
implementations being able to enumerate the value(s) of the environment item.
That's not possible in general for extlists, and we definitely don't want to
require that it be possible.

    require ["environment", "extlists", "variables"];
    if environment :list "extlists" "nice-lists" { ... }

Checking to see if a member of list of lists is listed  on a list of lists... A
bit too Cantorian for me!

Now, I suppose we could say that the only MATCH-TYPE for this environment item
is :is. But I'd really rather not impose such an arbitrary restriction just to
ram this into environment; it's grotty.

Ihave avoids this particular problem but has other issues. In particular, ihave
is designed so it can checked at compile time. (And even if ihave isn't done at
compile time, require is, and require and ihave accept the same arguments by
definition.) So if we defined, say, an extlist-URL convention, it would buy us
into some ihave tests occuring at compile time and others at run time. I really
don't like this inconsistency - this stuff's behavior is already tricky enough.

I therefore believe that a separate test is the best approach.

				Ned