Subject: Re: Sieve extensions

From: "Stan Bailes" <[email-masked]>
Date: Fri, 16 Jan 1998 08:08:20 -0800

>> Hmm.  Yes, this is a real problem.  I don't suppose you'd prefer
>> that unknown extensions return 'true' instead of 'false', right?
>
>No, I'd prefer that they return an error and leave the user's mail
>unprocessed.  They're essentially link errors and can be detected just
>after you finish parsing the script.

Ok, this makes sense to me.  The implementation needs to make sure
that any extensions that it doesn't support are guarded by the
appropriate 'support' tests *before* execution of the script commences.

(In the implementation I'm thinking of, where ACAP is used to store
the script on the server, the parsing happens when the script is stored,
and the error can be reported at that time, so a "bad" script will never
be executed by the filter agent.  I like this.)

>> I would still argue that the behavior of trying to execute an
>> unknown extension should be as I've proposed, though.
>
>I strongly disagree.  This will result in unpredictable scripts.  Every
>**typo** is processed as an unrecognized extension.

Ok, I'm convinced....

>> > [re: the mystery math extension]
>>
>> But that is really pretty ugly, and I don't think we're trying
>> to make Sieve be Turing complete via extensions, so I don't
>> see this "lack" as a real drawback.
>
>I consider ruling it out to be a serious drawback.

Well, OK.  Let me make my concern clear -- It makes no sense
to specify a 'support' test, which implicitly recognizes the possibility
of extensions not supported by an implementation, if you don't
at least specify the syntax that such extensions must follow.

I'm not particularly wedded to the syntax that I originally proposed,
I just picked it as the simplest possible generalization of the syntax
in the current draft.  If you want to propose a more general syntax,
that's fine.  However, I believe that it is unacceptable to leave the
situation as it is, where an extension is free to specify its own syntax.
The base draft should specify the grammar for the base language and
for *all* extensions -- an individual extension should specify its own
syntax *as a proper subset* of the base syntax.

Stan

---

From: Tim Showalter <[email-masked]>
Date: 17 Jan 1998 23:03:28 -0500

> From: "Stan Bailes" <[email-masked]>
> Date: Fri, 16 Jan 1998 08:08:20 -0800
>

The ACAP server isn't required to ensure validation of the data when it
stores it.  The client should validate the data, but there's no way to
enforce that requirement, and the mail server is forced to re-check the
data.

> at least specify the syntax that such extensions must follow.

That's true, but a support test still has value even if there are
exceptional extensions that it can't handle.

-- 
                                          Tim Showalter [email-masked]

---

From: Ned Freed <[email-masked]>
Date: Sun, 18 Jan 1998 11:43:47 -0800 (PST)

> enforce that requirement, and the mail server is forced to re-check the

Generally speaking the mail server has to assume that invalid scripts will
appear from time to time. Suppose the ACAP server doesn't validate properly?
What then?

As such, good error handling has to be part of the server, and we need to
make sure we specify what that is.

> exceptional extensions that it can't handle.

That's NOT true if the support is considered to be an assertion sort of
test. That is, if the support is there, the script is execute, and if not,
the script fails.

I think trying to support cases where scripts test for the presence of certain
features in the language is _way_ too complex and dangerous for what we're
trying to accomplish here. A simple list of the extensions used at the
beginning of the script which is then checked and the script doesn't execute if
they're not present is all we need to have.

I also don't like the compromises in language design having an extensible
language implies. We've adopted the position that the language we specify has
to be reasonably reader-friendly. And extensible languages more or less by
definition imply either syntactic rigidity (e.g., PPL, the simpler LISPs,
Forth, the simpler functional languages) or some sort of fairly sophisticated
macro facility (e.g., the more complex LISPs, the more complex functional
languages, Tex and MetaFont, ASN.1 1988). The rigidity we don't want and
the complexity we cannot tolerate.

				Ned

---

From: Rob Earhart <[email-masked]>
Date: Sun, 18 Jan 1998 16:38:19 -0500 (EST)

> the complexity we cannot tolerate.

  By "rigid", I assume you mean, "regular, such that the syntactic
rules are fairly simple, and there aren't any exceptions"?

  Actually, that's one of the big arguments in favor of having a
regular syntax.  A regular syntax doesn't imply that it's less
reader-friendly; it means that there's an abstract syntax for things
like "command" and "expression" which all extensions have to obey.

  Historically, I can't think of a single language whose syntax had
lots of exceptions which didn't turn out to either be a miserable flop
for programming (the unix shells, most noteable csh, are a pretty good
example of the phenomenon), or which greatly cleaned up its syntax as
it evolved (perl's a good example).

  )Rob

---

From: Tim Showalter <[email-masked]>
Date: 18 Jan 1998 18:04:52 -0500

> Date: Sun, 18 Jan 1998 16:38:19 -0500 (EST)
> From: Rob Earhart <[email-masked]>
> 
> > I also don't like the compromises in language design having an
> > extensible language implies. We've adopted the position that the
> > language we specify has to be reasonably reader-friendly. And
> > extensible languages more or less by definition imply either syntactic
> > rigidity (e.g., PPL, the simpler LISPs, Forth, the simpler functional
> > languages) or some sort of fairly sophisticated macro facility (e.g.,
> > the more complex LISPs, the more complex functional languages, Tex and
> > MetaFont, ASN.1 1988). The rigidity we don't want and the complexity we
> > cannot tolerate.
> 

A regular syntax means you can't add new constructs that don't look like
old constructs, you can only add things like "commands" and "expressions",
never "loops" or "variables" (unless you do something really bizarre).

> it evolved (perl's a good example).

Perl does not have a clean syntax.  It has a few lucky assumptions that let
the programmers and new features at will, and has been fairly extensible as
a result of that.

Having a standard syntax for tests and actions would be very beneficial;
it'd certainly prevent future extensions from doing anything dumb
unnecessarily.  The point of contention isn't that a regular syntax is good
or bad -- it's good -- but rather that extensions can extend the regular
syntax, if it's called for.

There's one difference between any language you care to name and Sieve.
The basic question is if old parsers always be able to parse scripts with
extensions.  Perl 4 can't parse perl5's syntax extensions; it's just
handled by a little line of common syntax at the beginning of the script
telling the kernel which interpreter it wants to pipe stuff through --
"#!/usr/local/bin/perl5".  C can't parse C++, old LISPs can't parse
CommonLISP backquoting.

I agree with Ned; I'd like some special metadata listing the extensions a
script needs to run, and I'd like to do away with support.

I want to add the generalized syntax to the draft; it'll simplify the
grammar anyway.

---

From: Rob Earhart <[email-masked]>
Date: Sun, 18 Jan 1998 20:36:36 -0500 (EST)

> never "loops" or "variables" (unless you do something really bizarre).

  It's actually not hard at all, if you plan it in advance.  (Granted,
it limits what you can do in the future, but it's possible to make the
syntax general enough so that it won't be a problem).

> the programmers and new features at will, and has been fairly extensible as

  The point was that perl 5's syntax is cleaner than perl 4's syntax;
the progression of the language has been to remove special-case syntax
elements.

> syntax, if it's called for.

  Okay - I flatly disagree.  I strongly feel that no extension should
ever mess with the syntax.  (To tie in with another thread, it'll be
*impossible* to write a generic middleman sieve storage mechanism
that'll validate a script and let the user know before it goes to the
MTA (or whatever) that their script has a syntactic-level problem,
something that shouldn't be required of the storage mechanism, but is
nevertheless *extremely* useful if the middleman is up to it).

> "#!/usr/local/bin/perl5".  C can't parse C++, old LISPs can't parse

  I'm not quite sure what you mean, here - what's the difference
between any other language and Sieve?

  When you make a change to the syntactic structure of a language, you
lose backwards compatibility - it's effectively a new language.  If
the language's syntax is sufficiently flexible, though, you don't need
to change the syntax when you add an extension - the extension extends
the semantic meaning of the program, without changing the syntax.  Old
interpreters might not be able to interpret parts of a script, but
they'll be able to parse it, and maybe do something useful with what
they get.

  )Rob

---

From: Tim Showalter <[email-masked]>
Date: 18 Jan 1998 23:53:22 -0500

> Date: Sun, 18 Jan 1998 20:36:36 -0500 (EST)
> Organization: cmu

You are proposing one or more of
* prohibiting future extensions from doing anything to change the grammar
* the addition of macros

Which one?

Are you aware of any languages that didn't have some changes made to their
syntax?  Why should we be any different?

> the progression of the language has been to remove special-case syntax

But perl gets to change its syntax.  Old interpreters don't get run on new
scripts.  If you do, they fail.  Problem solved.

I want the same flexibility.  There will be inevitable changes to the base
grammar.

(The counterscenario is that if we make Sieve have a non-extensible syntax,
we'll have old interpreters also not understanding new scripts, and nothing
changes.)

> between any other language and Sieve?

Other languages have changes to their syntax.  Many have macro facilities
for this purpose, some cleaner than others.  You're asking for this to be
banned forevermore in Sieve in the interest of having old parsers be able
to give subtly different error messages.

---

From: Ned Freed <[email-masked]>
Date: Sun, 18 Jan 1998 18:05:30 -0800 (PST)

> rules are fairly simple, and there aren't any exceptions"?

No, by "rigid" I mean that there's some small set of syntatic constructs in the
language that provide the only possible extension facilities. The one proposal
I've seen on the list certainly falls into this category -- extensions are done
by adding things that look like functions, even when the semantics being
expressed would look a lot better as additional statement forms. 

In fact I see such things as irregular rather than regular. Having a language
where some forms of statements are "built in" and hence are formatted as
readily recognizable statements and others are "extensions" and have to appear
as functions (or expressions or what have you) is pretty irregular in my book.

This probably would not matter if the core lanugage had a complete set of
"statements" in it. But it doesn't, and shouldn't, because we need to KISS.
However, the limits of the core lanuage have considerable effect on how we do
handle extensions. I do not want to get stuck with a language where assignments
and loops have to end up looking like function calls.

> like "command" and "expression" which all extensions have to obey.

A fixed extension mechanism may be syntactically regular, but no more so than a
more general extension mechanism if extensions done with the latter are done
right. Nobody is arguing for poor or difficult to parse syntax, so this is
basically nothing but a strawman on your part.

> it evolved (perl's a good example).

This is not a reading of history I agree with.  The fact of the matter is that
I see languages all over the place that are riddled with screwy syntax rules,
are astonishingly difficult to parse, and yet are extremely popular and
enduring. FORTRAN is certainly a good example of such a language -- take a look
at a full FORTRAN parser sometime (or better still, try and write one). Or try
COBOL. Or how about ASN.1-1988, where you get to invent new BNF for the
language any time you want? (This doesn't really count as an extension
mechanism, however, since for all its syntactic flexibility ASN.1-1988 isn't
truly extensible. It's all the pain and none of the gain.)

And if you want something _really_ gross, take a look at the shell language
behind the ALL-IN-1 Office Automation System. Despite Digital's best efforts to
abandon it or kill it, it still accounts for millions of user seats and people
still routinely blort out tons of code for it -- code that has to fit in the
comment fields of FMS forms (!), code where practically every statement in the
language uses different quoting, structuring, and other syntactic convensions.
Actually, to call this gross is an insult to gross languages everywhere.

I would, however, count many languages with limited statement forms and which
extend solely through the use of rigid syntax mechanisms as failures, mostly
because people want to be able to control the "look" of basic operations and
won't accept having to do obvious stuff in non-obvious ways. Some languages of
this sort have succeeded, but not many, and I suspect that the ones that have
succeeded have done so in spite of, rather than because of, their rigidity.

One language that deserves special mention here as illustrative of the pitfalls
in this area is PostScript. PostScript has a rigid syntax and is carefully
designed so that each extension (and there are dozens if not hundreds of them
around) can be parsed even when a given implementation doesn't support it.
Facilities are also provided so that PostScript programs can test for the
presence of a given extension and "work around" its absence -- exactly the sort
of thing being proposed here. PostScript manuals even go to the trouble of
explaining in detail how to use these facilities to write more portable
PostScript.

And what has been the result? It has been a total disaster. Extensions,
including highly proprietary and nonstandard ones, are used willy-nilly with
absolutely no attempt to test and make sure support for them is present. Your
typical PostScript interpretor has to define at least a dozen obscure
nonstandard extensions if it is to have any chance at all of handling a typical
PostScript program. Heck, one of the most common extensions you run into is one
where 68000 machine code is loaded into the printer and executed by print jobs!
(Needless to say, "correct" implementation of this consists of ignoring it.) 

And perhaps worst of all is the fact that all this support for extensions
didn't satisfy anyone. It is by no means uncommon for PostScript programs to
piddle around with the interpreter loop in an attempt to enhance the basic
syntax of the language. And don't confuse this with languages like LISP with
"nice" facilities like reader macros -- this stuff is *ugly*. But people still
do it.

Speaking as someone who has spent way too much time dealing with support issues
for one such language, I will add that having Sieve turn into another one is a
showstopper for me. For the most part I'm pretty comfortable with what we
have now (modulo filling in a few blanks and maybe fiddling around with the
punctuation some) and I don't want to see it ruined.

				Ned

---

From: Rob Earhart <[email-masked]>
Date: Mon, 19 Jan 1998 08:30:55 -0500 (EST)

> as functions (or expressions or what have you) is pretty irregular in my book.

  Fair enough.  So, if it were designed such that everything looked the
same, it'd be okay?  (I agree; assignments and loops should not look like
function calls).

> enduring.

  They're popular in the sense that they're used; they're not popular in
the sense that they're universally hated by programmers.  I think the
trend of language regularity => better language still applies (especially
since most of your examples are outmoded dinosaurs which I think we all
wish would just disappear).

  Re: PostScript: I think we want, in all cases, to have the interpreter
punt the program if it hits something it doesn't understand.  You can do
this by announcing at the top of your code the extensions you require
(and, actually, I like having this ability anyway), and you can also have
the processor plod along until it hits something it doesn't understand,
giving the script a chance to work around the lack of an extension.

  Nothing we do is going to prevent people from writing proprietary and
non-standard extensions, and nothing we're going to do is going to prevent
script writers from using them without testing anything.

  I think this is orthogonal to the question of whether or not old
interpreters should be able to *parse* code with extensions.  (Maybe not
interpret - but parse.) 

> "nice" facilities like reader macros -- this stuff is *ugly*. But people still

  Actually, this is exactly what I see happening to the current Sieve
proposal; there're a number of things which can't be done cleanly in the
current syntax, so people who want those things *will* extend the syntax.
Especially since we're suggesting it as a good thing for extensions to do.
I'd *much* rather just nail down the syntax once and for all...

  )Rob

---

From: Rob Earhart <[email-masked]>
Date: Mon, 19 Jan 1998 08:46:32 -0500 (EST)

> * prohibiting future extensions from doing anything to change the grammar

  Prohibiting future extensions from doing anything to change the grammar.
I don't think it's a horrible limitation if the fundamental grammar is
reasonably flexible.

> syntax?  Why should we be any different?

  Tcl and Scheme come to mind (although one could argue that Scheme's
learned from all the mistakes made in various LISPs).  Tcl, for all its
grossness, hasn't changed its syntax (AFAIK).  It's changed its semantics
quite a bit, but the syntax has been the same, and extensions don't mess
with that syntax.

  Scheme's model is simply (commmand stuff...), where "stuff" is atoms,
lists, strings, and numbers.  It's fairly simple, fairly clean, not
exactly easy to read or write (although I still don't think it's as hard
as people make it out to be, and I think it's *much* easier to write
parsers and generators which read and write it), but it's general enough
and flexible enough that extensions can be done simply by adding commands.

  As an example: one common thing to both languages is the ability to pass
a block of code to some command; this means that Tcl's and Scheme's loops
aren't actually syntax elements, they're just commands, like everything
else.  Adding this notion to Sieve would make adding structures such as
loops, exceptions, functions, and macros in the future trivial, without
changing the language's syntax. 

> to give subtly different error messages.

  No.  I'm asking for syntax changes to be banned so that old parsers have
a *prayer* of figuring things out.

  Incidentally: what happens when there're two extensions out there which
define new syntax elements which are mutually incompatible?  And you have
a single script interpreter which needs to understand scripts which use
one or the other?  If you allow extensions to define new syntax, I'll bet
that this'll happen... 

  )Rob

---

From: Tomas Fasth <[email-masked]>
Date: Mon, 19 Jan 1998 19:01:13 +0100

Rob Earhart wrote:

> learned from all the mistakes made in various LISPs).

AFAIK, Python is another example of such a language.

--Tomas

---

From: "Fred L. Drake" <[email-masked]>
Date: Mon, 19 Jan 1998 13:27:08 -0500

> AFAIK, Python is another example of such a language.

  Actually, Python's syntax has changed, but only in very minor ways.
The "access" statement was removed, and the "assert" statement was
added.  These actually required changes to the grammar and parser.
(The third argument to "raise" may fall into this category as well; I
think it was not in early versions of the language.)  Optional &
keyword parameters to functions also required changes.
  Of course, these have been very minor changes.  Old code is not
broken so long as the "access" statement wasn't used and there weren't 
too many assumptions made about implementation details of the
runtime.  New code is less likely to be runnable with old versions of
the interpreter, mostly because Python programmers are using the newer 
language features.  A lot of code can be written that is indenpendent
of the interpreter version.  Dependence on the standard library is
more likely to be a problem, where bug fixes can change the expected
behavior in some cases (like when someone uses an undocumented
interface that wasn't intended to be public, or for which a bug was
being worked around).  These really shouldn't be issues for Sieve.
  As long as there is some reasonable grammar which can allow new
statements and expressions to be recognized and skipped over, there's
unlikely to be any real problems with future extensibility.  The catch 
is to write the grammar that way.  ;-)

  -Fred

--
Fred L. Drake, Jr.
[email-masked]
Corporation for National Research Initiatives
1895 Preston White Drive
Reston, VA    20191-5434

---

From: Tim Showalter <[email-masked]>
Date: 20 Jan 1998 00:37:46 -0500

> Date: Mon, 19 Jan 1998 08:46:32 -0500 (EST)
> From: Rob Earhart <[email-masked]>
> 

Tcl has a quoting syntax; everything else is chaos and eval rules.  Tcl is,
uh, an interesting case considering the "syntax" of the language.

Scheme has had changes to it over the years; they're up to the Revised
Revised Revised Revised Report on Scheme.  It also has a macro facility.

> changing the language's syntax. 

Right now, the only control structure in Sieve is if, and changing if to
match a command means commands can take blocks, and that commands have to
end with ; so stupid interpreters can parse them.

That's
if = "if" WSP test WSP block [WSP "else" WSP block] [WSP];"

and that those have to be blocks, not commands.

> a *prayer* of figuring things out.

I'm not sure what this buys you other than subtly different error
messages.  It's unlikey that users who would write their own filters
wouldn't be able to understand "this requires an extension" in the
documentation.

> define new syntax elements which are mutually incompatible?

There's no way to work around this.  We have to establish some procedure
for extensions and hope that the peer review is clever enough to note a
direct conflict.

> And you have a single script interpreter which needs to understand
> scripts which use one or the other?  If you allow extensions to define
> new syntax, I'll bet that this'll happen...

If you allow the syntax to ever be changed, it's equally likely.
(Including definition of new commands.)

Actually, I think that if we don't allow some extension, including
arbitrary additions, someone else will probably it for us.  In fact, if
HTML is any like this, we have no choice in the matter.

---

From: Rob Earhart <[email-masked]>
Date: Tue, 20 Jan 1998 12:42:00 -0500 (EST)

> uh, an interesting case considering the "syntax" of the language.

  I think the idea that control structures can be done as commands is
reasonable, though.

> Revised Revised Revised Report on Scheme.  It also has a macro facility.

  The Scheme semantics have changed, and I expect that the Sieve
semantics will change; this doesn't affect syntax.  The macro facility
is an extension, not part of the base spec.

> and that those have to be blocks, not commands.

  Or you can do it the C way:

  if = "if" WSP test WSP block [WSP "else" WSP block]

  Syntactically, a block terminates a command, and else is a new
command (this can be parsed by a parser which expects a block to
terminate a command, even if the interpreter will lose if it tries to
evaluate it).  Semantically, else only makes sense right after an if.
I can't think of a single control structure from any language that
*doesn't* work in this model (although C's "for" loops are a stretch :-)

> wouldn't be able to understand "this requires an extension" in the

  Ah.  The idea is that if a script contains extensions and extension
tests, it's *possible* to write a script which works with older
parsers, as long as the syntax doesn't change.

  It's highly possible that people will write scripts which require
certain extensions, without using extension tests, forcing sieve
implementors to implement some set of common extensions.  OTOH, this
can happen anyway - it's simply a question of whether you'll have to
go in and muck with your parser or not to deal with them.

  With extensions affecting language syntax, implementors must
incorporate extensions into their parsers; this isn't pretty even with
compile-time extensions, and if you want dynamic plugins (something I
don't think should be ruled out), it's *very* difficult to parse a
dynamic syntax, whereas if extensions are not allowed to affect
syntax, dynamicly plugging them in as extra commands is trivial.

  And lastly, it'd be nice to be able to do *some* checking of a
script without knowing exactly which extensions the final script
interpreter has available; if extensions are allowed to modify syntax,
one can not assume anything about the script without complete
knowledge of every extension in use, whereas without syntax
modification, many common errors ("You forgot a '{'") can be caught
immediately.

  )Rob

---

From: Tim Showalter <[email-masked]>
Date: 20 Jan 1998 17:20:29 -0500

> Date: Tue, 20 Jan 1998 12:42:00 -0500 (EST)

Perhaps.

> is an extension, not part of the base spec.

The R4RS *is* the current base spec.  I should hope that the semantics are
well defined (but in Scheme, they're not).

> *doesn't* work in this model (although C's "for" loops are a stretch :-)

I'm not sure I like the idea of having a block or a semicolon terminate a
command.
 
> > > No.  I'm asking for syntax changes to be banned so that old parsers
> > > have a *prayer* of figuring things out.
> > 

Which gets you nothing when the validator rejects the script.

> certain extensions, without using extension tests, [...]

I'm removing extension tests unless you complain right now.

> modification, many common errors ("You forgot a '{'") can be caught

These two I can understand more.  The question of a general validator is
not one I consider to be especially interesting, but allowing for plugin
extensions might be pretty useful especially for implementations where
source isn't availible.

---

From: Rob Earhart <[email-masked]>
Date: Wed, 21 Jan 1998 00:02:04 -0500 (EST)

> I'm not sure I like the idea of having a block or a semicolon terminate a

  Just curious: why?

> Which gets you nothing when the validator rejects the script.

  ... except that the validator can say, "Hey, I don't understand this
command, but I know the syntax is bad; you might want to fix it."  It's a
lot like the way a C compiler can compile a function call without actually
knowing that the function will exist at link-time.

  With feature tests, it's possible to write a script which will work on
older implementations and take advantage of newer features (vacation, some
sort of persistant storage, a small AI, a way to access a list of
known spammers, etc...).

> I'm removing extension tests unless you complain right now.

  I'm complaining right now; I really like having both feature tests and
feature requirements.  (Although, heh, extension tests could easily be
done as an extension... :-)

  )Rob

---

From: Tim Showalter <[email-masked]>
Date: 21 Jan 1998 01:45:25 -0500

> Date: Wed, 21 Jan 1998 00:02:04 -0500 (EST)

Why not do this on the client, which needs to do it anyway to get the
communication with the user right?

Otherwise, there's no good way to alert the user that "Hey, that vacation
thing you thought you had -- well, you didn't."

---

From: Chris Newman <[email-masked]>
Date: Fri, 23 Jan 1998 16:05:08 -0800 (PST)

> parsers, as long as the syntax doesn't change.

A client pre-processor does this better than mucking up the language.
Besides, I figure at most 5% of the users will write their own scripts and
probably 1% of those will have more than one server.  This is such a rare
case it's not worth worrying about in the server.  I want to see the
"support" command dropped from the language.

> go in and muck with your parser or not to deal with them.

Parsers are cheap and easy to build.  Semantics are expensive -- if the
user doesn't like the semantics then the language doesn't get used.
Constraining the semantics to make the parser easier is backwards.  I'm
not against trying to design commands that are easy to parse -- but it's
much more important that the commands are easy for humans to think about
and understand.

> syntax, dynamicly plugging them in as extra commands is trivial.

Just give the plugin access to the raw script, parsing callbacks and have
it return how many characters it consumed.  Trying to layer the plugins on
top of a generic parser just adds an unnecessary layer.  Unnecessary
layers are *bad* -- they produce layering violations and constrain what
you can do in the future.

> interpreter has available;

This isn't sufficient justification for restricting the future semantics
of the language.  There's a big difference between a protocol (designed
with parsing as a primary goal and human readability as a secondary goal)
and a language (opposite priority).  The power users who write these
scripts will get a client which supports the extensions they want to use.

		- Chris

---

From: Rob Earhart <[email-masked]>
Date: Sun, 25 Jan 1998 14:31:27 -0500 (EST)

> "support" command dropped from the language.

  Pre-processing is a good indication that one's language isn't
powerful enough.  Supporting "support" in an interpreter is *trivial*
- it's just a table lookup.  What's the problem?

  In addition, I'd like to be able to write a client that
automatically generates scripts which use "support" to dynamically
discover interpreter capabilities.  I like the idea of writing a
client which can take advantage of new features but can produce
scripts which work with older servers as well.

> much more important that the commands are easy for humans to think about

  I don't think anyone was trying to argue that we should restrict the
semantics of the language, merely the way in which they're expressed.
And frankly, I don't see what the problem is with the proposed
restriction; the tagged block model produces code which looks just
like C, and works with *every* control structure I can think of.

  I'd personally much rather feed plugins a datastructure produced by
a generic parser.  "Just give the plugin access to the raw script" is
just *begging* for disaster, IMHO - plugins *will* get the parsing
wrong ("Well, in a while() block, you can use newline and CRLF, but in
a try/catch block you can only use CRLF..." No).  Having multiple
plugins trying to deal with each other when each one tries to use its
own little parser to do its own little thing will be a nightmare.
Having plugins do parsing is bad design (although you're welcome to do
it in your implementation if you want); defining Sieve in such a way
as to force server writers to have plugins do their own parsing seems
almost criminal.

  )Rob

---

From: Ned Freed <[email-masked]>
Date: Sun, 25 Jan 1998 13:03:29 -0800 (PST)

> - it's just a table lookup.  What's the problem?

I disagree. The use or non-use of a preprocessor basically says nothing about
the underlying language. There are all sorts of reasons one might want a
preprocessor. Some are good and some aren't. The fact that one of the most
common uses of preprocessing (the C langage) happens to be a case where the
preprocessor is used to fix up underlying language weaknesses is beside the
point.

Nor is the problem isn't that "support" isn't easy for servers to implement. It
is.

The problem lies in how "support" affects the way the language is used. People
will use the presence of "support" to justify all sorts of extensions. These
extensions will then be used willy-nilly, and they will be used *without*
support tests being made. We will then end up with massive interoperability
problems.

You are recreating the PostScript situation here. Nothing more and nothing less.

> scripts which work with older servers as well.

I have no problem with facilities that discover the abilities of a given server
and then write scripts in accordance with those abilities. I do not, however,
think this justifies having a "support" language construct so that scripts can
be generated with conditionals around code that some servers don't understand.

> like C, and works with *every* control structure I can think of.

I don't much care for the present structure because of the inter-block
communication required to implement control structures. However, I think I can
live with it if that's what people want.

> as to force server writers to have plugins do their own parsing seems

All you have done here is erect a strawman and then attack it. Nobody is
arguing for multiple parsers. A single parser will do the job just fine.

The reason this multiple parser argument is a strawman is that many if not most
extensions will simply add functional forms to the language, not control
structures. Frankly, I don't _want_ to expose the ability to add control
structures via plugins in my implementation.

				Ned

---

From: Rob Earhart <[email-masked]>
Date: Sun, 25 Jan 1998 17:02:44 -0500 (EST)

> support tests being made. We will then end up with massive interoperability

  Your argument hinges on the assumption that people will not write all
sorts of extensions unless "support" (done as a test or as a macro or
whatever) is available.  I disagree; I think people will write extensions
anyway.

  Removing support will not stop people from writing extensions and from
writing code that uses those extensions (requiring everyone who wants to
be interoperable to support the common extensions).  The only thing
removing support will do is prevent people who'd like to be able to write
portable code and still take advantage of extensions from being able to do
so. 

> You are recreating the PostScript situation here. Nothing more and nothing less.

  Removing "support" will not help avoid this.

  The only solution I can think of, actually, requires "support": forbid
servers from permitting extensions to be used unless the existance of the
extension has been tested as a condition for entering the block of code in
which the extension occurs. So,

	if (support "vacation") {
	  vacation;
	}

would be legal;

	vacation;

occuring on its own, outside of a block wrapped with a "support" test,
would be forbidden.

  (At that point, it'd probably be much simpler for implementors if
"support" were its own control structure with a required "else" clause,
giving us something like

	ifsupport "vacation" {
	  vacation "I'm on vacation";
	} else {
	  reply "I'm on vacation";
	}
)

  It's kind of a gross idea, but it's the only way that comes to mind to
*make* script writers do feature tests before using features.

> be generated with conditionals around code that some servers don't understand.

  I'd like to be able to generate scripts without any knowledge of which
extensions the server supports; discovering the list of extensions is
another level of complexity, and prevents the generated scripts from
taking advantage of new extensions as they're introduced to old
interpreters.

> structures.

  That still involves parsing (you could have a general lexical analysis
module for the plugin parsers to call, but the extension modules will
still be able to do whatever parsing they want).  If most extensions will
simply add functional forms, why not just hand them a parse tree to
manipulate?

> Frankly, I don't _want_ to expose the ability to add control

  Reasonable.  :-)

  )Rob

---

From: Ned Freed <[email-masked]>
Date: Sun, 25 Jan 1998 17:26:36 -0800 (PST)

> whatever) is available.  I disagree; I think people will write extensions

Not at all. It isn't a question of what people do but how they do it. I don't
care if lots of people write extensions. I do care if people expect them all to 
work universally. And this is the license that a inline support test
effectively gives to implementors. Its presence will result in it not being
used properly. This isn't logical, but it is nevertheless the lesseon learned
from other languages that used this approach.

Note that I have no problem with having a series of support tests at the time
of script submission which have to pass or else the script won't be accepted. I
don't even have a problem with phrasing this as part of the "language", if
that's how people want it to work. This sort of "in your face" rejection will
have the desired effect of weeding out extensions that don't make the grade and
forcing support of those that do.

We have tons of experience that this extension model works well and does not
hinder the development of extensions.

> portable code and still take advantage of extensions from being able to do

I disagree; I think it will have exactly this effect.

>   Removing "support" will not help avoid this.

I disagree; I think it will.

> occuring on its own, outside of a block wrapped with a "support" test,

PostScript tried this approach, more or less, in that you were supposed to
bracket the use of certain optional facilities with tests to make sure the
necessary facilities were there. Implementations were allowed and even
encouraged to reject the use of extensions not so marked. But nobody did the
tests and implementations were forced to handle extensions without the tests.

Your proposal differs slightly in that your tests would be mandatory and the
presence of a test would be a bit easier to spot. I do not think, however, that
this is enough of a difference.

> *make* script writers do feature tests before using features.

I don't think it will have this effect.

You are arguing the logic of the situation here. Unfortunately, my experience
says that your logic, while valid, doesn't match what actually happens. Like it
or not, people's behavior in this area isn't logical.

> taking advantage of new extensions as they're introduced to old

Rob, let me be blunt. I think this is a recipe for disaster. Nothing more and
nothing less. This is a showstopper for me and I will not support or contribute
to the development of a language with these characteristics. If the group's
consensus is to pursue this model I am not interested in chairing it or
participating in it further.

				Ned

---

From: Rob Earhart <[email-masked]>
Date: Sun, 25 Jan 1998 20:59:31 -0500 (EST)

Okay - it seems like we fundamentally disagree on how people writing
extensions and scripts will behave; I don't see any way to resolve it.
I'll defer to your experience; let's remove "support".

  (So, how badly will you hurt me if I write up "support" as an extension? :-)

  )Rob