Subject: Re: "extensible" grammar

From: "Stan Bailes" <[email-masked]>
Date: Sat, 24 Jan 1998 10:55:09 -0800

Chris Newman <[email-masked]> writes:

>On Wed, 21 Jan 1998, Rob Earhart wrote:
>>   Nope - you *can* do your semantic checking in your parser, even
>> under this scheme.  I'd argue that this makes your parser needlessly
>> complex and hard to maintain, when the work's better split up and done
>> at a higher level, but that's just my opinion.
>
>Functional separation is good.  Layers are bad.  I've never been a fan of
>multi-level grammars -- in my experience they make things less reliable
>and harder to understand.  Every time you introduce a layer you partition
>part of the problem into very complex cross-layer communication issues
>(your if-then-else example is one case).  Even a layer as necessary and
>delinated as TCP has these problems (think IP security, urgent data,
>SIGPIPE).

[snip]

>So I see this as a proposal to constrain the human readability of the
>script for all time in order to save a few programmer hours.  This seems
>like a really bad tradeoff to me.

Your argument seems inconsistent to me.  First you suggest that a
"multi-level" grammar is complex and hard to understand, then you
argue that it will "save a few programmer hours", (one can only assume
that's because it's actually *less complex*).  But you're opposed to that
because it "constrains human readability".  Then, in another note,
you argue that less than 5% of users will write their own scripts, which
leaves me wondering why human readability matters so much....

>If we want to compromise on this issue, I suggest we define the rules for
>skipping a command (look for unquoted ";"), and let extensions which add
>new control structures do whatever they wish.  New control structures will
>be rare.  I also don't object to suggestions for what the syntax of
>extensions should look like -- just don't make them recommendations or
>requirements.

I think this is on the right track, though I would prefer to see a more
formal
rule than "look for unquoted semi".  Tim's recently posted grammar seems
like a good starting point.  (Maybe somebody could post an example of
a "reasonable" extension which can't be accomodated by this grammar?)

This will at least allow implementations to use a conventional
AST (abstract syntax tree) implementation, and to easily support dynamically
loaded extensions without requiring (horror!) parsing callbacks and the
like.

Stan

---

From: Ned Freed <[email-masked]>
Date: Sat, 24 Jan 1998 23:39:18 -0800 (PST)

> that's because it's actually *less complex*).

You're confusing any number of things here. For one thing, a multi-level
grammar can be quite easy to parse, especially when you only do the outer
generic level. This does not, however, mean that the thing is simple or that
people -- even implementors -- really understand what is going on. What seems
to happen is that they think they do but they don't.

We have decades of experience with one multi-level grammar called RFC822 that
argue this point rather overwhelmingly.

Second, the point is that parsing is the *easy* part. All this whoopla about
language syntax is nothing but an attempt to optimize the already-easy part of
the problem, at the expense of grotting up the hard part -- the semantics --
considerably. Shaving 10% off of the 10% part while adding 10% to the 90% part
is a bad idea, and that's what I see happening here.

> But you're opposed to that because it "constrains human readability".

The issue with human readability is that people like to have a fair number of
visual cues as to what is going on. The use of a very regular syntax tends to
compromise the ability to add such cues.

> Then, in another note,

Once again you're confusing different things. Yes, it is true that only a small
percentage of users (I expect the number to be far lower than 5%) will write
their own scripts. But this only means that it is critically important that the
language appeal to the authors of script generators. This is a fairly demanding
audience, and if the language looks awkard (and I think the latest proposal
looks _very_ awkward) they aren't going to like it.

				Ned

---

From: Tim Showalter <[email-masked]>
Date: 26 Jan 1998 12:00:02 -0500

> Date: Sat, 24 Jan 1998 23:39:18 -0800 (PST)
> From: Ned Freed <[email-masked]>

> Once again you're confusing different things. Yes, it is true that only a
> small percentage of users (I expect the number to be far lower than 5%)
> will write their own scripts. But this only means that it is critically
> important that the language appeal to the authors of script
> generators. This is a fairly demanding audience, and if the language
> looks awkard (and I think the latest proposal looks _very_ awkward) they
> aren't going to like it.

Other than the control structure weirdness, what looks awkward?
(Everything?)

Other than support, which is apparently gone, I like having more
flexibility in the argument order, although it may need to be cleaned up
some.  In particular, the tagged argument stuff will probably be easier to
specify in the grammar this way.  (I'm assuming tagged arguments are a good
thing; I'd like to use them for comparators and to replace lots of the
random keywords.)

I'll point out that it is possible to implement "if" without actually doing
it with commands so that it makes actual sense in the parser, even if the
spec specifies that all commands are control structures, and that all
commands are created equal.  I certainly intend to do it this way.

-- 
                                          Tim Showalter [email-masked]

---

From: Ned Freed <[email-masked]>
Date: Mon, 26 Jan 1998 10:09:39 -0800 (PST)

> Other than the control structure weirdness, what looks awkward?

I was talking about the awkward coupling between successive commands if you
elect to go the "pure parser" route. I don't
like this much but I think I can live with it, especially if it means we
have a viable consensus for moving forward.

> thing; I'd like to use them for comparators and to replace lots of the

I've always liked tagged arguments and would like to see a proposal that uses
them. I use them routinely when writing code in lanuages that support them and
I find the resulting code to be much more reliable and readable. And they are
certainly better than the current situation in regard to comparators.

> commands are created equal.  I certainly intend to do it this way.

Exactly. This is why I believe your current proposal is acceptable.

				Ned

---

From: Tim Showalter <[email-masked]>
Date: 26 Jan 1998 15:02:36 -0500

> Date: Mon, 26 Jan 1998 10:09:39 -0800 (PST)

> I've always liked tagged arguments and would like to see a proposal that
> uses them. I use them routinely when writing code in lanuages that
> support them and I find the resulting code to be much more reliable and
> readable. And they are certainly better than the current situation in
> regard to comparators.

I'll try and change the vacation proposal to use them.  Eventally it'll
probably appear in the Internet-Drafts repository, but the one I submitted
was barely changed from the last one I sent to the list.

The comparator stuff in draft 02 is broken in a very major way; it
references the ACAP spec, yet doesn't look anything like it.  The
"match-keyword" stuff, along with comparator stuff, could be done far more
elegantly with tagged arguments, so I'll try and draw up a proposal for
that.