Subject: (#3) 4.1.1 Grammar for Search Mechanism

From: Doug Royer <[email-masked]>
Date: Sun, 27 Jan 2002 14:15:16 -0700

Attempt #3, PLEASE REVIEW

 o  I added the '(' ')' expressions.

 o  I changed the NOTES - PLEASE READ.

 o  One of the notes that is NEW is how to compare DURATION and
    DTEND values and the fact that they are ONE SECOND off from
    each other.  I thin that I have entered what the WG has had
    on this list on and off for over a year on this - PLEASE REVIEW.

 o  Fixed typo's sent to me.

--------------------------------------------------------------------------

4.1.1 Grammar for Search Mechanism

  (1) All components look like tables for the purpose of
      a VQUERY including VALARM. And their contained properties
      looked like columns in those tables.

  (2) All VAGENDAs and CS's look like tables for the purpose of a
      VQUERY. And all of their properties look like columns in
      those tables.

  (3) You CAN NOT do any cross component-type joins. And that means
      you can ONLY have one component, OR one VAGENDA OR one CALSTORE
      in the the FROM clause.

  (4) Everything in the SELECT and WHERE clauses MUST be from the
      component type, or VAGENDA OR CALSTORE in the FROM clause.

  (5) The '.' means <table>.<column> when needed for contained
      properties or their associated contained components.
      As long as all were contained (as defined by iCalendar or CAP)
      in the SINGLE component type named in the FROM clause OR
      its contained component.
      
  (6) A contained component without a '.' it is the same as
      <component>.* with the result being a properly formatted
      <component>(s) in the data stream, and correctly formatted
      in the contained component(s) in iCalendar (RFC2445) format.

        VALID:

              (a) SELECT VEVENT.<a-property-name> FROM VEVENT

              (b) SELECT VEVENT.VALARM FROM VEVENT

              (c) SELECT VALARM FROM VEVENT

              (d) SELECT VEVENT.* FROM VEVENT

              (e) SELECT * FROM VEVENT

              (f) SELECT * FROM VEVENT WHERE
                   VALARM.TRIGGER < 20020201T000000Z
                   AND VALARM.TRIGGER > 20020101T000000Z

              Note: (a) Selects all instances of <a-property-name>
                    from all VEVENTs.

                    That (b), and (c) yield the same results.
                    And that is select all VALARMS from all VEVENTS.

                    Note that (d) does not enclose them in BEGIN/END
VALARM
                    because you selected the properties from the VALARM
                    and not the VALARM.

                    (e) Selects every property and every component
                        that is in any VEVENTs.

                    (f) Selects all properties and all contained
                    components in all VEVENTS that have a VALARM
                    with a TRIGGER property value between the provided
                    dates and times.
                  
        NOT VALID:

              (g) SELECT VEVENET.VALARM.TRIGGER FROM VEVENT

              (h) SELECT DTSTART,UID FROM VEVENT WHERE
                   VTODO.SUMMERY = "Fix typo in CAP"

               Note: (g) Is NOT valid because it contains
                      two '.' characters in the SELECT clause.

                     (h) Is NOT valid because it mixes VEVENT
                     and VTODO properties in the same VQUERY.

4.1.1.1 "VQUERY ABNF"

    NOTE: Properties in iCalendar objects can be in any order. This ABNF
          below specifies the valid contents of a BEGIN/END VQUERY. The
          ordering of the properties is not important. Any two VQUERY
          components that contain exactly the same properties in any
random
          order are equivalent.

    search     = "BEGIN:VQUERY" CRLF
                 [expand] querycomp
                 "END:VQUERY" CRLF

               # If not provided, EXPAND default to FALSE
    expand     = "EXPAND" *(xparam) ":" ( "TRUE" / "FALSE") CRLF

    comp-name  = "VEVENT"    / "VTODO"   / "VJOURNAL"
               / "VTIMEZONE" / "VALARM"  / "VFREEBUSY"
               / "VAGENDA"   / "VCAR"    / "CALSTORE"
               / "VQUERY"    / iana-name / x-name

    querycomp  = queries / ( queryname queries) / queryname

    queryname  = "QUERYNAME" *(xparam) ":" text CRLF

    queries    = query
               / queries "," query

    query      = "QUERY" *(xparam) ":" capselect CRLF

                 # NOTE: There is exactly one space separating
                 # the various parts of capselect
                 #
    capselect  = "SELECT" SP cap-cols SP
                 "FROM"   SP comp-name SP
		 "WHERE"  SP cap-expr

               / "SELECT" SP cap-cols SP
                 "FROM"   SP comp-name

               / "SELECT  SP cap-cols SP
                 "FROM"   SP comp-name  SP
                 cap-uselist
                 "WHERE"  SP cap-expr

    cap-uselist = cap-using
                / cap-uselist "," cap-using
                 
    cap-using   = "USING_PROPERTIES" SP cap-col SP cap-local

    cap-col     = # Any property name found in the component
                  # named in the comp-tbl used in the FROM clause.
                  #
                  #   SELECT ORGANIZER FROM VEVENT ...
		  #
                  # OR
                  #
                  # A component name of an existing component contained
                  # inside of the cmp-tbl used in the FROM clause.
                  #
                  #   SELECT VALARM FROM VEVENT ...

                  # NOTE: there is NO space around the "," on
                  # the next line
    cap-cols    = ( cap-col / cap-col "," cap-cols )
                  / "*"

    cap-param   = # Any parameter that may be contained in the cap-col
                  # in the supplied PARM() function
    cap-local   = # Any string that is composed of the characters 
                  # that could be a cap-col name, but is not any
                  # cap-col name. It is suggested that the
                  # string start with "x-" to ensure it does not
                  # conflict with any existing or future cap-col name.
                  # This name MUST BE defined in the cap-using and
                  # can only be used in cap-expr of the same query.
                  # And this name is only known and valid for the
                  # provided query and only for the lifetime of
                  # the query. If multiple QUERY properties exist
                  # in the same VQUERY, it is only valid and usable
                  # in the same QUERY property where it was supplied.

    col-literal = "'" literal-data "'"

   literal-data = # Any data that matches the value type of the
                  # column that is being compared. That is you can
                  # not compare PRIORITY to "some string" because
                  # PRIORITY has a value type of integer. If it is
                  # not preceded by the LIKE element, any '%' and '_'
                  # characters in the literal data are not treated as
                  # wildcard characters and do not have to be backslash
                  # escaped.
                  #
                  # OR
                  # 
                  # If the literal-data is preceded by the LIKE
                  # element it may also contain the '%' and '_'
                  # wildcard characters. And if the literal data
                  # that is comparing contains any '%' or '_'
characters,
                  # they MUST BE backslash escaped as described in
                  # the notes below in order for them not to be
                  # treated as wildcard characters.

    cap-ucol	= cap-col / cap-local

    cap-expr    = "(" cap-expr ")"
                / cap-term

    cap-term    = cap-expr SP cap-logical SP cap-expr
                / cap-factor

    cap-factor  = cap-lhs SP cap-oper SP col-literal
                / cap-lhs SP "NOT LIKE" SP col-literal
                / cap-lhs SP "LIKE" SP col-literal
                / cap-lhs SP "IS NULL"
                / cap-lhs SP "IS NOT NULL"
                / "NOT CONTAINS(" cap-lhs "," col-literal ")"
		/ "CONTAINS(" cap-lhs "," col-literal ")"

    cap-lhs     = cap-ucol
                / "PARAM(" cap-ucol "," cap-param ")"

    cap-oper    = "=" 
                / "!="
                / "<"
                / ">"
                / "<="
                / ">="

    cap-logical = "AND" / "OR"

    SP          = # A single white space ascii character (value in HEX
0x20).

4.1.2 CAP-QL notes

      (1) There is no ORDERBY.  Sorting will take place in the order the
      columns are supplied in the command.

      Float and integer values MUST BE sorted by their numeric value.

      This means the result of a sort on an integer value type will be:
          
                1, 2, 100, 1000

        and not

                1, 100, 1000, 2

      This means the result of a sort on an float value type will be:
          
                1.1, 2.23, 100.332, 1000.12

        and not

                1.1, 100.332, 1000.12, 2.23

      Date and date time values will be sorted by their equivalent
      value in UTC. No matter what the returned time zone in the result
      set returns. This is so that if multiple components are returned
      each in a unique time zone, the results will be sorted in UTC.
      This does not mean the values must be converted to UTC in the
      data returned to the CUA. It means the CS must do the sort in UTC.

      All other values are sorted according to the locale sorting order
      as specified in the calendar. Or the CS locale if the calendar
      does not have any locale set, or the host operating system
      locale if the CS does not specify a locale. And the locale to
      use for the sort is determined in that order.

      (2) The CS MUST sort at least the first column.
          The CS MAY sort additional columns.

      (3) If the cap-cols is only "*" and nothing else, then:

            If EXPAND=FALSE sorting will be by the DTSTART value
            ascending.

            If EXPAND=TRUE sorting will be by the RECURRENCE-ID value
            ascending.

          If one or more DTSTART or RECURRENCE-ID components have
          exactly the same value, the order for those matching
          components is unspecified.

      (4) All literal values are surrounded by single quotes ('), not
          double quotes ("), and not without any quotes. If the value
          contains quotes or any other ESCAPED-CHAR, they must be
          backslash escaped as described in section "4.3.11 Text" 
          of RFC2445. Any LIKE wildcard characters that are part
          of any literal data that is preceded by a LIKE clause and
          is not intended to mean wildcard search, MUST BE escaped as
          described in not (7) below.

      (5) When comparing DATE-TIME to DATE value types and when
          comparing DATE to DATE-TIME value types, the result will
          be true if the DATE value is on the same day as the DATE-TIME
          value. And they are compared in UTC no matter what time zone
          the data may actual have been stored in.
          
            VALUE-1             VALUE-2            Compare Results

            20020304            20020304T123456    TRUE
            (in UTC-3)          (in UTC-3)

            20020304            20020304T003456    FALSE
            (in UTC-4)          (in UTC-4)

            20020304T003456Z    20020205T003456    FALSE
            (in UTC-0)          (in UTC-7)

         When comparing DATE and DATE-TIME values with the LIKE
         clause the comparison will be done as if the value is
         a RFC2445 DATE or DATE-TIME string value.

                LIKE '2002%' will match anything in the year 2002.

                LIKE '200201%' will match anything in January 2002.

                LIKE '%T000000' will match anything at midnight.

                LIKE '____01__T%' will match anything for any year or
                                  time that is in January.
                                  (Four '_', '01', two '_' 'T%').

         Again all comparisons will be done in UTC.

	 Using a LIKE value of "%00%, would return any value that
         contained two consecutive zeros. 
      
      (6) DTEND and DURATION.

          When a VQUERY contains a DTEND value, then the CS MUST also
          evaluate any existing DURATION property value and determine
          if it has an effective end time that matches the VQUERY
supplied
          DTEND value or any range of values supplied by the VQUERY.

          When a VQUERY contains a DURATION value, then the CS MUST also
          evaluate any existing DTEND property value and determine
          if it has an effective duration that matches the VQUERY
supplied
          DURATION value or any range of values supplied by the VQUERY.

          As DTEND is the first time that is excluded from a components
          time range, any DURATION supplied by the VQUERY that is
exactly
          one second less than DTEND MUST match the VQUERY. And if the
          DURATION ends exactly at the computed DTEND it MUST NOT match.

          Any DTEND supplied by the VQUERY that is exactly one second
more
          than an end time computed from a DURATION MUST match the
VQUERY.
          Any end time that is computed from a DURATION that exactly
matches
          the supplied DTEND MUST NOT match.

	    (6.1) Given a meeting room reserved with a component
                  that contains:

		DTSTART:20020127T000000Z
		DTEND:20020127T010000Z

                The reservation is really from January 27th, 2002
00:00:00
                                          to   January 27th,
2002,00:59:59

            (6.2) Given another meeting room reserved with a component
                  that contains:

		DTSTART:20020127T000000Z
		DURATION:P59M59S

                The reservation is really from January 27th, 2002
00:00:00
                                          to   January 27th,
2002,00:59:59

            (6.3) A VQUERY that contains:

		... VEVENT.DTSTART = '20020127T00000'
                AND VEVENT.DTEND = '20020127T010000'

                MUST match both (6.1) and (6.2).

            (6.4) A VQUERY that contains:

		... VEVENT.DTSTART = '20020127T00000'
                AND DURATION = 'P59M59S'

                MUST match both (6.1) and (6.2).
                
      (7) [NOT] LIKE notes:

         The pattern matching characters is the '%' that matches
         zero or more characters, and '_' that matches exactly one
         character (where character does not always mean octet).

         LIKE pattern matches always cover the entire string. To match
         a pattern anywhere within a string, the pattern must start and
         end with a percent sign. 

         To match a '%' or '_' in the data and not have it interpreted
         as a wildcard character, they must be backslash escaped.
         That is to search for a '%' or '_' in the string:

               LIKE '%\%%'    Matches any string with a '%' in it.
               LIKE '%\_%'    Matches any string with a '_' in it.

         Strings compared using the LIKE clause MUST BE performed
         using case in-sensitive comparisons. ('a' = 'A').

         If LIKE is preceded by 'NOT' then there is a match when
         the string compare fails.

      (8) [NOT] "CONTAINS(" cap-lhs "," col-literal ")"

          This is similar to the LIKE element, except it does value
          matching and not string comparison matches.

              property:value1,value2

              CONTAINS(property, 'value1')   would match
              CONTAINS(property, 'value')    would NOT match

              LIKE(property, 'value%')       would match

          The CS must understand the objects being compared and
          understand how to determine how any multi valued property
          or parameter values are separated, quoted, and backslash
escaped
          and perform the comparisons as if each value existed by itself
          and not quoted or backslash escaped when comparing using the
          CONTAINS() element.

          If CONTAINS() is preceded by 'NOT' then there is a match when
          the value does not exist in the property or parameter value.

---

From: Patrice Lapierre <[email-masked]>
Date: 28 Jan 2002 11:20:15 -0500

> Attempt #3, PLEASE REVIEW

  Looks good, I think we are not far from a working proposition.
I included some suggestions and corrections in the text below.

>...
 
  I don't think this point is necessary, the ABNF should be enough. 
The word "everything" also seems too strong, the WHERE clause accepts
expressions and its elements can also include identifiers from the
USING_PROPERTIES clause.

>                    AND VALARM.TRIGGER > 20020101T000000Z
 
1. According to ABNF cap-literal must be quoted:
 
            (f) SELECT * FROM VEVENT WHERE
                    VALARM.TRIGGER < '20020201T000000Z'
                    AND VALARM.TRIGGER > '20020101T000000Z'
 
2. As mentioned previously be Alan, I think that the example 
is ambiguous, we don't know that the two sub-expressions refers 
to the same VALARM.
 
It was recommended that a new USING_COMPONENTS could be used to
address this issue (I included the construct in the ABNF below).
 
I would also suggest that the "SELECT" clause should accept
identifiers from USING_COMPONENTS and USING_PROPERTIES.
 
e.g.,

   To return only the matching VALARM of a VEVENT:
   
        SELECT alarm FROM VEVENT
        USING_COMPONENTS VALARM alarm
        WHERE alarm.TRIGGER < '20020201T000000Z'
             AND alarm.TRIGGER > '20020101T000000Z'
 
   To return only the matching ATTENDEE 
             
        SELECT att, DTSTART, DTEND FROM VEVENT
        USING_PROPERTIES ATTENDEE att
        WHERE att = [email-masked]'

>...
 
Typo: should be VEVENT (not VEVENET).

>                    VTODO.SUMMERY = "Fix typo in CAP"
 
Typo: should be SUMMARY.

> 4.1.1.1 "VQUERY ABNF"
...

>    queryname  = "QUERYNAME" *(xparam) ":" text CRLF

Should be:

   queryname  = "QUERYNAME" *(";" xparam) ":" text CRLF

>                / queries "," query
 
The "," makes it invalid iCalendar.
 
        queries = 1*( query )

*** NOTE *** : Multiple QUERY properties in a VQUERY is a new addition. 

  I'm not opposed, however I can see some potential side effects that 
were not mentioned:
 
 - Are the result of different QUERYs returned in the same VCALENDAR?
 - What is the orderings of the returned components?

>     capselect  = "SELECT" SP cap-cols SP
 
Must also support:
 
   SELECT VEVENT.VALARM
   SELECT VEVENT.*
 
>                  "FROM"   SP comp-name SP
 
comp-name won't accept VEVENT.VALARM.
 
> ...
 
To be consistent with other SQL clauses the proposition was:
 
    USING_PROPERTIES ATTENDEE att1, ATTENDEE att1
 
not
 
    USING_PROPERTIES ATTENDEE att1, USING_PROPERTIES ATTENDEE att2

>... 
 
Typo: should be PARAM() not PARM()
 
>                  # can only be used in cap-cmps of the same query.
>                  # the query.

    Local identifiers prefixed with "x-" won't ensure that there is no 
conflict. The simplest way to resolve this issue is simply stating the 
local identifiers introduced in the USING_PROPERTIES clause have
precedence over the other properties.

------------------------------------------------------------

  Here's the ABNF with the modifications mentioned above and 
following additions:
 
 - Support for an arbitrary number of spaces between tokens.

 - Add the USING_COMPONENTS clause.

 - SELECT clause can accept identifiers declared in USING_PROPERTIES
   and USING_COMPONENTS

 - Use ';' instead of '#' for comments (to conform to RFC2234).

 - cap-col was used both rhs the WHERE and the SELECT clause. 
   However in the WHERE clause component should not be accepted.

 - Add definitions to elements that contained only comments
   (reuse definitions found in RFC2445).

                 [ expand ] querycomp
                 "END:VQUERY" CRLF
 
               ; If not provided, EXPAND default to FALSE
    expand     = "EXPAND" *( ";" xparam) ":" ( "TRUE" / "FALSE" ) CRLF

    querycomp  = queries / ( queryname queries ) / queryname
 
    queryname  = "QUERYNAME" *(";" xparam) ":" text CRLF
 
    queries    = query *( "," query )
 
    query      = "QUERY" *(";" xparam) ":" capselect CRLF
 
    capselect  = "SELECT" 1*WSP cap-select-elems 1*WSP
                 "FROM"   1*WSP cap-from-elem 1*WSP
                 [ cap-using-components 1*WSP ]
                 [ cap-using-properties 1*WSP ]
                 "WHERE"  1*WSP cap-expr
 
               / "SELECT" 1*WSP cap-select-elems 1*WSP
                 "FROM"   1*WSP cap-from-elem
 
   cap-using-components = "USING_COMPONENTS" 1*WSP cap-local-comps
 
   cap-local-comps = cap-local-comp *( *WSP "," *WSP cap-local-comp )
   
   cap-local-comp = comp-name 1*WSP cap-comp-identifier
 
   cap-comp-identifier = iana-token / x-token
         ; identifier declared in the USING_COMPONENTS clause
         
   cap-using-properties = "USING_PROPERTIES" 1*WSP cap-local-props
 
   cap-local-props = cap-local-prop *( *WSP "," *WSP cap-local-prop )
   
   cap-local-prop = cap-prop 1*WSP cap-prop-identifier
 
   cap-prop-identifier = iana-token / x-token
      ; identifier declared in the USING_PROPERTIES clauses.
   
   cap-comp = comp-name *( "." comp-name )
                     ; component name  (e.g., VEVENT.VALARM)
            / cap-comp-identifier  
                 ; Identifier declared in the USING_COMPONENTS clause.
                 ; NOTE: when an identifier of the USING_COMPONENTS has
                 ; the same name as a component, the identifier has
                 ; precedence and hides the components name in the 
                 ; query.
            
   cap-select-elems = cap-select-elem *( *WSP "," *WSP cap-select-elem )
   
   cap-select-elem = cap-col
                   / "*"
                   / cap-comp [ ".*" ]
                     ; A component name of an existing component
                     ; contained inside of the cap-comp used in the 
                     ; FROM clause.
                     ;
                     ;   e.g., SELECT VALARM FROM VEVENT ...

   name = x-name / iana-token
                        ; name of a property as defined in RFC2445   
                        
   cap-prop    = [ cap-comp "." ] name
                  ; Any property name found in the component
                  ; named in the cap-from-elem used in the FROM clause.
                        
   cap-col     = [ cap-comp "." ] name
               / cap-prop-identifier
                  ; Identifier defined in the USING_PROPERTIES clause.
                  ; NOTE: The identifiers of the USING_PROPERTIES clause
                  ; have precedence over the property name.
   
   cap-from-elem = comp-name *( "." comp-name )

   literal-data = text
                  ; Any data that matches the value type of the
                  ; column that is being compared. That is you can
                  ; not compare PRIORITY to "some string" because
                  ; PRIORITY has a value type of integer. 
                  ; 
 
   like-expr = "'" text "'"
                  ; The characters '%' and '_' are wildcard characters. 
                  ; To match literally match '%' or '_', they
                  ; MUST BE backslash escaped as described in
                  ; the notes below in order for them not to be
                  ; treated as wildcard characters.

   cap-expr    = "(" *WSP cap-expr *WSP ")"
               / cap-term
 
   cap-term    = cap-expr 1*WSP cap-logical 1*WSP cap-expr
               / cap-factor
 
   cap-factor  = cap-lhs 1*WSP cap-oper 1*WSP col-literal
               / cap-lhs 1*WSP [ "NOT" 1*WSP ] "LIKE" 1*WSP like-expr
               / cap-lhs 1*WSP "IS" [ 1*WSP "NOT" ] 1*WSP "NULL"
               / [ "NOT" 1*WSP ] "CONTAINS" *WSP "(" *WSP 
                     cap-lhs *WSP "," *WSP col-literal *WSP ")"

   cap-lhs     = cap-col
               / "PARAM" *WSP "(" *WSP 
                 cap-col *WSP "," *WSP param-name *WSP ")"

   param-name = iana-token / x-token

   cap-oper    = "=" 
               / "!="
               / "<"
               / ">"
               / "<="
               / ">="

   SP   = %x20         ; space
   HTAB = %x09         ; horizontal tab
   WSP  =  SP / HTAB

---

From: Doug Royer <[email-masked]>
Date: Mon, 28 Jan 2002 17:11:38 -0700

>   I'm not opposed, however I can see some potential side effects that

It was discussed - just never made it into the ABNF before.

>  - Are the result of different QUERYs returned in the same VCALENDAR?

As if two seperate VQUERYs had been submitted.

>  - What is the orderings of the returned components?

Same as if two seperate VQUERYs had been submitted.

---

From: Doug Royer <[email-masked]>
Date: Mon, 28 Jan 2002 17:16:12 -0700

> precedence over the other properties.

That will not work. What if:

	SELECT * FROM VEVENT

And what if 'att1' is a (future) valid PROPERTY name?

We need to make sure that they are not valid future IANA names.
If not with 'X-', then how about 'L-' for LOCAL names?
And we reserve L- names to be local to VQUERY only?

---

From: Alan Davies <[email-masked]>
Date: Mon, 28 Jan 2002 19:48:21 -0500

>And what if 'att1' is a (future) valid PROPERTY name?

Then the att1 property would be ignored in this query, because "local
identifiers introduced in the USING_PROPERTIES clause have precedence
over the other properties".

If the creator of the query wanted to refer to the "att1" property, they
could just use different USING_PROPERTIES identifier names for the ATTENDEEs.

Why won't it work?

--Alan

---

From: Alan Davies <[email-masked]>
Date: Mon, 28 Jan 2002 19:56:22 -0500

>     cap-using   = "USING_PROPERTIES" SP cap-col SP cap-local

I think this would be clearer, and correct an error in cap-using:

                / "SELECT  SP cap-cols SP
                  "FROM"   SP comp-name SP
                  "USING_PROPERTIES" SP cap-uselist SP
                  "WHERE"  SP cap-expr

     cap-uselist = cap-using

     cap-using   = cap-col SP cap-local

--Alan

---

From: Doug Royer <[email-masked]>
Date: Mon, 28 Jan 2002 18:32:45 -0700

> could just use different USING_PROPERTIES identifier names for the 

Because if the 'att1' property was added to the component,
you can no longer get '*' without re-compiling your
CUA with a new VQUERY command.

---

From: Doug Royer <[email-masked]>
Date: Mon, 28 Jan 2002 18:48:38 -0700

>     USING_PROPERTIES ATTENDEE att1, ATTENDEE att1

Did you mean (two '1's):

	USING_PROPERTIES ATTENDEE att1, ATTENDEE att2

If so, yes - I misunderstood.

>     USING_PROPERTIES ATTENDEE att1, USING_PROPERTIES ATTENDEE att2

And as to WSP vs  SP.

No - we are not pretty printing the data, it is for
computer processing, not humans. Even the BEEP protocol
uses exactly one SPACE between elements.

---

From: Patrice Lapierre <[email-masked]>
Date: 29 Jan 2002 08:44:10 -0500

> CUA with a new VQUERY command.

I don't understand, if you do the following:

	SELECT * FROM VEVENT
	USING_PROPERTIES ATTENDEE UID
	WHERE UID = [email-masked]'

The query means: return all the properties of the VEVENTs
with an ATTENDEE property having the value [email-masked]'

There is not substitution of property names in the returned
elements. Only in the search condition.

i.e, In the returned VEVENTs the UID property will still be an
unique identifier, not an attendee.

---

From: Alan Davies <[email-masked]>
Date: Tue, 29 Jan 2002 09:50:03 -0500

>CUA with a new VQUERY command.

I still don't see why it won't work; if my CUA sends a "SELECT * FROM..."
query to a CS that supports the new "att1" property, is should recieve
the att1 property no matter when it was 'compiled'. The "*" is expanded
by the CS, and has nothing to do with the CUA.

--Alan

---

From: Patrice Lapierre <[email-masked]>
Date: 29 Jan 2002 11:01:11 -0500

> If so, yes - I misunderstood.

You are correct, I meant att1 and att2.

> uses exactly one SPACE between elements.

  BEEP use exactly one space only in frame header 
(a low-level construct visible only to the transport 
layer), no such limitation is present in the payload
where all CAP elements reside. 

  CAP-QL is closer to SQL, which does allow more than one
spaces. I don't see the arm in permitting an arbitrary 
number of spaces. It's not harder to implement, and queries 
can still be as compact when generated by a machine.

---

From: Patrice Lapierre <[email-masked]>
Date: 29 Jan 2002 11:41:18 -0500

On Mon, 2002-01-28 at 19:11, Doug Royer wrote:
...
> Same as if two seperate VQUERYs had been submitted.

  OK great, this would work. A note should be added to the
search command to indicate that distinct "ANS" messages will
be received for each QUERY (not just for different targets).

---

From: Doug Royer <[email-masked]>
Date: Tue, 29 Jan 2002 10:27:37 -0700

> be received for each QUERY (not just for different targets).

Or, they can be in the same RPY just like before BEEP:

	BEGIN:VCALEDNAR
	....
	....
	END:VCALENDAR
	BEGIN:VCALENDAR
	...
	...
	END:VCALENDAR

iCalendar allows for N number of BEGIN/END segments per MIME
body. That was outlined in 05 and was not a TRANSPORT issue.

---

From: Doug Royer <[email-masked]>
Date: Tue, 29 Jan 2002 10:50:13 -0700

> by the CS, and has nothing to do with the CUA.

So if a 'att1' property exists, then this will get it?

        SELECT att1 FROM VEVENT
	 WHERE att1.att1 = 'foo'
         AND att2.att1 = 'fee'

---

From: Alan Davies <[email-masked]>
Date: Tue, 29 Jan 2002 13:23:19 -0500

>          USING_PROPERTIES ATTENDEE att1, ATTENDEE att2

I don't understand that example, and don't see how it relates
to the answer that I gave about the expansion of "*".

Are you treating ATTENDEE as if it's a subcomponent, not a
property with the ATTENDEE.att1 type syntax?

I still don't see why Patrice's quote on this issue from some way
back up the thread doesn't solve the problem that I think you're
trying to demonstrate:

--Alan

---

From: Patrice Lapierre <[email-masked]>
Date: 29 Jan 2002 13:33:40 -0500

...
> body. That was outlined in 05 and was not a TRANSPORT issue.

  Multiple VCALENDARs in the same BEEP message also works.
But shouldn't we chose one alternative and document it in the
draft.

---

From: Doug Royer <[email-masked]>
Date: Tue, 29 Jan 2002 13:03:13 -0700

> to the answer that I gave about the expansion of "*".

My point is that for that property, it would be the
same as that property as part of '*'? If not, then I don't
understand the USING_PROPERTIES yet.

> property with the ATTENDEE.att1 type syntax?

I am lost - :-) 

Maybe that is my point. I need more text to understand it.
I thought I understood it when it COULD NOT be part of
the SELECT clause.

> back up the thread doesn't solve the problem that I think you're

I was just trying different examples in order for me to understand
the difference. I was not proposing any solution - yet - as
I don't understand what the USING_PROPERTIES local-name does
when it is in the SELECT clause.

If you are saying that it is treated just like the property
name (attendee in this example), then why not just enter
attendee?  If you are saying that it is NOT treated just
like the property that it is an ~alias~ for, then how will
it know how to treat it if it is part of the expanded '*',
or named specificly in the future as an existing property
in the component in the FROM clause?

What am I missing?

---

From: Doug Royer <[email-masked]>
Date: Tue, 29 Jan 2002 13:16:22 -0700

Patrice Lapierre wrote:
ND:VCALENDAR
> But shouldn't we chose one alternative and document it in the

I have no problem saying that the CS MAY reply with multiple
ANS messages. But for slow connections, I would prefer
if the CS could reply with one RPY and avoid all of the
chattiness of the protocol when needed.

Without byte reduction in the BEEP layer, the BEEP transport
overhead easily exceeds the iCalendar object size. At least
doubling the bandwidth consumed. So when bandwidth is a
more important issue - one RPY is best.

Also, there are valid iCalendar objects that contain multiple
objects, so a CS and CUA MUST support that. 

For a LARGE query on a SLOW CS - there is an advantages
to multiple ANS messages if a person is watching the data
transfers - the perceived performance goes up. As in 100 objects
each arriving one second apart is perceived to be faster to a user
than one big message coming after a 100 second delay.

But if the CUA is not a person waiting for the data, why waste
the bandwidth with the BEEP overhead, just send one RPY. Something
the impmlementors will have to take into account depending on their
application and the CS and CUA needs.

I think a CUA MUST allow for both RPY and ANS. With no
LESS than one iCalendar object per ANS or RPY.

If the CUA gets a ANS, it keeps processing until the last message.
If the CUA gets a RPY - done.

---

From: "Graham Gilmore" <[email-masked]>
Date: Tue, 29 Jan 2002 15:33:13 -0500

> > >          USING_PROPERTIES ATTENDEE att1, ATTENDEE att2

    No; because the USING_PROPERTIES redefines 'att1' (in the context of
this CAP-QL text *ONLY*) to be a specific ATTENDEE property that matches the
WHERE clause.
    If 'att1' becomes a property name at some later point, it doesn't
matter, because the local definition in USING_PROPERTIES (or
USING_COMPONENTS) will override it (although it still could/would be
selected in a "SELECT *".

> when it is in the SELECT clause.

    Basically, rather than SELECTing all (to use an example) ATTENDEE
properties in the components referred to in the FROM clause, *only* those
ATTENDEE properties that match the definition of 'att1' (i.e., that fit the
WHERE clause) will be SELECTed.

> in the component in the FROM clause?

    I don't know if it's possible to use the named property (or component)
in the FROM clause?  But assuming it was (for the sake of argument), then
there are 2 cases :  (a) You have an existing VQUERY using the new property
name as a USING_* variable name that you don't want to change.  Because of
the local precedence rule, you're fine here; the semantics of the VQUERY
won't change.  (b) You want to modify a VQUERY to use the new property name
(or component name) in the FROM clause, but you've already used the name as
a USING_* variable.  In that case, since you're already modifying the
VQUERY, just change the variable name :)

    Graham

---

From: Doug Royer <[email-masked]>
Date: Tue, 29 Jan 2002 14:46:47 -0700

> selected in a "SELECT *".

So in other words - if that happens - your toast and your
CUA can never again get out what it put it. Using the same
original VQUERY over time. What about the ROM CUA - hand held
devices and phones?

Today - you want to get out the UID and ATTENDEE entries, but
only for ATTENDEE is 'foo' or 'fee'. If I understand the
proposal, this is the QUERY:

	SELECT UID,att1 from VEVENT
	 USING_PROPERTIES ATTENDEE att1, ATTENDEE att2
         WHERE att1 = 'foo'
	 OR att2 = 'fee'

A year from today the 'att1' property is added as a valid property
that has NOTHING to do with ATTENDEE and without your CUA knowing
that it was added in the component stored in the CS. Now the same
VQUERY will not return the same results? The same VQUERY will now
return the UID and the value of the att1 property? If not how can
I then get the UID and value of the att1 property when the ATTENDEE
is 'foo' or 'fee'.

I think your past reply stated that the CUA must determine
the QUERY line dynamically? That won't work ether as any CUA
at any point in time will not know if the 'att1' property
really exists in the component or not.

You also stated that the USING_() value overrides the SELECT
clause. How will the CS know not to when the property names in
the components are NOT known until after the CS does the VQUERY?
How would it know that 'this' time I want the value of the
property name that I don't know exists, until after I do
the VQUERY? How could the CUA make a smart VQUERY using
only unknown property names - before it knows the property names
in the components that will be in the result set?

That is why I am proposing we define reserved name schema
that will never be a property name.

> VQUERY, just change the variable name :)

To what? The CUA will NOT know what non-existant property names
exist until after the VQUERY.

---

From: Patrice Lapierre <[email-masked]>
Date: 29 Jan 2002 16:47:45 -0500

An example may help clarifying the proposition.

Assuming that the target of a VQUERY only contains the following 
component:

   BEGIN:VEVENT
   UID:abc12234
   ...
   ATTENDEE:[email-masked]
   ATTENDEE:[email-masked]
   ATTENDEE:[email-masked]
   ATTENDEE:[email-masked]
   ATTENDEE:[email-masked]
   att1:imaginary property to demonstrate scoping.
   END:VEVENT

(1) The following VQUERY:

   BEGIN:VQUERY
   QUERY:SELECT UID,ATTENDEE FROM VEVENT 
     USING_PROPERTIES ATTENDEE att1, ATTENDEE att2
     WHERE att1 = [email-masked]' AND
     att2 = [email-masked]'
   END:VQUERY

would return the following component (omitting the 
surrounding VCALENDAR,...)

   BEGIN:VEVENT
   UID:abc12234
   ATTENDEE:[email-masked]
   ATTENDEE:[email-masked]
   ATTENDEE:[email-masked]
   ATTENDEE:[email-masked]
   ATTENDEE:[email-masked]
   END:VEVENT

(2) The following VQUERY:

   BEGIN:VEVENT
   QUERY:SELECT UID,att1 FROM VEVENT 
     USING_PROPERTIES ATTENDEE att1, ATTENDEE att2
     WHERE att1 = [email-masked]' AND
     att2 = [email-masked]'
   END:VEVENT

would return the component:

   BEGIN:VEVENT
   UID:abc12234
   ATTENDEE:[email-masked]
   END:VEVENT

NOTE: only the ATTENDEE properties matching the att1 in 
the query are returned.

(3) The following VQUERY:

   BEGIN:VEVENT
   QUERY:SELECT * FROM VEVENT 
     USING_PROPERTIES ATTENDEE att1, ATTENDEE att2
     WHERE att1 = [email-masked]' AND
     att2 = [email-masked]'
   END:VEVENT

would return the component:

   BEGIN:VEVENT
   UID:abc12234
   ...
   ATTENDEE:[email-masked]
   ATTENDEE:[email-masked]
   ATTENDEE:[email-masked]
   ATTENDEE:[email-masked]
   ATTENDEE:[email-masked]
   att1:imaginary property to demonstrate scoping.
   END:VEVENT

Now of course if your query explicitly refers to the "att1" 
property, then you'll need to use another name for local 
variable in your query.

e.g,
   
   BEGIN:VEVENT
   QUERY:SELECT att1,UID,att3 FROM VEVENT 
     USING_PROPERTIES ATTENDEE att3, ATTENDEE att2
     WHERE att3 = [email-masked]' AND
     att2 = [email-masked]'
   END:VEVENT

Would return:

   BEGIN:VEVENT
   UID:abc12234
   ATTENDEE:[email-masked]
   att1:imaginary property to demonstrate scoping.
   END:VEVENT

The same pattern would apply for components used in 
conjunction with the USING_COMPONENTS proposition.

---

From: Alan Davies <[email-masked]>
Date: Tue, 29 Jan 2002 17:01:47 -0500

>original VQUERY over time. What about the ROM CUA - hand held

No, your query will return the same results. A query like
"SELECT * from VEVENT" would return the new property, but
a query that names the attributes, or provides aliases for
them with "USING_PROPERTIES", would always return the same
result.

>         USING_PROPERTIES ATTENDEE att1, ATTENDEE att2

This query should be written:

SELECT UID, att1 from VEVENT
         USING_PROPERTIES ATTENDEE att1
        WHERE att1 = 'foo'
         OR att1 = 'fee'

or

SELECT UID, ATTENDEE from VEVENT
        WHERE ATTENDEE = 'foo'
         OR ATTENDEE = 'fee'

>I then get the UID and value of the att1 property when the ATTENDEE

As stated by Patrice and repeated by me, the local alias used for
the property would override any future property names in that query
and that query only.

>really exists in the component or not.

I don't understand what you mean; at the time of evaluation the
whole query should be present, and the "USING_PROPERTIES" clause
can be parsed.

>in the components that will be in the result set?

I really don't understand this, the CUA doesn't have to know
about unknown property names.

>that will never be a property name.

I don't think that's necessary when a 1-line statement about the
scope that the names are valid in is enough.

>exist until after the VQUERY.

Of course it doesn't, and it doesn't need to.

--Alan

---

From: "Graham Gilmore" <[email-masked]>
Date: Tue, 29 Jan 2002 18:28:38 -0500

> USING_PROPERTIES ATTENDEE att1, ATTENDEE att2

    I get the feeling you don't quite understand the proposal (or, more
specifically, Patrice's scoping addition); the VQUERY to "get out the UID
and ATTENDEE entries, but only for ATTENDEE is 'foo' or 'fee'" would be:

    The incidence of the token 'att1' in the USING_PROPERTIES clause
reserves the name 'att1' to mean the ATTENDEE property (from the
USING_PROPERTIES clause) that fits whatever description is forthcoming in
the WHERE clause.  Think of it as a pointer variable definition in C.
    When 'att1' is used in the WHERE clause, it is stating exactly which
ATTENDEE property is referenced by the 'att1' name, by giving value
constraints.  Thus, in the example above, 'att1' becomes any ATTENDEE
property whose value is 'foo' or 'fee'.  Think of it as assigning a value to
your pointer variable.
    Finally, in the SELECT clause, the 'att1' name is used as a placeholder
for the full definition which has been set up by the USING_PROPERTIES and
WHERE clauses.  Think of it as dereferencing your pointer variable.

    Maybe it would be more clear if the USING_PROPERTIES line came before
the SELECT, like this? :

> I then get the UID and value of the att1 property when the ATTENDEE

    I'm afraid you've misunderstood my post; my apologies if it wasn't clear
enough.  If, a year from today the 'att1' property is added as a valid
property (that has NOTHING to do with ATTENDEE and without my CUA knowing
that it was added in the component stored in the CS), THEN my VQUERY will
return the SAME results as it does today; because now, one year from now,
and until the end of time (or at least until a newer version of CAP changes
the scoping rules ;), 'att1' will mean "the ATTENDEE whose value is 'foo' or
'fee'" inside my VQUERY, and it will NEVER mean any new component or
property or parameter or command or what have you.  Yes, this means that if
I continue to use the name 'att1' then in that same VQUERY I will not be
able to *specifically* select a property named 'att1', because I will only
get the local VQUERY definition rather than the global definition (but
remember that a wildcard SELECT will still retrieve the property.)

> in the components that will be in the result set?

    I think this point is moot if you consider my (hopefully sufficiently
clear this time) explanation above.  The CS can figure out that 'att1' means
some user-defined substitution described in the VQUERY itself based solely
on the USING_PROPERTIES, regardless of the existence or non-existence of a
property named 'att1'.

    Graham

---

From: Doug Royer <[email-masked]>
Date: Tue, 29 Jan 2002 17:01:21 -0700

> and ATTENDEE entries, but only for ATTENDEE is 'foo' or 'fee'" would be:

Your right!

But I don't know how else to figure it out without going over
this. Someone (Patrice?) send a nice summary. I'll digest
that in the AM when my brain is still feeding on food and
not sugar and Caffeine.

Assuming that the proposal goes "as is", I do think that
we need more text to explain the proposal. Bruce (when he
catches up), seem so far to have similar questions to me.
So I am guessing other readers will have the same questions.

---

From: Patrice Lapierre <[email-masked]>
Date: Wed, 30 Jan 2002 09:48:42 -0500

Doug Royer wrote:
...
> this. Someone (Patrice?) send a nice summary. 
...

Here's an informal summary of my understanding of the USING_PROPERTIES
clause.

1. Overview of USING_PROPERTIES (without new suggestions).
   
The USING_PROPERTIES clause was introduced to manipulate
properties that may occur more than one in a component
(e.g., ATTENDEE). 

Informally, a clause of the form:

   USING_PROPERTIES PROPERTYNAME varname  

has the following semantic:
   
   There exist at least one property PROPERTYNAME (referred to
   as varname) that satisfies the query.

Here's an example of its usage:

   SELECT * FROM VEVENT 
   USING_PROPERTIES ATTENDEE att WHERE att = [email-masked]'

  This means select all VEVENTs having at least one property ATTENDEE 
with the value [email-masked]'

Alan also proposed that when referring directly to a multi-value 
property in the WHERE clause, the semantic would be "for all".

e.g.,

   SELECT * FROM VEVENT WHERE PARAM( ATTENDEE, PARTSTAT ) = 'ACCEPTED'

 Means select all VEVENT where ALL attendees have their participation
status set to ACCEPTED.

2. The problem of potential name conflict.

  The USING_PROPERTIES does introduce new identifiers that
may conflict with existing or future properties (or components).

  As a solution, Doug suggested the use or a prefix (i.e, "x-" or 
"l-").

  The other alternative, debated yesterday, is to give precedence 
to the local identifiers (e.g., problem similar the scope of
variables in programming languages such as C).

3. Proposition to allow the USING_PROPERTIES identifiers in the
   SELECT clause.

As mentioned in (1), the USING_PROPERTIES construct allows 
the manipulate individual instances of multi-value properties.

It appears that the same concept could also apply to the SELECT clause.

i.e.,

 - SELECT MULTIPROPERTYNAME FROM COMPONENT...

   means return ALL the instances of MULTIPROPERTYNAME.

 - SELECT ident FROM COMPONENT USING_PROPERTIES ident ...

   means return only the instances of MULTIPROPERTYNAME where
   the expression in the WHERE clause can be satisfied if
   the ident is bounded to the property.

e.g., 

   SELECT ATTENDEE FROM VEVENT
   USING_PROPERTIES ATTENDEE att WHERE att = [email-masked]'

would mean:

   Return ALL the attendee properties of the VEVENT containing
   [email-masked]' as an attendee.

And

   SELECT att FROM VEVENT
   USING_PROPERTIES ATTENDEE att WHERE att = [email-masked]'

Would mean:

   Return only the attendee(s) with value = [email-masked]' for
   all VEVENTs having [email-masked]' as an attendee.

Examples of (2) and (3) can be found here:

   http://www.imc.org/ietf-calendar/mail-archive/msg03622.html

4. USING_COMPONENTS.

  Alan mentioned that the multi-value problem is similar to
the multiple occurrences of sub-components.

e.g. VEVENTs in VAGENDA, VALARMs in VEVENT

  It seems that the introduction of a USING_COMPONENTS clause,
could address the issues of multi sub-components in a similar 
manner.

--
Patrice