Subject: Re: updated path data text

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 1 Feb 2005 09:52:09 -0500

Since things went momentarily quiet, let me take a crack at what I think we 
have discussed.

Weiming has asked for some efficiency in referencing related components.
I have asked that we use unique, pre-declared keys.
Jamal (and others I think) have asked for block operations.
We are still discussing where index (subscript) values go.  For this 
description, I am going to put them with content keys.
     I would really like to be able to put the common case of simple subscripts
     into the data path.  This can be done using a flag if we restrict content
     keying to the leaf reference.  As we do not have agreement on that, I am
     NOT including that in this description.

I think that would give:

OPER TLV := <PATH-DATA>+
PATH-DATA := <PATH> [DATAINFO]
PATH := flags IDcount <IDs>
DATAINFO := <SELECTOR>* <DATA>*
SELECTOR := BLKINFO | KEYINFO | IDXINFO
DATA := DATARAW | <PATH-DATA>+
KEYINFO := KeyID DATARAW
IDXINFO := uint32
DATARAW := encoded data representing ForCES LFB information.
     This can represent a simple value, a structure, a string, an array,
     or any other declared ForCES type.
     The questions of whether this is a packed representation, a TLV
     structure, or something else is a separable, and separate, discussion.

IDs are identifiers of elements of LFBs or structures.
The number of Selectors must either the same as the number of non-terminal 
array types selected by the IDs, plus an additional optional selector if 
the full ID list selects an array and a single element of the array is 
being referenced rather than the whole array.

A Selector is either a blk / range selector TLV or it is an index TLV 
holding an unsigned integer, or it is a keying TLV holding a key identifier 
and a DATARAW.

The question of the interaction between blk selectors and the following 
data element(s) is not fully resolved.  It seems desirable to allow the 
following two cases:
    If a block selector is followed by a single DATA TLV, then the Data
    (inner selection and/or value) is applied to each of the elements of 
the block.
    If a block selector is followed by a number of DATA TLVs equal to the
    size of the block, then each DATA TLV corresponds to one element of the 
block.
Do we need both of these?
Presumably, if the number is not 1 and does not match it is an error.

Note that using two BLKINFO selectors relative to nested arrays is legal, 
and should be defined to work, but is very prone to unexpected 
results.  (The number of elements that actually match the selector may not 
be the same for different inner array occurrances.)

Inner PATH-DATA will normally be used only when more than one item is being 
selected.  It allows the factoring out of the common subexpressions.  This 
may allow some efficiency in implementation and does allow some efficiency 
in bits on the wire.

Yours,
Joel M. Halpern

At 03:42 AM 1/29/2005, Wang,Weiming wrote:
> > OPER TLV : = <PATH-DATA>+
> > PATH-DATA := <PATH> [DATAINFO]
> > PATH TLV := flags IDcount <IDs>
> > DATAINFO TLV := [SELECTOR]  [DATARAW]
> > SELECTOR :=  [BLKINFO] [KEYINFO]* [IDINFO]
> > DATARAW := somedata* [DATARAW TLV]*
> > IDINFO := [INDINFO] [FIELDINFO]
> > FIELDINFO TLV := <zero or more fieldIDs [FIELDINFO]*>
> > INDINFO TLV :=  one or more indexes
> > KEYINFO TLV := KEYID KEY_DATA
> > BLKINFO TLV := [BLK_RANGE_INDEX] | [BLK_CNT_INDEX]

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 1 Feb 2005 14:27:38 -0500

Following up on this, I have been batting an idea for the path / index / 
key encoding back and forth.  The following changes the PATH and Selector 
encoding, with the goal of simplifying the common cases:

There is one more variation that occurs to me.  This is not what I sent to 
the list, and has some advantages and disadvantages.  Tell me what you think.

Suppose we declare that block and content indexing can only be applied at 
the end of a path. Given nesting, this does not restrict what can be 
described, just where one has to split the TLVs.
Then, we declare that indexes are included in the path sequence.

Suppose we take two flag bits, and declare the meanings as:
    00 last not a table (normal case), includes the case where the last
       is an indexed selection of a table element.
    01 last is the entire table
    10 last is a block selection from the table.  See BLK TLV
    11 last is a content keyed selection from the table, see KEY TLV
And the there is exactly one BLK or KEY TLV, and that is present only if 
the bits are set.
(Arguably, the 01 combination is redundant, but the bits are available and 
it provides clarity.)

So, to select x1 from table2 entry 4 inside table1 entry 10 we would have
OPER TLV
     PATH flag = 00, count = 5, IDS = 1.10.2.4.1
as we would like.

To select, from all table1s, the x1s of the 4the element of table2, we 
would have
OPER TLV: (whatever operation is being performed
    PATH-DATA TLV
        flag = 01, count=1, IDS = 1
        PATH-DATA TLV
            flag = 00, count=3, IDS = 2.4.1

To select an x2 based on the value of x1 (where x1 as a key has KeyID 1) 
from the 10th element of table1, we would have
OPER TLV: (whatever operation is being performed)
    PATH-DATA TLV
       flag = 11, count = 3, IDS = 1.10.2
       KEYTLV, KEYID = 1, DATARAW = x1value
       PATH-DATA TLV
          flag = 00, count = 1, IDS = 2 (to select x2)

If we go with this, the syntax would become:
OPER TLV := <PATH-DATA>+
PATH-DATA := <PATH> [DATAINFO]
PATH := flags IDcount <IDs>
DATAINFO := [SELECTOR] <DATA>*
SELECTOR := BLKINFO | KEYINFO
DATA := DATARAW | <PATH-DATA>+
KEYINFO := KeyID DATARAW
IDXINFO := uint32
DATARAW := encoded data representing ForCES LFB information.

Where IDs would include subscripts, and
SELECTOR would only occur if the flags indicated it should, and would only 
apply to the table that would be the end of the PATH at the TLV 
level.  This would mean that nested PATH-DATA is used both for multiple 
selection with common part extraction and for intermediate content keyed/ 
block selection.

Yours,
Joel

At 09:52 AM 2/1/2005, Joel M. Halpern wrote:
>Since things went momentarily quiet, let me take a crack at what I think 
>we have discussed.
>
>holding an unsigned integer, or it is a keying TLV holding a key 
>identifier and a DATARAW.
>
>    size of the block, then each DATA TLV corresponds to one element of 
> the block.
>Inner PATH-DATA will normally be used only when more than one item is 
>being selected.  It allows the factoring out of the common 
>subexpressions.  This may allow some efficiency in implementation and does 
>allow some efficiency in bits on the wire.
>
>Yours,
>Joel M. Halpern
>

---

From: Jamal Hadi Salim <[email-masked]>
Date: Tue, 1 Feb 2005 15:47:27 -0500

Looks very good and resolves the table of  tables issue i had.

Should cover both what i need as well as Weimings IMO.
I will run through all use cases later; and if we get an ACK from
Weiming i will fixup the doc with this update.

cheers,
jamal

---

From: Weiming Wang <[email-masked]>
Date: Mon, 31 Jan 2005 21:43:42 +0800

Jamal,

----- Original Message -----
From: "Jamal Hadi Salim" <[email-masked]>
Subject: Re: updated path data text

> >    INDINFO TLV:= [ENUM_INDEX] | [BLK_RANGE_INDEX] | [BLK_CNT_INDEX]
> > !ENUM_INDEX TLV is for enumerated indexs.
>
> Can you provide an example of ENUM_INDEX?
Maybe the word Enum is not so relevant, bu what I mean for ENUM_INDEX is for the
descrete index expressions like:

ENUM_INDEX TLV the Type = ENUM_INDEX_TLV
the Value = {1, 3, 5, 7, 9} representing the index of 1,3,5,7,9.

Jamal, I see your other remain comments relates to the other post, please allow
me reply in the followed later post.

Cheers,
Weiming

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 2 Feb 2005 09:49:20 -0500

For simplicity and generality, and to avoid accidentally limiting 
ourselves, my view is that field identifiers and table indexes should both 
be represented in 32 bit unsigned inttegers.  If we were to leave 
subscripts separate from the path elements, we could argue that subscripts 
would be 32 bits, but field identifiers would be 16 bits.    On the other 
hand, one can argue that it is easier to process the path components if 
they are 32 bits long.

To put it differently, I would prefer to concentrate on whether the 
combined path is the right thing.  If it is, then we can afford to use 32 
bit field identifiers.  If it is not right to combine them, then we can 
decide whether to use 16 or 32 bit field identifiers.

Yours,
Joel

At 08:53 AM 2/2/2005, Weiming Wang wrote:
>Joel,
>
>Before we decide how index is placed in the path, I have a related question to
>ask first. I see that the bits for index may be different from that for 
>fields,
>for index may probably range more than 64k, but fields may be enough with
>32bits. I haven't followed the related discussions before, therefore, do 
>we have
>any consensus on the bits for fields and indices?
>
>Yours,
>Weiming