Subject: Re: [Forces-protocol] RE: Protocol/model discussion?

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 11 Aug 2004 11:45:59 -0400

[We realized this conversation belonged on the list.  Sorry if it seems to
be missing some context.  Feel free to ask about background.]

I would expect that LFB class elements would be referenced by a TLV, but
that the pieces of that reference would not be a TLV.  For a GET, the TLV
would indicate that what follows is a identifier of some information, while
for a SET, the TLV would indicate that what follows is the information
identifier and the information value.  (One might use one TLV for the
target identifier all the time, and then a TLV for the value, to allow
representing structured values.)

We will define the format to be used for identifiers and values.
Since (at the moment) all LFB class elements and struct elements and array
subscripts are defined to be 32 bit unsigned values, we might define the
structure of an identifier (within a TLV, as the number of subidentifiers
followed by a sequence of 32 bit quantities, each one being context
specific, where the first number is the attribute identifier in the LFB
Class, and the succeeding values (if present) are either struct element
identifiers or array subscripts, where the proper struct or array is known
by walking the class definition.
(This looks rather like an SNMP OID< but with less tree at the top and
subscripts in line instead of at the end.)

This is certainly not the only possible representation, and there are
probably some problems I have overlooked in this representation of the
identifier of the target piece of the LFB instance for the SET / GET.

The assumption in the above is that the LFB Instance (and either explicitly
or implicitly the LFB Class) and the operation are outside of what I am
describing.

Yours,
Joel

At 10:43 PM 8/10/2004 -0400, Jamal Hadi Salim wrote:

> >  If one parses through a sequence of element identifying
> > values, and gets to an array, either there are no further identifiers, in
> > which case the whole array is being referenced, or there is a further
> > identifying value in which case that value is the subscript for the
> > array.  Clearly, we need a way to have a sequence of <attribute>
> {<element>
> > | subscript}* and to tell where that sequence ends.
>
>I am still not very XML literate and so tend to think in TLV mode (yes
>sir, binary); and i find it hard to visualize the difference between an
>element and an attribute - to me they (their IDs at least) seem to map
>to the T in TLV.
>Could we maybe get a convergence on this - i think it would help moving
>forward. IOW, i am not sure how to parse your staments above.
>
> > I think we will want to allow arrays with holes in them.
> > That implies a delete operation (to turn an entry back into a hole.)
> > Whether it implies a create operation, or just a write with a flag
> > indicating "create if non-existant" does not much matter to me.  There is
> > an elegance to the write w/ flag approach.
> > Clearly we will need to be able to return an error for attempting to
> read a
> > non-existant entry.
>
>We can resolve this as we go along. There are certain known schemes
>(some of which i listed) that would be useful to not reinvent.
>
> > What adjunct variables do we need for an array?  Do we need
> > "highestUsedIndex"?  Do we need "lowestNonExistantIndex"?  Are special
> > attributes that automatically exist in whatever container the array is in
> > the right answer?  (If so, you can not simply put an array in an array,
> but
> > must but a struct in between to give a place for the other elements.)
> > Another way to do it is to declare that special subscripts have meaning
> > (steal array.0, array.1, or negative subscripts?)  That seems ugly.
>
>I think "UsedIndex" needs to be tracked in the minimal; when you run to
>the end of the table you can wrap back and find the next index unused.
>Seems to be per array/table attribute.
>can always steal array.0 to store things (i have seen hardware that isnt
>that cheap play these sorts of games ;->); array.0 becaomes reserved and
>semantiacally an overlay.
>
>cheers,
>jamal

---

From: Zsolt Haraszti <[email-masked]>
Date: Wed, 11 Aug 2004 13:24:25 -0400

> subscripts in line instead of at the end.)

This is indeed a very obvious way to do it.  Until we find reason
to change it, lets assume we use this.  I will call the above sequence
of 32-bit IDs the "path specifier" or simply a "path".

The ForCES messages that operate on LFB capabilities and attributes
will conceptually composed as:

<FE-address><LFB-address><operator><path>[<op-and-path-specific-data>]

The first two are already have their TLVs in the draft. How the rest
will map to TLVs (or something else), I don't care at the moment.

I do, however, see an advantage of
1. if requests to multiple LFBs can be bundled into same message
   (the current draft supports that).  This implies that the
   <FE-address> as a TLV encloses the rest (nesting).
2. if multiple requests to different attributes within the same LFB
   can be bundled inside same request without repeating the
   <LFB-address> part.  A recent note from Jamal shows similar
   thinking.  This also implies further nesting.  I see no problem
   with that.

/zsolt

--
Zsolt Haraszti                Phone:  +1 919-765-0027/2017
Modular Networks              Mobile:      +1 919-522-2337
                              Email:  [email-masked]

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 11 Aug 2004 16:26:13 -0400

This seems to be moving along the lines of a couple of other comments
looking for associative arrays.
a) If the associative array is indexed by a simple uint32, I suppose it
does not really matter whether it is a normal array or an associative one.

b) I believe what is being asked for is arrays index by more complex things.
I do not think the CE needs this.  The data plane needs to be able to match
on more complex structures.  But the CE is just responsible for maintaining
table entries.
Yes, some CAMs support delete and create by value.  But if you have the CAM
mapped already, you can maintain it purely by entry index.

I would strongly suggest that given the information the CE has, we just use
normally indexed arrays, not associative lookups <bold> for CE access
</bold>.

Yours,
Joel M. Halpern

At 03:58 AM 8/12/2004 +0800, Weiming Wang wrote:
>[Weiming] I also thought like this with my early work with the GRMP, i.e., to
>use an attribute type id to do query op. But it was denied very soon when I
>began to have implementation experience with it. The problem is, very
>often, we
>need to have attribute values asssociated with the type id even when we do a
>qeury op. E.g., when we query a routing table for some specific routes, the
>matching conditions should be associated. This is quite like the op for
>database
>management, where retrive conditions are often needed.

---

From: Steven Blake <[email-masked]>
Date: Wed, 11 Aug 2004 16:51:14 -0400

> normally indexed arrays, not associative lookups <bold> for CE access

I can give a counter example where it would be desirable for the CE to
be able to access a FE table by value and not by index: a prefix table.
The CE will have to maintain one or more FIBs, each distributed to one
or more (or all) FEs.  It is fairly easy for the CE to maintain the FEs'
prefix tables by feeding updates (inserts/delete by value) to them as
its FIBs are updated, without maintaining copies of each FE's prefix
tables.

Regards,

// Steve

---

From: Jamal Hadi Salim <[email-masked]>
Date: Wed, 11 Aug 2004 21:17:04 -0400

> of 32-bit IDs the "path specifier" or simply a "path".

Nice.

> <FE-address><LFB-address><operator><path>[<op-and-path-specific-data>]

May I rewrite this to:
"<FE-address> <LFB-address <operator <path
[<op-and-path-specific-data>]*>+>+>+ "

["" are to avoid mail client from thinking the > is a quotation]

In other words:
- there can be one or more LFB instances addressed (batch)
- There can one or more operation on an addressed LFB instance (batch)
- operation can apply on zero or more paths
- op-and-paths may have one or more data values associated

The way i see it is starting at TLV levels, every "<" indicates the
begining of a TLV. So we may have up to 4 nested TLV levels.
We could collapse the operator and path into one and reduce the
nesting level[1] - which probably makes sense since the combo of
operator and path would be unique i.e you will not need to carry the
same operation on the same path more than once in a given message.

cheers,
jamal

[1] it becomes:
"<FE-address> <LFB-address <operator-path
[<op-and-path-specific-data>]*>+>+ "

---

From: Weiming Wang <[email-masked]>
Date: Thu, 12 Aug 2004 03:58:11 +0800

Hi Joel,

Just one comment, for it's 4:00am here :(

Cheers,
Weiming

----- Original Message -----
From: "Joel M. Halpern" <[email-masked]>
Subject: Re: [Forces-protocol] RE: Protocol/model discussion?

[Weiming] I also thought like this with my early work with the GRMP, i.e., to
use an attribute type id to do query op. But it was denied very soon when I
began to have implementation experience with it. The problem is, very often, we
need to have attribute values asssociated with the type id even when we do a
qeury op. E.g., when we query a routing table for some specific routes, the
matching conditions should be associated. This is quite like the op for database
management, where retrive conditions are often needed.

---

From: Zsolt Haraszti <[email-masked]>
Date: Wed, 11 Aug 2004 23:56:19 -0400

I fully agree with all you said below.

/zsolt

---

From: Jamal Hadi Salim <[email-masked]>
Date: Thu, 12 Aug 2004 08:46:58 -0400

I have attached a txt file which i think captures discussion so far.
It may be totaly off - comment.

Sorry, i also jumped into the table discussion with this - hopefully its
not distracting the main topic at hand.
I use the term "key" to describe what content is used to lookup in table
although the illustrated NOOP LFB contains "index" as the key.
I also noticed (and model folks have expressed concern about this
before), that the  LFBTLV use to address the LFB instance may be
insufficient space with 16 bits; ignore that issue for now - we will
revisit.

cheers,
jamal

---

From: "Wang,Weiming" <[email-masked]>
Date: Thu, 19 Aug 2004 15:55:30 +0800

Hi Jamal,

To be honest, although I have been trying to accord with your scheme, 
I just find it still hard for me to be the same page with you on the 
overall message format, even I don't try to use the flexible state diagram 
based TLVs format I proposed last time , and try to conform with the 
static TLV format. 

I'm not sure the LFBTLV and Operation TLV in your scheme actually mean. 
Is it a separate TLV or is it a part in a TLV? Even in the static TLVs 
scheme, I thought that the LFBTLV and the OPTLV should be separate and 
independant TLVs, that is, the message format should be look like as below:

Msg body = TLV:TLV:...

Every TLV has following format:

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|    Type = 'TLVforConfigMsg'   |               Length          |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~                          LFB TLV                              ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~                          OP TLV                               ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~                          Data TLV                             ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~                          ...                                  ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~                          OP TLV                               ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~                          Data TLV                             ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~                          ...                                  ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

By using a separate LFBTLV and OPTLV for LFB addressing and Operation 
description, we can define quite flexible LFB addressing scheme and 
operation scheme.

Moreover, to observe the above bit map, we can find that above first 
level TLV is actually not necessary. We can rewrite the message body 
directly as follows:

Msg body =  

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~                          LFB TLV                              ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~                          OP TLV                               ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~                          Data TLV                             ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~                          ...                                  ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~                          OP TLV                               ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~                          Data TLV                             ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 
~                          ...                                  ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~                          ...                                  ~
~                          ...                                  ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~                          LFB TLV                              ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~                          OP TLV                               ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 
~                          ...                                  ~
~                          ...                                  ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

I just think this is more tidy and efficient compared with the above 
one, not mentioning that it will ease parsing for implementation. 

Best regards,
Weiming

---

From: Jamal Hadi Salim <[email-masked]>
Date: Thu, 19 Aug 2004 07:17:55 -0400

> Every TLV has following format:

Actually what i proposed is:
Mainhdr:LFBTLV:LFBTLV:LFBTLV... As many as LFBTLV as you can fill in
one PDU. So at that level you can batch to multiple LFB instanceids.
Within the LFBTLV you can batch multiple Operations on multiple targets
(another batching level).
We also discussed short cutting getting to the OPeration and target.
We introduced a OP_path TLV.
Target is a specific attribute within a LFB instanceid which could go to
the granulaty of a cell within a table.
Joel has some concerns about the op_path which is based on efficiency
reasons (as opposed to layout reasons you have). Hes on vacation if i am
not mistaken and pops up once in a while - I was hoping to wait until he
is operational before furthering the discussion some more; however,
since your concern is different, i will post a slightly dated (but still
valid version) of the layout.
Please comment on this one and lets move from there.

We really need to make progress with this to move on to the next
steps in the protocol. I sensed that the model team has agreed with
what i posted except for Joel whose concerns are resolvable in a short
period after he gets better internet access.

cheers,
jamal

---

From: Alex Audu <[email-masked]>
Date: Thu, 19 Aug 2004 09:54:38 -0500

Hello Weiming,

You always need the top level "msg type" and  "length" indication to
guide the parser
as to how much of the coming data actually partains to the "msg type"
being processed.
The length information is necessary to prevent  reading into neighbor
region. It could also
be used to signal how much memory to allocate for the message.

Regards,
Alex.

Wang,Weiming wrote:

> From: "Jamal Hadi Salim" <[email-masked] <mailto:[email-masked]>>
> To: <[email-masked] <mailto:[email-masked]>>

---

From: Weiming Wang <[email-masked]>
Date: Sat, 21 Aug 2004 00:18:09 +0800

Jamal,

Thank you for the quick response. Pls see some more replies inline

Cheers,
Weiming

----- Original Message ----- 

> one PDU. So at that level you can batch to multiple LFB instanceids.

[Weiming]I actually understand this well. My debate is I see this LFBTLV
is in fact a general TLV that is for a specific message type, i.e., the 
LFBTLV you mean here is message specific, a LFBTLV for configure message 
will be different from a LFBTLV for query message.

What I mean for an LFBTLV is, it is specific for LFB addressing, i.e., 
it can be applied to all kind of messages where LFBs should be addressed. 
Then we may define a LFBTLV with following format:

 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |    Type = 'LFBTLV'            |               Length          |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |     LFB Type ID #1            |     LFB Instance ID #1        |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 ~                             ...                               ~
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |     LFB Type ID #N            |     LFB Instance ID #N        |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

(Pls don't pay attention too much on the bit length of the id used)

We may also define a moer complex LFBTLV for more complex LFB addressing, 
e.g., we then may be able to allow addressing for a range of LFBs like 
LFB A with the instance id from #1 to #10.

What I'm not very clear about your scheme in your LFBTLV is, it is still 
not very clearly how LFB will finally be addressed in the LFBTLV format. 
I just feel that the actual LFB IDs will still be shown as a field in the 
LFBTLV rathan than another TLV. Could you just present a more detailed 
message format so that we can see it more clear?

It is obvious that to use a specific LFBTLV for LFB addressing is more 
powerful and is also more tidy for the protocol text, for we only need 
to define it one time and then can be used in all messages.
[/Weiming]

> Within the LFBTLV you can batch multiple Operations on multiple targets
[Weiming]I think a specific LFBTLV just for LFB addressing can also easily 
do batching, and moreover, it can also cope with the case where we need to 
address many types of LFBs while with the same operation. I'm thinking  
it will be a little more complex to do this in your LFBTLV scheme.

> We introduced a OP_path TLV.
[Weiming]Even if we need such OP path, I think we can just use a OP TLV to 
include this. What I proposed a OP TLV in my approach is also a 'real' TLV 
which can describe different kind of operations and even with operation 
parameters like the path (we suppose the notion of operatioin parameter 
is different from operation Data which is include by Data TLV)

> period after he gets better internet access.
[Weiming]I think we need to solve the LFBTLV issue first before we move 
forward, because it is the top level issue for message format and so important 
for the followed work, and on the other hand I do feel the scheme in your text 
for it is really not so much satisfied. Pls believe I'm not a guy who is 
interested in setting obstacles. 

I agree some description for Data TLV has been discussed very well, but I don't 
think the top level message format has been discussed enough, both by the 
protocol team and also by model team.

---

From: Weiming Wang <[email-masked]>
Date: Sat, 21 Aug 2004 00:21:41 +0800

Hi Alex,
  ----- Original Message ----- 
  From: Alex Audu 
  To: [email-masked] 
  Sent: Thursday, August 19, 2004 10:54 PM
  Subject: Re: [Forces-protocol] RE: Protocol/model discussion?

  Hello Weiming,

  You always need the top level "msg type" and  "length" indication to guide the parser 
  as to how much of the coming data actually partains to the "msg type" being processed.
  The length information is necessary to prevent  reading into neighbor region. It could also
  be used to signal how much memory to allocate for the message.

  [Weiming]This is not a problem actually, because the 'msg type'and the 'msg length' is 
  is in the msg header and the issue we are discussing is only related to the msg body.
  The msg header is the same.
  Thank you for the comments.

  Weiming

  Regards,
  Alex.

  Wang,Weiming wrote:

    Hi Jamal,

    Msg body = TLV:TLV:...

    Msg body =  

    Best regards,
    Weiming