Subject: Re: [forces] comments #14

From: Jamal Hadi Salim <[email-masked]>
Date: Thu, 1 Sep 2011 05:14:19 -0400

Weiming,

On Thu, Sep 1, 2011 at 12:22 AM, Wang,Weiming <[email-masked]> wrote:
ent more:
>
> ----------
> 5,5.1. BasicMetadataDispatch
>
>
> A basic medatata dispatch LFB is defined to abstract the process in which a packet is
> dispatched to some path based on its associated  metadata value.
>
>
>
> 5.5.1.1. Data Handling
>
>
>
> The BasicMetadataDispatch LFB provides the function to dispatch input packets to a group output according to a metadatum and a dispatch table.
>
> The BasicMetadataDispatch has only one singleton input known as "PktsIn".

I think the above two sentences would be better to move to  5.5.1 above so that
we only talk about processing in this section.

> Every input packet should be associated with a metadatum that will be used by the LFB to do the dispatch.
> If a packet is not associated with such a metadatum, the packet will be dropped inside the LFB.
>
>
> This LFB contains a dispatch table named MetadataDispatchTable that can be set by CE so as to decide
> which metadatum is used for dispatch.

"...is configured by the CE to define which metadata is to be used for
dispatching."

>The table contains entires of a Metadata ID, a Metadata value  and an OutputIndex.
>
>The LFB uses all existing metadata associated with every input packet with their
> metadata IDs and metadata values as the content keys to look up the table to find out the relevant OutputIndex.
> If a packet failed lookup of the table, the packet will just be dropped inside the LFB.
>

Ok, now i realize my confusion was technical not editorial. It seems
sensible that
we should avoid content keys if we can get away with indexed tables. I am sorry
if i am rehashing past discussion: Why was this not a table inside a
table indirection?
First you use the Metadata ID to index into table 1 to find the table for
that Metadata. Drop if non-existent. Next you lookup the second table
using the metadata value to find the OutputIndex. Drop if non-existent.

>
>
> The BasicMetadataDispatch LFB has one output defined as a group output known as "PktsOut". A packet having found an OutputIndex will be output to the group port instance with the corresponding index.
>
>
>
> As an example, if the CE decides to dispatch packets according to physical port ID (PHYPortID), the CE may set the metadata ID of PHYPortID and the PHYPortID actual value (the metadata value), and assigned OutputIndex to the  dispatch table. When a packet arrives, metadata associated with the packet are used to look up the dispatch table to find out an output port instance for the packet.
>
>
>
> Currently the BasicMetadataDispatch LFB only allows the metadata value of the dispatch table entry be integer.

makes more sense to say "32-bit value".

>A metadatum with other types of value is not supported in this version. A more complex metadata dispatch LFB may be defined in future version of the library. In that LFB, multiple tuples of metadata with more value types supported may be used to dispatch packets.
>
>
>
> 5.5.1.2. Components
>
>
>
> This LFB has only one component named MetadataDispatchTable which is defined as an array. Each row of the array is a struct containing a   Metadata ID, a Metadata value and the OutputIndex to select the output port from the group.
>
>

I have no issues with the rest of the text.

cheers,
jamal

---

From: "Wang,Weiming" <[email-masked]>
Date: Thu, 1 Sep 2011 21:33:12 +0800

Fixed all except on the following point. 
Chuanhuang,pls double check the possibility to use a table in table here. 

thanks,
Weiming
----- Original Message ----- 
From: "Jamal Hadi Salim" <[email-masked]>

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 01 Sep 2011 10:08:19 -0400

I am probably missing a bit of context.
As a general rule, we do not use metadata IDs as table indices.  The 
reason is that they are allocated across all the uses of metdaa, so they 
are not a dense set.  Using them as an index produces a large, sparse, 
table.  Not particularly good.

We can use the values of a metadatum as an index if that particular set 
of values is designed to be dense, and start from a very small number 
(0, 1, or so).  Some metadata are designed that way, some aren't.

Yours,
Joel

On 9/1/2011 9:33 AM, Wang,Weiming wrote:
> From: "Jamal Hadi Salim"<[email-masked]>

---

From: Jamal Hadi Salim <[email-masked]>
Date: Thu, 1 Sep 2011 12:50:04 -0400

Those are good reasons - and i will be happy if you kept the
representation as is
as long as you say something of whats below in the text.

cheers,
jamal

On Thu, Sep 1, 2011 at 10:08 AM, Joel M. Halpern <[email-masked]> wrote:
> As a general rule, we do not use metadata IDs as table indices.  The reason
> is that they are allocated across all the uses of metdaa, so they are not a
> dense set.  Using them as an index produces a large, sparse, table.  Not
> particularly good.
>
> We can use the values of a metadatum as an index if that particular set of
> values is designed to be dense, and start from a very small number (0, 1, or
> so).  Some metadata are designed that way, some aren't.
>
> Yours,
> Joel
>

---

From: "Chuanhuang Li" <[email-masked]>
Date: Fri, 2 Sep 2011 09:11:08 +0800

Hi,All

As an alternative, i think we can define MetadataDispatchTable by using table in table as Jamal said.
Pherhaps, this definition is more convinient. 
name: MetadataDispatchTable
type: MetadataDispatchTableType 
array of:
{
    MetadataID  /*the ID value of the metadata.*/
    array of:
    {  
        MetadataValue,   
        OutputIndex   /*group output port index*/
     }
}

First, We use the ID value of the metadata to lookup the table (Note, not finding internal index, 
but the MetadataID member in the entry.), and will get a sub-table. Drop if non-existent. 
Next we lookup the sub-table using the metadata value to
 find the OutputIndex. Drop if non-existent.

Yours,
Chuanhuang	

======= 2011-09-02 00:49:15 Jamal Hadi Salim, wrote: =======

---

From: Jamal Hadi Salim <[email-masked]>
Date: Fri, 2 Sep 2011 13:44:11 -0400

Chuanhuang,

Based on what Joel said (the sparsity issue) - I am open to leaving it as it was
as long as the text reflects the reasoning.
I will leave it up to the authors to make that call.

cheers,
jamal

---

From: "Wang,Weiming" <[email-masked]>
Date: Thu, 22 Sep 2011 20:59:42 +0800

Chuanhuang and Jamal,

I'm ok with the definition. Moreover, as you mentioned, the MetadataID may have to be defined as a 'content key'. By use of the content key, we may avoid using it is the table index to look up the key, which, as Joel pointed, may not be dense enough.

How about the solution?

thks,
Weiming

----- Original Message ----- 
From: "Jamal Hadi Salim" <[email-masked]>
Subject: Re: comments #14

Chuanhuang,

cheers,
jamal

---

From: "Chuanhuang Li" <[email-masked]>
Date: Fri, 23 Sep 2011 09:59:12 +0800

Hi, all
I think there is no sparse issue, whatever format we define this table as -- 
array in array or just one array. We do not use metadata IDs as table indices.
The table indices still are: 1,2,3,...... The metadata ID is the content of the 
table (one of the member of the structure.). I prefer to the array in array definition 
as Jamal mentioned.

 name: MetadataDispatchTable
 type: MetadataDispatchTableType
 array of:
 {
    MetadataID /*the ID value of the metadata.*/
    array of:  
    {
       MetadataValue,
       OutputIndex /*group output port index*/
    }
}   
 we use MetadataValue as the content key in inner table,.
 we use the value of MetadataID as the content key in outer table.

Yours,
Chuanhuang 	

======= 2011-09-22 20:57:31 Wang,Weiming, wrote: =======

---

From: "Wang,Weiming" <[email-masked]>
Date: Fri, 23 Sep 2011 10:45:23 +0800

Pls provide a full XML definition so that we can review it again and write the text.

thks,
Weiming

---

From: "Chuanhuang Li" <[email-masked]>
Date: Fri, 23 Sep 2011 14:11:44 +0800

Array in array, the XML defined as follows:
 
***********************
     <dataTypeDef>
         <name>MetadataDispatchSubtableEntryType</name>
         <synopsis>Entry type for the subtable of metadata dispatch table.</synopsis>
         <struct>
            <component componentID="2">
               <name>MetadataValue</name>
               <synopsis>metadata value.</synopsis>
               <typeRef>uint32</typeRef>
            </component>
            <component componentID="3">
               <name>OutputIndex</name>
               <synopsis>group output port index.</synopsis>
               <typeRef>uint32</typeRef>
            </component>
         </struct>   
      </dataTypeDef> 

      <dataTypeDef>
         <name>MetadataDispatchSubtableType</name>
         <synopsis>Type for the subtable of metadata dispatch table. </synopsis>
         <array type="variable-size">
           <typeRef>MetadataDispatchSubtableEntryType</typeRef>
           <contentKey contentKeyID="1">
           <contentKeyField>MetadataValue</contentKeyField>
           </contentKey>
         </array>
      </dataTypeDef>

      <dataTypeDef>
         <name>MetadataDispatchType</name>
         <synopsis>Entry type for metadata dispatch table.</synopsis>
         <struct>
            <component componentID="1">
               <name>MetadataID</name>
               <synopsis>metadata ID</synopsis>
               <typeRef>uint32</typeRef>
            </component>
            <component componentID="2">
               <name>MetadataDispatchSubtable</name>
               <synopsis>Metadata dispatch table.</synopsis>
	           <array type="variable-size">
               <typeRef>MetadataDispatchSubtableType</typeRef>
               </array>
            </component>
         </struct>   
      </dataTypeDef> 

      <dataTypeDef>
         <name>MetadataDispatchTableType</name>
         <synopsis>Type for Metadata dispatch table.</synopsis>
         <array type="variable-size">
           <typeRef>MetadataDispatchType</typeRef>
           <contentKey contentKeyID="1">
           <contentKeyField>MetadataID</contentKeyField>
           </contentKey>           
         </array>
      </dataTypeDef>

Yours,
Chuanhuang	

======= 2011-09-23 10:43:15 Wang,Weiming, wrote: =======

---

From: Jamal Hadi Salim <[email-masked]>
Date: Fri, 23 Sep 2011 11:26:24 -0400

I will let the authors make the call on whats best ...
My take was on clarity for the reader

cheers,
jamal

---

From: "Wang,Weiming" <[email-masked]>
Date: Wed, 28 Sep 2011 10:56:47 +0800

Resend to the list because of no response from the list. 

thks,
Weiming

----- Original Message ----- 
From: "Weiming Wang" <[email-masked]>
To: "Chuanhuang Li" <[email-masked]>; "Jamal Hadi Salim" <[email-masked]>
Cc: "Joel M. Halpern" <[email-masked]>; "forces" <[email-masked]>; "Haleplidis Evangelos" <[email-masked]>; "Kentaro Ogawa" <[email-masked]>; "Weiming Wang" <[email-masked]>
Sent: Wednesday, September 28, 2011 10:32 AM
Subject: Re: comments #14

> When doing the update of the BasicMetadataDispatch LFB, I came up with a suggesttion that we may have also to define an "ExceptionOut" for the LFB. Currently, we defined that a failed dispatch packet will be dropped. This is unreasonable in some case. In some use case, it may only require that packets unable to be dispatched just pass throught the LFB for other processing rather than disappear. 
> 
> thanks,
> Weiming
>

---

From: "Chuanhuang Li" <[email-masked]>
Date: Thu, 29 Sep 2011 09:30:39 +0800

I agree with you. we only defined a group output in BasicMetadataDispatch currently.
If you all have no other opinion, i'll add an ExecptionOut port to this LFB. 
The execption case is :
o There is no matching when looking up the metadata dispatch table.

Now, the LFB ports definition is as follows:

Inputs:
PktsIn,  singleton input
frameExpected: Arbitrary
metadataExpected: Arbitrary

Outputs:
PktsOut,group output 
frameProduced: Arbitrary 
metadataProduced:  none 

ExceptionOut, single output
frameProduced: Arbitrary
metadataProduced: ExceptionID

Yours,
Chuanhuang

======= 2011-09-28 10:29:21 Weiming Wang, wrote: =======

---

From: "Wang,Weiming" <[email-masked]>
Date: Thu, 29 Sep 2011 17:30:20 +0800

Hi Chuanhuang, Jamal and Joel, 

We currently only define a single tuple dispatch table for the BasicMetadataDispatch LFB.  For this purpose, I don't see the need for an array in array structure for it. Rather, we may only need to define two component for the purpose, something like below, pls comment it: 

***********************
      <dataTypeDef>

    <LFBClassDef LFBClassID="16">
         <name>BasicMetadataDispatch</name>
......  
        <outputPort>
            <name>ExceptionOut</name>
....
         
         <components>
            <component componentID="1">
                <name>MetadataID</name>
                <synopsis>the metadata ID for dispatching</synopsis>
            </component>
            <component access="read-write" componentID="2">
                  <name>MetadataDispatchTable</name>
                   <synopsis>Metadata dispatch table.</synopsis>
                   <typeRef>MetadataDispatchTableType</typeRef>
            </component>      
         </components>
       </LFBClassDef>  
***********************

Accordingly，the description text for the BasicMetadataDispatch LFB is as follows, (Jamal especially) pls help to review it, thanks a lot.

---------- 
5.5.1. BasicMetadataDispatch
 
The BasicMetadataDispatch LFB is defined to abstract the process in which a packet is dispatched to some output path based on its associated metadata value. 
 
5.5.1.1. Data Handling
   
The BasicMetadataDispatch has only one singleton input known as "PktsIn". Every input packet should be associated with a metadata that will be used by the LFB to do the dispatch. This LFB contains a Metadata ID component a dispatch table named MetadataDispatchTable, all configured by the CE. The Metadata ID specifies which metadata is to be used for dispatching packets. The MetadataDispatchTable contains entries of a Metadata value and an OutputIndex, specifying that the packet with the metadata value must go out from the LFB group output port instance with the OutputIndex. 
 
Two output LFB ports are defined. 
 
The first output is a group output port known as "PktsOut". A packet with its associated metadata having found an OutputIndex by successfully looking up the dispatch table will be output to the group port instance with the corresponding index. 
   
The second output is a singleton output port known as "ExceptionOut", which will output packets for which the data processing failed, along with an additional ExceptionID metadata to indicate what caused the exception. Currently defined exception types include: 
 o There is no matching when looking up the metadata dispatch table.
 
As an example, if the CE decides to dispatch packets according to a physical port ID (PHYPortID), the CE may set the ID of PHYPortID metadata to the LFB first. Moreover, the CE also sets the PHYPortID actual values (the metadata values) and assigned OutputIndex for the values to the dispatch table in the LFB. When a packet arrives, a PHYPortID metadata is found associated with the packet, the metadata value is further used as a key to look up the dispatch table to find out an output port instance for the packet. 
 
Currently the BasicMetadataDispatch LFB only allows the metadata value of the dispatch table entry be 32-bits integer. A metadata with other types of value is not supported in this version. A more complex metadata dispatch LFB may be defined in future version of the library. In that LFB, multiple tuples of metadata with more value types supported may be used to dispatch packets.
 
5.5.1.2. Components
 
This LFB has two components. One component is MetadataID and the other is MetadataDispatchTable. Each row of the dispatch table is a struct containing a metadata value entry and the OutputIndex entry. Note that currently, the metadata value is only allowed to be 32-bits integer. The metadata value is also defined as a content key for the table. The concept of content key is a searching key for tables which is defined in the ForCES FE Model (RFC 5812).  See this document and also the ForCES Protocol (RFC 5810) for more details on the definition and use of a content key.
---------------------

thanks,
Weiming

---

From: "Chuanhuang Li" <[email-masked]>
Date: Thu, 29 Sep 2011 23:16:54 +0800

Exactly, the original purpose is letting the LFB support dispatch function according to 
arbitrary metadata in one BasicMetadataDispatch instance.
Yes, maybe there is something we haven't considered before. If there are more than 
one input metadata, how will the LFB do? Which metadata should be used to look 
up the dispatch table? There is no contraints.
So i agree to this modification.

Yours,
Chuanhuang

======= 2011-09-29 17:30:22 Wang,Weiming, wrote: =======