Subject: [manet] Regarding packetbb constraints on ordering and uniqueness.

From: Thomas Clausen <[email-masked]>
Date: Tue, 23 Oct 2007 12:24:15 +0200

Dear all,

As Ian hinted in his previous email, the issues regarding the  
constraints on TLV ordering and uniqueness may not have been  
expressed in a way which was clear to everybody. I will try to do my  
best to remedy that in this email, and I am sure that my co-authors  
will help supplement me where I am insufficient.

I should add as a disclaimer that I am using a couple of examples in  
this email, and that these are chosen explicitly to show the ideas  
and principles -- to be illustrative only.

I think that it is important to up-front recall what packetbb is and  
is not.

Designed as a packet format for initially OLSRv2, since generalised  
based on discussions within the MANET wg, its goal is to efficiently  
encode the information exchanged as part of MANET routing protocols.  
Essentially, this means encode IP addresses (IPv4 and IPv6 both) in a  
compact way, associating attributes to such IP addresses, and  
encoding other information, specified by and exchanged as part of, a  
MANET routing protocol. Other than efficiently encoding what is being  
sent "on the wire" or "over the air", an additional goal has been to  
ensure that this encoding can be efficiently and uniquely parsed (I  
will define exactly what I mean by "parsing" later in this email)  
upon being received -- to put in simple terms: that addresses and  
associated attributes can be extracted efficiently.

packetbb is not, however, a specification of a protocol and so does  
not specify or mandate any interpretation of or response to what is  
being exchanged. It is purely an encoding format, to be used by a  
protocol or protocols which will specify or mandate such  
interpretation or responses.

While the issue being discussed at present is one of the constraints  
on TLVs in TLV blocks, I will take a step back and describe briefly  
some of the basics of packetbb, before discussing the specifics of  
TLVs and TLB constraints.

IP addresses as commonly deployed, v6 and v4 indiscriminately, have a  
basic structure. In IPv4, for example, addresses deployed within the  
same administrative structure commonly share the same prefix, for  
example, 192.168.1/24, with each configured interface within that  
administrative structure having an unique last-octet. This is also  
true in IPv6 where, additionally, IP addresses assigned to the same  
interface commonly will share the same "host ID" (last 64 bits), with  
different prefixes. For example, an interface configured with a Link  
Local Address as well as a Global Addresses will use the same host ID  
as the latter 64 bits of either address. As specified, packetbb  
allows a protocol to exploit this by allowing addresses which have  
common parts to be represented together without repetition of the  
common part. Such addresses with common parts in them are grouped  
together in an "address block".

Consider the following, simple, example. One has 10 IPv4 addresses  
and, considering a classic Link State protocol, a LS message may wish  
to advertise these as directly adjacent:
	
	192.168.1.1, 192.168.1.10, 192.168.1.20, 192.168.1.30, 192.168.1.40
	62.168.1.1, 62.168.1.10, 62.168.1.20, 62.168.1.30, 62.168.1.40

Explicitly representing these as above will "cost" 40 octets.

The encoding structure in packetbb allows a protocol implementation  
to encode these addresses explicitly, or aggregated in two address  
blocks as follows:

	192.168.1.{1,10,20,30,40}
	62.168.1.{1,10,20,30,40}

Which "costs" only 16 octets. Obviously, for IPv6 where addresses are  
larger, the gain may potentially be more significant.

What is important to extract from the above is that, concerning  
packetbb:

	o an address being included in one or another address block is not,  
by packetbb,
	   given any specific meaning. A protocol may or may not assign a  
specific
	   meaning to addresses being in one or another address block;

	o packetbb allows representing addresses with "common bits" in an  
aggregated
	   fashion;

	o specifically, packetbb does not state that "thou shall process  
these addresses in a
	   specific order". It is up to the protocol to decide such --  
packetbb is, on this matter,
	   only concerned with an efficient way of encoding addresses for  
exchange over
	   the wire/air.

Now, assuming that our Link State protocol wants to say "Up" or  
"Down" for a links to adjacent nodes. Such information would be an  
attribute associated with an address, and in packetbb such are  
represented by TLVs. Assuming that the TLV concept in general is well  
understood, I will here focus only on that which is specific to  
packetbb. Still, TLVs have a type and a value (hence, the T and V),  
and so for the sake of this example we consider a TLV with type  
"STATE" and possible values of "UP" and "DOWN".

Rather than having to include one explicit TLV for each address  
(which would again imply repetition of information -- something  
packetbb is intended to avoid, although packetbb also allows doing  
this if a protocol should so desire), TLVs associated to addresses in  
packetbb can be associated in different ways:

	o to a single address
	o to a range of addresses
	o to all addresses in an address block

Finally, TLVs in packetbb can be of the "multivalue" kind -- which I  
will come back to shortly.

So, in a subset of the example above, assuming that the attributes  
"STATE=UP" and "STATE=DOWN" is to be encoded for the following  
addresses according to the following:

	UP:
  	192.168.1.1, 192.168.1.20, 192.168.1.30

	DOWN:
	192.168.1.10, 192.168.1.40

packetbb allows that this can be done thus:

	Address Block:	192.168.1.{1,20,30,10,40}
	TLV Block	T=STATE, V=UP, START=1, STOP=3
				T=STATE, V=DOWN, START=4, STOP=5

Note that the order of the addresses within the address block is  
chosen such that only two TLVs (with start and stop indices) are  
needed. An alternative encoding, also supported by packetbb, could be  
to separate the addresses into two address blocks thus:

	Address Block:	192.168.1.{1,20,30}
	TLV Block	T=STATE, V=UP, START=1, STOP=3

	Address Block:	192.168.1.{10,40}
	TLV Block	T=STATE, V=DOWN, START=1, STOP=2

What is important to extract from the above is that, concerning  
packetbb:

	o addresses can be collected into address blocks, and the addresses  
within
	   an address block ordered such that it will allow the most  
efficient (compact)
	   representation of the information exchanged by a given protocol;

	o packetbb provides structure which a protocol can optimise the use  
of, and
	   packetbb does not mandate or suggest a specific usage;

	o the choice as to how addresses are collected into address blocks,  
and how
	   TLVs are associated with these is a matter for the specification  
of the protocol(s)
	   using packetbb.

I promised to return to the "multivalue" TLV kind, which I will do  
now. Assuming that, in the example above, we wanted to also associate  
a value -- say, cost -- to each address, and assuming that these  
costs would be different for each address, such as this:

	100: 192.168.1.1
	2: 192.168.1.10
	42: 192.168.1.20
	1: 192.168.1.30
	4: 192.168.1.40

Rather than to have to include five different TLVs, the multivalue  
TLV allows us to define a "COST" TLV and do the following:

	Address Block:	192.168.1.{1,20,30,10,40}
	TLV Block	T=STATE, V=UP, START=1, STOP=3
				T=STATE, V=DOWN, START=4, STOP=5
				T=COST, V={100, 42, 1, 2, 4}, START=1, STOP=5

Notice that I included the start and stop indices also on the COST  
TLV. While this is not necessary since the TLV applies to all  
addresses in the TLV block, I included it in this example to show  
that also multivalue TLVs of course can apply to ranges of addresses.

It should be noted that the choice of which TLV "kind" to use  
(multivalue, range-indexed, ....) when generating a message is not  
mandated by packetbb, since it depends on the given deployment, the  
protocol(s) etc.

A final note on the structure of packetbb (simplified):

	o a set of address blocks, plus their associated address-block-tlv- 
blocks are included
	    in a message. A message has a header (for controlled-scoped  
forwarding etc),
	    and contains optionally a message TLV block.

	o a set of messages are included for transmission in a packet --  
which also has an
	   optional packet TLV block.

The idea of collecting multiple messages in a packet and transmit  
these in one transmission is to reduce the MAC overhead -- something  
which has been observed to have a measurable impact when running  
MANETs -- but which will not be discussed further in this email.

Now, let us look at the constraints. The specification of packetbb  
lists a number of constraints that must be satisfied when including  
TLVs in a given TLV block. For now, I will only discuss address block  
TLV blocks (i.e. TLV blocks associated to addres-blocks), but I will  
get back to message- and packet-TLV blocks later.

The constraints as given and as discussed in recent emails on this  
mailing list are, essentially, that within an address block TLV block:
	
	o TLVs must be sorted in non-descending order according to their type;
	o no two TLVs of the same type may be associated with the same address.

I will discuss these in the context of address block tlv blocks,  
however first I think that it is important to note one thing -- which  
is essentially a repeat from the above part of this email:

	o packetbb is concerned only with the encoding, not with the  
processing, specifically
	o the processing of entities (addresses) and attributes (that which  
is associated to
	   addresses through TLVs), including the order of processing, is  
neither suggested
	   nor mandated by packetbb.

I want at this point to be very clear on one additional thing, which  
is that when in this email I talk about "parsing", I simply mean  
extracting entities and their associated attributes. For example, if  
I receive and parse a message as the following:

	Address Block:	192.168.1.{1,20,30,10,40}
	TLV Block	T=STATE, V=UP, START=1, STOP=3
				T=STATE, V=DOWN, START=4, STOP=5
				T=COST, V={100, 42, 1, 2, 4}, START=1, STOP=5

the result of parsing is a data structure containing:

	192.168.1.1		STATE=UP		COST=100
	192.168.1.10	STATE=DOWN	COST=2
	192.168.1.20	STATE=UP		COST=42
	192.168.1.30	STATE=UP		COST=1
	192.168.1.40	STATE=DOWN	COST=4

Which may (of course) be represented and interpreted by the protocol  
in any which way and/or order it desires.

This, in a way, is the crux of this discussion: packetbb is concerned  
only with encoding a data structure as the one above for transmitting  
over the wire/air and decoding the data structure at the receiver --  
not with how or in which order the data structure is processed. So,  
in that sense, packetbb parsing comes before protocol processing, and  
protocol processing is concerned only with the decoded data structure  
-- not how this data structure happened to be encoded for transmission.

Consider two protocols using identical TLVs -- say "STATE" and "COST"  
as above. Protocol A may, for correct functioning, require that  
receivers process the STATE TLV before the COST TLV. Protocol B,  
however, requires for correct  functioning that COST TLVs be  
processed before STATE TLVs. The specification of each protocol would  
state exactly which order is required (and the implementations,  
correspondingly, doing exactly that), and this would be entirely  
independent of how these are encoded for transmission.

What is important to extract from the above is that, concerning  
packetbb:

	o the order in which TLVs are encoded by packetbb for transmission is
	   independent from the order in which TLVs are processed by a protocol
	   specification;

	o a protocol specification SHOULD specify, assuming that the processing
	   order is important, what this processing order is.

I will touch upon the benefits of mandating an encoding-order in  
packetbb a bit later in this email, since it should become clearer  
what they are once the second constraint -- uniqueness -- has been  
discussed.

The second constraint for address-block tlv-blocks is that:

	o no two TLVs of the same type may be associated with the same address.

If we consider the previous example with one small modification:

	Address Block:	192.168.1.{1,20,30,10,40}
	TLV Block	T=STATE, V=UP, START=1, STOP=3
				T=STATE, V=DOWN, START=3, STOP=5
				T=COST, V={100, 42, 1, 2, 4}, START=1, STOP=5

the result of parsing is a data structure containing:

	192.168.1.1		STATE=UP			COST=100
	192.168.1.10	STATE=DOWN		COST=2
	192.168.1.20	STATE=UP			COST=42
	192.168.1.30	STATE=UP/DOWN	COST=1
	192.168.1.40	STATE=DOWN		COST=4

we note that the address 192.168.1.30 now has both UP and DOWN  
associated. Clearly, the sending node transmitted ambiguous  
information -- and while a discussion as to what this means and how  
to treat that belongs elsewhere (i.e. not in packetbb), it suffices  
to simply observe this to be the case. On the mailing-list, a number  
of suggestions as to how to handle this have been suggested, as  
alternatives to the constraint in packetbb. While diverse, these  
suggestions essentially come down to something like the following:

	- pick the first/last TLV and ignore the other(s) to disambiguify;

Now, on the "pick the last/first TLV and ignore the other(s), this  
won't work. Consider the following example:

	Address Block:	192.168.1.{1,20,30,10,40}
	TLV Block	T=STATE, V=UP, START=1, STOP=2		(i)
				T=STATE, V=DOWN, START=2, STOP=4	(ii)
				T=STATE, V=UP, START=4, STOP=5		(iii)
				T=COST, V=10, START=1, STOP=2		(iv)
				T=COST, V=1, START=3, STOP=5			(v)

This address-block-TLV-block has three TLVs of type STATE which are  
overlapping.  The roman numbers to the right is to allow references  
to an exact TLV in the following discussion.

The result of parsing is a data structure containing:

	192.168.1.1		STATE=UP			COST=100
	192.168.1.10	STATE=DOWN		COST=2
	192.168.1.20	STATE=UP/DOWN	COST=42
	192.168.1.30	STATE=UP/DOWN	COST=1
	192.168.1.40	STATE=UP			COST=4

Ok, ambiguous information, let's apply the "Last TLV wins" rule. How  
do we do that?

	For the address 192.168.1.20, the first TLV with type STATE is (i)  
and the last is (ii)
	For the address 192.168.1.30, the first TLV with type STATE is (ii)  
and the last is (iii)

So, which TLV is "last"? If we go by "last TLV per address", then we  
will end up using some of (i) and some of (ii), but certainly not all  
of (i) nor all of (ii) -- leading to partial and incoherent use of  
the information in the TLVs.

So, the "pick the first/last TLV" solution, while perhaps  
disambiguifying matters for a single address do not solve the problem  
-- the protocol receiving the data structure resulting from parsing  
still becomes exposed to incoherent information. The constraint of  
"no two TLVs of the same type may be associated with the same  
address" does solve this.

Now, I carefully mentioned that I would discuss this for address- 
block-tlv-blocks, and then return to message-tlv-blocks and packet- 
tlv-blocks later. As can be noted from the packetbb I-D, the  
constraint of uniqueness is relaxed for message- and packet- tlv- 
blocks, noting that:

  	o Packet and message TLVs MUST be defined so as to indicate whether
	   two TLVs with the same tlv-fulltype are, or are not, allowed in
	   the same packet or message TLV block, respectively.

The reason that this is permissible in message- and packet- tlv- 
blocks is, that tlvs in those tlv-blocks associate to one and only  
one entity: the message or the packet. As such, while ambiguity still  
may persist, the risk of "partial" or "incoherent" association (and,  
thus, usage) of part of one and part of another tlv does not exist.

I promised to touch upon the benefits of these constraints, which I  
will do in the following -- but will otherwise refer to the emails  
from Chris Dearlove, Justin Dean and myself on the list for more  
details.

The "uniqueness" constraint of address block TLVs is there for the  
reason of ensuring non-ambiguity and coherence in the data structure  
resulting from parsing an incoming message. Keeping the TLVs within a  
TLV block ordered by TLV type means that verifying (at both the  
sending side, prior to transmitting a message, and on the receiving  
side during parsing but prior to presenting the resulting data  
structure to the protocol) this uniqueness criteria can be done in a  
single linear pass. Having the TLVs unordered would imply  
backtracking and maintaining additional data structures.

I want to emphasise that the order of inclusion of TLVs in a TLV  
block is not indicative or suggestive of the order of processing of  
the TLVs within that TLV block, just as the order of inclusion of  
addresses within an address block (or address blocks) is not  
indicative or suggestive of the order of processing of these  
addresses. Indeed, such is, if processing order is important, to be  
specified by the protocol using packetbb.

An additional advantage of the ordering is, that if one is looking  
for a specific TLV (e.g. a protocol or protocol extension where  
processing the message is not worth it unless tlv type XXX is  
included) then one can terminate parsing the message (or packet or  
address block) early, i.e. once parsing encounters a tlv with type  
 >XXX. A clear example hereof would be "do not process this message  
unless a signature TLV is included". It has been pointed out that for  
that reason it would be helpful if a range of TLV types were  
allocated which "comes before everything else", and that guidelines  
be provided to IANA as to how types from that range are assigned --  
and this point has been updated in the I-D.

A new I-D (draft-ietf-manet-packetbb-10.txt) has been submitted to  
the IETF, in which:

	o a range of (the numerically first, in order to retain the benefits  
of ordering)
	   address-block-, message- and packet- TLV types are reserved for  
assignment
	   where it is important that these "come before anything else", as  
indicated in the
	   preceding paragraph) is specified;

	o clarifications have been provided regarding the ordering of TLV  
inclusion in TLV
	   blocks being a matter of encoding-order only, not processing order.

The I-D should appear shortly (there were some issues with the I-D  
submission tool slowing things down, but thanks to Michael, the tool- 
man at the IETF, I believe that these have been resolved and the I-Ds  
started flowing).

I hope that this email clarifies these issues, and that otherwise you  
will not hesitate to ask clarifying questions.

Sincerely,

Thomas

-- 
Thomas Clausen
[email-masked]
http://www.thomasclausen.org/
http://hipercom.thomasclausen.org/

_______________________________________________
manet mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/manet

---

From: "Joseph Kopena" <[email-masked]>
Date: Tue, 23 Oct 2007 13:08:59 -0400

> will help supplement me where I am insufficient.

Hi Thomas,

Thank you for the email.  It is very clear & informative, and the
examples simple & useful.

> backtracking and maintaining additional data structures.

What is the consensus on the best way to represent a graph in
PacketBB?  This seems very simple & reasonably efficient if non-unique
TLVs are allowed, simply list all addresses and have a set of possibly
non-unique TLVs reporting edge pairs.  The only solution I've come up
with off-hand which maintains uniqueness is having 1 TLV per node and
stuffing all neighbor indexes into that value.  That's admittedly
slightly more efficient bit-wise, but requires the parent object to do
more of its own parsing and requires tighter interaction with PacketBB
(because the parent object is now operating on indexes of this address
block, rather than asserting & retrieving addresses).  What's the
recommended idiom for this, if there is one?

> specified by the protocol using packetbb.

If processing order is actually important, which I could believe is a
fairly common scenario, doesn't sorting the TLVs for parsing actually
slow things down in many cases?  For example, suppose I need to
process STATE before COST.  If the TLVs are not sorted, the parent
object can place them in that order, and then simply parse & process
them linearly on the receiver end.  If PacketBB sorts those TLVs
however, the protocol designer may have no control over the order if
they're following IANA numbers or mixing protocol elements/existing
code.  They could quite possibly wind up in the wrong order from the
processing perspective.  Necessarily sorting all TLVs therefore avoids
the aforementioned possible costs in parsing (discussed later), only
to incur them in processing.

> and this point has been updated in the I-D.

This shortcut-parsing advantage is worth keeping in mind, but
ultimately not a very strong argument because it works contrary to an
important supporting argument for the constraints---that they don't
effect processing order.  It should likely also be cause for concern
that there is already extra management effort & resource costs
(restrictions on the number space) being incurred by the necessity of
having to carefully choose identifiers for at least this feature.

It seems like this same shortcutting can be achieved if PacketBB
simply has to preserve the ordering generated by the parent object.
In addition to being simpler & more flexible from protocol designers'
perspectives, that has the additional benefit that the parent can
ensure the signature is first, allowing the receiver to shortcircuit
all of the parsing if it's not there.  In the current scheme it has to
parse any TLVs which just happen to have an identifier lower than
whatever the signature fulltype happens to be.

But, in general, I think this whole discussion hinges on the
importance of uniqueness and how costly it can be to detect
non-uniqueness.  I'm curious to hear opinions on representing graphs,
as that seems to be a reasonable example which is naturally
represented with non-unique TLVs.  It also seems to me that the costs
of sorting the TLVs are no better than determining uniqueness given
unsorted TLVs, while also making the spec less flexible & forcing
everyone to incur that cost (which is not true for unsorted, possibly
non-unique TLVs).

Even if that weren't true, the costs for detecting non-uniqueness over
unsorted TLVs if required by some protocol only seem to be an issue
for the most severely limited devices.  Based on my experiences
developing for a number of PDA platforms, I would not be overly
concerned about incurring that cost in return for a simpler, more
flexible spec.  Uniqueness and/or sorting also could, and maybe
should, be left up to the system or client protocols, system-wide or
child-protocol specification amenable to compile-time
inclusion/exclusion, rather than mandated on everyone.

Thx!

-- 
- joe kopena
  right here and now.

---

From: "Teco Boot" <[email-masked]>
Date: Wed, 24 Oct 2007 22:44:16 +0200

Thomas, thanks for the clarification.
Comments and questions inline.
Teco.

> Which "costs" only 16 octets. 
This is marketing specs. The compact representation needs additional fields
providing information for decoding.
Generally spoken, I agree that when large ranges of addresses are sent,
compressing helps a lot.

> 	192.168.1.40	STATE=UP			COST=4
I can't follow. I would come up with something very different:

	192.168.1.1		STATE=UP		COST=10
	192.168.1.10	STATE=UP/DOWN	COST=1
	192.168.1.20	STATE=UP/DOWN	COST=10
	192.168.1.30	STATE=DOWN		COST=1
	192.168.1.40	STATE=UP		COST=1

Can you provide all encoding to decoding to encoding info for this example?

I can't come up with an example where real problems are introduced. When
ambiguous info is being transported using packetbb, I do not expect packetbb
"repairing" the garbage.

Can you suggest how repeating information is transported? Assume for example
DOWN/UP timestamps:

192.168.1.1    STATE=UP    COST=10  DU=8.20-8.44 DU=9.00-9.06
192.168.1.10   STATE=DOWN  COST=1	
192.168.1.20   STATE=DOWN  COST=10  DU=9.30-9.45
192.168.1.30   STATE=DOWN  COST=1
192.168.1.40   STATE=UP    COST=1

Don't ask why sending this info. It is just an example.
(hint: think of forecasted outages).

---

From: Thomas Clausen <[email-masked]>
Date: Thu, 25 Oct 2007 00:09:44 +0200

Dear Teco,

See below...

> providing information for decoding.

Certainly, a small number of fields are required - however I did not  
plan on repeat the entire content of the I-D in this email ;)

<SNIP>

> Can you provide all encoding to decoding to encoding info for this  

You're right, my parsing was incorrect - copy-paste error, I  
apologise. I'm trying to collect these things in non-perishable  
format somewhere, you'll get a kind word in the acknowledgements for  
catching that ;) THANKS!

> ambiguous info is being transported using packetbb, I do not expect  

To use the terminology you propose ;) the point is not one of  
"repairing" garbage, but to be able to "detect" garbage in an  
efficient way -- i.e. to ensure that either parsing succeeds (and is  
unique) or it is ambiguous and therefore "garbage".

> Don't ask why sending this info.

OK, Teco, you should know me by now.....WHY?  ;)

I think I will answer by indicating that we already have a TLV type  
with repeating information in existence in form of this I-D:

	http://www.ietf.org/internet-drafts/draft-ietf-manet-timetlv-02.txt

Without referencing your specific example, I'd probably consider  
mechanisms as in the I-D above for repeated information.

You'll be in Vancouver, yes?

Cheers,

Thomas

---

From: "Charles E. Perkins" <[email-masked]>
Date: Wed, 31 Oct 2007 07:22:19 -0700

Hello Thomas,

Thanks for the long email.  I have some follow-up comments...

ext Thomas Clausen wrote:

<good discussion about address aggregation elided>
> mandated by packetbb, since it depends on the given deployment, the 

 Yes, noted and agreed.

> MANETs -- but which will not be discussed further in this email.

 I reckon this function would be better situated in the MAC
layer itself, but I understand this topic is not the main point of
your email.  However, I do think the solution as currently
specified is a layer violation and relatively difficult to put into
practice.  One might ask why aggregation is better for relatively
sparse control packet when it does not admit the more likely
gains from bundling with data traffic.

>     o no two TLVs of the same type may be associated with the same 
 
Up until this point in your note, there has not been a shred of
discussion to support the need for this.  Reading through the
rest of it, one still does not find any supportive discussion.

>     o the processing of entities (addresses) and attributes (that 
> which is associated to

 Noted.  Does not support ordering requirement.
>
> ... packetbb parsing comes before protocol processing, and protocol 
> processing is concerned only with the decoded data structure -- not 
> how this data structure happened to be encoded for transmission.

  Noted.  Does not support ordering requirement.

>
> <discussion about protocol example>
>
> What is important to extract from the above is that, concerning packetbb:
>

 Noted.  Does not support ordering requirement.

> <... example elided ...>
> of (i) nor all of (ii) -- leading to partial and incoherent use of the 
> information in the TLVs.
 This is because the type values in the TLVs you define in the example are
mutually exclusive.  A node may occupy only one of the STATEs in your TLV,
and it may only have a single cost [presumably the cost metric has already
been previously agreed upon, and so on...]
> still becomes exposed to incoherent information. The constraint of "no 
> two TLVs of the same type may be associated with the same address" 
> does solve this.
 Great!  Then you should enable this to be used for TLVs where it is needed,
without mandating it for all address TLVs.  For instance, as has been 
suggested
many times, some typed values for addresses would NOT have the uniqueness
requirement.  Some addresses may need to be described by multiple values 
from
the same type space, for some types of information.

>
> Now, I carefully mentioned that I would discuss this for 
> address-block-tlv-blocks, and then return to message-tlv-blocks and 
> packet-tlv-blocks later. As can be noted from the packetbb I-D, the 
> constraint of uniqueness is relaxed for message- and packet- 
> tlv-blocks, noting that:
>
> The reason that this is permissible in message- and packet- tlv-blocks 
> is, that tlvs in those tlv-blocks associate to one and only one 
> entity: the message or the packet. As such, while ambiguity still may 
> persist, the risk of "partial" or "incoherent" association (and, thus, 
> usage) of part of one and part of another tlv does not exist.

 As I understand it, this really destroys your point.  For one thing, 
consider
the follow sentence fragment, slightly reworded to refer to addresses:
> tlvs in those tlv-blocks associate to one and only one entity: the address

So, if I apply the same logic, any such tlv where the value is to be
associated with a single address, is a good candidate for allowing
a multiplicity of values.

Also note that up until now, no justification has been provided
for the ordering constraint.

But, in point of fact, the constraint for applicability to a single
address is quite arbitrary already and not necessary -- except,
of course, for TLVs where the values are from a domain where
uniqueness is sensible.

> resulting from parsing an incoming message.
These are characteristics that are _not_ naturally applicable to all 
types of
information that would be of interest for addresses.

> Keeping the TLVs within a TLV block ordered by TLV type means that 
> verifying (at both the sending side, prior to transmitting a message, 
> and on the receiving side during parsing but prior to presenting the 
> resulting data structure to the protocol) this uniqueness criteria can 
> be done in a single linear pass.
  ... fine, when uniqueness is desired, except of course unless the 
effort of
sorting typically outweighs the effort of parsing.  And, even so, it was 
never
shown that sorting on the sending side is "better" than sorting on the
receiving side!
> Having the TLVs unordered would imply backtracking and maintaining 
> additional data structures.
... in fact, very similar to sorting...  ... but all postulating that 
uniqueness is drastically important,
which is, of course, unsupported by the evidence provided except for 
selected boolean or
single-valued types.  And even the evidence shown does not imply the 
need for sorting.
>
> I want to emphasise that the order of inclusion of TLVs in a TLV block 
> is not indicative or suggestive of the order of processing of the TLVs 
> within that TLV block, just as the order of inclusion of addresses 
> within an address block (or address blocks) is not indicative or 
> suggestive of the order of processing of these addresses. Indeed, such 
> is, if processing order is important, to be specified by the protocol 
> using packetbb.

 Noted.  Does not support ordering requirement.
>
> An additional advantage of the ordering is, that if one is looking for 
> a specific TLV (e.g. a protocol or protocol extension where processing 
> the message is not worth it unless tlv type XXX is included) then one 
> can terminate parsing the message (or packet or address block) early, 
> i.e. once parsing encounters a tlv with type >XXX.

 While true, I have never seen this in practice (for valid TLVs).  In 
fact, if such were
the case, it would be a _lot_ smarter for the sender to just not send 
the superfluous data.
In fact, if I were writing the protocol, sending such superfluous data 
would be strongly
discouraged if not outright prohibited.  It is difficult to make a 
general statement for
this since there may indeed be cases where decorative/advisory TLVs are 
useful, but
usually, by the time the packet has been received, the major costs have 
been incurred
and scanning the TLVs is really in the noise.

> A clear example hereof would be "do not process this message unless a 
> signature TLV is included". It has been pointed out that for that 
> reason it would be helpful if a range of TLV types were allocated 
> which "comes before everything else", and that guidelines be provided 
> to IANA as to how types from that range are assigned -- and this point 
> has been updated in the I-D.

 That signature block ought to be the last TLV anyway.

>
> A new I-D (draft-ietf-manet-packetbb-10.txt) has been submitted to the 
> IETF, in which:
>
>     o a range of (the numerically first, in order to retain the 
> benefits of ordering)
  Basically, you still have not shown the benefits.

>        where it is important that these "come before anything else", 
> as indicated in the

 Such ordering can be imposed regardless of the numerical value of the
type.

Regards,
Charlie P.

---

From: "Dearlove, Christopher (UK)" <[email-masked]>
Date: Wed, 31 Oct 2007 15:49:52 -0000

>> these in one transmission is to reduce the MAC overhead

> I do think the solution as currently
> practice.

We do it in OLSRv1. It works, lots of people use it. OLSRv2,
and hence since we spun it out, packetbb, is using already
proven technology. This is self-evidently therefore not
difficult to put into practice.

As for the rest of your points, I'm not going to go through
it all again. It has been pointed out many times how ordering
permits one-pass easy verification of non-duplication.

********************************************************************
This email and any attachments are confidential to the intended
recipient and may also be privileged. If you are not the intended
recipient please delete it from your system and notify the sender.
You should not copy it or use it for any purpose nor disclose or
distribute its contents to any other person.
********************************************************************

---

From: "Charles E. Perkins" <[email-masked]>
Date: Wed, 31 Oct 2007 09:14:45 -0700

Hello Chris,

> difficult to put into practice.
The issue is whether or not one has protocol packets of
dissimilar varieties to collect together into the bundle.  This is
a matter of timing, or perhaps delaying some messages.

It is not a comment on the ability to write the code taking
the packets as input parameters.  I would be curious to
read about performance studies using realistic data traces,
comparing the bundling vs. non-bundling approach during
"normal" web traffic.  I note that the IPv6 working group
already rejected such an approach.  Perhaps you claim to
know better somehow.

I feel that your responses are, more than almost anyone
else's, quite dismissive of concerns that have been raised.
> it all again.
 I do not find this surprising.  It takes a long time.  It took me a long
time to compose the message.  It would take you a long time to
present any realistic and engaging response.  You have to decide
whether it is worth the effort.

>  It has been pointed out many times how ordering
 It has been pointed out many times that this feature is
not universally applicable, most likely not economical, and
certainly not inspired by current IETF practice.

Regards,
Charlie P.

---

From: "Dearlove, Christopher (UK)" <[email-masked]>
Date: Wed, 31 Oct 2007 17:43:10 -0000

> a matter of timing, or perhaps delaying some messages.

Which of course we do using jitter. There's a useful synergy
between the two, even though they are for different reasons.

> I would be curious to
> "normal" web traffic.

When being used for OLSRv2, one case that packetbb supports,
normal web traffic is a red herring. OLSRv2 is a proactive
protocol, what it does is independent of that traffic,
except insofar as the two get in each other's way. And
reducing the number of packets is a step in that direction.
And I have done simulations that compare the numbers of OLSR
packets with and without piggybacking, and found a significant
reduction in OLSR control packets, without increasing the
jitter delay that we want anyway.

> I note that the IPv6 working group

I just claim to know that what's appropriate to a wireless
ad hoc network may be slightly different from a general
IPv6 consideration.

> else's, quite dismissive of concerns that have been raised.

When the concerns are that something that's been done for
several years is "relatively difficult to put into practice"
and which has been in packetbb for years without a whisper
suddenly becomes an issue when we are trying to get the
document finished, a long over-due step, then yes, my
answer will be short, and may appear dismissive.

---

From: "Stan Ratliff (sratliff)" <[email-masked]>
Date: Wed, 31 Oct 2007 13:51:43 -0400

>-----Original Message-----
>From: Dearlove, Christopher (UK) 
>[mailto:[email-masked]] 
>Sent: Wednesday, October 31, 2007 11:50 AM
>To: Charles E. Perkins; ext Thomas Clausen
>Cc: Manet
>Subject: RE: [manet] Regarding packetbb constraints on 
>ordering and uniqueness.
>
[snip]
>permits one-pass easy verification of non-duplication.

Ditto on the "not going through it all again". Suffice it to say
that there are still dissenting opinions out here.

Regards,
Stan

---

From: "Charles E. Perkins" <[email-masked]>
Date: Wed, 31 Oct 2007 11:34:01 -0700

Hello Chris,

> jitter delay that we want anyway.

 To be clear, I am not lobbying for removal of the bundling
feature.  As you describe it, bundling is useful for reducing
the number of proactive packets periodically produced by
particular protocols.   Besides having lots of 'p's, anything that
reduces a significant fraction of the network-layer load on
the system is to be 'p'refered.

My intuition is more shaped by  the case where the
network signaling is less of a fraction of the total traffic.

> ad hoc network may be slightly different from a general
 
This is more debatable, but it is true that reducing network
load is probably not so urgent in wired networks.
 
> answer will be short, and may appear dismissive.

 Your discussion in this message was a lot more what I was looking
for until this last sentence.  Maybe a brief but responsive answer to
issues that have been raised would in the end be more economical
with your time and that of the working group, under the assumption
that people really want to understand and make progress.  There is
little or no reason to create a divisive atmosphere.

Regards,
Charlie P.