Subject: [manet] Re: performance comparaison between Ad Hoc routing protocols

From: [email-masked]
Date: Thu, 7 Mar 2002 11:36:15 +0100 (CET)

Richard,

I suspect serious problems with your control overhead measures. Please see
below.

On Wed, 20 Feb 2002 [email-masked] wrote:

> 
> Thomas,
> 
> > Richard,
> > 
> > I am not quite sure what to make from your simulations. It seems to me
> > that they are from a network in overload condition - in which case buffer
> > management has a critical impact on performance. We've been simulating
> > OLSR in conditions, similar to those you've been showing, and our results
> > differ significantly. We experience that a large abmount of (data and
> > control) traffic in these situations are rejected from the various
> > buffers.
> 

<SNIP> 

> To ensure that the protocols were compared on a fair basis,
> since AODV and TBRPF gave control traffic higher priority than
> data traffic, we did the same for OLSR.  It has long been considered
> a good idea to give control traffic higher priority.  This could 
> explain why your results differ from mine.
> You can do this simply by adding the line
>                 case PT_OLSR:
> to the function PriQueue::recv() in priqueue.cc.
> 
> > Also, I notice that we obtain much less overhead for OLSR when counting
> > MAC/IP overhead. Have you alterated the ns2 code to make it output more
> > detailed information (e.g. physical overhead etc.) ?
> 
> No we did not.  In addition to the 20-byte IP header, ns2 adds 52 bytes 
> at the MAC layer.  
> 

<SNIP> 

I've tried to reproduce your published results for OLSR, using a large set
of scenario files, corresponding to the parameters you've used.

Unfortunately I fail to achieve the same results as those you present
(both with and without control traffic priority): in particulary in the
overload condition (packet rate of 6-8 packets/second) I find the overhead
of OLSR to be significantly lower (up to 25-30%) than what your
simulations show.

Also, as we were discussing in another mail, my simulations showed that
the "control packet size" bug resulted in approximately 50% more overhead,
whereas you cited your simulations as showing in the orders of 90-120%
more overhead. I have not had success in reproducing the 90-120% figures 
either.

I am basically looking for an explanation on why we have such different
results.

I am well aware that a paticulary "bad" or "good" choise of scenario (even
corresponding to the same general parameters such as mobility and traffic
patterns) may trigger particulary good or bad results, so I've run a
number of different scenarios through, all with roughly the same results.
This left me wondering if our different results were just a matter of
computing the overhead differently?

The way in which I extract overhead information from the ns2 tracefiles is
to look at all "send" events on the "MAC" layer from the agent
"OLSR". This should yield both the number of packets being transmitted as
well as the number of bytes, actually reaching the MAC layer and not
being dropped somewhere else.

However, when I tried to disregard the layer (MAC, RTR, AGT etc), and just
look at all the lines in the tracefile wish an "s" event from "OLSR", it
was possible to end up with overhead figures very close those that you
cite. I do believe, though, that this is a wrong way of accounting, since
(1) control packets, dropped in the protocol stack (and thus not
contributing to diffusion of topology information) are accounted as
protocol overhead and, (2) control packets are accounted multiple times
(each time a layer is sending the packet "downwards" in the stack.)

-- 

-------------------------------------------
  Thomas Heide Clausen
  Civilingeniør i Datateknik (cand.polyt)
  M.Sc in Computer Engineering

  E-Mail: [email-masked]
  WWW:    http://www.cs.auc.dk/~voop
-------------------------------------------

_______________________________________________
manet mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/manet

---

From: [email-masked]
Date: Fri, 08 Mar 2002 08:09:06 -0800

Thomas,

> (each time a layer is sending the packet "downwards" in the stack.)

We counted packets exactly the same way you did, i.e., 
only at the MAC layer.  

To double check, I did the following:

grep "s " out.tr | grep MAC | grep OLSR   

and got exactly the same number of OLSR packets
as our perl script did.

I did this for the following case:
100 nodes, 670x670 area, no mobility, 20 sources, packet rate = 6
For 500 seconds, there were 154744 OLSR packets totaling 20684584, 
so the average size of an OLSR packet (including the IP and MAC 
headers) is 133 bytes. 

I also would like to find out why our results differ.
If you have any more ideas, please let me know.

Regarding the 90-120% increase of overhead for the bug fix, 
that was for 100 nodes.  I later tried 50 nodes and found
the increase to be roughly 75-90%.

Richard

---

From: [email-masked]
Date: Fri, 08 Mar 2002 08:09:06 -0800

Thomas,

Richard

---

From: [email-masked]
Date: Fri, 08 Mar 2002 17:01:20 -0800

Thomas,

So that we can determine why our results differ, I will
provide additional details and scripts, so that you can run
the same experiment I ran.  I will give you the cbr 
and scenario scripts in a separate email.

I will refer to the case with 100 nodes, 20 sources, 670x670 area, 
no mobility, and a packet rate of 6 packets/second/source.  
Each stream begins at a random time between 10 and 40 seconds, and 
changes its destination to a randomly selected node every 30 seconds. 
The size of each data packet is 512 bytes. 
The total number of packets generated in 500 sec is 57261,
which is slightly less than 60000 = 6 * 20 * 500.
Is this similar to what you ran?

Richard

---

From: [email-masked]
Date: Fri, 08 Mar 2002 17:01:20 -0800

Thomas,

Richard