Subject: Re: [iccrg] IKR SimLib-QEMU: TCP Simulations Integrating Virtual Machines follow up discussion mic

From: David Wagner <[email-masked]>
Date: Wed, 7 Aug 2013 13:35:23 +0200

Hi Damien, *,

I had a closer look on DCE. 
I appreciate the DCE approach, I think it's a big step forward from NSC and is very useful for many use cases. Nevertheless it is a completely different concept than our IKR SimLib-QEMU approach. 
I think the central issue ist that DCE still depends on the structure of the kernel code. While IKR SimLib-QEMU can use any old or future kernel that runs on x86 hardware and offers the same user space API, i.e. sockets, without _any_ modifications to neither the simulation framework nor the kernel. 

This dependence of DCE is very well documented: 
  http://www.nsnam.org/~thehajime/ns-3-dce-doc/dce-kernel.html#net-next-2-6-to-linux-stable-3-4-5
In order to use a newer kernel (3.4.5) the user needed to 
1. change includes in an existing header file
2. add a header file
3. change in sim method 
4. change in the DCE makefile and 
5. change in kernel source (!, but only concerning slab)
6. faking newly existing kernel function generic_file_aio_read

In the end, I end up with the simulation environment depending on the used kernel, the one for Linux 2.6 not working for 3.4.5 and vice versa. 
In this case, good luck, no central networking was concerned, but nevertheless using this newer kernel required deep understanding of the kernel (!) and the simulation framework (!). I do see that this documents a hard case, covering a distance of many kernel versions. Nevertheless, any new kernel version can contain basic changes in the code that requires changes to DCE and / or the kernel itself. 

Just an example to showcase the difference to IKR SimLib-QEMU: 
Assume I'd want a student to compare the exact behavior of the cwnd for default Linux congestion control for the last 30 kernel versions. 
With IKR SimLib-QEMU, he just calls `make` for these 30 kernels, replaces one line of the simulation parameters identifying the kernel image, and just runs these 20 simulation runs. So replacing one kernel by any other, there is not one line of code necessary (because the user space API, sockets etc, is stable in contrast to the kernel internals). 
With DCE cradle, I could run into some of the described issues at any step, requiring kernel and dce knowledge to build a  simuation environment specific to that kernel. 

And another example: if we at some would like to support Windows protocol stacks, we just need to write a windows user space program relaying the socket calls etc. But then we could include Windows stacks, too, without access to any source files. 
For our approach, we only need a known user space API. 

I hope this clarifies the fundamental difference between the two concepts and I'd be happy to discuss further. 

David

On Wednesday 31 July 2013 16:47:38 Damien Saucez wrote:
> Mirja,
> 
> let's continue the discussion we started at the mic...
> 
> So it seems that there are problems with ns3 and DCE to run
> experiments with new versions of the linux kernel. Could
> you discuss with the ns3 people the problem and see how to
> make all this work fine? It would have been very constructive
> to fix ns3/DCE as there is a large user basis. Don't hesitate 
> also to present your work to ns3 people because it is very
> similar, in the concept at least, to DCE.
> 
> Thank you,
> 
> Damien Saucez
> _______________________________________________
> iccrg mailing list
> [email-masked]
> https://www.irtf.org/mailman/listinfo/iccrg
>

---

From: Damien Saucez <[email-masked]>
Date: Wed, 7 Aug 2013 14:00:58 +0200

David,

> I hope this clarifies the fundamental difference between the two concepts and I'd be happy to discuss further. 

Yes, it makes it clear, thank you.

I just re-read the slides and it seems very straightforward with your solution to distribute
the simulation/emulation on several machines, have you ever tried?

Thanks again for the time spent answering,

Damien Saucez

---

From: David Wagner <[email-masked]>
Date: Wed, 7 Aug 2013 14:53:09 +0200

Hi Damien, 

> the simulation/emulation on several machines, have you ever tried?

Unfortunately that's not so easy: any time a VM executes kernel code, this (unknown) code may set a timer sooner than any other event in the simulation. This event could change the the following events and therefore the simulation run in general. 
Nevertheless we are planning to head in that direction, basically taking advantage of the delay of the simulated network: if two stacks are "25ms apart from each other", this can be used as guard interval for parallelization (not implemented yet). 

Thanks for your comments, I think it's important to exchange ideas and have discussions like this also on the tools we use for our research.

David 

-- 
Dipl.-Inf. David Wagner
Institute of Communication Networks and Computer Engineering (IKR)
University of Stuttgart
Pfaffenwaldring 47, D-70569 Stuttgart, Germany

web: www.ikr.uni-stuttgart.de   email: [email-masked]
phone: +49 711 685-67965        fax: +49 711 685-57965
-------------------------------------------------------------------