Subject: [dtn-users] 1 Hour Data Flow Test Results - DTN2.8.0 Changeset 3550 plus StorageThread patch

From: "Zoller, David A. (MSFC-EO60)[HOSC SERVICES CONTRACT]" <[email-masked]>
Date: Tue, 9 Oct 2012 12:51:27 -0500

Greetings again,
My next step was to consolidate most of the database updating to a Storage Thread. I left the GlobalStorage class as-is so that all changes to those values get written to the database immediately. All of the database updates to the bundles, registrations and links were changed to post an event to the storage thread which then periodically applied the accumulated updates en masse. The database update interval was set to 1 millisecond for this test. This pushes most of the delay due to updating the database to the storage thread hopefully leaving all the other threads screaming along.

In this case the BunRcvPost count is the number of bundles that were processed by the input thread and queued to the Storage Thread to be written to the database in 1 second. No guarantee that they have all actually been inserted which brings up a point to consider about this modification. A custody signal is generated before the bundle is officially in the database so it might be better to have the completion of the add to the database trigger the custody signal.

Best regards,
DZ

David Zoller
COLSA Corporation
Marshall Space Flight Center

DTN2 - One Hour Data Flow with the Receiver (Node2) Brought Online After 30 Minutes

Configuration

*        DTN2 changeset 3550 plus patches up to and including the StorageThread patch plus some statistics collection code.

*        OASYS changeset 2298 with all patches applied.

*        Computer (Node1)

o   HP xw4400 Workstation

o   Intel(R) Core(TM)2 CPU          6300  @ 1.86GHz   (dual core)

o   4GB memory

o   BerkeleyDB hosted on a Solid State Drive configured as an ext2 partition with the noatime and nodiratime options

*        Data flow is 160 non-custody bundles per second for 60 minutes

Bottom Line Results

*        576,000 UDP packets were transmitted (1316 byte payload)

*        549,390 packets were processed into bundles  (1316 byte payload)

*          26,610 packets were lost due to UDP socket buffer overflow

o   40MB socket buffer holds ~30k packets or ~187 seconds of data

*        Still not there and in these particular runs was a bit worse than the immediate db updates!!

o   Bundle ingest rate dropped to average 62 bundles/sec over 500 second interval

Timeline Highlights
Seconds after Node1 started

12

Data flow started at 160 packets/sec

12 - 1818

Successfully received 289,081 bundles at 160 bundles/sec

1819

Node2 makes contact (link-0 created) - reroute_all_bundles entered - ingest (BunRcvPost) continues at 160 bundles/sec but BunRcvProc halts

1834

reroute_all_bundles finished - elapsed 14 seconds

1834 - 1836

Burst processing posted BundleReceived events, ingest at 160 bundles/sec and transmit averages 534

1836

Node1 establishes connection to Node2 - reroute_all_bundles entered - ingest (BunRcvPost) continues at 160 bundles/sec but BunRcvProc and Transmission halts

1852

reroute_all_bundles finished - elapsed 15.6 seconds

1852 - 1859

Burst processing posted BundleReceived events, ingest at 160 bundles/sec and transmit averages 567

1860 - 2361

ingest avg=62 bundles/sec,  rcvd bundles routed avg=62 bundles/sec,   transmit avg=600 bundles/sec

2362 - 2410

ingest avg=596 bundles/sec,  transmit avg=37 bundles/sec

2411 - 3612

ingest avg=160 bundles/sec,  transmit avg=160 bundles/sec after initial choppiness and spike to 36495 1 second

3612

Data flow terminates

3612

Processing complete in 1 hour