Subject: Re: [dtn-users] bundle fragmentation

From: Amy Alford <[email-masked]>
Date: Mon, 29 Apr 2013 13:38:20 -0400

I submitted a patch based on the discussion in this thread, but predicting
the fragment lengths really is a nightmare.  With a lot of effort, I do it
badly.  So, I was rethinking this again.  What if fragments were based off
of xmit_blocks but we actually then ran those blocks through
produce/consume to create recv_blocks?  Would that clear up all the issues
with bundle validity?  The downside is that this may create headaches with
BAB since we're going to try to validate the blocks we were planning to
send as though we just received them, and then we're going to try to add
BAB again.  I haven't thought through how to deal with those issues.

It feels like what we need is a way to call produce that produces
serialized fragments to send out a convergence layer.

On Tue, Feb 26, 2013 at 7:33 AM, Elwyn Davies <[email-masked]> wrote:

> Hi, Amy.
>
> On Mon, 2013-02-25 at 20:21 -0500, Amy Alford wrote:
> >
> > Why is it pointless to copy the xmit_blocks?  They certainly need
> > patching up, but I've been able to do that AFAICT.  It ends up
> > behaving as though you sent the original bundle on the link, then the
> > next hop fragmented it.
> Perhaps pointless was the wrong word. The problem is that the Bundle
> structure that is created for each fragment is not properly 'valid'.
> Things will probably work out as long as it remains on the link for
> which the xmit blocks were created, but things will a.s. go very pear
> shaped if the daemon is shutdown and restarted because the bundle, as
> stored in persistent storage, will have no blocks since the storage is
> driven from recv_blocks and api_blocks which remain empty.  The
> resulting bundle read back will be flagged as invalid because it has no
> primary or payload blocks - at best it will be flagged and discarded -
> but there is a good chance it will trigger an assert somewhere.  A
> similar problem will occur if a new link is opened along which the
> fragment bundle could be routed - reroute_all_blocks will be called and
> the fragments may end up on other links. An attempt to create
> xmit_blocks for this new link will also barf. The same will happen if
> the the link is closed down and/or restarted.
>
> > Whether that's what you want for BSP (as opposed to fragmenting then
> > adding BSP blocks), I don't know.
> Right.. that depends on what sort of blocks you were adding.  The
> hop-by-hop integrity checks might have to be different - I don't know
> whether BAB blocks for hop-by-hop would be verified at the next security
> destination before or after reassembly (or indeed whether fragmentation
> of such a bundle would be allowed).
>
> Generated metadata blocks could also be a problem although that is more
> of a theoretical problem at the moment because the generated metadata
> block capability is not really used at present (the BPQ stuff that I am
> working on is nearest to it AFAICS as it generates bundles in response
> to queries if it has a match in the bundle cache and these have metadata
> blocks).
> >
> > On Mon, Feb 25, 2013 at 12:02 PM, Elwyn Davies <[email-masked]>
> > wrote:
> >         Looking at the two create_fragment methods, the second one is
> >         essentially useless (that's the one with 'link' in its
> >         parameters - it
> >         should probably be deleted as it is pointless copying the xmit
> >         blocks as
> >         Amy discovered) and the first one needs some refinement to
> >         make it work
> >         properly. If we actually generated a new recv_blocks list (and
> >         possibly
> >         an api_blocks list - but that is probably not vital) the new
> >         fragments
> >         would get transmitted properly without needing to fudge
> >         queue_bundles
> >         and also without deleteing the original full size bundle (I
> >         believe).
> >         However I think recording the MTU size to which the bundle has
> >         been
> >         fragmented would help as we could avoid refragmenting the
> >         whole bundle
> >         if it got tested on the same  link again (as it might after a
> >         new link
> >         opened).
> >
>
>