Subject: [Rmt] flute comments

From: [email-masked]
Date: Fri, 24 Oct 2003 13:53:04 +0300

Hi All

A few of us have been implementing flute and have performed some initial interoperability tests. We actually got three separate implementations working together in a single day, so I that means the FLUTE spec is splendid, but I'm writing to ask for improvements rather than to praise :) We spotted a few issues that would be helpful to deal with in the spec - clarifications to make a coder's life that bit easier.

All of these are about making the spec nicer and we think it can all be handled in in editorial changes after last call (i.e. if these delay last call, we'd be amazed).

1. A flag usage

ALC allows packets without TOI, which is useful for end of session (A flag) no payload packets. FLUTE does not allow this - just an oversight.

FLUTE-02 speficication says (Page 6, 3th paragraph): "* The TOI field MUST be used in ALC/LCT packets"

It can be changed to "* When sending an empty A flag packet the use of TOI is optional, otherwise the TOI field MUST be used in ALC/LCT packets"

It's a pretty triavial change, but would allow this ALC feature to be also usable for FLUTE.

2. Default SBN & ESI lengths

ALC gives an example with FEC ID 128 (SBN & ESI lengths are 4 bytes each). In our tests we made null fec - fec id 0 - the default, so 2 byte lengths in each. The specifications are correct, but two coders seperately overlooked this when adding FLUTE on top of ALC. It would be helpful to warn unwarey coders to take care with these lengths and check the specifics of the fec codes used - and if that doesn't prevent someone tripping up, they'll have to find out through the hex dumps :)

3. Expires time insufficiently described

Some improvements to the FDT Instance expiration time information (3.3 bullet 7 of ID_02) would be:
(1) provide a reference to the NTP spec
(2) NTP can be 64 bit, so you mean the most significant half - the other 32bits are for fractions of a second. This needs a mention "...value in seconds (note, NTP also offers an additional 32 bits to describe fractions of a second, these are not used for FLUTE)"
(3) wrap around will occur (2034?) so prehaps an advisory would be helpful like "Wrap-around of the 32 bit time is to be handled according to NTP [NTP]. For the purpose of identifying past and future times, it is recommended that 2^31 integers in advance of the current time are interpreted as future times and (2^31)-1 integers preceeding the current time are interpreted as past. For systems where sub-second synchronisation accuracy of clocks is not feasible, a system jitter of 24 hours (86400 seconds) may be used (as jitter tolerence) to avoid the situation where there is ambiguity whether the time is past or future. In this case some times would not be reliable: before (current_time - (2^32)-1 + (86400/2)); and after (current_time + 2^32-1 - (86400/2))."
(4) Sometimes you want to say "never expires". Adding this informational text would be helpful: "To factilitate something close to a 'never expires' option, a time long into the future, e.g. 2^30 ahead of the current time, may be used. As this represents many years into the future, this specification assumes that it is sufficient approxomation to 'never expires'."  Note, other alternatives come with problems that aren't worth solving (e.g. make the expires field optional, not allow 'never expires', have a special 'key word' instead of a number (zero would be a bad idea as it is a valid NTP time)).

So add [NTP], and add to the end of that bullet "(Note, NTP also offers an additional 32 bits to describe fractions of a second, these are not used for FLUTE). To factilitate something close to a 'never expires' option, a time long into the future, e.g. 2^30 ahead of the current time, may be used. As this represents many years into the future, this specification assumes that it is sufficient approxomation to 'never expires'. Wrap-around of the 32 bit time is to be handled according to NTP [NTP]. Wrap-around of the 32 bit time is to be handled according to NTP [NTP]. For the purpose of identifying past and future times, it is recommended that 2^31 integers in advance of the current time are interpreted as future times and (2^31)-1 integers preceeding the current time are interpreted as past. For systems where sub-second synchronisation accuracy of clocks is not feasible, a system jitter of 24 hours (86400 seconds) may be used (as jitter tolerence) to avoid the situation where there is ambiguity whether the time is past or future. In this case some times would not be reliable: before (current_time - (2^32)-1 + (86400/2)); and after (current_time + 2^32-1 - (86400/2))."

Note, this addition deals with rare and unusual problems so may seem overly detailed or complex compared with the simplicity of the problem. Maybe it would be added as an informational appendix instead?

4. Modifying a FDT Instance

There are some problems with allowing modified FDT Instances as opposed to "complementing". for complementing it, you are only adding (optional) info which does not inhibit (or confuse) reception. For modifying, you can do anything, e.g. change the MD5 checksum. There may be a use case for this but as far as I can see it is much simpler just to force a TOI change if reception-critical parameters change. The spec fix to this is in 3.3 bullet 6:
change "..., not to include it, complement the description or modify the description. In the last..." to "..., not to include it or complement the description."

If someone really needs to keep the same TOI, some limitations related to modifying the description could be made instead, but we have all sorts of exceptions about updating all receivers, not sending packets from two fifferent files on the same TOI, etc. The above proposal is definately the simplest.

5. Fixing source block lengths

We are not absolutely sure if a change is needed here, because we haven't done enough implementation and testing to work out the optomisation needs. In anycase here are some observations.

How necessary are variable length source blocks? I know this came up once before so I missed some of the lgic (maybe someone just needs to explain this).

To enable on-the-fly one step reception/decoding/saving/ready_for_use, it is essential for a receiver to know the source block lengths so it can decode and save to the right part of storage. Otherwise it is forced to store individual source blocks and then afterwards sort/defrag/copy to storage. Enabling the one step option cuts down on memory and time (and maybe total processing cycles). We haven't stress tested the implementations yet, but I feel this may be important for battery powered devices - espcially for wireless/mobile reception. Fixing this requires more stable source block lengths.

Fixing lengths gives 3 options:
a. all source blocks are the same. simple but requires stuffing in the last source block (wasted congestion) if the file is not an exact interger multiple of the source block size
b. all but the last source block is the same. Removes the stuffing but means that you ought to receive packets (encoding symbols) from as least two source blocks before you are ready for one-step-receive/store (incase the first synbols are from the last, shorter block).
c. all but the last 2 source blocks are the same and the last two are the same length as each other. Similar to the "all but the last" but you must get symbols from at least 3 blocks to be sure. On the other hand, it was suggested that keeping the minimum source block size no smaller than half the maximum is a good idea - but can someone explain (my head is spinning now :)

2 of the 3 implementations went for option b, and the other only used one souce block anyway, but we used null fec (id 0) so I may have missed something simple here. Even if no chnage is made, we can pick this up again in as an experimental RFC, which is the normal place from implemenation experience changes I guess.

Cheers, Rod.

_______________________________________________
Rmt mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rmt