Subject: [Rmt] FLUTE revision

From: "Luby, Michael" <[email-masked]>
Date: Mon, 24 Jan 2011 11:42:16 -0800

Dear RMTers, We are in the process of revising FLUTE –11, to produce FLUTE –12, and it is clear at this point that the FLUTE version number will transition from 1 to 2.  One question that has arisen: Can an FDT instance have a TOI > 0 with an EXT_FDT LCT extension header containing the FDT Instance ID?  I think it is and should be restricted to TOI = 0 for carrying FDT instances, but there is some ambiguities in the text here and there that might be construed to suggest otherwise.
Mike

---

From: <[email-masked]>
Date: Tue, 25 Jan 2011 09:38:33 +0100

FDT instances on TOIs other than 0 are allowed by design. There is no requirement for a minimal FLUTE-only client to parse them (there is no assurance that the client would parse all to TOI0 FDT instances either). However, FDTi's must follow the FDT rules regardless of TOI they come on (e.g. They use the same "TOI namespace" for the FLUTE session).  This allows more flexible extensions built on top of FLUTE. (The mmusic IMG work, that spawned the first FDALC draft, was to use this multi-TOI feature).

Disclaimer: apologies if I misunderstood the question as I didn't parse the bit on "with an EXT_FDT LCT extension header containing the FDT Instance ID" as it would require re-digesting the spec before replying. If the question is limited to this case with that header, are there any positive or negative implications of explicitly allowing or disallowing this?

Cheers, Rod.

On 24 Jan 2011, at 21:42, "ext Luby, Michael" <[email-masked]<mailto:[email-masked]>> wrote:

Dear RMTers, We are in the process of revising FLUTE –11, to produce FLUTE –12, and it is clear at this point that the FLUTE version number will transition from 1 to 2.  One question that has arisen: Can an FDT instance have a TOI > 0 with an EXT_FDT LCT extension header containing the FDT Instance ID?  I think it is and should be restricted to TOI = 0 for carrying FDT instances, but there is some ambiguities in the text here and there that might be construed to suggest otherwise.
Mike
_______________________________________________
Rmt mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/rmt

---

From: Jani Peltotalo <[email-masked]>
Date: Tue, 25 Jan 2011 10:51:07 +0200

Dear RMTers,

One side effect of the version number change is problem with ALC and LCT
versions. Version number is signalled in the LCT header field and in
RFCs 5775 and 5651 (FLUTE revised will be working on top of these RFCs)
version number is specified to be 1.

So it is not possible to signal FLUTE version 2 and ALC/LCT version 1 in
the same header field.

BR,
Jani

> Dear RMTers, We are in the process of revising FLUTE –11, to produce
> FLUTE –12, and it is clear at this point that the FLUTE version number
> will transition from 1 to 2.  One question that has arisen: Can an FDT
> instance have a TOI > 0 with an EXT_FDT LCT extension header containing
> the FDT Instance ID?  I think it is and should be restricted to TOI = 0
> for carrying FDT instances, but there is some ambiguities in the text
> here and there that might be construed to suggest otherwise.
> Mike  
> 
> 
> 
> [email-masked]

---

From: "Gillies, Don" <[email-masked]>
Date: Tue, 25 Jan 2011 19:32:43 +0000

Rod,

In reading the specification, I found in two or three places, text that suggested that FDT TOI = 0 and FDT TOI > 0 were both possible, as long as both files were delivered using FDT Instance in the EXT_FDT header of LCT.

So, I agree with you that this is how the present spec reads, TOI > 0 is possible for FDTs.

Some questions I had, however, were :
  - would there be a separate FDT Database for each TOI, or
    would all the different TOIs share the same FDT Database?
  - is it required at all to use TOI = 0 before using TOI > 0 ?

Do you think we should answeri these questions in the FLUTE spec?

Thanks,

- Don Gillies
Qualcomm Incorporated

From: [email-masked] [mailto:[email-masked]]
Sent: Tuesday, January 25, 2011 12:39 AM
To: Luby, Michael
Cc: [email-masked]; Gillies, Don; Luby, Michael
Subject: Re: [Rmt] FLUTE revision

Cheers, Rod.

---

From: "Luby, Michael" <[email-masked]>
Date: Tue, 25 Jan 2011 16:43:08 -0800

Hi Jani,
I'm missing where the version of FLUTE must be the same as the version of
ALC and LCT. It seems that if we define this specification to be FLUTE
version 2, and this FLUTE specification explicitly says that it must be used
with ALC version 1 as specified in RFC 5775 and that it must be used with
LCT version 1 as specified in RFC 5651 then there is no ambiguity, but maybe
I'm missing something?  I think what you are saying is that there is an
overloaded field that carries both the FLUTE version and the ALC version and
the LCT version?  If so, can you point to that?
Mike

On 1/25/11 12:51 AM, "Jani. Fi" <[email-masked]> wrote:

>> Dear RMTers, We are in the process of revising FLUTE 11, to produce
>> FLUTE 12, and it is clear at this point that the FLUTE version number

---

From: Jani Peltotalo <[email-masked]>
Date: Wed, 26 Jan 2011 09:02:57 +0200

Hi Mike,

Of course we can/must specify in the text as you mention below. However
in the packet only place to signal the version number is LCT version
number (V) field:

LCT version number (V): 4 bits

Indicates the LCT version number.  The LCT version number for this
specification is 1.

In RFC 5775 it is then mentioned:

"The version number of ALC specified in this document is 1. The version
number field of the LCT header MUST be interpreted as the ALC version
number field. This version of ALC implicitly makes use of version 1 of
the LCT building block defined in [RFC 5651]."

This kind of text is missing from the experimental FLUTE RFC and from
the revised FLUTE, but if we want to overload this field with FLUTE
version number (as we have done in our implementation based in
experimental RFC) to enable receivers to distinguish FLUTE versions 1
and 2, the packet is not anymore align with the ALC and LCT RFCs.

It is also possible to leave this field for ALC/LCT version numbers and
use for example reserved bits or LCT header extension to signal FLUTE
version. All in all, in my opinion it is necessary to signal FLUTE
version 2 somewhere in the FLUTE packet to be backwards compatible with
the receivers based on the experimental FLUTE RFC.

BR,
Jani

---

From: "Gillies, Don" <[email-masked]>
Date: Wed, 26 Jan 2011 07:31:37 +0000

I had one more question about FLUTE FDTs after reading the r12 FLUTE spec.

Is it required that the FDT have a table entry for the FDT itself (i.e. should the
FDT be self-describing?)  So for a typical flute session does TOI=0 have to appear
in the table of attributes?

I could not tell if the FDT had to reference itself.  Since only TOI and URI are 
required for a file in the FDT, it seems useful to me to have the URI in case I
wanted to fetch the latest version of an FDT via unicast.

- Don Gillies
Qualcomm Incorporated

---

From: "Luby, Michael" <[email-masked]>
Date: Wed, 26 Jan 2011 00:33:27 -0800

Thanks Jani for bringing this to our attention. We will take a look at this.
The easiest solution off the top of my head is to specify in the FLUTE draft
that the LCT version number in the LCT header MUST be interpreted as the
FLUTE version number field (just like what ALC does when it uses LCT).
However, we'll look at it more closely.
Best, Mike

---

From: "Luby, Michael" <[email-masked]>
Date: Wed, 26 Jan 2011 00:44:30 -0800

Actually, looking at the FLUTE doc, it seems that the only place where the
FLUTE version number is carried is in the FDT Instance header EXT_FDT, which
is an ALC PI specific LCT header extension, i.e., in EXT_FDT there is a
FLUTE version number that MUST be set to 2.  If this is correct, then it
seems fine that the version number carried in the LCT header is the ALC
version number, which is 1.  Comments on this?

---

From: Jani Peltotalo <[email-masked]>
Date: Wed, 26 Jan 2011 11:20:23 +0200

Hi Mike&all,

Yes, this is one possibility to signal the FLUTE version for whole FLUTE
session. It still have negative influence on FLUTE version 1 receivers,
which might buffer quite a lot unnecessary data, if the FDT Instance is
not received early enough.

BR,
Jani

---

From: Vincent Roca <[email-masked]>
Date: Wed, 26 Jan 2011 12:11:21 +0100

Hello everybody,

I have two main comments:

1- I am extremely surprised to see people say that
TOIs > 0 is possible for FDTs, as if it was obvious.
That's the contrary IMHO. For instance section 3.3 of
draft-ietf-rmt-flute-revised-11 says:

    *  The TOI value of '0' MUST be reserved for delivery of FDT
       Instances.  The use of other TOI values for FDT Instances is
       outside the scope of this specification.

There are other sentences that only refer to value 0.
For instance, beginning of Section 3.4:
      "FDT Instances are carried in ALC packets with TOI = 0..."
and Figure 1 explicitely mentions TOI = 0.

I understand that saying that "TOI 0 MUST be for FDT" is
not the same as saying "FDT MUST use TOI 0". However
the current specification does not explicitly say that other
values must be considered. It only says that is is "out of
the scope of this specification" which is different.

Said differently, even if I'm co-author and FLUTE/ALC
implementer, I never ever realized that the definite test
to determine if an incoming packet contains an FDT
Instance is the presence of the EXT_FDT rather than
TOI=0.

So I see a high risk of non-inter operable FLUTE
implementations here if we don't clarify in the text the
direction we want to choose.

BTW, clarifying this point is also a good incentive to move
to a version 2!

2- Let's assume we accept TOI > 0 for FDT Instances.
If **all** FDT Instances are sent on TOI > 0, a FLUTEv1
receiver that drops such packets (high probability) will never
consider any EXT_FDT. Since the FLUTE version number is
  carried in EXT_FDT only, this receiver has no way to
identify that it is a FLUTEv2 session. He will wait forever
an FDT Instance on TOI 0.

So we need to mandate that, from time to time, a packet
containing an EXT_FDT be sent on TOI 0.
I'm not sure, but it's may be possible to send an empty
FDT Instance to that purpose (e.g. if the senders wants to
avoid having to send the same FDT Instance on multiple
TOIs) (TBC).

Cheers,

    Vincent

>> On 1/26/11 12:33 AM, "Luby, Michael"<[email-masked]>  wrote:
>>
>>> On 1/25/11 11:02 PM, "Jani. Fi"<[email-masked]>  wrote:
>>>
>>>> Hi Mike,
>>>>
>>>>> On 1/25/11 12:51 AM, "Jani. Fi"<[email-masked]>  wrote:
>>>>>
>>>>>> Dear RMTers,
>>>>>>
>>>>>>> instance have a TOI>  0 with an EXT_FDT LCT extension header containing

---

From: <[email-masked]>
Date: Wed, 26 Jan 2011 14:31:05 +0100

Hi Vincent et al.

The discussion on multi-TOI FDT use is so historic that my memory is patchy. It was about the same time as the discussion that, unlike ALC, FLUTE session signalling doesn't have to indicate that it contains an object because it will always contain at least one FDT instance on TOI0. Also about the same time as the first discussion that an IANA mimetype for the FDT instances would be a good idea so FDT parsing had an easy way to detect non-TOI0 FDTs. Why IANA registration was not included in the existing FLUTE RFC is beyond my recall right now. (I remember we became preoccupied with security corner cases to ensure RFC Editor success, so maybe we were distracted?)

The current spec text "beyond the scope" is a pseudonym for "clients are not required to parse" to provide minimum interoperability, much the same as the allowing extensions to the FDT XML without spec'ing them.

Of course, there is nuanced sementic difference between "an FDT instance on TOI!=0 contributing, possibly redundantly, to the session control information" and "a file of FDT mimetype on TOI!=0 relevant to some 'out of scope' higher layer application but not contributing to the control information of the current session". And this could cause an IOP issue (although an FDT attribute addition would fix that without breaking IOP). Clarifying text on this wouldn't harm providing we haven't inadvertently sent implementers in two different directions already.

On the subject of version number, I am still very confused. No one has yet offered a compelling reason for changing it. The headers don't break things (at least the same version LCT/ALC changes are rolled into FLUTE and obviously shouldn't prompt a flute version increment either). The reason to revise the FDT schema hasn't yet been explained. SDOs using the existing flute will be hurt (or possibly the extra effort performed in the IETF will be rendered insignificant/obsolete where the deploying community consensus rejects the change). So why?

BTW, the revised FDT schema modifies effective functionality in two ways: it allows multiple FDT Instances per FDT Instance (I assume this is an error); it allows extensibility by elements as well as by attributes (sounds like a "nice to have" but the gain is considerably less than the SDO disharmony pain). (Incidentally, both schema allow from zero file elements per FDT instance, so "empty FDT Instances" are permitted).

Cheers, Rod.

PS I thought our primary goal with the RMT standards track revisions was to preserve interoperability and avoid trashing the considerable amount of work that went into aligning so many SDOs behind a common IETF approach, so interoperability preservation ought to be our default approach to spec text modifications.

On 26/01/2011 13:11, ext Vincent Roca wrote:

Hello everybody,

Cheers,

    Vincent

>> On 1/26/11 12:33 AM, "Luby, Michael"<[email-masked]><mailto:[email-masked]>  wrote:
>>
>>> On 1/25/11 11:02 PM, "Jani. Fi"<[email-masked]><mailto:[email-masked]>  wrote:
>>>
>>>> Hi Mike,
>>>>
>>>>> On 1/25/11 12:51 AM, "Jani. Fi"<[email-masked]><mailto:[email-masked]>  wrote:
>>>>>
>>>>>> Dear RMTers,
>>>>>>

---

From: "Luby, Michael" <[email-masked]>
Date: Wed, 26 Jan 2011 09:04:17 -0800

Hi Jani,

I guess you are thinking that the FLUTE  version 1 receivers might buffer
data for files and then when they get the FDT that is for the new FLUTE spec
the receivers realize that they don't support FLUTE version 2 and have to
discard all of this data because they don't know what to do with it?  If
that is the concern, there is advice in the FLUTE specification that
recommends that the FDT instances should be sent before the actual data for
the file, and that FDTs should be sent with higher reliability than the file
data. There is stronger and clearer wording on this in the current FLUTE
draft than in RFC 3926, but there is wording on this in the RFC 3926 as
well, e.g.,"It is RECOMMENDED that FDT Instance that contains the file
description entry for a file is sent prior to the sending of the described
file within a file delivery session."

Maybe I got this wrong?
Best, Mike

---

From: Jani Peltotalo <[email-masked]>
Date: Thu, 27 Jan 2011 09:03:30 +0200

Hi Mike,

You are right if there are no late-joiners and/or the first FDT Instance
is not lost.

In the revised-11 draft there is indeed better wording:

"It is RECOMMENDED that an FDT Instance that contains the file
description entry for a file is sent prior to the sending of the
described file within a file delivery session. This recommendation is
intended to minimize the amount of file data which may be received by
receivers in advance of the FDT Instance containing the entry for a file
(such data must either be speculatively buffered or discarded).  Note
that this possibility cannot be completely eliminated since the first
transmission of FDT data may be lost."

However, receiver will buffer at most that data which will be sent
between two FDT Instances and FDT Instance interval is of course an
implementation level issue. On the other hand, if the version number is
in every packet, FLUTE version 1 receiver will not buffer FLUTE version
2 packets at all.

BR,
Jani