Subject: [MMUSIC] Re: [AVT] new I-D on temporal URI addresses

From: Magnus Westerlund <[email-masked]>
Date: Thu, 06 Mar 2003 11:14:54 +0100

Hi, Silvia.

I see the possibility of this proposal. However I have a number of 
comments on this proposal.

First is the possibility to specify this as a general mechanism. It 
seems that you want to have this to apply to all URI schemes. Is that 
really possible? How do you know that the use of @ will not collide with 
any usage? For example the rtsp scheme does not define how to interpret 
the fragment. So this is up to the server responding to a request. So 
this can potentially already be used by some implementations. Also does 
it make sense to be defined for all schemes. As this is currently 
related to media it seems that for the moment it applies to HTTP(S) and 
RTSP(U|S).

So a general question, what is the actually possibilities to standardize 
such a solution?

Second, is the usage of the fragment identifier. It seems that you tries 
to redefine the meaning of the fragment identifier. See below.

 From RFC 2396:

  "When a URI reference is used to perform a retrieval action on the
   identified resource, the optional fragment identifier, separated from
   the URI by a crosshatch ("#") character, consists of additional
   reference information to be interpreted by the user agent after the
   retrieval action has been successfully completed.  As such, it is not
   part of a URI, but is often used in conjunction with a URI.

      fragment      = *uric

   The semantics of a fragment identifier is a property of the data
   resulting from a retrieval action, regardless of the type of URI used
   in the reference.  Therefore, the format and interpretation of
   fragment identifiers is dependent on the media type [RFC2046] of the
   retrieval result.  The character restrictions described in Section 2
   for URI also apply to the fragment in a URI-reference.  Individual
   media types may define additional restrictions or structure within
   the fragment for specifying different types of "partial views" that
   can be identified within that media type."

Two things strikes me as not being taken into account by your draft in 
relation to this quote. The first is from the first paragraph of the 
quote. The fragment identifier is interpreted by the user agent. While 
in chapter 4 of your draft you clearly specifies that the server shall 
interpret the fragment. This redefines the behavior of the fragment 
identifier.

Secondly the interpretation of the fragment identifier is in relation to 
the data resulting in the request. So in fact it seems that you need to 
define the behavior on per media basis.

I think the use of fragment to specify where in a media resource is 
appropriate in some case. For example an RTSP URL that has a fragment of 
the proposed type is useful. However as RTSP works the server shall not 
interpret the fragment as this is pointless. Only the client needs to do 
this and use that to determine the range to include in the PLAY request. 
So again the usage rules in chapter 4 seems to be wrong.

It seems that the time specification should be complemented to allow 
specifying ranges. The result of a search or the part of your friends 
wedding video that you want to point out would in most cases only be a 
part of the video. Therefore both a start and stop time seems useful. 
Then it is also a very short step to having multiple ranges allow one to 
point out multiple parts of the same resource that are interesting.

Also I think you have a few strange thing in the definition. First you 
specify an attribute "start" that I think are pointless. Either you 
totally skip the fragment or you can use "npt=0". Secondly it the 
defaulting to "npt". This clashed with the general parameters. If you 
use start then you have actually wrote "npt=start" which isn't defined. 
"now" works as long as this is based on the RTSP definitions of NPT, 
however from your BNF definitions this is evidently not the case. So my 
proposal is to skip having any default time scheme if you are going to 
have general attributes that does not belong to a time scheme.

If you would like the server to take the range into account would not 
the use of query (?) be better? However I hope that somebody that are 
more will versed in URIs can give further suggestions.

I can see the usefulness of this proposal, however there is need to 
resolve a number of issues:
- A general standardization or target certain URI schemes?
- Applicability on different media types and protocols needs to better 
understood.
- Use of fragment or other URI syntax element, like parameter or query?
- Need for having ranges?
- The actual need for general parameters?

Regards

Magnus Westerlund 

Multimedia Technologies, Ericsson Research ERA/TVA/A
----------------------------------------------------------------------
Ericsson AB                | Phone +46 8 4048287
Torshamsgatan 23           | Fax   +46 8 7575550
S-164 80 Stockholm, Sweden | mailto: [email-masked]

_______________________________________________
mmusic mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/mmusic

---

From: [email-masked]
Date: Thu, 06 Mar 2003 13:16:13 +1100

Hi all,

we have submitted a proposal to specify temporal URI fragments on 
time-continuous Web resources (such as audio and video files/streams). I 
would like to ask for your brains on whether this is a problem that 
you've stumbled across, too, and whether you like the solution that we 
are proposing.

http://www.ietf.org/internet-drafts/draft-pfeiffer-temporal-fragments-00.txt

THE PROBLEM:
------------
We can see a need for the direct specification of points of interest in 
a URI for a/v. There are several uses for it:

1) Representation of search results:
When typing a search term into your favourite search engine, you would 
like it to search on a/v and bring up the link to an offset in a video 
where the search term matches. e.g. goal in last week's soccer matches

2) Favourite links:
When sending a link to a video to a friend, you'd really like to be able 
to specify the main part of your interest in it. e.g. look at my new 
cool sunglasses in my friend's wedding video

THE SOLUTION:
-------------
What we want is to address fragment offsets into videos. Therefore, the 
"#fragment" part of a URI naturally lends itself for including this 
address specification. For specification of the time points, we were 
inspired by the timestamp specification in RTSP. Therefore, we are 
proposing the following pattern:

#@npt=14.5

-> the "#" part signifies a fragment specification
-> the "@" tells us to look "at" that time specification
-> the "npt=" part is a time scheme just like in RTSP and we are also 
proposing npt, smpte, smpte-30-drop, smpte-25, and clock
-> the time format itself depends on the time scheme, this one 
signifying a 14.5 seconds offset

[for more details refer to the I-D]

THE USE:
--------
Here are examples for temporal URI fragment offsets:

http://foo/bar.mov#@smpte=00:00:14:15
(start downloading at 14 seconds and 15 frames into bar.mov)

rtsp://foo/bar.avi#@utc=20030308T143000.00Z
(start streaming at 8th March 2003 at half past 2pm)

http://foo/bar.mpg#@start
(start downloading from the start)

rtsp://foo/bar.rm#@now
(start streaming now)

---
We wanted to make it simple, intuitive and human readable and have 
therefore come up with this solution. Please let me know what you think.

Cheers,

Silvia.

---

From: [email-masked]
Date: Wed, 12 Mar 2003 14:06:04 +1100

Hi Magnus, all,

 > - The actual need for general parameters?

Great to hear you liked the idea. Many thanks for your elaborate answer 
and well thought through comments, which I am very happy to address.

1) General mechanism questions, applicability to different media types 
and protocols:

> really possible? 

The question that you are addressing goes beyond whether our mechanism 
applies to all URI schemes. In 
http://www.w3.org/DesignIssues/Fragment.html Tim Berners-Lee states as 
an axiom that "The fragment ID spec for a new MIME type should be part 
of the MIME type registration process". This implies that the meaning of 
a fragment specification depends on the MIME type of the addressed 
resource. Therefore, the interpretation of a fragment has to be 
specified for each MIME type, and then probably again for each MIME type 
on each different scheme. Our I-D can only propose a generic way to 
specify time-offsets for time-continuous data and we are proposing it 
for http and rtsp where they make sense.

As an example take QuickTime files. Apple registered the MIME type 
video/quicktime but the documentation does not have a specification of 
how to interpret fragment parts when the files are used in URIs. Our I-D 
would not automatically enable timed fragment offsets to be used with 
QuickTime files - it needs an active move from Apple to change their 
MIME type specification. The same holds true for RealMedia and 
WindowsMedia and any other time-continuous MIME type. In fact, I believe 
that no MIME type registration except for text/html includes a fragment 
interpretation section.

 > How do you know that the use of @ will not collide with
> any usage?

The "@" sign is a reserved character in the URI Generic Syntax 
(RFC2396). It is allowed as part of a fragment, but its use is reserved 
for special cases. No special case of the use of "@" is know to us or 
published with IETF and therefore we asume there will be no collisions.

> For example the rtsp scheme does not define how to interpret 
> this can potentially already be used by some implementations.

As this has never been suggested before, there won't be any existing 
implementations that use it. Streaming servers should however find it 
easy, to support this mechanism as they only have to copy the 
specification after the "@" sign into the appropriate rtsp timestamp field.

> So a general question, what is the actually possibilities to standardize 

It needs some minor adjustments in the URI standard. As that standard is 
currently being changed/adapted, this could be the right time to get it 
adjusted. Then our mechanism can be provided as a Generic Syntax 
proposal and MIME type specifications for time-continuous data can elect 
to support it. That's my take.

> interpret the fragment. This redefines the behavior of the fragment 

We are aware that with the current URI specification, fragments only get 
interpreted on the user agent side. One way to overcome this problem is 
to use the Query part of the URI ("?") instead, but this creates other 
problems, iin the least overloading the meaning of "querying". 
Inherently, a temporal offset is a specification of a "fragment" in its 
core meaning and therefore we would prefer to use the "fragment" part. 
It makes everything much simpler. If there is no burning need to 
interpret the fragment identifier at the user agent, a change of the 
standard may be adequate. As the URI standard is currently reworked, we 
may get a chance to just do that (see URIBOF at next IETF meeting).

> define the behavior on per media basis.

Agreed.

> So again the usage rules in chapter 4 seems to be wrong.

Thanks. We'll take that into account when working over it.

> point out multiple parts of the same resource that are interesting.

As we've had several requests for that feature, we should probably work 
that as an option into the next version of the document.

> totally skip the fragment or you can use "npt=0". 

Possibly. But sometimes you have a file that does not start at 0 but 
rather with a time offset (because e.g. it is mirroring a timecode from 
an analog band from where it got digitized). In this case, "start" makes 
sense and does not default to "npt=0".

 > Secondly it the defaulting to "npt".
 > This clashed with the general parameters. If you

That's a matter of parsing sequence and probably adjustment of the BNF 
that we wrote in the document. The use of "npt" as default came from our 
experience with using the mechanism and the short form added much usability.

> have general attributes that does not belong to a time scheme.

I guess, a change of the BNF definition is more appropriate. Also, 
"start" and "now" seem to be useful for streaming and thus should be 
covered in URIs, too.

Best Regards,

Silvia.

---

From: Magnus Westerlund <[email-masked]>
Date: Wed, 12 Mar 2003 12:14:31 +0100

Hi Silvia,

First I have a question on how you are going to go forward in IETF. Are 
you planning to take this work into any WG or do a individual submission?

Then more comments below:

[email-masked] wrote:

> of the MIME type registration process". This implies that the meaning 
> of a fragment specification depends on the MIME type of the addressed 
> specified for each MIME type, and then probably again for each MIME 
> type on each different scheme. Our I-D can only propose a generic way 
> to specify time-offsets for time-continuous data and we are proposing 
> it for http and rtsp where they make sense.
>
> how to interpret fragment parts when the files are used in URIs. Our 
> I-D would not automatically enable timed fragment offsets to be used 
> with QuickTime files - it needs an active move from Apple to change 
> their MIME type specification. The same holds true for RealMedia and 
> WindowsMedia and any other time-continuous MIME type. In fact, I 
> believe that no MIME type registration except for text/html includes a 
> fragment interpretation section.

Thanks, for the explanation, it greatly clarifies this. However to my 
understanding the usage of MIME types is actually depending on the URI 
scheme. One simple example is to compare HTTP and RTSP. In HTTP you: 1) 
download the object specified by the URI. 2) Look at the MIME type of 
the downloaded object and sends it to its parser together with the 
fragment. 3) The parser takes the fragment into account when displaying 
the content.

However for RTSP URI with a fragment one possible interpretation is to 
setup the presentation indicated by the URI. This setup results in that 
a media description is retrieved. This description has of course a MIME 
type. Is it to this description the the fragment applies? Or is it to a 
the presentation in general which does not have a MIME type, or is it to 
the media streams which can contain one or more different encodings 
perhaps identified by MIME types.

My point with this example is that the schemes seems to be taken into 
account. For RTSP it seems in fact that it is time to start defining 
some fragment identifier behavior. 

> (RFC2396). It is allowed as part of a fragment, but its use is 
> reserved for special cases. No special case of the use of "@" is know 
> to us or published with IETF and therefore we assume there will be no 
> collisions.

Ok.

>
>> For example the rtsp scheme does not define how to interpret the 
>> fragment. So this is up to the server responding to a request. So 

I don't think we need to consider servers in this case, rather the 
clients as fragments should have been treated in accordance with the URI 
specification saying that fragment is a user agent problem.

>
>> So a general question, what is the actually possibilities to 
>> standardize such a solution?
>
>
> It needs some minor adjustments in the URI standard. As that standard 
> is currently being changed/adapted, this could be the right time to 
> get it adjusted. Then our mechanism can be provided as a Generic 
> Syntax proposal and MIME type specifications for time-continuous data 
> can elect to support it. That's my take.

I am not fully aware of what is happening with this. However I have seen 
a draft for making URI generic syntax to standard. Is that what you have 
in mind? To my knowledge the changes performed on a specification when 
going from draft to full standard are only clarifications. Technical 
changes are very limited if allowed at all.

I think that any such extension as you are proposing will be a proposed 
standard. 

>> Second, is the usage of the fragment identifier. It seems that you 
>> tries to redefine the meaning of the fragment identifier. See below.
>>
>>  From RFC 2396:
>>
>> Two things strikes me as not being taken into account by your draft 
>> in relation to this quote. The first is from the first paragraph of 
>> the quote. The fragment identifier is interpreted by the user agent. 
>> While in chapter 4 of your draft you clearly specifies that the 
>> server shall interpret the fragment. This redefines the behavior of 
>> the fragment identifier.
>
>
> We are aware that with the current URI specification, fragments only 
> get interpreted on the user agent side. One way to overcome this 
> problem is to use the Query part of the URI ("?") instead, but this 
> creates other problems, in the least overloading the meaning of 
> "querying". Inherently, a temporal offset is a specification of a 
> "fragment" in its core meaning and therefore we would prefer to use 
> the "fragment" part. It makes everything much simpler. If there is no 
> burning need to interpret the fragment identifier at the user agent, a 
> change of the standard may be adequate. As the URI standard is 
> currently reworked, we may get a chance to just do that (see URIBOF at 
> next IETF meeting).

For me it seems very bad step to redefine this. I think one should look 
at both use cases. There will still be plenty of reasons to use fragment 
on the client side. And as there is a large number of HTTP clients out 
there using the current definition it seems a bad move to redefine it.  
However one should also look into how to address this when one wants the 
server to perform the action. I can clearly understand the use case of 
having a server reduce large media files to only contain what one is 
interested of. 

I looked through the URIBOF agenda: 
http://www.ietf.org/ietf/03mar/uribof.txt . I could however not see that 
this issue would be part of the BOF. Are you planning to take up your 
proposal there?

Best Regards

Magnus Westerlund

---

From: [email-masked]
Date: Fri, 14 Mar 2003 11:08:00 +1100

Hi Magnus,

> you planning to take this work into any WG or do a individual submission?

I am most happy to take it into a WG but am uncertain which one that 
would be.

> fragment. 3) The parser takes the fragment into account when displaying 

Not necessarily. That's just the way current browsers work because of 
the recommendations for the use of the fragment. I know for a fact that 
the Apache Web server can receive, parse and handle fragment offsets.

> perhaps identified by MIME types.

That should be discussed, but my take is that it applies to each one of 
the media streams that are part of the general presentation.

> some fragment identifier behavior.

I agree, we definately need to take the schemes into account when 
describing the expected behaviour of servers and clients when 
encountering these temporal offsets in URIs.

> specification saying that fragment is a user agent problem.

It is important to consider servers, too. I does not make sense to 
burden the Internet or the user with a huge amount of video data when 
all that the user requires is a clip 20 minutes into the video. I am 
aware that the current use of fragments does not conform to this idea 
and therefore suggested to discuss the possibilities of changing that 
usage specification.

> changes are very limited if allowed at all.

As far as I understand it, there are more changes than just 
clarifications happening to RFC2396. But if you are right, we are 
probably forced to go with the query "?" solution rather than fragments, 
which makes it a lot less elegant. Why should we be using a different 
scheme for addressing offsets in audio/video than in html? Also, the use 
of query is already overloaded and therefore a sensible use of fragments 
for media would make sense.

> I think that any such extension as you are proposing will be a proposed 

I'd be more than happy to have it become that and through an IETF WG.

> on the client side.

Could you please tell me some? I tend to disagree. I believe that as 
time-continuous media are usually high data rates, fragment addressing 
of such data only makes sense if we can avoid having to serve out the 
complete file before we can address the offset. By the way, even with 
HTML I have often been upset about having to wait (especially on a modem 
line) for the complete document to download when all I wanted was to 
read a specific paragraph in it.

> And as there is a large number of HTTP clients out 

No. That would  not affect them. It is still possible to say that if the 
user (agent) wishes to interpret the fragment offset locally, he's free 
to strip off the fragment offset from the URI (as happening today) and 
not sending that to the server. However, if a server receives a fragment 
offset request, he requires to send data only from that offset onwards.

> this issue would be part of the BOF. Are you planning to take up your 

I only found out about the URIBOF agenda after having submitted the I-D. 
I have however sent the I-D to the relevant mailing lists and hope to 
get them to pick it up and discuss the issues. I cannot attend the 
meeting personally, unfortunately.

Best Regards,

Silvia.

---
Silvia Pfeiffer
Research Scientist
CSIRO Australia