Subject: [AVT] Update (-08) Timed Text draft

From: "Rey Jose" <[email-masked]>
Date: Thu, 16 Dec 2004 16:07:50 +0100

Hi Magnus, Colin, All,

I've just sent an update to the ID manager.  You can get it here until
is published
http://www.prdcg.panasonic.de/ietf/docs/draft-ietf-avt-rtp-3gpp-timed-te
xt-08.txt

Below are the comments and actions taken in the draft -08. The first
part is the issues pending (until the "Non-issues" mark); the second one
is the the non-issues; this part is just there for quick checking of
each comment. Not necessarily to be read, if one reads the draft anyway.

I have introduced many figures, 2 new sections on relation to RFC2793
and RFC3640. Also re-arranged into introduction, motivation, basics of
the format, requirements and limitations at the beginning.  I hope it is
easier to read now.

------
[CP]
In Section 3.1.2, SDUR has 3 bytes, since two bytes gives limited
maximum duration. This will be true no matter what: is it appropriate to
place a fixed maximum? Or to define a way in which event duration can be
extended by a following packet to allow unlimited duration events?
[JR]
"o	SDUR (24 bits) "Text Sample Duration": indicates the sample
duration in RTP timestamp units of the text sample.  For this field, a
length of 3 bytes is preferred to 2 bytes.  This is because, for a
typical clockrate of 1000 Hz, 16 bits would allow for a maximum duration
of just 65 seconds, which might be too short for some streams.  On the
other hand, 24 bits at 1000 Hz allow for a maximum duration of about 4.6
hours, while for 90 KHz, this value is about 3 minutes.  These values
should be enough for streaming applications.  However, if a larger
duration is needed, the extension mechanism specified in Section 4.2
SHALL be used."

An extension mechanism has been defined. Thereby a copy of the sample
with adjusted timestamp and duration is sent:
"I.e., a sample with timestamp TS and duration SDUR can be sent as two
samples having timestamps TS1 and TS2 and durations SDUR1 and SDUR2,
such that TS1=TS, TS2=TS1+SDUR1 and SDUR= SDUR1+SDUR2."

Comments?
--
[JR]
Also the draft states now explicitly that a limitation is not being able
to convey packet sizes larger than a 16 bits integer.  This is a design
choice and it is considered enough for streaming apps. There is no
extension mechanism defined for this reason. Comments?
--
[MW]
> 14. Section 8.2.2: Last bullet: The usage of "sver" in multicast? Is
it really possible to offer more 
> than one version that all uses? Especially if the version result in
that different parsing of the samples  > must be used, then one can 
>really only offer one. Otherwise one needs to agree on the version
used.

[JR]
I think so: as "sver" is defined currently if a newer extension to the
3GPP TS26245 is done which is not backwards-compatible with previous
versions or cannot be discarded, then this version will not be listed in
the "sver" param list (per definition). So this case cannot happen in
the first place. 

Thus, if alternative non-compatible versions are offered, then this
should be done using different payload types. Is this acceptable?

On the other hand, it is the question whether the "sver" may be
obviated. We say that parameters that are not understood by the SDP
consumer are ignored and the timed text format itself is
backwards-compatible.  then there's no need, correct? 

--
[JR]

I've been asked by some MPEG people whether it would be possible to
introduce a header that uses char counter to allow the number of chars
lost to be known. Obviously I have no time to do this.  But, could a new
header be added later? There are three types remaining "TYPE 0, 6 and 7
are reserved." Could the number be used in a later complementary draft?
---
[JR]

I've done some more emphasis on packet repetition as a means for
resilient transport, in particular on carousel data transmission in
section about resilient transport.  Since one of the main applications
will be for multicast and timed text has low bitrates, I think RTP
carousel should be a good trade-off, instead of choosing more complex
redundancy methods such as rfc2733 or reliable multicast.  Comments?
--
[MW]
11. Section 8.2.1, bullet 1, a, i: Should it also be added, that unless
special 
preference exist the offered value should be used?

[JR]

Hhmmm... a) if not used then the answerer MUST ensure that the
preference of the offerer are compatible with the values offered, how
makes this sense?? and b) what should be done if this is not so?? A
behaviour for error case shall be defined...no?

Accordingly, in the second bullet the second MUST changed to SHOULD.

This is what I wrote:

"
a.	"tx", "ty" and "layer": these are parameters describing where
the received text track is placed.  Depending on the directionality:
i.	MUST appear in all sendrecv offers and answers and in all
recvonly offers and answers(thus applying to the incoming stream).  In
the case of sendrecv offers and answers and in recvonly offers, these
values SHOULD be used by the sender of the stream unless it has a
particular preference, in which case, it MUST make sure that these
different values do not corrupt the presentation.  For recvonly answers,
the answerer MAY accept the proposed values for the incoming stream (in
a sendonly offer, see bullet below) or respond with different ones.  The
offerer MUST use the returned values.  

ii.	MAY appear in sendonly offers and MUST appear in sendonly
answers.  In sendonly offers they specify the values that the offerer
proposes for sending (see example in Section 9.3).  In sendonly answers
these values SHOULD be copied from the corresponding recvonly offer upon
accepting the stream, unless a particular preference by the receiver if
the stream exists, as explained in the previous bullet "
---

[MW]
7. Section 7.1: I think that a number of parameters are misclassified.
Any 
parameter that do not need to be included in all messages belongs under
Optional 
parameters. I think that include: max-w, max-h, width, and height. This
also has 
effect in 8.1.
[JR]
What means are "all" messages? Sendrecv or recvonly O & A? I thought
about all possibilities and created a new table:

+..---------------------------+----------+----------+----------+
|   ``--..__  Directionality/ | sendrecv | recvonly | sendonly |
+ Type of   ``--..__   O or A +----------+----------+----------+
|    Parameter      ``--..__  |   O/A    |   O/A    |   O/A    |
+--------------+------------``+----------+----------+----------+
| Declarative  |tx, ty, layer |   M/M    |   M/M    |   m/M    |
|              |              |          |          |          |
+--------------+--------------+----------+----------+----------+
| Display      |max-h, max-w  |   M/M    |   M/M    |   -/-    |
| Capabilities |              |          |          |          |
+--------------+--------------+----------+----------+----------+
| Stream       |height, width |   M/M    |   -/(M)  |   M/M    |
| properties   |tx3g          |   m/m    |   -/-    |   m/m    |
|              |              |          |          |          |
+--------------+--------------+----------+----------+----------+
|  Negotiable  |sver          |   M/M    |   M/M    |   M/M    |
|              |              |          |          |          |
+--------------+--------------+----------+----------+----------+
|  Symmetric   |rate          |   M/M    |   M/M    |   M/M    |
+--------------+--------------+----------+----------+----------+
Table 1. Parameter usage in Unicast Offer /Answer.

Key:
o M means MUST be present
o m means MAY be present (such as proposed values)
o (m) or (M) means MUST or MAY, when applicable
o means MUST NOT be present.
---

[MW]
15. Section 8.3:
"In this case the answerer's "max-h" and
   "max-w" are compatible with the offerer's "height" and "width".
   Otherwise, the answerer would have to remove this stream and the
   answerer issue a new offer taking the answerer's capabilities into
   account."

I would like to make you aware that the offerer will not receive a
payload type 
at all if the answerer has to refuse the stream due to that height and
width are 
larger then the supported max-w and max-h. The only way to ensure that
this 
would happened would be do define multiple payload types, with at least
one that 
is guaranteed to be acceptable, thus allowing the answerer to give his
max-w and 
max-h parameters. Then the offerer can re-issue the offerer with the
knowledge 
about the answerers capability.

[JR]
I added the following: 
"In this case the answerer's "max-h" and "max-w" are compatible with the
offerer's "height" and "width".  Otherwise, the answerer would have to
remove this stream and the offerer would have to issue a new offer
taking the answerer's capabilities into account.  This is possible only
if multiple payload types are present in the initial offer so that at
least one of them matches the answerer's capabilities as expressed by
"max-h" and "max-w" in the negative answer.  Note also that the
answerer's text box dimensions fit within the within the maximum values
signalled in the offer.  Finally, the answerer chooses to use version 60
of the timed text format."
---

----------
----------
Non-issues
----------
----------

[JR]
I've introduced a section (after Colin's comment) on relation to RFC3640
and RFC2793.  AS well as example payloads.
---
[CP]
"Section 2.3.3 (last bullet point on page 9) mentions that "it does not
seem reasonable to include such additional headers". What headers are
these? The text preceding talks about intelligent fragmentation, but
it's not clear that this needs extra headers."

Added:

Informative note: ensuring that partially received modifiers can be
applied to text strings in all cases (for all modifier types and for all
fragment loss constellations) would place additional requirements on the
payload format.  In particular this would require that: a) senders
understand the semantics of the modifier boxes (where does it field of
the modifier start and end, what the fields indicate,...) and b)
specific fragment headers for each of the modifier boxes are defined (in
addition to the payload formats defined below.)  This would be necessary
to apply each piece of the modifiers received independent of previous
packet losses.  This requires complex headers.  Given the low
probability of fragmentation and the desire to keep the requirements
low, it does not seem reasonable to specify such modifier box specific
headers.

[CP]
"In section 2.2, requirement 1 is to keep the 3GP text sample structure.

This is justified by stating that a receiver SHALL be able to rebuild
the text samples from the received RTP packets. I agree that rebuilding
the text samples is an appropriate goal, but don't see why this requires
the payload format to keep the 3GP text sample structure? We have other
payload formats that modify the data format to optimise media transport,
but allow reconstruction of the original data."

[JR]
It should rather be new reqs 1&2. It should rather be new reqs 1&2:

"1.	It SHALL enable both live streaming and streaming from a 3GP
file.  

..

2.	Furthermore, it SHALL be possible for an RTP receiver using this
payload format, and capable of storing in 3GP format, to obtain all
necessary information from the RTP packets for storing the received text
contents according to the 3GP file format.  This file MAY or MAY NOT be
the same as the original file."

[CP]
"In section 2.2, requirement 3 states that it may be more appropriate to
send large sample descriptions in-band. What is the justification for
this? In-band transmission would seem inappropriate for large
descriptions, since they're more likely to be affected by packet loss
than smaller descriptions that fit into a single packet."

[JR]
Fixed:
3.	It SHALL enable efficient and resilient transport of timed text
contents over RTP.  The following measures are taken:

a.	Enable the transmission of the sample descriptions both by
out-of-band and in-band means.  Sample descriptions are important
information, which potentially apply to several text samples.  These
default formatting settings are typically transmitted out-of-band
(reliably) once at the initialization phase.  If sample descriptions are
needed in the course of a session, these may be sent also out-of-band or
in-band.  In-band transmission, although unreliable, may be more
appropriate for sending sample descriptions if these should be sent
frequently, as opposed to establishing an additional communication
channel for SDP, for example.  It is also useful in cases where an
out-of-band channel may not be available and for live streaming, where
contents are not known a priori.  In order to cover this wide range of
scenarios, the payload format SHALL enable both in-band and out-of-band
transmission of sample descriptions.  Section 4.1.6 specifies a payload
header for transmitting sample descriptions in-band.  Section 9
specifies how sample descriptions are mapped to SDP.

[MW]

"18. Section 12.1: The algorithm description. I have some problem with
the current description. Bullet 3, has an IF without an clear argument.
Also I think it fails to properly describe if the new X must be the next
value or values above the current active set. Or if one is allowed to
send an X=127 in a session that had the inactive set to be "

[JR]
Better:   Let X be the SIDX of the last received sample description.
Let Y 
  be a value within the allowed range for dynamic SIDX: [0,127], and 
  different from X.

1.	Initialize all dynamic SIDX values as inactive.  For stored
content, read the sample description index in the Sample to Chunk box
("stsc") for that sample.  For live streaming, the first value MAY be
zero or any other value in the interval above.  The initial value is
SIDX=X.  Go to step 2.
2.	First in-band sample description with SIDX=X is received. Go to
step 3.
3.	Set SIDX=Y inactive if Y inside the (inclusive) interval [X+1
modulo(128), X+64 modulo(128)].  Otherwise, set SIDX=Y as active.  Go to
step 4.
4.	Wait for next sample description.  Upon reception of a sample
description with SIDX=X do: 
a.	If X is currently active, then wait for next SIDX (do nothing).

b.	Else go to step 3.

Informative note: note that it is allowed to send any value of SIDX=X in
the interval [0,127].  E.g. if [64..127] is the current active set and
65 is sent a new sample description is defined and an old one deleted
(64).  Similarly one could send X=127, thus inverting the active and
inactive sets.

[CP]
"Section 2.3.4 specifies how packets are formed from samples and
modifiers. The rules listed in the numbered list at the start of the
sections allow empty packets, but the preceding text specifies that at
least one unit of data must be included. Which is correct?"

-A small sentence has been added "At least one unit of either type MUST
be present.  "

[CP]
"The first observation in section 2.3.4 (near the end of page 10) states
that type 5 units may be placed anywhere in the aggregate, and SHALL NOT
be regarded for calculating the timestamp (indeed, they MUST be ignored
when calculating the timestamp). The numbered rules for aggregate
payload earlier in the section allow a packet that comprises only a type
5 unit. What is the RTP timestamp for such a packet, given that type 5
units MUST be ignored when calculating timestamps?"

[JR]
Yes, good point.

The text now is:

o	TYPE 5 units MAY be placed anywhere in the aggregate and they
SHALL NOT be regarded for calculating the timestamp of the subsequent
units.  This is because they usually do not belong to any text sample in
particular, but may apply to several.  For timestamp calculations, TYPE
5 units MUST simply be ignored, i.e. by jumping to the next unit.  For
setting the timestamp in packets containing TYPE 5 units refer to
Section 3, timestamp definition.

[CP][MW]
Section 3 again notes that TYPE 5 units do not make use of the
timestamp. All RTP packets need a valid timestamp; if only TYPE 5 units
are included, what value does it take?

Secondly, I think some clarification on when Type 5 packets shall be
assumed to become active. From the point they are received, when they
are unpacked, i.e. order in the RTP packet may have some meaning, or
when other packets type the type 5 was attached to is utilized. I think
this has some impact, although the guard band for dynamic types normally
would prevent it."

[JR]
Now:
"Note that TYPE 5 units do not make use of the timestamp and therefore
the above does not apply to them.  TYPE 5 units become active upon
reception and not at the time instant indicated by the timestamp.  In
RTP packets containing only TYPE 5 unit(s), the RTP timestamp SHALL be
set to the current value of RTP timestamp." 

[CP]
Section 3.1.2 has an "example that illustrates how units of unknown
duration MUST be presented". If there's a rule, it need to be defined
explicitly, not by a single example.
[JR]
Text samples have generally a known duration at the time of
transmission.  However, in some cases like live streaming, the time for
which a text piece shall be presented might not be known a priori.  For
this case, the value zero SDUR=0 (0x000000) is reserved to signal
unknown duration.  The amount of time that a sample of unknown duration
is presented is determined by the timestamp of the next sample that
shall be displayed at the receiver.  Text samples of unknown duration
SHALL be displayed until the next text sample becomes active, as
indicated by its timestamp.  Note that only text samples to be actually
displayed SHALL replace them, while discarded samples (e.g. because of
missing fragments) or sample descriptions alone SHALL NOT terminate
display of a sample of unknown duration.

[CP]
Section 3.1.2 (top of page 22) says that only TYPE 5 units can follow
units of unknown duration otherwise it's not possible to calculate the
timestamp. Which timestamp does that refer to? The RTP timestamp of the
packet or that (implied) for the following unit? Please clarify.
[JR]
Note that samples of unknown duration SHALL NOT use features, which
require knowledge of the duration of the sample up front.  Such features
are scrolling and karaoke in [1].  Furthermore, only sample descriptions
(TYPE 5) MAY follow units of unknown duration in the same aggregate
payload.  Otherwise, it would not be possible to calculate the timestamp
of these other units.  

[CP]
Section 3.1.2 (last two bullets) explains that the TLEN is needed by the
decoder to know where the modifiers in the payload start. It's not clear
what this means: does the text string include modifier boxes? I thought
TYPE 1 units transported text samples only, and that any modifiers would
be part of a TYPE 5 unit later?
[JR]
No, TYPE 1 units carry text samples (defined in Terminology). Text
samples carry text strings + zero or more modifiers.  TYPE 5 OTH carry
sample descriptions, which specify default settings that may be modified
by modifiers.  I added minimum changes to the text:
"o	Finally, the actual text sample follows the TLEN field.  As
defined in Section 3, a text sample consists of a string of characters
encoded using either UTF-8 or UTF-16, followed by zero or more
modifiers.  Note also, that no BOM and no byte count are included in the
strings carried in the payload (as opposed to text samples stored in 3GP
files [1])."

[CP]
Similar comment for section 3.1.3, where SLEN indicates the size of the
text string plus any modifier boxes. From the description earlier, it
can be read that there are no modifiers and only text here?
[JR]
I included also a reference to the terminology section.

[CP]
Section 3.1.3 states that LEN MUST be greater than 9. What is the
response to a packet where LEN <= 9? (similar comments apply for section
3.1.4, 3.1.5 and 3.1.6)
[JR]
Done. Units should be discarded.

[CP]
Section 3.1.3 what should a receiver do if it receives fragments with
differing SLEN values?
[JR]
Discard. Added:
"If a receiver receives fragments of the same text sample (identified by
the timestamp) with differing SLEN, these should be discarded.  Either
the SLEN is known at the moment of sending the packet or it is unknown
but it SHALL NOT change for different fragments."

[CP]
Sections 3.1.3, 3.1.4 and 3.1.5, what should a receiver do if it
receives units where THIS > TOTAL? Or if TOTAL = 0?
[JR]
Discard.Added:
"In all cases (TYPEs 2, 3 and 4), if THIS > TOTAL or if TOTAL = 0, the
fragment SHALL be discarded."

[CP]
Section 7.1 states that a receiver MUST ignore any "unspecified"  
parameter. Can you clarify what this means? Did you intend to write
"unrecognised"?
[JR]
Yes.

[CP]
Section 7.1 defines a number of parameters as "16 bit integers" or as
"16 bit unsigned integers". Does this mean that the parameters are in
binary format? Or do you intend (as in the examples in section 8) that
the decimal representation is used?
[JR]
Decimal representation.

[CP]
Section 8.1, last bullet point says that unknown parameters SHALL be
ignored. Does this mean that they're not copied into the SDP (i.e.  
ignored by the device generating the SDP)? Or are they copied, but
ignored by the device that later uses the SDP? Please clarify.
[JR]
Added:

"o	Any unknown parameter to the device that uses the SDP SHALL be
ignored.  E.g. parameters added in later media format specifications MAY
be copied into the SDP and SHALL be ignored by receivers that do not
understand them."
"
[CP]
Please add some examples of complete RTP packets and their contents, for
several combinations of unit types, including all the headers and data.
[JR]
Added.

[CP]
These lead to a bigger question: lease clarify the contents of the
payload. Is it a single logical block, with the headers giving pointers
to sub-parts of the block? Or is it multiple logical blocks (some text,
some modifiers) stacked after the headers in order?
[JR]
Hopefully (!) clarified with  examples.

---------------------------------------------
Jose Rey              mailto:[email-masked]
P R D C G - Panasonic R&D Center Germany GmbH
Phone/Fax: +49(0)6103766-134/166
--------------------------------------------- 

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt