Subject: [AVTCORE] Magnus Westerlund's Discuss on draft-ietf-payload-rtp-ttml-03: (with DISCUSS and COMMENT)

From: Magnus Westerlund via Datatracker <[email-masked]>
Date: Mon, 21 Oct 2019 07:02:23 -0700

Magnus Westerlund has entered the following ballot position for
draft-ietf-payload-rtp-ttml-03: Discuss

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.

The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-payload-rtp-ttml/

----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------

James and WG,

I do have a couple of issues I want to have your feedback on if they should be
corrected or not before proceeding to publication. Note they are for discussion
and in cases where things have been discussed and there is consensus please
reference that so that I can take that into consideration when we resolve these.

1. Section 4.1:
        Timestamp:
        The RTP Timestamp encodes the time of the text in the packet.

        As timed text is a media that has duration, from a start time to an end
        time, and the RTP timestmap is a single time tick in the chose clock
        resolution the above text is not clear. I would think the start time of
        the document would be the most useful to include?

        I think the text in 4.2.1.2 combined with the above attempts to imply
        that the RTP timestamp will be the 0 reference for the time-expression?

        I think this needs a bit more clarification. Not having detailed
        studied TTML2/1 I might be missing important details. But some more
        information how the document timebase:media time line connects to the
        RTP timestamp appears necessary.

2. A Discuss Discuss: As Timed Text is directly associated with one or more
video and audio streams and requires synchronization with these other media
streams to function correct. This leads to two questions.

        First of all is application/ttml+xml actually the right top-level media
        type? If using SDP that forces one unless one have BUNDLE to use a
        different RTP session. Many media types having this type of properties
        of being associated with some other media types have registered media
        types in all relevant top-level media types.

        Secondly, this payload format may need some references to mechanisms in
        RTP and signalling that has the purpose of associating media streams? I
        also assume that we have the interesting cases with localization that
        different languages have different time lines for the text and how long
        it shows as there are different tranditions in different countries and
        languages for how one makes subtitles.

        This may also point to the need for discussing the pick one out of n
        mechanism that a manifest may need.

3. Section 7.1:

        It may be appropriate to use the same Synchronization
   Source and Clock Rate as the related media.

        Using the same SSRC as another media stream in the same RTP Session is
        no-no. If you meant to use multiple RTP sessions and associate them
        using the same SSRC in diffiernt, yes it works but is not recommended.
        This points to the need for a clearer discussion of how to achieve
        linkage and the reasons for why same RTP timestamp may be useful or not.

4. Fragmentation:
        I think the fragmentation of an TTML document across multiple RTP
        payloads are a bit insufficiently described. I have the impression that
        it is hard to do something more clever than to fill each RTP payload to
        MTU limtiation, and send them out insequence. However, I think a firm
        requirement to apply RTP sequence number for a single document in
        consecutive numbers. Also the re-assebly process appear to have to
        parts for detecting what belongs together, same timestamp and last
        packet of document should have marker bit set. As a receiver can loose
        the last packet in the previous document, still know that it has
        received everything for the following document. However, if the losses
        are multiple, inspection of the re-assemblied document will be
        necessary to determine if the correct beginning is present. I have the
        impression that a proper section discussing these matter of
        fragmentation and re-assembly are necessary for good interoperability
        and function.

5. Lack of definition of parameter types in the media type when using SDP
Offer/answer.

As the application/ttml media type do contain parameters (charset and profile)
there is a need to define what SDP O/A interpretations they need to have. See
section 3.4.2.1 of RFC 8088 for discussion of these different types.

----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

A. Section 6.
        To my understanding the TTML document is basically not possible to
        encode better. A poor generator can create unnecessary verbose XML
        which could be shorter, but there are no possibility here to trade-off
        media quality for lower bit-rate. I think that should be made more
        explicit in Section 6.

B. Section 7.
        Wouldn't using 90kHz be the better default? 1kHz is the minimal from
        RTCP report that will work decently. However, if the timed text is
        primarily going to be synchronized with video 90k do ensure that
        (sub-)frame precise timing is possible to express. I don't see any need
        raster line specific for time text so the SMPTE 27 MHz clock is not
        needed. And using non default for subtitling radio etc appears fine.

C. Repair operations and relation to documents. Based on basic properties of
TTML documents, I do think the repair operations should be highly targeting
single documents as there is likely seconds between documents, while the
fragments of a document will be sent in a rather short interval. That
recommendation would be good to include.

---

From: Magnus Westerlund <[email-masked]>
Date: Tue, 22 Oct 2019 10:34:30 +0000

Hi,

This is an updated Discuss with the issue of missing SDP O/A parts that Christer
Holmberg did notice. I put this on discuss level as it really need to be fixed.
I think we all missed it due to the fact that the media type definition is
elsewhere. 

Cheers

Magnus

On Mon, 2019-10-21 at 07:02 -0700, Magnus Westerlund via Datatracker wrote:
> _______________________________________________
> Audio/Video Transport Core Maintenance
> [email-masked]
> https://www.ietf.org/mailman/listinfo/avt
-- 
Cheers

Magnus Westerlund 

----------------------------------------------------------------------
Network Architecture & Protocols, Ericsson Research
----------------------------------------------------------------------
Ericsson AB                 | Phone  +46 10 7148287
Torshamnsgatan 23           | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden | mailto: [email-masked]
----------------------------------------------------------------------

---

From: James Sandford <[email-masked]>
Date: Fri, 25 Oct 2019 10:12:10 +0000

Hello,
Apologies for only replying to the one thread. I have uploaded a new I-D (-04) which I believe addresses the Discuss items and Comments raised.

Changes include:
* Improvements to typesetting
* Improvements to the clarity of various sentences/terms
* A new section of fragmentation
* A note on the RTCP jitter calculation being unusable with this payload format
* A new section on protection against data loss
* A new section on offer/answer considerations

I would like to highlight that the new section on protection against data loss does not define usage of RFC 2198 redundancy. This is intentional and the reasoning is two fold. I have referenced SMPTE ST2022-7 which defines a means of sending duplicate streams over diverse paths. This provides similar functionality and is the most common method of sending duplicate streams in the broadcast industry, one of the more common users of TTML. Secondly, I could not convince myself that RFC 2198 with usage similar to RFC 4103 Section 4 would work effectively for fragmented payloads in a lossy environment. For this reason, I opted to stick with a reference to ST2022-7 as an example of a redundant transmission mechanism rather than re-inventing the wheel in this spec. This is alongside the usual references for FEC and re-transmission.

Regards,
James

==========
James Sandford
R&D Project Engineer

BBC Research and Development
5th Floor
Dock House
MediaCityUK
Salford
M50 2LH

Tel: 030304 (09549)
Web: http://www.bbc.co.uk/rd

________________________________________
From: Magnus Westerlund [[email-masked]]
Sent: 22 October 2019 11:34
To: [email-masked]
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Re: [AVTCORE] Magnus Westerlund's Discuss on draft-ietf-payload-rtp-ttml-03: (with DISCUSS and COMMENT)

Hi,

Cheers

Magnus

--
Cheers

Magnus Westerlund

---

From: Magnus Westerlund <[email-masked]>
Date: Tue, 29 Oct 2019 11:47:09 +0000

Hi James, 

Thanks for the  many updates in -04. However, I think there are a couple of
adjustments still needed. 

Section 8:

When a document spans more than one RTP packet, the entire document
is obtained by concatenating User Data Words from each contributing
packet in ascending order of Sequence Number.

I think this can be further clarified by adding "consecutive"

When a document spans more than one RTP packet, the entire document
is obtained by concatenating User Data Words from each consecutive
contributing
packet in ascending order of Sequence Number. 

What I think is unclear is what is considered contributing packets. It is
quite common that one determine fragments based on timestamp and that may be
assumed by some. I don't know if that is a dangerous assumption here. To my
understanding one can determine the set of fragments by looking at the
marker bit for the packets. From first 0 after a 1, until and including the
packet with a m=1. If that is your intention for how one should do it, so
that it works for multiple documents to share epoch and thus RTP timestamp
documents I think this needs to be made explicit. 

In section 11.1 it says: 

In these situations, it is RECOMMENDED that streams use
the same Synchronization Source and Clock Rate as the related media.

You do need to insert "Time" before Synchronization source to not be
misinterpret to mean SSRC. Or maybe better is to say "clock source".

Cheers

Magnus Westerlund

---

From: James Sandford <[email-masked]>
Date: Tue, 29 Oct 2019 13:52:05 +0000

Thank you for your further comments.

I will make the suggested changes to Section 8 and Section 11.1.

With regards to further clarifying the fragmentation of documents, I propose the following:

Section 6 OLD:
  If a TTML document is assessed to be invalid then it MUST be discarded. When processing a valid document, the following requirements apply.

Section 6 NEW:
  If a TTML document is assessed to be invalid then it MUST be discarded. This includes empty documents, i.e. those of zero length. When processing a valid document, the following requirements apply.

Section 8 ADDITIONAL PARAGRAPH:
  As described in Section 6, only zero or one TTML document may be active at any point in time.  As such, there MUST only be one document transmitted for a given RTP Timestamp.  Furthermore, as stated in Section 4.1, the Marker Bit MUST be set for a packet containing the last fragment of a document.  A packet following one where the Marker Bit is set contains the first fragment of a new document.  The first fragment might also be the last.

Regards,
James

==========
James Sandford
R&D Project Engineer

Tel: 030304 (09549)

________________________________________
From: Magnus Westerlund [[email-masked]]
Sent: 29 October 2019 11:47
To: James Sandford; [email-masked]
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: RE: [AVTCORE] Magnus Westerlund's Discuss on draft-ietf-payload-rtp-ttml-03: (with DISCUSS and COMMENT)

Hi James,

Section 8:

In section 11.1 it says:

Cheers

Magnus Westerlund

---

From: Magnus Westerlund <[email-masked]>
Date: Tue, 29 Oct 2019 14:13:40 +0000

Hi James,

Looks good to me. 

Thanks,

Magnus

On Tue, 2019-10-29 at 13:52 +0000, James Sandford wrote:
>   If a TTML document is assessed to be invalid then it MUST be discarded. When
> processing a valid document, the following requirements apply.
> 
> Section 6 NEW:
>   If a TTML document is assessed to be invalid then it MUST be discarded. This
> includes empty documents, i.e. those of zero length. When processing a valid
> document, the following requirements apply.
> 
>   As described in Section 6, only zero or one TTML document may be active at
> any point in time.  As such, there MUST only be one document transmitted for a
> given RTP Timestamp.  Furthermore, as stated in Section 4.1, the Marker Bit
> MUST be set for a packet containing the last fragment of a document.  A packet
> following one where the Marker Bit is set contains the first fragment of a new
> document.  The first fragment might also be the last.
> 
> 
> Regards,
> James
> 
> 
> ==========
> James Sandford
> R&D Project Engineer
> 
-- 
Cheers

Magnus Westerlund 

Networks, Ericsson Research

---

From: James Sandford <[email-masked]>
Date: Tue, 29 Oct 2019 14:24:13 +0000

Changes have been submitted in -05. https://datatracker.ietf.org/doc/draft-ietf-payload-rtp-ttml/

Regards,
James

==========
James Sandford
R&D Project Engineer

Tel: 030304 (09549)

Sent: 29 October 2019 14:13

Hi James,

Looks good to me.

Thanks,

Magnus

--
Cheers

Magnus Westerlund

---

From: Christer Holmberg <[email-masked]>
Date: Tue, 29 Oct 2019 14:52:48 +0000

Hi,

I see that you have added an SDP Offer/Answer section, but the only statement there is "All parameters are declarative".

I assume (please correct me if I am wrong) it shall be possible for two endpoints to negotiate a TTML RTP flow, in which case you need to describe the procedures for the Offerer and the Answerer.

Regards,

Christer

________________________________
From: avt <[email-masked]> on behalf of James Sandford <[email-masked]>
Sent: Tuesday, October 29, 2019 4:24 PM
To: Magnus Westerlund <[email-masked]>; [email-masked] <[email-masked]>
Cc: [email-masked] <[email-masked]>; [email-masked] <[email-masked]>; [email-masked] <[email-masked]>
Subject: Re: [AVTCORE] Magnus Westerlund's Discuss on draft-ietf-payload-rtp-ttml-03: (with DISCUSS and COMMENT)

Regards,
James

==========
James Sandford
R&D Project Engineer

Tel: 030304 (09549)

Hi James,

Looks good to me.

Thanks,

Magnus

--
Cheers

Magnus Westerlund

---

From: Christer Holmberg <[email-masked]>
Date: Tue, 29 Oct 2019 14:57:35 +0000

Hi,

I see that you have now added text on protection of packet loss. But, you just list a number of different mechanisms, without mandating (or even recommending) one of them - you just say that implementations must support *A* mechanism. But, that is most likely going to cause interoperability problems, if different implementations support different mechanisms.

Regards,

Christer

Regards,
James

==========
James Sandford
R&D Project Engineer

Tel: 030304 (09549)

Hi James,

Looks good to me.

Thanks,

Magnus

--
Cheers

Magnus Westerlund

---

From: Magnus Westerlund <[email-masked]>
Date: Tue, 29 Oct 2019 15:20:30 +0000

Hi,

RTP is way to much a framework, that the different usages normally have to
specify what mechanisms is actually to be used. That is true both for security
as well as for repair / redundancy mechanisms. I don't see this payload format
having that specific requirement that it actually makes sense to lock down to a
single solution for all usages. 

Cheers

Magnus

On Tue, 2019-10-29 at 14:57 +0000, Christer Holmberg wrote:
> Hi,
> 
> I see that you have now added text on protection of packet loss. But, you just
> list a number of different mechanisms, without mandating (or even
> recommending) one of them - you just say that implementations must support *A*
> mechanism. But, that is most likely going to cause interoperability problems,
> if different implementations support different mechanisms.
> 
> Regards,
> 
> Christer
> 
> Cc: [email-masked] <[email-masked]>; 
> [email-masked] <[email-masked]>; 
> [email-masked] <[email-masked]>
> Changes have been submitted in -05. 
> > Subject: RE: [AVTCORE] Magnus Westerlund's Discuss on draft-ietf-payload-
> rtp-
> > ttml-03: (with DISCUSS and COMMENT)
> >
> > Hi James,
> >
-- 
Cheers

Magnus Westerlund

---

From: Magnus Westerlund <[email-masked]>
Date: Tue, 29 Oct 2019 15:23:57 +0000

Hi Christer,

Actually, if they are all declarative it is actually sufficient. That results in
all parameters behaving as defined in RFC 3264 and you actually don't have to go
into details. 

Or do you disagree with that assessment? 

Cheers

Magnus

On Tue, 2019-10-29 at 14:52 +0000, Christer Holmberg wrote:
> 
> Hi,
> 
> I see that you have added an SDP Offer/Answer section, but the only statement
> there is "All parameters are declarative".
> 
> I assume (please correct me if I am wrong) it shall be possible for two
> endpoints to negotiate a TTML RTP flow, in which case you need to describe the
> procedures for the Offerer and the Answerer.
> 
> Regards,
> 
> Christer
> 
> 
> 
> 
-- 
Cheers

Magnus Westerlund

---

From: James Sandford <[email-masked]>
Date: Tue, 29 Oct 2019 15:32:28 +0000

Hi,

I am hesitant to mandate a specific mechanism for the reason Magnus has noted. The most likely use case for TTML over RTP, in my view, is in the broadcast industry. The industry standard for protection in this case is SMPTE ST2022-7 or the provision of an appropriately robust network. But that is a broadcast-oriented solution that may not be appropriate for other use cases. I'm hesitant to mandate support for mechanisms that may never be use by many/any implementations.

James

==========
James Sandford
R&D Project Engineer

Tel: 030304 (09549)

________________________________________
From: Magnus Westerlund [[email-masked]]
Sent: 29 October 2019 15:20
To: James Sandford; Christer Holmberg; [email-masked]
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Re: [AVTCORE] Magnus Westerlund's Discuss on draft-ietf-payload-rtp-ttml-03: (with DISCUSS and COMMENT)

Hi,

Cheers

Magnus

--
Cheers

Magnus Westerlund

---

From: Christer Holmberg <[email-masked]>
Date: Tue, 29 Oct 2019 15:33:05 +0000

Hi,

The way it is written now it sounds like you can't use this with SDP Offer/Answer, which I assume is not the intention.

The Offerer and Answerer procedures may be very simple, but we have previously agreed that they should be there.  For example, one piece of important information is what (if any) changes are allowed in subsequent offers/answers.

There are existing procedures that can be used as template, and I'm also happy to help if needed.

Regards,

Christer

On 29/10/2019, 17.24, "Magnus Westerlund" <[email-masked]> wrote:

    Hi Christer,

    
    Cheers
    
    Magnus

    -- 
    Cheers
    
    Magnus Westerlund

---

From: Christer Holmberg <[email-masked]>
Date: Tue, 29 Oct 2019 15:33:32 +0000

Fair enough.

Regards,

Christer

On 29/10/2019, 17.32, "James Sandford" <[email-masked]> wrote:

    Hi,

    James
    
    ==========
    James Sandford
    R&D Project Engineer

    Tel: 030304 (09549)

    Hi,

    Cheers
    
    Magnus
    
    --
    Cheers
    
    Magnus Westerlund

---

From: James Sandford <[email-masked]>
Date: Tue, 29 Oct 2019 15:47:21 +0000

Hi,
Could you explain what you think prevents the use of offer/answer? I think the relevant parameter is codecs which specifies the content profile(s) the documents adhere to. That is declarative and there is no mechanism in TTML to negotiate content profiles used.

Regards,
James

==========
James Sandford
R&D Project Engineer

Tel: 030304 (09549)

________________________________________
From: Christer Holmberg [[email-masked]]
Sent: 29 October 2019 15:33
To: Magnus Westerlund; James Sandford; [email-masked]
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Re: [AVTCORE] Magnus Westerlund's Discuss on draft-ietf-payload-rtp-ttml-03: (with DISCUSS and COMMENT)

Hi,

Regards,

Christer

    Hi Christer,

    Cheers

    Magnus

    --
    Cheers

    Magnus Westerlund

---

From: Christer Holmberg <[email-masked]>
Date: Tue, 29 Oct 2019 16:33:01 +0000

Hi,

Since you are not defining any new SDP attributes etc, I don't think you need the SDP O/A procedure structure we normally use.

After a second thought, maybe the easiest way would be to simply rename Section 11.2 to "SDP Considerations" (I think "Mapping to SDP" sounds confusing), and then remove section 11.3.

Regards,

Christer

On 29/10/2019, 17.47, "James Sandford" <[email-masked]> wrote:

    Hi,
    
    Regards,
    James
    
    ==========
    James Sandford
    R&D Project Engineer

    Tel: 030304 (09549)

    Hi,

    
    
    Regards,
    
    Christer

    
    
        Hi Christer,

    
        Cheers
    
        Magnus

        --
        Cheers
    
        Magnus Westerlund

---

From: James Sandford <[email-masked]>
Date: Tue, 29 Oct 2019 16:38:45 +0000

Hi,

I'm happy to do that. I'll hold off making the change until Adam has had a chance to review in case he has anything to add.

Regards,
James

==========
James Sandford
R&D Project Engineer

Tel: 030304 (09549)

________________________________________
From: Christer Holmberg [[email-masked]]
Sent: 29 October 2019 16:33
To: James Sandford; Magnus Westerlund; [email-masked]
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Re: [AVTCORE] Magnus Westerlund's Discuss on draft-ietf-payload-rtp-ttml-03: (with DISCUSS and COMMENT)

Hi,

Regards,

Christer

    Hi,

    Regards,
    James

    ==========
    James Sandford
    R&D Project Engineer

    Tel: 030304 (09549)

    Hi,

    Regards,

    Christer

        Hi Christer,

        Cheers

        Magnus

        --
        Cheers

        Magnus Westerlund