Subject: Re: [AVT] I-D Action:draft-ietf-avt-rtp-svc-19.txt

From: "Charles Eckel (eckelcu)" <[email-masked]>
Date: Tue, 20 Oct 2009 15:59:06 -0700

Perhaps I am missing some fundamental aspects of the draft; however, in
reading over it, I do not understand how one would convey the desired
layering scheme. For example, if when using SVC it is desired for the
base layer to be AVC compatible with a given resolution and frame rate,
then for temporal and spatial enhancement layers to be added on top, how
would one signal that. The example in section 7.3.1 illustrates offering
an AVC compatible stream and an SVC compatible stream; however, it does
not illustrate how to request that if SVC is supported (97) a specific
layering scheme is used that includes an AVC compatible base layer with
a specific resolution and frame rate.
Being able to negotiate the layering scheme in SDP seems useful to me.
Was this considered?

Thanks,
Charles

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
Ye-Kui Wang
> Sent: Thursday, September 17, 2009 9:57 AM
> To: [email-masked]
> Subject: Re: [AVT] I-D Action:draft-ietf-avt-rtp-svc-19.txt
> 
> 
> 
> Changes compared to -18 are list below, among other minor editorial
changes:
> 
> - alignment of recent changes to the RFC 3984bis draft
> - update of some references for which the RFCs are now available
> - SDP examples are now in line with RFC5583
> - change of Stephan's affiliation and address
> - change of Thomas' email address
> 
> I hope Chairs could issue a second WGLC for this draft.
> 
> BR,YK
> 
> > -----Original Message-----
> > From: [email-masked] [mailto:[email-masked]] On
> > Behalf Of [email-masked]
> > Sent: Thursday, September 17, 2009 12:45 PM
> > To: [email-masked]
> > Cc: [email-masked]
> > Subject: [AVT] I-D Action:draft-ietf-avt-rtp-svc-19.txt
> >
> > A New Internet-Draft is available from the on-line
> > Internet-Drafts directories.
> > This draft is a work item of the Audio/Video Transport
> > Working Group of the IETF.
> >
> >
> > 	Title           : RTP Payload Format for SVC Video
> > 	Author(s)       : S. Wenger, et al.
> > 	Filename        : draft-ietf-avt-rtp-svc-19.txt
> > 	Pages           : 103
> > 	Date            : 2009-09-17
> >
> > This memo describes an RTP payload format for Scalable Video Coding
> > (SVC) as defined in_Annex G of ITU-T Recommendation H.264,
> > which is technically identical to Amendment 3 of ISO/IEC
> > International Standard 14496-10.  The RTP payload format
> > allows for packetization of one or more Network Abstraction
> > Layer (NAL) units in each RTP packet payload, as well as
> > fragmentation of a NAL unit in multiple RTP packets.
> > Furthermore, it supports transmission of an SVC stream over a
> > single as well as multiple RTP sessions.  The payload format
> > defines a new media subtype name "H264-SVC", but is still
> > backwards compatible to [I-D.ietf-avt-rtp-rfc3984bis] since
> > the base layer, when encapsulated in its own RTP stream, must
> > use the H.264 media subtype name ("H264") and the
> > packetization method specified in [I-
> > D.ietf-avt-rtp-rfc3984bis].  The payload format has wide
> > applicability in videoconferencing, Internet video streaming,
> > and high bit-rate entertainment-quality video, among others.
> >
> > A URL for this Internet-Draft is:
> > http://www.ietf.org/internet-drafts/draft-ietf-avt-rtp-svc-19.txt
> >
> > Internet-Drafts are also available by anonymous FTP at:
> > ftp://ftp.ietf.org/internet-drafts/
> >
> > Below is the data which will enable a MIME compliant mail
> > reader implementation to automatically retrieve the ASCII
> > version of the Internet-Draft.
> >
> 
> 
> _______________________________________________
> Audio/Video Transport Working Group
> [email-masked]
> https://www.ietf.org/mailman/listinfo/avt

---

From: Ye-Kui Wang <[email-masked]>
Date: Wed, 21 Oct 2009 10:04:03 -0400

Once you are able to signal layering scheme in SDP, you are able to
negotiate layering scheme. To indicate specific resolution and frame rate
values, you can use the media type parameter sprop-operation-point-info.
When providing an SVC stream, it is by default (and mandated) that the base
layer is AVC compatible. Therefore I don't see anything missing. 

If the above does not answer your question well, perhaps you could try to
re-elaborate your question.

BR, YK 

> From: Charles Eckel (eckelcu) [mailto:[email-masked]] 
> Sent: Tuesday, October 20, 2009 6:59 PM
> To: Ye-Kui Wang; [email-masked]
> Subject: RE: [AVT] I-D Action:draft-ietf-avt-rtp-svc-19.txt
> Importance: Low
> 
> Perhaps I am missing some fundamental aspects of the draft; 
> however, in reading over it, I do not understand how one 
> would convey the desired layering scheme. For example, if 
> when using SVC it is desired for the base layer to be AVC 
> compatible with a given resolution and frame rate, then for 
> temporal and spatial enhancement layers to be added on top, 
> how would one signal that. The example in section 7.3.1 
> illustrates offering an AVC compatible stream and an SVC 
> compatible stream; however, it does not illustrate how to 
> request that if SVC is supported (97) a specific layering 
> scheme is used that includes an AVC compatible base layer 
> with a specific resolution and frame rate.
> Behalf Of
> > > From: [email-masked] [mailto:[email-masked]] 
> On Behalf 
> > > Of [email-masked]
> > > (SVC) as defined in_Annex G of ITU-T Recommendation 
> H.264, which is 
> > > technically identical to Amendment 3 of ISO/IEC International 
> > > Standard 14496-10.  The RTP payload format allows for 
> packetization 
> > > of one or more Network Abstraction Layer (NAL) units in each RTP 
> > > packet payload, as well as fragmentation of a NAL unit in 
> multiple 
> > > RTP packets.
> > > Furthermore, it supports transmission of an SVC stream 
> over a single 
> > > as well as multiple RTP sessions.  The payload format 
> defines a new 
> > > media subtype name "H264-SVC", but is still backwards 
> compatible to 
> > > [I-D.ietf-avt-rtp-rfc3984bis] since the base layer, when 
> > > encapsulated in its own RTP stream, must use the H.264 
> media subtype 
> > > name ("H264") and the packetization method specified in [I- 
> > > applicability in videoconferencing, Internet video streaming, and 
> > > high bit-rate entertainment-quality video, among others.
> > >
> > > Below is the data which will enable a MIME compliant mail reader 
> > > implementation to automatically retrieve the ASCII version of the 
> > > Internet-Draft.
> > >
> > 
> >

---

From: "Charles Eckel (eckelcu)" <[email-masked]>
Date: Wed, 21 Oct 2009 15:44:40 -0700

Hi YK,

Thanks for the response. I have been looking into
sprop-operation-point-info in more detail, but I unclear on a few key
concepts that I hope you can clarify for me. Taking the example in
7.3.2, the following sprop-operation-point-info may be offered.

      a=rtpmap:97 H264-SVC/90000
      a=fmtp:97 profile-level-id=53000c; packetization-mode=1;
       sprop-parameter-sets={sps0},{sps1},{pps0},{pps1};
       sprop-operation-point-info=<1,0,0,0,4d400a,3200,176,144,128,
      256>,<2,1,1,0,53000c,6400,352,288,256,512>;

First, take the simple case in which the receiver supports SVC and wants
to choose the highest operation point. In this case, the answer includes
scalable-layer-id=2. Does this imply that a base layer should be sent
corresponding to  scalable-layer-id=1, with one of more enhancement
layers being sent to a maximum of scalable-layer-id=2?

If the receiver supports something higher than scalable-layer-id=1 but
lower than scalable-layer-id=2, does it respond with scalable-layer-id=1
and send a base layer corresponding to scalable-layer-id=1 but send one
of more enhancement layers exceeding that defined for
scalable-layer-id=1, or does it respond with scalable-layer-id=2 and
send a base layer corresponding to scalable-layer-id=1 and one or more
enhancement layers but stops short of that defined for
scalable-layer-id=2.

Is it legal to include more than two operation points in the
sprop-operation-point-info? The following text makes it sound to me as
if there is a limit of two.

         If the sprop-operation-point-info is followed by exactly one
operation-
         point-description vector, this describes the highest operation
         point contained in the RTP session.  If there are exactly two
         operation-point-description vectors, the first describes the
         lowest and the second describes the highest operation point
         contained in the RTP session.

Thanks,
Charles

> From: Ye-Kui Wang [mailto:[email-masked]]
> Sent: Wednesday, October 21, 2009 7:04 AM
> To: Charles Eckel (eckelcu); [email-masked]
and

---

From: Ye-Kui Wang <[email-masked]>
Date: Wed, 21 Oct 2009 22:26:51 -0400

Hi Charles,

See inline below.

> Sent: Wednesday, October 21, 2009 6:45 PM
> sprop-operation-point-info in more detail, but I unclear on a 
> few key concepts that I hope you can clarify for me. Taking 
> the example in 7.3.2, the following 
> sprop-operation-point-info may be offered.
> 
> First, take the simple case in which the receiver supports 
> SVC and wants to choose the highest operation point. In this 
> case, the answer includes scalable-layer-id=2. Does this 
> imply that a base layer should be sent corresponding to  
> scalable-layer-id=1, with one of more enhancement layers 
> being sent to a maximum of scalable-layer-id=2?
> 

Yes. All the layers, including the base layer, required for decoding the
operation point corresponding to scalable-layer-id=2 will be sent in this
case. 

> If the receiver supports something higher than 
> scalable-layer-id=1 but lower than scalable-layer-id=2, does 
> it respond with scalable-layer-id=1 and send a base layer 
> corresponding to scalable-layer-id=1 but send one of more 
> enhancement layers exceeding that defined for 
> scalable-layer-id=1, or does it respond with 
> scalable-layer-id=2 and send a base layer corresponding to 
> scalable-layer-id=1 and one or more enhancement layers but 
> stops short of that defined for scalable-layer-id=2.
> 

In this case, the answerer responds with scalable-layer-id=1 and sends the
bitstream for the operation point with scalable-layer-id=1, and the offerer
also sends the bitstream for the operation point with scalable-layer-id=1.
Only when the offer explicitly includes more operation points, with some
corresponding to the intermediate layers, the answerer can choose one of
them. 

> sprop-operation-point-info? The following text makes it sound 
> to me as if there is a limit of two.
> 

Yes, you can explicitly include many operation points. The following text
does not have that implication (at least to me). 

BR, YK

> > When providing an SVC stream, it is by default (and 
> mandated) that the
> base
> > If the above does not answer your question well, perhaps 
> you could try
> to
> > > Perhaps I am missing some fundamental aspects of the 
> draft; however, 
> > > in reading over it, I do not understand how one would convey the 
> > > desired layering scheme. For example, if when using SVC it is 
> > > desired for the base layer to be AVC compatible with a given 
> > > resolution and frame rate, then for temporal and spatial 
> enhancement 
> > > layers to be added on top, how would one signal that. The 
> example in 
> > > section 7.3.1 illustrates offering an AVC compatible 
> stream and an 
> > > SVC compatible stream; however, it does not illustrate how to 
> scheme is 
> > > used that includes an AVC compatible base layer with a specific 
> > > resolution and frame rate.
> > > > Audio/Video Transport Working Group [email-masked]

---

From: "Charles Eckel (eckelcu)" <[email-masked]>
Date: Thu, 22 Oct 2009 08:46:06 -0700

Hi YK,

This all sounds good. The sprop-operation-point-info parameter provides
more flexibility and functionality than I originally thought. Based on
my new understanding, I suggest changing the sentence:

         If there are exactly two
         operation-point-description vectors, the first describes the
         lowest and the second describes the highest operation point
         contained in the RTP session.
to:
         If there are two or more
         operation-point-description vectors, the first describes the
         lowest and the last describes the highest operation point
         contained in the RTP session.

At the end of the description of sprop-operation-point-info, where it
currently reads:

         A receiver
         MAY indicate its choice of one layer using the optional media
         type parameter scalable-layer-id.

I suggest change to:

         A receiver
         MAY indicate its choice of the highest layer it wants to send
and/or receive using the optional media
         type parameter scalable-layer-id.
         
I also think it would be good to have one of the SDP examples illustrate
this by adding a third operation point to the example. It might also be
a good idea to illustrate that the values of scalable-layer-id do not
need to be consecutive. 

One other minor thing I noticed while reading the draft:

Section 7.1

- max-bitrate:  This value specifies the average bit rate of
         the operation point.  This parameter is given as an integer in
         kbits per second and describes the maximum bitrate per each
         one second window.  Note that this parameter is provided in
         the scalability information SEI message in bits per second and
         is  calculated over a variable time window.  This field MAY be
         empty, indicating that the value is unspecified.

First line should say maximum bit rate, not average.

Thanks,
Charles       

> Sent: Wednesday, October 21, 2009 7:27 PM
> > > negotiate layering scheme. To indicate specific resolution and
frame
> > rate
> > > > Being able to negotiate the layering scheme in SDP seems useful
to
> > me.
> > > > > > implementation to automatically retrieve the ASCII version
of
> > the

---

From: Ye-Kui Wang <[email-masked]>
Date: Thu, 22 Oct 2009 12:07:46 -0400

Hi Charles, many thanks for your suggestions, which are all agreeable to me.

Hi All, If no one objects these changes, I will implement them and submit a
new version. Please let me know as soon as possible if you do have
objections. 

See also inline below. 

> Sent: Thursday, October 22, 2009 11:46 AM
> This all sounds good. The sprop-operation-point-info 
> parameter provides more flexibility and functionality than I 
> originally thought. Based on my new understanding, I suggest 
> changing the sentence:
> 
>          If there are exactly two

OK to me. 

> At the end of the description of sprop-operation-point-info, 
> where it currently reads:
> 
>          A receiver
>          MAY indicate its choice of the highest layer it 
> wants to send and/or receive using the optional media

Good. 

> I also think it would be good to have one of the SDP examples 
> illustrate this by adding a third operation point to the example. 

OK to me. To be simpler, we could just udpate the existing sample to include
three operation points. 

> It might also be a good idea to illustrate that the 
> values of scalable-layer-id do not need to be consecutive. 
> 

OK to me. 

> First line should say maximum bit rate, not average.

Definitely. 

BR, YK

> > > sprop-operation-point-info in more detail, but I unclear on a few 
> > > key concepts that I hope you can clarify for me. Taking 
> the example 
> > > in 7.3.2, the following sprop-operation-point-info may be offered.
> > >
> > > First, take the simple case in which the receiver 
> supports SVC and 
> > > wants to choose the highest operation point. In this case, the 
> > > answer includes scalable-layer-id=2. Does this imply that a base 
> > > layer should be sent corresponding to 
> scalable-layer-id=1, with one 
> > > of more enhancement layers being sent to a maximum of 
> > > scalable-layer-id=2?
> > >
> > 
> > > scalable-layer-id=1 but lower than scalable-layer-id=2, does it 
> > > respond with scalable-layer-id=1 and send a base layer 
> corresponding 
> > > to scalable-layer-id=1 but send one of more enhancement layers 
> > > exceeding that defined for scalable-layer-id=1, or does 
> it respond 
> > > with
> > > scalable-layer-id=1 and one or more enhancement layers but stops 
> > > short of that defined for scalable-layer-id=2.
> > >
> > 
> > In this case, the answerer responds with 
> scalable-layer-id=1 and sends
> the
> > corresponding to the intermediate layers, the answerer can 
> choose one
> of
> > them.
> > 
> > > sprop-operation-point-info? The following text makes it 
> sound to me 
> > > as if there is a limit of two.
> > >
> >

---

From: Alex Eleftheriadis <[email-masked]>
Date: Sun, 25 Oct 2009 21:52:35 +0200

They look fine to me too.

--Alex