Subject: [AVT] Sigalling for layered media -- do it generic or do it payload specific

From: "Stephan Wenger" <[email-masked]>
Date: Fri, 11 Nov 2005 17:09:24 -0800

Folks,

I walked out of the AVT meeting with an action item to rouse up the list on
the subject of generic vs. payload specific signaling for layered media.
Let me throw in a few random thoughts on both options.  Perhaps one or two
bother to comment :-)

Regards,
Stephan

Generic:
- If we decide to work towards a generic mechanism for the signaling of
layered media, the work would likely end up in MMUSIC.  Cross-WG (and
perhaps area) coordination necessary etc. etc.
- Doing something generic is more risky and more difficult than something
specific (one has to try to envision upcoming technologies, but even the
best informed guess could be wrong).
- Doing something generic takes more time, and the way the IETF goes, work
on non-generic signaling would be blocked on the generic work. 
- Result: a risk of not having the SVC payload available when we need it.
- When possible, the IETF prefers generic mechanisms over specific ones,
even if they are a bit more costly (in terms of cycles, bits,
standardization time, page count, whatever)
- A generic mechanism can include hooks for specific enhancements.  The
opposite direction (generalization of a specific mechanism) has much lower
chances of success.
- With a generic mechanism we could perhaps add support for legacy layered
codecs (MPEG-2, MPEG-4).  Not that anyone uses those in scalable mode,
though.

Payload specific
- Signaling can be tailored to the media format, to an extend no generic
mechanism can possibly achieve
- Signaling can utilize signaling support built into the media standard.
See the SVC payload draft for an example and discussion
- As we are normally talking about compressing codecs nowadays, chances are
that the signaling developed elsewhere is more lightweight than (say)
straightforward SDP
- Work on signaling support can be pushed away into committees, which
understand their stuff better than AVT or MMUSIC.
- Work in the media standardization committees is encouraged.  This may (in
the long term) lead to a better and more IETF-friendly media codecs and
system designs.
- Generally, media standardization committees are faster than the IETF --
shorter time to market 

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

---

From: Colin Perkins <[email-masked]>
Date: Sat, 26 Nov 2005 15:49:34 +0000

On 12 Nov 2005, at 01:09, Stephan Wenger wrote:
> I walked out of the AVT meeting with an action item to rouse up the  
> list on the subject of generic vs. payload specific signaling for  
> layered media. Let me throw in a few random thoughts on both  
> options.  Perhaps one or two bother to comment :-)
>
> Regards,
> Stephan
>
> Generic:
> - Signaling can be tailored to the media format, to an extend no  
> generic mechanism can possibly achieve
> - Signaling can utilize signaling support built into the media  
> standard. See the SVC payload draft for an example and discussion
> - As we are normally talking about compressing codecs nowadays,  
> chances are that the signaling developed elsewhere is more  
> lightweight than (say) straightforward SDP
> - Work on signaling support can be pushed away into committees,  
> which understand their stuff better than AVT or MMUSIC.
> - Work in the media standardization committees is encouraged.  This  
> may (in the long term) lead to a better and more IETF-friendly  
> media codecs and system designs.
> - Generally, media standardization committees are faster than the  
> IETF -- shorter time to market

One additional comment - payload specific signalling will still need  
some generic component, to indicate in the SDP that multiple media  
streams are parts of a single session. It need not indicate how they  
are related, but must indicate that they *are* related.

Colin

---

From: "Thomas Schierl" <[email-masked]>
Date: Sun, 27 Nov 2005 17:59:13 +0100

Colin,

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Colin Perkins
> Sent: Saturday, November 26, 2005 4:50 PM
> To: Stephan Wenger
> Cc: [email-masked]
> Subject: Re: [AVT] Sigalling for layered media -- do it generic or do
> itpayload specific
> 

I think different, especially if the receiver has the full knowledge about
the video codec.
For SVC, there is one base layer, which is H.264 compliant and which has to
be received and decoded anyway. If a receiver (or a mixer/translator as
well) is also aware of SVC, it will realize that there are additional video
enhancement layers in different RTP sessions "offered", which are SVC
compliant. Therefore it needs its specific knowledge of SVC and the SVC
specific "binary" SDP signalling in order for being able to request and
decode the full quality. 
Or is there somewhere a definition, which I overlooked, that each video RTP
session (signalled in SDP) must be an equivalent of the others and/or must
be independently decode-able? With video and audio it's similar, they are
related to each other, but signalling is done implicitly, or not? And there
should be no restrictions on the number of different video streams in a
conference or streaming scenario. 
Regarding this issue, I only found sections 2.4, 8.3 and 11. of RFC3550,
which are not opposing my comments.

Thomas Schierl,
----------------
Fraunhofer HHI
Berlin, Germany
http://ip.hhi.de

---

From: Colin Perkins <[email-masked]>
Date: Sun, 27 Nov 2005 17:25:54 +0000

Thomas,

On 27 Nov 2005, at 16:59, Thomas Schierl wrote:
...
> For SVC, there is one base layer, which is H.264 compliant and  
> which has to be received and decoded anyway. If a receiver (or a  
> mixer/translator as well) is also aware of SVC, it will realize  
> that there are additional video enhancement layers in different RTP  
> sessions "offered", which are SVC compliant. Therefore it needs its  
> specific knowledge of SVC and the SVC specific "binary" SDP  
> signalling in order for being able to request and decode the full  
> quality.

Sure.

> Or is there somewhere a definition, which I overlooked, that each  
> video RTP session (signalled in SDP) must be an equivalent of the  
> others and/or must be independently decode-able?

Yes. If you offer two "m=video" lines in SDP they are treated as  
separate and unrelated RTP sessions. You need to use the SDP grouping  
framework (RFC 3388) to indicate that they actually form two parts of  
the same session. This issue affects the ULP format which groups  
sessions for FEC (hence draft-ietf-mmusic-fec-grouping-00.txt) in  
addition to layered coding.

Colin

---

From: Stephen Casner <[email-masked]>
Date: Sun, 27 Nov 2005 09:51:53 -0800 (PST)

> addition to layered coding.

We tend to use the word "session" somewhat loosely with multiple
definitions in different places.  However, an "RTP session" is defined
very specifically (and after considerable discussion) in RFC 3550.
The two "m=video" lines will be separate RTP sessions whether they are
grouped or not.  They will, indeed, be unrelated if not grouped.

                                                        -- Steve

---

From: "Thomas Schierl" <[email-masked]>
Date: Mon, 28 Nov 2005 22:13:52 +0100

Colin, Steve,

> grouped or not.  They will, indeed, be unrelated if not grouped.

Understood!
But again, is there a real need of signalling the grouping or the relation
of 'm='-lines describing layers of an SVC stream in a generic way within
SDP?
The application/mixer/translator has anyway to have the full knowledge about
the media in order to do anything useful with or on it and that is also said
in the introduction of RFC 3388.

In section 7. of RFC 3388 the definitions for media streams and RTP sessions
are summarized and I see that there could be a reason of signalling the
grouping or relation of 'm='-lines describing layers of a layered media
stream, since this point seams not to be clearly defined. But that would
require extending the grouping-semantics, since with the current grouping
types (LS, FID) the SVC layer dependency would not be able to be signalled
(RFC 3388, 7.5.2).

But what could be a reason of signalling the relation from the
implementation's point of view?
An application or a similar instance does not need it. There could be only a
reason for enabling media-unaware instances doing something with the stream,
but do we really want that? 
A receiver's SDP parser should be anyway robust enough to ignore 'm='-lines
with unknown media types, thus only SVC media-aware receivers will process
the additional 'm='-lines, with or without explicit relation signalling. 

Thomas.

---

From: Magnus Westerlund <[email-masked]>
Date: Tue, 29 Nov 2005 09:25:22 +0100

Hi Thomas,

This is actually quite easy to motivate. One case is definitly when you 
have two different video screens part of the same multi-media session. 
Both are scalable and it is the session logic that is going to setup 
which RTP sessions that belong together and should go to the same 
decoder. First of all does SVC actually allow you to safely and 
efficently determine this case. Secondly, in most case you don't want to 
invoke the media decoder to determine which component (codecs, 
processing, output, etc.) and how they are connected in your system.

Another case is as you mention middleboxes. If you don't allow 
middleboxes to have easy understanding of what is going on they are more 
likely to do something bad to your stream rather than letting them 
through. Also there a number of cases when certain middleboxes are a 
necessary part of the picture.

Cheers

Magnus

-- 

Magnus Westerlund

Multimedia Technologies, Ericsson Research EAB/TVA/A
----------------------------------------------------------------------
Ericsson AB                | Phone +46 8 4048287
Torshamsgatan 23           | Fax   +46 8 7575550
S-164 80 Stockholm, Sweden | mailto: [email-masked]

---

From: "Thomas Schierl" <[email-masked]>
Date: Tue, 29 Nov 2005 16:48:19 +0100

Hi Magnus,

I see that it could be useful to make it easier for the receiver to
understand the layer dependencies on a higher level. 
But there will be anyway a need of having detailed knowledge about the SVC
data, e.g. switching up/down between different layers will not be possible
at each frame in the stream. I mean, if a middlebox decides to lower or
increase the quality by dropping or adding layers, it has to be that
media-aware to understand the full SVC stream semantics.

Nevertheless that is a matter of taste.
I am in the opinion that a middlebox as well as a receiver has to have the
full knowledge about the media and therefore it must be able to parse all
binary SVC description. But it could make life easier if the
dependency/relation would be signalled via e.g. extended group-semantics.
But for SVC there is no strong demand on doing signalling in a generic way.

Thomas.