Subject: Kikuchi-san proposal as subset of generic MPEG-4 formats

From: [email-masked]
Date: Thu, 14 Sep 2000 12:03:08 +0200

Dear all,

As promised in a previous email, I would like to review the problems
associated with making the proposal of Kikuchi-san a subset of the set of
generic formats that hopefully will be produced in La Baule. My assumption
is that there will be "generic formats" for several MPEG-4 streams, such as:
- MPEG-4 audio elementary streams (allowing for carriage of the remainder of
an SL header)
- MPEG-4 video elementary streams (allowing for carriage of the remainder of
an SL header)
- MPEG-4 system streams
For now, I would like to constrain the discussion to the first two, as these
two are also addressed in the proposal of Kikuchi-san. In the following
review I make a separation between video specific issues and audio specific
issues. The general issue of the additional bytes is discussed extensively
and I don't think there is a need to recap that here. The only clarification
that may be useful in this context is that the additional bytes can be used
to carry the remainder of an SL header in cases where needed, thereby
providing the same functionality as in the proposal of Civanlar et al.

Video specific issues

My proposal would be to use for the generic MPEG-4 video ES format exactly
the same payload type and names as Kikuchi-san. I also suggest to use the
same fragmentation rules. So from my perspective no need to change anything
specific to that.

Audio specific issues

Also for the generic MPEG-4 audio ES format I suggest to use the same
payload type, names and fragmentation rules as Kikuchi-san. With audio we
have the "LATM problem". However, with some pragmatism we may be able to
solve this. In Paris we agreed to allow for more than one access unit within
one SL packet. We could slightly modify this by allowing LATM as multiplex
structure within SL. (Note that the FlexMux provides a multiplex structure
on top of SL.) To indicate usage of LATM within an SL packet a currently
reserved bit in the SL header could be assigned. If we agree on his, then
the important question is whether we need also define carriage of MPEG-4
audio ES over IP without LATM. Perhaps not. LATM adds only little overhead
in case of a single large AU and for a small AU it is likely that LATM is
applied anyhow. A final remark on this issue: LATM is also used for carriage
of MPEG-4 audio ES over MPEG-2, pretty much consistent with usage of LATM
within SL and for transport over IP.

Along these lines I will propose specific wording changes to the proposal of
Kikuchi-san. Please comment if you disagree with the approach described. I
hope that my proposal for usage of LATM is not too big of a shock. In my
view it is an acceptable price to pay for compatibility with the proposal
from Kikuchi-san. And I believe such compatibility is important for the
success of MPEG-4 on the market place.

Best regards,

Jan

---

From: [email-masked]
Date: Thu, 14 Sep 2000 16:42:54 +0200

Zvil,

I fully agree that signaling at OD level is a better solution. With respect
to migration towards other solutions, I think that it is better to have one
simple and straightforward solution for carriage of MPEG-4 A/V ES over IP
then the situation in which "you have many standards to choose from".
Specifically in case of thin clients.

With respect to audio, the reason of my suggestion to always use LATM is to
achieve a single solution for audio. I believe that this is often useful,
while the pain in terms of overhead and processing is very acceptable.

Regards,

Jan

-----Original Message-----
From:	[email-masked] [mailto:[email-masked]]
Sent:	donderdag, 14 september, 2000 14:15
To:	[email-masked]; Jan vanderMeer
Cc:	[email-masked]
Subject:	RE: Kikuchi-san proposal as subset of generic MPEG-4 formats

Dear Jan, all,

Even after we talked I wasn't convinced to agree to your approach. The main
thing is that you are talking about upgrading the Kikuchi-san proposal,
while I see no reason to do it. Specifically, I cannot think if any in-band
info I would like to add in the future to a stream that is delivered by the
ES format. Theoretically crazy ideas can always come up, but practically
the chances for this are not more than for any content carried over RTP.
And other payload formats didn't care about such extensions. It seems that
all the fancy enhancements that you, and Sam, and actually also me and most
people in MPEG want to have will be done through addition of extra streams
on top of the audio/video streams for which the Kikuchi-san proposal works
well.

Therefore I see no reason to change the Kikuchi-san proposal, as long as
future applications will be able to smoothly integrate streams using this
format and the generic format, which is indeed seems the situation.

Some other comments regarding your proposal:

1. The Generic payload format is for every MPEG-4 stream, including audio.
You seem to restrict the use of this format by requiring that whenever
audio is transmitted, the format cannot be used natively and LATM must be
used. I don't see a point and this restriction. LATM is for systems that
don't use MPEG-4 Systems. If an MPEG-4 server doesn't care about
compatibility with other systems why should we force it? Same argument for
video fragmentation.

2. Just like we made the ES proposal for video a subset of the Generic
format from MPEG-4/SL point of view, we can do that for audio, by allowing
(NOT requiring) LATM as an MPEG-4 stream. You want do indicate that in a
bit in the SL header, which I find no reason for. We need this indication
just once per stream. It can be in the SLConfig, but since LATM is only for
audio, it's a kind of format that should be indicated in the
DecoderConfigDescriptor. Maybe in objectTypeIndication.

Regards,

z
soon the whole world will STREAM
==================================================================
Zvi Lifshitz				Phone +972(2)679-4788
[email-masked]			Fax +972(2)679-4789
Optibase Ltd.				Mobile: +972(53)461-787
http://www.optibase.com		US voice mail/fax  +1(206) 888-4149
Send SMS from: 			http://isend.cellcom.co.il/English/Login.htm
==================================================================
> Come see us at:
Streaming Media Europe Earls Court Centre, Booth # 628, London, October
10th-12th, 2000

-----Original Message-----
From:	[email-masked] [SMTP:[email-masked]]
Sent:	Thursday, September 14, 2000 12:31 PM
To:	[email-masked]
Cc:	[email-masked]
Subject:	Kikuchi-san proposal as subset of generic MPEG-4 formats

Dear all,

As promised in a previous email, I would like to review the problems
associated with making the proposal of Kikuchi-san a subset of the set of
generic formats that hopefully will be produced in La Baule. My assumption
is that there will be "generic formats" for several MPEG-4 streams, such
as:
- MPEG-4 audio elementary streams (allowing for carriage of the remainder
of
an SL header)
- MPEG-4 video elementary streams (allowing for carriage of the remainder
of
an SL header)
- MPEG-4 system streams
For now, I would like to constrain the discussion to the first two, as
these
two are also addressed in the proposal of Kikuchi-san. In the following
review I make a separation between video specific issues and audio specific
issues. The general issue of the additional bytes is discussed extensively
and I don't think there is a need to recap that here. The only
clarification
that may be useful in this context is that the additional bytes can be used
to carry the remainder of an SL header in cases where needed, thereby
providing the same functionality as in the proposal of Civanlar et al.

Video specific issues

Audio specific issues

solve this. In Paris we agreed to allow for more than one access unit
within
applied anyhow. A final remark on this issue: LATM is also used for
carriage

Along these lines I will propose specific wording changes to the proposal
of

Best regards,

Jan

---

From: Zvi Lifshitz <[email-masked]>
Date: Thu, 14 Sep 2000 17:59:24 +0200

[[email-masked]:]

[Reply:]

But you want to further complicate the solution that we already have, simple or not..

[Original message:]

[Reply:]

There is something in between - the standard that best fits to every case.

[Original message:]

With respect to audio, the reason of my suggestion to always use LATM is to
achieve a single solution for audio.

[Reply:]

And what is the reason to achieve a single solution for audio?

z
Send SMS from: 			http://isend.cellcom.co.il/English/Login.htm ==================================================================
> Come see us at:
Streaming Media Europe Earls Court Centre, Booth # 628, London, October 10th-12th, 2000

---

From: [email-masked]
Date: Thu, 14 Sep 2000 17:01:25 +0200

Dear Guido,

As also said in my reply to Zvi, I have no problems to remove the signaling
from SL level. Zvi and you are right.

With respect to a solution in the control plane, that is fine too, as long
as it is required that clients MUST support both cases (with and without the
additional bytes). I agree with your analysis that such solution may be more
elegant; however, taking more time for study is only OK as long as we take
into account that the ITU needs their "compatible solution" really fast. If
the "control plane" solution can not be achieved quickly, I propose to focus
on the solution I suggested.

Perhaps for me the best thing to do is wait whether there are some more
reactions, possibly also from the US. Tomorrow I have a 12 hr+ flight back
from the Far East and than I may be able to prepare a more detailed
solution. To be distributed after late tomorrow, European time.

Kind regards,

Jan

-----Original Message-----
From:	[email-masked] [mailto:[email-masked]]
Sent:	donderdag, 14 september, 2000 14:29
To:	Jan vanderMeer; [email-masked]
Cc:	[email-masked]
Subject:	RE: Kikuchi-san proposal as subset of generic MPEG-4 formats

Dear Jan,

I am not really happy to signal LATM usage (which is an audio specific tool)
at the SL level (which is generic). I would prefer to expand the audio
specific configuration parameters.

Also, I think that while compatibility at the level of RTP packets is
reachable (e.g. by means of the extra byte), the real technical difficulty
resides in the control plane. Once somebody finds a solution for the control
plane issues, then maybe we will find more than one possible approach for
the data plane.
The usage of one extra byte (thus a payload header) in all RTP packets is a
possible technical solution, but also the usage of an indicator at config
time telling whether such extra byte (thus a payload header) is present or
not in the RTP packets could be considered (this way you would not force the
extra byte all the time). And maybe there are more solutions.
To conclude: I would not assert that the ONLY way to fulfill your
requirement is to force one extra byte all the time. I understand and agree
with the requirement, but personally don't like the proposed solution. I
would appreciate some more investigation in the control plane aspects.

Also, let's remember that the SDP and the MPEG-4 Object Descriptors serve
two very similar purposes. In IETF current practice, the 'negotiation' is
mostly done using SDP, while when using MPEG-4 Systems it is done through
ODs.
A solution aiming at serving with the same bits of information devices
compliant to the IETF current practice as well as devices compliant to
MPEG-4 Systems MUST consider the overlap between SDP and ODs.

IMHO, we can start this work, but we cannot rush it.
And this work would involve the MMUSIC group.

Regards
Guido

> ----------
> From: 	[email-masked][SMTP:[email-masked]]
> Sent: 	Thursday, September 14, 2000 12:30 PM

---

From: Franceschini Guido <[email-masked]>
Date: Thu, 14 Sep 2000 19:12:01 +0200

Dear Jan,

> on the solution I suggested.
The point is I believe that your extra-byte proposal does not work as is,
since the 'old' device, in presence of 'new' content, would know from the
control plane only that the RTP Payload Format is the 'generic' one, but it
would not know what media is being carried.

Bye
Guido

---

From: John Lazzaro <[email-masked]>
Date: Thu, 14 Sep 2000 11:28:13 -0700

[[email-masked] writes]

> If we agree on his, then
> applied anyhow. 

Here's an app where it may be worthwhile to carry MPEG-4 audio ES over IP 
without LATM: using SA_access_units to transmit MIDI Events between a set
of Structured Audio decoders over a low-latency network for real-time 
musical performance. Since SA_access_units's are defined to hold a single
MIDI command, most often 3 bytes, and in many cases there won't be another
audio codec being used along with the Structured Audio decoder, the LATM
overhead will be significant and won't be adding any value.

-------------------------------------------------------------------------
John Lazzaro -- Research Specialist -- CS Division -- EECS -- UC Berkeley
lazzaro [at] cs [dot] berkeley [dot] edu     www.cs.berkeley.edu/~lazzaro
-------------------------------------------------------------------------