Subject: [AVT] Re: FYI: Structured-Audio-related IETF AVT individual contribution

From: John Lazzaro <[email-masked]>
Date: Tue, 28 Aug 2001 12:28:07 -0700

> Lim, Young-Kwon" <[email-masked]> wrote:

> It doens't require to multiplex multiple ESs in one RTP stream.

Yes, I didn't read draft-ietf-avt-mpeg4-multiSL-01.txt closely enough
before responding, and I didn't realize that it could be stripped down
as much as it can be.

So, let's assume that its possible to get the recovery journal
resilency system working inside of draft-ietf-avt-mpeg4-multiSL-01.txt,
or at least that if there is a problem, its a subtle one and not an 
obvious one.

Given that, it seems like the first-order questions concern how
to "break up" the technology in draft-lazzaro-avt-mwpp-midi-mpeg4-00.txt
so it can be maximally useful to everyone. The real core technologies 
(which we'll call MCT -- the MIDI Core Technologies) in the document are

-1- Handling "late" packets gracefully: modeling the channel to 
    determine the nominal latency, and executing the appropriate
    recovery semantics when a packet arrives later than nominal.

-2- Handling lost packets gracefully: using the recovery journal
    mechanism for keeping a recent MIDI history that is referenced
    to an early checkpoint packet, in a way which senders can track
    the last RTP packet received by each receiver via RTCP.

---

Note MCT is really about MIDI and RTP at its core -- we bring in
Structured Audio because its a normative reference for MIDI semantics
which does a great job of pinning down exact semantics, which is
necessary for concisely specifying the recovery journal semantics.

But if, say, CSound (a precursor to Structured Audio out of the MIT
Media Lab, that is much more popular than Structured Audio in the
music community) users were willing to accept MCT's technology
decisions, they could incorporate MCT into CSound and do network
musical performance w/o any MPEG technologies at all.

And MPEG 4 Systems comes into the picture because Structured Audio is
a part of MPEG 4, and so its natural to want Structured Audio with MCT
to plug-and-play with other MPEG 4 codecs via MPEG 4 Systems.

Ideally, we'd like to specify MCT in a way that would allow three
sorts of developers to be happy:

[1] Structured Audio developers who want to plug-and-play with
MPEG 4 Systems, and want a normative way to do it. These folks
would use draft-ietf-avt-mpeg4-multiSL-01.txt with as many or
as few options as they need, and they would refer to some other
IETF document which describes how to make MCT work with 
draft-ietf-avt-mpeg4-multiSL-01.txt.

[2] Non-Structured-Audio developers who want to use MCT in their
systems (like the CSound example above). MPEG 4 Systems is not
interesting for these folks -- they want the most efficient RTP
packetization of MCT possible, both in terms of

  -1- bandwidth
  -2- sending and receiving processing overhead 
  -3- software development time, dominated by the time a non-networking
      person needs to take to understand all of the standards before
      real work can begin.

In practice, -3- may be most important -- most implementors will be
domain experts in music synthesis, and it takes effort to convince
(some of) them of the benefit of using pre-existing networking
standards at all, their natural inclination is to use TCP, or failing
that, to start from raw IP packets and define their own standard for
scratch.  Adding MPEG 4 Systems to the list of things to learn
decreases the odds they'll accept RTP into their worldview. To
understand this issue better, read this article written by a game
developer, which captures it perfectly (and is a very entertaining
read too):

http://www.gamasutra.com/features/19990903/lincroft_01.htm

[3] Structured Audio developers who see no need for MPEG 4 Systems
support, and whose worldview are basically the same as [2] above. 

---

So, it seems like we'd need a set of memos:

  -1- The MCT core technology memos. All of the Appendices of 
      draft-lazzaro-avt-mwpp-midi-mpeg4-00.txt, along with several
      of the main sections of the memo.
  -2- A "how to use MCT with draft-ietf-avt-mpeg4-multiSL-01.txt" memo.
  -3- The RTP packetization of MCT described in 
      draft-lazzaro-avt-mwpp-midi-mpeg4-00.txt, for people who
      expressly want to use MCT without MPEG 4 Systems. This 
      ensures we don't get N different non-standard versions of
      a non-MPEG-4-Systems RTP packetization, done by people who
      try to read draft-ietf-avt-mpeg4-multiSL-01.txt and give up.
  -4- Some sort of framework memo to serve as an introduction.

> Would you explain more about "recovery journal"?

The shortest, easiest-to-read introduction is Sections 6 and 7 in
the main text of this paper:

http://www.cs.berkeley.edu/~lazzaro/sa/pubs/pdf/nossdav01.pdf

If you're familiar with MIDI, just skip to Section 7. To learn more,
the appendices in the nossdav01 paper are the next step; for the full
details, see the

http://www.ietf.org/internet-drafts/draft-lazzaro-avt-mwpp-midi-mpeg4-00.txt

document.

> Would you continue this discussion there?

I changed the lists to be IETF AVT and [email-masked].

-------------------------------------------------------------------------
John Lazzaro -- Research Specialist -- CS Division -- EECS -- UC Berkeley
lazzaro [at] cs [dot] berkeley [dot] edu     www.cs.berkeley.edu/~lazzaro
-------------------------------------------------------------------------

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
http://www1.ietf.org/mailman/listinfo/avt

---

From: John Lazzaro <[email-masked]>
Date: Wed, 29 Aug 2001 12:14:45 -0700

>Colin Perkins writes:
>
>
> >--> [email-masked] writes:
> > ...
> > your target is low delay, single SL stream:
> > "draft-ietf-avt-mpeg4-multiSL-01.txt"
> > is what you need !
>
> Perhaps, but doesn't the MIDI format need a
> non-MPEG specified recovery journal too?

Yes, I'm starting to formulate an approach for
rewriting the I-D, that recasts the recovery
journal technology as something that stands
on its own as a resilency technique for MIDI,
as something apart from MPEG 4 Structured Audio.

Phillipe's and Young's comments have highlighted
the need to write this I-D in a way that makes it
usable in contexts other than a raw RTP packet;
I think I can provide this capability in a 
general way, not as something specialized to a
specific MPEG 4 Systems packetization.

Ideally, in fact, the recovery journal will be
useful for folks that see RTP as a possible future
transport layer for their own MIDI application 
layer, as an alternative to custom UDP formats. 
For example, I spent a few hours yesterday talking
f2f to the Open Sound Control folks at CNMAT:

http://cnmat.cnmat.berkeley.edu/OSC/

to get a sense of what an I-D that would be useful
to them would need to look like, and if one reads
pages 75-106 of this document describing Apple's
Mac OS X MIDI systems services:

http://developer.apple.com/audio/pdf/coreaudio.pdf

one can get a sense of what a recovery journal 
packetization would need to look like in order
to fit transparently into an API designed for 
local MIDI wire protocol bundles.

I'm hoping this reformulated I-D gets written in
the next month or so, it will probably have a new
name that reflects the refocusing of the memo.