Subject: [AVT] Re: [linux-audio-dev] Introducing DMIDI

From: John Lazzaro <[email-masked]>
Date: Tue, 18 Dec 2001 10:10:28 -0800

[AVT folks -- CC'ing an MWPP-related thread from the linux-audio-dev
 mailing list ...]

> "martijn sipkema" <[email-masked]> writes
>
>> John Lazzaro writes
>> 
>> [...]
>>
>> http://www.ietf.org/internet-drafts/draft-lazzaro-avt-mwpp-midi-nmp-00.txt
>>
>
> this is great! it doesn't handle sysex does it? perhaps a seperate
> protocol for sysex dumps over tcp would work. there are some
> synthesizers that use short sysex messages for control data, i don't
> think i have a synthesizer that does that though, but perhaps this is
> just bad design.
>
> --martijn

In its present incarnation, it handles the subset of MIDI that 
MPEG 4 Structured Audio uses -- all of the MIDI commands except
for the MIDI System commands (0xF*). Because one of the roles
of the packetization is to be a normative transport for SA, 
there would need to be at least one mode of operation where this
restriction is kept, since SA demands it.

However, there are reserved bits in both the "MIDI Command Section"
and in the recovery journal, intentionally left to facilitate MIDI
System implementation (including the SysEx commands), as well as 
to facilitate SASL/MIDI integration within Structured Audio. There
are a few distinct issues here:

-- Many of the MIDI System commands fall into the "real-time sync 
   of machines that play MIDI data" category:

0xf2    song position
0xf3    song select 
0xf8   timing clock
0xfa   start
0xfb   continue
0xfc   stop

   Right now, the packetization rests pretty heavily on the idea 
that the players are humans -- the latency over interesting 
Internet distances are comparable to the acoustic latency of 
musicians in a room (Berkeley-Stanford, 40 miles apart, has
an acoustic latency of 2.4 ft/0.7 meters/2.1ms; Berkeley-Caltech,
375 miles apart, has an acoustic latency of 16ft/4.9 meters/14.2ms).
Just like players can adjust to these latencies in the same room,
they can adjust over the net -- including small shifts in the
latency overy time. Figuring out a sensible way to implement the
real-time sync commands in that context is challenge, because the
players are robots, not humans, and they need to "listen" to the
latency of the link and adjust. 

-- The other major MIDI System category is SysEx -- the "reliable
bulk data" usages of SysEx are really a bad match for RTP, and
need to be shunted to a TCP link; the "manufacturer-specific 
control data" usage IS a good fit for RTP, though, and the 
challenge here is to define a good recovery journal semantics
and bit-encoding that works in the context of MWPP (the recovery
journal is the forward-error-correction system built into MWPP to
gracefully handle lost packets).

----

These two issues (and a few other minor ones) were sufficiently
complex, and sufficiently removed from the primary target application
of the packetization (identical softsynths on all network nodes,
to do network musical performance), that I thought the right thing
to do for now was to leave reserved bits in all the right places
in the packetization, so that when someone is motivated to solve
these problems well, the packetization can be upgraded in a 
backward-compatible way. If someone is inspired to work on the
problem now, I think that's a great idea -- you could write a  
companion I-D in the AVT working group for MIDI Systems semantics,
defining how to handle MIDI Systems inside of MWPP. 

                                                        --jl

-------------------------------------------------------------------------
John Lazzaro -- Research Specialist -- CS Division -- EECS -- UC Berkeley
lazzaro [at] cs [dot] berkeley [dot] edu     www.cs.berkeley.edu/~lazzaro
-------------------------------------------------------------------------

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
http://www1.ietf.org/mailman/listinfo/avt

---

From: John Lazzaro <[email-masked]>
Date: Thu, 20 Dec 2001 15:04:27 -0800

[ resending reply over to AVT, didn't realize the CC had remained ...]

> Dominique Fober writes:
>
> [... algorithm in Appendix B of NOSSDAV paper, does it have clock skew
>      issues?]

Note the use of RTCP in this algorithm -- when new RTCP packets arrive,
we check their timestamp to see if they are on time, and if so we update
the ACO model parameters. This on-going parameter re-estimation (about
once per 5 seconds for sessions with small numbers of receivers) handles
the garden-variety "sender and receiver clocks unsynced" issue pretty
well -- in practice on CalREN 2, we only see the "late window exhaustion"
happen for rarer network and host events that mess up local timing. 

> Are RTP timestamped packets enough ? I'm not sure ...

Another feature of RTCP can come in handy here too -- each RTCP packet
supplies both the RTP timestamp and its corresponding wall-clock time,
using a 64-bit NTP encoding of time. RTP timestamps are supposed to 
be "nominal", not "actual" -- the way RTP systems are supposed to get
a handle on skew, sync, and other issues is to use these NTP/RTP pairs
in RTCP to maintain sync. We didn't use this in our simple "Appendix B"
algorithm, but a more sophisticated algorithm could ... 

> However, fixing the limits (the skew tolerance and the corresponding
> time limit) would be useful for the protocol implementation).

This is one possible way to proceed -- another is to take a look at
whether normative "late/ontime" semantics need to be in the MWPP I-D
at all. At Salt Lake, this was another issue that came up -- people
were skeptical how needed this normative text was, given that 
the method of deciding late/ontime was totally implementation dependent.

It might be better to simply state that sound playout is implementation
dependent, and offer non-normative references to the NOSSDAV paper, the
RTESP-Wedel.pdf, and perhaps others as examples for implementors to
consider ...