Subject: [AVT] draft-ietf-avt-rtp-midi-07: (Issue 36) Multiple channel blocks

From: Magnus Westerlund <[email-masked]>
Date: Tue, 11 Jan 2005 11:04:13 +0100

Hi John and the WG.

What to me appears to be the biggest issue is how the RTP payload format 
handles multiple blocks of 16 channels, i.e. MIDI name spaces. My 
current understanding after having read the document is that each new 16 
midi namespace requires a new RTP payload format (Issue 3 and 36) which 
is identified with the musicport parameter. My initial understanding in 
Issue 3 seem to be wrong, however the usage of SSRC in this payload 
format should be further explained.

So what one can start asking is how often more than a single MIDI name 
space is needed. It seems that they in certain use cases can be quite 
common. In a streaming session of a SMF file, they do appear to be less 
likely.

With the current mechanism specified, the usage of multiple requires 
each name space to use its own RTP payload type (PT). This include 
complete configuration information for each midi name space. That midi 
namespace is then shared between all participants in the session. So, is 
the configuration information needed for each midi namespace and how big 
parts of it is actually shared between name spaces.

Would it be a better to include in the RTP payload format a identifier 
for the namespace, either for the complete RTP payload, for each command.

John, can you please share your design reasoning behind this?

Cheers

Magnus Westerlund

Multimedia Technologies, Ericsson Research EAB/TVA/A
----------------------------------------------------------------------
Ericsson AB                | Phone +46 8 4048287
Torshamsgatan 23           | Fax   +46 8 7575550
S-164 80 Stockholm, Sweden | mailto: [email-masked]

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

---

From: lazzaro <[email-masked]>
Date: Tue, 11 Jan 2005 14:35:34 -0800

On Jan 11, 2005, at 2:04 AM, Magnus Westerlund wrote:
> So what one can start asking is how often more than a
> single MIDI name space is needed.

Rarely.  A MIDI DIN cable codes one name space, and an
SMF track codes one name space.  There is no standard
way to code a name space identifier on a MIDI DIN cable
(non-standardized methods exist).  I am not certain, but I
believe the mechanism for coding a name space identifier
for an SMF track is also non-standard -- I do not recall seeing
a "MIDI Port" meta-event in an MMA document, and I have
read most of the MMA documents.

16 MIDI channels (i.e. one MIDI name space) code a lot of
instruments, especially since all percussion instruments usually
share 1 MIDI channel, and each drum gets mapped to one key
of the channel.

So, multiple name spaces are a specialist feature, that we
include for use by high-end applications. As an analogy,
video conferences that use several cameras are rare (so
far) and thus, to my knowledge, most RTP video formats do
not multiplex several cameras onto a single payload format.
That's the right comparison -- not the individual sound channels
of a stereo or 5.1 audio RTP stream.

> Would it be a better to include in the RTP payload format a
> identifier for the namespace, either for the complete RTP
> payload, for each command.

If the format were designed in this way, each RTP packet in
the stream would need to include the recovery journal for
each MIDI name space.  This is necessary because if a
receiver sees a sequence number break, it has no way of
knowing which MIDI name spaces were coded in the lost
packets.  So, it needs to do recovery for each of the name
spaces, and so it needs the recovery journal for each name
space in the packet that ends the sequence number break.

One might say "maybe the sender could figure out which
journals need to be included for each packet", but this
approach does not work in practice -- the sender cannot
know which packets were lost en route to receivers,
apart from points in the stream where an RTCP packet
has been received with a "last extended sequence number
received" field.

Multiple recovery journals per packet (one per name space)
have bad side effects (a few are listed below):

[1] The maximum packet size in a stream grows, and for
streams with a few dense name spaces could easily bump
into the smaller MTU sizes.

[2] The bandwidth of the stream needlessly grows

[3] Guard packet algorithms (see the guidelines document)
become harder to conceptualize and implement.

[4] Receivers need to do more recovery work after
each packet loss, in the general case.

There are other practical problems too -- the protocol would
need a bit-field redesign, as the need for a multiplexing point
arises in several places. New coding and packing problems
would appear and would need fresh solutions ... etc.

---
John Lazzaro
http://www.cs.berkeley.edu/~lazzaro
lazzaro [at] cs [dot] berkeley [dot] edu
---

---

From: Stephen Casner <[email-masked]>
Date: Tue, 11 Jan 2005 22:43:30 -0800 (PST)

> of a stereo or 5.1 audio RTP stream.

But many times we have talked about one participant having multiple
video cameras, such as one for the face and one for objects or
documents.  In these scenarios, each camera generates its own stream
of RTP packets distinguished by a different SSRC identifier.

                                                        -- Steve

---

From: Magnus Westerlund <[email-masked]>
Date: Wed, 12 Jan 2005 11:24:37 +0100

Hi John,

Please continue this discussion so that we can determine if the way you 
have selected is the most appropriate, or if there is some much better 
or necessary way. We do definitely need a definition for the SSRC as it 
appears to be a fundamental part of the picture.

As Steve says, RTP has a common way of introducing multiple sources with 
the SSRC, however as the MIDI RTP payload format is mostly a wire 
replacement, this is not an obvious fit for the model. As I understand, 
a MIDI DIN cable would in a basic case only connect two devices. But a 
colleague told me there exist cases where you connect multiple renders 
and configure each renderer to know which channels to listen to. However 
with RTP you get the possibility with a shared cable that has up to 4 
billion senders and receivers. That is why there is need to define the 
behavior in regards to SSRCs.

So is the intended behavior in regards to SSRC, that each sender can 
generate MIDI commands, that is processed by all receivers according to 
their render configuration. As different senders will use different 
SSRCs, there is the possibility for multiple timing spaces that needs to 
be synchronized and that simultaneously use the same midi channels. Are 
there any special problems with this, as it may exist multiple sources 
for commands which doesn't appear to be a normal case in midi. With this 
definition there may be further work on the recovery journal, as the 
status of a channel is not any longer guaranteed to be the result of a 
single source.

An alternative usage of SSRC would be that each SSRC defines its own 
MIDI name space. That way we ensure that there is no risk for collisions 
of commands within the same name space. That SSRC is also solely 
responsible for the content of the recovery journal. However this usage 
would prevent any responses at all within the same name space, i.e. the 
commands become strictly unidirectional unless extra support is included 
in the RTP payload format for this. So is bi-directional transmission 
within the same name space required (If yes, then how common are they 
and what are they used for)? Multiple payload types would only need to 
be use if different render configurations are necessary. However it 
would place a requirement on the receiver/render to handle multiple SSRC 
if they occur.

In the first case the handling of the different name spaces would need 
to be handled in some other way. There are a few possibilities:
- Require different RTP session, which has high overhead in terms of 
system resources and signalling.
- Use the PT, which also generate overhead, mostly in the signalling 
space. Limits the number of midi name spaces per session.
- Define it in the RTP payload format, which generates overhead in each 
packets.

Which is the most appropriate depends mostly on the need to handle 
multiple channels. If the renders are usually configured differently for 
different name spaces, there is still need for different configuration 
information through multiple PTs or sessions. If multiple name spaces 
are seldomly used and also usually few as you indicate, there is may 
also be acceptable to use the PT field for this, however it is a 
multiplexing point which main usage is to determine the payloads type 
and its processing requirements.

See also comments on the side effects below

> into the smaller MTU sizes.

Yes, the possibility to disperse multiple name spaces into different 
packets if aggregation multiple command lists and recovery journals 
results in larger than MTU size payloads would be needed.

> [2] The bandwidth of the stream needlessly grows
I don't understand how it needlessly grows. If you need to send multiple 
name spaces in a payload you will of course have more data to send, but 
that comes from the application, not the protocol design. There will be 
an increased overhead, however that would be the price to support this 
style of multiplexing of the name space.

> each packet loss, in the general case.

Yes, as commonly in RTP payload formats, the more you aggregate the more 
you lose. However the trade off between aggregation of data and 
robustness, is something that should be dynamically adjustable in a 
sender application as it is a main tool for congestion control in RTP 
applications.

In conclusion, lets try to nail down how the SSRC is used first. It 
seems to be an important factor in the puzzle if the PT is to be used 
for name space separation or not.

Cheers

Magnus Westerlund

---

From: lazzaro <[email-masked]>
Date: Wed, 12 Jan 2005 11:08:24 -0800

On Jan 12, 2005, at 2:24 AM, Magnus Westerlund wrote:
> Please continue this discussion so that we can determine if the way 
> you have selected is the most appropriate, or if there is some much 
> better or necessary way.

Let's put this discussion on hold until March or April.  I'm
teaching an undergraduate course starting next week, and
I need to devote my energies to the course for the next
few months -- I was not expecting such a fundamental
lack of consensus to appear so late in the process, and
I haven't budgeted time for it.  To try to continue now will
put the course in danger, and at Cal students come first.

For the next few months, I'll have enough cycles to
help with getting contrans through the system, but
that's about it ...

---
John Lazzaro