Subject: Re: [AVT] Working group last call: MIDI Payload Format and Implementation Guide

From: Magnus Westerlund <[email-masked]>
Date: Mon, 10 Jan 2005 19:04:13 +0100

Hi,

I have reviewed MIDI RTP payload format (version 07) and have the 
following comments. Please note that I didn't read Appendix A and B 
completely. I have to little MIDI knowledge to give any valuable 
comments on these. If you don't agree with comments, please raise this 
and provide some motivation why not. However there are a number of 
comments that needs to be discussed in the below.

I will see if I can get through the guidelines document tomorrow. I do 
know that I am late, however there is need that someone actually reads 
and comments on these.

It is also unfortunate that this payload format has become so huge. I 
really wished it was simpler and much shorter.

1. Section 1, third paragraph. I haven't seen anything in this payload 
format that makes it RTP profile specific. Therefore I would like to 
have removed or changed all occurances where it specifies that it is to 
be used under AVP. If you like to maintain the profile reference then 
make it as an example. I would suggest the following change in this 
sentence:

Was:
After a call is set up, audio data would flow between the parties using
the Real Time Protocol (RTP, [2]) under the Audio/Visual Profile (AVP,
[3]).

Change to:
After a call is set up, audio data would flow between the parties using
the Real Time Protocol (RTP, [2]) under any applicable profile, e.g. the 
Audio/Visual Profile (AVP, [3]).

In addition I think one can change basically all references to "RTP/AVP" 
to "RTP".

2. Section 1, last paragraph page 5:
"To provide a foundation for RTP MIDI applications, this memo extends 
two of the IETF tools (RTP and SDP) to support MIDI. Sections 2-5 and
Appendices A-B extend RTP/AVP by adding a MIDI payload format. Section
6 and Appendices C-D extend SDP by adding session configuration tools to
customize the stream behavior (including the MIDI rendering method)
during session setup."

As an RTP payload format extends neither the basic RTP functionality or 
SDP, I am against using the word "extends" in this context. I would 
instead propose that you change this paragraph to:

"To enable MIDI applications using RTP, this memo defines a RTP payload 
format and its media type. Section 2-5 and Appendices A-B defines the 
RTP payload format. Section 6 and Appendices C-D defines the media types 
identifying the payload format and all the parameters needed for 
configuration and how they are utilized in SDP."

3. Section 2.1: I am missing a explanation how to consider the SSRC in 
this payload format. If I understand correctly, the intention is to 
utilize multiple SSRCs for each block of 16 midi channels to transmit. I 
think it is important to discuss this here.

4. Section 2.1: Timestamp resolution. There is no discussion on the 
timestamp resolution issues that exist. Especially when it comes to high 
performance application.

5. Section 2.2, 5th paragraph on page 9:
"A sender MUST NOT knowingly create RTP MIDI UDP packets whose size 
exceeds the MTU of the underlying network."

I think the MUST should be changed to SHOULD, due to the problems of 
ensuring that it is actually followed. Also at least in IPv4 avoiding 
fragmentation is mostly an performance issues, rather than something 
breaks.

6. Same paragraph as 5:
"If these steps are not successful at limiting packet size to fit within 
an MTU, the session MUST be terminated."

I am not comfortable requiring that the session is terminated. What is 
the motivation behind this drastic measure? Also what are the thresholds 
for considering one unsuccessful?

7. Section 3: third paragraph on page 11:
"A MIDI list MAY consist a single delta time (Delta Time 0) without an
associated command (MIDI Command 0).  In this case, the MIDI list codes
the interval of time over which no commands appear in the stream."

I am not certain I understand this paragraph. So is the interval of time 
in which no commands are present: From TS to TS+Delta_Time_0? I think 
you need to explain this better. Also the "(Delta Time 0)" is a bit 
confusing. I misunderstood this at first to refer to a delta time value 
of 0 rather than the first value present in the list. Also some 
explanation what the benefit would be good.

8. Section 3.1: First paragraph on page 12:
"Note that delta time values may be
legally encoded in multiple formats; for example, there are four legal
ways to encode the zero delta time (0x00, 0x8000, 0x800000, 0x80000000)."

I think this example is in error, shouldn't they be 0x808000 and 
0x80808000?

9. Section 3.2:
"Although the first channel command in the MIDI list MUST include a
status octet, the corresponding command in the original MIDI source data
stream might not have a status octet (in this case, the source would be
coding the command using running status).  If the status octet of the
first channel command in the MIDI list does not appear in the source
data stream, the P (phantom) header bit MUST be set to 1.  In all other
cases, the P bit MUST be set to 0."

Must the packetization generate the status octet or is it allowed to 
excluded it if you have the P bit set to 1?

10. Section 3.2, page 15, the cancellation mechanism. I think there are 
a problem here.

"If a sender
determines that the command whose transmission is in progress should not
appear on the RTP stream, it cancels the command."
Which I interprets as you don't need to transmit more than up to the 
cancellation of the SYSEX.

Then you have the following text:
"A cancelled transmission SHOULD be ignored by the receiver, if 
possible. However, if a receiver has forwarded earlier command segments 
to a MIDI destination that is not able to ignore commands, the receiver 
SHOULD forward the data octets coded in the "cancel" sublist, and end 
the forwarded SysEx command with an 0xF7."

If you can ignore the cancel message, does not the complete message need 
to be transmitted then? Or is this something I am missing in the MIDI 
part here?

11. Section 5, page 22: The ordering of the bit list. Shouldn't the 
definition of the bits be equal to the order they appear in the packet? 
That would be easier to read.

12. Section 5, the definition of R and LENGTH. I think that these as 
they are commonly used should be defined in the 1.1 Terminology section.

13. Section 5, is the CHAN field also in MIDI encoded as channel 0 to 15 
so that they are directly mapped?

14. Section 6.1, bullet 5:
"If more than one minimal rtp-midi stream appears in a session,
      the MIDI name spaces for these streams are independent: channel
      1 in the first stream does not reference the same MIDI channel
      as channel 1 in the second stream."

What is the definition of "stream" in this sentence?

15. Section 6.1:
"By default, an RTP session, and thus a minimal RTP MIDI session, is
bidirectional ("sendrecv", as defined in [6]).  Each participant acts as
sender and receiver, as in traditional telephony."

The above sentence is only true in the SIP context. Other application 
context may not necessarily share this default value.

16. IANA section: I think it can be made more clear that this draft is 
registering a new mode for mpeg-generic. This repository is defined but 
not created by IANA, which will need to happen now.

17. IANA Consideration. I think it would be much better if the rules for 
registering values to the the extensible parameters are put into a 
subsection of the IANA consideration section. Also the initial statement 
on the action for IANA must be made clearer. It should list all the 
repositories that shall be created, and their initial values.

I would like to see something like:

IANA is kindly request to perform the following actions:

Register one MIME type "audio/rtp-midi" which defines 5 repositories for 
extensions:
    j_sec: Registrations in this repository is done according to section
           18.1.1.
           Initial values in the repository are:
           xyz : RFC XXXX, section x.y.z

    j_update:  ....

--- End of example ----
That way the actions to actually create the repositories will not be 
missed by IANA. Also any one registering things in the future can't miss 
the rules and how to do it.

17. Section 6.2, the AudioSpecifiConfig, how big can this become part 
become?

18. Section 6.2,
"If more than one minimal mpeg4-generic stream appears in
      a session, each stream uses an independent instance of the
      Audio Object Type coded in the config parameter value."

I don't understand this sentence. What is here meant with "stream" and 
how is the mapping between the different instances accomplished?

19. Section A.1, definition of N-active and C-active. Isn't there a way 
to define these command in a inclusive way, instead of exclusive way. I 
am especially concerned with future extensions and if they automatically 
should be included or excluded in these sets. However I don't at all 
understand what the intention with these definitions are, and what is 
the main properties of these. But that may be my lack of knowledge about 
MIDI.

20. Appendix C: Section title, I would like to remove "SDP" from it. The 
basic configuration is expressed as a series of parameter and values. 
These can for example be mapped to SDP, however in general we like to 
separate the basic requirement for configuration and therefore express 
it as Media Types parameters, and then map these to SDP. That way the 
work will be simpler when one take the step to another format for 
configuration.

21. Section C.1.1: 5th paragraph page 82:
"Special j_sec issues arise when sessions are managed by the Real Time
Streaming Protocol (RTSP, [17]).  In many streaming applications, the
session description in the response to the DESCRIBE method does not code
the transport details (such as UDP or TCP) for the session.  Instead,
server and client negotiate transport details using the SETUP method."

Similar behavior exist in other applications using declarative media 
configuration. This include MBONE application using SAP or 
pre-distributed SDP files. So I would recommend that you refer to the 
two cases as "Declarative usage" and "Negotiated usage". Where 
declarative usage can be exemplified with RTSP and SAP, while 
"Negotiated usage" is exemplified with Offer/Answer in SIP.

22. Section C.1.2.2, page 85:
"To address this issue, the closed-loop policy states that if a receiver
participates in a session where it may have access to a stream before
the sender is aware of the receiver, the receiver MUST take actions to
ensure that its rendered MIDI performance does not contain indefinite
artifacts.  The receiver MUST NOT discontinue these protective actions
until it is certain that the sender is aware of its presence."

How will a receiver be able to determine that the sender is aware of its 
presence? In any case where some other entity (multicast, conference 
focus, RTP translator/mixer) then the sender is distributing the traffic 
to multiple receivers, the receiver can't really determine if the sender 
has received its feedback.

23. Section C.1.2.2, page 85-86:
"At some point, the sender may need to take action in order to limit the
bandwidth of the stream.  The closed-loop policy states that if this
situation occurs, and if the nature of the session permits a sender to
stop transmitting packets to the offending receiver, the sender MUST
stop transmitting packets to this receiver.  In other words, it is not
permissible for a sender to no longer use M(k) in computing the
checkpoint packet identity but still send the stream to receiver k, if
it is possible for the sender to actively cut off receiver k from the
stream."

As the default method for feedback is RTCP, I think one should discuss 
the action when one times out an SSRC. To my understanding a timed out 
SSRC, does not need to be provided tailored recovery journals for.

24. Section C.1.2.2, page 83:
"The closed-loop policy relies on feedback from receiver to sender.  The
policy assumes that a receiver periodically informs the sender of the
highest sequence number it has seen so far in the stream, coded in the
32-bit extension format defined in [2].  In sessions that use RTCP,
receivers transmit this information in the Extended Highest Sequence
Number Received (EHSNR) field of Receiver Report (RR) packets.  However,
applications MAY use any method of feedback to implement the closed-loop
policy."

I think there is a need for a more strict requirement on RTCP in this 
case. Although transport events may result in the loss of RTCP feedback 
from a specific receiver, the receiver should really send feedback. Thus 
I think that the following formulation should be added.
"RTCP Sender or Receiver Reports SHALL be sent by any participant in a 
session with closed loop sending policy, unless other feedback mechanism 
has been agreed upon."

25. Section C.1.2.2, page 86:
"If the receiver detects this condition, it MUST leave the session, and 
close down the rendered MIDI performance in a manner that is free of
indefinite artifacts."

I do understand the need to stop the rendered MIDI performance, however 
I don't see why he must leave the session. The only way of getting back 
is hanging in there and try get his feedback through. Thus enabling him 
to get the necessary recovery information.

26. Section C.1.3, page 90: It is not explained how the special 
semantics is inserted into the channel list. Is there any restrictions 
on combination of special channel lists and the chapters?

27. Section C.1.3, page 93:
"To guard against indefinite artifacts, the MIDI Program Change command
and several MIDI Control Change controller numbers are assigned to
ch_anchor.  Note that the ordering of the ch_anchor chapter C assignment
after the ch_unused command acts to override the ch_unused assignment
for the listed controller numbers (7 and 64)."

I don't quite understand how "ch_anchor=C7.64;" can override something 
that is not included in the "ch_unused" list?

28. Section C.2.1:
"If comex is used to transcode MIDI time-of-arrival sources (such as 
MIDI 1.0 DIN cables [1]), the performance rendered from the MIDI stream 
may exhibit temporal jitter relative to the original source, as comex
semantics differ from MIDI time-of-arrival source semantics."

Is this temporal jitter a result of the inprecision of the RTP timestamp 
to correctly encode the execution time, or is it depending on something 
else. If it is depending on the conversion of the time scale, I think a 
note that the size of the jitter is dependent on the RTP timestamp rate 
should be added.

29. Section C.2.2:
"If the octpos parameter
does not appear in the media description, a timestamp MAY reflect the
time value of any octet of the command."

Is this no definition resulting in uncertainty a good thing. Why not 
default it to either value? Same applies to octpos in section C.2.3.

30. Section C.3.1:
"The RTP/AVP profile [3] permits audio packet media times to range from 
0 to 200 ms."

The implications of the usage of "permits" is a bit to strong. RFC 3551 
says in section 4.2 (Operating Recommendations): A receiver SHOULD 
accept packets representing between 0 and 200 ms of audio data.

This a recommendation not a fixed rule, and for formats for which it 
makes sense, even more media time may be used. So I would change this 
sentence from "permits" to "recommend".

31. Section C.4.2: The zerosync non-zero behavior does not work. For it 
to be useful it will need to encode all SSRCs TS offset compared to the 
reference SSRC.

32. Section C.4.2: the zero-behavior. I do not like it, as it further 
the view that this behavior as completely acceptable. I am not certain 
that it is sufficiently studied the effects of this usage. I do not have 
time to think of what potential issues this usage may produce beyond 
weakening ssecurity.

33. Section 5.2:
The "fasym" and "rasym" values indicate that the stream uses the client-
server semantics of MIDI 1.0 DIN network cable pairs.  The "fasym" value
indicates the forward flow from server to client; the "rasym value codes
the reverse flow from client to server.  In a session that uses client-
server semantics, the host that receives the "fasym" flow runs an
instance of the renderer, but the host that receives the "rasym" flow
does not.

I think the second sentence is a bit unclear, as the parameter is 
defined for the stream one receives if I interpret the first paragraph 
on the page. Also the parameters relation to the SDP directional 
attribute is unclear.

Also I am concerned with the definition implying that they also has 
capability declaring effects. What I have learned is that it is unwise 
to connect stream properties with capabilities. It is better to separate 
them so that one actually can express both at the same time and with 
different values. There are clearly benefits to being able to express 
that I will receive the rasym, but is also capable of symmetric and the 
reverse.

34. Section 5.3: Why is opaque render types allowed at all? Isn't it 
simpler to require the usage of a MIME type?

35. Section 5.4: For the "url" type, is any specific download protocols 
required to be implemented? If not interoperability seems to be at stake 
in many case, preventing usage of this method.

36. Section C.5: If I understand the implications of these parameters is 
that for a specific source to be able to use more than 16 channels with 
different renders, it must assign one payload type for each block of 16 
channels. Is this acceptable? I am considering the SDP size, and also 
complexity of configuring the MIDI renderers for each PT.

37. Section C.6: In general I think their should be more MUSTs in this 
section. All the tools necessary to get decent performance in a number 
of situations should be made mandatory to support. The problem with 
SHOULD in a interoperability specification is that it prevents the 
clients from being sure that the parameter will be understood. Thus even 
more code will go to handle these cases and work arounds.

38. Comment around guard time. If one uses RTCP, a receiver will 
continue to get RTCP reports from the SSRC that prior sent packets. Thus 
as long as this happens it is possible to determine that this sender is 
still present, but not actively sending for the moment.

39. ABNF check of syntax in section D:
I used http://www.apps.ietf.org/abnf.html
to produce this results.

undefined rule: hex-octet
undefined rule: type
undefined rule: uri
undefined rule: msg-id
undefined rule: subtype
undefined rule: token
undefined rule: base64
ABNF validation (version 1.0) completed

Of these, at least "msg-id" is lacking a complete ABNF definition that 
is easily implementable. Can't you define this in a more explicit way? 
Especially considering that you define +1 operations on it.

40. Section D. Comment on "rinit" def.
I think the rinit MIME type needs to have room for parameters that a 
MIME type may have.

41. Section H. "rate" parameter for audio/rtp-midi
Please change the definition from being dependent on the rtpmap line.

42. Section H. The Author / Change Controller:
This section should be split into two headings. Put yourself under 
"Author" and Change Controller should be written as follows:
"IETF Audio/Video Transport Working Group delegated from the IESG"

43. The audio/asc type, that is nothing that MPEG has defined is it? 
This format, both its definition and name is done in this draft?

Cheers

Magnus

-- 

Magnus Westerlund

Multimedia Technologies, Ericsson Research EAB/TVA/A
----------------------------------------------------------------------
Ericsson AB                | Phone +46 8 4048287
Torshamsgatan 23           | Fax   +46 8 7575550
S-164 80 Stockholm, Sweden | mailto: [email-masked]

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

---

From: lazzaro <[email-masked]>
Date: Mon, 10 Jan 2005 11:02:34 -0800

On Jan 10, 2005, at 10:04 AM, Magnus Westerlund wrote:
> following comments.

Thanks for the comments.  Realistically, I don't think point-by-point
postings to AVT are an efficient way to resolve 43 issues.  So as I did
with contrans, I'll do a -08 of these two documents, and address each
of your points in the Change Log.  Ideally, if you could turn the 
guidelines
document around soon, that would be great -- otherwise, there will 
probably
be a -09 for both documents to address your guidelines comments.

I'd also encourage anyone else in WG reading through the I-Ds
to send in your comments by the 11th deadline as well (or soon
thereafter), so I can address all remaining objections in one set
of document revisions.  I'd prefer this over taking a serial approach,
with WG last call after WG last call, each devoted to one person's
comments.

---
John Lazzaro
http://www.cs.berkeley.edu/~lazzaro
lazzaro [at] cs [dot] berkeley [dot] edu
---

---

From: Magnus Westerlund <[email-masked]>
Date: Tue, 11 Jan 2005 08:39:29 +0100

Hi John,

I also strongly suggest that anyone with an interest perform their 
review now, rather than later. The WG last call is intended to be the 
last chance to comment and a ensuring that the WG support the document. 
Therefore it is important to provide comments, even if they simple are: 
"I read it, it looks fine."

I would also note that it isn't required to do a second WG last call 
after one has been done. We usually do if there is substantial comments. 
However if there are no or only minor technical changes, there is 
usually just a short review period of the changes and we request the 
document to be published.

However back to my comments. There are a number of comments, that to 
some degrees question. I would be thankful if you separated these out 
and responded to them. Preferably one mail per question/issue.

I will raise my biggest concern in a separate mail.

Cheers

Magnus

-- 

Magnus Westerlund

---

From: lazzaro <[email-masked]>
Date: Tue, 11 Jan 2005 12:59:43 -0800

> and responded to them. Preferably one mail per question/issue.

OK, I'll do that now.  Each will have a subject line that begins with 
"RTP MIDI Issue: "

---
John Lazzaro