Subject: Re: [MMUSIC] [Technical Errata Reported] RFC2326 (4199)

From: Magnus Westerlund <[email-masked]>
Date: Tue, 27 Jan 2015 11:35:51 +0100

Hi,

Regarding this errata I recommend that it is being rejected. The reason
is that is is technical change that can cause compatibility issues, and
further does not resolve the issue as it is missing the required
escaping definitions (another technical change). In addition we know
that RFC 2326 is quite buggy and attempting to fix it with Errata does
not work.

However, as this issue is retained in RTSP 2.0 I like to comment on it
in that context. Thus, the question becomes should we fix this in RTSP 2.0?

Both version of RTSP allows binary non-escaped data in two places. The
message-body as well as the data portion of the Embedded binary data.
These parts can clearly contain a octet value of decimal 36 ("$"). Thus
these must be skipped by any parser using the provided length fields.
Thus, a very simple parser based on just looking for $ in a byte stream
will not work anyway. The parser must know what part of the message
structures it is in to be able to correctly parse and consume incoming
bytes over the RTSP connection.

Thus a parser must work on message basis. Which means that requests,
responses and embedded data must be possible to separate. This must be
done on the beggining of each chunk. Response starts with RTSP/2.0 and
can't contain $. Requests starts with the method name. Unfortunately the
ABNF is not explicit on forbidding a method name to start with a $.
However, the text in Section 13 is. We should clearly consider fixing
the ABNF so that it is explicit also in the ABNF.

However, taking the proposed step and moving $ into the reserved
category of characters will have much more significant impact. For
example, it will result in an URI format that is more restricted than
the general one. I haven't analysed the full impact of this. Nor have I
looked through all the headers which may contain values to see what
impact this has.

However, the current specification does not prevent correct separation
of RTSP messages from interleaved. It does require a message aware
parser and that parser could be tripped by malformed messages. If this
issue had been raised early in the RTSP 2.0 work, we could have resolved
it to a much greater satisfaction and been able to make the protocol
more robust.

Cheers

Magnus

On 2014-12-12 00:48, RFC Errata System wrote:
> The following errata report has been submitted for RFC2326, "Real
> Time Streaming Protocol (RTSP)".
> 
> -------------------------------------- You may review the report
> below and at: 
> http://www.rfc-editor.org/errata_search.php?rfc=2326&eid=4199
> 
> -------------------------------------- Type: Technical Reported by:
> Julius Friedman <[email-masked]>
> 
> Section: 4
> 
> Original Text ------------- 4 RTSP Message
> 
> RTSP is a text-based protocol and uses the ISO 10646 character set
> in UTF-8 encoding (RFC 2279 [21]). Lines are terminated by CRLF, but 
> receivers should be prepared to also interpret CR and LF by 
> themselves as line terminators.
> 
> Text-based protocols make it easier to add optional parameters in a 
> self-describing manner. Since the number of parameters and the 
> frequency of commands is low, processing efficiency is not a concern.
> Text-based protocols, if done carefully, also allow easy 
> implementation of research prototypes in scripting languages such as
> Tcl, Visual Basic and Perl.
> 
> The 10646 character set avoids tricky character set switching, but is
> invisible to the application as long as US-ASCII is being used. This
> is also the encoding used for RTCP. ISO 8859-1 translates directly
> into Unicode with a high-order octet of zero. ISO 8859-1 characters
> with the most-significant bit set are represented as 1100001x
> 10xxxxxx. (See RFC 2279 [21])
> 
> RTSP messages can be carried over any lower-layer transport protocol 
> that is 8-bit clean.
> 
> Requests contain methods, the object the method is operating upon
> and parameters to further describe the method. Methods are
> idempotent, unless otherwise noted. Methods are also designed to
> require little or no state maintenance at the media server.
> 
> Corrected Text --------------
> 
> 
> 4 RTSP Message
> 
> RTSP is a text-based protocol and uses the ISO 10646 character set
> in UTF-8 encoding (RFC 2279 [21]). Lines are terminated by CRLF, but 
> receivers should be prepared to also interpret CR and LF by 
> themselves as line terminators.
> 
> Text-based protocols make it easier to add optional parameters in a 
> self-describing manner. Since the number of parameters and the 
> frequency of commands is low, processing efficiency is not a concern.
> Text-based protocols, if done carefully, also allow easy 
> implementation of research prototypes in scripting languages such as
> Tcl, Visual Basic and Perl.
> 
> The 10646 character set avoids tricky character set switching, but is
> invisible to the application as long as US-ASCII is being used. This
> is also the encoding used for RTCP. ISO 8859-1 translates directly
> into Unicode with a high-order octet of zero. ISO 8859-1 characters
> with the most-significant bit set are represented as 1100001x
> 10xxxxxx. (See RFC 2279 [21])
> 
> RTSP messages can be carried over any lower-layer transport protocol 
> that is 8-bit clean.
> 
> Requests contain methods, the object the method is operating upon
> and parameters to further describe the method. Methods are
> idempotent, unless otherwise noted. Methods are also designed to
> require little or no state maintenance at the media server.
> 
> *Please note the hexadecimal octet 0x36 should not be included in any
>  part of a RTSP message, if present it should be replaced with binary
>  escaped sequence as defined in: <consensus>*.
> 
> Notes ----- Section [15.1 Base Syntax] makes the following
> definitions: .. safe               =  "$" | "-" | "_" | "." | "+" .. 
> reserved           =  ";" | "/" | "?" | ":" | "@" | "&" | "=" extra
> =  "!" | "*" | "$'$" | "(" | ")" | "," unreserved         =  alpha |
> digit | safe | extra xchar              =  unreserved | reserved |
> escape
> 
> However it doesn't explicitly state the association of "$"
> [hexadecimal 0x24]
> 
> Section [10.12 Embedded (Interleaved) Binary Data] specified a
> mechanism of transferring RTSP messages over a TCP connection with a
> application layer header consisting of the hexadecimal octet 0x24;
> followed by a single octet channel identifier and the RFC4571 length
> specifier of the frame.
> 
> Due to the fact 0x24 violates section 4's requirements based on the
> fact it cannot be masked with the octet (AND 0xc3) to ensure it is a
> valid character I am filing this errata.
> 
> The chosen octet may also be present, specifically in the types of
> RTSP messages which are interleaved during playback such as ANNOUNCE.
> E.g. 0x24 could be a part of the configuration specifying the bits
> per pixel or audio bit depth as part of the SDP inter alia.
> 
> In such cases the underlying parser would interpret this octet as the
> start of the defined application header and cause framing errors
> which could cause data loss as allow for buffer overflow attacks of
> carefully crafted binary data.
> 
> There are also definitions which need to be made in the stadard for
> what to do when less then the amount of bytes indicated by the
> application layer frame header are or are not present. (e.g. more or
> less bytes are required then are present during reception of the
> given the [$CXX] application header sequence).
> 
> It is based on these considerations among others that I recommend
> that "$" [hexadecimal octet 0x24] be added to the reserved list and
> if necessary an escaping mechanism be defined where it is replaced
> with an escaped sequence which is agreed upon after consensus.
> 
> Due to the fact that drafts also specify that RTSP can be extended
> with any type of message so long as the data is not "$" the first
> character it is ambiguous may also appear elsewhere in the status
> line.
> 
> There are also issues with the ambiguous definition of pdu
> fragmentation defined in the latest draft, e.g. the mechanism defined
> as possibly required when pdu's larger then 65535 bytes are used
> however it specified no way to convey how this occurs or why.
> 
> The same can also be said for "sink" and "source" however I will
> address those definitions et al when appropriate.
> 
> Thank you for your time!
> 
> Instructions: ------------- This erratum is currently posted as
> "Reported". If necessary, please use "Reply All" to discuss whether
> it should be verified or rejected. When a decision is reached, the
> verifying party (IESG) can log in to change the status and edit the
> report, if necessary.
> 
> -------------------------------------- RFC2326 (no draft string
> recorded) -------------------------------------- Title
> : Real Time Streaming Protocol (RTSP) Publication Date    : April
> 1998 Author(s)           : H. Schulzrinne, A. Rao, R. Lanphier 
> Category            : PROPOSED STANDARD Source              :
> Multiparty Multimedia Session Control Area                : Real-time
> Applications and Infrastructure Stream              : IETF Verifying
> Party     : IESG
> 
> _______________________________________________ mmusic mailing list 
> [email-masked] https://www.ietf.org/mailman/listinfo/mmusic
> 

-- 

Magnus Westerlund

----------------------------------------------------------------------
Services, Media and Network features, Ericsson Research EAB/TXM
----------------------------------------------------------------------
Ericsson AB                 | Phone  +46 10 7148287
Färögatan 6                 | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden | mailto: [email-masked]
----------------------------------------------------------------------

---

From: [email-masked] (Dale R. Worley)
Date: Thu, 29 Jan 2015 22:18:08 -0500

Taking a first look at the erratum, two things occur to me:

The erratum quotes a large chunk of the text of section 4 in both the
"says" and "should say" sections.  IMO the duplicated text should have
not been put in either, and instead just the changed part, the insertion
of:

should have been mentioned in "should say".

The second item is that the proposed addition clearly is not what was
intended.  The text says "the hexadecimal octet 0x36", which is the
ASCII code for the digit "6".  What was intended was "the octet 36",
which is the ASCII code for the character '$'.

Dale

---

From: Julius Friedman <[email-masked]>
Date: Fri, 30 Jan 2015 13:42:47 -0500

Magnus, Et al,

Then Please, adjust what you / we can control which is the Binary
Interleaved Data.

One simple way would be to at least ensure that the application framing
lengths can be conveyed in a manner consistent with the underlying
transport *AND* with respect to any Blocksize in use on the session.

E.g.

A binary interleaved frame MAY NOT contain a frame size larger then any
Blocksize in use.

A binary interleaved frame MAY NOT contain less than 12 bytes for RTP or
RTCP.

A binary interleaved frame MUST be verified by using the packet header
fields e.g. (sequence number, timestamp, ssrc)

Instead of skipping the data what should happen is that the data propagated
to the lower layer transport in chunks.

Skipping data is causing the problem which is also defined as an
instruction when a packet is received for an unknown context, which should
again be handled by propagating the data to the lower layer.

I put this logic in my RtpClient implementation and it works well, and
never becomes unframed unless the client intentionally sends a large packet
which hides data inside (which should be related to the packet anyway)

https://net7mma.codeplex.com/SourceControl/latest#Rtp/RtpClient.cs

You will see in my unit tests I can receive all of the messages which
overflow most other client implementations cannot due to framing errors.

Using the information given and if needed my code as a reference an
implementation can make much better choices and actually never become
unframed and if they do recovering is always possible.

As a take way from this and final note I suppose that someone has also
looked at using independent TCP on a session which also uses Binary
Interleaved data transport, either for RTSP 2 or 1 if not I will ask anyway.

The reason for this question is I am revising some parts of my server and
client to be more dependent on a 'session' and that session is typically
tied to a underlying 'Transport'..

My question is

"What happens when a client signals in a single session that it wants to
use TCP Interleaved on one media item and Independent on another"

The result for my implementation is that if the end points overlap then the
framing will contain two additional bytes...

How is this being handled or how SHOULD it be so I can adjust my
implementation to allow for it as such.

I can see how it would work if the end points didn't overlap, the protocol
wouldn't have the RTSP Binary Interleaved header for the media flows using
the independent transport, correct?

But if and when they DO how is that handled?

Should it not be allowed that Binary Interleaved Transfer and Independent
transfer occur in the same session?

If that is the case then what about Binary (TCP/RTP/AVP) and (UDP/)RTP/AVP,
can they occur in the same session? If not why and where does one make that
determination given the information in the RFC either 2326 or BIZ).

I allow for this in my implementation as well and there is no issue (but
caused me to notice that the Session is still active when TEARDOWN occurs
and hence why I filled the other erratum).

Thanks!

On Fri, Jan 30, 2015 at 5:41 AM, Magnus Westerlund <
[email-masked]> wrote:

> Hi,
>
> I would still prefer that this discussion was done on the mailing list.
>
> The content of the Errata is formal broken. But, lets us not care about
> that.
>
> As I have written before, I still don't see how we can make the change
> Julius proposes even to RTSP 2.0. The core of the proposal to address
> this would make so that $ can only occur as starting point of an
> interleaved binary datagram. That appears impossible to me as the URI
> format for example do allow $ to occur in various parts. Putting in
> escaping at all these places at this stage will also be problematic.
>
> Yes, it is sub-optimal it will force clients or server to close the RTSP
> message transport if they become out of synch with message boundaries.
>
> So, in conclusion, I don't see how we can make any corrections or
> improvements on this.
>
> Cheers
>
> Magnus
>
>
> On 2015-01-30 04:30, Julius Friedman wrote:
> > I wrote in about that in a later email.
> >
> > Do you need my comments on that again?
> >
> > This issue is real and does need to be addressed,  either by using
> > hexadecimal notation in all header fields or otherwise by checking the
> > data using the timestamp and sequence numbers of the packets.
> >
> > Invalid packets (length or otherwise) and packets with no context are
> > also issues I ha e brought up.
> >
> > This is much worse when independent TCP is used if used with interleaved
> > transport in 1.0 as allowed because even though $ was the worst choice
> > based for a framing value it allows something better then what is
> > required when NO framing at all is used and every read has to be
> > verified to be in sync using the protocol level fields such as timestamp
> > and sequence numbers.
> >
> > I have already addressed this in my software as well if reference is
> > needed.
> >
> > Thanks,
> > Julius
> >
> > On Jan 29, 2015 10:18 PM, "Dale R. Worley" <[email-masked]
> > <mailto:[email-masked]>> wrote:
> >