Subject: Re: [AVT] draft-schierl-avt-rtp-multi-session-transmission-00.txt

From: "Ingemar Johansson S" <[email-masked]>
Date: Thu, 13 Nov 2008 16:27:12 +0100

Hi

I have a comment on draft-schierl-avt-rtp-multi-session-transmission-00.txt

=============
7.2.6.  RTP header extension

   The RTP header extension may be used to add generic signaling about
   Data Alignment to RTP packets.

7.2.6.1.  Identified problems

   RTP header extensions are required to be ancillary information which
   can safely be discarded by receivers which do not understand them.
   Data alignment mechanisms do not satisfy this requirement.
=============

My question is ..how serious is the identified problem ?. 
My gut feeling is that a receiver that implements e.g decoding of G.718 content will likely update the RTP stack to recognize header extensions if header extensions are used to carry e.g the NTP timestamp related to the RTP timetstamp in the RTP header. 
I see many benefits with using the RTP header extension for this purpose: 
1) It is relatively compact 
2) Easy to make generic (I would believe that the NTP is all info that is needed)
3) The clock-skew issue is easily solved as you don't need to schedule the transmission of the header extension like for RTCP-SR. For instance for a layered audio codec you typically add the header extension for all layers corresponding to the same exact time instant.

Regards
Ingemar
******************************************* 
Ingemar Johansson 
Senior Research Engineer, IETF "nethead" 
EAB/TVP - Multimedia Technologies 
Ericsson Research Ericsson AB 
Box 920 S-971 28 Luleå, Sweden 
Tel: +46 (0)8 4043042 
ECN: 850-43042 
Mobile: +46 (0)730 783289 
******************************************* 
_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www.ietf.org/mailman/listinfo/avt

---

From: <[email-masked]>
Date: Thu, 13 Nov 2008 18:31:54 +0200

I tend to agree with Ingemar that the problem of the RTP header extension mechanism is not serious. If the RTP header extension mechanism is based on decoder order numbers of coded data units, then I think it is really generic and applies to any layered or multi-source codec, including flexible scalable video codecs such as SVC.

BR, YK 

>-----Original Message-----
>From: [email-masked] [mailto:[email-masked]] On 
>Behalf Of ext Ingemar Johansson S
>Sent: 13 November, 2008 17:27
>To: [email-masked]
>Subject: Re: [AVT] 
>draft-schierl-avt-rtp-multi-session-transmission-00.txt
>
>Hi
>
>I have a comment on 
>draft-schierl-avt-rtp-multi-session-transmission-00.txt
>
>
>=============
>My gut feeling is that a receiver that implements e.g decoding 
>of G.718 content will likely update the RTP stack to recognize 
>header extensions if header extensions are used to carry e.g 
>the NTP timestamp related to the RTP timetstamp in the RTP header. 
>3) The clock-skew issue is easily solved as you don't need to 
>schedule the transmission of the header extension like for 
>RTCP-SR. For instance for a layered audio codec you typically 
>add the header extension for all layers corresponding to the 
>same exact time instant.
>
>Regards
>Ingemar

---

From: Randell Jesup <[email-masked]>
Date: Thu, 13 Nov 2008 13:51:09 -0500

<[email-masked]> writes:
>I tend to agree with Ingemar that the problem of the RTP header extension
>mechanism is not serious. If the RTP header extension mechanism is based
>on decoder order numbers of coded data units, then I think it is really
>generic and applies to any layered or multi-source codec, including
>flexible scalable video codecs such as SVC.

Ingemar Johansson writes;
>>draft-schierl-avt-rtp-multi-session-transmission-00.txt

What if an intermediary deletes the header extension?  (SBC, B2BUA,
raw message store, etc).

This isn't to say it might not be a good idea - but requiring header
extensions to pass through may not be a good idea.  You also have issues
where you can't use multiple header extensions in the same RTP packet, so
moving too much to them is a problem.

-- 
Randell Jesup, Worldgate (developers of the Ojo videophone), ex-Amiga OS team
[email-masked]
"The fetters imposed on liberty at home have ever been forged out of the weapons
provided for defence against real, pretended, or imaginary dangers from abroad."
		- James Madison, 4th US president (1751-1836)

---

From: Thomas Schierl <[email-masked]>
Date: Fri, 14 Nov 2008 07:52:22 +0100

Let us go a step further. If such information for an "enhanced"
decoding order recovery mode may be discarded, there would have to be a
fall back. All applications relying on such mechanism would also have
to support  the time stamp based solution.

--
Thomas Schierl
--------------
Fraunhofer HHI

Randell Jesup wrote:

I have a comment on

=============

----

Visit us at

MEDICA 2008 / Duesseldorf, Germany / 19 - 22 November 2008 / Hall 16, Booth D55

http://www.medica.de

SOCCEREX 2008 / Gauteng, South Africa / 23 - 26 November 2008 / Booth V1-B1 a

http://www.soccerex.com

---

From: "Ingemar Johansson S" <[email-masked]>
Date: Fri, 14 Nov 2008 09:39:18 +0100

Hi

Comments inline below 

Regards 
Ingemar

> From: Randell Jesup [mailto:[email-masked]] 
> Sent: den 13 november 2008 19:51
> To: [email-masked]
> Cc: Ingemar Johansson S; [email-masked]
> Subject: Re: [AVT] 
> >I tend to agree with Ingemar that the problem of the RTP header 
> >extension mechanism is not serious. If the RTP header extension 
> >mechanism is based on decoder order numbers of coded data 
> units, then I 
> >think it is really generic and applies to any layered or 
> multi-source 
> >codec, including flexible scalable video codecs such as SVC.
> 
> 
> >>My gut feeling is that a receiver that implements e.g decoding of 
> >>G.718 content will likely update the RTP stack to recognize header 
> >>extensions if header extensions are used to carry e.g the NTP 
> >>timestamp related to the RTP timetstamp in the RTP header.
> 
> What if an intermediary deletes the header extension?  (SBC, 
> B2BUA, raw message store, etc).
There is of course a risk that this may happen. 
However I am curious as to what extent this may actually happen as it
would also cause the authentication to fail in case SRTP is used.

> 
> This isn't to say it might not be a good idea - but requiring 
> header extensions to pass through may not be a good idea.  
> You also have issues where you can't use multiple header 
> extensions in the same RTP packet, so moving too much to them 
> is a problem.
Ok. A fallback is always to use RTCP SR (which is sent anyway), the
convergence will be slower and clock-skew compensation will complicate
an implementation somewhat but it will anyway be a fallback solution.
The additional method (if possible to use) will speed up convergenge.
As for the problem with multiple header extensions, I would not believe
that you need to add the "timing alignent" header extension in every
packet. Also I believe it is up to the sender to prioritize between the
header extensions.

> 
> --
> Randell Jesup, Worldgate (developers of the Ojo videophone), 
> ex-Amiga OS team [email-masked] "The fetters imposed on 
> liberty at home have ever been forged out of the weapons

---

From: Randell Jesup <[email-masked]>
Date: Fri, 14 Nov 2008 10:38:33 -0500

>would also cause the authentication to fail in case SRTP is used.

SRTP doesn't use header extensions.  (I think ZRTP was doing so, but
I don't know if they kept that or if there were alternatives, and I
think ZRTP just would consider that a case where the channel couldn't be
secured if there's no alternative channel (RTCP?) for key exchange.)

There's no easy way to know (except maybe by asking the ZRTP people) how
often this happens. Certainly Asterisk and things like it would be a major
blocker to this (they act as kind-of B2BUA's -- don't get me started on
what a mess their implementations are).  I haven't seen a true SBC that
stripped them - but on the other hand I haven't used header extensions
except occasionally.

>packet. Also I believe it is up to the sender to prioritize between the

That does cover the requirement that header extensions can't be mandatory,
though the details and consequences of the fallback need the be worked out.

Adding it to every packet is a big no-no as mentioned.  Which packets would
it need to be added to, taking into account packet loss, startup, etc?

The scheduling issues may mean that these extensions don't get transmitted
for a while, but another issue is that the inserters of the extensions
aren't always coordinated -- witness ZRTP which in the current
implementation acts as a wrapper layer - if the main RTP stack adds an
extension, that means that ZRTP can't use that packet for an extension.
This should be ok, and ZRTP has to account for this possibility (as well as
packet loss).

---

From: Thomas Schierl <[email-masked]>
Date: Sat, 15 Nov 2008 12:39:37 +0100

I think it is nice to go into a deeper discussion on a particular
solution, but we should first solve the following questions:

"Do we need a new solution or an extension to the existing mechanism?
And, are the arguments collected in the draft convincing reasons for
going that way?"

Let us assume one special problem goes away - the delay in presence of
multiple receivers allowing bidirectional communication (see:
draft-perkins-avt-rapid-rtp-sync-00).

Thanks,

Thomas

--
Thomas Schierl
--------------
Fraunhofer HHI

Randell Jesup wrote:

----

Visit us at

http://www.medica.de

http://www.soccerex.com

---

From: <[email-masked]>
Date: Sat, 15 Nov 2008 15:57:46 +0200

>What if an intermediary deletes the header extension?  (SBC, B2BUA,

What are the (good) reasons for an intermediary to delete header
extensions? Isn't so that if there is something an intermediary does not
understand, it'd better not to touch the something?

BR, YK 

>From: ext Randell Jesup [mailto:[email-masked]] 
>Sent: 13 November, 2008 20:51
>To: Wang Ye-Kui (Nokia-NRC/Tampere)
>Cc: [email-masked]; [email-masked]
>Subject: Re: [AVT] 
>ex-Amiga OS team

---

From: "Ingemar Johansson S" <[email-masked]>
Date: Sun, 16 Nov 2008 13:27:00 +0100

Hi

Comments below
Regards
Ingemar
> Sent: den 14 november 2008 16:39
> To: Ingemar Johansson S
> Cc: [email-masked]; [email-masked]
> Subject: Re: [AVT] 
> SRTP doesn't use header extensions.  (I think ZRTP was doing 
> so, but I don't know if they kept that or if there were 
> alternatives, and I think ZRTP just would consider that a 
> case where the channel couldn't be secured if there's no 
> alternative channel (RTCP?) for key exchange.)
Maybe I misunderstand things here but..Tried to read through RFC3711 but
I failed to find anything that metions something like that SRTP
precludes the use of header extensions. It is correct that the header
exension is sent as open information but I don't see this as an issue as
long as the information in the header extension does not reveal any
encrypted information. 

> 
> There's no easy way to know (except maybe by asking the ZRTP 
> people) how often this happens. Certainly Asterisk and things 
> like it would be a major blocker to this (they act as kind-of 
> B2BUA's -- don't get me started on what a mess their 
> implementations are).  I haven't seen a true SBC that 
> stripped them - but on the other hand I haven't used header 
> extensions except occasionally.
> 
> >> extensions to pass through may not be a good idea.
> extensions 
> >> in the same RTP packet, so moving too much to them is a problem.
> That does cover the requirement that header extensions can't 
> be mandatory, though the details and consequences of the 
> fallback need the be worked out.
OK

> 
> Adding it to every packet is a big no-no as mentioned.  Which 
> packets would it need to be added to, taking into account 
> packet loss, startup, etc?
OK, needs to be clarifies. 

> 
> The scheduling issues may mean that these extensions don't 
> get transmitted for a while, but another issue is that the 
> inserters of the extensions aren't always coordinated -- 
> witness ZRTP which in the current implementation acts as a 
> wrapper layer - if the main RTP stack adds an extension, that 
> means that ZRTP can't use that packet for an extension.
> This should be ok, and ZRTP has to account for this 
> possibility (as well as packet loss).
I believe that for cases like SVC and G.718 it should be possible to
synchronize the transmission of "alignment-info" as the layers after all
stem from the same encoder and it is probably in these cases that the
need to ensure perfect alignment is the biggest (for audio codecs it
must match to the sample)

---

From: <[email-masked]>
Date: Sun, 16 Nov 2008 18:46:24 +0200

Thanks for the clarification, which is important. According to this, it
appears that if the RTP header extenstion contains Decoder Order Number
information, then there is no problem, as this information not related
to anything in the RTP header. A good thing is that this solution is
really generic!

BR, YK 

>Sent: 16 November, 2008 17:19
>True - but if you modify the main package, passing through the header
>extension becomes tricky.  For example, a B2BUA may leave the payload
>alone, but almost always rewrites the RTP header, and may 
>generate it's own
>(or modify passed-through) RTCP packets.  If the header extension
>references the header (sequence or timestamps), you could be 
>broken.  Or if
>the header extension references the SSRC, or data carried in the RTCP
>stream (like NTP time), you could also be broken, and there's no way to
>know if you don't understand (and comply with) the extension.
>
>I mention things like this because Asterisk does those 
>actions, I think.
>
>--

---

From: Randell Jesup <[email-masked]>
Date: Sun, 16 Nov 2008 10:14:59 -0500

"Ingemar Johansson S" <[email-masked]> writes:
>> >There is of course a risk that this may happen.  However I am curious
>> >as to what extent this may actually happen as it would also cause the
>> >authentication to fail in case SRTP is used.
>> 

>long as the information in the header extension does not reveal any

Header extensions are fine with SRTP (I've used them)>  You said:

You're correct that an SBC/B2BUA/etc can't just delete anything from an
SRTP packet (as you know).  (Extensions are authenticated but not
encrypted, so sensitive information cannot be transmitted in them.)  A
B2BUA typically would need to decrypt and re-encrypt anyways, and an SBC
might be able to depending on the key and encryption setup.  (For example,
if it's SIP-over-TLS with sdesc, then the SBC probably has access to the
encryption keys.)

However, quite a few SBC's/etc pretty much ignore SRTP and treat it like
RTP.  Also, using either best-effort-srtp methods or cap-neg drafts, the
SBC might not even be aware that the stream is an SRTP stream.

I worry most about things like Asterisk, which is a pseudo-B2BUA.  I
suspect strongly it doesn't pass-through header extensions.

>must match to the sample) 

My point was that you can't synchronize with the prototypical ZRTP
implementation, since that occurs downstream of the encoder and main
application, in a black box (pseudo-proxy), so you have to leave enough
slots open for applications like that.

---

From: Randell Jesup <[email-masked]>
Date: Sun, 16 Nov 2008 10:19:03 -0500

True - but if you modify the main package, passing through the header
extension becomes tricky.  For example, a B2BUA may leave the payload
alone, but almost always rewrites the RTP header, and may generate it's own
(or modify passed-through) RTCP packets.  If the header extension
references the header (sequence or timestamps), you could be broken.  Or if
the header extension references the SSRC, or data carried in the RTCP
stream (like NTP time), you could also be broken, and there's no way to
know if you don't understand (and comply with) the extension.

I mention things like this because Asterisk does those actions, I think.

---

From: Colin Perkins <[email-masked]>
Date: Sat, 15 Nov 2008 17:47:20 -0600

Ye-Kui,

> understand, it'd better not to touch the something?

Perhaps, but RTP doesn't give you that guarantee.  An RTP header  
extension is designed to be optional and ignorable, and middleboxes  
are within their right to strip it out.  You can't rely on a header  
extension.

-- 
Colin Perkins
http://csperkins.org/

---

From: Colin Perkins <[email-masked]>
Date: Sun, 16 Nov 2008 20:01:57 +0000

Err, no. Things like decoding order number go into an RTP payload  
header, as part of a payload format, not into a header extension.

Colin

-- 
Colin Perkins
http://csperkins.org/

---

From: "Ingemar Johansson S" <[email-masked]>
Date: Mon, 17 Nov 2008 17:45:27 +0100

Hi
 
Some comments after reading in more detail thorugh the docs.
I believe that Colin makes a very strong nd valid point that the RTCP-SR
rate will likely be high enough to yield a fairly fast synchronization. 
The problem I see is in clock-skew and clock-resolution. Clock-skew
compensation requires that a number of RTCP-SR are received to be able
to compensate for the clock-skew. The clock-resolution (10-15ms
mentioned for Windows Vista) can be a problem. 
In order to this to become a problem the lack of accuracy should in e.g
the G.718 case be a frame-slip (20ms) i.e a layer is applied with a
N*20msec offset. Now how big is this risk ?. I would believe that
clock-skew is a relatively small risk given the relative fast RTCP-SR
rate. Clock accuracy can however be a problem and I would not neglect
it.
Also.. How do one detect that the layers are not synchronized?. Must
admit hear that it is some time since I read the G.718 draft, what is
the method to detect this.
As for the proposed solution in
draft-schierl-avt-rtp-multi-session-transmission to use the same RTP
timestamp for teh different layer. Given the weakened security is
acceptable this is probably useful. One limitation is of course that it
assumes the same RTP timestamp clock for the different layers, perhaps
not an issue today but what about the future ?.
Header extensions: Although header extensions are quite simple and would
make it simple to overcome the clock-skew and clock-accuracy issue there
seems to be a number of serious issues. However my feeling is that some
of the issues are sort of corner cases. When services are created codecs
are bundled with transports some of these issues would go away. I recall
that header extensions are proposed to be used in some 3GPP service
despite the possible issues.
 
My 5 cents
/Ingemar

________________________________

	From: Thomas Schierl [mailto:[email-masked]] 
	Sent: den 15 november 2008 12:40
	To: Randell Jesup
	Cc: Ingemar Johansson S; [email-masked]; [email-masked];
Colin Perkins; Jonathan Lennox
	Subject: Re: [AVT]
draft-schierl-avt-rtp-multi-session-transmission-00.txt

	I think it is nice to go into a deeper discussion on a
particular solution, but we should first solve the following questions: 
	"Do we need a new solution or an extension to the existing
mechanism? And, are the arguments collected in the draft convincing
reasons for going that way?" 
	Let us assume one special problem goes away - the delay in
presence of multiple receivers allowing bidirectional communication
(see: draft-perkins-avt-rapid-rtp-sync-00).
	
	Thanks,
	Thomas 

	-- 
	Thomas Schierl
	--------------
	Fraunhofer HHI

	Randell Jesup wrote: 

		"Ingemar Johansson S" <[email-masked]>
<mailto:[email-masked]>  writes:

				My question is ..how serious is the
identified problem ?. 
				My gut feeling is that a receiver that
implements e.g decoding of 
				G.718 content will likely update the RTP
stack to recognize header 
				extensions if header extensions are used
to carry e.g the NTP 
				timestamp related to the RTP timetstamp
in the RTP header.

				What if an intermediary deletes the
header extension?  (SBC, 
				B2BUA, raw message store, etc).

			There is of course a risk that this may happen. 
			However I am curious as to what extent this may
actually happen as it
			would also cause the authentication to fail in
case SRTP is used.

		SRTP doesn't use header extensions.  (I think ZRTP was
doing so, but
		I don't know if they kept that or if there were
alternatives, and I
		think ZRTP just would consider that a case where the
channel couldn't be
		secured if there's no alternative channel (RTCP?) for
key exchange.)
		
		There's no easy way to know (except maybe by asking the
ZRTP people) how
		often this happens. Certainly Asterisk and things like
it would be a major
		blocker to this (they act as kind-of B2BUA's -- don't
get me started on
		what a mess their implementations are).  I haven't seen
a true SBC that
		stripped them - but on the other hand I haven't used
header extensions
		except occasionally.

				This isn't to say it might not be a good
idea - but requiring 
				header extensions to pass through may
not be a good idea.  
				You also have issues where you can't use
multiple header 
				extensions in the same RTP packet, so
moving too much to them 
				is a problem.

			Ok. A fallback is always to use RTCP SR (which
is sent anyway), the
			convergence will be slower and clock-skew
compensation will complicate
			an implementation somewhat but it will anyway be
a fallback solution.
			The additional method (if possible to use) will
speed up convergenge.
			As for the problem with multiple header
extensions, I would not believe
			that you need to add the "timing alignent"
header extension in every
			packet. Also I believe it is up to the sender to
prioritize between the
			header extensions.

		That does cover the requirement that header extensions
can't be mandatory,
		though the details and consequences of the fallback need
the be worked out.
		
		Adding it to every packet is a big no-no as mentioned.
Which packets would
		it need to be added to, taking into account packet loss,
startup, etc?
		
		The scheduling issues may mean that these extensions
don't get transmitted
		for a while, but another issue is that the inserters of
the extensions
		aren't always coordinated -- witness ZRTP which in the
current
		implementation acts as a wrapper layer - if the main RTP
stack adds an
		extension, that means that ZRTP can't use that packet
for an extension.
		This should be ok, and ZRTP has to account for this
possibility (as well as
		packet loss).

	----
	Visit us at
	
	MEDICA 2008 / Duesseldorf, Germany / 19 - 22 November 2008 /
Hall 16, Booth D55
	http://www.medica.de
	
	SOCCEREX 2008 / Gauteng, South Africa / 23 - 26 November 2008 /
Booth V1-B1 a
	http://www.soccerex.com