Subject: [AVT] MPEG2-TS preamble drafts

From: "Ingemar Johansson S" <[email-masked]>
Date: Tue, 10 Nov 2009 03:12:30 +0100

Hi

I have tried to compare the two competing ID's on MPEG2-TS preamble. Not being an expert in this area I anyway try to get an understanding

As I see it draft-xia-avt-mpeg2ts-preamble-00.txt is a simple version of draft-begen-avt-rtp-mpeg2ts-preamble-03.txt as the former does not need a specification of a new preamble format (for simplicity I call the drafts XIA and BEGEN)
The benefit XIA is that the endpoints (that understands RFC4588) does not need to be upgraded to understand a new preamble format, I cannot put any weight on this possible benefit. 

The benefit with XIA compared to BEGEN comes with some drawbacks
1) Increased transmission overhead : BEGEN compresses the preamble info into one packet while XIA needs to transmit each packet that contains the preamble. This naturally means that the overhead increases. Does anybody have some figures as regards to how much this overhead increases ?.
2) Vulnerability to packetloss: XIA transmits the packets with preamble in several RFC4588 packets, which means that if one of these packets is lost the whole preamble is rendered useless. BEGEN compresses the preamble and can can possibly use this "compression gain" to transmit two identical preamble packets to cope with lossy environments.

In addition, if I was to come up with a solution to the preamble problem I would first consider the XIA alternative then the more elaborate BEGEN alternative. I am curious to know if the authors of BEGEN originally considered the alterantive outlined in XIA and if so what the motivation was to go for the current proposal.    

Unfortunately I cannot say do this or that but I hope that my questions will help to advance this in a direction that is satisfactory for all. 

/Ingemar

=================================
INGEMAR JOHANSSON  M.Sc. 
Senior Research Engineer 

Ericsson AB
Multimedia Technologies
Labratoriegränd 11
971 28, Luleå, Sweden
Phone +46-1071 43042
SMS/MMS +46-73 078 3289
[email-masked]
www.ericsson.com 
Visit http://labs.ericsson.com !
=================================

---

From: David R Oran <[email-masked]>
Date: Tue, 10 Nov 2009 11:25:14 +0900

On Nov 10, 2009, at 11:12 AM, Ingemar Johansson S wrote:

> Hi
>
> I have tried to compare the two competing ID's on MPEG2-TS preamble.  
> Not being an expert in this area I anyway try to get an understanding
>
> As I see it draft-xia-avt-mpeg2ts-preamble-00.txt is a simple  
> version of draft-begen-avt-rtp-mpeg2ts-preamble-03.txt as the former  
> does not need a specification of a new preamble format (for  
> simplicity I call the drafts XIA and BEGEN)
> The benefit XIA is that the endpoints (that understands RFC4588)  
> does not need to be upgraded to understand a new preamble format, I  
> cannot put any weight on this possible benefit.
>
> 1) Increased transmission overhead : BEGEN compresses the preamble  
> info into one packet while XIA needs to transmit each packet that  
> contains the preamble. This naturally means that the overhead  
> increases. Does anybody have some figures as regards to how much  
> this overhead increases ?.
> 2) Vulnerability to packetloss: XIA transmits the packets with  
> preamble in several RFC4588 packets, which means that if one of  
> these packets is lost the whole preamble is rendered useless. BEGEN  
> compresses the preamble and can can possibly use this "compression  
> gain" to transmit two identical preamble packets to cope with lossy  
> environments.
>
> In addition, if I was to come up with a solution to the preamble  
> problem I would first consider the XIA alternative then the more  
> elaborate BEGEN alternative. I am curious to know if the authors of  
> BEGEN originally considered the alterantive outlined in XIA and if  
> so what the motivation was to go for the current proposal.
>
Yes, we (actually I) did when I designed this general mechanism  
(although not the encoding into an RTP Payload format) about 3 years  
ago.

I considered just using retransmission  for about 5 minutes and  
realized that it used an absurd amount of bandwidth and sent a ton of  
useless data, when the amount of data actually needed to prime the  
decoder ahead of the I-frame was extremely small. it also had the  
hazard confusing the decoder by feeding it stale data; our real world  
experience with a variety of STBs is that they are very sensitive to  
the incoming data stream and especially on the positioning and timing  
of ECMs.

A secondary consideration is that pure retransmission may also require  
that the RS keep a lot of history in its buffer than is not needed  
since it will never need to be sent, so instead we extract the needed  
information from the MPEG2TS s it arrives and package it up for  
transmission when needed. This turns out to be very efficient and a  
small extra work factor over the work needed to parse the incoming  
MPEG2TS to locate the random access points.

> Unfortunately I cannot say do this or that but I hope that my  
> questions will help to advance this in a direction that is  
> satisfactory for all.
>
> /Ingemar
>
> _______________________________________________
> Audio/Video Transport Working Group
> [email-masked]
> https://www.ietf.org/mailman/listinfo/avt

---

From: Frank Xia <[email-masked]>
Date: Mon, 09 Nov 2009 21:05:48 -0600

Hi Ingemar

Thank you for the review.

BR
Frank
----- Original Message ----- 
From: "Ingemar Johansson S" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>; <[email-masked]>
Sent: Monday, November 09, 2009 8:12 PM
Subject: MPEG2-TS preamble drafts

Hi

I have tried to compare the two competing ID's on MPEG2-TS preamble. Not 
being an expert in this area I anyway try to get an understanding

As I see it draft-xia-avt-mpeg2ts-preamble-00.txt is a simple version of 
draft-begen-avt-rtp-mpeg2ts-preamble-03.txt as the former does not need a 
specification of a new preamble format (for simplicity I call the drafts XIA 
and BEGEN)
The benefit XIA is that the endpoints (that understands RFC4588) does not 
need to be upgraded to understand a new preamble format, I cannot put any 
weight on this possible benefit.

The benefit with XIA compared to BEGEN comes with some drawbacks
1) Increased transmission overhead : BEGEN compresses the preamble info into 
one packet while XIA needs to transmit each packet that contains the 
preamble. This naturally means that the overhead increases. Does anybody 
have some figures as regards to how much this overhead increases ?.

Frank=> I once tried to give a calculation in my
previous email.  It is quoted below "
let's say preamble consists of 20 TSes which has 20*188*8 bits.
Assume unicast RTP burst last 3 seconds with 2M bandwith
which is 3*2M bits in total. Preamble occupies around 0.5% of total 
unicast."
I may be wrong.

I would like to ask Avi a question. How many RTP packets
needs in his solution for carrying preamble?
I don't think there are big difference between the two solutions

2) Vulnerability to packetloss: XIA transmits the packets with preamble in 
several RFC4588 packets, which means that if one of these packets is lost 
the whole preamble is rendered useless. BEGEN compresses the preamble and 
can can possibly use this "compression gain" to transmit two identical 
preamble packets to cope with lossy environments.
Frank=>This question is a variant of the above question.
In fact, each of packets in RAMS needs retransmission if one is lost.
The possibilty of  preamble packets loss have very limited impact
given the small volume of the traffic.

In addition, if I was to come up with a solution to the preamble problem I 
would first consider the XIA alternative then the more elaborate BEGEN 
alternative. I am curious to know if the authors of BEGEN originally 
considered the alterantive outlined in XIA and if so what the motivation was 
to go for the current proposal.

Unfortunately I cannot say do this or that but I hope that my questions will 
help to advance this in a direction that is satisfactory for all.

/Ingemar

Ericsson AB
Multimedia Technologies
Labratoriegränd 11
971 28, Luleå, Sweden
Phone +46-1071 43042
SMS/MMS +46-73 078 3289

---

From: Frank Xia <[email-masked]>
Date: Mon, 09 Nov 2009 21:14:58 -0600

Hi David

Thank you for sharing you hands-on experience.
Please check my quick reply.

BR
Frank

----- Original Message ----- 
From: "David R Oran" <[email-masked]>
To: "Ingemar Johansson S" <[email-masked]>
Cc: <[email-masked]>
Sent: Monday, November 09, 2009 8:25 PM
Subject: Re: [AVT] MPEG2-TS preamble drafts

On Nov 10, 2009, at 11:12 AM, Ingemar Johansson S wrote:

> specification of a new preamble format (for  simplicity I call the drafts 
> XIA and BEGEN)
> 1) Increased transmission overhead : BEGEN compresses the preamble  info 
> into one packet while XIA needs to transmit each packet that  contains the 
> 2) Vulnerability to packetloss: XIA transmits the packets with  preamble 
> in several RFC4588 packets, which means that if one of  these packets is 
> lost the whole preamble is rendered useless. BEGEN  compresses the 
> preamble and can can possibly use this "compression  gain" to transmit two 
> identical preamble packets to cope with lossy  environments.
>
> In addition, if I was to come up with a solution to the preamble  problem 
> I would first consider the XIA alternative then the more  elaborate BEGEN 
> considered the alterantive outlined in XIA and if  so what the motivation 
> was to go for the current proposal.
>

I considered just using retransmission  for about 5 minutes and
realized that it used an absurd amount of bandwidth and sent a ton of
useless data, when the amount of data actually needed to prime the
decoder ahead of the I-frame was extremely small. it also had the
hazard confusing the decoder by feeding it stale data; our real world
experience with a variety of STBs is that they are very sensitive to
the incoming data stream and especially on the positioning and timing
of ECMs.
Frank=>Could you give me a simple picture about  "a
ton of useless data"?   Additionally, IMHO, a variety of STB
favor  STB-agnostic solution :-)

A secondary consideration is that pure retransmission may also require
that the RS keep a lot of history in its buffer than is not needed
since it will never need to be sent, so instead we extract the needed
information from the MPEG2TS s it arrives and package it up for
transmission when needed. This turns out to be very efficient and a
small extra work factor over the work needed to parse the incoming
MPEG2TS to locate the random access points.
Frank=>Keeping a history is mandatory for both solutions because
retransmission is common processing when a preamble packet
is lost.

> Unfortunately I cannot say do this or that but I hope that my  questions 
> will help to advance this in a direction that is  satisfactory for all.
>
> /Ingemar
>

---

From: David R Oran <[email-masked]>
Date: Tue, 10 Nov 2009 12:15:53 +0900

On Nov 10, 2009, at 12:05 PM, Frank Xia wrote:

> ----- Original Message ----- From: "Ingemar Johansson S" <[email-masked] 
> >
> To: <[email-masked]>
> Cc: <[email-masked]>; <[email-masked]>; <[email-masked] 
> >
where do you get this? The ECM or PAT/PMT you need may be 1/2 a GOP  
back in the data stream. That could be a megabit or more of useless  
traffic.

---

From: Peilin YANG <[email-masked]>
Date: Tue, 10 Nov 2009 11:32:42 +0800

Thank Ingemar and David,

Increased transmission overhead(Ingemar)： I don't think it is a drawback. We got all of MPEG2-TS streamings from operators which only need two MPEG2-TS infromation - PAT and PMT, CAT is necessay if it exists. I cannot ensure that all of MPEG2-TS streaming only need these two PAT and PMT infromation, but I estimated that at least above 80% MPEG2-TS streaming only need PAT and PMT. If it only has two TSs(188*2 = 673 bytes), the overhead is si
mialr to the new format. PCR is dense and its time is early than Video'PTS and Audio'PTS, for other information SPS,PPS,SEI,PTS,etc. I think at least above 95% ES and PES contain these information.
  
Vulnerability to packetloss: if it is only one packet, two drafts are the same.

"I considered just using retransmission  for about 5 minutes and realized that it used an absurd amount of bandwidth and sent a ton of useless data, when the amount of data actually needed to prime the decoder ahead of the I-frame was extremely small. it also had the hazard confusing the decoder by feeding it stale data; our real world experience with a variety of STBs is that they are very sensitive to the incoming data stream and especial
ly on the positioning and timing of ECMs."-From David

I didn't sent a ton of useless data to test like David, but I think MPEG2-TS preamble and packets of I-frame are equally important. if packets of I-frame is on problem using RFC4588, MPEG2-TS preamble is identical. 

"A secondary consideration is that pure retransmission may also require that the RS keep a lot of history in its buffer than is not needed since it will never need to be sent, so instead we extract the needed information from the MPEG2TS s it arrives and package it up for transmission when needed. This turns out to be very efficient and a small extra work factor over the work needed to parse the incoming MPEG2TS to locate the random access 
points."-From David

I don't completely understand the meaning. According to my understanding if our draft needs to keep a lot of history in its buffer, i also think new format  needs to keep this information the same history. if one of this information lost, it also need to retransmit unless it is impossible to lose.

Peilin

******************************************************************************************
 This email and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained here in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this email in error, please notify the sender by phone or email
 immediately and delete it!
 *****************************************************************************************

----- 原邮件 -----
发件人: David R Oran <[email-masked]>
日期: 星期二, 十一月 10日, 2009 上午10:25
主题: Re: [AVT] MPEG2-TS preamble drafts
收件人: Ingemar Johansson S <[email-masked]>
抄送: [email-masked]

> > Not being an expert in this area I anyway try to get an 
> understanding>
> > Labratoriegr?nd 11
> > 971 28, Lule?, Sweden

---

From: David R Oran <[email-masked]>
Date: Tue, 10 Nov 2009 12:37:54 +0900

We're clearly mis-communicating. Let's defer this to the meeting this  
afternoon. Unfortunately I can't make it, but Ali and Jeff will be  
there and they understand the stuff as well or better than I do.

Thanks, DaveO.

On Nov 10, 2009, at 12:32 PM, Peilin YANG wrote:

> Thank Ingemar and David,
>
> Increased transmission overhead(Ingemar)： I don't think it is a  
> drawback. We got all of MPEG2-TS streamings from operators which  
> only need two MPEG2-TS infromation - PAT and PMT, CAT is necessay if  
> it exists. I cannot ensure that all of MPEG2-TS streaming only need  
> these two PAT and PMT infromation, but I estimated that at least  
> above 80% MPEG2-TS streaming only need PAT and PMT. If it only has  
> two TSs(188*2 = 673 bytes), the overhead is si
> mialr to the new format. PCR is dense and its time is early than  
> Video'PTS and Audio'PTS, for other information SPS,PPS,SEI,PTS,etc.  
> I think at least above 95% ES and PES contain these information.
>
> "I considered just using retransmission  for about 5 minutes and  
> realized that it used an absurd amount of bandwidth and sent a ton  
> of useless data, when the amount of data actually needed to prime  
> the decoder ahead of the I-frame was extremely small. it also had  
> the hazard confusing the decoder by feeding it stale data; our real  
> world experience with a variety of STBs is that they are very  
> sensitive to the incoming data stream and especial
> I didn't sent a ton of useless data to test like David, but I think  
> MPEG2-TS preamble and packets of I-frame are equally important. if  
> packets of I-frame is on problem using RFC4588, MPEG2-TS preamble is  
> identical.
>
> "A secondary consideration is that pure retransmission may also  
> require that the RS keep a lot of history in its buffer than is not  
> needed since it will never need to be sent, so instead we extract  
> the needed information from the MPEG2TS s it arrives and package it  
> up for transmission when needed. This turns out to be very efficient  
> and a small extra work factor over the work needed to parse the  
> incoming MPEG2TS to locate the random access
> points."-From David
>
> I don't completely understand the meaning. According to my  
> understanding if our draft needs to keep a lot of history in its  
> buffer, i also think new format  needs to keep this information the  
> same history. if one of this information lost, it also need to  
> retransmit unless it is impossible to lose.
>
> Peilin
>
> This email and its attachments contain confidential information from  
> HUAWEI, which is intended only for the person or entity whose  
> address is listed above. Any use of the information contained here  
> in any way (including, but not limited to, total or partial  
> disclosure, reproduction, or dissemination) by persons other than  
> the intended recipient(s) is prohibited. If you receive this email  
> in error, please notify the sender by phone or email

---

From: Frank Xia <[email-masked]>
Date: Mon, 09 Nov 2009 22:15:36 -0600

Hi David

Please check my quick reply

BR
Frank
----- Original Message ----- 
From: "David R Oran" <[email-masked]>
To: "Frank Xia" <[email-masked]>
Cc: "Ingemar Johansson S" <[email-masked]>; <[email-masked]>
Sent: Monday, November 09, 2009 9:15 PM
Subject: Re: [AVT] MPEG2-TS preamble drafts

where do you get this? The ECM or PAT/PMT you need may be 1/2 a GOP
back in the data stream. That could be a megabit or more of useless
traffic.
Frank=>I am curious how the preamble can occupy 1/2 GOP.
Probably, you can us give a rough illustration.

Let's say it is true. Then, what is the gain after your compression?
Remember PAT/PMT only needs 2 TS in most cases.
ECM has more, but your compression can contribute little.