Subject: Re: [AVTCORE] Call for WG item on RTCP for inter-destination media synchronization

From: "Schooler, Eve M" <[email-masked]>
Date: Tue, 10 May 2011 01:06:19 -0700

Unfortunately I don't remember much about the actual RTP implementation. 
However a few additional details about the experiment were captured in an 
earlier post to the [e2e] mailing list.

http://www.postel.org/pipermail/end2end-interest/2001-August/001314.html

As Steve points out, one focus of the work was to understand the proper hooks 
needed by a session orchestration protocol (pun intended!) to be able to 
establish and negotiate synchronization constraints among session participants.

E.

Eve M. Schooler, PhD
Intel Labs, SC12-303
3600 Juliette Lane                         w: +1 408.765.1591
Santa Clara, CA 95054                      f: +1 408.653.8511

-----Original Message-----
From: Stephen Casner [mailto:[email-masked]] 
Sent: Monday, April 18, 2011 12:38 PM
To: Magnus Westerlund
Cc: IETF AVTCore WG; Schooler, Eve M
Subject: Re: [AVTCORE] Call for WG item on RTCP for inter-destination media synchronization

At the 29th IETF in Seattle in 1994, Julio Escobar gave a presentation
about the Flow Synchronization Protocol developed along with Craig
Partridge and Debra Deutsch at BBN.  This protocol was designed to
support multiple synchronization goals including inter-destination
media playout synchronization.  The proceedings from that meeting say
this:

    At the end of the second session, there were two additional
    presentations.  Julio Escobar gave a report on the use of RTP to
    support the Synchronization Protocol developed at BBN.  The only
    requirement not satisfied by RTP was the need to communicate the
    calculated synchronization delays among the destination
    processors; this could be added to the control packet as
    application-specific data.

This was during the period when RTPv2 was being designed, so the
important question was whether the base RTP protocol needed any
additions.  As the paragraph states, the conclusion was that all that
was needed was some information in RTCP.  As I recall, the decision at
that time was not to define any specific RTCP packet types yet, but to
just use the APP packet for experiments.  Note that the Flow
Synchronization Protocol also included messages of its own that would
be used in conjunction with something more like a session
establishment protocol.

This protocol was used for a demonstration at the ACM Multimedia
conference in October, 1994 of a distributed music trio.  From the
proceedings:

    This demonstration will show the ability to display (play out) a
    music orchestra at the conference site by remotely synchronizing
    the audio and video from musicians across the country.  It will
    serve to demonstrate the current capabilities of multi-media
    synchronization.

    The demonstration will synchronize across the continental USA two
    live players and one repeater site, this delivering a synchronized
    music trio performance the the demo site in spite of delay
    variations through the internetworks that carry the aduio and
    video flows.

    The demonstration will also include Graphical User Interface
    displays of the measured and managed delays in the system to show
    in real-time the way in which the synchronization protocol
    operates.  The audience will be able to experience the difference
    between synchronized and unsynchronized music performances.

For this demonstration, the synchronization goal was different from
synchronizing playout at multiple sites at the same time.  Instead, it
was to synchronize the mixing of multiple sound sources that were
created at multiple sites at the same "logical" time.  In this case,
the piano part of a Haydn trio was pre-recorded and served as the
conductor or metronome.  It was originated from one network site and
sent to three other sites where two live performers (Eve Schooler and
Martha Steenstrup) listened to the piano part and played along the
violin and cello parts that were transmitted in new streams, while the
third site repeated (forwarded) the piano part.  Then listeners in
multiple other sites received the three streams and played them in
sync.  Playout could have been sychronized to be at the same time as
well, but that goal was unnecessary for this demonstration.

I apologize that I do not remember the details of the implementation
that we did with RTP for this demo.  Perhaps Eve remembers.  I might
be able to find something in old email if it turns out to be useful.

I urge those who are interested in this synchronization work to take a
look at the BBN Flow Synchronization Protocol, if you have not already
done so, and to consider a more general set of synchronization goals.

I found these references:

A paper in IEEE/ACM Transactions on Networking from April, 1994, in
the ACM Digital Library:

http://portal.acm.org/citation.cfm?id=187042

An earlier paper from March, 1991 that is freely available:

ftp://cs.ucl.ac.uk/nets/docs/distributed-systems/msfs.ps.Z

Another free paper, undated but later than February 1992 based on a
reference date:

ftp://cs.ucl.ac.uk/mice/misc_papers/flow_sync_control.ps.gz

A slide presentation by Eve Schooler from the Network Music Festival
in Septemper 1993 that reported on an earlier demo of the same
scenario for our funding sponsors is here:

ftp://ftp.isi.edu//pub/hpcc-papers/mmc/NetMusicFest.ps

Ghostscript barfed on that, but the Mac was able to translate to PDF,
so I put it here:

ftp://ftp.packetdesign.com/outgoing/casner/NetMusicFest.pdf

                                                        -- Steve

On Mon, 18 Apr 2011, Magnus Westerlund wrote:

> WG,
>
> At the WG meeting in Prague we had a show of hands if we should create a
> WG work item for RTCP for inter-destination media synchronization. Along
> the lines described by
> https://datatracker.ietf.org/doc/draft-brandenburg-avtcore-rtcp-for-idms/.
>
> There where 5 persons in the room indicating interest. We now would like
> to judge the interest on the mailing list for a new WG item. Please
> indicate your support and willingness to contribute. If you think it
> shouldn't be done please voice that opinion also.
>
> This call runs for 2 weeks until Wednesday the 4th of May.
>
> Regards
>
> Magnus Westerlund
>
> AVTCore chair
> ----------------------------------------------------------------------
> Multimedia Technologies, Ericsson Research EAB/TVM
> ----------------------------------------------------------------------
> Ericsson AB                | Phone  +46 10 7148287
> F?r?gatan 6                | Mobile +46 73 0949079
> SE-164 80 Stockholm, Sweden| mailto: [email-masked]
> ----------------------------------------------------------------------
> _______________________________________________
> Audio/Video Transport Core Maintenance
> [email-masked]
> https://www.ietf.org/mailman/listinfo/avt
>

---

From: "Kevin Gross" <[email-masked]>
Date: Wed, 25 May 2011 14:17:55 -0600

I have reviewed the draft and offer these comments:

MSAS acronym is used on p.5 before its definition on p.7.

IDMS is applicable to the commercial AV applications I work on - stadiums,
legislative, concerts, general public address, etc. I'd be happy to provide
a use case for section 4 if that would be helpful.

Consider using the 64-bit NTP format for Packet Presented NTP timestamp. The
current 32-bit 15 microsecond resolution is longer than some audio sample
periods. The resolution could be a limitation in some use cases - stereo
imaging and audio beam forming.

Need clarification or cleanup of the last sentence of first paragraph in
section 8. Synchronized clocks should not drift beyond the accuracy of the
synchronization mechanism.

Move second paragraph in section 8 to section 4. This is a use-case
discussion.

Add comments on p.14 to explain what "gps", "gal" and "ptp" are.

p.14: Format of ptp-id depends on the version of PTP in use. IEEE 1588-2002
uses an ASCII string. IEEE 1588-2008 uses a 7-bit identifier. IEEE
802.1AS-2011 uses a 64-bit identifier.

p.14: As per above, we probably need to add some means for identifying PTP
version/profile in use.

p.14: Consider increased precision for last-synced. PTP can deliver updates
up to 100 times per second.

Add justification to section 12 for new packet type. Why is the ETSI TISPAN
approach deficient?

Section 13 does not give a complete enough description of how presentation
time alignment would be accomplished for us to verify whether the protocol
delivers enough information for it to be done accurately and reliably. The
Boronat2009 reference also does not provide the necessary details. How does
the system deal with changing latency due to changing network conditions and
media formats? How do we handle devices that don't output directly to users
(e.g. STB connected to HDTV with unknown delay in the HDTV)?

Kevin Gross
AVA Networks

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Kevin
Gross
Sent: Friday, April 22, 2011 7:01 AM
To: 'IETF AVTCore WG'
Subject: Re: [AVTCORE] Call for WG item on RTCP for inter-destination media
synchronization

This is of interest to me and I should be able to review it and contribute
comments.

Kevin Gross
AVA Networks

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Magnus
Westerlund
Sent: Monday, April 18, 2011 8:49 AM
To: IETF AVTCore WG
Subject: [AVTCORE] Call for WG item on RTCP for inter-destination media
synchronization

WG,

Regards

Magnus Westerlund

AVTCore chair
Färögatan 6                | Mobile +46 73 0949079

---

From: "Brandenburg, R. (Ray) van" <[email-masked]>
Date: Thu, 26 May 2011 07:04:44 +0000

Hi Kevin,

Thanks for your extensive review. Very helpful. We will start work on a new version which incorporates your comments. 

> IDMS is applicable to the commercial AV applications I work on - stadiums, legislative, concerts, general public address, etc. I'd be happy to provide a use case for section 4 if that would be helpful.

That would be great!

Kind Regards,

Ray van Brandenburg

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Kevin Gross
Sent: woensdag 25 mei 2011 22:18
To: 'IETF AVTCore WG'
Subject: Re: [AVTCORE] Call for WG item on RTCP for inter-destination media synchronization

IDMS is applicable to the commercial AV applications I work on - stadiums, legislative, concerts, general public address, etc. I'd be happy to provide a use case for section 4 if that would be helpful.

Consider using the 64-bit NTP format for Packet Presented NTP timestamp. The current 32-bit 15 microsecond resolution is longer than some audio sample periods. The resolution could be a limitation in some use cases - stereo imaging and audio beam forming.

Need clarification or cleanup of the last sentence of first paragraph in section 8. Synchronized clocks should not drift beyond the accuracy of the synchronization mechanism.

Move second paragraph in section 8 to section 4. This is a use-case discussion.

p.14: Format of ptp-id depends on the version of PTP in use. IEEE 1588-2002 uses an ASCII string. IEEE 1588-2008 uses a 7-bit identifier. IEEE
802.1AS-2011 uses a 64-bit identifier.

p.14: As per above, we probably need to add some means for identifying PTP version/profile in use.

p.14: Consider increased precision for last-synced. PTP can deliver updates up to 100 times per second.

Add justification to section 12 for new packet type. Why is the ETSI TISPAN approach deficient?

Section 13 does not give a complete enough description of how presentation time alignment would be accomplished for us to verify whether the protocol delivers enough information for it to be done accurately and reliably. The
Boronat2009 reference also does not provide the necessary details. How does the system deal with changing latency due to changing network conditions and media formats? How do we handle devices that don't output directly to users (e.g. STB connected to HDTV with unknown delay in the HDTV)?

Kevin Gross
AVA Networks

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Kevin Gross
Sent: Friday, April 22, 2011 7:01 AM
To: 'IETF AVTCore WG'
Subject: Re: [AVTCORE] Call for WG item on RTCP for inter-destination media synchronization

This is of interest to me and I should be able to review it and contribute comments.

Kevin Gross
AVA Networks

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Magnus Westerlund
Sent: Monday, April 18, 2011 8:49 AM
To: IETF AVTCore WG
Subject: [AVTCORE] Call for WG item on RTCP for inter-destination media synchronization

WG,

At the WG meeting in Prague we had a show of hands if we should create a WG work item for RTCP for inter-destination media synchronization. Along the lines described by https://datatracker.ietf.org/doc/draft-brandenburg-avtcore-rtcp-for-idms/.

There where 5 persons in the room indicating interest. We now would like to judge the interest on the mailing list for a new WG item. Please indicate your support and willingness to contribute. If you think it shouldn't be done please voice that opinion also.

Regards

Magnus Westerlund

AVTCore chair

_______________________________________________
Audio/Video Transport Core Maintenance
[email-masked]
https://www.ietf.org/mailman/listinfo/avt
This e-mail and its contents are subject to the DISCLAIMER at http://www.tno.nl/emaildisclaimer