Subject: RE: [MMUSIC] RE: Transport of Internet Media Guides - Work/Service Split

From: [email-masked]
Date: Mon, 16 Dec 2002 14:22:35 +0200

Hi Henning

Your comments are very helpful. I am starting to think the matrix was a pretty 
good idea since we've already found some of the important issues to resolve in 
the drafts. It seems we are thinking along the same lines on most of these 
issues and I've made specific responses below just to check this and test some
of the implications.

>> Is there a need
>> for any more detail at this stage?
> No. I actually believe that only simplex and duplex (or, if you prefer, 
> unidirectional/bidirectional). Hybrid is not meaningful. No connection 
> is ever perfectly symmetric, including your plain old modem. In many 
> cases, the asymmetry is not visible to the end system in any event.

You are right, only simplex and duplex should be considered by the IMG 
transport protocol instantiations and related link characteristics this can 
be made transparent to the server(s). After reading my earlier postings I 
realise the purpose of the hybrid case is not very clear. It is meant to 
provoke discussion on how we should deal with this when it comes to actually 
deploying the implementations and so perhaps it's a little early in the img 
life cycle to dedicate too much time to. To this extent, I feel that it may 
be worth briefly describing the hybrid case as an environmental consideration 
or applicability statement or even usage scenario in the eventual IMG 
framework document if the implementation environments and the experience we 
gain from them show this is valuable. Does this seem rational?

To reflect this the matrix should be simplified thus...
          |unidirectional |    hybrid     |bi-directional |
          |  access/link  |  access/link  |  access/link  |
          | (1 x simplex) | (2 x simplex) | (duplex)      |
...changes to...
          |unidirectional |bi-directional |
          |   (simplex)   |   (duplex)    |
...and let the text explain the rest. (i.e. hybrid is indeed only of 
consideration to the links, and server configuration such that, if needed, it 
should only be mentioned in the IMG framework - and on the reflector :)

>> Can you see any situations where additional sender
>> knowledge would be needed to abstract away the link-layer?
> No. That's why I'm arguing that we should only worry about the one-way 
> and two-way cases. Bandwidth constraints are a separate issue.

Great, our life will be simpler this way! Bandwidth constraints and perhaps
congestion control are a more difficult issue in terms of whether they are 
transport protocol only, framework only or both. For instance, a scheme like
SAPv2 would require that the sender acquires information on other announcement
IP flows to calculate its own announcement bandwidth/frequency. The usage of a
changing bandwidth seems to be transport protocol scope but the replacement
strategy for the metadata is more of an issue to the framework (and clearly
impacts the metadata specification too). There's definitely room for 
contributions in this space.

> Again, I don't see a clear boundary here. Updates and full state are 
> content functions, not message type functions. 

Yes. Updates and full state are metadata/content functions (I guess IMG 
metadata = IMG content) and so the same message type be used for both. In 
fact, it is reasonable to suppose that if a unidirectional multicast notify 
is required, then this could also use the same kind of framing/header (and 
maybe same message type). In this sense a single push transport protocol could 
be used for both abstract services: img notify and img push. 

We definitely need some kind of identification for an (atomic) element of the 
IMG which can be distinguished from others - relating to a "block" of 
metadata, possibly a single object, single session or a complex relationship 
of these. This is the only way to perform "precision" replacement, change, 
insertion and deletion. A notify can say that this "IMG element/block X has 
changed" and a push "this is the IMG element/boundary X (new or repeated)" 
[and naturally many such announcements can appear in a single push or notify]. 
Also (at least conceptually) a piggy-backed message can say "X has changed, 
this is the new X".

At this point the separation of push and notify seems reasonable clear to me 
(push contains metadata, notify only identifies it). However, this does not 
imply that a separate transport protocol is needed for them (it would be less 
complex to use a single one) and these may even be considered "content 
functions". Even so, it is useful to see the two concepts of push and notify 
as distinct since the reaction of the client application will be different 
(and the use of the server likewise).

Note, we also have the case of a kind of piggy-back which does not give the 
whole element (only sub-atomic particles). So that notify gives "X has 
changed" and push "in this way X.1, X.7". The sub-element metadata is not a 
complete element but obviously must include the identifier. The usefulness 
of this feature depends on the bandwidth saving and that will be hard to 
predict until the specifications are stable enough to model and simulate, or 
prototype and test. 

> I can see SUBSCRIBEing to 
> a certain type of program and then get notified when, say, my favourite 
> actor is on. As you say, with SAP, you want something more personalized 
> than a small set of multicast groups, so you will need a form of 
> "subscription" to convey those preferences.

I agree. It seems we have 3 distinct cases relating to IMG subscription 
(I'm listing so anyone can easily spot if there's a flaw in this idea):
i.   personal subscribe: subscribe to a point-to-point IMG service
ii.  shared subscribe:   subscribe to a point-to-multipoint IMG service
iii. zero subscribe:     just listen for point-to-multipoint IMG services
(IMG service = push or notify in this context)

> For robustness, my preference is actually to make each update 
> self-contained and complete, i.e., while it may contain update data 
> (say, a time-shift) in the body, there should be a reference in the 
> notification which allows the receiver to retrieve the whole item. This 
> is necessary since the state coupling between sender and receiver is 
> going to be very loose. It can easily happen that the receiver "forgot" 
> earlier announcements for some reason.

Yes, where a well equipped return channel is available this is an excellent 
solution (it should probably get a "RECOMMENDED" key word for this case in 
the drafts) (however, note RFC2357). Where there is no return channel or 
whether the use of one is not a foregone conclusion** we must consider 
robustness of the unidirectional downlink channel. Both forward error 
correction (FEC) and repeat transmission may be considered as they provide 
different compromises between bandwidth usage, complexity, timeliness and 
suitability to different types of transmission error. It should be worth 
simulating the first case (using the return channel to retrieve the detailed 
or erroneous data) in combination with some FEC to see what kinds of compromise 
might result in the least congestion.

** this may be the case if the return channel is high latency (like dial-up), 
relatively high cost or massive scalability is required. In any case, any 
reliable IP multicast specification in the IETF must deal with the needs of 
RFC2357 and the use of uplink signalling following a multicast error would 
require congestion control.

> Making notifications and announcements mostly references also has the 
> advantage that this works well for small devices and for devices where I 
> may want to delay retrieving information until more bandwidth becomes 
> available (think of a hybrid 3G/802.11 device or a device that's roaming 
> into expensive data territory). There is no way that the sender should 
> keep track of receiver bandwidth costs and availability.

Excellent, this is my thinking too. Especially metadata which does not effect 
the basic human selection of media and machine reception/retrieval of that 
media (e.g. pop star's favourite food) fits this scheme exceptionally well. 
However, this is more of an implementation issue (and "recommendations in 
the framework") as the push method must allow alternative cases of 
unidirectional-only, high urgency metadata and high value metadata (worth 
burning wireless bandwidth for). (I guess I've already emphasised the 
alternative requirements of the unidirectional-case enough in one mail 
already :)

Also, just so as not to be misinterpreted: I agree with your comments on 
bandwidth costs and availability <for links>. It's worth pointing out that 
"bandwidth availability" is a serious issue to the sender in terms of 
announcement frequency/timeliness and datarate control (e.g. as in SAPv2) 
and, also the receiver, in terms of congestion control.

>> We have also identified some short-comings of SAP that it would be
>> necessary to address for IMG transport:
...
>> - Prioritisation and timeliness of some announcements over others 
>> is desirable
> I agree with most of these, except the last one. That's a transport 
> issue. 

I agree that it is a transport issue. There's plenty of opportunity to 
debate whether SAP is a transport protocol (I feel that is its main job 
shared with UDP). In any case, SAPv2 has a very real effect on prioritisation 
and timeliness - i.e. they are totally "wild" and it is not uncommon to wait 
10 minutes before a particular message of any priority arrives. Apart from 
not helping user-friendly UI (where more important/popular parts are available 
faster), it is not feasible to expect a battery powered terminal to wait 10 
minutes for the interesting piece of information with it's RF powered on 
(WLANs are an exception, but do burn batteries very fast).

So, the issues of prioritisation and timeliness MUST (IMHO) be considered by 
IMG and especially with respect to the unidirectional/multicast transport.

> The problem with large packet size is that you then have to do 
> application-layer fragmentation.

Yes and since UDP expects packets to already be segmented, our SAP-like 
protocol must do this. SAPv2 does this by giving the size limitation to the 
application layer. Although application-layer fragmentation is not the 
greatest invention known to man, it must be considered. If we are to 
synchronise a subsection of a server-based IMG database which is split into 
identifiable IMG elements, we do have to select which elements go at what 
time/instant at the (server) application layer. On the other hand, if this 
results in sets of data larger than the UDP/MTU allows, then the transport 
can segment this for UDP/L2-friendly sized sets of data. I hope we will get a 
significant debate on this issue as the topic of application grouping of 
elements (into a single announcement event) comes up.

All the best,
Rod.

-----Original Message-----
From: ext Henning Schulzrinne [mailto:[email-masked]]
Sent: 09 December, 2002 16:49
To: Walsh Rod (NRC/Tampere)
Cc: [email-masked]; [email-masked]; Luoma Juha-Pekka
(NRC/Tampere)
Subject: Re: [MMUSIC] RE: Transport of Internet Media Guides -
Work/Service Split

> I agree that it is not necessary or desirable to get tied up in link
> details. It is sufficient that the IMG specifications take account of
> the general environments in which they are used and I expect the
> three kinds of links described (unidirectional, bi-directional and
> hybrid) are sufficient at this level of abstraction. Is there a need
> for any more detail at this stage?

No. I actually believe that only simplex and duplex (or, if you prefer, 
unidirectional/bidirectional). Hybrid is not meaningful. No connection 
is ever perfectly symmetric, including your plain old modem. In many 
cases, the asymmetry is not visible to the end system in any event.

> 
> 
>> In many cases, the sender may not know, for example, whether the
>> receiver is on a hybrid link. After all, the Internet model is
>> about abstracting away link-layer details.
> 
> 
> This is true. The sender would know what kinds of subscribes it has
> (for notify_quiz, notify_push and push; and for p-t-p and p-t-m), and
> may be configured (manually?) to send certain items to certain p-t-m
> channels be default. This should be sufficient to handle each of the
> link cases. Can you see any situations where additional sender
> knowledge would be needed to abstract away the link-layer?

No. That's why I'm arguing that we should only worry about the one-way 
and two-way cases. Bandwidth constraints are a separate issue.

> My use of img_PUSH is to send IMG metadata. The metadata itself can
> contain references to further metadata but this does not effect the
> use of img_PUSH. My use of img_NOTIFY is to inform of events in the
> IMG - i.e. new items, modifications, deletions. The notify may just
> identify the effected part of the IMG, or may also reference the IMG
> metadata which has changed (e.g. 9pmFootball, or
> 9pmFootball_destinationIPaddress). The notify may also include the
> change (i.e. be piggy-backed with a push), but this only makes sense
> for small changes that are not frequent (otherwise bandwidth and
> scalability could be baddly effected).

Again, I don't see a clear boundary here. Updates and full state are 
content functions, not message type functions. I can see SUBSCRIBEing to 
a certain type of program and then get notified when, say, my favorite 
actor is on. As you say, with SAP, you want something more personalized 
than a small set of multicast groups, so you will need a form of 
"subscription" to convey those preferences.

For robustness, my preference is actually to make each update 
self-contained and complete, i.e., while it may contain update data 
(say, a time-shift) in the body, there should be a reference in the 
notification which allows the receiver to retrieve the whole item. This 
is necessary since the state coupling between sender and receiver is 
going to be very loose. It can easily happen that the receiver "forgot" 
earlier announcements for some reason.

Making notifications and announcements mostly references also has the 
advantage that this works well for small devices and for devices where I 
may want to delay retrieving information until more bandwidth becomes 
available (think of a hybrid 3G/802.11 device or a device that's roaming 
into expensive data territory). There is no way that the sender should 
keep track of receiver bandwidth costs and availability.

> We have also identified some short-comings of SAP that it would be
> necessary to address for IMG transport: - no reliability (some kind
> of FEC building block is needed) - descriptor size limitation - a
> descriptor (e.g. SDP) must fit in one SAP packet and will be no
> larger than the MTU or 4kB - state information needs to be maintained
> for the SAP "session" as well as the SDP session (this is quite minor
> relating to SAP ("session") message identifier (hash), but is only
> useful for encrypted descriptors/payload) - IMG descriptors need the
> insertion, expiry, deletion, replacement policy details - not the
> announcement transport protocols frames (this seems to require that
> encrypted descriptors must provide a clear text identifier (and
> version?) or be discarded by hosts without the keys. - Sender
> listening to other senders may not be necessary in all environments
> for bandwidth control. Sender listening when descriptors are
> encrypted also forces the need for SAP "session" state maintenance as
> an expiry mechanism must be used without the session time information
> in SDP. - Prioritisation and timeliness of some announcements over
> others is desirable

I agree with most of these, except the last one. That's a transport 
issue. The problem with large packet size is that you then have to do 
application-layer fragmentation.

_______________________________________________
mmusic mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/mmusic

---

From: Henning Schulzrinne <[email-masked]>
Date: Tue, 17 Dec 2002 22:13:07 -0500

> gain from them show this is valuable. Does this seem rational?

It may indeed be helpful to note that there are environments with very 
asymmetric bandwidth availability.

> contributions in this space.

There are two ways to look at this problem in more general terms: 
multicast congestion control or multicast "bandwidth division". The 
former is a hard problem, the second somewhat easier. Indeed, it is 
possible that the RTCP "reconsideration" algorithm for scaling feedback 
may solve this problem.

> Also (at least conceptually) a piggy-backed message can say "X has changed, 

XPath may be the appropriate tool here for identifying elements, 
assuming that content is XML:

"The primary purpose of XPath is to address parts of an XML [XML] 
document. In support of this primary purpose, it also provides basic 
facilities for manipulation of strings, numbers and booleans. XPath uses 
a compact, non-XML syntax to facilitate use of XPath within URIs and XML 
attribute values. XPath operates on the abstract, logical structure of 
an XML document, rather than its surface syntax. XPath gets its name 
from its use of a path notation as in URLs for navigating through the 
hierarchical structure of an XML document."

> (push contains metadata, notify only identifies it). However, this does not 

I don't see this division. Both can contain references to existing data 
(e.g., via external references). Notify might actually update data if it 
is small - after all, there is no point spending a whole packet to say 
that four characters (the port number, say) has changed and then require 
that the user fetch this data.

> IMG and especially with respect to the unidirectional/multicast transport.

I agree that in carousel mode, more important data should be transmitted 
more frequently. It is not clear that, for a large group of receivers, 
that there is data that is significantly more important than others. If 
the interests are sufficiently diverse and evenly distributed, everybody 
will have to wait. Clearly, the proliferation of cable channels in the 
US and the relative decrease of the importance of the standard broadcast 
channels attest to this.

I suspect that for expensive channels, a system where the receiver 
ditches most announcements is not particularly viable. On the last-hop 
wireless link, you don't gain anything by doing multicast. You want a 
server that recognizes that the mobile has joined and then push the data 
down in one quick session.

Satellite seems to be the only really interesting one-way connection. 
Few satellite terminals are on small batteries. (XM and Sirius satellite 
radio might be one exception, although it currently seems to only be 
offered for cars and home.)