Subject: RE: JPEG in INVITE

From: Taji Rachid <[email-masked]>
Date: Thu, 23 Dec 1999 11:35:09 +0100

Hi alvir,

I think you can do that in SDP part like this:

c= "IN"  "rfc2543"  IpAdressOfReceiver   
m= "image" portOfThisIPAdress "UDP" " JPEG"
a= "fmtp:JPEG"  "uri:"http://mycampany/xxx.jpeg 

portOfThisIPAdress :  is the port to wich the image will be sent  
uri   : is the uri of the image
IN    :stands for internet.
You can get more information in RFC 2327 SDP protocole

Merry christmass an Happy new Year

> -----Original Message-----
> From:	Alvir Alisic (ECS) [SMTP:[email-masked]]
> Sent:	Donnerstag, 23. Dezember 1999 10:13
> To:	[email-masked]
> Subject:	JPEG in INVITE
> 
> Hey!!!
> 
> I want to send JPEG pictures in my INVITE messages, but I don't know how I
> can do that. Is there anyone who can help me???
> 
> Marry Christmas and Happy New Year
> 
> Kind Regards
> Alvir
> 
> Alvir Alisic
> M. Sc Candidate - Datacom Software
> Ericsson Mobile Communications AB
> Lund, Sweden
> mailto:[email-masked]  
> phone:+46 (0)46 19 38 39
> 
>

---

From: Jonathan Rosenberg <[email-masked]>
Date: Thu, 23 Dec 1999 09:45:29 -0500

There are two basic ways to do this, both of which have now been
mentioned on the list. The first is to treat the image as part of the
signaling, and include it somehow in the INVITE. This would be
accomplished, as Scott has suggested, using MIME multipart, with one
type possibly being image/jpeg. We have discussed using the MIME
Content-Disposition header to allow the UAS to know the purpose of the
MIME content. To date, we have seen MIME bodies needed for several
reasons:

1. for session descriptions
2. for display
3. for tunnelled signaling (ISUP)
4. for billing/accounting (OSP tokens)

The reason I mention this is that I think rather than including the
image in SIP itself, its a bit better to include the type text/uri-list,
which is just a list of URIs. Then, one of the URIs is an http URL that
points to the JPEG image. When one does this, the Content-Disposition
header becomes important. This is because the URI might point to any one
of the 4 content disposition types above, and to know how to handle it,
the UAS needs to know which before its even fetched. We will also need,
I think, an additional parameter which indicates whether the body is
optional or mandatory. Scott has suggested that for multipart, any
unknown bodies are discarded. I'd rather be explicit about that.

The second approach is to have the image be a media stream. This would
be done by including an additional m line in the SDP. The formatting
suggested by Taji is not quite right though. It would look like:

m=image 49170 RTP/AVP 55
a=rtpmap:55 jpeg
c=IN IP4 10.1.1.1

So, whats the difference? Which is the "right" way? In this case, I
believe inclusion in SIP is the right way. There are several reasons for
this:

1. The purpose of the image is to assist the user in deciding whether to
accept or reject the call. We have headers like Subject in display name
which serve a similar function; these are part of the signaling
function.

2. Security can be better when this is placed in signaling. When a UAC
includes a jpeg image or URL, its outgoing proxy can verify these as
being correct (based on a preconfigured list of allowed content and
URLS, perhaps), and then resign the request. This provides a form of
enhanced caller ID service, as the identity of the caller is verified by
a service provider now, whose key a called party is more likely to have.

3. We save the complexity and overhead of establishing a media channel
just for a single image.

The line between what to do in SIP, and what to do in a media stream, is
hard to draw. If the content is not interactive (like a static image
that already exists), it affects how the user is likely to handle the
call, and it can be obtained through redirection, SIP might be the
better choice.

-Jonathan R.

-- 
Jonathan D. Rosenberg                       200 Executive Drive
Chief Scientist                             Suite 120 
dynamicsoft                                 West Orange, NJ 07052
[email-masked]                     FAX:   (732) 741-4778
http://www.cs.columbia.edu/~jdrosen         PHONE: (732) 741-7244
http://www.dynamicsoft.com

---

From: Henning Schulzrinne <[email-masked]>
Date: Thu, 23 Dec 1999 10:00:15 -0500

Another alternative is to define a new header, e.g.,

Face: http://sln.fi.edu/franklin/inventor/images/belllg.jpg

There's a trade-off in that this mechanism is more in line with its
function (and much simpler than parsing through body parts and URI
lists), but a receiver would have to recognize the header name rather
than just a generic "display this and don't ask what it's for" approach
of an attachment. I suspect for common functionality such as this,
headers will turn out to be a better approach since they can be more
explicit about function.
-- 
Henning Schulzrinne   http://www.cs.columbia.edu/~hgs

---

From: "Doug Sauder" <[email-masked]>
Date: Thu, 23 Dec 1999 10:16:32 -0500

Just brainstorming here a little...   The "cid:" URL protocol allows the referenced object to be one of the MIME body parts.  Thus, an HTTP connection would not have to be opened to get the image.

--
Doug Sauder
Broadsoft, Inc.

---

From: Jonathan Rosenberg <[email-masked]>
Date: Thu, 23 Dec 1999 11:01:53 -0500

I kind of like using a multipart type, since this allows the case of
directly included and indirectly included (via a URL) to be the same. It
also means we can include things for a broader purpose than just
"faces".

-Jonathan R.

---

From: Jonathan Rosenberg <[email-masked]>
Date: Thu, 23 Dec 1999 11:05:41 -0500

Well, my point was that you want to do that in many cases. SIP is not a
transfer protocol; running ontop of UDP means large bodies will be
problematic. Plus, you don't want to be shipping all these bodies around
if the far side doesn't need them. By putting a URL, the UAS can get the
image if its capable of displaying it, otherwise, not much is lost.

-Jonathan R.

---

From: Jonathan Rosenberg <[email-masked]>
Date: Fri, 24 Dec 1999 13:59:35 -0500

My concern with doing this is that there is no way we could hope to
categorize the reasons for obtaining an image. You yourself list some
five different purposes. I don't object to a text phrase or something
which can be displayed to the user, but anything with protocol semantics
is likely to make things more restrictive than we want to.

-Jonathan R.

"Daniel G. Petrie" wrote:
> 
> I do not really have any issue with sending the image as a url list or
> file in the body.  However if this is the way we decide, it is very
> important that context is provided for the image.  The image as
> a field provides this (i.e. faces).  The content-disposition or what
> ever the content labeling mechanism used, needs to give the UA
> a hint as to what the image represents.
> 
> Is it a picture of the caller, group phone of those on the conference,
> the conference host, a view out the window, an animated gif
> indicating please hold until a call representative is avialable. A
> context free image makes it difficult for the UA to do the right
> thing.
> 
> For example the UA may already have or know how to
> get an image for a caller.  If it knows that the image in the
> message is of the caller (i.e. face), it would OVERRIDE the
> default image retrieval.  If the image in the invite is not a face
> it might display BOTH the default caller image and the image
> (reference or contained) in the message.
> Cheers,
> Dan
> 
> Jonathan Rosenberg wrote:
>

---

From: "Daniel G. Petrie" <[email-masked]>
Date: Thu, 23 Dec 1999 16:20:07 -0500

---

From: Scott Petrack <[email-masked]>
Date: Sat, 25 Dec 1999 19:40:29 -0500 (EST)

1. One important reason not to use an indirection (i.e. including only the
URL of the image and not the image) is call set-up time. An indirection
requires at least one extra round trip time to get the image. Of course,
this is a standard trade-off, and in any case,
including a huge bitmap is not good for call set-up time either. But
sending along a relevant thumbnail is much less harmful to call set-up 
time than requiring the extra round-trips.

2. Perhaps this is what was meant by using the Content-Disposition header,
but I would suggest that there is lots of room in the tags of Content-Type
and Content-Disposition to signal a rich set of contexts and uses for the
image. Even just something like:

Content-Disposition: attachment; filename="caller id.tif"

would already be helpful. In any case, I doubt that we are ready to
standardize a set of such names yet.

---

From: [email-masked] (Henning Schulzrinne)
Date: Sun, 26 Dec 1999 12:07:09 +0100

There are clearly some common cases, such as visual caller id. In those
cases, I think that the URI-in-body method adds anything but ambiguity.
The callee application may already have a database of pictures. Unless
the purpose of the image is clearly indicated, it may end up rendering
both in two windows. If we go down that route, we may as well get rid of
headers altogether and just wrap all but To, From and Call-ID in body
parts. If we standardize functional declarations ("this body part shows
the caller's likeness"), we might as well standardize a header. As was
pointed out, if we really want to carry the JPEG along (say, the caller
doesn't have an externally accessible web page), then the cid: method
seems like a reasonable approach.

Thus, in general, headers seem to be appropriate where clear functional
labeling is useful and data volume is modest (and where the content is
for the whole session), while body parts can serve more as a "just
render this and let the human figure out what it means" methods.

---

From: Scott Petrack <[email-masked]>
Date: Sun, 26 Dec 1999 07:08:49 -0500 (EST)

Henning makes a good point. All I ask is that it be *possible* to include
the actual content within the message, for those cases where it makes
sense. In PINT, we wrestled with this problem, and allowed using the
standard MIME content-id to enable having something in the INVITE refer to
an included body-part. Thus, nothing new needs to be added. I'll bet one
could even use the multipart/alternative stuff to enable *both* pointing
to the data via a URL and including it within the message, to signal that
these are two different ways to get the same information.

Even when both sides have access to the same database of pictures (say
within a company-wide directory), it is still a lot faster to set up the
call by including the picture within the INVITE. Sometimes it makes sense
to do this, sometimes not. Since MIME signalling is already rich enough to
allow both, let's not dis-allow one by design.

Scott

---

From: Jonathan Rosenberg <[email-masked]>
Date: Sun, 26 Dec 1999 20:56:36 -0500

> these are two different ways to get the same information.

I wasn't arguing that it shouldn't be possible, but that I would put a
statement like "using URIs instead of the content itself is
RECOMMENDED". 

The issue is, if the content is referenced by indirection, is it better
to be in the body or in a header. Henning is arguing in favor of a
header. I agree with him that this is simpler. Both he and Dan are also
arguing that we need a way to identify the purpose of the content.
Providing such data also argues for it being a header. My concern,
though, is that there are so many uses for displaying of content, we
have no hope of enumerating them all. I suppose we could enumerate a few
and leave others to future standardization. Also, using a URI list in
the body is something thats doable now (since we support MIME already),
whereas a new header is yet-another-extension that I'd rather avoid. The
notion of a generic "body contains content to display", with the body
either in-line or referenced via a URI list, is appealing to me because
of its generality. Yes, we could have both, but then this seems to make
things even more complicated for a designer that now has to choose
whether to use the header or the indirect content in the body.

Am I the only one who prefers the indirect references in the body
approach?

-Jonathan R.

---

From: Sean Olson <[email-masked]>
Date: Sun, 26 Dec 1999 20:57:44 -0600 (CST)

> time than requiring the extra round-trips.

I would hope that you would do this in parallel with the call setup. I
can't think of a reason not to proceed with the call while fetching the
image in parallel.

-- 
-----------------------------------------------------------------
Sean Olson            E-mail: [email-masked]
Ericsson Inc.         Voice: (972) 583-5472 
                      FAX: (972) 669-0154

---

From: Scott Petrack <[email-masked]>
Date: Mon, 27 Dec 1999 04:29:22 -0500 (EST)

The callee needs to see the pic before s/he accepts the call.
Why is "caller ID" a useful feature? Because it gives the callee a way
to screen the call. It would be silly to give me a caller ID string in
parallel with accepting the call. 

In the future, let's imagine a little camera in a public phone
booth, so that the caller can call collect and the callee gets a thumbnail
sketch of the caller, to decide if s/he wants to accept the call. This
will reduce the pain of setting up a collect call immensely. (And there
are variants which are useful even if you don't believe that there will be
public phone booths in the future....)

To me, this is an example of a truly useful service which SIP enables. 
So yes, I need to see the picture in order to know if I want to take the
call.

Scott

---

From: Scott Petrack <[email-masked]>
Date: Mon, 27 Dec 1999 05:12:46 -0500 (EST)

My point was that even if we decide we need to signal the purpose
explicitly within a header, I believe that we don't have to add any new
extension at all. This is because MIME headers like Content-Type and 
Content-Disposition already allow semicolon-separated tags. Looking
through my own mail, I see lines like:

Content-Type: application/octet-stream; name="Board Presentation.ppt";

Could someone out there tell us where the complete list of such tags is to
be found? I'll bet that there are enough out there that we don't need to
invent anything new at all. And even if there are not, I would sooner
define a new tag to be used in Content-Type, Content-Disposition, etc. 
headers, than invent an entirely new header extension. 

This seems to me to give the best of both worlds: you use a new body part
to contain the information (be it content or a pointer to the content),
and you use tags in the headers that deal with that content to describe
various attributes and features of them.

This also keeps features orthogonal, which I think is very important:
since one CAN put a body of Content-Type: application/URI, and this can be
made to work for our purpose, we shouldn't invent a different way to do
the same thing. Especially if we can use tags in the existing headers to
signal the stuff we need. Implementations that don't understand these tags
already have to throw them on the floor, so it adds no backward
compatibility problems.

Are there any experts who can tell us what tags are available?

Scott

On Mon, 27 Dec 1999, Henning Schulzrinne wrote:

> In general, even though it does add "yet another extension",
> explicitness of purpose is good, since it allows the receiver to deal
> with things appropriately. In the photo case, this may mean updating an
> internal picture database.
> 
> This is a generic problem. Clearly, we could put things like priority or
> subject or other call information in a web page, but it would be
> unstructured information.
> 
> Another issue is that the display mechanism only works with a single
> attachment, unless you want to have windows pop up all over the screen.
> If we want to provide additional call-identifying information beyond a
> single item, we clearly need better labeling and "first class" protocol
> status rather than just a collection of attachments.
> 
> Having a generic display capability for web pages (which includes JPEG
> pictures as a special case) is a good thing, too. So I don't see this as
> an either-or issue.
> 
> I don't think the delay is much of an issue for this application since
> the callee should be able to accept the call even before the picture
> rendition is complete (e.g., if she recognizes the caller by name). Even
> at the typical modem download speed of around 3 kB, a typical picture
> would take about 3 seconds to download, probably faster than most of us
> either run to the phone or even turn to the phone device. If the callee
> is a fashion magazine or the Miss Universe contest office, it might
> design its phones to only ring once the image has been received...
> 
> Henning
>

---

From: Jonathan Rosenberg <[email-masked]>
Date: Mon, 27 Dec 1999 10:53:47 -0500

There are two issues we have been discussing:

1. What is the right way to access content indirectly? A new header in
SIP, or a body with application/URI-list?

2. How do you convey the reason for the content?

These are really separate problems, although the solution for (2)
depends a lot on the solution for (1).

Your suggestion below is to use tags in Content-Type. That solves the
second problem given the solution to the first is to use a body with
application/URI-list.

Others have suggested that indirect content be accessed directly with a
SIP header, such as:

INVITE sip:[email-masked] SIP/2.0
Display: http://www.dynamicsoft.com;purpose=web,
http://www.dynamicsoft.com/pictures/jdrosen;purpose=face

In this case, (2) is solved with some new paramters to this new header.

-Jonathan R.

---

From: Dean Willis <[email-masked]>
Date: Mon, 27 Dec 1999 11:16:36 -0600

> time than requiring the extra round-trips.

Another reason not to indirect is the security/firewall/web-posting
interworking complexity.

User A is behind the firewall of company B. User C is behind the firewall of
company D.

The firewalls of C and D must pass or proxy SIP for A and B to make a SIP
call.

A wants to call B and include a caller-id image, from a phone "UAC(A)" that
includes a caller-ID camera.

With in-SIP images:

A activates the UAC, dials B, producing image I.
UAC(A) INVITES UAS(B) with image
UAS(B) receives I and displays it.

With out in-SIP, case 1, UAs have HTTP clients and servers.

A activates the UAC, dials B, producing image I.
UAC(A) posts I to internal HTTP server
UAC(A) INVITES UAS(B)with URL(I)
UAS(B) receives INVITE with URL(I)
UAS(B) opens HTTP connection to UAC(A), gets and displays I.

Do you really think the corporate security people at company C are going to
allow HTTP GET operations from arbitrary DHCP addesses at D to come in thru
their firewall to arbitrary IP DHCP addresses at C? Not likely.

So, we have a new scenario.

We introduce WS(C), the web server at company C.

A activates the UAC, dials B, producing image I.
UAC(A) posts I to WS(C). (authentication? URL construction?)
UAC(A) INVITES UAS(B)with URL(I)
UAS(B) receives INVITE with URL(I)
UAS(B) opens HTTP connection to WS(C), gets and displays I.
Question: How does WS(C) know to let UAS(B) get I? It would seem UAC(A)
would need to provide B's identity information in the posting process.

So, to relieve the cost of sending a small image in an invitation, we have
added:
1) An external web server.
2) HTTP posting software in the UAC.
3) HTTP getting softeare in UAS.
4) A business and security relationship between the UAC and the web server.
5) An unspecified content management problem around the UAC and the web
server.
6) An unspecified authentication/authorization problem relating to the UAS
and the web server.
7) And, as Scott points out, a rather huge hit to call setup time. I count:
	Two TCP connection establishments
	Two HTTP authentications
	One HHTP POST
	One HTTP GET
	Two TCP teardowns
which adds up to a COUPLE OF DOZEN RTTs.

That strikes me as a rather painful alternative to a SIP INVITE with
multipart payload including a JPEG and transmitted over TCP, optionally
secured with SIP auth and encryption.

--
Dean

---

From: Sean Olson <[email-masked]>
Date: Mon, 27 Dec 1999 11:55:40 -0600 (CST)

> parallel with accepting the call. 

You are assuming of course that you can trust the UAC on the other end to
give you their own thumbnail and not that of someone else. If you want
to authenticate a call, there are better ways, PGP being one that comes
to mind.

> So yes, I need to see the picture in order to know if I want to take the

I can definitely appreciate the utility of such an application, but I can
never see it as a form of caller id for the purpose of screening calls.
Again, I think something like PGP is better suited for these purposes.

---

From: Scott Petrack <[email-masked]>
Date: Mon, 27 Dec 1999 09:19:22 -0500 (EST)

> to authenticate a call, there are better ways, PGP being one that comes

I would get whatever assurances the surrounding SIP security would give.
Perhaps the INVITE from a public phone would be secured with the public
phone company's key, and if I believe this key, then I would believe that
the picture is indeed sent by a camera in that phone. 

I was not suggesting that the people who operate the missiles controlled
by the "red button" would just believe the call because it came with a
thumbnail sketch of the President.

> Again, I think something like PGP is better suited for these purposes.

PGP would be used by the SIP system elements to help make sure that the
pictures are what they purport to be. 

We are debating how the information is put into the SIP message (will it
be in a header, or in the body, and what are the details?). However we
decide, the usual SIP security will help us. 

Scott

---

From: Jonathan Rosenberg <[email-masked]>
Date: Mon, 27 Dec 1999 14:32:22 -0500

> to authenticate a call, there are better ways, PGP being one that comes

It is possible to have the signature cover the thumbnail too. This would
allow you to be sure that the person who is calling inserted the
picture. But, that doesn't guarantee that the picture is actually of
that person.

However, this can be done. Lets say the UAC inserts a jpeg thumbnail
into the INVITE. The user has provided some 9 or 10 different thumbnails
that the user would like to use in different contexts. The service
provider has (somehow) verified that these are really pictures of the
person. Offline, there are numerous ways this can be done. For example,
the service provider could check public records from the Dept. of Motor
Vehicles and verify that the picture is of that person. Now, when the
user makes a call, they insert one of these pictures. The user uses an
outgoing proxy server. This server checks the included image, checks it
against the list of allowed pictures, and if its OK, resigns the
request, with the signature covering the picture. It also adds some kind
of flag to indicate that the caller identity has been verified. When a
UAS receives this, it can be certain that the picture is of the actual
caller, so long as the UAS trusts the service provider of the caller.

There are some definite problems with this, but interestingly its more
or less conceptually how the PSTN guarantees caller ID service (sans the
crypto, of course). The service provider verifies the identity of the
caller's terminal based on the physical wires the call comes in on, and
inserts the name of the caller. This name is assumed known from offline
administrative procedures.

I'm not suggesting we do this, but its a thought.

-Jonathan R.

---

From: Scott Petrack <[email-masked]>
Date: Mon, 27 Dec 1999 10:51:03 -0500 (EST)

You have said it very well, Jonathan. I am merely arguing that we can
satisfy Henning's requirements (the ones which led him to suggest using
headers) without inventing a new special purpose header, and that this is
the better approach. It gives him his headers and you your body ;-). 

Goaded on by ScottB, I found and read RFC 1806. It defines the allowed
types in "Content-Disposition:" header. The one we want is "inline":

"2.1  The Inline Disposition Type

   A bodypart should be marked `inline' if it is intended to be
   displayed automatically upon display of the message. Inline bodyparts
   should be presented in the order in which they occur, subject to the
   normal semantics of multipart messages."

In fact, here is the first example in the RFC:

"3.  Examples

   Here is a an example of a body part containing a JPEG image that is
   intended to be viewed by the user immediately:

         Content-Type: image/jpeg
         Content-Disposition: inline
         Content-Description: just a small picture of me

         <jpeg data>"

This is just what we need. URI-list fans would send:

	 Content-Type: application/URI
         Content-Disposition: inline
         Content-Description: just a small picture of me

         <URI>

So I guess that the header which means "display this data along with the
display of the INVITE"  already exists. Given this, what is the reason NOT
to use it, along with the URI-list in the body? I mean, I believe that the
above signalling is perfectly standard SIP, so why have a different way to
do the same thing?

Scott

(By the way, Sean, the second example in section 3 of RFC 1806 suggests an
authentication method for INVITEs which is superior even to PGP ;-)).

Scott

---

From: [email-masked] (Henning Schulzrinne)
Date: Mon, 27 Dec 1999 23:37:53 +0100

There's always the well-established challenge-response crypto method:

401 Authentication required
Challenge: "Close your left eye"

INVITE (retransmitted)
Response: uses camera, has at most a few seconds to get better-looking
coworker in front of camera

---

From: [email-masked] (Henning Schulzrinne)
Date: Mon, 27 Dec 1999 09:17:58 +0100

Henning

---

From: Jonathan Rosenberg <[email-masked]>
Date: Tue, 28 Dec 1999 00:13:33 -0500

Presumably the picture is already on a publically accessible web server,
and thus there is no need for an explicit POST at the time of call
setup. 

Scott Petrack writes:
> ou have said it very well, Jonathan. I am merely arguing that we can

Actually, at the last meeting, we discussed using Content-Disposition,
and inline was one of the parameters that seemed to work for us. I agree
that the body mechanism is more general; I think some people on the list
want (1) more information on *why* this information is being displayed,
and (2) an easier way to get at it than looking in bodies. 

I can understand (1), and we can solve this through additional
parameters in Content-Disposition that can be defined. I'm still not
convinced that making indirect references headers is so much faster and
easier its worth an extension plus eliminating a more general solution.

-Jonathan R.