Subject: [MMUSIC] comments on draft-johansson-mmusic-image-attributes-00

From: "Even, Roni" <[email-masked]>
Date: Tue, 3 Jun 2008 14:52:15 +0300

Hi,
I read the draft and have some comments

1. On section 5 issue 2 you suggest that the encoder will crop the
image, this may lead to having an irrelevant part of the image. For
example cropping a QCIF size from a HD camera may give a small top left
part of the image with no content that may be in the middle.

2. My understanding is that the size negotiation is to enable a decoder
with low processing power to decode without any scaling requirements.
For H.264 and future codecs this may still lead to geometric distortion
due to  different picture and sample aspect ratios. Example 4:3 or 16:9.
To have a full solution you should be able to declare your preferred
aspect ratio. Such work was approved last month in ITU-T H.241

Roni Even

_______________________________________________
mmusic mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/mmusic

---

From: "Ingemar Johansson S" <[email-masked]>
Date: Tue, 3 Jun 2008 21:18:49 +0200

Hi

Thanks Roni and Colin for the comments. I skimmed through RFC4175 and my
immediate reaction is that the parameters width and height applies to
the encoding. The proposed SDP attribute draft tries to separate the two
as the encoder may only be able to encode certain sizes with a limited
granularity.

Issue 1 below:
It is true that cropping of the image may cause a problem but I would
believe that the example below is a bit extreme. A more likely example
is maybe that the image is cropped 30%, which may indeed cause problems
with missing heads etc. One solution is that the sender can see what the
other party receives for instance in a small window but he consequences
must probably be studied more.

Issue 2 below:
Geometric distorsion is a problem, in an early drop of this draft I had
aspect ratio as an open issue but I removed it for some reason. I guess
aspect ratio is a bit tricky as it is a rational number. Perhaps one
could steal something from H.241 here. 

Besides video I can see other uses such a shared whiteboard
applications. In a office environment there is often problem with the
sender in e.g a sametime session that use a way to large screen. In the
other end the screen is resampled, something that often renders the
image useless. If the sender image size is adjusted to fit the receivers
the image will be much better as e.g the vector graphics and truetype
faces are scaled without distorsion. OK we may have scalability issue
here.

BTW!
What working group does this discussion belong to ?, it seems like it is
a mix of payload format and session setup. I cross-posted to avt this
time (promise I will not do it again).
 
Regards
Ingemar

> -----Original Message-----
> From: Even, Roni [mailto:[email-masked]] 
> Sent: den 3 juni 2008 13:52
> To: [email-masked]
> Cc: Ingemar Johansson S
> Subject: comments on draft-johansson-mmusic-image-attributes-00
> 
> Hi,
> 1. On section 5 issue 2 you suggest that the encoder will 
> crop the image, this may lead to having an irrelevant part of 
> the image. For example cropping a QCIF size from a HD camera 
> may give a small top left part of the image with no content 
> that may be in the middle.
> 
> 2. My understanding is that the size negotiation is to enable 
> a decoder with low processing power to decode without any 
> scaling requirements.
> For H.264 and future codecs this may still lead to geometric 
> distortion due to  different picture and sample aspect 
> ratios. Example 4:3 or 16:9.
> To have a full solution you should be able to declare your 
> preferred aspect ratio. Such work was approved last month in 
> ITU-T H.241
> 
> Roni Even
> 
> 
>

---

From: "Ingemar Johansson S" <[email-masked]>
Date: Wed, 11 Jun 2008 09:06:56 +0200

Hi

Some comments to Roni's comments below.

1) Cropping issues
  Only way to avoid cropping of the image with this approach is that the
sender rescales the input video.        In cases that the sender has the
processing capacity for this it should probably be possible. In the case
that the sender is also low end, cropping is probably the only way
forward and the problems that comes with this can maybe be avoided with
simple techniques such as the display of a small low quality "what the
reciver see" image on the senders screen. 

2) Sample and picture aspect ratio.
  After reading the H.241 amendment and H.264, this is how I believe
things would fit with SDP and the offer/answer model.

I am not sure that PAR (picture aspect ratio) is needed as the x and y
ranges are specified, possibly it can be used to limit the number of x
and y combinations.

SAR (sample aspect ratio) may be useful to avoid distorsion. Here I
assume that the receiver tells the sender the sample aspect ratio of the
receiver display and the sender adjusts its encoder image accordingly.
In other words the rescaling burden of the image is put on the sender
side, which I believe  makes sense as the purpose of this draft is to
make possible to decode and display video in the received with low
complexity. A sender that does not have the processing capacity should
not include SAR in the offer. This may happen with two low-end terminals
communicating with each other.

The offer may then look like 
Offer:
a=imageattr:98 landscape [sar=[1:16],x=800:-16:480,y=640:-16:320]
  Meaning, the sender can support appropriate rescaling for all
aspect_ratio_idc in tabell E1 i H.264

The answer may look like
Answer:
a=imageattr:98 landscape [sar=2,x=800,y=640]
  Meaning that the reciver display has a sample aspect ratio of 12:11
according to table E1 in H.264.

Alternatively the below offer may be used for support for extended SAR
Offer:
a=imageattr:98 landscape [sar=255,x=800:-16:480,y=640:-16:320]
Answer:
a=imageattr:98 landscape [sar_w=16,sar_h=11,x=800,y=640]

/Ingemar