Subject: comments on draft-ietf-fax-tiff-fx-Extension1-00.txt

From: "Platt, Tim" <[email-masked]>
Date: Wed, 21 Feb 2001 08:59:03 -0500

To whom it may concern,

Hello

Not having commented on this list before, allow me to introduce myself.
I am a developer with ScanSoft Inc., the leading provider of digital imaging
software. ScanSoft develops OCR, Web publishing, electronic forms, document
management, and media asset management software.

I have reviewed the draft and in general it is fine.  My comments appear
below in descending order of importance.  At the end of the document are
editing comments that you may or may not want to pay attention to.

Section E1.7, JBIG2 Color Extension of M Profile issues

Consider a page that contains mostly small text, which has evenly colored
characters.  There is however, large text in a title or headline that has a
color gradient running from top to bottom.  The top of the title characters
is a light color and the bottom is a dark version of the same color. For
generality, assume that the large title or headline text cannot be isolated
to a strip or stripe of its own.

I would like to take advantage of the extensions proposed to make the
smallest file possible.  I want, however to be able to run OCR on all text,
including the title or headline text for indexing purposes.

To encode the large text with the color gradient faithfully, and yet use
color tags to increase compressions I need to separate the title or headline
text into a separate IFD.  I have to put the text I wish to encode using the
color tags in a separate IFD.  Since the title or headline text may overlap
other text in the Y direction I will need to use the n-layer extension of
the M profile to put the title or headline text into a separate layer.  This
means I will have to OCR separate layers to retrieve all text for indexing.
Depending on the OCR engine used it may be more or less difficult.  I would
need to combine the results of the separate passes into a single OCR result
page to allow for indexing for searching, but also to allow hit highlighting
on any page displayed for the search hit list.

Why can't the color tags be associated with the mask layer?  They are
already there anyway.  Then we could have a foreground layer associated with
that mask layer that had actual color image data.  In the mask layer, a
JBIG2 symbol that had color tag data would be rendered with that color.
JBIG2 symbols without color tag data would be rendered with the color from
the foreground.  If a sufficient amount of the foreground color is missing
or some don't care color (like black) it would compress very well.  This
would handle the situation I have given above and yet leave all the text in
the mask layer so only one OCR pass would be required.

Another member of my group, who is leading the segmentation development
effort, has recently asked me to ask you gentlemen if there would be such a
provision to mix color tags and foreground color data for a single mask
layer.  He did this before I reviewed the your proposal, so it can be
considered an independent corroboration of my request.

Now, my fall back position is to have some way to mark in the file what mask
layers are to be considered for OCR purposes.  Perhaps a bit can be assigned
form the NewSubFileType field that would indicate this.  Note that this is a
fall back position and not what I would really like to see.

Section E1.6, T profile - Lossy and Lossless JBIG2 Black-and-White Fax
profile issues

Much of the same arguments about OCR and simpler mask layer planes above
apply to black and white images where there is reversed video text.  In this
case, one could encode in a full M profile.  The foreground and background
layers would be used but with only black and white content. Presumably, even
though this black and white foreground/background data were encoded with >
one bits per pixel, they would still compress well.  Better yet, if JBIG2 is
used, then such a page could be encoded with a binary background, and a
mask/foreground combination with color tags.  This would require allowing a
binary background.  I do not have an opinion as to whether this should be a
relaxation to the M profile or and extension to the T profile.  If you did
not want to use JBIG2 for the Primary IFD for some reason, then you cannot
enable color tags.  Why should you not be able to have a binary background
AND foreground to indicate areas of reversed video while allowing OCR on all
characters in the mask layer?

Other issues

In section E1.5.4.1 "may include a list of IDs in a SharedDataList".  It
might be good to note that SharedData IDs may be used in the future in some
other way.  You do not want to require a single SharedData ID to have a
two-byte count of one in front of it.  For instance, a stamp annotation may
wish to point to a single SharedData ID so to keep stamp annotation small we
do not want the extra two byte count overhead of a SharedDataList.

Editing comments

E1.4.1 (around line 11) does not read very well (to me)
E1.4.4 (just before E1.4.5 line 48/49) this does not appear different from
8/10/00 version of RFC 2301

Thank you very much for your attention

Timothy J. Platt
[email-masked]
(978) 977-2068

---

From: "McIntyre, Lloyd" <[email-masked]>
Date: Wed, 21 Feb 2001 16:13:49 -0800

Tim,
Thanks you for your very constructive comment.

Please see my response below,
Lloyd

> -----Original Message-----
> From: Platt, Tim [mailto:[email-masked]]
> Sent: Wednesday, February 21, 2001 5:59 AM
> To: [email-masked]'
> Subject: comments on draft-ietf-fax-tiff-fx-Extension1-00.txt
> 
> 
> To whom it may concern,
> 
> Hello
> 
> text into a separate IFD.
This is close but not quite accurate. The title or headline text needs to be
placed in a separate TIFF strip from the one containing the regular text,
however both strips would be contained within a single mask layer IFD. The
gradient color of the title or headline text would be represented in a
foreground IFD that corresponds to the title or headline text strip within
the mask layer IFD. The colors of the regular text (i.e. flat non-gradient
colors) would be in a separate foreground IFD.
                   Page Image
             _________________________
            |  Gradient Color Title   |
            |                         |
            |      Regular text,      |
            | composed of flat non-   |
            | gradient colors.        |
            |  dkjtieh;jgjgjgejhjjjig |
            |  thogh ghotgjrepggh hf. | 
            |_________________________|

                                      JPEG coded Foreground layer
   JBIG2 coded Mask layer IFD                    IFD 1
         with 2 strips                  -------------------------
   -------------------------           ]      Gradient Color     [
  ]  Gradient Color Title   [           -------------------------
  ]+++++++++++++++++++++++++[
  ]      Regular text,      [      Color Tag coded Foreground layer
  ] composed of flat non-   [                    IFD 2
  ] gradient colors.        [           -------------------------
  ]  dkjtieh;jgjgjgejhjjjig [          ]   Regular flat colors   [
  ]  thogh ghotgjrepggh hf. [          ]     colors   colors     [   
   -------------------------           ]     colors   colors     [  
                                       ]     colors   colors     [ 
                                       ]     colors   colors     [ 
                                        -------------------------

For more complex pages where it is not possible to split the color gradient
text and the flat color text into two or more strips, it may be best to
forego use of color tags and simply JPEG encode the entire foreground layer.
The file will not be as small, however, you will retain a simple structure
that is conducive to OCR.

>  I have to put the text I wish to 
> encode using the
Given the representation above, which contains a single mask IFD with two
stripes (i.e. stripe 1 containing title or headline text plus any regular
text that may be overlapped in the Y direction and stripe 2 containing
regular text) and two foreground IFDs, there is no need to use n-layers. OCR
can be run on the single mask layer to retrieve all text for indexing.

> Why can't the color tags be associated with the mask layer? 
They ARE associated with the mask layer.  Please see Section E1.7.7.2 on
color tag terms of use, the color tags are augmented to the binary JBIG2
portion, but referenced together with the binary image as the foreground
image.  

> They are
> the foreground. 
There is no way to handle two separate color encodings within a single
foreground IFD, additionally mask layers are strictly binary - mask layer
IFDs cannot have color content. The TIFF rules require different encodings
to be placed in separate IFDs. In the representation above, the binary mask
layer IFD is JBIG2 encoded (i.e. each symbol instance is represented by a
triplet consisting of its XPosition, YPosition plus symbol ID). Foreground
IFD1 is associated with the first strip of the mask IFD and it contains the
gradient colors of the title or headline text. Foreground IFD1 is coded with
a convention color coder such as JPEG. Foreground IFD2 is associated with
the second strip of the mask IFD and it contains the flat colors of the
regular text, which is color tag encoded. This means that the foreground
image is also JBIG2 encoded and each JBIG2 symbol instance is augmented with
a color tag or color name. This results in quad, consisting of XPosition,
YPosition, symbol ID plus color tag. Obviously the JBIG2 coded XPosition,
YPosition and symbol ID portion of the foreground quad is identical to that
of the mask triplet. The draft recognizes this redundancy and accommodates
it by requiring only a single representation of the JBIG2 bi-level data. The
mask IFD would contain the ColourTagsFollow flag, alerting the decoder that
the foreground will point to the same block of data that is used by the mask
IFD. The byte count is, however, increased to include the color tags.

> If a sufficient amount of the foreground 
> color is missing
As stated above, using a single mask IFD that contains two or more strips
plus two or more corresponding foreground IFDs, make it possible to mix both
color tags and normal bit-map color representation while using a single
JBIG2 mask.

> fall back position and not what I would really like to see.
I hope that you are now comfortable that this is not necessary.

> layers would be used but with only black and white content. 
Profile T does allow 3 layers, with the explicit reasoning of allowing the
reverse video scenario.  In this case only black and white are allowed as
background and foreground colors, and are set using the ImageBaseColor
field, and the 0-byte strips option. This is addressed specifically in
Section E1.6.2.1, StripByteCounts.

> Presumably, even
Given that the draft does handle reverse video within the context of the
rules, there is no need to bend or break the MRC model or TIFF-FX rules.

> do not want the extra two byte count overhead of a SharedDataList.
If the intention is to reduce the file size, then the alternative that I
think you're implying is to use a 1-byte count instead.  This might be
limiting, in that a single data block could only reference up to 255 shared
data items.  This may or may not be reasonable, but it would be foolish to
limit it like this.  I believe that the savings in using a shared resource
for something larger than two bytes will make up for something that would
otherwise be more than two bytes, repeated in every page.
If you suspect that many data items will have a count of zero, and that they
will all incur the 2-byte overhead, then define the data stream such that no
SharedDataList is ever used.  It's only for data that might use a shared
resource, and in this case, must at least have the 2-byte count of zero.
I believe it is appropriate to strengthen the requirement by changing the
word "may" in the first sentence of E1.5.4.1 to "SHALL".

> E1.4.1 (around line 11) does not read very well (to me)
I will revise.

> 8/10/00 version of RFC 2301
I will delete.