Subject: Re: [rfc-i] character sets, Bug in RFC Search page...

From: Leonard Rosenthol <[email-masked]>
Date: Sun, 1 Oct 2017 17:23:06 +0000

There is a standard BOM for UTF-8, just as there is for UCS2.  Seems like a good idea, though most systems don't use it...

Leonard

-----Original Message-----
From: rfc-interest [mailto:[email-masked]] On Behalf Of John Levine
Sent: Wednesday, September 27, 2017 1:54 PM
To: [email-masked]
Cc: [email-masked]
Subject: Re: [rfc-i] character sets, Bug in RFC Search page...

In article <[email-masked]> you write:
>Given how browsers are quite inconsistent in the characer sets they 
>load, it would be nice to have explicit indication of the character 
>sets required to render a document. I've seen PDF documents where i had to go to page 100 before some crucial text was not rendered because it used some character set not available to me.

Those are buggy PDFs.  A PDF is supposed to contain all the fonts it uses beyond a small standard set, but there is a lot of PDF crudware that wrongly assumes that whatever fonts it finds on the machine where it's running will exist everywhere.  

>There is a lot of tool chains out there that will not render UTF-8 
>correctly, therefore it is IMHO extremely viable to have a clear 
>indication of the minimum toolchain features required to render a 
>document correctly (in the first page header of an RFC wold be great).

Hey, maybe we could put a couple of bytes at the begining that say this is a UTF-8 document.

R's,
John
_______________________________________________
rfc-interest mailing list
[email-masked]
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.rfc-editor.org%2Fmailman%2Flistinfo%2Frfc-interest&data=02%7C01%7C%7Cbe3f71e51afd492186ab08d50819e6f5%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636423829726381788&sdata=rmOSOTx4azrI97SsPWivmcIKHdwQy%2Fcfk7aA0hupTnc%3D&reserved=0
_______________________________________________
rfc-interest mailing list
[email-masked]
https://www.rfc-editor.org/mailman/listinfo/rfc-interest

---

From: Martin J. Dürst <[email-masked]>
Date: Mon, 2 Oct 2017 10:27:31 +0900

On 2017/09/28 03:04, Toerless Eckert wrote:
> On Wed, Sep 27, 2017 at 05:53:48PM -0000, John Levine wrote:
>> Those are buggy PDFs.  A PDF is supposed to contain all the fonts it
>> uses beyond a small standard set, but there is a lot of PDF crudware
>> that wrongly assumes that whatever fonts it finds on the machine where
>> it's running will exist everywhere.
> 
> I have stopped trying to really understand all those details for the last
> two decades, but i fear what you describe sounds like the original intention
> of adobe when they thought fonts embedded in PDFs could not be extracted
> and (illegally) be reused outside of the PDF. Once that hack happened (2 decades ago ?),
> the problem of licensed/payware fonts and their use has AFAIK lead to
> less inclusion of fonts into PDF.

For us, that should be irrelevant, because the fonts that we are 
(planning on) using are open source, and can be embedded without 
problems. Also, as far as I understand, the variant that we choose for 
the official PDFs actually requires that all fonts used (except for a 
very small base set) are included.

Regards,   Martin.

P.S.: In addition, please note that font embedding usually uses 
subsetting, and our documents have very few non-ASCII characters, so the 
additional memory needed for font embedding isn't much of a deal.
_______________________________________________
rfc-interest mailing list
[email-masked]
https://www.rfc-editor.org/mailman/listinfo/rfc-interest

---

From: Martin J. Dürst <[email-masked]>
Date: Mon, 2 Oct 2017 10:35:15 +0900

On 2017/09/27 07:32, Matthew Kerwin wrote:

> Emoji are astral codepoints, aren't they? 🤞🏻

Many of them indeed are, but not all of them

> Given how browsers are quite inconsistent in the characer sets they load,
> it would be nice to have explicit indication of the character sets required
> to render a document.

> If you mean font support (I'm not going to trip myself up over the
> difference between character sets and encodings and all that, but I'm
> pretty sure 'Unicode' has you covered for characters/codepoints) that's a
> mostly-solved problem in the modern web, with webfonts and the like.

Webfonts is one thing. The other is that even if we move beyond ASCII, 
we are still very conservative. The chance that an RFC soon will use 
some of the characters accepted in some recent version of Unicode is 
quite low (unless somebody wants to make a point, in which case the RFC 
editor will probably stop them). And for the characters from older 
Unicode versions, the OS and/or the browser has the necessary fonts 
available, unless somebody is using a hopelessly outdated OS/browser 
(which would be a bad idea anyway).

Regards,   Martin.

---

From: Martin J. Dürst <[email-masked]>
Date: Mon, 2 Oct 2017 10:41:46 +0900

On 2017/09/27 06:57, Toerless Eckert wrote:

> There is a lot of tool chains out there that will not render UTF-8 correctly,
> therefore it is IMHO extremely viable to have a clear indication of the
> minimum toolchain features required to render a document correctly (in the first
> page header of an RFC wold be great).

First, most of the 'toolchain' doesn't render anything, and for most 
purposes, will just pass through UTF-8 if it passes through all 8 bits.

Second, rendering usually works so that there might be something 
illegible, but the rest of the document (in ASCII) isn't affected. And 
the guidelines by the RFC editor are designed so that the document can 
still be read in such a case.

Third, I think overall it's much easier to fix the remaining tool chains 
than to have everybody be detracted by additional distinctions that 
should be irrelevant in this day and age.

Regards,   Martin.

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 2 Oct 2017 03:57:23 +0200

Most users would not even be able to figure out why something is not diplsayed
right with some tool unless they have some easy way to map the problem to
some type of content that they are aware of. Therefore such a header
would get very useful content information.

On Mon, Oct 02, 2017 at 10:41:46AM +0900, Martin J. D?rst wrote:
> illegible, but the rest of the document (in ASCII) isn't affected.
> And the guidelines by the RFC editor are designed so that the
> document can still be read in such a case.
> 
> Third, I think overall it's much easier to fix the remaining tool
> chains than to have everybody be detracted by additional
> distinctions that should be irrelevant in this day and age.
> 
> Regards,   Martin.

---

From: Leonard Rosenthol <[email-masked]>
Date: Mon, 2 Oct 2017 14:59:16 +0000

According to RFC 7995:
For RFCs, require PDF/A-3 with conformance level "U".  This
      captures the archivability and long-term stability of PDF 1.7
      files, mandatory Unicode mapping (Sections 14.8.2.4.2 ("Unicode
      Mapping in Tagged PDF") and 9.10.2 ("Mapping Character Codes to
      Unicode Values") of [PDF]), and many of the requirement features.

PDF/A-3 requires that *all* fonts be embedded – so this should never be a problem.

Leonard

On 10/1/17, 9:28 PM, "rfc-interest on behalf of Martin J. Dürst" <[email-masked] on behalf of [email-masked]> wrote:

    
    
    Regards,   Martin.
    
    https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.rfc-editor.org%2Fmailman%2Flistinfo%2Frfc-interest&data=02%7C01%7C%7Ce4f3c149651b412d8c9b08d50934ca37%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636425044797831153&sdata=PKyyCPHSCVBoLn4%2BxsydsCeXES6LekLkAydieM1vjew%3D&reserved=0

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 2 Oct 2017 17:46:35 +0200

Nice! 

I just do not know why there seems to be disinterest to have
human readable document format information/disclaimer. Heck, every other
interest group has managed to get considerations & disclaimers into RFCs
(security, IANA, TSV, rfc2119, et. pp).

"This PDF RFC was created to ensure maximum compatibility. It requires
no external data to display (such as fonts), and the formatting
does ot use features newer than PDF 1.7 defined in 2007. If you encounter problems
with this document a viewer/renderer newer than 2012, please call 1-800-RFC-EDIT"

;-P

Cheers
    Toerless

On Mon, Oct 02, 2017 at 02:59:16PM +0000, Leonard Rosenthol wrote:
> PDF/A-3 requires that *all* fonts be embedded ??? so this should never be a problem.
> 
> Leonard
>

---

From: "John R Levine" <[email-masked]>
Date: 2 Oct 2017 12:09:45 -0400

> with this document a viewer/renderer newer than 2012, please call 1-800-RFC-EDIT"

"This RFC is in PDF/A-3, rather than what some randome buggy PDF software 
creates.  If you encounter problems with this document using a viewer, you 
need a better viewer."

There is no secret to creating PDFs that work, other the secret that you 
really should read the specs to know what to do.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly
_______________________________________________
rfc-interest mailing list
[email-masked]
https://www.rfc-editor.org/mailman/listinfo/rfc-interest