Subject: [Xml-sg-cmt] new sourcecode types for "cbor-diag" and "cbor-hexdump" - RFC-to-be 9090 [RV]

From: Alice Russo <[email-masked]>
Date: Wed, 7 Jul 2021 09:44:37 -0700

Hi all,
Re: the list of preferred types for <sourcecode> (https://www.rfc-editor.org/materials/sourcecode-types.txt)
Q1- May we add "cbor-diag" and "cbor-hexdump" as Carsten suggested? Details are below, including his review of existing usage of type="cbor".
Q2- If changing to the finer-grained types, should "cbor" be removed from the list?

The current use cases are in this document in AUTH48:
https://www.rfc-editor.org/authors/rfc9090.html

Thanks,
Alice
--

[from mail regarding RFC-to-be 9090]

>> a) In Figures 1-6, we updated <artwork> to <sourcecode>. For Figures 2, 4, 5,
>> and 6, we used type="cbor”.
> 
> 
> CBOR will never actually be used in an RFC, as it is a binary format.
> 
> I made a quick, non-scientific survey of what we are doing today.
> 
> RFC 8727:
> uses <sourcecode type="cbor"
> for automatically generated "pretty" CBOR hex-dumps
> 
> RFC 8742:
> uses <artwork type="CBORdiag"
> for CBOR diagnostic notation
> 
> RFC 8746:
> uses <sourcecode type="CBOR"
> for automatically generated, but then annotated "pretty" CBOR hex-dumps
> 
> RFC 8747:
> uses <sourcecode type="cbor"
> for CBOR diagnostic notation
> 
> RFC 8949:
> uses <artwork type="hex-dump"
> for a less standard, manually annotated variant of "pretty" CBOR hex-dumps
> (That RFC uses sourcecode only for actual C and Python as well as for pseudocode)
> 
> So there are two CBOR notations we need to take care of:
> * CBOR diagnostic notation (defined in Section 8 of RFC 8949 and Appendix G of RFC 8610), and 
> * various forms of prettified, possibly annotated CBOR hex-dumps.
> 
> Actually, ASN.1/BER has the same problem here, except that there is a name for BER that could be used here, and “asn.1” stands for what would be CDDL for CBOR as well as the value notation as used in RFC 9090-to-be.
> 
> Figure 2, 4, 6 are prettified hex-dumps of CBOR data items.
> These should not be mixed up with figure 5, which is CBOR diagnostic notation.
> 
> We can use type=“cbor” only for one of these two.

[...]

> The existing sourcecode variants I found are:
> CBOR diagnostic notation is called “CBORdiag” and “cbor” in the above survey.
> Prettified CBOR hex-dumps are called “cbor”, “CBOR” and also <artwork type=“hex-dump”.
> 
> It seems the “standard” sourcecode types are all lower case, so “CBORdiag” and “CBOR” are out.  We could invent names such as “cbor-diag” and “cbor-hexdump”.
> 
> I am really not sure how to handle this situation.

---

From: "John R Levine" <[email-masked]>
Date: 7 Jul 2021 13:15:56 -0400

> https://www.rfc-editor.org/authors/rfc9090.html

I realize that CBOR is computer data but it doesn't smell like source 
code to me.  It's not like you can take the hex dump and feed it to a CBOR 
compiler.  I think artwork would be more appropriate here.

R's,
John

>> I am really not sure how to handle this situation.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

---

From: Alice Russo <[email-masked]>
Date: Thu, 8 Jul 2021 10:30:56 -0700

Thanks for your reply, John. Re:
> I realize that CBOR is computer data but it doesn't smell like source code to me.  It's not like you can take the hex dump and feed it to a CBOR compiler.  I think artwork would be more appropriate here.

Seems to get back to the past discussion of defining <artwork> vs. <sourcecode>, where the latter is not limited to what can be fed to a compiler. These figures don't look like "character-based art" or diagrams to me, so I'm surprised by <artwork>. 

https://www.rfc-editor.org/authors/rfc9090.html#figure-1
https://www.rfc-editor.org/authors/rfc9090.html#figure-2
https://www.rfc-editor.org/authors/rfc9090.html#figure-3
https://www.rfc-editor.org/authors/rfc9090.html#figure-4
https://www.rfc-editor.org/authors/rfc9090.html#figure-5
https://www.rfc-editor.org/authors/rfc9090.html#figure-6

Any other thoughts here? 

Thanks,
Alice

> On Jul 7, 2021, at 10:15 AM, John R Levine <[email-masked]> wrote:
> 
> I realize that CBOR is computer data but it doesn't smell like source code to me.  It's not like you can take the hex dump and feed it to a CBOR compiler.  I think artwork would be more appropriate here.
> 
> R's,
> John
> 
>

---

From: "John R Levine" <[email-masked]>
Date: 8 Jul 2021 15:18:28 -0400

>> Seems to get back to the past discussion of defining <artwork> vs. <sourcecode>, where the latter is not limited to what can be fed to a compiler. These figures don't look like "character-based ar…

Well, that's the problem.  If you look at RFC 7991, <artwork> is clearly 
for ASCII art, and <sourcecode> for source code and formal languages like 
ABNF.  But they don't seem to have considered examples that are chunks of 
text in a fixed layout that are neither in a formal language nor artwork 
other than to say that hex-dump is artwork.

> The real question is what is the result of choosing between artwork and sourcecode.

In practice, not much.  You can tag sourcecode with CODE BEGINS / CODE 
ENDS, and you can provide alignment hints for artwork.

> As long as the »type« attributes of the two elements differ in their 
> power, this appears to be the most important criterion for choosing.  I 
> think it would be good if the two machine-readable formats cbor-diag 
> (which is, essentially, JSON plus some salt) and cbor-pretty (which is a 
> graphically enhanced hex-dump, both artwork and tool input) are clearly 
> identified for future tools to grab them.  So with the current 
> interpretation of »type« for artwork, this would force »sourcecode«.

If you believe there will someday be tools that can do something with 
them and the formats are well enough defined to say what is a cbor-diag 
or cbor-pretty and what isn't, I'm OK with calling them sourcecode.

Regards,

---

From: Sandy Ginoza <[email-masked]>
Date: Fri, 9 Jul 2021 22:32:09 -0700

Hi all,

Adding another data point.  draft-ietf-cose-rfc8152bis-struct-15 <https://www.ietf.org/archive/id/draft-ietf-cose-rfc8152bis-struct-15.html#section-appendix.c> includes the following:

   The examples can be extracted from the XML version of this document via an XPath             
   expression, as all of the source code is tagged with the attribute type='CBORdiag'.

It then uses <sourcecode type="CBORdiag”> throughout.  

> It seems the “standard” sourcecode types are all lower case, so “CBORdiag” and “CBOR” are out.  We could invent names such as “cbor-diag” and “cbor-hexdump”.

I think “cbor-diag” would work here, but it’s not clear to me how/if these should be tagged and the text updated.  

Thanks,
Sandy 

> On Jul 8, 2021, at 12:18 PM, John R Levine <[email-masked]> wrote:
> 
> Well, that's the problem.  If you look at RFC 7991, <artwork> is clearly for ASCII art, and <sourcecode> for source code and formal languages like ABNF.  But they don't seem to have considered examples that are chunks of text in a fixed layout that are neither in a formal language nor artwork other than to say that hex-dump is artwork.
> 
> In practice, not much.  You can tag sourcecode with CODE BEGINS / CODE ENDS, and you can provide alignment hints for artwork.
> 
>> As long as the »type« attributes of the two elements differ in their power, this appears to be the most important criterion for choosing.  I think it would be good if the two machine-readable formats cbor-diag (which is, essentially, JSON plus some salt) and cbor-pretty (which is a graphically enhanced hex-dump, both artwork and tool input) are clearly identified for future tools to grab them.  So with the current interpretation of »type« for artwork, this would force »sourcecode«.
> 
> If you believe there will someday be tools that can do something with them and the formats are well enough defined to say what is a cbor-diag or cbor-pretty and what isn't, I'm OK with calling them sourcecode.
> 
> Regards,
> Please consider the environment before reading this e-mail. https://jl.ly-- 
> Xml-sg-cmt mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/xml-sg-cmt

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 8 Jul 2021 20:31:33 +0200

> On 2021-07-08, at 19:30, Alice Russo <[email-masked]> wrote:
> 

(Except for figure 5), they are two-dimensional, which is particularly visible in figure 1.
By design of the tool that was used, figure 2, 4, 6 are machine readable as well.

I tend to agree with John on the question “are these *really* »source code«?”.

But that is the sophistic question.
The real question is what is the result of choosing between artwork and sourcecode.

Grüße, Carsten

---

From: Alice Russo <[email-masked]>
Date: Mon, 12 Jul 2021 09:58:52 -0700

Hi Carsten,

Per CMT discussion today, these two have been added to the list on https://www.rfc-editor.org/materials/sourcecode-types.txt :
> cbor-diag (which is, essentially, JSON plus some salt) and cbor-pretty (which is a graphically enhanced hex-dump, both artwork and tool input) 

>>>> Figure 2, 4, 6 are prettified hex-dumps of CBOR data items.
--> type="cbor-pretty"

>>>> figure 5, which is CBOR diagnostic notation.
--> type="cbor-diag"
 
(Fig. 1 and 3 remain w/ empty type.)

Thanks,
Alice

> On Jul 8, 2021, at 11:31 AM, Carsten Bormann <[email-masked]> wrote:
> 
> 
>

---

From: Sandy Ginoza <[email-masked]>
Date: Mon, 12 Jul 2021 12:20:36 -0700

Hi again,

draft-ietf-cose-rfc8152bis-struct was submitted in v3 and seems to suggest adding a new sourcecode type: XPATH

        <sourcecode type="XPATH"><![CDATA[                                                             
//sourcecode[@type='CDDL']/text()                                                                      
]]></sourcecode>

<sourcecode type="XPATH"><![CDATA[//sourcecode[@type='CDDL']/text()]]></sourcecode>

Should “xpath” be added to the list of types <https://www.rfc-editor.org/materials/sourcecode-types.txt> or should these snippets be marked as something else?  

Thanks,
Sandy 

> On Jul 9, 2021, at 10:32 PM, Sandy Ginoza <[email-masked]> wrote:
> 
> Hi all,
> 
>> On Jul 8, 2021, at 12:18 PM, John R Levine <[email-masked] <mailto:[email-masked]>> wrote:
>> 
>> John Levine, [email-masked] <mailto:[email-masked]>, Taughannock Networks, Trumansburg NY
>> Please consider the environment before reading this e-mail. https://jl.ly-- <https://jl.ly--/> 
>> Xml-sg-cmt mailing list
>> [email-masked] <mailto:[email-masked]>

---

From: Alice Russo <[email-masked]>
Date: Mon, 12 Jul 2021 12:47:59 -0700

CMT, Carsten,

Here are a few questions that Carsten raised on the 9090 thread re: https://www.rfc-editor.org/materials/sourcecode-types.txt

> This still has “cbor”, which now no longer can occur.

CMT said to keep it; alternatively, it can be removed.

> Is there a way to deprecate an entry?

It can be removed from the 'preferred' list. (As far as I know, there is no 'deprecated' list.)

> Shouldn’t there be explanations for each of the entries?

IIRC, there was a decision not to do that in the past, perhaps to avoid being prescriptive. However, seems it would be helpful to the RPC (and presumably authors) for the list to include 'intended use'. Currently, precedent is the only indication.

e.g.
xdr    
intended use: XDR code (e.g., "source code that might be contained in a .x file and fed to rpcgen") per https://mailarchive.ietf.org/arch/msg/nfsv4/nlLTz-HurpBLqW4M7WfhJ81IWXs/

Thanks,
Alice

> On Jul 12, 2021, at 9:58 AM, Alice Russo <[email-masked]> wrote:
> 
> Hi Carsten,
>

---

From: "John R Levine" <[email-masked]>
Date: 12 Jul 2021 16:05:02 -0400

> Should “xpath” be added to the list of types <https://www.rfc-editor.org/materials/sourcecode-types.txt> or should these snippets be marked as something else?

xpath is part of XSLT, but it's also used outside XSLT so it's OK with me 
to make it a sourcecode type.

Regards,

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 12 Jul 2021 22:07:39 +0200

> Should “xpath” be added to the list of types <https://www.rfc-editor.org/materials/sourcecode-types.txt> or should these snippets be marked as something else?  

Xpath is a rather common format in the XML world and lends itself to extraction very nicely.
(We don’t use XML that much any more as the subject of RFCs, so there aren’t that many examples.)

Xpath expressions can be very short, so they might be used inline in a paragraph. E.g. (rfc8794.xml):

            <t pn="section-11.1.15.1-1">Within an EBML Schema, the XPath of the <tt>@type</tt> attribute is
<tt>/EBMLSchema/element/extension/@type</tt>.</t>

This example shows a limitation of the vocabulary that we use for inline codespans (<tt>) — we cannot identify the type of such codespans.

But if they are used in display (block) mode, <sourcecode type=“xpath” …> sounds right.

Grüße, Carsten

---

From: Sandy Ginoza <[email-masked]>
Date: Mon, 12 Jul 2021 15:02:20 -0700

John and Carsten,

Thanks for your replies - xpath has been added to the list.

Thanks,
Sandy 

> On Jul 12, 2021, at 1:07 PM, Carsten Bormann <[email-masked]> wrote:
>