Subject: [DNSOP] hostnames vs domain names vs RFC1034/1035 vs RFC2818 vs Wikipedia etc

From: "Adrien de Croy" <[email-masked]>
Date: Fri, 08 Apr 2016 01:37:30 +0000

Hi all

I guess you're all aware of the issue of what constitutes a valid domain 
name, what characters are valid in labels etc.  So forgive me for what 
must be me re-raising an ancient maybe long-thought-put-to-rest issue...

but there's a serious problem out there.

RFC1034 secion 3.5 which is almost copied in RFC1035 section 2.3.1, both 
labelled "preferred name syntax" clearly define

<domain> ::= <subdomain> | " " <subdomain> ::= <label> | <subdomain> "." 
<label> <label> ::= <letter> [ [ <ldh-str> ] <let-dig> ] <ldh-str> ::= 
<let-dig-hyp> | <let-dig-hyp> <ldh-str> <let-dig-hyp> ::= <let-dig> | 
"-" <let-dig> ::= <letter> | <digit> <letter> ::= any one of the 52 
alphabetic characters A through Z in upper case and a through z in lower 
case <digit> ::= any one of the ten digits 0 through 9 Note that while 
upper and lower case letters are allowed in domain names, no 
significance is attached to the case. That is, two names with the same 
spelling but different case are to be treated as if identical. The 
labels must follow the rules for ARPANET host names. They must start 
with a letter, end with a letter or digit, and have as interior 
characters only letters, digits, and hyphen. There are also some 
restrictions on the length. Labels must be 63 characters or less.
which allows DNS labels (not just host names) to contain alphanumeric 
and hyphen only.  There doesn't seem to be a MUST level requirement to 
use this, but there doesn't seem to be any specification elsewhere in 
the documents either.

RFC2818 on the other hand says

The DNS itself places only one restriction on the particular labels that 
can be used to identify resource records. That one restriction relates 
to the length of the label and the full name. The length of any one 
label is limited to between 1 and 63 octets. A full domain name is 
limited to 255 octets (including the separators). The zero length full 
name is defined as representing the root of the DNS tree, and is 
typically written and displayed as ".". Those restrictions aside, any 
binary string whatever can be used as the label of any resource record. 
Similarly, any binary string can serve as the value of any record that 
includes a domain name as some or all of its value (SOA, NS, MX, PTR, 
CNAME, and any others that may be added).

So how did we get from alphanumeric+hyphen to "any binary"?

If we truly allow "any binary" why the need for special ascii-compatible 
encodings for IDN?

Later RFCs (the ones I checked) seem to corroborate RFC2818, but I'm 
pretty sure the last time I tried to register a domain I couldn't enter 
any special chars.  So there's a (probably mixed) de facto standard in 
use anyway.

Plus the countless pages on various answer sites about "what is a valid 
DNS name" which state alphanumeric+hyphen, and seem to gloss over the 
underscore used for SRV records.

Is this just a mess that it's been decided we can't really adequately 
fix?

Thanks

Adrien

---

From: Ted Lemon <[email-masked]>
Date: Thu, 7 Apr 2016 23:03:17 -0300

No, you're confusing hostnames and domain names.   Read Ed Lewis' draft,
https://tools.ietf.org/html/draft-lewis-domain-names-02

On Thu, Apr 7, 2016 at 10:37 PM, Adrien de Croy <[email-masked]> wrote:

> name, what characters are valid in labels etc.  So forgive me for what must
> be me re-raising an ancient maybe long-thought-put-to-rest issue...
>
> <domain> ::= <subdomain> | " "
>
> <subdomain> ::= <label> | <subdomain> "." <label>
>
> <label> ::= <letter> [ [ <ldh-str> ] <let-dig> ]
>
> <ldh-str> ::= <let-dig-hyp> | <let-dig-hyp> <ldh-str>
>
> <let-dig-hyp> ::= <let-dig> | "-"
>
> <let-dig> ::= <letter> | <digit>
>
> <letter> ::= any one of the 52 alphabetic characters A through Z in
> upper case and a through z in lower case
>
> <digit> ::= any one of the ten digits 0 through 9
>
> Note that while upper and lower case letters are allowed in domain
> names, no significance is attached to the case.  That is, two names with
> the same spelling but different case are to be treated as if identical.
>
> The labels must follow the rules for ARPANET host names.  They must
> start with a letter, end with a letter or digit, and have as interior
> which allows DNS labels (not just host names) to contain alphanumeric and
> hyphen only.  There doesn't seem to be a MUST level requirement to use
> this, but there doesn't seem to be any specification elsewhere in the
> documents either.
>
>
>    The DNS itself places only one restriction on the particular labels
>    that can be used to identify resource records.  That one restriction
>    relates to the length of the label and the full name.  The length of
>    any one label is limited to between 1 and 63 octets.  A full domain
>    name is limited to 255 octets (including the separators).  The zero
>    length full name is defined as representing the root of the DNS tree,
>    and is typically written and displayed as ".".  Those restrictions
>    aside, any binary string whatever can be used as the label of any
>    resource record.  Similarly, any binary string can serve as the value
>    of any record that includes a domain name as some or all of its value
>    (SOA, NS, MX, PTR, CNAME, and any others that may be added).
>
>
>
> pretty sure the last time I tried to register a domain I couldn't enter any
> special chars.  So there's a (probably mixed) de facto standard in use
> anyway.
>
> Is this just a mess that it's been decided we can't really adequately fix?
>
> Thanks
>
> Adrien
>
> _______________________________________________
> DNSOP mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dnsop
>
>

---

From: "Adrien de Croy" <[email-masked]>
Date: Fri, 08 Apr 2016 02:15:36 +0000

------ Original Message ------
From: "Ted Lemon" <[email-masked]>
To: "Adrien de Croy" <[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Sent: 8/04/2016 2:03:17 p.m.
Subject: Re: [DNSOP] hostnames vs domain names vs RFC1034/1035 vs 
RFC2818 vs Wikipedia etc

>No, you're confusing hostnames and domain names.   Read Ed Lewis' 
>draft, https://tools.ietf.org/html/draft-lewis-domain-names-02
Yeah I was trying to avoid confusing them.  I did read that draft which 
is how I ended up at this point.

Section 3.5 of 1034 does not limit this definition to hostnames by my 
reading of it.

Paragraph 1 alludes to the existence of some specification of domain 
names and says that they are "as general as possible".  It also says we 
should be prudent when assigning names to objects.  I read a lot of the 
prose of the document and there's very little could be construed as a 
formal specification for domain names.

Paragraph 2 follows on from the concept of assigning names to objects 
and discusses additional restrictions for mail domains, and also host 
names.

Paragraph 3 starts off the definition.  It doesn't say that it's a 
definition for any particular type of name (mail domain or host name), 
it just helpfully suggests that "The following syntax will result in 
fewer problems with many applications...."

etc.

So, someone reading only this specification could be excused for 
concluding (as many seem to have done) that the BNF isn't just for 
hostnames.  You'd have to make some leaps and assumptions to conclude 
that.

understood the thing was written in 1987.

So is the encoding scheme for IDN due to the fact that everyone else got 
this wrong, or is it only for encoding internationalised hostnames?

Has ICANN issued any domains that use anything other than ASCII 
alphanumeric + hypen?

Adrien

>>I guess you're all aware of the issue of what constitutes a valid 
>>domain name, what characters are valid in labels etc.  So forgive me 
>>for what must be me re-raising an ancient maybe 
>>long-thought-put-to-rest issue...
>>
>>RFC1034 secion 3.5 which is almost copied in RFC1035 section 2.3.1, 
>>both labelled "preferred name syntax" clearly define
>>
>>
>><domain> ::= <subdomain> | " " <subdomain> ::= <label> | <subdomain> 
>>"." <label> <label> ::= <letter> [ [ <ldh-str> ] <let-dig> ] <ldh-str> 
>>::= <let-dig-hyp> | <let-dig-hyp> <ldh-str> <let-dig-hyp> ::= 
>><let-dig> | "-" <let-dig> ::= <letter> | <digit> <letter> ::= any one 
>>of the 52 alphabetic characters A through Z in upper case and a 
>>through z in lower case <digit> ::= any one of the ten digits 0 
>>through 9 Note that while upper and lower case letters are allowed in 
>>domain names, no significance is attached to the case. That is, two 
>>names with the same spelling but different case are to be treated as 
>>if identical. The labels must follow the rules for ARPANET host names. 
>>They must start with a letter, end with a letter or digit, and have as 
>>interior characters only letters, digits, and hyphen. There are also 
>>some restrictions on the length. Labels must be 63 characters or less.
>>If we truly allow "any binary" why the need for special 
>>ascii-compatible encodings for IDN?
>>
>>pretty sure the last time I tried to register a domain I couldn't 
>>enter any special chars.  So there's a (probably mixed) de facto 
>>standard in use anyway.
>>
>>Plus the countless pages on various answer sites about "what is a 
>>valid DNS name" which state alphanumeric+hyphen, and seem to gloss 
>>over the underscore used for SRV records.
>>

---

From: "Adrien de Croy" <[email-masked]>
Date: Fri, 08 Apr 2016 02:21:20 +0000

------ Original Message ------

>>through 9
if this was a BNF production only for hostnames, why call it <domain>, 
<label> etc.

There's no other BNF for domain name in the spec.

Adrien

---

From: Ted Lemon <[email-masked]>
Date: Thu, 7 Apr 2016 23:24:33 -0300

Have you read the rest of the documents?   E.g.,:

https://tools.ietf.org/html/rfc2181#section-11

---

From: "Adrien de Croy" <[email-masked]>
Date: Fri, 08 Apr 2016 02:29:28 +0000

yeah, paragraph 2 of that section is what I quoted as at odds with 
1034/1035.

I've been reading this stuff all day....

Looks like 1034/1035 should be obsoleted.

Thankfully not many people nowadays need to write DNS resolvers.

Adrien

------ Original Message ------
From: "Ted Lemon" <[email-masked]>
To: "Adrien de Croy" <[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Sent: 8/04/2016 2:24:33 p.m.
Subject: Re: [DNSOP] hostnames vs domain names vs RFC1034/1035 vs 
RFC2818 vs Wikipedia etc

>>>>"." <label> <label> ::= <letter> [ [ <ldh-str> ] <let-dig> ] 
>>>><ldh-str> ::= <let-dig-hyp> | <let-dig-hyp> <ldh-str> <let-dig-hyp> 
>>>>::= <let-dig> | "-" <let-dig> ::= <letter> | <digit> <letter> ::= 
>>>>any one of the 52 alphabetic characters A through Z in upper case 
>>>>and a through z in lower case <digit> ::= any one of the ten digits 
>>>>0 through 9

---

From: Ted Lemon <[email-masked]>
Date: Thu, 7 Apr 2016 23:31:13 -0300

The document I mentioned updates RFC 1034.   That's how we do things in the
IETF!

On Thu, Apr 7, 2016 at 11:29 PM, Adrien de Croy <[email-masked]> wrote:

> Subject: Re: [DNSOP] hostnames vs domain names vs RFC1034/1035 vs RFC2818
> vs Wikipedia etc
>
>

---

From: "Adrien de Croy" <[email-masked]>
Date: Fri, 08 Apr 2016 02:33:47 +0000

I've seen a lot of RFCs marked as obsoleted.

When you see updated by about 20 more RFCs, and you start to click 
through them, and see mostly they relate to DNSSEC etc, it can be a bit 
of an impediment.

maybe an errata would be more visible for this kind of issue.

thanks for your patience!

------ Original Message ------
From: "Ted Lemon" <[email-masked]>
To: "Adrien de Croy" <[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Sent: 8/04/2016 2:31:13 p.m.
Subject: Re: [DNSOP] hostnames vs domain names vs RFC1034/1035 vs 
RFC2818 vs Wikipedia etc

>The document I mentioned updates RFC 1034.   That's how we do things in 
>the IETF!
>
>>>>>><domain> ::= <subdomain> | " " <subdomain> ::= <label> | 
>>>>>><subdomain> "." <label> <label> ::= <letter> [ [ <ldh-str> ] 
>>>>>><let-dig> ] <ldh-str> ::= <let-dig-hyp> | <let-dig-hyp> <ldh-str> 
>>>>>><let-dig-hyp> ::= <let-dig> | "-" <let-dig> ::= <letter> | <digit> 
>>>>>><letter> ::= any one of the 52 alphabetic characters A through Z 
>>>>>>in upper case and a through z in lower case <digit> ::= any one of 
>>>>>>the ten digits 0 through 9
>>>>if this was a BNF production only for hostnames, why call it 
>>>><domain>, <label> etc.
>>>>

---

From: Ted Lemon <[email-masked]>
Date: Thu, 7 Apr 2016 23:35:40 -0300

No worries.   FYI, errata don't actually update documents, so they're not
the right solution.   The thing about 1034/1035 is that they are very
readable, and a good place to start.   Think of it as being like Talmudic
commentary (or TIbetan Buddhist, for that matter)--you just have to read
all the layers to get the full story.

On Thu, Apr 7, 2016 at 11:33 PM, Adrien de Croy <[email-masked]> wrote:

> When you see updated by about 20 more RFCs, and you start to click through
> them, and see mostly they relate to DNSSEC etc, it can be a bit of an
> impediment.
>

---

From: Mark Andrews <[email-masked]>
Date: Fri, 08 Apr 2016 12:33:40 +1000

IDN is because there isn't a character set for the non-ascii domain names
and because some hostname lookup libraries validate their inputs.

rfc952 as modified by rfc1123 define hostnames.  hostnames are a subset
of domain names.  Both fold the case of ascii letters.

The DNS is supposed to be case preserving.  Most servers don't do
this though modern versions of named do.  This exposes bugs in DNS
clients which try to check the owner names of returned records in
a case sensitive manner.

Mark
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: [email-masked]

---

From: "Adrien de Croy" <[email-masked]>
Date: Fri, 08 Apr 2016 02:49:35 +0000

Whenever I see a "errata exist" I click through.... anyway.

I guess if we consider that the BNF production in RFC1034/1035 is for 
hostname not domain name we run into the next problem which is that if 
we ever have a domain name that doesn't also comply with the rules for 
hostname, we can't include any hostname records in it, since the 
production for <label> is reused for all the labels in the name down to 
the root.

That kinda puts a damper on it.

But it's good to see a clear statement from 1987 about desirability of 
supporting alternate protocols (although they use CLASS for that).  
Maybe onion should have used a new CLASS :)

I'm also struggling to see where the restriction on using ASCII is 
coming from, especially since 2818 s. 11 states binary, not ASCII, and 
so IDN shouldn't need to exist except for coding hostnames.

Is a hostname library one that is used purely to resolve addresses (e.g. 
A and AAAA records)?

Adrien

------ Original Message ------
From: "Ted Lemon" <[email-masked]>
To: "Adrien de Croy" <[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Sent: 8/04/2016 2:35:40 p.m.
Subject: Re: [DNSOP] hostnames vs domain names vs RFC1034/1035 vs 
RFC2818 vs Wikipedia etc

>No worries.   FYI, errata don't actually update documents, so they're 
>not the right solution.   The thing about 1034/1035 is that they are 
>very readable, and a good place to start.   Think of it as being like 
>Talmudic commentary (or TIbetan Buddhist, for that matter)--you just 
>have to read all the layers to get the full story.
>
>>through them, and see mostly they relate to DNSSEC etc, it can be a 
>>bit of an impediment.
>>
>>>The document I mentioned updates RFC 1034.   That's how we do things 
>>>in the IETF!
>>>
>>>>>>>><let-dig> ] <ldh-str> ::= <let-dig-hyp> | <let-dig-hyp> 
>>>>>>>><ldh-str> <let-dig-hyp> ::= <let-dig> | "-" <let-dig> ::= 
>>>>>>>><letter> | <digit> <letter> ::= any one of the 52 alphabetic 
>>>>>>>>characters A through Z in upper case and a through z in lower 
>>>>>>>>case <digit> ::= any one of the ten digits 0 through 9

---

From: Mark Andrews <[email-masked]>
Date: Fri, 08 Apr 2016 12:48:44 +1000

Additionally the rules have not changed.  "preferred syntax" means
just that "preferred syntax".  It is not "required syntax".  It was
guidance for what is likely to be usable.  Underscore also turned
out to be usable especially as a way to differentiate hostnames
from other types of names that are stored in the DNS.

UTF8 strings can be stored in the DNS.  The case of a-z and A-Z is
not always preserved.  Lookups of using UTF8 strings are insensitive
to a-z and A-Z.  The presentation format of UTF8 strings in master
files is all ascii using DNS escaping conventions.

Similar things can be said about other 8 bit character sets when used
in the DNS,

---

From: Suzanne Woolf <[email-masked]>
Date: Fri, 8 Apr 2016 07:10:33 -0400

> On Apr 7, 2016, at 10:49 PM, Adrien de Croy <[email-masked]> wrote:
>  
> But it's good to see a clear statement from 1987 about desirability of supporting alternate protocols (although they use CLASS for that).  Maybe onion should have used a new CLASS :)
>  

See draft-sullivan-dns-class-useless (which will be discussed in the WG meeting in a few hours) for some analysis of why that’s impractical.

Suzanne