Subject: [apps-discuss] Fwd: FW: New Version Notification for draft-kerwin-file-scheme-13.txt

From: Matthew Kerwin <[email-masked]>
Date: Fri, 26 Sep 2014 11:13:22 +1000

FYI: this is my file URI draft, now listing apps-discuss as the official
place for discussion.

A new version of I-D, draft-kerwin-file-scheme-13.txt
has been successfully submitted by Matthew Kerwin and posted to the
IETF repository.

Name:           draft-kerwin-file-scheme
Revision:       13
Title:          The file URI Scheme
Document date:  2014-09-26
Group:          Individual Submission
Pages:          16
URL:
http://www.ietf.org/internet-drafts/draft-kerwin-file-scheme-13.txt
Status:         https://datatracker.ietf.org/doc/draft-kerwin-file-scheme/
Htmlized:       http://tools.ietf.org/html/draft-kerwin-file-scheme-13
Diff:           http://www.ietf.org/rfcdiff?url2=draft-kerwin-file-scheme-13

Abstract:
   This document specifies the "file" Uniform Resource Identifier (URI)
   scheme, replacing the definition in RFC 1738.

   It attempts to document current practices, while at the same time
   defining a common core which is intended to interoperate across the
   broad spectrum of existing implementations.

Note to Readers (To be removed by the RFC Editor)

   This draft should be discussed on the IETF Applications Area Working
   Group discussion list <[email-masked]>.

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat

-- 
  Matthew Kerwin
  http://matthew.kerwin.net.au/

---

From: Daniel Stenberg <[email-masked]>
Date: Fri, 26 Sep 2014 09:19:14 +0200 (CEST)

> http://www.ietf.org/internet-drafts/draft-kerwin-file-scheme-13.txt

Hey,

I too wouldn't mind seeing a refreshed and more accurate file: URI document. 
Around 9% of curl users say they use file:, which probably scales up to quite 
a large amount of people. I'm not convinced it is worth the trouble and the 
arguments to pull this draft into a real RFC.

I have some problems with this draft as it currently stands.

I would rather have a new spec straigten up and tighten the language somewhat 
so that we can get a stricter interpretation of how a file:// is supposed to 
work. Sure, there will then possibly be a few non-compliant parsers out there 
that accept a few non-standard formats but it should be clear to those who 
generates or writes file:// URIs which the correct way is. That way we could 
potentially move into a future with less file:// URI variations rather than 
more. The rest of my comments are based on this basic stand.

This document rather opens up pandora's box for someone like me. Suddenly this 
claims that all sorts of horrid abuses (in my view) we've seen through the 
years are fine and should be supported by my parser (if I want to be 
compliant) and it would surprise me if not a few other parser authors don't 
feel the same.

I also think that the 'host' part of the file: URI was a mistake to begin 
with. It was always a mystery to readers of RFC1738 and I think it should be 
better clarified here as well that it is useless and not interoperable.

The query part. It didn't exist in 1738, so introducing that now will break a 
lot of old parsers (like mine) even if I recognize that 3986 says that it 
should be supported globally. This, for just a vague extension model without 
any clear purpose or use case.

-- 

  / daniel.haxx.se

---

From: "Martin J. Dürst" <[email-masked]>
Date: Fri, 26 Sep 2014 17:09:00 +0900

On 2014/09/26 16:19, Daniel Stenberg wrote:
> I too wouldn't mind seeing a refreshed and more accurate file: URI
> document. Around 9% of curl users say they use file:, which probably
> scales up to quite a large amount of people. I'm not convinced it is
> worth the trouble and the arguments to pull this draft into a real RFC.
>
> I would rather have a new spec straigten up and tighten the language
> somewhat so that we can get a stricter interpretation of how a file://
> is supposed to work. Sure, there will then possibly be a few
> non-compliant parsers out there that accept a few non-standard formats
> but it should be clear to those who generates or writes file:// URIs
> which the correct way is. That way we could potentially move into a
> future with less file:// URI variations rather than more. The rest of my
> comments are based on this basic stand.

I think this would be ideal, but there's really a lot of 'crap' out 
there in the case of the file: scheme. So it may make sense to:
a) Document the core syntax for people creating new URIs, and parsers 
that want to be strict.
b) Document frequently seen (and rather widely accepted) variations to 
help people who want (or have) to implement lenitent parsers. This 
should very clearly be labeled as optional.

> This document rather opens up pandora's box for someone like me.
> Suddenly this claims that all sorts of horrid abuses (in my view) we've
> seen through the years are fine and should be supported by my parser (if
> I want to be compliant) and it would surprise me if not a few other
> parser authors don't feel the same.
>
> I also think that the 'host' part of the file: URI was a mistake to
> begin with. It was always a mystery to readers of RFC1738 and I think it
> should be better clarified here as well that it is useless and not
> interoperable.
>
> The query part. It didn't exist in 1738, so introducing that now will
> break a lot of old parsers (like mine) even if I recognize that 3986
> says that it should be supported globally. This, for just a vague
> extension model without any clear purpose or use case.

It is the first time that I have seen somebody claim that RFC 3986 says 
that all URI schemes have to support query parts. I'd like to know where 
in RFC 3986 you found that. Indeed my understanding is that most schemes 
don't understand or include query parts. The main (but very important) 
exceptions as far as I'm aware of are http:/https: and mailto:.

Regards,   Martin.

---

From: Daniel Stenberg <[email-masked]>
Date: Fri, 26 Sep 2014 10:41:52 +0200 (CEST)

On Fri, 26 Sep 2014, "Martin J. Dürst" wrote:

>> The query part. It didn't exist in 1738, so introducing that now will break 
>> a lot of old parsers (like mine) even if I recognize that 3986 says that it 
>> should be supported globally. This, for just a vague extension model 
>> without any clear purpose or use case.
>
> It is the first time that I have seen somebody claim that RFC 3986 says that 
> all URI schemes have to support query parts. I'd like to know where in RFC 
> 3986 you found that. Indeed my understanding is that most schemes don't 
> understand or include query parts. The main (but very important) exceptions 
> as far as I'm aware of are http:/https: and mailto:.

Maybe this is just me, but that's my reading of RFC 3986 section 3:

    3.  Syntax Components

    The generic URI syntax consists of a hierarchical sequence of
    components referred to as the scheme, authority, path, query, and
    fragment.

There exists language in 3986 suggesting that things differ between protocols 
but reading that section 3 doesn't make it obvious to me that the query part 
can be opted out (even if as I already mentioned by own parsers do). Not even 
3.4 that details query mentions that.

Section 2.2 also lists '?' as a "Reserved character" also in a generic way and 
it is listed as a delimiter but then goes on to rather mysteriously say:

   "URI producing applications should percent-encode data octets that
    correspond to characters in the reserved set unless these characters
    are specifically allowed by the URI scheme to represent data in that
    component."

... but the file: section of RFC1738 doesn't mention what letters that are 
legal in the path part so it's not clear to me if '?' is a delimiter or part 
of the path, spec-wise.

-- 

  / daniel.haxx.se

---

From: Julian Reschke <[email-masked]>
Date: Fri, 26 Sep 2014 12:07:16 +0200

On 2014-09-26 10:41, Daniel Stenberg wrote:
>> It is the first time that I have seen somebody claim that RFC 3986
>> says that all URI schemes have to support query parts. I'd like to
>> know where in RFC 3986 you found that. Indeed my understanding is that
>> most schemes don't understand or include query parts. The main (but
>> very important) exceptions as far as I'm aware of are http:/https: and
>> mailto:.
>
> There exists language in 3986 suggesting that things differ between
> protocols but reading that section 3 doesn't make it obvious to me that
> the query part can be opted out (even if as I already mentioned by own
> parsers do). Not even 3.4 that details query mentions that.
>
> Section 2.2 also lists '?' as a "Reserved character" also in a generic
> way and it is listed as a delimiter but then goes on to rather
> mysteriously say:
>
> ... but the file: section of RFC1738 doesn't mention what letters that
> are legal in the path part so it's not clear to me if '?' is a delimiter
> or part of the path, spec-wise.
 > ...

Yes, RFC 3986 defines a generic syntax. Of course individual schemes can 
profile that (query is not allowed, or query will be ignored). What we 
be bad is to deviate from the generic syntax with respect to delimiters.

Best regards, Julian

---

From: t.petch <[email-masked]>
Date: Fri, 26 Sep 2014 12:36:37 +0100

----- Original Message -----
From: "Martin J. DÃ¼rst" <[email-masked]>
To: "Daniel Stenberg" <[email-masked]>; "Matthew Kerwin"
<[email-masked]>
Cc: "IETF Apps Discuss" <[email-masked]>
Sent: Friday, September 26, 2014 9:09 AM
> should very clearly be labeled as optional.

That approach would motivate me.

I think that getting rough consensus will be hard, since there is so
much out in the wild and there will be those, probably at IETF Last
Call, who will want their variations included or not excluded or who
will want a blow-by-blow history.

If we did no more than Obsolete RFC1738 s3.10, then I would see that as
a valuable step forward.  Perhaps have the core syntax as Normative and
the frequently seen variations as Informative appendices.

Tom Petch

> It is the first time that I have seen somebody claim that RFC 3986
says
> _______________________________________________
> apps-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/apps-discuss

---

From: Sean Leonard <[email-masked]>
Date: Fri, 26 Sep 2014 07:30:25 -0700

> will want a blow-by-blow history.
A significant reason for the variations is that operating 
systems/platforms map network file paths in some way into the file 
system namespace...such as UNC paths.

For the sake of implementation, at a minimum, URIs (even if provisional) 
for network file paths such as SMB / CIFS [draft-crhertel-smb-url-12], 
NFS [RFC2224], and AFP [draft-ietf-svrloc-afp-service, see also 
https://www.iana.org/assignments/uri-schemes/prov/afp and 
http://support.apple.com/kb/ht4538] should advance. They should also be 
referenced in draft-kerwin-file-scheme as the better or "other" way to 
do that sort of thing. Otherwise, I guarantee you that people will 
continue to use file: "in diverse ways" to represent network paths. (Of 
course, it should always be acceptable to use file: to represent mounted 
network paths, since mounted network paths are part of the local file 
system.)

I think a blow-by-blow history should be done, if for no other reason 
than to placate those who want it. If in fact sufficient people want it. 
That depends on what makes it into the file URI spec. The best way 
forward would be a separate I-D that discusses the history, making it 
informational or BCP.

Sean

---

From: John C Klensin <[email-masked]>
Date: Fri, 26 Sep 2014 12:35:33 -0400

FWIW,

This discussion of a series of connected questions such as:

 -- what elements of the RFC 3986 definition are expected
	to be used by all URI types, 
	
 -- whether 3968 specifies semantics for those elements
	or whether what appear to be its semantic definitions
	and restrictions are just examples that can be replaced
	on a per-scheme basis, 
	
 -- which elements can be eliminated by a per-scheme
	profile, and 
	
 -- under what circumstances delimiters that might be
	associated with a particular URI component (or element)
	type can be used for something else if that component is
	not used by that scheme,

dominated the discussions in URNBIS for some months, exploded
briefly onto the IETF list after the now-dead
draft-ietf-urnbis-urns-are-not-uris draft was posted as an early
attempt to deal with the disagreements, and has been the source
of a great many "it means X, no, it means Y and maybe not-X"
exchanges among normally reasonable people.

May I suggest that it would be desirable to avoid repeating that
discussion  here, either for the specific case of "file://" or
for the next URI proposal to come along that does not closely
align with the "http:" UNL model?  It seems to me that a review
of the URNBIS discussion archive for the last nine months (there
was less traffic there than one might have wished for) and
minutes of the Toronto (IETF 90) WG meeting might be in order,
as would be some leadership from WG Chairs and ADs?

As a more or less rhetorical question, if we really can't agree
on what 3986 says and means, is it time to freeze _all_ new and
revised URI definitions and registrations until it is possible
to develop and complete a clarification to that rather basic
document?

thanks,
   john

--On Friday, September 26, 2014 10:41 +0200 Daniel Stenberg
<[email-masked]> wrote:

>>> The query part. It didn't exist in 1738, so introducing that
>>> now will break  a lot of old parsers (like mine) even if I
>>> recognize that 3986 says that it  should be supported
>>> globally. This, for just a vague extension model  without
>> It is the first time that I have seen somebody claim that RFC
>> 3986 says that  all URI schemes have to support query parts.
>> I'd like to know where in RFC  3986 you found that. Indeed my
>> understanding is that most schemes don't  understand or
>> include query parts. The main (but very important) exceptions 
> There exists language in 3986 suggesting that things differ
> between protocols but reading that section 3 doesn't make it
> obvious to me that the query part can be opted out (even if as
> I already mentioned by own parsers do). Not even 3.4 that
> details query mentions that.
> 
> Section 2.2 also lists '?' as a "Reserved character" also in a
> generic way and it is listed as a delimiter but then goes on
> to rather mysteriously say:
> 
> ... but the file: section of RFC1738 doesn't mention what
> letters that are legal in the path part so it's not clear to
> me if '?' is a delimiter or part of the path, spec-wise.

---

From: Matthew Kerwin <[email-masked]>
Date: Mon, 29 Sep 2014 15:36:06 +1000

On 26 September 2014 18:09, "Martin J. Dürst" <[email-masked]>
wrote:

> On 2014/09/26 16:19,
> ​​
> Daniel Stenberg wrote:
> I think this would be ideal, but there's really a lot of 'crap' out there
> in the case of the file: scheme. So it may make sense to:
> a) Document the core syntax for people creating new URIs, and parsers that
> want to be strict.
> help people who want (or have) to implement lenitent parsers. This should
> very clearly be labeled as optional.
>
>
​I think that's where I would have ended up if I'd worked on it in
isolation for a few more years. I always get nervous and back down when I
get introspective and wonder if what I'm proposing is better, or just want
I want to see. I suppose the best way forward is to canonise the common
cases "file:///foo/bar" and "file://smb.example.com/foo/bar"​, and mark
every other variant as an optional extension. Personally I'd like
"file:/foo/bar" to be the standard for local files, because there's less
guff.

> ​
> ​
> ​​
> Daniel Stenberg
> :
>> ​
You're right, and I didn't see that when I was writing it. My twofold goal
was to document what currently happens, and highlight the bits I think are
"ideal." I just didn't go far enough. Although I'm still a bit nervous
about peoples' reactions when they find out their unique interpretations
aren't enshrined as the One True Way.

>> ​
>> ​
>> ​
>> ​
>> ​
​They just didn't think about it enough. ;) From what I've heard, mapping
UNC paths into file URIs is a pretty common case, so it's one we'd do well
to document. It's also one that results in a lot of interop issues (notably
Firefox's fifth slash), so it's worth resolving one way or another. UNC has
gotten more and more prominent in the text as I've iterated, in response to
comments and suggestions. This is also the point where I've deviated the
most from RFC 1738, taking my lead from Microsoft and Chrome, and using the
host as "the samba server" instead of "the only machine that can
dereference this URL."

There's still the fact that this scheme is defined in isolation (without a
protocol attached). The "methods" are necessarily vague, because they
depend on the file system, and where one system might be perfectly happy
supporting a URL with a host (e.g. Windows' CreateFile(PathCreateFromUrl())
stack) another might not. Because there aren't proper methods, this URI
scheme is effectively an interchange format for exchanging filenames (i.e.
a string), and as such I'm reluctant to say "some people can't do this
thing when they interpret this part of this string this way, so no one can
use that part of the string."

I guess we could push to tighten up the methods, but I think that would sap
my enthusiasm somewhat. I already referenced POSIX, that's enough for me
for now.

​
>> ​
>> ​
>> ​
I'm happy drop it. If anyone using OpenVMS is desperate to refer to a
particular version within a file URI, I guess they can use a semicolon.
It's a reserved sub-delimiter, which is perfectly appropriate in this case.

---

From: Sean Leonard <[email-masked]>
Date: Mon, 29 Sep 2014 10:17:16 -0700

In reviewing this draft, the history, and the comments, I noticed that 
there is a lot of text devoted to Windows, but little on POSIX (the 
interoperable standard for Unix), and none on Mac OS X. Mac OS X is 
rooted in POSIX, but it is notable that a lot of Apple Frameworks use 
the file:/// URL format.

In particular, there is zero discussion of this fascinating tidbit from 
the Apple Developer documentation:
Specifying the Path to a File or Directory 
<https://developer.apple.com/library/mac/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/AccessingFilesandDirectories/AccessingFilesandDirectories.html#//apple_ref/doc/uid/TP40010672-CH3-SW6>

    For most URLs, you build the URL by concatenating directory and file
    names together using the appropriate|NSURL|methods until you have
    the path to the item. A URL built in that way is referred to as
    a*path-based URL*because it stores the names needed to traverse the
    directory hierarchy to locate the item. (You also build string-based
    paths by concatenating directory and file-names together, with the
    results stored in a slightly different format than that used by
    the|NSURL|class.) In addition to path-based URLs, you can also
    create a*file reference URL*, which identifies the location of the
    file or directory using a unique ID.

    All of the following entries are valid references to a file
    called|MyFile.txt|in a user’s|Documents|directory:

    *Path-based URL:*|file://localhost/Users/steve/Documents/MyFile.txt|

    *File reference URL:*|file:///.file/id=6571367.2773272/|

    *String-based path:*|/Users/steve/Documents/MyFile.txt|

    Different file systems rely on different separator characters.
    Because of these changes, you should create your URL objects using
    the methods provided by the NSURL class.

    You create URL objects the|NSURL|methods and convert them to file
    reference URLs only when needed. Path-based URLs are easier to
    manipulate, easier to debug, and are generally preferred by classes
    such as|NSFileManager
    <https://developer.apple.com/library/mac/documentation/Cocoa/Reference/Foundation/Classes/NSFileManager_Class/Reference/Reference.html#//apple_ref/occ/cl/NSFileManager>|.
    An advantage of file reference URLs is that they are less fragile
    than path-based URLs while your app is running. If the user moves a
    file in the Finder, any path-based URLs that refer to the file
    immediately become invalid and must be updated to the new path.
    However, as long as the file moved to another location on the same
    disk, its unique ID does not change and any file reference URLs
    remain valid.

Other Refs:
https://developer.apple.com/library/mac/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/
https://developer.apple.com/library/mac/documentation/Cocoa/Conceptual/URLLoadingSystem/

FWIW I don't think documenting legacy Mac OS is useful, so I am not 
advocating for its inclusion. Otherwise you would have to talk about ":" 
specifically as a path separator. But by the same token, I would 
question why we are bothering with documenting legacy MS-DOS and Windows 
(pre Windows 2000/Windows XP), except to the extent that those issues 
survive in contemporary versions of Windows.

-Sean

On 9/25/2014 6:13 PM, Matthew Kerwin wrote:
> FYI: this is my file URI draft, now listing apps-discuss as the 
> official place for discussion.
>
> >>>
>
[SNIP]

---

From: Sean Leonard <[email-masked]>
Date: Mon, 29 Sep 2014 11:35:57 -0700

Colleagues:

We ought to step back for a moment and ask ourselves what exactly the 
goals are of getting a common file URI scheme through the IETF process.

Ultimately we are all here to promote "interoperability". There is a lot 
of software out there that needs to identify local resources in the OS 
file system in the same data type (URI) as Internet resources. The way 
to distinguish these local resources in the URI way is--at a minimum--to 
tack on "file:".

File system resources are normally addressed by the operating system's 
convention. We can come up with all sorts of notations, but at the end 
of the day the OS system call (POSIX open, Win32 CreateFile, etc.) is 
what matters. The OS needs to be consistent with itself and with 
applications on it; it's also important that two instances of the same 
OS (or OS families) agree on things, so that applications and data 
written with that OS (family) in mind can work on different machines.

But from the perspective of Internet architecture, all those things are 
a "local convention"--that magical escape that in IETF-speak means "do 
whatever at the endpoints, it doesn't affect the network". 
draft-kerwin-file-scheme talks about local conventions but the overall 
gist is trying to stuff local conventions into a common format, and 
flailing about by addressing these disparate use cases (mostly Windows, 
but some OpenVMS) that have cropped up.

It is not surprising, by the way, that Windows causes difficulties while 
Unix/POSIX is less problematic, since the URI convention of "/" for path 
separators and "/" as the root path directly descends from Unix/POSIX 
(and from FTP [RFC 959]).

I would like to suggest a totally different approach:
1. file: identifies local filesystem resources (this includes filesystem 
resources accessible via network paths such as UNC, since the filesystem 
facility of the OS accommodates them).
2. # is the fragment.
3. everything else (the RFC 3986 <hier-part> and <query> productions) is 
a local convention.

The draft can still include all of the current content, but move it into 
informative appendices.

If an application encounters a file URI for a local filesystem resource, 
it should deal with it using the local convention. If an application 
encounters a file URI for some non-local filesystem resource, it should 
*leave it alone*. The end.

In other words: why bother parsing a file URI if you know it's not going 
to refer to a resource that you can make use of? If you're on Ubuntu 
14.04, why do you care about interpreting a file: URI that's meant for 
Windows 7? The only thing this does is invite security problems, because 
attackers will tell you (via the network) to perform operations on local 
resources that they don't have the authority to say anything about.

Another issue is the matter of non-ASCII character encoding, which has 
always been dicey. [POSIX] is very clear that a paths as a sequence of 
*bytes*, not characters. Other than "/", <NUL>, and the relative 
pathnames "." and "..", it's anything-goes. The mapping between bytes 
and characters depends on the locale. In Windows NT, the kernel treats 
pathnames as Unicode (specifically UCS-2/UTF-16); translations are done 
in user-space. In Mac OS X, the HFS Plus file system converts all file 
names to decomposed Unicode, i.e., Normalization Form D [HFSPLUSREF].

It is kind of seen as a moral imperative in the IETF to use UTF-8 (and 
Normalization Form C), possibly because of [BCP18] and [RFC5198]. But 
here, the only thing common is diversity--and each operating system 
(vendor) has its own good reasons for its individual decisions. Since 
file: URIs are always stored *in context*, context (the operating system 
documentation, the LC_ALL environment variable, the declared encoding of 
the text document, the identity of the local machine, etc.) provides all 
the info that you need. Just say that the file: URI (outside of the 
fragment component) encodes with local conventions--which are provided 
by context--and be done with it.

If you are taking things like file: URIs out of context...three suggestions:

1. Translate the file URI into the destination context. For example: if 
you are in an HTML editor and you really want to reference 
<file:///D:/pics/catpic.jpg>, but you are saving the HTML document in a 
workgroup share, change the reference to 
<file://mycomputername/pics/catpic.jpg>, and ensure that D:\pics is 
shared as \\mycomputername\pics with appropriate access. Better yet: 
upload the resource to an Internet-accessible location such as 
<http://i.imgur.com/ziZNc2s.jpg>, which for purposes of Internet 
architecture, provides a more universal context.

2. Dereference the file URI and put whatever content you need into the 
context. For example: in an HTML e-mail, don't use <img 
src="file:///D:/catpic.jpg">; use <img src="cid:catpic93298@myemail"> 
and put the cat pic in the e-mail per [RFC2392] and [RFC2387].

3. Don't do that. It doesn't make sense.

Best regards,

Sean

[RFC959]: https://tools.ietf.org/html/rfc959
[POSIX]: http://pubs.opengroup.org/onlinepubs/9699919799/
[BCP18]: http://tools.ietf.org/html/bcp18
[RFC5198]: http://tools.ietf.org/html/rfc5198
[HFSPLUSREF]: https://developer.apple.com/library/mac/qa/qa1235/_index.html
[RFC2392]: http://tools.ietf.org/html/rfc2392
[RFC2387]: http://tools.ietf.org/html/rfc2387

---

From: Matthew Kerwin <[email-masked]>
Date: Tue, 30 Sep 2014 09:37:05 +1000

On 30 September 2014 03:17, Sean Leonard <[email-masked]> wrote:

> interoperable standard for Unix), and none on Mac OS X. Mac OS X is rooted
> in POSIX, but it is notable that a lot of Apple Frameworks use the
> file:/// URL format.
> ​
> ​
>
>
​There's not much on POSIX because it's not exceptional (i.e. if you start
from a POSIX file path, you don't have to do much to get to a file URI).
Windows is known to be exceptional, so those exceptions got some focus.

> <https://developer.apple.com/library/mac/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/AccessingFilesandDirectories/AccessingFilesandDirectories.html#//apple_ref/doc/uid/TP4001…

​The biggest reason for that omission is that, quite simply, I'd not seen
it. (I figured, right from the start, that people would be ready and
willing to point out all my shortcomings and direct me to their favoured
implementations' documentation and justifications.)

>From what I see, the issues are: 1) the "Path-base URL" uses RFC 1738-style
"localhost", which I've broken (and explicitly called out). And 2) the
"File reference URL" ... er, I ... don't know; it's syntactically
compatible with the generic syntax (barring a reserved sub-delimiter, which
is probably alright), and to my eye it looks like it still works according
to the "translating to/from file paths" idea.

> specifically as a path separator. But by the same token, I would question
> why we are bothering with documenting legacy MS-DOS and Windows (pre
> Windows 2000/Windows XP), except to the extent that those issues survive in
> contemporary versions of Windows.
>
>
I don't have to talk about ":" as a path separator in MacOS 8 for the same
reason I didn't talk about "\" as a path separator in DOS, or the wonderful
[FOO.BAR] thing in OpenVMS -- it's irrelevant. As long as the sequence of
directories that comprise a file path is iterable, all you do to map that
path to a file URI is concatenate the directory names together with "/",
and in reverse you explode them on "/" and "translate to the local system's
representation of file paths." Granted draft 13 doesn't have the "translate
URI to file path" text yet -- there's a TODO in the kramdown source.

Regarding the amount of text dedicated to Windows, all I said was: "they
have drive letters, which you map this way," and: "some people actually map
drive letters this way instead, but that's not good." I had to add a bit
more text to justify the "|" because, as was pointed out to me, that isn't
compatible with the generic syntax. If the "/C|/" pattern moves to an
informational section the number of words it receives will matter even less.

UNC comes from Microsoft, but it's not what I'd call a Windows-specific
tech, and it's definitely not legacy. It's also a legitimate alternative to
file URIs, so it gets special mention.​

​Is there somewhere else I've disproportionately represented legacy
DOS/Windows cases to the exclusion of other systems?

---

From: Matthew Kerwin <[email-masked]>
Date: Tue, 30 Sep 2014 12:21:54 +1000

> goals are of getting a common file URI scheme through the IETF process.
I agree. I can say, from the start, my original reason for taking on the
draft was to address a single use-case: there are a bunch of really cool
"URI access" libraries out there, and people like to use those to access
local files[1]. So, the goal is: a means of specifying file names that is
compatible with the generic syntax of RFC 3986, and can be common across
all the "URI access" libraries and other similar consumers.

[1]: https://bugs.ruby-lang.org/issues/8544

> of software out there that needs to identify local resources in the OS file
> system in the same data type (URI) as Internet resources. The way to
> distinguish these local resources in the URI way is--at a minimum--to tack
> on "file:".
> ​
> ​
>
> ​​
> convention. We can come up with all sorts of notations, but at the end of
> the day the OS system call (POSIX open, Win32 CreateFile, etc.) is what
> matters. The OS needs to be consistent with itself and with applications on
> it; it's also important that two instances of the same OS (or OS families)
> agree on things, so that applications and data written with that OS
> (family) in mind can work on different machines.
>
> But from the perspective of Internet architecture, all those things are a
> "local convention"--that magical escape that in IETF-speak means "do
> draft-kerwin-file-scheme talks about local conventions but the overall gist
> is trying to stuff local conventions into a common format, and flailing
> about by addressing these disparate use cases (mostly Windows, but some
> OpenVMS) that have cropped up.
>
>
The apparent "flailing" could just be me not being able to express myself
very well. The path-to-URI conversion is really simple, the only weirdness
being the various conventions for file system roots. Converting back can be
trickier to specify, so it's more tempting to hand-wave there.

But aside from that detail, the interoperability I was aiming for only goes
as far as the syntax. The only operations/methods I explicitly address are
parsing, and converting to and from file paths (once you've done that it's
not a file URI anymore, so of course the local system's syscalls are
appropriate), and the conversions are necessarily system dependent.

> 3. everything else (the RFC 3986 <hier-part> and <query> productions) is a
> local convention.
>
>
1. Yep.

​2. I originally removed the fragment because fragment semantics depend on
the content-type of the addressed resource, and files don't have a
content-type. I kept it off because as far as I'm aware no file system
conventions include fragments (or equivalent semantics), and the as only
operations are converting to/from file system conventions, any fragments
would be immediately lost anyway.

3. Pretty sure this breaks the driving goal, which is: can be parsed by any
generic URI parser. Unless you mean to introduce local URI conventions
that, while compatible with RFC 3986, might not be compatible with other
local URI conventions. That ... offends me in some way I can't well
articulate. <https://www.youtube.com/watch?v=5tkM6Zfr47o>

> *leave it alone*. The end.
Again, this is all what happens after the conversion.​ It doubtless comes
across as a bit weak when I keep saying, "I don't care what you do with it
once you've converted it to a file path", but this whole discussion
reinforces to me the fact that we can't reasonably define any more methods.
They all fall short.

> to refer to a resource that you can make use of? If you're on Ubuntu 14.04,
> why do you care about interpreting a file: URI that's meant for Windows 7?
> The only thing this does is invite security problems, because attackers
> will tell you (via the network) to perform operations on local resources
> that they don't have the authority to say anything about.
>
>
Catch 22: how can you know it's not going to refer to a particular resource
without parsing it? And I could try to tell you, on your Ubuntu 14.01
system, to dump /etc/passwd without resorting to Windows drive letters.

> *bytes*, not characters. Other than "/", <NUL>, and the relative pathnames
> "." and "..", it's anything-goes. The mapping between bytes and characters
> depends on the locale. In Windows NT, the kernel treats pathnames as
> Unicode (specifically UCS-2/UTF-16); translations are done in user-space.
> In Mac OS X, the HFS Plus file system converts all file names to decomposed
> Unicode, i.e., Normalization Form D [HFSPLUSREF].
> ​
> ​
>
>
Yep, I'm on board here (arguments about how accurately OS X maps to NFD
aside), and this is the bit I want most to get right.

> Normalization Form C), possibly because of [BCP18] and [RFC5198]. But here,
> the only thing common is diversity--and each operating system (vendor) has
> its own good reasons for its individual decisions. Since file: URIs are
> always stored *in context*, context (the operating system documentation,
> the LC_ALL environment variable, the declared encoding of the text
> document, the identity of the local machine, etc.) provides all the info
> that you need. Just say that the file: URI (outside of the fragment
> component) encodes with local conventions--which are provided by
> context--and be done with it.
>
>
I think the moral imperative is to provide interoperability, and when all
else fails, Unicode with NFC is a good (or at least convenient)
fallback/default.

Regarding encoding, I took​ my lead from Microsoft and went as far as
defining conversions to IRIs (which are NFC Unicode, and thus unambiguous
and universal, but not necessarily UTF-8), and said that you /can/ go the
next step and convert them to URIs (if you want.) In draft 13 there's the
beginnings of a reason to leave it at IRIs, sketched out as examples.

> workgroup share, change the reference to <file://mycomputername/pics/catpic.jpg>,
> and ensure that D:\pics is shared as \\mycomputername\pics with appropriate
> access. Better yet: upload the resource to an Internet-accessible location
> such as <http://i.imgur.com/ziZNc2s.jpg>, which for purposes of Internet
> src="file:///D:/catpic.jpg">; use <img src="cid:catpic93298@myemail"> and
> put the cat pic in the e-mail per [RFC2392] and [RFC2387].
>
​That's a different use-case: writing one URI that can be consumed
anywhere. All I want is one syntax that can be used to write or parse a URI
anywhere.

I agree that if you *do* want to share a file you either make it accessible
to everyone who will receive the URI (e.g. SMB), or use a different scheme
(either n:1 like ftp/http/etc, or n:n like cid/data/...).

At our university, central IT has set up lots of SMB shares and mapped
drives, etc., so it's very convenient to email links to, e.g.
<file:///i:/section/minutes/some-meeting.doc> because we all share the same
I-drive, and because Outlook lets us interact with those links "properly."
I just want to make sure that the one guy who insists on using Thunderbird
doesn't miss out because Thunderbird has a different parser.

---

From: t.petch <[email-masked]>
Date: Tue, 30 Sep 2014 10:46:07 +0100

----- Original Message -----
From: "Sean Leonard" <[email-masked]>
To: <[email-masked]>
Sent: Monday, September 29, 2014 7:35 PM

> to distinguish these local resources in the URI way is--at a
minimum--to

Well, yes and no.  My working life revolves around Windows (as it does
for most of the world:-) so it is the Windows flavour of file: that
concerns me, and I don't need an RFC for that.  But in the IETF, we
periodically stumble across the status of RFC1738 and so we do need an
RFC to obsolete s3.10 of that, and that motivates me.  Recall that it
says
"This scheme, unlike most other URL schemes,
   does not designate a resource that is universally accessible over the
   Internet."
so that is where our standards currently stand; any movement forward
from that would be welcome to me.  And, as I said before, I think that
there will not be much that we can achieve rough consensus on when it
comes to IETF Last Call.

Tom Petch

> [HFSPLUSREF]:
https://developer.apple.com/library/mac/qa/qa1235/_index.html