Subject: Updated I18N proposal

From: Nicolas Williams <[email-masked]>
Date: Wed, 10 May 2006 13:46:53 -0500

Offline Jeff Altman argues that the problem with my proposal is
GSS_C_CES_*.

So I offer, instead, this:

 - In the GSS-APIv3 deprecate (note that deprecate != remove):

    - GSS_Import_name() for any name type other than GSS_C_NT_EXPORTED_NAME
    - GSS_Display_name()
    - GSS_Display_status()

 - In the GSS-APIv3 add versions of the above functions that specify the
   use of Unicode in the abstract API, UTF-8 in the C bidnings, etc...

    - GSS_Import_name_unicode() (gss_import_name_utf8() in C bindings)
    - GSS_Display_name_unicode() (gss_display_name_utf8() in C bindings)
    - GSS_Display_status_unicode() (gss_display_status_utf8() in C bindings)

 - In the GSS-APIv3 add *optional* _utility_ function versions of the
   above that utilize the CES of the current locale:

    - GSS_Import_name_locale() (gss_import_name_locale())
    - GSS_Display_name_locale() (gss_display_name_locale())
    - GSS_Display_status_locale() (gss_display_status_locale())

   Suitable warnings about the limited applicability of these functions
   and the limited portability of applications that rely on these
   interfaces will be included in the specs.

 - In the GSS-APIv2u2 add a note about implementation history of the
   Latin-1 requirements of the GSS-APIv2u1, but otherwise change
   nothing.

   Though we may want to add the above to the GSS-APIv2u2 as well; I see
   no reason not to.

I believe this should make Jeff Altman happy (no GSS_C_CES_* symbols,
values, no registry or reuse of existing registries), as well as Martin.
It will certainly make me happy, and probably Jeff Hutzelman as well.

Cheers,

Nico
-- 

_______________________________________________
Kitten mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/kitten

---

From: Michael B Allen <[email-masked]>
Date: Wed, 10 May 2006 16:33:10 -0400

On Wed, 10 May 2006 13:46:53 -0500

Why not do:

>     - GSS_Import_name_latin1() (gss_import_name_latin1())
>     - GSS_Display_name_latin1() (gss_display_name_latin1())
>     - GSS_Display_status_latin1() (gss_display_status_latin1())

and leave the original functions alone but simply remove the "latin1"
requirement? That's basically how most implementations work now so you're
just aligning the spec with reality. Yes, I know for C implementations
"just-use-8" behavior will not be to spec but it wasn't before. Then
implementations can slowly add the necessary conversions fix things as
bugs pop up in existing code.

Mike

---

From: Nicolas Williams <[email-masked]>
Date: Wed, 10 May 2006 15:39:27 -0500

> >     - GSS_Display_status_latin1() (gss_display_status_latin1())

I wouldn't oppose _latin1() versions of these functions as long as
they're optional, and if they were likely to be useful, but methinks
they'd not be useful.

> bugs pop up in existing code.

The originals have been implemented both ways: just-use-locale's-codeset
and Latin-1.  Leaving them alone is not an option since some
implementations would be left with no standards compliant way to do what
they do without also modifying applications to do what the _locale()
utility functions would do anyways.

Nico
--

---

From: Martin Rex <[email-masked]>
Date: Wed, 10 May 2006 23:07:53 +0200 (MET DST)

Nico,

I like your updated proposal.

>     - GSS_Import_name() for any name type other than GSS_C_NT_EXPORTED_NAME

 + retain the current behaviour of existing implementations,
   but check out and non-ambiguously document what it **really** does!

>    use of Unicode in the abstract API, UTF-8 in the C bindings, etc...
> 

I appreciate that.  :)

>     - GSS_Display_status_locale() (gss_display_status_locale())

I know little about Java.
Is there something like "locale" on Java -- I mean at the GSS-API level?
There may be something when calling to the native code interface,
but I'm really wondering whether this applies to the Java GSS-API bindings.

-Martin

---

From: Michael B Allen <[email-masked]>
Date: Wed, 10 May 2006 17:08:50 -0400

On Wed, 10 May 2006 15:39:27 -0500

> they're optional, and if they were likely to be useful, but methinks

You're right. I don't know what I was thinking.

> utility functions would do anyways.

But implementations of these functions never followed the spec in the
first place.

What about all the existing code that uses gss_display_name? As
mechs start to use extended charsets more and more the old code
will start to break. It's a lot easier to fix libgssapi as opposed
to running around adding "_locale" to all your code. In fact, I'll
make a bold prediction - implementors will make gss_import_name and
gss_import_name_locale behave exactly the same.

Mike

---

From: Nicolas Williams <[email-masked]>
Date: Wed, 10 May 2006 16:19:04 -0500

> I like your updated proposal.

Excellent.  We're close to consensus then.

Expect an I-D soon, and a respin of my expired I-Ds.

>  + retain the current behaviour of existing implementations,

Oh, I expect that existing implementations will do what they do.

>    but check out and non-ambiguously document what it **really** does!

In the IETF document?  I suppose we could, though maybe not
exhaustively.

> but I'm really wondering whether this applies to the Java GSS-API bindings.

I was hoping to leave the Java bindings to Java experts, but I can
research this and get back to you.  The point of saying "unicode" in the
abstract API is to leave the actual details to the bindings -- some
might use UTF-16, etc...  (some languages might not support anything
other than, say, US-ASCII, whatever).

Note also that I don't want to require Unicode 4.x, or 3.x or any
particular version, planes, etcetera in the abstract API.  Why?  Because
some language bindings that may be limited to UCS-2 may not be able to
comply.

Nico
--

---

From: Jeffrey Altman <[email-masked]>
Date: Wed, 10 May 2006 17:20:00 -0400

> but I'm really wondering whether this applies to the Java GSS-API bindings.

I do not believe these will be implemented in the Java Bindings.
Java uses UCS2 Unicode character representation.  When calling out
to a native GSS-API mechanism it will want to use the gss_xxx_unicode()
versions.  Same for C#.

Jeffrey Altman

---

From: Nicolas Williams <[email-masked]>
Date: Wed, 10 May 2006 16:20:19 -0500

> gss_import_name_locale behave exactly the same.

Existing implementations can't be required to be made to comply with the
original spec.  Running code will keep running.  Code that doesn't run
will continue to not run.  End of story :)

---

From: Michael B Allen <[email-masked]>
Date: Wed, 10 May 2006 18:12:14 -0400

On Wed, 10 May 2006 16:20:19 -0500

> will continue to not run.  End of story :)

If gss_display_name is supposed to maintain just-use-8 behavior and
someone now starts using a Cyrillic character in their username what is
the implementation supposed to do?

---

From: Nicolas Williams <[email-masked]>
Date: Wed, 10 May 2006 17:28:57 -0500

> If gss_display_name is supposed to maintain just-use-8 behavior and

Whatever was implemented.  Martin has an implementation that does what
RFC2743 said to do.

> the implementation supposed to do?

Whatever it does now.

---

From: Martin Rex <[email-masked]>
Date: Thu, 11 May 2006 00:50:23 +0200 (MET DST)

> the implementation supposed to do?

When the term "just-use-8" was being used to describe the "encoding"
of Kerberos principals in rfc1510 implementations, it meant that
integers in the range 0 to 127 were considered to be 7-Bit ASCII
characters while integers in the range -128 to -1 (or 128 to 255)
were treated as mere integers.  There is no such thing as a
cyrillic character for a "just-use-8" gssapi mechanism implementation.

*Applications* may interpret certain integers outside of 7-bit US-ASCII
as cyrillic characters (or more precisely, users might perceive those
integers as as cyrillic characters based on the font that is used
for visualization and/or the keyboard driver in use).

-Martin

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Thu, 11 May 2006 09:36:45 -0400

> to a native GSS-API mechanism it will want to use the gss_xxx_unicode()

In fact, now that I think about it, we need to be a little careful here.
The GSS-API has generally been defined as an abstract API plus language 
bindings for one or more languages.  There are protocol specifications 
written in terms of the abstract API, and I expect there will be more in 
the future.  We _don't_ want to get into a situation where protocols are 
specifying the use of GSS_Import_name_unicode() or GSS_Import_name_locale() 
explicitly -- that choice should be up to the needs of the application 
implementor.

I think that at the abstract API level, we get to hand-wave away the 
charset issue, as long as we make sure it is dealt with appropriately in 
the concrete language bindings.  So, Nico's proposal applies to the C 
language bindings, but not to C# or Java, and not to the abstract API.  One 
thing we do need to do is _remove_ the Latin-1 requirement from the 
abstract API, where it doesn't belong, and push it down to the C-language 
bindings.

There is still the question of when the *_locale versions must be 
implemented, and what charsets are required under what conditions.  My 
previous proposal in this area is still relevant, and I have seen neither 
comments on that nor alternate proposals.

-- Jeff

---

From: Martin Rex <[email-masked]>
Date: Thu, 11 May 2006 16:15:21 +0200 (MET DST)

Jeffrey Hutzelman wrote:
> language bindings, but not to C# or Java, and not to the abstract API.

I also think we need not and should not touch the Astract / high-level
API specification, only the language bindings -- here the C-Bindings
(rfc-2744)

>
> One thing we do need to do is _remove_ the Latin-1 requirement from the 

There is no reference to Latin-1 at the abstract API today,
and I believe there never was.

-Martin

---

From: Nicolas Williams <[email-masked]>
Date: Thu, 11 May 2006 09:53:47 -0500

> abstract API, where it doesn't belong, and push it down to the C-language 

I agree!  The abstract API spec should say something about I18N though,
something that amounts to saying that language bindings authors need to
take care of it, possibly recommending the use of Unicode.

> comments on that nor alternate proposals.

I think they must be present but may return an error if not implemented.

Of course, why not implement them on any platforms that have non-UTF-8
locales?

Nico
--

---

From: Ken Raeburn <[email-masked]>
Date: Thu, 11 May 2006 10:57:31 -0400

>> to a native GSS-API mechanism it will want to use the  

Then the Java/C# bindings probably need to have a defined mechanism  
for handling received names outside the BMP.  (Raise an exception?)

> I think that at the abstract API level, we get to hand-wave away  
> the charset issue, as long as we make sure it is dealt with  
> appropriately in the concrete language bindings.  So, Nico's  
> proposal applies to the C language bindings, but not to C# or Java,  
> and not to the abstract API.

I agree.

Ken

---

From: Jeffrey Altman <[email-masked]>
Date: Thu, 11 May 2006 11:17:24 -0400

> and I believe there never was.

The only reference to Latin-1 is in the C Bindings Section 3.2.2
Character Strings.

Jeffrey Altman

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Thu, 11 May 2006 11:21:06 -0400

On Thursday, May 11, 2006 04:15:21 PM +0200 Martin Rex <[email-masked]> 
wrote:

>> abstract API, where it doesn't belong, and push it down to the
>> C-language  bindings.
>

You're right; the only reference to Latin-1 is in the C-language bindings.
Even that text is not very precise; it only says that "certain" data items 
"may be regarded" as Latin-1 strings; aside from a few examples, it doesn't 
say which ones.  The abstract API should be precise about what things are 
strings, and the language bindings should make it clear what the charset is 
for every string parameter.

-- Jeff

---

From: Martin Rex <[email-masked]>
Date: Thu, 11 May 2006 20:55:22 +0200 (MET DST)

> take care of it, possibly recommending the use of Unicode.

Although I do agree it would be good to have that, and I definitely
want to see in when rfc2743 requires real changes (e.g. GSS-API v3),
I think it would be overkill if we opt for updating only the C-Bindings
in a GSS-API v2u2 style.

> I think they must be present but may return an error if not implemented.

I strongly disagree.

Several functions in GSS-API v2 are new (not present in GSS-API v1)
and optional (such as gss_export_sec_context/gss_import_sec_context),
and furthermore the message protection calls were renamed

  gss_seal/gss_unseal -> gss_wrap/gss_unwrap,

so having to provide a function pointer for a function that you
don't want to implement is somewhat alien to the gssapi history.

Applications have to deal with absence of optional functions and
renamed functions anyway -- and I will definitely will not provide
any of the locale functions in gssapi libraries that I create/distribute.

-Martin

---

From: Nicolas Williams <[email-masked]>
Date: Thu, 11 May 2006 13:59:05 -0500

> any of the locale functions in gssapi libraries that I create/distribute.

For v2u2, I agree, for v3 we can require that all of these be present...

Nico
--

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Thu, 11 May 2006 15:10:34 -0400

> For v2u2, I agree, for v3 we can require that all of these be present...

For v2u2, none of these will be present, as we are not empowered to change 
the existing API, only to clarify it.

For v3, I happen to agree with Martin's position that the *_locale 
functions should simply not exist if not implemented.  However, I very 
strongly object to his method of argument, which basically boils down to 
"you better do it my way because if you don't I'm going to ignore the spec 
and do it my way anyway".

-- Jeff

---

From: Nicolas Williams <[email-masked]>
Date: Thu, 11 May 2006 14:13:47 -0500

> the existing API, only to clarify it.

That depends on what the new charter says w.r.t. I18N.  I've not seen
the text.

> and do it my way anyway".

I see no reason why in the v3 baseline we couldn't require all functions
to exist, but I also don't take a very strong position on this
particular matter.

---

From: Martin Rex <[email-masked]>
Date: Thu, 11 May 2006 21:58:51 +0200 (MET DST)

> and do it my way anyway".

Sorry, I didn't mean to offend you personally, just wanted to clarify
that I take the liberty to entirely ignore a detail in a spec that
I consider a poor/inapropriate design choice.  There are a few things
in rfc3280 and in the X.509v3 Standard that I handle this way,
including the "subjectKeyIdentifier" presence, isCA presence in
EndEntity certs and criticality of certain attributes.

In GSS-API v2 there is unnecessary bloat with the OID_set utility
functions (which I did implement just for the same of the fun of it,
but which I would never expect to be provide from a gssapi v2 mechanism,
and the fubar-ed gss-api call gss_add_cred().

I do not mind if some gssapi mechanism implemented the three gssapi
call we were discussing to use the locale (and correct codepage-conversions)
instead of iso-8859-1 as the C-bindings says, because it is at least
slightly ambiguous.  John Wray (the author of the C-Bindings document)
posted a significantly more explicit statement on the use of ISO-8859-1
on the CAT Mailing list -- but that is not part of the official standard,
and one cannot blame implementors for this.

Really, it is much easier if you can check whether some library implements
a functionality with a simple "nm libxyz.so" or "dumpbin /exports xyz32.dll"
than having to write code, compile, run and look at the return code in
order to find out.

-Martin

---

From: Nicolas Williams <[email-masked]>
Date: Thu, 11 May 2006 15:08:15 -0500

> EndEntity certs and criticality of certain attributes.

Er, OK then.

Obviously specs can have mistakes, or parts of them may be much too
difficult to implement.  So we have to take some liberties.  We should
prefer to take as few such liberties as possible.

> and the fubar-ed gss-api call gss_add_cred().

Remind me, off-list, what's wrong with gss_add_cred().

> and one cannot blame implementors for this.

I didn't either, but now that it's clear that you have implementations
which conflict with ours on this I do mind and I do want a better
solution than "ignore what the spec says."

> than having to write code, compile, run and look at the return code in

Ideally there should be pre-processor defines...  But one can do all
this work at run-time (provided one has a modern run-time linker).
Still, it'd be even better if all functions in the baseline API are
guaranteed to be present, even if all some optional ones do is return
errors.

Nico
--

---

From: Nicolas Williams <[email-masked]>
Date: Thu, 11 May 2006 15:31:47 -0500

On Thu, May 11, 2006 at 04:12:42PM -0400, Jeffrey Hutzelman wrote:
> Things that require running code to perform a configure test are evil.
> If you have to run code to do a test, you can't do that test in a 
> cross-compile environment.

I didn't say that compile-time tests are bad...

And, BTW, I don't agree with what you say, witness Java.

Given a sufficiently rich run-time then relying on run-time detection of
features is a good thing.

Nico
--

---

From: Jeffrey Altman <[email-masked]>
Date: Thu, 11 May 2006 16:32:41 -0400

> features is a good thing.

I want my cake and eat it to.  I would like run-time detection
and compile time detection.

Jeffrey Altman

---

From: Nicolas Williams <[email-masked]>
Date: Thu, 11 May 2006 15:44:34 -0500

> and compile time detection.

Exactly.

The only problem is that for the C bindings we can only portably specify
compile-time feature detection, and we can't even specify exactly how to
find the header file.

Yes, C can suck.

Nico
--

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Thu, 11 May 2006 16:56:14 -0400

> and compile time detection.

Right, and my point is that "compile time detection" cannot consist of 
compiling and running a test program that uses the run-time detection to 
tell you if the feature is available, because that doesn't work in 
cross-compile environments.

It turns out that the art of build-time feature detection is fairly well 
understood, at least in the UNIX/C world, and not breaking it is nice.  It 
turns out that tests that check for existence of a type, function, 
preprocessor symbol, etc are fairly easy to do.  Tests that check the value 
of a preprocessor symbol are a bit harder, and tests that require that you 
_call_ a function and check its behavior or return value are very poor.

---

From: Nicolas Williams <[email-masked]>
Date: Thu, 11 May 2006 16:05:44 -0500

> _call_ a function and check its behavior or return value are very poor.

I see this point, but C code that would use functions that may
optionally do nothing and return errors SHOULD always do run-time
detection of this and SHOULD NOT rely on compile-time detection of such
features.

Again, I'm not remotely opposed to have pre-processor defines for
compile-time detection of features.

Nico
--

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Thu, 11 May 2006 17:17:35 -0400

On Thursday, May 11, 2006 04:05:44 PM -0500 Nicolas Williams 
<[email-masked]> wrote:

>> understood, at least in the UNIX/C world, and not breaking it is nice.
>> It  turns out that tests that check for existence of a type, function,
>> preprocessor symbol, etc are fairly easy to do.  Tests that check the
>> value  of a preprocessor symbol are a bit harder, and tests that require
>> that you  _call_ a function and check its behavior or return value are
>> very poor.
>

Ah, but if those functions aren't implemented, then maybe I want to compile 
in additional code that emulates them.  To do that, I _need_ to know the 
answer at compile time.

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Thu, 11 May 2006 17:24:43 -0400

> compile-time detection of features.

Then I don't think we have a problem.

---

From: Nicolas Williams <[email-masked]>
Date: Thu, 11 May 2006 16:25:55 -0500

> in additional code that emulates them.  To do that, I _need_ to know the 

And I will repeat this one last time: I don't oppose having C
prep-processor defines for this.

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Thu, 11 May 2006 16:12:42 -0400

On Thursday, May 11, 2006 03:08:15 PM -0500 Nicolas Williams 
<[email-masked]> wrote:

>> Really, it is much easier if you can check whether some library
>> implements a functionality with a simple "nm libxyz.so" or "dumpbin
>> /exports xyz32.dll" than having to write code, compile, run and look at
>> the return code in order to find out.
>

-- Jeff

---

From: Martin Rex <[email-masked]>
Date: Fri, 12 May 2006 18:37:05 +0200 (MET DST)

> compile-time feature detection, and we can't even specify exactly how to

Not if you do it correctly.  We use binary plug'n'play exclusively,
i.e. runtime dynamic loading of a third party shared library via
dlopen/LoadLibrary on all of our platforms with any of the
15 gssapi mechanism that are certified for interoperability and a few more.

You're out for much worse experiences when you have your application
compiled with C++ compiler A and try to dynamically load at runtime
a shared library from a third party vendor that was compiled with
a different C++ compiler and different class libraries.  I think
the problem occurs because the shared library is incorrectly built
(in combination with the global symbol space of the Unix loader)
but I'm still waiting for the resolution.

We have been using a gssapi_2.h header file based on the appendix
or the C-Bindings document with slight modifications.
We do not use the broken xom.h OID/OID_set definitions, but instead the
fallback definitions compatible to what MIT Kerberos 5 has been
using in 1995/1996 (the difference affects 64-bit platforms),
because we needed to be binary compatible to MIT Kerberos on Digital Unix.

Determining the features of a gssapi mechanism by the functions/symbols
which can be loaded/resolved dynamically at runtime works fine and reliably.
Dynamic runtime loading does not have any problem loading several seperate
shared libraries that export the exact same symbols/functions without
getting confused (about the dynamically loaded functions).
As mentioned, if the shared library has internal symbols not fully
resolved, your're up for unpleasant surprises for all meet-in-the-wild
combinations of apps and shared libs on Unix. 

GSS-API v2 is sufficiently clear that a portable application do
not need to include any product or mechanism specific header files
at compile time in order to be able to use it.  You might need that
for some dirty mech-specific tricks, though.

-Martin

---

From: Nicolas Williams <[email-masked]>
Date: Fri, 12 May 2006 11:44:56 -0500

> 15 gssapi mechanism that are certified for interoperability and a few more.

Clearly.  We have a very nice runtime linker in Solaris.

That's not my complaint though.  My complaint is that nice runtime
linking isn't part of the C standard.

So, can we mention runtime linking in the C bindings spec?  I suspect
there will be resistance.  Certainly we can't offer much _detail_ as the
details of modern linking vary by platform.

E.g., on Solaris I'd recommend that mechglue use dlopen() with
RTLD_GROUP and RTLD_LOCAL flags to load mechanisms.  Does Linux's rtld
provide similar options?  Does MS'?  Can we be sufficiently abstract in
this regard?

Nico
--

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Fri, 12 May 2006 12:53:13 -0400

> provide similar options?  Does MS'?  Can we be sufficiently abstract in

Linux's (glibc's) rtld provides RTLD_LOCAL; in fact, it is the default, as 
it should be.  But it doesn't provide RTLD_GROUP, which is fairly essential 
to the problem you're thinking of.

In any case, no, I don't think the GSS-API C bindings spec is the right 
place to get up on that particular soapbox.  Doing so is IMHO more likely 
to be confusing than helpful, especially since the rest of the document 
specifies an API, not an ABI.  For example, that means that applications 
and mechanisms (or whatever glue is behind the API) _do_ need to agree on 
the exact declarations of things, what the structure types are, the values 
of various constants, and so on.  So no, an application does not need to be 
built against a mechanism-specific header, but the app and meechanism need 
to agree on an ABI, which is outside the scope of the GSS-API spec.

-- Jeff

---

From: Ken Raeburn <[email-masked]>
Date: Fri, 12 May 2006 16:52:28 -0400

> 15 gssapi mechanism that are certified for interoperability and a  

I'm a bit surprised that this works, given the latitude granted in  
the C bindings spec.  Stuff like "pointer or integral type", where  
some platforms handle pointers and integers differently in function  
calls, two Windows function calling conventions, and other stuff,  
some of which is discussed in RFC 2744 Appendix B.  But if you've  
found that most or all implementations do behave consistently enough  
to have an ABI, perhaps we should see if that appendix should be  
updated with more specifics, or an informational doc published.

Or is it the case that there are other implementations you can't use,  
and they're just not "certified for interoperability"?

Ken

---

From: Nicolas Williams <[email-masked]>
Date: Fri, 12 May 2006 17:11:19 -0500

> updated with more specifics, or an informational doc published.

I think we might be able to have a largish non-normative appendix on ABI
considerations, including advice on dynamic linking, run-time link/
loading, etc...  Martin argues, privately, that we could do that (or at
least that's what I understood, Martin), and I think it would be good to
do that, even if it means having some platform-specific contents in
there.

Nico
--

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Sat, 13 May 2006 17:18:04 -0400

> do that, even if it means having some platform-specific contents in

Maybe instead of a largish non-normative appendix, it would be appropriate 
to have a largish non-normatve separate document ?

---

From: Jeffrey Altman <[email-masked]>
Date: Sat, 13 May 2006 18:29:35 -0400

Jeffrey Hutzelman wrote:

> Maybe instead of a largish non-normative appendix, it would be
> appropriate to have a largish non-normatve separate document ?

I would prefer an informational document as well.

Jeffrey Altman