Subject: Using SASL C API from a library

From: Greg Hudson <[email-masked]>
Date: Sun, 29 Jun 2003 09:00:15 -0700

Hi.  I'd like to point two deficiencies in the SASL C API as
documented in draft-newman-sasl-c-api-01.txt and implement in Cyrus
SASL.  First, the API includes functions:

  int sasl_client_init(const sasl_callback_t *callbacks);
  int sasl_server_init(const sasl_callback_t *callbacks,
                       const char *appname);
  static void sasl_done(void);

These are reasonable for using SASL from within an application, such
as sendmail, but involve too much global state to safely use SASL from
within a library, such as a Subversion repository access mechanism.
If I use the SASL C API from within my library, any application which
uses my library must not use the same API itself, nor can it link
against any other library which wants to use that API.

(There are some exceptional cases where things might work:

  * Using the API in a non-interleaved fashion, e.g. library A calls
    sasl_client_init(), creates and tears down some connections, and
    calls sasl_done(), then library B does the same thing.

  * Using the APIs in an interleaved fashion, but using a NULL
    default callbacks structure and never calling sasl_done().

But in the general case, the interaction won't succeed.)

Second, according to the draft, the sasl_client_init functions must be
called for the first time in a single-threaded environment.  A library
does not necessarily have the luxury of performing any calls from a
single-threaded environment.

I don't have a specific recommendation for fixing these problems.  But
they mean I can't in good conscience use the SASL C APIs from within a
library; I will either have to do that work within a subprocess,
implement my own SASL mechanisms, or abandon SASL for my protocol.

---

From: Lawrence Greenfield <[email-masked]>
Date: Sun, 29 Jun 2003 19:10:51 -0400

Date: Sun, 29 Jun 2003 09:00:15 -0700
   From: Greg Hudson <[email-masked]>
[...]
   These are reasonable for using SASL from within an application, such
   as sendmail, but involve too much global state to safely use SASL from
   within a library, such as a Subversion repository access mechanism.
   If I use the SASL C API from within my library, any application which
   uses my library must not use the same API itself, nor can it link
   against any other library which wants to use that API.

We've discussed this problem, though I'm not certain any text ever
made it into a draft. The ultimate recommendation is to prohibit
libraries from using global callbacks: they _must_ pass NULL to
sasl_client/server_init(). SASL libraries should do reference counting
so that only the last sasl_done() will actually destroy global state.

If an application wishes to use global callbacks, it can, safely with
the knowledge that the library will never override it's global
callbacks.

It's also necessary for the application to call sasl_set_mutex()
before invoking any library call that might initialize the SASL
library. There's no way for the library to know what locking
primitives to use.

[...]

You can create this luxury by having your own init() function that you
require be called in a single threaded environment. Unix processes
have global state---it is impossible for the library to work around
this. Unix also has multiple threading libraries---there are no locks
that will work in all cases.

This is a problem that all Unix shared libraries face. If you're
that concerned about it, you pretty much can't use _any_ library that
needs to manipulate process-wide state (from loading dynamic modules
to file locks to environment variables).

Larry

---

From: Greg Hudson <[email-masked]>
Date: 29 Jun 2003 19:29:16 -0400

On Sun, 2003-06-29 at 19:10, Lawrence Greenfield wrote:
> The ultimate recommendation is to prohibit
> sasl_client/server_init().

> the knowledge that the library will never override it's global

But won't the application's callbacks interfere with the library?

> library. There's no way for the library to know what locking

I've never heard of any other API using this approach to thread safety. 
It seems like a bizarre argument, especially when Unix systems converged
on the POSIX threads API years and years ago.

If there really are multiple different valid choices for the locking
primitives, what happens if the library decides to use one set of
primitives and the application decides to use another one?

> require be called in a single threaded environment.

If I am implementing a predefined standard API--like GSSAPI, although
that's not a realistic example--then I cannot create this luxury because
I do not control the interface.  The SASL C API is unusual in its
requirement (though not absolutely unique; I think OpenSSL has the same
problem), and can only get along with other APIs which impose the same
requirement.

> to file locks to environment variables).

I think all of this reasoning flows from the flawed premise that there
are no standard locking primitives in Unix which it is reasonable for a
library to use directly.  If the SASL C API continues to take this
premise to its logical conclusion and no one else does, the SASL API
will wind up living within a ghetto where it can only be used directly
from an application, and not from other libraries.

---

From: Rob Siemborski <[email-masked]>
Date: Sun, 29 Jun 2003 19:13:59 -0400 (EDT)

> single-threaded environment.

It is just as reasonable for the library to require its own initialization
to occur within a single threaded environment.

Your other concern is a known problem with the C API.  In general, it can
be solved by not using global callbacks except in the outermost
application, and having a reference count within the implementation that
does the right thing for calls to sasl_done().  I'm pretty sure a group of
us discussed this at the last IETF.

There is a more severe problem however.  It is unclear what to do in the
case where an application calls sasl_client_init and a library calls
sasl_server_init, and then the library calls sasl_done.

-Rob

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski | Andrew Systems Group * Research Systems Programmer
PGP:0x5CE32FCC | Cyert Hall 207 * [email-masked] * 412.268.7456
-----BEGIN GEEK CODE BLOCK----
Version: 3.12
GCS/IT/CM/PA d- s+: a-- C++++$ ULS++++$ P+++$ L+++(++++) E W+ N o? K-
w O- M-- V-- PS+ PE++ Y+ PGP+ t+@ 5+++ R@ tv-@ b+ DI+++ G e h r- y?
------END GEEK CODE BLOCK-----

---

From: Greg Hudson <[email-masked]>
Date: 30 Jun 2003 13:51:12 -0400

(leg forgot to cc the mail I'm responding to to ietf-sasl, so I'll
include it in its entirety at the end.)

> Callbacks for a given sasl_conn_t are used before the global
> callbacks.

Sure, but from the draft, it looks like there will be an IANA registry
of callback types, which means new callback types may be added.  So
there's no way my library can provide a complete set of callbacks for
all time, which means calls to the SASL API from my library may wind up
invoking the application's callbacks.

> There are lots of different threading APIs.  this is why all of the
> "portable" APIs abstract them, from the NSPR (netscape) to the APR
> (apache).

APR does abstract thread functions, but it uses pthreads on all Unix
systems.  It certainly doesn't ask the invoker of APR to pass in thread
functions to use.  At first glance, NSPR appears to behave similarly, at
least with the default build options.

> Obviously things don't work. All have to agree on them, which is why the 
> SASL API lets the application tell the library what locking primitives 
> to use.

And how are they supposed to agree?  Are you saying my interface not
only has to contain a library initialization function which must be run
in a single-threaded environment, but also a function which accepts
mutex functions to use?  That isn't very transparent at all.

> > living within a ghetto where it can only be used directly

> Plenty of libraries already use the SASL API (such as the OpenLDAP ldap 
> library).

And apparently they don't mind walking on a minefield.  The OpenLDAP
ldap library uses a global callbacks structure, contains a "/* XXX not
threadsafe */" comment in the function which calls sasl_client_init(),
and also contains a call to sasl_set_mutex() which sets the mutexes to
the LDAP private mutex routines.  I don't think they considered the case
where the application or another library also uses the SASL API, or they
considered it and threw up their hands in disgust.

On Mon, 2003-06-30 at 13:11, Lawrence Elias Greenfield wrote:
> Greg Hudson wrote:
> Callbacks for a given sasl_conn_t are used before the global callbacks.
> 
> No, they haven't. There are lots of different threading APIs. This is 
> why all of the "portable" APIs abstract them, from the NSPR (netscape) 
> to the APR (apache). Some high performance servers use things like state 
> threads or other cooperative threading libraries.
> 
> Obviously things don't work. All have to agree on them, which is why the 
> SASL API lets the application tell the library what locking primitives 
> to use.
> 
> Plenty of libraries already use the SASL API (such as the OpenLDAP ldap 
> library).
> 
> Larry
> 
>

---

From: Rob Siemborski <[email-masked]>
Date: Mon, 30 Jun 2003 14:50:44 -0400 (EDT)

> on the POSIX threads API years and years ago.

Solaris has a non-pthreads threading API.  There's no reason the SASL
C API should dictate what threading API to use.

-Rob

-----BEGIN GEEK CODE BLOCK----
Version: 3.12
------END GEEK CODE BLOCK-----

---

From: Rob Siemborski <[email-masked]>
Date: Mon, 30 Jun 2003 14:52:18 -0400 (EDT)

> library. There's no way for the library to know what locking

Isn't this a red herring?

Presumably the application and the library aren't accessing the same
objects in a thread-safe manner (or, if the application is mucking with
the SASL internals, its asking for its own problems).

As long as whatever mutex APIs the library is using work in a threaded
environment on a given OS, it shouldn't matter that the application and
the library are using different mutex APIs -- they're accessing different
objects!

Or am I missing some counterexample?

-Rob

-----BEGIN GEEK CODE BLOCK----
Version: 3.12
------END GEEK CODE BLOCK-----

---

From: Greg Hudson <[email-masked]>
Date: 30 Jun 2003 15:05:06 -0400

> C API should dictate what threading API to use.

An API generally has no reason to say anything about threads at all; it
should just avoid mandating global state.  The problems are that the
SASL API is currently saying too much about threads, and that it
mandates global state for the global callbacks structure.

The implementation of an API, on the other hand, has every right to use
the standard thread interfaces for the platforms it runs on, if for some
reason it has to use locks or other kinds of thread primitives under the
hood.  Unfortunately, that means you have to use pthreads on Unix and
something else on Windows, or you have to find a third-party abstraction
which covers both, but that's on different from the situation with GUI
code or anything else which isn't standard between the two platforms.

There might be cases where application programmers using a non-standard
thread interface will be frustrated by a library which uses different
interface, but this frustration would hardly be specific to a SASL
implementation.  Other libraries will already frustrate them--anything
which uses apr or NSPR, and probably just about every other library
which needs to do its own locking.

I note that the SASL API approach becomes very unwieldly when you go
beyond the realm of simple locking and into other thread interfaces like
condition variables and thread creation.

---

From: Lawrence Elias Greenfield <[email-masked]>
Date: Mon, 30 Jun 2003 15:22:20 -0400

>Or am I missing some counterexample?
Sure. Any application using cooperative threading can deadlock if a 
library uses pthread mutexes. Cooperative threading packages on Unix 
frequently work by intercepting socket calls and performing a task 
switch on any EAGAIN.  Scenario, library acquires a pthread mutex and 
then performs a blocking system call (our KERBEROS_V4 plugin could do 
this). A task switch is performed and a different thread (but the same 
kernel thread) attempts to acquire the mutex. This blocks forever, since 
the mutex is already locked and a task switch is never performed. 
(Alternatively, if the mutex is recursive, the lock is acquired, 
violating mutual exclusion.)

Larry

---

From: Philip Guenther <[email-masked]>
Date: Mon, 30 Jun 2003 12:24:40 -0700

>C API should dictate what threading API to use.

The question isn't whether there's a system that has multiple threading
APIs.  The question is whether there's a system that
a) has multiple threading APIs, one of which is POSIX, and
b) its pthreads mutexes don't provide mutual-exclusion for threads
   created via other APIs, and
c) the SASL community cares about the platform.

Solaris's pthreads mutexes work, no matter which system thread API you
use to create the actual threads.  (Indeed, the mutexes in its POSIX and
UI threads are completely interchangable, not that doing so would be
good style...)

I believe the Netscape and Apache portable runtimes both build on top of
whatever APIs are supported by the system (instead of implementing
user-level threading themselves), so they fail (b) above and therefore
don't serve as counter-examples either.

Philip Guenther

---

From: "Paul W. Nelson" <[email-masked]>
Date: Mon, 30 Jun 2003 14:39:54 -0500

Just FYI - this is already a real-world problem for me.  Our software
contains a plug-in that is loaded by Apple's directory services daemon, and
uses SASL.  Apple also loads other plug-ins that use SASL too.  The result
is a broken mess.

-- 
Paul W. Nelson
Thursby Software Systems, Inc.

> Date: 30 Jun 2003 15:05:06 -0400
> To: Rob Siemborski <[email-masked]>
> Cc: [email-masked]
> Subject: Re: Using SASL C API from a library
> 
>

---

From: Lawrence Greenfield <[email-masked]>
Date: Mon, 30 Jun 2003 20:07:07 -0400

From: Greg Hudson <[email-masked]>

Global state exists, whether or not we want it to. Some existing
authentication systems rely on environment variables. Dynamically
loaded libraries are global whether or not we want them to be.

So we have a choice: we can either ignore the fact that global state
exists and pretend to have solved the problem, or we can make it
explicit to the user of the library. Some things need to be done once
in a process; we can either hide those things from the library user
(by removing sasl_xxx_init()) or we can make them explicit.

I don't understand what you think is accomplished by hiding these
things. The problem can't possibly get any easier.

   I note that the SASL API approach becomes very unwieldly when you
   go beyond the realm of simple locking and into other thread
   interfaces like condition variables and thread creation.

Yes, that's true. It means that a SASL library can't rely on those
features and expect to be portable across multiple different
programming paradigms. That's ok: library implementors then have the
choice of being universal or being more specific.

Larry

---

From: Greg Hudson <[email-masked]>
Date: 30 Jun 2003 20:31:39 -0400

On Mon, 2003-06-30 at 20:07, Lawrence Greenfield wrote:
> authentication systems rely on environment variables.

I can't find any calls to putenv() or setenv() in Cyrus SASL, and it's
not common practice for a library to expect calling code to set
environment variables.  Can you point out where this is a practical
issue?

> Dynamically loaded libraries are global whether or not we want them to be.

SUSv3 requires that dlopen() be thread-safe, and I have no reason to
believe that the situation is otherwise on any modern platform. 
(Solaris documents it as such.) 

> things. The problem can't possibly get any easier.

If, in the final analysis, there really are things which absolutely must
be performed only once, they can be protected with a mutex, using the
standard threading primitives of the platforms the library runs on. 
Although this solution introduces some Unix/Windows portability issues,
it is superior to requiring an initialization function because the
callers may not be able to execute any code from a single-threaded
environment.  (This is because the callers may be libraries implementing
an externally specified interface which does not contain a
single-threaded intialization function.  I raised this argument before
and it was not addressed.)

Since people have grown silent on the topic, I'll note that no excuse
has been given for the global callbacks structure, and no answer has
been given to my argument that library calls to the SASL C API could
accidentally trigger invocations of the application's global callbacks
no matter what the library does to guard against this possibility.  If
transition for users of Cyrus SASL is the only concern, then the API
spec can preserve the parameter as a dummy and mandate that all callers
must pass NULL, as has been done in Unix with gettimeofday() and the
timezone parameter.

---

From: Rob Siemborski <[email-masked]>
Date: Mon, 30 Jun 2003 21:56:46 -0400 (EDT)

> environment variables.  Can you point out where this is a practical

Well, for example, Kerberos libraries rely on enviornment variables to
name the ticket cache.

> standard threading primitives of the platforms the library runs on.

This works, until an application is running using an incompatible
threading API (see Larry's answer to my earlier question).

Note that this isn't just to get around things that have to be done once,
some libraries that the SASL mechanisms themselves make use of may not be
thread safe.

> must pass NULL, as has been done in Unix with gettimeofday() and the

Unless I am misremembering, this is basically what we discussed in San
Francisco (I thought I had mentioned that).

Note that this is difficult for situations such as the getopt callback,
where path information to loadable modules (or similar information that is
requried for the library to initialize) may need to be obtained.  In this
case (since it is part of the one-time library startup) the application
would need to supply a getopt callback.  Since its one-time startup state,
your library couldn't change it if it wanted to.

-Rob

-----BEGIN GEEK CODE BLOCK----
Version: 3.12
------END GEEK CODE BLOCK-----

---

From: Greg Hudson <[email-masked]>
Date: 01 Jul 2003 00:07:48 -0400

> Well, for example, Kerberos libraries rely on enviornment variables to

So what?  *Reading* environment variables is perfectly thread-safe. 
(And actually, writing them tends to be thread-safe too, although it's
not required by the standards I've consulted.  And of course a
read-modify-write cycle will have a race.)

> threading API (see Larry's answer to my earlier question).

Let's say we accept the bizarre hypothesis that non-standard threading
APIs are important enough to worry about, and also the hypothesis that
they can't be accomodated by interposing an implementation of the
pthreads interface at link time.  (The second hypothesis may well be
justified; interposition on Unix requires making the ABIs compatible,
and I have no idea whether it's feasible at all on Windows.)

If both of those are true, then the SASL API (or better yet, a
non-standardized extension to the SASL API) can accomodate them by
creating an *optional* API call to set the mutex functions.  The call
would have to be made from a single-threaded environment, but it
wouldn't have to be made at all if the application isn't doing something
screwy.  And hopefully a library trying to be thread-safe wouldn't be
doing anything that strange.

> some libraries that the SASL mechanisms themselves make use of may not be

Point taken.  If a SASL implementation wants to be thread-safe and wants
to make use of non-thread-safe libraries, it must lock.  (And even that
won't help if other libraries or the application wants to use the same
non-thread-safe libraries, since there's no way for the SASL
implementation and the other caller to agree on a mutex.)

I don't think this affects my central argument.

> > If transition for users of Cyrus SASL is the only concern, then the API

> Francisco (I thought I had mentioned that).

(The minutes do not appear to document any discussion of the SASL C API,
incidentally.)

Previously, you said "it can be solved by not using global callbacks
except in the outermost application," which is very different from "not
using global callbacks at all."

> Note that this is difficult for situations such as the getopt callback
> requried for the library to initialize) may need to be obtained.

There is really no limit to the number of awful things you could justify
putting into the API if you want to satisfy every possible foible of a
back-end library.  Perhaps some back-end libraries want to demand a
special version of malloc(), so we'd better force the application to ask
SASL what malloc to use and use that for every allocation.  Perhaps some
back-end libraries want to contact a smart-card reader, so we'd better
make the application pass in callbacks for that.  The insanity would
never end.

So, unless you can demonstrate a present practical issue, I don't think
this argument is even vaguely compelling.

Franky, the getopt callback is a great big hole in the spec for
unspecified behavior (both mechanism-specific and
implementation-specific), since there is no standardization of what the
options mean.  Presumably the idea is that you would provide the answers
from a configuration file or from command-line options, giving a sort of
direct communications channel between the user and the SASL library or
specific mechanism.  That may sound attractive to implementors, but it's
a totally unacceptable idea, because it's impossible to document such a
config file or command-line syntax without knowing the intimate (and
version-specific) details of your SASL implementation, and because it's
impossible to provide reasonable answers in a situation where a config
file or command line doesn't make sense.

---

From: Philip Guenther <[email-masked]>
Date: Mon, 30 Jun 2003 21:22:45 -0700

...
>threading API (see Larry's answer to my earlier question).

I knew I should have replied to Larry's answer.  I find his example
unconvincing.  He has suggested that there may exist a cooperative
threading library running on a system that has a system level threading
implementation and that intercepts blocking I/O calls (i.e., supplies
symbols like connect(), read(), etc) such that it intercepts the calls
of unmodified libraries, but that *doesn't* intercept calls to the
system's threading primitives.  (If that's not what you're suggesting,
Larry, please correct me.)

Such a library could not be safely used with another library that
performed (nominally) blocking I/O in a thread-safe fashion using the
system mutexes.  On systems supporting POSIX threads, that would include
the stdio implementation in libc.  Any library that multiplexes access
to a socket via a system level mutex will be broken.  I hope you don't
need to call gethostname_r(), getaddrinfo(), or getnameinfo(), because
they can all deadlock with such a library.

To be charitable, I'll call such a library implementation "incomplete,
not suitable for production use".

Oh, and if the system doesn't even have a threading implementation, then
such an intercepting library would render any function or library that
uses a blocking socket call unsafe to use in multiple threads.

If a threading library is going to pretend to be transparent in use by
intercepting I/O calls, it MUST also intercept whatever primitives are
used by the system to make code thread-safe.  If the system doesn't have
threads already, then the library MUST NOT intercept calls.  Otherwise,
the library is so dangerous that the set of applications that it is
suitable for has an insignificant intersection with the set of
applications using the SASL library.

Philip Guenther

---

From: Lawrence Greenfield <[email-masked]>
Date: Sun, 29 Jun 2003 19:10:51 -0400

[...]

Larry

---

From: Greg Hudson <[email-masked]>
Date: 29 Jun 2003 19:29:16 -0400

---

From: Rob Siemborski <[email-masked]>
Date: Sun, 29 Jun 2003 19:13:59 -0400 (EDT)

-Rob

-----BEGIN GEEK CODE BLOCK----
Version: 3.12
------END GEEK CODE BLOCK-----

---

From: Greg Hudson <[email-masked]>
Date: 30 Jun 2003 13:51:12 -0400

---

From: Rob Siemborski <[email-masked]>
Date: Mon, 30 Jun 2003 14:50:44 -0400 (EDT)

-Rob

-----BEGIN GEEK CODE BLOCK----
Version: 3.12
------END GEEK CODE BLOCK-----

---

From: Rob Siemborski <[email-masked]>
Date: Mon, 30 Jun 2003 14:52:18 -0400 (EDT)

-Rob

-----BEGIN GEEK CODE BLOCK----
Version: 3.12
------END GEEK CODE BLOCK-----

---

From: Greg Hudson <[email-masked]>
Date: 30 Jun 2003 15:05:06 -0400

---

From: Lawrence Elias Greenfield <[email-masked]>
Date: Mon, 30 Jun 2003 15:22:20 -0400

Larry

---

From: Philip Guenther <[email-masked]>
Date: Mon, 30 Jun 2003 12:24:40 -0700

Philip Guenther

---

From: "Paul W. Nelson" <[email-masked]>
Date: Mon, 30 Jun 2003 14:39:54 -0500

---

From: Lawrence Greenfield <[email-masked]>
Date: Mon, 30 Jun 2003 20:07:07 -0400

Larry

---

From: Greg Hudson <[email-masked]>
Date: 30 Jun 2003 20:31:39 -0400

---

From: Rob Siemborski <[email-masked]>
Date: Mon, 30 Jun 2003 21:56:46 -0400 (EDT)

-Rob

-----BEGIN GEEK CODE BLOCK----
Version: 3.12
------END GEEK CODE BLOCK-----

---

From: Greg Hudson <[email-masked]>
Date: 01 Jul 2003 00:07:48 -0400

---

From: Philip Guenther <[email-masked]>
Date: Mon, 30 Jun 2003 21:22:45 -0700

Philip Guenther

To: Rob Siemborski <[email-masked]>
Cc: Greg Hudson <[email-masked]>, Lawrence Greenfield <[email-masked]>, [email-masked]
Subject: Re: Using SASL C API from a library 
Date: Mon, 30 Jun 2003 21:22:45 -0700
From: Philip Guenther <[email-masked]>
Sender: [email-masked]

Philip Guenther

Subject: Re: Using SASL C API from a library
From: Greg Hudson <[email-masked]>
To: Rob Siemborski <[email-masked]>
Cc: [email-masked]
Organization: 
Date: 01 Jul 2003 00:07:48 -0400
Sender: [email-masked]

Date: Mon, 30 Jun 2003 21:56:46 -0400 (EDT)
From: Rob Siemborski <[email-masked]>
To: Greg Hudson <[email-masked]>
cc: Lawrence Greenfield <[email-masked]>, [email-masked]
Subject: Re: Using SASL C API from a library
Sender: [email-masked]

-Rob

-----BEGIN GEEK CODE BLOCK----
Version: 3.12
------END GEEK CODE BLOCK-----

Subject: Re: Using SASL C API from a library
From: Greg Hudson <[email-masked]>
To: Lawrence Greenfield <[email-masked]>
Cc: [email-masked]
Organization: 
Date: 30 Jun 2003 20:31:39 -0400
Sender: [email-masked]

Date: Mon, 30 Jun 2003 20:07:07 -0400
From: Lawrence Greenfield <[email-masked]>
To: Rob Siemborski <[email-masked]>, Greg Hudson <[email-masked]>
Cc: [email-masked]
Subject: Re: Using SASL C API from a library
Sender: [email-masked]

Larry

Date: Mon, 30 Jun 2003 14:39:54 -0500
Subject: Re: Using SASL C API from a library
From: "Paul W. Nelson" <[email-masked]>
To: <[email-masked]>
Sender: [email-masked]

To: Rob Siemborski <[email-masked]>
Cc: Greg Hudson <[email-masked]>, Lawrence Greenfield <[email-masked]>, [email-masked]
Subject: Re: Using SASL C API from a library 
Date: Mon, 30 Jun 2003 12:24:40 -0700
From: Philip Guenther <[email-masked]>
Sender: [email-masked]

Philip Guenther

Date: Mon, 30 Jun 2003 15:22:20 -0400
From: Lawrence Elias Greenfield <[email-masked]>
To: Rob Siemborski <[email-masked]>
CC: Lawrence Greenfield <[email-masked]>, [email-masked], Greg Hudson <[email-masked]>
Subject: Re: Using SASL C API from a library
Sender: [email-masked]

Larry

Organization: 
Sender: [email-masked]

Date: Mon, 30 Jun 2003 14:52:18 -0400 (EDT)
From: Rob Siemborski <[email-masked]>
To: Lawrence Greenfield <[email-masked]>
cc: [email-masked], Greg Hudson <[email-masked]>
Subject: Re: Using SASL C API from a library
Sender: [email-masked]

-Rob

-----BEGIN GEEK CODE BLOCK----
Version: 3.12
------END GEEK CODE BLOCK-----

Date: Mon, 30 Jun 2003 14:50:44 -0400 (EDT)
From: Rob Siemborski <[email-masked]>
To: Greg Hudson <[email-masked]>
cc: Lawrence Greenfield <[email-masked]>, [email-masked]
Subject: Re: Using SASL C API from a library
Sender: [email-masked]

-Rob

-----BEGIN GEEK CODE BLOCK----
Version: 3.12
------END GEEK CODE BLOCK-----

Subject: Re: Using SASL C API from a library
From: Greg Hudson <[email-masked]>
To: Lawrence Elias Greenfield <[email-masked]>
Cc: [email-masked]
Organization: 
Date: 30 Jun 2003 13:51:12 -0400
Sender: [email-masked]

Subject: Re: Using SASL C API from a library
From: Greg Hudson <[email-masked]>
To: Lawrence Greenfield <[email-masked]>
Cc: [email-masked]
Organization: 
Date: 29 Jun 2003 19:29:16 -0400
Sender: [email-masked]

Date: Sun, 29 Jun 2003 19:13:59 -0400 (EDT)
From: Rob Siemborski <[email-masked]>
To: Greg Hudson <[email-masked]>
cc: [email-masked]
Subject: Re: Using SASL C API from a library
Sender: [email-masked]

-Rob

-----BEGIN GEEK CODE BLOCK----
Version: 3.12
------END GEEK CODE BLOCK-----

Date: Sun, 29 Jun 2003 19:10:51 -0400
From: Lawrence Greenfield <[email-masked]>
To: [email-masked], Greg Hudson <[email-masked]>
Subject: Re: Using SASL C API from a library
Sender: [email-masked]

[...]

Larry

Date: Sun, 29 Jun 2003 19:02:58 -0400
From: Lawrence Greenfield <[email-masked]>
To: [email-masked]
Subject: Re: Comments on the SASL C API
Sender: [email-masked]

   Date: Sun, 29 Jun 2003 15:36:22 -0400
   From: Greg Hudson <[email-masked]>

   On 29 Jun 2003 19:15:31 +0200, Simon Josefsson wrote:
   > int sasl_decode(sasl_conn_t *conn,
   >             const char *input,
   >             unsigned int inputlen,
   >             const char **output,
   >             unsigned int *outputlen);

   > Consider what happens if the input/inputlen fields of the
   > sasl_decode() contains two tokens from sasl_encode(), or one SASL
   > token and a few bytes of the next SASL token.

   Presumably it is the responsibility of the sasl_decode()
   implementation to remember (using state associated with conn) the
   partial bytes.  If there are two or more tokens in input, the
   resulting output tokens can presumably be concatenated together; I
   don't think applications are supposed to rely on knowing anything
   about the security later's framing.

That is correct. Because of the limitations on the buffer sizes, the
SASL library can always save this "intermediate" state in a constant
amount of space (at most the max SASL token size).

I believe it's more efficient for the library to do this than
requiring the application to "remember" unconsumed encoded data. It is
also less error prone for application programmers.

The Cyrus SASL documentation has the following hints. If both sides
follow these guidelines---or their equivalent---you get efficient
communication.

* Before starting authentication, set maxbufsize in your security
  properties to be the buffer size that you pass to the read() system
  call---that is, the amount of data you're prepared to read at any
  one time.

* After authentication finishes, use sasl_getprop() to retrieve the
  SASL_MAXOUTBUF value, and call sasl_encode() with chunks of data of
  that size or less. sasl_encode() will throw an error if you call it
  with a larger chunk of data, so be careful!

Larry

Date: Sun, 29 Jun 2003 15:36:22 -0400
From: Greg Hudson <[email-masked]>
To: [email-masked]
Subject: Re: Comments on the SASL C API
Sender: [email-masked]

On 29 Jun 2003 19:15:31 +0200, Simon Josefsson wrote:
> int sasl_decode(sasl_conn_t *conn,
>             const char *input,
>             unsigned int inputlen,
>             const char **output,
>             unsigned int *outputlen);

> Consider what happens if the input/inputlen fields of the
> sasl_decode() contains two tokens from sasl_encode(), or one SASL
> token and a few bytes of the next SASL token.

Presumably it is the responsibility of the sasl_decode()
implementation to remember (using state associated with conn) the
partial bytes.  If there are two or more tokens in input, the
resulting output tokens can presumably be concatenated together; I
don't think applications are supposed to rely on knowing anything
about the security later's framing.

> This require that mechanisms are able to figure this information
> out, and looking at e.g. DIGEST-MD5 I'm not sure this is the case.
> How does DIGEST-MD5 decode() know that a SASL encode token ends?
> Some framing seem to be required.  I may be missing something, even
> though I have implemented this, and can't recall how this problem is
> solved in my implementation.

RFC 2222 states:

   Once the security layer is in effect, the protocol stream is
   processed by the security layer into buffers of cipher-text.  Each
   buffer is transferred over the connection as a stream of octets
   prepended with a four octet field in network byte order that
   represents the length of the following buffer.

To: [email-masked]
Subject: Comments on the SASL C API
From: Simon Josefsson <[email-masked]>
Date: Sun, 29 Jun 2003 19:15:31 +0200
Sender: [email-masked]

This message contains a description of a problem in the proposed API,
and a proposed extension for the API.  The extension allows moving
session contexts between processes, or even machines in a clustered
environment.  The rationale for the extension is included in the
proposal below.  Both are taken from old personal communication, but
as I don't know if the problem has a solution, and to gage interest in
the extension, I'd like to mention them in public.

Problem:

When I considered supporting the API, I discovered the following
problem.  Recall from 4.1:

int sasl_decode(sasl_conn_t *conn,
            const char *input,
            unsigned int inputlen,
            const char **output,
            unsigned int *outputlen);

      This function decodes a block of data received using security
      layer (if any). The output and outputlen are filled in with the
      decoded data and its length respectively. If there is no security
      layer the input buffer is returned in the output. Otherwise, the
      output is only valid until a next call to sasl_decode or sasl_dis-
      pose.

Consider what happens if the input/inputlen fields of the
sasl_decode() contains two tokens from sasl_encode(), or one SASL
token and a few bytes of the next SASL token.  If I understand the
specification correctly, the application have no way of splitting up
incoming data as separate SASL tokens, so it must simply pass on
whatever is returned from read() to sasl_decode().  If the server
always sends entire SASL tokens using one write(), it will probably
work, but if the server concatenates two sasl_encode() output tokens
and sends them, or if it buffers them somehow, the client will be
confused.

If this isn't a problem in sasl-c-api (I haven't read it carefully),
please point me to how it is handled.

The solution I would suggest is to replace "int inputlen" with "int
*inputlen" and have it populate the output variable with how many
bytes it actually used from the input buffer, so the application can
use the remaining bytes (possibly concated with new incoming data) and
call the function again.  This require that mechanisms are able to
figure this information out, and looking at e.g. DIGEST-MD5 I'm not
sure this is the case.  How does DIGEST-MD5 decode() know that a SASL
encode token ends?  Some framing seem to be required.  I may be
missing something, even though I have implemented this, and can't
recall how this problem is solved in my implementation.

Extension:

Here is a writeup on an idea I have received.  It might cause some
complications and likely needs more pondering.  However, if the
rationale for adding this makes sense to you, I'd appreciate if you
spent the time incorporating the functions.  If you are familiar with
the GSS-API, compare this to the gss_export_sec_context and
gss_import_sec_context functions that were added to GSS-API version 2
(see RFC 2744).

 5.2.   Exporting and importing security layer state

      Some application environments wishes to separate the initial and
      short authentication step from the potentially long lived
      application step, where the application uses sasl_encode() and
      sasl_decode() to transfer application data.  This separation may
      be between processes (separation across hosts is more
      complicated, as the security layer state is often dependent on
      the host address, but still conceivable).  The reason for such
      separation may be to guarantee that long-term secrets (e.g.,
      passwords) cannot be recovered by compromising the application
      that handles existing session.

      The functions defined in this section allows the application to
      export the security layer state associated with a session as an
      opaque character string, and later import the opaque character
      string.  The opaque state typically contains an encryption key,
      initialization vectors, what encryption algorithm to use, packet
      number etc.

  5.2.1. sasl_export_security_layer_state function

Arguments:
            sasl_conn_t *conn,
            const char **state,
            unsigned int *statelen

Results:
            SASL_OK        -- Success
            SASL_NOTDONE   -- Security layer negotiation not finished

      This function exports the security layer state required to
      implement the sasl_encode() and sasl_decode() functions after
      the state has been imported.  The security layer state is an
      opaque data string.  The state and statelen are filled in with
      the opaque state data and its length respectively. If the
      security layer negotiation has not finished, SASL_NOTDONE is
      returned.  After calling this function, the application should
      treat the SASL session as concluded and call sasl_dispose().  In
      particular, the application MUST NOT call sasl_encode() or
      sasl_decode() [XXX sasl_dencode should simply return a error
      code when it happens].  The state data is only valid until a
      call to sasl_dispose(), and should be copied before calling
      sasl_dipose() if necessary.

  5.2.2. sasl_import_security_layer_state function

Arguments:
            const char *state,
            unsigned int statelen
            sasl_conn_t **conn,

Results:
            SASL_OK        -- Success
            SASL_BADPARAM  -- Invalid parameter supplied

      This function create a SASL session, for use with sasl_encode(),
      sasl_decode() and sasl_dispose() only, from the given security
      layer state.  The security layer state is an opaque data string,
      as returned from sasl_export_security_layer_state().

Thanks,
Simon

Date: Sun, 29 Jun 2003 09:00:15 -0700
To: [email-masked]
From: Greg Hudson <[email-masked]>
Subject: Using SASL C API from a library
Sender: [email-masked]

To: IETF-Announce: ;
Cc: [email-masked]
From: [email-masked]
Reply-to: [email-masked]
Subject: I-D ACTION:draft-ietf-sasl-crammd5-00.txt
Date: Tue, 10 Jun 2003 07:53:09 -0400
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Simple Authentication and Security Layer Working Group of the IETF.

	Title		: The CRAM-MD5 SASL Mechanism
	Author(s)	: L. Nerenberg
	Filename	: draft-ietf-sasl-crammd5-00.txt
	Pages		: 7
	Date		: 2003-6-9
	
This document defines a simple challenge-response authentication
mechanism, using a keyed-hash digest, for use with the Simple
Authentication and Security Layer (SASL)

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-sasl-crammd5-00.txt

To remove yourself from the IETF Announcement list, send a message to 
ietf-announce-request with the word unsubscribe in the body of the message.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-ietf-sasl-crammd5-00.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-sasl-crammd5-00.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="[email-masked]"

Content-Type: text/plain
Content-ID:	<[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-ietf-sasl-crammd5-00.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-sasl-crammd5-00.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

--OtherAccess--

--NextPart--

Date: Fri, 6 Jun 2003 23:55:35 -0400
From: Lawrence Greenfield <[email-masked]>
To: "Kurt D. Zeilenga" <[email-masked]>, Lyndon Nerenberg <[email-masked]>
Cc: SASL list <[email-masked]>
Subject: Re: rfc2222bis comments
Sender: [email-masked]

   Date: Fri, 6 Jun 2003 20:12:14 -0600
   From: Lyndon Nerenberg <[email-masked]>

   >  From an informal poll showed sizeof("all those implementations") == 0.
   > If the sizeof > 0, then I'd retract the suggestion.  Is it?

   There is no way of knowing. You cannot possibly enumerate every 
   instance of SASL that has been deployed. Therefore "20" is a constant 
   until the end of time.

I agree with Lyndon. I see no compelling reason why this should be
lifted.

Larry

Date: Fri, 6 Jun 2003 20:12:14 -0600
Subject: Re: rfc2222bis comments
Cc: "RL 'Bob' Morgan" <[email-masked]>, SASL list <[email-masked]>
To: "Kurt D. Zeilenga" <[email-masked]>
From: Lyndon Nerenberg <[email-masked]>
Sender: [email-masked]

>  From an informal poll showed sizeof("all those implementations") == 0.
> If the sizeof > 0, then I'd retract the suggestion.  Is it?

There is no way of knowing. You cannot possibly enumerate every 
instance of SASL that has been deployed. Therefore "20" is a constant 
until the end of time.

--lyndon

Date: Fri, 6 Jun 2003 19:54:06 -0600
Subject: Re: rfc2222bis comments
Cc: [email-masked]
To: "Kurt D. Zeilenga" <[email-masked]>
From: Lyndon Nerenberg <[email-masked]>
Sender: [email-masked]

On Friday, June 6, 2003, at 07:18  PM, Kurt D. Zeilenga wrote:

> I suggest we change the 20 character maximum length to a
> higher number so that OIDs can be more easily used as mechanism
> names.  I don't think this will break anyone... and it will make
> GSS API mechanism specification much easier.

This will break any existing implementation that uses a fixed 20 octet 
array to store the mechanism name.

--lyndon

Date: Fri, 06 Jun 2003 18:45:13 -0700
To: "RL 'Bob' Morgan" <[email-masked]>
From: "Kurt D. Zeilenga" <[email-masked]>
Subject: Re: rfc2222bis comments
Cc: SASL list <[email-masked]>
Sender: [email-masked]

At 06:35 PM 6/6/2003, RL 'Bob' Morgan wrote:

>On Fri, 6 Jun 2003, Kurt D. Zeilenga wrote:
>
>> I suggest we change the 20 character maximum length to a higher number
>> so that OIDs can be more easily used as mechanism names.  I don't think
>> this will break anyone... and it will make GSS API mechanism
>> specification much easier.
>
>Eh ... it will break all those implementations that allocate a 20-char
>buffer for a mech-name, won't it?

 From an informal poll showed sizeof("all those implementations") == 0.
If the sizeof > 0, then I'd retract the suggestion.  Is it?

>> The "authorization identity" should clearly be described as an
>> optional proxy authorization identity.
>
>I disagree.  Proxy is only one example of the utility of authz-id.  A
>system might permit my authenticated user credential to assume several
>roles, letting me specify the role via authz-id.

Okay.  I was trying to figure out how to stress even more (than
the SHOULD NOT derive statement) that clients should not specify
an authzid unless the user provided one with the credentials.

Kurt 

Date: Fri, 6 Jun 2003 18:35:39 -0700 (PDT)
From: "RL 'Bob' Morgan" <[email-masked]>
To: "Kurt D. Zeilenga" <[email-masked]>
cc: SASL list <[email-masked]>
Subject: Re: rfc2222bis comments
Sender: [email-masked]

On Fri, 6 Jun 2003, Kurt D. Zeilenga wrote:

> I suggest we change the 20 character maximum length to a higher number
> so that OIDs can be more easily used as mechanism names.  I don't think
> this will break anyone... and it will make GSS API mechanism
> specification much easier.

Eh ... it will break all those implementations that allocate a 20-char
buffer for a mech-name, won't it?

> The "authorization identity" should clearly be described as an
> optional proxy authorization identity.

I disagree.  Proxy is only one example of the utility of authz-id.  A
system might permit my authenticated user credential to assume several
roles, letting me specify the role via authz-id.

Your other points I agree with ... 8^)

 - RL "Bob"

Date: Fri, 06 Jun 2003 18:18:49 -0700
To: [email-masked]
From: "Kurt D. Zeilenga" <[email-masked]>
Subject: rfc2222bis comments
Sender: [email-masked]

I have a few technical comments for the WG to consider.
(I also have a number of editorial comments which I send
directly to the editor.)

I suggest we change the 20 character maximum length to a
higher number so that OIDs can be more easily used as mechanism
names.  I don't think this will break anyone... and it will make
GSS API mechanism specification much easier.

The "authorization identity" should clearly be described as an
an optional proxy authorization identity.  Empty and no strings
are semantically equivalent. Mechanisms should not allow both
empty and no string to be transferred (we've had interop problems
with DIGEST-MD5 and EXTERNAL because implementations didn't always
treat them as being the same).  Also, replace "entire repertoire
of Unicode Normalization form KC [UNICODE-NORMALIZATION]" with
"entire Unicode repertoire".  Also, use same repertoire here
in first paragraph of 4.2.

The last paragraph of 4.2 needs some word-smithing.

The wording of this section 5 paragraph is problematic.
>   A protocol profile MAY further refine the definition of an         
>   authorization identity by adding additional syntactic restrictions
>   and protocol-specific semantics.

Profiles MUST specify the form of the authorization identity
well as how authorization identities are to be compared.  It
may be appropriate to encourage use of simple user names here
(and, hence, SASLprep).

More later....