Subject: Re: Scram as a gs2 mechanism ABNF

From: Sam Hartman <[email-masked]>
Date: Wed, 06 Aug 2008 14:45:44 -0400

>>>>> "Chris" == Chris Newman <[email-masked]> writes:

    Chris> --On July 29, 2008 8:12:40 -0400 Sam Hartman
    Chris> <[email-masked]> wrote:
    >> Here are Nico and my notes on scram as a GSS-API mechanism.
    >> 
    >> My hope is that Chris and others will find this acceptable.

    Chris> FYI, I am not likely to find this protocol acceptable, but
    Chris> I want to make sure I understand the protocol before I
    Chris> commit one way or the other.  At this point, I do not
    Chris> understand the "mac" rule and need to confirm my
    Chris> assumptions.

We discussed some of this in person, but I want to follow up for the list.

    Chris> My basis for evaluating this protocol relative to the
    Chris> current SCRAM protocol is as follows:

    Chris> I recognize a modest benefit in code sharing,
    Chris> deployment/implementation/interoperability if a SCRAM-GS2
    Chris> mechanism and a SCRAM-SASL mechanism use the same
    Chris> bits-on-the-wire as both a GSSAPI-guru and a SASL-app-guy
    Chris> can implement in their comfort zone.  I contrast that with
    Chris> separate SCRAM-GS2 used for things like NFS that are GSSAPI
    Chris> only, and SCRAM-SASL used for things that support SASL.  In
    Chris> that case, there's risk of the GSSAPI mechanism leaking
    Chris> through in a SASL-based protocol (even if we say SHOULD
    Chris> NOT) and introducing interop problems -- much the way
    Chris> GSS-SPNEGO leaked into SASL.  

I think that if we design separate gss-scram and scram SASL mechanisms
we should expect that some applications will use GSS-SCRAM as a SASL
mechanism.  I would object to us saying that this SHOULD NOT happen
because there is no reasonable way for a SASL implementation to know whether some GSS-API mechanism corresponds to another SASL mechanism or not.
With SPNEGO, we got lucky because we  needed to revise the GSS-API SASL binding.
The only way I can see limiting this is to assume that by default a GSS-API mechanism should not be used with SASL.
I would object  to that in the strongest possible terms.

So, I think that if we design separate mechanisms we should expect
that platforms that tend to use GSS-API will offer the GSS version in
SASL dialogues.  Whether these same platforms will also offer SASL
versions will depend on a lot of factors.  Of course whether an
application supports one or both will depend on a lot of factors.
Applications that tend to use a full-featured SASL framework will be
more likely to support both versions than applications that hard-code
a set of SASL mechanisms.

    Chris> Regardless, if SCRAM is to
    Chris> succeed it needs to be implemented in multiple frameworks
    Chris> (HTTP-Auth, SASL, GSSAPI, SSH-auth, perhaps EAP).  

Note that HTTP, SSH and SASL all support GSS-API.  I can understand
claiming that the HTTP support for GSS-API is a bit esthetically displeasing.

I agree though that for scram to be successful it needs to be
available in all these situations.  My primary goals are two-fold: to
reduce security analysis and to make sure that choice of application
substrate does not influence choice of security technology.  So, if
SCRAM was not going to have the same over-the-wire bits as GS2, then
I'd be a lot happier if we did more than just a SASL mechanism.
First, that would allow us to do the security analysis for all the
mechanisms at once.  Second, it would give confidence that scram was
at least going to have the standardization energy necessary to
succeed.

    Chris> So the benefit of potentially eliminating one framework is
    Chris> only a modest benefit.

Obviously we disagree on this point.

    Chris> I contrast that with the additional complexity of
    Chris> implementing a pure SASL mechanism.  GSSAPI is not an
    Chris> app-level protocol; it's an OS-level protocol when present.
    Chris> Deploying things in the OS often takes longer than
    Chris> deploying them in the app layer.  So I consider the cost of
    Chris> deploying the SASL mechanism the overriding consideration
    Chris> in terms of actually getting this mechanism deployed.

    Chris> Constant binary strings are ugly, but tolerable especially
    Chris> if they are few and only a prefix or postfix.  A length is
    Chris> much more problematic (regardless of whether or not it is
    Chris> binary).  Each computed length is an opportunity for a
    Chris> buffer overflow implementation bug and thus makes the
    Chris> protocol itself less secure when it enters the real world.
    Chris> A security advisory saying SCRAM is insecure due to a
    Chris> buffer overflow in someone's implementation would do harm
    Chris> to SCRAM deployment.

I think that this argument applies equally well to GS2 as it does to
scram.  We should not have standards for what is good protocol design
that differ between two protocols in the same working group.  So, if
lengths are bad, let's remove them from GS2.  If lengths are OK, then
they should be OK for scram.  I don't care very much whether we decide
lengths are fine or problematic.

    Chris> I am assuming the "4*octet" is a big-endian 32-bit unsigned
    Chris> integer octet count of part of the protocol.  I am assuming
    Chris> ctx-length covers the "context" (for each exchange
    Chris> respectively: scram-c1, scram-s1, scram-c2-ctx,
    Chris> scram-s2-ctx).  I am assuming wrap-length covers the
    Chris> "wrapped" portion of each exchange (respectively: 0, 0,
    Chris> scram-c2-wrap, scram-s2-wrap).
Your assumptions are correct to the best of my understanding.

    >> der-definite-length = %00-%7F / %81 octet / %82 2*octet / %83
    >> 3*octet / %84 4*octet
    >> 
    >> gss-init-ctx-header = %60 der-definite-length "<constant
    >> encoded OID octet string here>"

    Chris> I am assuming that the "der-definite-length" is the length
    Chris> of the encoded OID only.  I presume gss-init-ctx-header
    Chris> will collapse to a constant octet string for each SCRAM
    Chris> mechanism name and no SCRAM implementation will be required
    Chris> to generate a DER ASN.1 length for this purpose.
Unfortunately, no. Section 3.1 of RFC 2743 says that the length needs to cover the entire token.
If the working group agrees that this is too ugly, I think we have two approaches:

* Move the complexity into general purpose GS2 mechanisms.
Require that if the GSS-API token starts with the header defined in RFC 2743  section 3.1, then that header is removed and replaced with a SASL-friendly representation.
The over-the-wire encoding might look something like 
"OID=1.2.3.99212.24 DATA=base64stuffgoeshere"
The receiving side reconstructs the GSS-API glop.
The assumption here is that it's OK if a general purpose GS2 mechanism needs to be more complicated than the wire encoding for SASL.

* Relax RFC 2743 3.1 so that the length covers items 2-5 but MAY or MAY NOT need to cover  the following mechanism-specific data.
I actually suspect that would be a harmless change.
The assumption here is that GSS-API is too complicated for everyone and we should fix it.

    >> 
    >> mac = "m=" base64 ";" ;; This is a GSS wrap with integrity
    >> token for SCRAM; normally ;; this is all it will be, but a true
    >> SCRAM GSS mechanism may ;; have sequence numbers too

    Chris> I do not understand where the content of the base64 comes
    Chris> from.  How is the content computed by a SCRAM
    Chris> implementation with no GSSAPI library present?
This would simply be the hmac-whatever of the data being mac'd.  Nico
wants the ability to have extensions in the MAC string used for
sequencing that would not show up in SASL.  In other words, we'd like
to include non-critical extensions there.
You would use those extensions in  providing sequencing for raw GSS-API applications.

    >> channel-binding = *(octet)
    >> 
    >> scram-c2-ctx = nonce "," proof scram-c2-wrap = mac "," 4*%00
    >> channel-binding 4*octet %00 [channel-binding] [authzid]
    >> scram-c2 = gs2-header scram-c2-ctx scram-c2-wrap

    Chris> I am confused about why the channel-binding may appear
    Chris> twice.  Can you explain?

I am too.
I gave you an explanation for this in person which seems wrong looking at the ABNF.
Channel binding should appear once in the wrap part of the message.

Nico, is this just an error?

---

From: Nicolas Williams <[email-masked]>
Date: Wed, 6 Aug 2008 14:48:53 -0500

On Wed, Aug 06, 2008 at 02:45:44PM -0400, Sam Hartman wrote:
>     Chris> [...]

Assuming Crhis' main objection is to the binary/non-textual bits on the
wire, then we could:

 - base64-encode GSS tokens in GS2 and remove the length fields in GS2
 
 and

 - "compress" the standard initial token mechanism OID header, as
   discussed before

Then an all-ASCII ABNF for GS2/SCRAM will be possible.

Chris, Simon, please comment.

Nico
--

---

From: Nicolas Williams <[email-masked]>
Date: Wed, 6 Aug 2008 15:02:56 -0500

> Nico, is this just an error?

The first appearance of it in scram-c2-wrap was meant to be
channel-binding-length.  So yes, it's a typo.

Nico
--

---

From: Sam Hartman <[email-masked]>
Date: Wed, 06 Aug 2008 16:01:15 -0400

>>>>> "Nicolas" == Nicolas Williams <[email-masked]> writes:

    Nicolas> On Wed, Aug 06, 2008 at 02:45:44PM -0400, Sam Hartman
    Nicolas> wrote:
    Chris> [...]
    >>  I think that this argument applies equally well to GS2 as it
    >> does to scram.  We should not have standards for what is good
    >> protocol design that differ between two protocols in the same
    >> working group.  So, if lengths are bad, let's remove them from
    >> GS2.  If lengths are OK, then they should be OK for scram.  I
    >> don't care very much whether we decide lengths are fine or
    >> problematic.

    Nicolas> Assuming Crhis' main objection is to the
    Nicolas> binary/non-textual bits on the wire, then we could:

    Nicolas>  - base64-encode GSS tokens in GS2 and remove the length
    Nicolas> fields in GS2

You probably do not want to base64 encode the GSS-data.  That will get
you ascii, but with multiple layers of base64, which may be as bad as
lengths.  I think you want to have an escaped terminator and avoid as much encoding as possible.

---

From: Simon Josefsson <[email-masked]>
Date: Wed, 06 Aug 2008 22:15:24 +0200

> Chris, Simon, please comment.

Textual protocols are easier to debug and can be easier to implement in
some languages.  One problem is if textual protocol permits complex
rules like DIGEST-MD5.  It will require some work to rewrite GS2 though.

If rewriting GS2 will make SCRAM become a GSS-API mechanism (and used in
SASL under GS2), I'll happily do the GS2 work involved.  But if SCRAM
remains as a SASL mechanism, I don't see the advantage in changing GS2
for this.

/Simon

---

From: Nicolas Williams <[email-masked]>
Date: Wed, 6 Aug 2008 16:08:15 -0500

> remains as a SASL mechanism, I don't see the advantage in changing GS2

Yes, the point is to end up with SCRAM as a GSS-API mechanism only, but
such that SASL implementors can implement SCRAM without having to
implement GS2 and GSS frameworks.

Nico
--

---

From: Nicolas Williams <[email-masked]>
Date: Wed, 6 Aug 2008 16:06:54 -0500

> much encoding as possible.

Multiple layers of base64 encoding don't hurt, though we could certainly
make GS2 avoid it.  Escaping would work.

---

From: Chris Newman <[email-masked]>
Date: Wed, 06 Aug 2008 20:22:11 -0700

> Chris, Simon, please comment.

It's my opinion the amount of data in this context never requires a length 
field and such fields are complex and error prone at this layer (length 
fields are useful for large objects and chunking of large objects).  I 
point to the numerous security bugs in ASN.1, PKIX, and Kerberos libraries 
as evidence that lengths (especially nested lengths) are a security problem 
for design.  While textual protocols that use delimiters have security bugs 
too, they are significantly less frequent in practice in my experience.

For protocol design of small fields like this, I prefer (in order of 
preference):
 * fixed size fields
 * SASL exchanges are self-delimiting (the SASL profile delimits the SASL 
message), so if
   there's only one variable length or binary field at the end of the of 
each exchange,
   that's better.
 * delimited fields where the delimiter can not appear in the field
 * delimited fields where the fields have a regular encoding (base64, 
ascii-hex, etc).
 * delimited fields with escapes (rarely used escapes are more likely to 
create interop
   bugs due to implementation flaws than a regular encoding)

I also prefer textual over binary as it simplifies examples and test 
vectors, but that's secondary to the above priority list.

While I can't speak for my co-author, I am almost certain to support making 
SCRAM a GS2 mechanism is the proposed simplifications to GS2 are made.

		- Chris

---

From: Nicolas Williams <[email-masked]>
Date: Wed, 6 Aug 2008 23:07:23 -0500

> For protocol design of small fields like this, I prefer (in order of 

In GS2 the context and per-msg tokens cannot be fixed size fields.

> * SASL exchanges are self-delimiting (the SASL profile delimits the
>   SASL message), so if there's only one variable length or binary
>   field at the end of the of each exchange, that's better.

In GS2 we have a round-trip optimization that sometimes requires having
two variable-length opaque tokens in one message, so putting one last
doesn't help.

> * delimited fields where the delimiter can not appear in the field

GSS context and per-message tokens are opaque octet strings, therefore
any octet value can appear, therefore there is no suitable delimiter.

> * delimited fields where the fields have a regular encoding (base64,

GS2 could base64-encode GSS tokens.

> * delimited fields with escapes (rarely used escapes are more likely
>   to create interop bugs due to implementation flaws than a regular
>   encoding)

GS2 could escape some octet value (and the escape itself).

Given that SCRAM itself could be all-ASCII (or all-UTF-8) we could make
it so SCRAM implementations not done via GS2 need never do any escaping.

I think that makes escaping the winner :)

And we can always compress the standard initial GSS context token header
as a bonus.  That way a non-GSS SCRAM implementation can also avoid that
binary goo and be truly all-text.

> SCRAM a GS2 mechanism is the proposed simplifications to GS2 are made.

I support such simplifications.

Nico
--

---

From: Sam Hartman <[email-masked]>
Date: Thu, 07 Aug 2008 10:54:59 -0400

Chris, I'd like to start discussing a GS2 encoding.

At a conceptual level, the GS2 layer needs to include two objects in
its challenges and responses.  First, it needs to include a context
token.  Secondly, it needs to include some integrity-protected stuff.
Depending on the state of the exchange, one of these objects may be
absent.

I think there are two basic approaches without lengths.  The first is
to have two base64 blobs.
So, ignoring the OID issue, the encoding might be as simple as 
gs2-mesage = base64 SP base64 
(You might want something more complicated as an initial space may be undesirable to indicate that a context token is absent)

The potential problem with this approach is that it involves a lot of
layers of base64.  For example in IMAP, a SCRAM message would look
like a base64 blob containing two base64 blobs.

Another approach is to use some delimiter.  However the context token
can contain arbitrary binary data.  At least one part of the integrity
protected data can include arbitrary binary data.  In the specific
instance of SCRAM, we can make everything textual or base64.  However
at a general level we'd need to have some escaping mechanism in case
the delimiter appeared in the data.  I'd be a bit concerned though if
SCRAM tended not to use that escaping mechanism that people would get
it wrong.

There may be a third approach that involves using gss_getmic rather
than gss_wrap (thus giving GS2 a bit more control of layout) where the
MIC is base64 and the context token if present goes at the end of the
message.  That might actually work well for SCRAM, although it's
fairly odd to have the context token (which you process first) go at
the end of the message.

I think somewhere in here we're going to need a conference call to
walk through some options.  However I'd like to have a couple rounds
of email to see what some of the constraints are so perhaps we could
draw up some specific options to discuss on such a call.

---

From: Simon Josefsson <[email-masked]>
Date: Thu, 07 Aug 2008 17:20:12 +0200

> SCRAM tended not to use that escaping mechanism that people would get

How about using 0x00 as a delimiter?

If 0x00 occurs in the data to encode, encode that as 0x00 0x00.

Use 0x00 0xFF to denote end of the data.

Using 0x00 rather than other values has the advantage that normal
zero-termination string function would work if the mechanism tokens are
all-ASCII (modulo the initial OID).  A disadvantage would be that binary
protocols tend to contain a lot of 0x00's.

Personally, my preference is to use base64 though, even if it leads to
double or even triple encoding.  It is easier to get right in
implementations, and easier to debug since it is all ASCII.

/Simon

---

From: Sam Hartman <[email-masked]>
Date: Thu, 07 Aug 2008 11:45:31 -0400

>>>>> "Simon" == Simon Josefsson <[email-masked]> writes:

    Simon> Sam Hartman <[email-masked]> writes:
    >> Another approach is to use some delimiter.  However the context
    >> token can contain arbitrary binary data.  At least one part of
    >> the integrity protected data can include arbitrary binary data.
    >> In the specific instance of SCRAM, we can make everything
    >> textual or base64.  However at a general level we'd need to
    >> have some escaping mechanism in case the delimiter appeared in
    >> the data.  I'd be a bit concerned though if SCRAM tended not to
    >> use that escaping mechanism that people would get it wrong.

    Simon> How about using 0x00 as a delimiter?

    Simon> If 0x00 occurs in the data to encode, encode that as 0x00
    Simon> 0x00.

    Simon> Use 0x00 0xFF to denote end of the data.
This is a fine approach if we believe escaped delimiter is the way to
go.

---

From: Nicolas Williams <[email-masked]>
Date: Thu, 7 Aug 2008 11:17:52 -0500

> This is a fine approach if we believe escaped delimiter is the way to

Escaping means that a non-GSS-based SCRAM implementor wouldn't need to
do any escaping if SCRAM is all-ASCII or all-UTF-8.

---

From: Chris Newman <[email-masked]>
Date: Thu, 07 Aug 2008 17:03:50 -0700

--On August 7, 2008 10:54:59 -0400 Sam Hartman <[email-masked]> 
wrote:
> (You might want something more complicated as an initial space may be
> undesirable to indicate that a context token is absent)

I'd use something other than a space as the delimiter for the reason you 
mention, but this is an acceptable approach.

> like a base64 blob containing two base64 blobs.

I don't see this as a problem.  The implementation can likely reuse the 
base64 code and it's not difficult to write an encoder/decoder that 
supports in-place conversion (I'd be glad to share my code if that's 
helpful).  That leaves two places the implementer can screw up security 
(b64 encoder and b64 decoder) and they're both already necessary for most 
SASL profiles.

> SCRAM tended not to use that escaping mechanism that people would get

While I understand the concern, I suspect we only have to worry about two 
classes of implementers: those who want both SCRAM & Kerberos and use a 
GSSAPI framework and those who only want SCRAM.  If we can design this so 
the latter can use delimiters without escaping and the GS2 spec includes 
Kerberos test vectors showing use of escaping, then I think we're fine. 
That may be better than the extra-base64 option.

> fairly odd to have the context token (which you process first) go at

That sounds even better, if the GS2 folks are willing.

> draw up some specific options to discuss on such a call.

Now that we're discussing how to simplify rather than whether to simplify, 
I'm not sure it's controversial enough to require a conference call.

		- Chris

---

From: Nicolas Williams <[email-masked]>
Date: Thu, 7 Aug 2008 20:01:14 -0500

> That sounds even better, if the GS2 folks are willing.

I don't see how that works.  MIC tokens are typically fixed-sized, but
there's no way to know a priori what that size will be, so we still
have two abritriray-size binary blobs in one message.

> I'm not sure it's controversial enough to require a conference call.

My first choice is escaping.

---

From: Sam Hartman <[email-masked]>
Date: Fri, 08 Aug 2008 11:33:31 -0400

>>>>> "Nicolas" == Nicolas Williams <[email-masked]> writes:

    Nicolas> On Thu, Aug 07, 2008 at 05:03:50PM -0700, Chris Newman
    Nicolas> wrote:
    >> >There may be a third approach that involves using gss_getmic
    >> rather >than gss_wrap (thus giving GS2 a bit more control of
    >> layout) where the >MIC is base64 and the context token if
    >> present goes at the end of the >message.  That might actually
    >> work well for SCRAM, although it's >fairly odd to have the
    >> context token (which you process first) go at >the end of the
    >> message.
    >> 

    Nicolas> I don't see how that works.  MIC tokens are typically
    Nicolas> fixed-sized, but there's no way to know a priori what
    Nicolas> that size will be, so we still have two abritriray-size
    Nicolas> binary blobs in one message.

GS2 base64 encodes the MIC token.  SCRAM's GSS-API mechanism uses a
binary MIC token.  GS2 does not base64 encode the context
token. SCRAM's mechanism uses an ASCII context token.

The over-the-wire-result is that GS2-SCRAM has a base64 blob for the
MIC (encoded by GS2) and a ascii context token.  The whole result is
ASCII.  This would mean that Kerberos would have binary context tokens
and a base64 MIC token as part of GS2.  It would also mean that
GSS-SCRAM (non-SASL) would have ASCII context tokens and binary MIC
tokens.

I think this third approach may be playing fast and loose with the
layers a bit much.

I think I can put together ABNF for the three approaches next week
provided that work does not suddenly consume my time.  However the
energy I can dedicate to this will go down at some point: currently my IETF work is unfunded.

--Sam

---

From: Nicolas Williams <[email-masked]>
Date: Fri, 8 Aug 2008 11:00:45 -0500

> token. SCRAM's mechanism uses an ASCII context token.

Oh, you're proposing that GS2 base64 encode the MIC token.  That works
for me.

> I think this third approach may be playing fast and loose with the

Not any more than any of the other solutions.

---

From: Simon Josefsson <[email-masked]>
Date: Sun, 10 Aug 2008 12:47:58 +0200

> Oh, you're proposing that GS2 base64 encode the MIC token.  That works

How is the MIC token delimited from the context token in GS2?  With a
non-base64-alphabetic character such as SP?  That could work.

/Simon

---

From: Nicolas Williams <[email-masked]>
Date: Sun, 10 Aug 2008 22:13:07 -0500

> non-base64-alphabetic character such as SP?  That could work.

That's what I gathered.  So GS2 messages would look like several
all-text fields (maxbuf, min layers, max layers, layers_if_cb, MIC
token) and one binary token following the text (GSS context token).
SASL prepends a length field, thus there's no ambiguity.

I like this, a lot.

Nico
--