Subject: Transport I-D: KEXINIT reserved field needs description

From: Nicolas Williams <[email-masked]>
Date: Mon, 21 Jul 2003 15:17:42 -0700

The transport I-D defines the SSH_MSG_KEXINIT message:

     byte      SSH_MSG_KEXINIT
     byte[16]  cookie (random bytes)
     string    kex_algorithms
     string    server_host_key_algorithms
     string    encryption_algorithms_client_to_server
     string    encryption_algorithms_server_to_client
     string    mac_algorithms_client_to_server
     string    mac_algorithms_server_to_client
     string    compression_algorithms_client_to_server
     string    compression_algorithms_server_to_client
     string    languages_client_to_server
     string    languages_server_to_client
     boolean   first_kex_packet_follows
     uint32    0 (reserved for future extension)

Nothing is said about the semantics of the last field.

Can implementors describe how they handle non-zero values for the last
field of SSH_MSG_KEXINIT?

I propose the following change:

 - Name the last field of KEXINIT "extensions":

     byte      SSH_MSG_KEXINIT
     byte[16]  cookie (random bytes)
     string    kex_algorithms
     string    server_host_key_algorithms
     string    encryption_algorithms_client_to_server
     string    encryption_algorithms_server_to_client
     string    mac_algorithms_client_to_server
     string    mac_algorithms_server_to_client
     string    compression_algorithms_client_to_server
     string    compression_algorithms_server_to_client
     string    languages_client_to_server  
     string    languages_server_to_client
     boolean   first_kex_packet_follows 
     uint32    extensions

 - Describe the 'extensions' field and the semantics of extensions
   negotiation:

      extensions
	 Currently MUST always be set to zero (0).  If set to zero this
	 is the last item in this packet.  Future extensions to
	 SSH_MSG_KEXINIT may be defined by IETF RFCs which specify
	 non-zero values for this field and add fields to the end of the
	 packet.  Receipients MUST ignore any items in SSH_MSG_KEXINIT
	 packets past this field that if they do not know the extensions
	 value.

	 ["Implementations must be careful to include the entire
	   SSH_MSG_KEXINIT packets in the session ID hash, not just the
	   fields they know about." ??]

Cheers,

Nico
--

---

From: Markus Friedl <[email-masked]>
Date: Tue, 22 Jul 2003 11:41:14 +0200

> field of SSH_MSG_KEXINIT?

OpenSSH will accept non-zero values for the reserved uint32 value,
but will not allow more data after this field.

---

From: Joseph Galbraith <[email-masked]>
Date: Tue, 22 Jul 2003 09:00:58 -0600

>field of SSH_MSG_KEXINIT?
Currently, debug builds of VanDyke's implementation ASSERT that the 
value is 0,
but since these builds are not shipped (as a general rule), I believe we 
are okay with
the value being changed.

>	   fields they know about." ??]
I do think we want the sentance clarifying that the entire KEX packet must
be included in session id hash.  Should it say key exchange hash instead?
Since only the first key exchange hash becomes the session id.

I assume we don't want to define the extension format now because of 
last call?

We might be able to use the extensions field as a count field:

     boolean   first_kex_packet_follows 
     uint32    extension_count
     string    extension_name[1]
     string    extension_data[1]
     string    extension_name[2]
     string    extension_data[2]
     string    extension_name[...]
     string    extension_data[...]
     string    extension_name[extension_count]
     string    extension_data[extension_count]

And give extension name the same semantics that is used else where in the
document-- @dns name extendable.

Remember, what is being extended here is algorithm negotiation, not
(as the packet name might have you think) key exchange (well, key exchange
algorithm is part of what is being negotiated.)

So I think a flexible extension mechanism is called for.  I.e., some of 
the things
that might be negotiated here in the future might be:

- hash algorithm for failed kex packets.
- compatibility work arounds (My implementation is vandyke.com, and
  I've fixed bugs 1 & 2; if you were working around this bugs, please
  don't anymore.)
- Capabilities announcement (I support packets > 35K; I support key 
exchange retry; etc., etc.)

I wouldn't want to assume we are smart enough to anticipate all future 
needs, or
that everyone's needs will actually need to be standardized by the 
IETF.  By allowing
extensibility, we prevent those people that have needs not met by the 
IETF from
producing broken, non-complaint implementations.  (I believe that people 
will produce
implementations that meet their needs, whether they can do it and comply
with the RFC or not.)

But this is all probably discussion for a new I-D.

Joseph

---

From: Simon Tatham <[email-masked]>
Date: Tue, 22 Jul 2003 16:03:55 +0100

> field of SSH_MSG_KEXINIT?

PuTTY ignores it, and will also tolerate extra stuff in the packet
after it.

Cheers,
Simon
-- 
Simon Tatham         "I'm going to pull his head off. Ear by ear."
<[email-masked]>                          - a games teacher

---

From: Nicolas Williams <[email-masked]>
Date: Tue, 22 Jul 2003 08:10:08 -0700

> Since only the first key exchange hash becomes the session id.

Hmmm, doesn't matter much if it's "key exchange hash" or "session ID
hash" since re-keying is protected anyways.  But yes, it probably should
say "key exchange hash."

And of course, KEXINIT extensions should apply to re-keying as much as
to the initial kex.

> I assume we don't want to define the extension format now because of 

If we define the semantics of how we extedn KEXINIT we can let the core
drafts progress and describe any KEXINIT extensions in subsequent I-Ds.

> We might be able to use the extensions field as a count field:
[...]
> document-- @dns name extendable.

This is like Joel's named fields proposal and it does sound reasonable
to me, though in this form it does mean that the extensions field is a
one-shot extension device, mitigated by the permanent extensibility of
the named fields.

> algorithm is part of what is being negotiated.)

Yes.

> needs, or that everyone's needs will actually need to be standardized
> by the IETF.

Me neither.  As long as any extension to KEXINIT leaves in extensible we
should be ok.

> By allowing extensibility, we prevent those people that have needs not
> met by the IETF from producing broken, non-complaint implementations.

A very strong argument for Joel's and your named fields proposals.

> But this is all probably discussion for a new I-D.

Indeed.  A small amount of text will fix transport so we can fix KEXINIT
later.  Later is good too - we ought not hurry in this case, as long as
we fix the transport I-D.

Cheers,

Nico
--

---

From: Nicolas Williams <[email-masked]>
Date: Tue, 22 Jul 2003 15:44:42 -0700

> field of SSH_MSG_KEXINIT?

Thanks to all who did.

For those whose implementations ignore additional data past the reserved
field, is that data included in the key exchange hash?

I think the fact that some implementations do not tolerate KEXINIT
"extensions" should not precluse the proposed change.  Several
implementations already map the peer's version string to compatibility
notes and react accordingly.  While I hope that we can put stop adding
to this database of compatibility notes I think that this particular
issue (KEXINIT extensibility) is worth the trouble to fix.

(When the core I-Ds progress I think it might be worthwhile to built a
 registry of compatibility notes.)

Cheers,

Nico
--

---

From: "Paul Swartz" <[email-masked]>
Date: Tue, 22 Jul 2003 18:52:48 -0400

> field, is that data included in the key exchange hash?

Yes, Conch includes the entire packet, regardless of how much of the 
packet it parses.

-p
-- 
     Paul Swartz
(o_  http://twistedmatrix.com/users/z3p.twistd/
//\  [email-masked]
V_/_ AIM: Z3Penguin

---

From: Joseph Galbraith <[email-masked]>
Date: Tue, 22 Jul 2003 17:03:54 -0600

> field, is that data included in the key exchange hash?

VanDykes products use the entire kexinit packet,
including any excess data.

- Joseph

---

From: Markus Friedl <[email-masked]>
Date: Wed, 23 Jul 2003 09:51:49 +0200

> including any excess data.

OpenSSH would do the same if it would accept
badly formated KEXINIT packets.

---

From: Markus Friedl <[email-masked]>
Date: Wed, 23 Jul 2003 09:57:24 +0200

> issue (KEXINIT extensibility) is worth the trouble to fix.

I don't think it's a good idea to break the protocol at
this point.  This compatibility database is always a pain
and you always miss some implementations.

I think you could only assume that the peer is able to deal with
the extension if it sends a non-zero value in the reserved field,
but I doubt this helps for what you want.  On the other hand, you
could send the extension data in an extra packet if the reserved byte
is not zero.

---

From: Simon Tatham <[email-masked]>
Date: Wed, 23 Jul 2003 09:20:03 +0100

> field, is that data included in the key exchange hash?

At the risk of making a barely contentful `M33 T00!!!1!!' post ...
yes, PuTTY hashes the whole incoming packet, whether it recognises
it all or not.
-- 
Simon Tatham         "I thought I'd put my foot so far into my mouth I
<[email-masked]>    wouldn't be able to sit down without standing up."

---

From: Jon Bright <[email-masked]>
Date: Wed, 23 Jul 2003 14:12:45 +0200

> Yes, Conch includes the entire packet, regardless of how much of the 

PenguiNet too.

--
Jon Bright
Lead Programmer, Silicon Circus Ltd.
http://www.siliconcircus.com

---

From: Nicolas Williams <[email-masked]>
Date: Wed, 23 Jul 2003 08:00:37 -0700

> could send the extension data in an extra packet if the reserved byte

Of all the things in the compat db, this would be amongst the most
important things to have in there.

So if a peer has a version string that according to the compat db means
that the peer does not support KEXINIT non-zero reserved field, then you
send zero in your KEXINIT's reserved field and no extra data in that
packet.  Otherwise you are free to use KEXINIT extensions.

Cheers,

Nico
--

---

From: Jon Bright <[email-masked]>
Date: Wed, 23 Jul 2003 19:28:50 +0200

> could send the extension data in an extra packet if the reserved byte

If you wanted to get really hacky about it, you could put the additional 
data in the padding part of the SSH packet.  This restricts you to 251 
bytes of data (255 bytes max padding - 4 bytes of real padding) but 
should retain compatibility with all current clients.  Of course, it's 
also a truly unpleasant hack and probably messy to implement for 
everyone concerned.  But it's possible...

Overall, my personal opinion (FWIW) is:

a) using an extra packet would be best, and
b) implementations should ignore additional data within packets, or at 
least within packets with reserved fields allowing for extensions.  Not 
ignoring additional data basically means reserved fields are useless, 
since in every case the only decent solution will be a) above, and if 
that was the desired extensibility mechanism, one could just rely on 
sending new packet types, receiving SSH_MSG_UNIMPLEMENTED from old 
clients and then sending the old packet type in order to extend things.

I'm curious as to why OpenSSH does refuse extra packet data - this goes 
against the general liberal-in-what-you-receive principle, which makes 
me interested in the thinking behind doing things this way.

---

From: Nicolas Williams <[email-masked]>
Date: Wed, 23 Jul 2003 10:31:53 -0700

> clients and then sending the old packet type in order to extend things.

I believe not all implementations reply with SSH_MSG_UNIMPLEMENTED
before userauth or before userauth completes.

To avoid another avalanche of "me toos" :) I will ask for such
information to be sent to me which then I'll summarize.

Nico
--

---

From: Nicolas Williams <[email-masked]>
Date: Wed, 23 Jul 2003 10:38:09 -0700

> a) using an extra packet would be best, and

This would require the use of extra packets after kex to verify that
one's peer truly did not support the new packets.

---

From: Jon Bright <[email-masked]>
Date: Wed, 23 Jul 2003 19:46:21 +0200

> one's peer truly did not support the new packets.

Sorry, on this occasion, I meant an extra packet sent after receiving 
the other side's key if the reserved field from the remote side is 
non-zero (as Markus suggested).  It's an extra round-trip, but should be 
fairly foolproof.  Basically, both sides, upon receiving non-zero from 
the other side, embark on new-and-shiny-kex and forget about whatever 
they would have done with current-kex.

---

From: Markus Friedl <[email-masked]>
Date: Thu, 24 Jul 2003 09:51:29 +0200

> important things to have in there.

yes, that's why it's dangerous and it's likely that you miss versions
strings. you should not depend on this and break the protocol this late.

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Wed, 23 Jul 2003 07:19:12 -0400

> could send the extension data in an extra packet if the reserved byte

Actually, I think that would work.

Based on what people have said about what implementations do, I will 
suggest that the core transport draft specify the following behavior:

- MUST send 0 in the KEXINIT reserved field
- MUST ignored the received value in the KEXINIT reserved field
- MUST NOT add any fields after the reserved field
- MAY reject KEXINIT packets with extra fields

At which point we can (in a later document) use the reserved field to 
indicate support for extensions, which are then negotiated in a separate 
exchange of packets after the KEXINIT messages have been received.

-- Jeffrey T. Hutzelman (N3NHS) <[email-masked]>
   Sr. Research Systems Programmer
   School of Computer Science - Research Computing Facility
   Carnegie Mellon University - Pittsburgh, PA

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Wed, 23 Jul 2003 07:19:12 -0400

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Thu, 24 Jul 2003 13:39:08 -0400

> strings. you should not depend on this and break the protocol this late.

I have to agree with Markus here.  Implementations may wish to maintain 
information about noncompliant implementations, but that should never be 
required or even suggested as a way to implement the protocol correctly.

Given a choice between "do X unless the peer is broken, in which case we do 
Y, which works with all peers" and "always do Y, which works with all 
peers", the first is only acceptable if there is a pretty compelling reason 
why you'd want to do X.

In the present case, there is a clear way to specify the extension such 
that a correct implementation will work with existing implementations that 
do not support the extension, regardless of whether they can handle extra 
fields in the KEXINIT packet -- don't send any such fields, and trade a 
second set of messages instead.

Of course, we do still need to update the transport document to specify 
that its implementations MUST NOT consider the contents of the reserved 
field in deciding whether a KEXINIT packet is valid, but MUST include its 
actual contents in the hash anyway.  Fortunately, it sounds like all the 
existing implementations behave this way anyway.

-- Jeff