Subject: S/MIME Developer's Workshop #5 (SDW5)
From: Steve Dusse <[email-masked]>
To: "[email-masked]'" <[email-masked]>, "[email-masked]'" <[email-masked]>
Date: Wed, 20 Aug 1997 16:59:22 -0700

RSA Data Security, Inc. will be hosting the fifth S/MIME Developer's
Workshop on September 10th, 1997 at the Airport Marriot in Burlingame,
California.  (Directions/details to follow in a separate message.)  This
meeting is open to all interested, please feel free to forward this
message to appropriate individuals/groups.

The meeting will nominally go from 9am to 4pm.  Actual agenda will
follow sometime in the next week or so.

The significant topics to be discussed are:

- technical feedback on the current Internet Drafts
- getting gateways to DO THE RIGHT THING !!!
- progressing S/MIME within the IETF
- S/MIME compliance and interoperability test plans
- introduction of new features for next version of S/MIME

The last topic on new features will include some information about RSA
Labs revision of the underlying PKCS #7 standard as well as some
exciting work that is underway to evaluate the merging of S/MIME and MSP
(the Message Security Protocol used for government messaging).

If there is time in the afternoon, we may have a couple of
vendor/product presentations. 

If there are other topics of interest to the S/MIME developer community,
feel free to email to the list.

Cheers,
Steve Dusse
Chief Troublemaker
RSA

From: Blake Ramsdell <[email-masked]>
To: "[email-masked]'" <[email-masked]>, "[email-masked]'" <[email-masked]>
Subject: What would you do for S/MIME gateway support language?
Date: Fri, 8 Aug 1997 01:03:29 -0700
Sender: [email-masked]

Ned, I've taken a little bit of time to mull this over, and I guess I
need to ask this question:

What gateway-specific language would you put into S/MIME to satisfy the
concerns that you currently have?

Blake
--
Blake C. Ramsdell
Deming Internet Security, a Worldtalk Company
For current info, check http://www.deming.com/users/blaker
Voice +1 425 882 8861 x103  Fax +1 425 882 8060

Date: Tue, 05 Aug 1997 13:49:22 +0200
From: Ludwig Wiechers <[email-masked]>
Organization: ASW BA SC
To: [email-masked]
Sender: [email-masked]

This is a cryptographically signed message in MIME format.

--------------msBAFE7D0CF9AD5BBE129B7D7B
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

SUBSCRIBE

--------------msBAFE7D0CF9AD5BBE129B7D7B
Content-Type: application/x-pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature

kdOr
--------------msBAFE7D0CF9AD5BBE129B7D7B--

Date: Sun, 03 Aug 1997 23:03:25 -0700
To: Steve Dusse <[email-masked]>
From: Dave Crocker / IMC <[email-masked]>
Subject: RE: application/signed vs. application/mime vs. signedData
Cc: "[email-masked]'" <[email-masked]>
Sender: [email-masked]

At 10:39 PM 8/3/97 -0700, Steve Dusse wrote:
>behind sucky gateways.  signedData always works.  multipart/signed does
>not.
>
>Nuff said ?

	Well, Steve, no.

	Orchestrated tests have a pretty high success rate, but little to do with
reality.  In any event, I don't recall anyone saying that gateways lack
problems with multipart objects that need to stay together.

	What your line of argument is forgetting is the reason there was so much
pressure for use of Multipart/signed.

d/
--------------------
Dave Crocker
Internet Mail Consortium                               +1 408 246 8253
675 Spruce Dr.                                    fax: +1 408 249 6205
Sunnyvale, CA 94086 USA              [email-masked] , http://www.imc.org

Member, interim Policy Oversight Committee     http://www.gtld-mou.org

To: "[email-masked]'" <[email-masked]>
Subject: Re: application/signed vs. application/mime vs. signedData 
Date: Mon, 04 Aug 1997 11:50:08 +0200
From: "Frederik H. Andersen" <[email-masked]>
Sender: [email-masked]

--==_Exmh_-135972864P
Content-Type: text/plain; charset=us-ascii

[email-masked] said:
> We have once again achieved consensus:
>
> - multipart/signed is great.
> - Lot's of gateways suck because they screw up multipart/signed.

How does these "gateways" handle other multipart types?

> - Email signatures using multipart/signed simply won't work until sucky
>   gateways are fixed. 

If we fx. take multipart/alternative with 2 parts, will these 2 parts
be fu^h^hchanged by said gateways? 

If these gateways are not able to deliver the original content of any
multipart/* parts, how can these gateways stay in the market?

With "original content" I mean the content protected by the MIME transfer
protection mechanisme, which (AFAIK) takes a good deal of consideration
towards protection the original content against gateways.

-- 

===============================================    |   |      ====
===============================================   _|  _|  _   ====
===============================================  / | / | /_\  ====
===============================================  \_| \_| \_.  ====    

Frederik H. Andersen,  M.Sc.
Senior Software Engineer
Internet Development

Dansk Data Elektronik A/S                 Phone:   +45 42 84 50 11  
Herlev Hovedgade 199                      Fax:     +45 42 84 52 20
DK-2730 Herlev, Denmark                   Web:   http://www.dde.dk

Email:  [email-masked] (MIME accepted)
PGP Fingerprint:  6B BC FB 45 E4 69 CE A2  63 E1 62 1F 0C 65 C2 E4

==================================================================

--==_Exmh_-135972864P
Content-Type: application/pgp-signature

--==_Exmh_-135972864P--

From: Steve Dusse <[email-masked]>
To: "[email-masked]'" <[email-masked]>
Subject: RE: application/signed vs. application/mime vs. signedData
Date: Sun, 3 Aug 1997 22:39:04 -0700
Sender: [email-masked]

Wow !!  My head is starting to hurt.  Deja vu.  I distinctly remember
the last time this thread was discussed and how frustrated everyone
got...

We have once again achieved consensus:

- multipart/signed is great.
- Lot's of gateways suck because they screw up multipart/signed.
- Email signatures using multipart/signed simply won't work until sucky
gateways are fixed.

Now let's convert some of this energy into writing the specifications
for what gateways should and should not do.  If the IETF is unwilling or
unable to work on this problem then IMHO they are not being responsive
to their constituency and we should work on the problem without them.

Hmmm.  What to do in the mean time ?  Wait until the gateways are fixed
or implement a stopgap ?  I could be wrong, however, I think the
consensus is (still) to implement a stopgap.  (If there are any other
proponents of waiting until gateways are fixed to have email security,
please speak up now !!)

Yup, opacity of signed messages pisses a lot of people off.
Nonetheless, signedData is a stopgap that works.  This is not
speculation.  S/MIME signedData was recently mandated in a successful
EDI-over-the-internet pilot by major EDI vendors.  multipart/signed
didn't work.

Some of our customers' customers have implemented a security policy
mandating that _all_ email sent between them and their
suppliers/contractors/remote offices/etc. be signed.  Many of them sit
behind sucky gateways.  signedData always works.  multipart/signed does
not.

Nuff said ?

The issue on the table is whether to design some other stopgap measure
to (eventually) replace signedData.  If there is a compelling
alternative, let's give it the attention it deserves.  The longer we
wait, the harder it will get to make the transition.

-steve	
>

Date: Sun, 03 Aug 1997 08:44:27 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: RE: application/signed vs. application/mime vs. signedData
To: Blake Ramsdell <[email-masked]>
Cc: "'Ned Freed'" <[email-masked]>, "[email-masked]'" <[email-masked]>
Sender: [email-masked]

> > Whether or not multipart/signed is recent or not is irrelevant. It didn't
> > contain any language about gateway conformance. As such, it was simply ignored
> > by the customers seeking compliant gateways and hence it was also igored by
> > the gateway development community.

> Right, and now that at least three secure mail specifications are using
> it, customers will start asking for it, and gateways will start
> implementing it.  I don't see how gateway-specific language would change
> the adoption rate.  The market demands the change, which is what you're
> pointing out.

My experience says otherwise. I've run into many cases where gateways don't
handle various things properly, including but not limited to multipart/signed,
and when taxed with this by customers the gateway vendor simply said "show us
the standard that says we have to do this". Long silence, followed by an
admission that the functionality isn't required, at which point the vendor says
"we'll put it on our may implement someday list, nice talking to you". And the
person who wants the functionality, who happens not to be the ones who operate
the gateway, says "that's unacceptable, change the gateway". But the people
running the gateway say "sorry, since functionality you're asking for is a nice
to have feature, not a mandatory one, we're not interested in changing gateway
products to get it".

I've seen this scenario played out countless times in the marketplace. As I
said, I have 15 years of experience in the gateway market, so I think I know
what I'm talking about.

The problem here, of course, is a chicken and egg one. If there was serious
market pressure to deploy security services across gateways the vendors would
of course change their practices. But such pressure only arises _after_ such
services are deployed to some extent. And we cannot get deployment in place
without first getting some amount of deployment of security-capable gateways.
It's a loop, and the only way I know of to break it is through the standards
process. I originally got started doing standards because I realized this, as a
matter of fact.

The past five years of near-total non-deployment of security services
illustrates this point rather nicely. And do not confuse pressure in the market
for security services with pressure for secure end-to-end email. The market
isn't that aware of the issues. So what we're seeing deployed isn't end-to-end
services, it is stuff like encrypted tunnels and (soon) SSL/TLS secured email.
And no, from a security standpoint these solutions are inferior. But the market
doesn't see it that way. If we're to have any chance of getting end-to-end
security for email widely deployed we need to address the issues gateways bring
up.

Now, if I thought that use of application/[s]mime would enable enough
deployment to create the market and break us out of loop we're in I would
support it a lot more than I do. But I don't think it is sufficient to have
this effect. What we really need is to define gateway compliance. I believe
this will get us enough deployment of capable gateways that we'll then be
able to create the critical mass to change the rest of them.

> And why not put in client-specific language also?  Or X-specific
> language, where X is some other faction of the overall Internet
> community?

Because we already have client-specific language. The language in the
multipart/signed document specifically deals with what a conformant client must
do. So if I build clients I might actually care about this. Now, I happen to
think the language is inadequate in several ways (in particular, it doesn't
deal with the case where the verifier is separate from the client and what the
security concerns in this case are), but what's there is all we could get
through the process. We used to have more specific and detailed requirements
but the WG had us remove them.

> I don't understand why the consideration is to the gateway,
> and yet the client is not important to consider also.

The client is equally if not more important. But we've addressed the client
already, in detail. And we haven't said a single, solitary word _anywhere_
about gateways.

> Or the international concerns.  Or whatever.  The reason why gateway-specific
> issues can get passed over is because they are specific to some faction,
> regardless of its size.

Blake, you lack the experience in the process to make such assessments. And you
are simply wrong about this: The reasons gateway issues get passed over in the
IETF are complex, but they do not include the notion that they are specific to
a particular faction. In fact I would argue that gateways are by definition not
a single single faction -- this is part  of the problem dealing with them. But
in any case, dealing with faction-specific issues in the IETF all the time and
in fact have no problem in doing so.

But even if this was true, so what? The point is that this WG has identified a
problem that manifests most prevalently in gateways. But rather than try and
solve the problem, the group seems intent on building more infrastructure that
_will_ _not_ _solve_ _the_ _problem_. Why not simply deal with the problem?

This of course begs the question of why the IETF hasn't dealt with gateways at
all. One of the primary reasons is obvious: Gateways by their very nature
restrict services to the intersection of the capabilities of the environments
they interconnect. This is unpleasant, and to a bunch of engineers who work
very hard to develop services in a particular arena, this is in effect and
attack on what they do, and one which they regard as evil. (I've lost track of
the times I've heard IETFers speak of gateways as evil.) And the obvious
solution is to avoid them, on the principle of "lie down with dogs get up with
fleas", and to instead stick to our little mostly-standards-compliant world of
IP/TCP/SMTP/MIME or whatever.

For example, Donald Eastlake has been trying literally for years to get a
specification covering the issues that arise when security services are
provided at an intermediate point rather than an endpoint. I chaired one such
session, and I quickly learned that the G-word would bring us a veritable
hailstorm of protest. The amount of fancy footwork I had to do as a result
literally left me gasping.

> As soon as you introduce faction-specific
> language, you are inviting every other faction to participate in the
> same process.

Again, so what? What faction are you worried about? The fact of the matter
is that I would _welcome_ the opportunity to address additional issues that
cause interoperability problems.

> You and I are both capable of writing informational RFCs that document
> the support that some faction should have for some RFC.  Anything else
> written by an original author of an RFC should carry as much weight as
> the original RFC, but I might be mistaken.

I think you are mistaken, unfortunately. There are people out there who don't
pay attention to an RFC's standard's status. But when a vendor is asked to
implement something in an RFC they absolutely do pay attention, because they
have lots of things to do and they have to prioritize. And implementing the
random raving in an informational RFC gets low priority. And an informational
document will be ignored.

> I certainly associate a
> certain brand awareness with Dave Crocker's work, Nathaniel Borenstein's
> work and your work, so anything that these people write carries with it
> the expertise that I expect.

I have not noticed this phenomenon to any significant degree. And I think
I should be the one who is able to tell... Not that it doesn't sound nice...

> > If there's a significant number of gateways that support MIME but have now been
> > abandoned I am unaware of their existance. And I deal with gateways at
> > thousands of customer sites on a dailly basis. If you have specific examples
> > I've love to hear about them, but until I see a list of such gateways I'm not
> > inclined to believe in their existance, much less their widespread existance.

> I'm referring to the AOL, MCI, CompuServe and other online service
> gateways.

I have first-hand knowledge of one of these and second hand knowledge of the
the other two. And that knowledge says you're wrong about one of them (I am
under nondisclosure so I cannot say which one) and that it will in fact be
transitioning to software that handles multipart/signed correctly, largely
because of standards compliance issues. As for the other two, while my
knowledge is second-hand, I have found both of them to be quite responsive to
compliance to standards-track IETF documents, and I suspect that they would
rapidly deploy a compliant solution if there was any standard to conform to.

As such, I know you are wrong in your assessment here in one case and believe
you are wrong in the other two cases.

> "Abandoned" is too strong a word, and I apologize for its
> use.  Let's say "conservatively modified".  They cannot handle
> multipart/signed right now, since they treat multipart/signed as
> multipart/mixed, and the original MIME data is lost on the way to the
> client.  These gateways are not significant in number, but are
> significant in customer base.

Far from being abandoned, these are all gateways in active development. And if
the standards said that gateways need to provide the option to tunnel
multipart/signed I know one of them is already on this course and I believe the
other two would be as well if there was a standard to back it up.

> You cannot make a sweeping generalization about an entire market and
> claim that because of your experience that no one in that market will
> ever care about something, or that you have enough of a pulse that you
> can say definitively without hesitation that no one cares about that
> thing now.  We have dozens of customers that are lining up and buying
> our gateway product *EXPLICITLY* because of our consideration for S/MIME
> digital signatures.

I never said that nobody cared. Of course some people care, and I'm sure you
have customers lining up and so on.  I also have customers who care about this
stuff, for that matter. I would probably have imlemented proper handling of
multipart/signed in our products without them, but it was nice to have a
business justification for it.

But this is beside the point. A standard for secure email isn't successful if a
few people line up to buy into it. It is only successful if it is widely
deployed. And what I was saying is I have a nontrivial number of customers who
not only aren't jumping on the signedData bandwagon, they will be actively
hostile to the signedData approach and will go out of their way to deploy
solutions that wreck it. Not only do I have such customers, I have a lot more
of them than I have that fall into the "security first, readability second"
camp.

> Please, be very, very careful about your adamancy.  I qualified my
> statement as being my opinion, yet yours went so far as to put two
> qualifiers to emphasize how wrong I was.  The customers you talk to
> don't care, the customers we talk to do.  I consider at least some of
> them to be in the same market.

The problem is that you only talk to the customers who are at least partly sold
on end-to-end security as the right thing to do. But this entire effort depends
on the willingness of those who are not sold on your program to come to the
table as well. The nature of my business, however, is that I talk to every type
of gateway user you could imagine (and probably some nobody could imagine -- I
certainly couldn't have dreamt them up). This is why I claim that my
perspective is better than yours on this issue.

> No one said that it would require a superhuman level of intelligence to
> implement the ripping of the MIME data out of signedData (I'm a Pretty
> Smart Guy myself, and given something with similar complexity to PKCS
> #7, I could probably figure out how to get the MIME out if it too).  The
> point is that when you *do* implement it, you have identified it as PKCS
> #7, and then gotten the PKCS #7 spec.  And then you have determined that
> it is digitally signed data.  And then you have ripped it apart without
> thinking "gee, maybe someone had a reason for sending this, and I should
> incorporate some new level of intelligence in my gateway product for
> dealing with data of this type, since the universe is now changing to
> start using digital signatures".

I don't get paid to think this way. I get paid to solve problems for gateway
users. And if signedData presents a problem I will solve it in the fashion the
customer wants. And while I might be able to stave off a few of them with a
standard that says "gateways should not do this", without one I don't have a
leg to stand on.

And I'll tell you something else. There's a phenomenon here that I simply do
not understand, yet I've observed time and again to be true. The customers who
want proper handling of security services seem to have a significant degree of
overlap with the ones who cannot change out a gateway that doesn't handle
multipart/signed properly for one that does. But the customers who want things
to "just work", even if it compromising security services, are the ones who
will drop us as a vendor in a heartbeat even if what we're asked to do utterly
compromises security. I don't know why this alignment occurs, but it does.

> And when you do strip off the signature, we have a phone also.  And the
> customer on that phone will say "what happened to our signatures" and
> we'll say "some gateways like to yank apart our signedData messages
> which trashes the signatures", and they'll say "but it's because we want
> to scan for viruses" and I'll say "they can scan for viruses and do
> other things, and in the event that no content needs to be modified,
> they can send the original message through.  However if they need to
> modify the content such as removing a virus, then by definition you
> cannot have valid digital signatures, so the signature is useless
> anyway.  What kind of a gateway would remove the digital signature when
> they didn't perform any content modification?"  And when they get mad
> because they can't have virus scanning and digital signatures at the
> same time, I'll say "but of course you can if you buy our gateway
> product that does both!"

You're arguing something I never said (or at least never intended to
say). I brought up the point about virus scanning in response to your stated
intention to use encryption as a means to thwart gateway translations. In
looking back over my message I see I associated it with multipart/signed as
well. BUt that's incorrect -- I meant to associate it with encryption, not
signature. A gateway cannot take apart an encrypted object to scan for viruses
without the keys, and many companies have policies against using encryption
because of this. And this makes your approach of using encrypted data a
nonstarter for many people.

As a matter of fact our gateway can scan multipart/signed for viruses. And
multipart/encrypted for that matter if the necessary decryption software and
keys are available.

> You are saying that your customers are more important than our customers
> -- that the recipient mail gateway customer is more important than the
> sender mail client customer.  This is subjective, and I suspect that it
> isn't a good point to discuss, since it is futile.  I appreciate your
> loyalty to your customers, as I hope you appreciate mine.

This is not what I'm saying at all. I'm saying that the success of a secure
email solution depends on widespread buy-in, the existance of a significant
faction in the industy that will oppose a particular solution is a compelling
reason not to adopt that solution, and that my experience indicates that
such a group does in fact exist.

Nor do I claim that a standard for gateway behavior will solve the problem
completely. It won't. But make no mistake about it: It does provide a very
powerful tool and it _will_ change the shape of the marketplace significantly.

> > The one difference between Innosoft and another gateway vendor is that we'll
> > take care to implement the right sorts of tunneling options as well as
> > signature removal options. Other vendors will not do this.

> I'm glad you do, and as a scumbag client that always tries to thwart
> you, I thank you.  The other vendors that you refer to don't do this --
> at their own peril (the suicide that I mentioned).  These vendors are
> part of the market that we can sell our gateway to, I will enjoy
> participating in it, and I thank them for creating the market.

I've been trying to sell things this way for years. Sometimes it works and
sometimes it doesn't. But it works a heck of a lot better if there's a standard
to back it up with.

You seem to be arguing that such a standard will have some sort of harmful
effect. But you haven't given specifics as to what the harmful effect will be
other than to say that dealing with the "gateway faction" will cause us to be
deluged with work from other, unspecified factions.

> Once again, "Standards Track" is just as good as "STD" for me.  Is that
> what you mean by telling them that they have to implement it?  The fact
> that the word "gateway" doesn't appear in the document doesn't have
> anything to do with it, in my opinion.

On the contrary, it has everything to do with it. People have little check
boxes for standards compliance and will buy software on the basis of what boxes
get checked. And a gateway can claim full compliance to the security multiparts
standard and not handle multipart/signed correctly. And if you take issue with
them and say they were lying (actually this will never happen because the
customer just bought the other product and not yours and you don't know why),
they will counter by saying "show us the place that the specification says
gateways are supposed to do such and such".  And you won't be able to.

Creative reading of standards is real problem, and it is our job to make it as
hard to do as we can.

> If you are implementing MIME
> stuff, then you must follow, understand, and accommodate the RFCs that
> deal with MIME stuff.  Whether you are HTTP, a gateway, a mail client, a
> mail server, or a MIME filesystem (sorry, ran out of examples), you must
> understand and potentially interpret these documents in a way consistent
> with your customer's goals, and consistent with the philosophy behind
> those RFCs.  multipart/alternative is *completely* different than
> multipart/mixed which is *completely* different than multipart/signed,
> and everyone damn well better understand that.

All this "truth, justice, mom and apple pie" stuff sure is nice, but I'm sorry
to say that I don't find very much of it out in the real world.

> I think what we're talking about is the slow adoption of a particular
> standard.

No, the adoption by gateway vendors has for all intents and purposes been
instantaneous. This is because security multiparts imposed absolutely no
additional implementation burdeon on them at all. And this is the real
problem -- it should have.

> > The problem with RFC 1847 isn't that it isn't far enough along on the standards
> > track, it is that it doesn't impose any requirements on gateways. In fact the
> > word "gateway" appears nowhere in the document. Nor does the word "tunnel". (I
> > believe both of these words used to appear but the entire discussion got
> > removed because that's what the PEM WG wanted.)

> I'm upset that the concept was removed.  I believe that you are correct
> that the gateway tunneling should be considered, since I believe that we
> are trying to solve that very problem with signedData, application/mime,
> and application/signed.  If I were around for that discussion (before my
> time, alas), then I would have participated in favor of a tunneling
> mechanism.

I sure could have used your voice at the time. At the time I was the only one
saying this. (Jim Galvin, Steve Crocker, and the rest of the TIS crew also said
it but he came into this game somewhat late and by the time he was involved it
was mostly over.)

> > There is no such text in RFC1847. The text that is there talks specifically
> > about message transfer agents not altering the content. But an MTA isn't a
> > gateway. They are different things. As such, this text cannot possibly be read
> > to impose any sort of restriction on a gateway. A gateway can in fact claim
> > full compliance with RFC1847 and do whatever it likes with multipart/signed.

> So what you're saying is that because RFC1847 does not explicitly call
> out the fact that the content and the signature must be passed intact
> through the gateway, that's why gateways don't do it?

Close. I'm saying this is why gateways haven't had to do it. They originally
didn't do it because most of them were in development before mulipart/signed
even existed. But when it came out they looked and found nothing new to
implement.

> It seems to me that the interpretation really needs to be left up to the
> gateway.  As you have pointed out before, some users behind a gateway
> couldn't care less about the signature, and want the content.  As I have
> pointed out, some users behind a gateway care explicitly about the
> signature.  This is a semantic that must be enforced by the gateway, and
> is specific to the market in which the gateway vendor wishes to
> participate.  This can't be dictated in faction-specific language in the
> RFC, can it?  

Sure it can. A specification can say that a gateway may be called upon to
provide access to content for security-incapable clients as its primary goal or
security tunneling as its primary goal, that you cannot have both, and as such
a compliant gateway MUST implement both options, and SHOULD allow option choice
on  a per-destination basis.

Specifying that certain options be implemented is done routinely in IETF
standards. There is simply no problem with having such language in a standard
and writing it is easy.

> How will the customer be served by that?

The text I've suggested above would be the single best thing that has happened
to gateway users in a standards document in years.

> And these provisions that you speak of for dealing with MIME at the
> gateway should provide for the basis for your informational RFC.  I
> suspect that it cannot be standards track, because it tarnishes The
> Vision, and will prolong the amount of time that it takes for us to
> reach it.

If I thought an informational document would work I would have written one
years ago. My assessement is, and continues to be, that only a standards track
document will work, that this vision stuff you talk about is largely
irrelevant, and that the only real problem with getting such a document through
the process will be: the knee-jerk response to talking about gateways in a
standard.

A document which talks about removing encryption at the gateway, on the other
hand, would be problematic, because such removal obviously compromises
security. (I nevertheless think such a document should in fact be written, but
I doubt if it stands any chance of making it through the process if it is.)
Removing a signature on the other hand, doesn't compromise anything per se --
it may compromise functionality, but not security.

> To a certain extent, I think you are beating yourself up because you
> have not been able to change human / business nature.  When push comes
> to shove, someone can simply ignore or misread a document that says how
> you should do something.  You can document something until you are blue
> in the face, and it still won't get implemented right.

I'm not beating myself up on this issue at all. What I am doing is beating on
the IETF for having failed to deal with this issue and continuing to think that
more messing around with solutions like application/[s]mime and signedData will
accomplish something. My assessment is that it won't, and as such I do not
support the development of such things. MInd you, I don't oppose them either
(actually I do oppose signedData but as its already basically a nonstarter my
support or lack of it is irrelevant) since I think they're mostly harmless.

				Ned

To: [email-masked]
Path: news
From: John Gardiner Myers <[email-masked]>
Newsgroups: mcom.list.ietf-smime
Subject: Do we need an opaque signature type?
Date: Fri, 01 Aug 1997 16:48:17 -0700
Organization: Netscape Communications Corporation
Lines: 25
NNTP-Posting-Host: 205.217.229.208
Sender: [email-masked]

Ned Freed wrote:
> So the thing to do is to come up with standards that say gateways
> MUST implement the option to tunnel such material or not be compliant with
> Internet specifications, not because some sender is whining about their
> signature being busted, but because it is the best interests of recipients to
> allow this option.

Given this requiremnent of gateways, I then see some need for a MIME
type which gateways can use to tunnel such material.

I absolutely agree that it is not for the sender to insist on the
criticality of their signature.  They cannot even insist the receiver
trust the CA in the sender's certificate, much less know who the sender
is.

It is the purview of the receiver to assign importance to a signature,
and it is reasonable to require them to deploy infrastructure (possibly
edge gateways which can tunnel) to preserve what it is they find
important to preserve.  The party which has the most direct need for a
capability is the one most likely to put resources into whatever is
necessary to provide it.

And it is an unfortunate property of workarounds that they tend to
reduce or eliminate the desire for people to deploy a "real" fix.

To: [email-masked]
Path: news
From: John Gardiner Myers <[email-masked]>
Newsgroups: mcom.list.ietf-smime
Subject: Re: application/signed vs. application/mime vs. signedData
Date: Fri, 01 Aug 1997 16:07:26 -0700
Organization: Netscape Communications Corporation
Lines: 30
NNTP-Posting-Host: 205.217.229.208
Sender: [email-masked]

[For this message only, I'm ignoring Ned's issue of whether we need a
signature preservation type at all.]

Blake Ramsdell wrote:
> For signature preservation, three proposals exist.

Actually, five proposals exist.  Three of them are named
application/mime.

2a. There is the application/mime proposal as it existed three weeks
ago, where the application/mime contents may be transfer-encoded.  This
type appears to violate (or amend) the recursive encoding restriction of
MIME.

2b. There is the application/mime proposal which may not be
transfer-encoded.  This proposal is better named message/mime-entity. 
This type may contain an 8bit or binary object, but is not capable of
tunneling it through a transport which is not capable of handling 8bit
or binary data, respectively.

2c. Same as 3b, but the contained object is prohibited from having 8bit
or binary data at all.

> 3. application/signed.  Less generic, but still interesting to
> intermediate processors.  Pretty much unreadable by any current mail
> agent.  MIME tools exist that can process the data.

Like the current incarnation of application/pkcs7-mime, this requires
the MIME parser which processes the resulting data be capable of
handling binary MIME.

Date: Fri, 01 Aug 1997 13:32:02 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: RE: application/signed vs. application/mime vs. signedData
To: Blake Ramsdell <[email-masked]>
Cc: "'Ned Freed'" <[email-masked]>, "[email-masked]'" <[email-masked]>
Sender: [email-masked]

> (Sorry about the big quote).  Ned, I can't sit by and not have secure
> email until the gateway vendors evolve their products to accommodate
> this recent (multipart/signed is two years ago -- not so recent)
> development.

Whether or not multipart/signed is recent or not is irrelevant. It didn't
contain any language about gateway conformance. As such, it was simply ignored
by the customers seeking compliant gateways and hence it was also igored by the
gateway development community.

  There are some gateways that are simply not in active
> development, attached to networks that are simply too large to roll out
> a new version because this faction of the Internet community would like
> it to happen.  I would even argue that there are some gateways that will
> stay exactly the way they are until they are thrown out and replaced --
> they will never be modified.

Sure, there's a big pile of legacy gateways out there. Microsoft Mail's SMTP
gateway, for example. The old cc:Mail gateway pile that produces seriously
incomliant messages. And so on.

But these gateways are not a problem in practice, for a very simple reason:
They were abandoned before MIME came out. As such, they don't delve into MIME
at all, much less multipart/signed. Tunneling is in fact the default with this
junk.

If there's a significant number of gateways that support MIME but have now been
abandoned I am unaware of their existance. And I deal with gateways at
thousands of customer sites on a dailly basis. If you have specific examples
I've love to hear about them, but until I see a list of such gateways I'm not
inclined to believe in their existance, much less their widespread existance.

> As far as a gateway tunneling into signedData, I think at that point it
> is really over the edge.  The gateway has gone to the point where it
> would implement PKCS #7 unwrapping code, and it has deliberately not
> stopped to consider the consequences of modifying signed data.  This is
> generally destructive to the use of digital signatures, and is in my
> opinion a suicidal move in the face of their increasing popularity.

I work for a gateway vendor and I have over 15 years of experience in this
area. I routine deal with support issues at literally thousands of different
sites that use our products. And I think this gives me the right to say the
following: You are simply, completely, and totally wrong about this. Far from
being suicidal to implement such code, it would be suicidal, in the event that
use of signedData by itself becomes widespread, for us not to. (It is also a
lot easier to do than you might think -- I could probably crank out the code to
do it, given the support libraries we already have in our product, in a couple
of hours.) What will happen is that a customer will call up on the phone and
say "what is this goo" and we'll respond that it's a signedData object which
contains MIME inside and they'll say "why didn't you convert it" and we'll say
"because that will wreck the signature". They will then say "this must mean you
didn't scan it for viruses either" and we'll say "yes that's true". And then
they'll say "that's unacceptable and words cannot express how little we care
about the signature so either convert it and scan it or we'll switch to a
gateway that does".  And since we're not inclined to go out of business by
taking the high road and not implementing such support, you may count it as a
given that we will do it if this eventuality comes to pass.

The one difference between Innosoft and another gateway vendor is that we'll
take care to implement the right sorts of tunneling options as well as
signature removal options. Other vendors will not do this.

> Right now, the use of signedData might make gateway vendors hopping mad,
> but I still have my signatures intact.  Maybe the gateway vendors will
> take pause and understand that what I do is not out of malice or spite,
> but out of necessity to enforce the sender's will over that of the
> gateway, which does not understand the sender's goals of sending a
> signed message.

Gateways vendors owe you no loyalty whatsoever and simply do not care what your
reasons are for doing what you do. Gateway vendors do owe their customers
something, however, and that something is to continually enhance their products
to accomodate whatever wierd sorts of content is thrown at them, converting it
into usable forms.

> Right now we have accommodated for the sometimes conflicting goals of
> the gateway (make sure it gets taken apart as best as it can, so that
> the content is preserved in a meaningful way -- preservation of content)
> and the sender (usually, I don't care if the signature gets eaten so
> preservation of content is important.  But dammit, sometimes I want to
> make sure that the signature is preserved!)  If the recipient's gateway
> implements tunneling to attack signedData, then it is making a value
> judgement -- it believes that mangling the content is more important
> than the sender's wish that it not be mangled, even though the sender
> has explicitly made the decision to keep the signature intact
> (otherwise, he would have sent it unsigned or as multipart/signed).
> This is a hard issue.

Hard line, maybe, but not a hard issue. The sender of the message has the same
right to consideration that you do: None whatsoever. Your right to dictate what
gateways do to messages simply doesn't exist in any realistic sense. What does
exist, however, is a recipient's right to get messages in the form that is most
advantageous for them. This is where there is some leeway, not at the other end
of things. So the thing to do is to come up with standards that say gateways
MUST implement the option to tunnel such material or not be compliant with
Internet specifications, not because some sender is whining about their
signature being busted, but because it is the best interests of recipients to
allow this option. This managers will hear and take note of, and it will change
their demands on gateway vendors considerably.

In fact gateway vendors welcome this sort of thing because it gives them a way
to distinguish themselves in the market. We win all sorts of sales over larger
companies because our gateways continue to provide more flexibility and
capabilites than the competition. If we can say "we implement compliant
multipart/signed handling and they don't" we just might win a sale we would
not have otherwise.

> If PKCS #7 tunneling evolves without intelligent handling of the fact
> that it is munching a signature, then I will encrypt all of the messages
> also.

At which point we'll get called and told "you didn't convert this". We'll
respond, "we can't because it is encrypted and we don't have the keys". The
customer will then say "that means you didn't check for viruses and furthermore
our security policies may be in the process of being thwarted, right?" and
we'll say "yes that's true". At this point the customer will probably say
"relax, we see this one isn't your problem, but where's that division manager
who deployed S/MIME support -- we want to have words with him about his
upcoming lack of career prospects in this company". Or they could say "we'll
institute a policy where the keys have to be made available to you can you use
them" or "how about you simply reject all encrypted messages", both of which we
could implement. Whatever. The point is that this is far from the viable
last-ditch option you seem to think it is.

> At some point my goals of on-demand guaranteed integrity,
> non-repudiation and authenticity will be accomplished.

I admire your continued willingness to believe this. I do not share your
belief, however.

> You're right,
> this isn't healthy.  Neither is diving into PKCS #7 signedData to grab
> the MIME out and discarding the signature like a dirty tissue.

Depends on whether or not the customer regards it as a dirty tissue. If they do
then discarding it is the right thing to do.

> I am in the position of developing both client and gateway products.
> Our latest product is a gateway product that makes very hard, thoughtful
> decisions about monkeying with signedData.  In the event that a
> modification would need to happen to a signed part (signedData or
> multipart/signed), there are very clear, administrator definable rules
> as to what should happen.  These rules can be (among other things) the
> modification of the content (and removal of the signature), or the
> notification of the sender and quarantine of the message.  I believe
> that a similar level of intelligence is required for other gateway
> products to evolve to the point where the sender's wishes are not
> ignored due to the "I know better" attitude of some of the current
> gateway technology that is still stuck in the early '90s, which
> interferes with conducting business-grade email.

Innosoft has had all this support in place for years now for multipart/signed.
But other vendors didn't add comparable support for the simple reason that the
IETF  didn't tell them to put it in. We need to fix this by telling them that
they have to.

> In my opinion, I would
> prefer that gateways return messages that are signed to the sender with
> a notification that the gateway needs to perform processing on the
> message which would invalidate the signature.  At that point, the sender
> could come up with some way to deal with this development (send the
> message unsigned) and continue on his way.

We provide this option available as well in our products.

> OK, so you use it unconditionally which pisses off every single mail
> client that is not aware how to process these constructs.  What am I
> missing here?  I get the impression that you're not considering the
> clients right now.  The installed base, as John points out.

You're missing the fact that this is a lose-lose situation. The product that
produces signedData fails because it generates a firestorm of protest and
regardless of security concerns, regardless of the need to upgrade
infrastructure, the basic ability to communicate has to come first. And you
lose because gateways vendors will also seek to accomodate this usage, and not
knowing what really needs to be done they won't get it right and your
signedData won't tunnel when it needs to.

> Have we seen rapid response from gateway vendors with multipart/signed?

Of course not -- we didn't ask for a response so we didn't get one.

> Is the use not widespread enough?  I'm honestly asking.

It isn't a question of usage, it is a question of requirements. Believe it or
not, requirements often play a far more significant role in purchase cycles
than actual usage does. 1/3 of the mail on the Internet could be
multipart/signed and it would not be sufficient to make gateway vendors
implement tunnelling as an option. A requirement that tunneling be provided as
an option, on the other hand, will have an effect. It may take time, but
it will have an effect.

> No one is suggesting that we ignore the gateways, they are most
> certainly important.  Maybe what needs to happen is that gateways need
> to stop ignoring the goals of constructs like multipart/signed and
> signedData, and instead stop screwing them up and deliberately working
> around them.  You cited before that Internet standard non-compliance is
> a great wakeup call to gateway vendors to fix their products.  I guess
> that since RFC 1847 is just Standards Track, it isn't really a standard
> yet, so the vendors wait until that "STD" shows up, and then become
> compliant.  I'm not referring to you, of course, but to the ones that
> don't work and play well with multipart/signed.

The problem with RFC 1847 isn't that it isn't far enough along on the standards
track, it is that it doesn't impose any requirements on gateways. In fact the
word "gateway" appears nowhere in the document. Nor does the word "tunnel". (I
believe both of these words used to appear but the entire discussion got
removed because that's what the PEM WG wanted.)

> So what can we add to multipart/signed to clarify this?  There is
> already language that says that you ***MUST NOT*** modify the first part
> (which still happens), and the content is not always carried intact
> through the gateway to keep the integrity of the signature.

There is no such text in RFC1847. The text that is there talks specifically
about message transfer agents not altering the content. But an MTA isn't a
gateway. They are different things. As such, this text cannot possibly be read
to impose any sort of restriction on a gateway. A gateway can in fact claim
full compliance with RFC1847 and do whatever it likes with multipart/signed.

In fact I'll go further than this. The fact of the matter is that gateways
are pretty much required to delve into multipart/signed by existing standards.
In particular, RFC2049 talks about how subtypes of multipart that don't get
other sorts of special treatment must be handled as multipart/mixed. Now,
this requirement applies to mail user agents. But while a gateway isn't
an MTA, it actually is thought of as a sort of MUA. We talk about gateways
having the right to exercise "user agent authority" on messages all the time.
Well then, it stands to reason that gateways also have follow the requirements
for user agents. And more to the point, gateways these days want to claim
MIME conformance. And the only MIME conformance we've ever bothered to
define is that of a receiving user agent. And there you go.

> Is there
> something ambiguous about the way that multipart/signed is worded that
> seems to make gateway vendors think that modifying the first part, or
> not presenting the first part intact along with the signature
> information, is not necessary for signature validation?

Gateway vendors don't get this far. They simply see that there are no
requirements imposed on gateways here and they ignore the rest. They also
see what MIME conformance means and they take it from there.

> I don't think
> gateways are dumb -- it's just that sometimes they are behind the times,
> which is quite apparently the case with multipart/signed.

They may be behind the times but they are not behind in terms of standards
compliance. I'm afraid you have no right to complain about gateway behavior in
this regard. You may have a right to complain that we've botched the standards
in this area. But it wasn't because I didn't know what the consequences were. I
will accept some measure of blame for not pursuing this issue more vigorously
but none for not realizing that it existed.

> I still don't
> see what the problem is with having a stopgap measure (signedData /
> application/mime / application/signed / whatever) to get around
> compatibility with older technology.  I further don't see what the
> problem is with having two encapsulation methods with clear rules to
> decide when to use each one, so that as much compatibility can be
> achieved before some future time where the world is better.

I never said there was any real harm. What I have said over and over again  is
that this approach is basically a wash -- it neither helps nor hurts all that
much. What needs to happen is that we finally sit down and say what gateways
are supposed to do. I realize you think we've already done this, but I think
I've proved that not only have we not done so, we've gone out of our way to
avoid doing so. We can do this in conjunction with some stopgap measure or
without some stopgap measure. Frankly, I couldn't care less. But we absolutely
must do it or we will not have accomplished anything at all.

				Ned

Date: Fri, 01 Aug 1997 15:12:16 -0700
To: Ned Freed <[email-masked]>
From: Dave Crocker / IMC <[email-masked]>
Subject: Re: application/signed vs. application/mime vs. signedData
Cc: Blake Ramsdell <[email-masked]>, "[email-masked]'" <[email-masked]>
Sender: [email-masked]

At 09:08 AM 8/1/97 -0700, Ned Freed wrote:
>Well, I for one disagree with this assessment. The problem with having two
>formats is that it then becomes a matter for the sender to choose which
one to
>use. And that's simply wrong -- this choice cannot in general be made

	At base, I strongly agree with you.  The reason for suggesting a construct
like application/mime was to make sure that we COMPLETELY SEPARATE the
structuring/labeling activites for real data service, like s/mime, from
peculiar transport situations, such as needing to bullet-proof.  That's why
content-transfer-encoding is separate from content-type.

	I am not as sure as Ned about the ultimate and complete failure of any/all
bullet-proofing.  My guess is that it can be useful on a small scale and
disastrous on a large scale.  But I am very certain that the the issue must
be dealt with separately from the semantic efforts, as for s/mime.  

	Let's be clear:  There is nothing about s/mime which requires two ways of
doing the same thing.

	There ARE differences in particular communications scenarios which need
additional bullet-proofing.  Such bullet-proofing needs to be achieved --
assuming it can work at all -- by one or more separate mechanisms.

	app/mime is an experiment.  there is no question that some gateways will
know about it and mess with it.  At a minimum, the purpose of the
bullet-proofing is to serve as a very clear label to keep one's processing
mits off this thing unless the software knows what it is doing.  

	mulipart has a default handling defined.  it is fine for many situations
and is proving to be a problem for others.  that's what app/mime tries to
respond to.

	if the approach to bullet-proofing used by app/mime works, there needs to
be onlyone way of doing it, for a variety of data types.  If it doesn't
work, we need to be able to back out from using it easily.  If we have this
approach to bullet-proofing implemented deeply into each of several
different semantics efforts, we won't be able to back out.

d/
--------------------
Dave Crocker
Internet Mail Consortium                               +1 408 246 8253
675 Spruce Dr.                                    fax: +1 408 249 6205
Sunnyvale, CA 94086 USA              [email-masked] , http://www.imc.org

Member, interim Policy Oversight Committee     http://www.gtld-mou.org

From: Blake Ramsdell <[email-masked]>
To: "'Ned Freed'" <[email-masked]>
Cc: "[email-masked]'" <[email-masked]>
Subject: RE: application/signed vs. application/mime vs. signedData
Date: Fri, 1 Aug 1997 13:24:29 -0700
Sender: [email-masked]

On Friday, August 01, 1997 9:09 AM, Ned Freed
[SMTP:[email-masked]] wrote:
> Well, I for one disagree with this assessment. The problem with having two
> formats is that it then becomes a matter for the sender to choose which one
> to
> use. And that's simply wrong -- this choice cannot in general be made
> intelligently by the sender. (I'm sorry, but a lot of situations exist where
> knowledge of the recipient doesn't exist, and I've yet to see a proposal
that
> deals with what happens when the capabilities of the recipient change over
> time.) What you're again trying to do is second-guess any gateways that may
> be
> present. And while I continue to think that this is mostly harmless, I also
> continue to think that it is mostly futile. Use application/mime and my
> gateway
> will treat it as composite. Nothing gained or lost. Use application/smime
and
> my gateway will treat it as composite. Nothing gained or lost. Play around
> with
> encodings and my gateway will decode them all. Nothing gained or lost. Use
> signedData in the process raising the bar for gateways to play, and get
> widespread deployment and you'll get two things: You'll have pissed off the
> gateway vendors and you will force them to implement a lightweight PKCS#7
> parser. Here all you gain is the emnity of the gateway implementors, so that
> when you do decide to call upon them for a real solution you won't get
> anywhere.

(Sorry about the big quote).  Ned, I can't sit by and not have secure
email until the gateway vendors evolve their products to accommodate
this recent (multipart/signed is two years ago -- not so recent)
development.  There are some gateways that are simply not in active
development, attached to networks that are simply too large to roll out
a new version because this faction of the Internet community would like
it to happen.  I would even argue that there are some gateways that will
stay exactly the way they are until they are thrown out and replaced --
they will never be modified.

I know that sometimes the recipient capabilities are not known (I've
even pointed it out).  In that case, you shoot for content preservation,
not signature preservation.  As far as changing over time, at one point
we specified that the signingTime attribute in a message was coupled
with the SMIMECapabilities so that newer capabilities could be recorded.
 I will check on that, since you're right -- it is important, and I
don't see it in the current draft.

I am willing to yank out any stopgap measures that I have put in, as
soon as we reach Gateway Nirvana.  The preferSignedData attribute will
eventually fall out of use when we reach this point, and clients will
automatically start working "the right way".  Of course, this is likely
to be at the point where we don't have to use CTE and everything is
binary also, but that's beside the point.

As far as a gateway tunneling into signedData, I think at that point it
is really over the edge.  The gateway has gone to the point where it
would implement PKCS #7 unwrapping code, and it has deliberately not
stopped to consider the consequences of modifying signed data.  This is
generally destructive to the use of digital signatures, and is in my
opinion a suicidal move in the face of their increasing popularity.
Right now, the use of signedData might make gateway vendors hopping mad,
but I still have my signatures intact.  Maybe the gateway vendors will
take pause and understand that what I do is not out of malice or spite,
but out of necessity to enforce the sender's will over that of the
gateway, which does not understand the sender's goals of sending a
signed message.

Right now we have accommodated for the sometimes conflicting goals of
the gateway (make sure it gets taken apart as best as it can, so that
the content is preserved in a meaningful way -- preservation of content)
and the sender (usually, I don't care if the signature gets eaten so
preservation of content is important.  But dammit, sometimes I want to
make sure that the signature is preserved!)  If the recipient's gateway
implements tunneling to attack signedData, then it is making a value
judgement -- it believes that mangling the content is more important
than the sender's wish that it not be mangled, even though the sender
has explicitly made the decision to keep the signature intact
(otherwise, he would have sent it unsigned or as multipart/signed).
This is a hard issue.

If PKCS #7 tunneling evolves without intelligent handling of the fact
that it is munching a signature, then I will encrypt all of the messages
also.  At some point my goals of on-demand guaranteed integrity,
non-repudiation and authenticity will be accomplished.  You're right,
this isn't healthy.  Neither is diving into PKCS #7 signedData to grab
the MIME out and discarding the signature like a dirty tissue.

I am in the position of developing both client and gateway products.
Our latest product is a gateway product that makes very hard, thoughtful
decisions about monkeying with signedData.  In the event that a
modification would need to happen to a signed part (signedData or
multipart/signed), there are very clear, administrator definable rules
as to what should happen.  These rules can be (among other things) the
modification of the content (and removal of the signature), or the
notification of the sender and quarantine of the message.  I believe
that a similar level of intelligence is required for other gateway
products to evolve to the point where the sender's wishes are not
ignored due to the "I know better" attitude of some of the current
gateway technology that is still stuck in the early '90s, which
interferes with conducting business-grade email.  In my opinion, I would
prefer that gateways return messages that are signed to the sender with
a notification that the gateway needs to perform processing on the
message which would invalidate the signature.  At that point, the sender
could come up with some way to deal with this development (send the
message unsigned) and continue on his way.

> FWIW, I do not see application/[s]mime as a second format option. I would
> expect it to be used unconditionally if it is used at all. I would also
> expect
> to see a rapid response from gateway vendors if it comes into widespread
use.

OK, so you use it unconditionally which pisses off every single mail
client that is not aware how to process these constructs.  What am I
missing here?  I get the impression that you're not considering the
clients right now.  The installed base, as John points out.

Have we seen rapid response from gateway vendors with multipart/signed?
Is the use not widespread enough?  I'm honestly asking.

> No. What is really needed is a single format and rules for how gateways have
> to
> handle it. The IETF needs to stop pretending that gateways don't exist and
> can
> be ignored or worked around.

No one is suggesting that we ignore the gateways, they are most
certainly important.  Maybe what needs to happen is that gateways need
to stop ignoring the goals of constructs like multipart/signed and
signedData, and instead stop screwing them up and deliberately working
around them.  You cited before that Internet standard non-compliance is
a great wakeup call to gateway vendors to fix their products.  I guess
that since RFC 1847 is just Standards Track, it isn't really a standard
yet, so the vendors wait until that "STD" shows up, and then become
compliant.  I'm not referring to you, of course, but to the ones that
don't work and play well with multipart/signed.

So what can we add to multipart/signed to clarify this?  There is
already language that says that you ***MUST NOT*** modify the first part
(which still happens), and the content is not always carried intact
through the gateway to keep the integrity of the signature.  Is there
something ambiguous about the way that multipart/signed is worded that
seems to make gateway vendors think that modifying the first part, or
not presenting the first part intact along with the signature
information, is not necessary for signature validation?  I don't think
gateways are dumb -- it's just that sometimes they are behind the times,
which is quite apparently the case with multipart/signed.  I still don't
see what the problem is with having a stopgap measure (signedData /
application/mime / application/signed / whatever) to get around
compatibility with older technology.  I further don't see what the
problem is with having two encapsulation methods with clear rules to
decide when to use each one, so that as much compatibility can be
achieved before some future time where the world is better.

Blake
--
Blake C. Ramsdell
Deming Internet Security, a Worldtalk Company
For current info, check http://www.deming.com/users/blaker
Voice +1 425 882 8861 x103  Fax +1 425 882 8060

Date: Fri, 01 Aug 1997 12:40:56 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: RE: application/signed vs. application/mime vs. signedData
To: Steve Dusse <[email-masked]>
Cc: "'Ned Freed'" <[email-masked]>, "[email-masked]'" <[email-masked]>
Sender: [email-masked]

> The frustrating point that keeps coming up is this.  Although "fixing
> the gateways" is the right thing to do, it is simply unacceptable to not
> have secure email until all the gateways are fixed.

I'm afraid this is basically counterproductive. The reality is that there's no
path that gets us usable secure email without getting gateway support in place
first. So, while getting such support in place first may be unacceptable to
you, it really is the only way that secure email is going to happen. So the
choice becomes very simple: Do you want secure email or not? If you do, you
need to face up to the fact that the gateway problem is real and you cannot
work around it. Instead we have to solve it, and unfortunately then we have to
wait for implementations to catch up. This is frustrating, I agree, but I see
no other viable course that will work.

It isn't like we haven't tried to solve the problem without getting gateway
support in place. In fact we've been frantically trying to do things this way
for well over five years now. And after huge implementation efforts, tons of
discussion, lots of standards writing, and all sorts of furious ancillary
activity, where are we? Nowhere, that's where. We're still butting our heads up
against the same set of problems we had five years ago. And worse, had we
actually faced the problem head on five years ago and written a specification
for what gateways should do with MIME objects this problem would have been
solved by now since the initial implementation MIME in literally hundreds of
gateway products would have taken these issues into account.

In fact it would be fair to say that nothing we do now will ever be able to
recoup the the loss of the first and best window to get this sort of support
into gateways. We blew it, plain and simple. (I count myself as part of the
failure to get this done properly since I should have fought harder to keep the
language about gateways in multipart/signed, not to mention MIME and ESMTP.)

Now, you may argue that since we've missed the chance to do this right we might
as well give up and continue to try various half-assed ways to work around this
problem. But there's no evidence that such approaches will in fact work any
better in the future than they have worked thus far. And furthermore, there is
considerable evidence to suggest that new language in Internet specifications
will in fact cause gateway vendors to mend their ways. In fact there is
evidence to suggest that this will not only happen, it will happen rapidly.

So I suggest that rather than waste another five years with the same end
result we instead try to tackle the real problem. 

> Therefore, we have
> a need for two formats, hopefully evolving to one in the future.

> I do know one way to have a single format which serves both needs.  It's
> called content duplication (i.e. multipart/signed and signedData in a
> single message) and was the basis for the original S/MIME proposal 2
> years ago.  It's awfully unpopular and has a lot of wierd potential side
> effects but it seems to work.

Your own words argue explain why this is a nonstarter -- you say it was awfully
unpopular. And with all due respect, this is being kind. The use of this format
generated a firestorm of protest and probably has delayed S/MIME by at least a
year if not more. And not only is there no reason to believe that it will be
any more successful now than it was before, there is considerable reason to
believe that it will be _less_ successful.

Here's why. The previous attempt to use multipart/alternative in this way
happened at a very unfortunate time -- it was a time when a bunch of products
appeared that also generated extra MIME parts in messages in various
proprietary formats. Microsoft Exchange and the infamous MS-TNEF parts is the
most widely known example of this, but there are a bunch of others. So there
was a flood of messages containg material end users saw as useless crap.
It doesn't matter that this stuff is useful under some circumstances, the
point is that users didn't see it as useful. And while S/MIME's use
of multipart/alternative was small beer compared to the sins of others
like Microsoft, its coincidental presence caused it to be tarred with the
same brush.

And all this in turn forced a bunch of gateway vendors, myself included, to
implement something we had resisted implementing for many years -- we added
capabilities to summarily delete attachments from messages. We simply  had no
choice but to do this -- the market pressure was overwhelming. And these
capabilities are now widely deployed and system managers understand how to use
them.

So now when a product starts emitting messages with, say, a PKCS wrapper around
signedData, it won't take system managers more than a few seconds to add this
to the (long) list of "crap to delete". After all, it isn't like it damages
messages any to lose this part. All it does is preclude the deployment of
secure email products on either side of the gateway later. In fact I predict
that deployment of part removal will proceed in advance of deployment of
secure email, making secure email practically impossible to deploy across
gateways.

> Don't get me wrong, message bloat is a
> _really bad thing_.  Maybe the time is right to revisit if it is a
> _worse_ thing than having two signature formats...

The real problem with this format isn't message bloat or the security issues it
raises (which continues to make it a showstopper for me, BTW), it is the added
message complexity. A few lines of additional junk at the beginning of a
message are one thing, a vast sea of binary gunk at the end that looks like a
useful attachment but isn't is quite another. Users will not tolerate it and,
due to the previous use of this format and what it did to gateways, system
managers will be ready and able to take care of their user's demands in short
order now.

So what's it to be? Are we going to learn from our mistakes and solve the real
problem? Or are we going to not pay attention to our own dismal track
record and waste even more time?

				Ned

From: Steve Dusse <[email-masked]>
To: "'Ned Freed'" <[email-masked]>
Cc: "[email-masked]'" <[email-masked]>
Subject: RE: application/signed vs. application/mime vs. signedData
Date: Fri, 1 Aug 1997 11:26:23 -0700
Sender: [email-masked]

The frustrating point that keeps coming up is this.  Although "fixing
the gateways" is the right thing to do, it is simply unacceptable to not
have secure email until all the gateways are fixed.  Therefore, we have
a need for two formats, hopefully evolving to one in the future.

I do know one way to have a single format which serves both needs.  It's
called content duplication (i.e. multipart/signed and signedData in a
single message) and was the basis for the original S/MIME proposal 2
years ago.  It's awfully unpopular and has a lot of wierd potential side
effects but it seems to work.  Don't get me wrong, message bloat is a
_really bad thing_.  Maybe the time is right to revisit if it is a
_worse_ thing than having two signature formats...

-steve
>----------
>From: 	Ned Freed[SMTP:[email-masked]]
>Sent: 	Friday, August 01, 1997 9:08 AM
>To: 	Blake Ramsdell
>Cc: 	[email-masked]'
>Subject: 	Re: application/signed vs. application/mime vs. signedData
>
>> In the hopes of getting closer to consensus, I'm clearing the slate.
>
>> I think we've determined that there is still a need to have two formats,
>> one that is suited to preserving content, and one that suited to
>> preserving signatures.
>
>Well, I for one disagree with this assessment. The problem with having two
>formats is that it then becomes a matter for the sender to choose which one
>to
>use. And that's simply wrong -- this choice cannot in general be made
>intelligently by the sender. (I'm sorry, but a lot of situations exist where
>knowledge of the recipient doesn't exist, and I've yet to see a proposal that
>deals with what happens when the capabilities of the recipient change over
>time.) What you're again trying to do is second-guess any gateways that may
>be
>present. And while I continue to think that this is mostly harmless, I also
>continue to think that it is mostly futile. Use application/mime and my
>gateway
>will treat it as composite. Nothing gained or lost. Use application/smime and
>my gateway will treat it as composite. Nothing gained or lost. Play around
>with
>encodings and my gateway will decode them all. Nothing gained or lost. Use
>signedData in the process raising the bar for gateways to play, and get
>widespread deployment and you'll get two things: You'll have pissed off the
>gateway vendors and you will force them to implement a lightweight PKCS#7
>parser. Here all you gain is the emnity of the gateway implementors, so that
>when you do decide to call upon them for a real solution you won't get
>anywhere.
>
>FWIW, I do not see application/[s]mime as a second format option. I would
>expect it to be used unconditionally if it is used at all. I would also
>expect
>to see a rapid response from gateway vendors if it comes into widespread use.
>
>Note that another way to do this would be to simply replace the
>multipart/signed label with an application/signed label, and don't add any
>extra level.
>
>> Because we have not been able to arrive at one
>> format that satisfies both goals, two formats are needed.  Because we
>> have two formats, it is a client determination as to which one gets
>> sent.
>
>No. What is really needed is a single format and rules for how gateways have
>to
>handle it. The IETF needs to stop pretending that gateways don't exist and
>can
>be ignored or worked around.
>
>				Ned
>

Date: Fri, 01 Aug 1997 09:08:33 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: application/signed vs. application/mime vs. signedData
To: Blake Ramsdell <[email-masked]>
Cc: "[email-masked]'" <[email-masked]>
Sender: [email-masked]

> In the hopes of getting closer to consensus, I'm clearing the slate.

> I think we've determined that there is still a need to have two formats,
> one that is suited to preserving content, and one that suited to
> preserving signatures.

Well, I for one disagree with this assessment. The problem with having two
formats is that it then becomes a matter for the sender to choose which one to
use. And that's simply wrong -- this choice cannot in general be made
intelligently by the sender. (I'm sorry, but a lot of situations exist where
knowledge of the recipient doesn't exist, and I've yet to see a proposal that
deals with what happens when the capabilities of the recipient change over
time.) What you're again trying to do is second-guess any gateways that may be
present. And while I continue to think that this is mostly harmless, I also
continue to think that it is mostly futile. Use application/mime and my gateway
will treat it as composite. Nothing gained or lost. Use application/smime and
my gateway will treat it as composite. Nothing gained or lost. Play around with
encodings and my gateway will decode them all. Nothing gained or lost. Use
signedData in the process raising the bar for gateways to play, and get
widespread deployment and you'll get two things: You'll have pissed off the
gateway vendors and you will force them to implement a lightweight PKCS#7
parser. Here all you gain is the emnity of the gateway implementors, so that
when you do decide to call upon them for a real solution you won't get
anywhere.

FWIW, I do not see application/[s]mime as a second format option. I would
expect it to be used unconditionally if it is used at all. I would also expect
to see a rapid response from gateway vendors if it comes into widespread use.

Note that another way to do this would be to simply replace the
multipart/signed label with an application/signed label, and don't add any
extra level.

> Because we have not been able to arrive at one
> format that satisfies both goals, two formats are needed.  Because we
> have two formats, it is a client determination as to which one gets
> sent.

No. What is really needed is a single format and rules for how gateways have to
handle it. The IETF needs to stop pretending that gateways don't exist and can
be ignored or worked around.

				Ned

From: Blake Ramsdell <[email-masked]>
To: "[email-masked]'" <[email-masked]>
Subject: application/signed vs. application/mime vs. signedData
Date: Fri, 1 Aug 1997 01:01:28 -0700
Sender: [email-masked]

In the hopes of getting closer to consensus, I'm clearing the slate.

I think we've determined that there is still a need to have two formats,
one that is suited to preserving content, and one that suited to
preserving signatures.  Because we have not been able to arrive at one
format that satisfies both goals, two formats are needed.  Because we
have two formats, it is a client determination as to which one gets
sent.

The choice of multipart/signed for content preservation seems
uncontested.  Thank goodness.

For signature preservation, three proposals exist.  These proposals and
some of their criticisms are:

1. signedData.  Only works for PKCS #7 data and is unreadable by
non-S/MIME agents.
2. application/mime.  Very  generic, so intermediate processors will be
tempted to monkey with it as regular MIME data.  Pretty much unreadable
by any current mail agent.  MIME tools exist that can process the data.
3. application/signed.  Less generic, but still interesting to
intermediate processors.  Pretty much unreadable by any current mail
agent.  MIME tools exist that can process the data.

Some of their strengths are:

1. signedData.  More resilient against peeking inside of it (significant
code is needed to get at the inner MIME data).
2. application/mime.  Very generic.  Mail clients will soon support it
and its semantics of "if you don't understand what I am, don't try to
parse me".
3. application/signed.  Generic in that it represents the encapsulation
of any type of multipart/signed entity (MOSS, S/MIME, PGP, FOO, etc.)
Better semantics than application/mime for this particular case (it
always represents the armoring of signed content).

I can give my opinion:  let's talk about application/signed some more
because I like it better than the other two for this particular case.  I
don't think that application/mime is not useful, but I do think that
application/signed has better semantics for what we are trying to
accomplish.

Blake
--
Blake C. Ramsdell
Deming Internet Security, a Worldtalk Company
For current info, check http://www.deming.com/users/blaker
Voice +1 425 882 8861 x103  Fax +1 425 882 8060

Date: Thu, 31 Jul 1997 23:14:33 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: application/mime and binary data
To: John Gardiner Myers <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

> > I should say that I'm not fundamentally opposed to application/{signed or
> > mime}, I just want to make sure we agree on the facts and the reason why we
> > need it.

> That's similar to my position.  We need to understand that we have
> mutually incompatible requirements here.  The usual method of obtaining
> a specification through successive approximations made in response to
> raised issues does not work well in this situation.  To obtain a good
> solution, people need to keep in mind the entire problem set and keep
> straight which modes of operation address which requirements.  This is
> difficult to do in a typical list forum, where the group attention span
> is rarely longer than two messages.

While I understand and appreciate all this red vs. blue stuff, I myself have a
much simpler view of the world -- I view it from the perspective of a gateway
to a non-822-non-MIME environment. And when you look at things from this point
of view you realize that there are two cases:

(1) The gateway talks to clients that have MIME-ware and signature verification
    facilities available. In this case the correct thing for the gateway to
    do is to tunnel signed objects without translation.

(2) The gateway talks to clients that don't have MIME-ware and signature
    verification facilities available. In this case the correct thing for the
    gateway to do is to treat multipart/signed or whatever composite object
    is being used to hide the signed content apart (note that this includes
    signedData).

Neither of these two is right and neither of these two is wrong -- sometimes
one applies and sometimes the other applies. And no matter what you do, no
matter what sort of tricky MIME composite types you define, no matter what
encodings you use, no matter what ASN.1 enclosures you use, the issue never
goes away. 

This, of course, is a special case of a very general problem: The problem that
end-to-end security services are fundamentally incompatible with multiprotocol
networks. The minute you start talking about having two formats for application
data in a network while simultaneously supporting end-to-end security you have
to either tunnel one format inside of another (option (1)) or convert formats
and do whatever security you're able to do at something other than an endpoint
(option (2)). The issue is fundamental and can never be satisfactorily
reconciled. And BYW, if you think we're having problems, look at DMS sometime.
DMS is attached to X.400 at the hip. But the real world isn't all X.400, it is
a bunch of other stuff. So what you end up with is X.400 P2 floating around
inside of a security enclosure which in turn is inside of MIME. Anyone game to
tackle, say, the wide variety of BP15s, including but not limited to
FTBP, in a MIME environment?

So what do you end up with? The gateway has to be configurable and has to be
properly configured to do the right thing, and that's all there is to it. All
we're doing (and all we'll ever be able to do) is try and optimize behavior
based on what we think the defaults of most gateways are.

And this, in my opinion, is not the right way to develop standards. If we want
to mandate that gateways be able to choose (1) or (2), well then we should
simply mandate that and be done with it. And do not kid yourself -- the notion
of being incompliant with an Internet standard _does_ make a difference to
gateway vendors. We have proof of this in past actions, as a matter of fact --
the change in MIME to make the requirement that composite objects be supported
clearer worked wonders in the gatway world and got at least four gateways I
know of fixed.

This, to my mind, illustrates the real failure in the work we've done in the
IETF: We've fallen into exactly the same trap X.400 did and refused to deal
with gateway behavior. In fact we've done this over and over again. I
originally had language in MIME that talked about proper gateway behavior. It
was a long way from perfect, to be sure, but the feedback I got wasn't that it
needed to be corrected, it was that it clearly needed to be done away with. I
also had language in the ESMTP specifications that talked about gatewaying from
8bit to 7bit MIME -- it also was rejected and I had to remove it. I also had
language about the proper processing of signatures and decryption at
non-endpoints in the original security multiparts specification (actually,
Marshall Rose wrote it and I tweaked it -- this was before Jim and the other
TIS folks were even involved in the writing) and I had to remove it. And I
doubt if I'm the only one that has tried to deal with this area, and run into
this sort of resistance. And now, five years later, this shortsightedness has
come home to roost, causing us all to run around in tight little circles.

Fine, you say, but what about the issues that arise purely in the MIME world
with stuff switching around content types? Well, first of all, this really is a
gateway issue too -- specifically the one that should have been addressed in
the 8bitMIME ESMTP document. But even if it weren't, I think this problem has
been seriously overstated. In fact I am unaware of any current product that
gratuitously goes into all-7bit messages and changes the encodings or boundary
markers within multipart/signed. And even if there were such a product, I
seriously doubt that it is widely used. And even if it is, I think an argument
can be made that it doesn't comply with extant Internet standards, and as such
should be dealt with on the basis of its incompliance. (The same cannot be said
for gateways to LAN email systems since we've never bothered to address what
they should or should not do.) But no matter how you slice it the pure
SMTP/MIME gateway problem pales in comparison with the 10s of millions of users
that sit in front of non-MIME UAs that connect to the Internet via gateways.

There has also been a lot of talk about how the design of MIME didn't deal with
security issues correctly. With all due respect, I think this is hogwash.
Again, there are incompatible goals here that cannot ever be reconciled -- the
minute you develop a format for composite objects that has to coexist in a
world with many such formats you are at odds with end-to-end security. And we
had to have a composite format, so the only choice we could have made that
wouldn't have compromised future end-to-end security needs would have been to
abandon the development of MIME. For example, we could have allowed recursive
encodings in MIME and if anything it would have made matters worse, because
implementations would simply have pushed the encodings out to the leaves to
insure maxmimum legibility to non-MIME agents, and we would have ended up with
more message munging, not less. (And please don't tell me that they wouldn't
because I implemented my MIME parser when the specification still said that
recursive encoding was OK and this is exactly what it does.) Our only mistake
as far as I can tell is that we never specified what sorts of conversions could
be applied to MIME objects -- what a gateway does, in other words.

So where does this leave us? It leaves us with three choices:

(1) Continue to piddle around with MIME types in hopes of fixing something. I
    think this is mostly harmless but also mostly useless.
(2) Switch to using signedData. I think this is actively harmful and also
    doomed -- if it ever becomes widespread enough to matter gateways will
    simply start decoding it and discarding the signature, and we'll be back
    where we started.
(3) Try and fix the problem. This means writing some clear, crisp guidelines
    for what it means to have a compliant gateway. This is hard, unpleasant
    work -- make no mistake about it. And I'm sure it makes some people
    uncomfortable since it means they have to depend on other vendors to solve
    problem and not do anything themselves. The good feeling that comes from
    doing something, even something pointless, is gone.

The option I think is best should be obvious.

				Ned