Subject: Re: We do have a marketing problem ...

From: Dick Brooks <[email-masked]>
Date: Tue, 20 Apr 1999 16:13:24 -0500 (CDT)

>> With
>> e-mail, even using the EDIINT secure receipt, you don't know untill
>> later whether it succeeded or failed (including the use of a timeout if
>> you don't receive a response at all).
>
>It depends on whether a store-and-forward gateway is used. Most 
>people use a gateway by default since it's easier and automatically 
>queues if the recipient is inaccessable.

>With direct SMTP connection, the system knows when the 
>message has been received. A receipt is still important, since the 
>message could be lost on the recipients system before it's 
>processed.

In the case of HTTP POST, as in GISB's implementation, the data exchange
and positive acknowledgement occur in the same "transaction", whereas in 
e-mail based approaches the data exchange and acknowledgement occur as
separate transactions. With GISB a trading partner knows without doubt
if a file was received in its entirety by the destination host immediately,
via positive acknowledgement. 

>> So EDIINT takes more effort to
>> implement, since you have to keep track of it and save the original files,
>> and not know whether they are "checked off" until later.

>You'd have to do the same with FTP, or even with a VAN if you 
>wanted maximum reliability and detection of problems.

The real-time acknowledgement is built into GISB, no extra effort needed.

>> There is also the risk that your EDIINT transmission succeeded, but the
>> receipt that was sent back in response failed, in which case you resend
>> your original transmission after a timeout of waiting for the receipt.

>Most failures can be detected and the sender can retry. For 
>completeness, the original can be resent.

Yes, failures can be detected, but what if a failure occurs in a relay host
that isn't aware of the need to notify parties in the event of failures.

>> Maybe we need a
>> definition of "real-time".

>Messaging within a few seconds, on a non-overloaded link. Of 
>course this is what SMTP can do technically, not necessarily what 
>implementations do.

>> You don't have a session or a virtual
>> connection with e-mail.

>With SMTP you have a socket connected to the recipients system, 
>and using the TURN operation, the same socket can be used for 
>the reply. Otherwise, a separate socket would be used.

>The asynchronous wait for reply includes processing of the 
>message. If the recipient system can't process immediately, then
>the checkoff of the reply needs to wait.

>The smtp (or http post) is synchronous in that the incoming 
>message is immediately delivered to a processing application. 
>With FTP, normally, the file system must be polled and checked 
>for changes. There is a problem in that it is not always easy to tell 
>when the FTP file transfer is complete and correct. Errors can 
>occur, undetected to the sender, leaving partially complete files 
>laying around. I suppose FTP could be made to work 
>synchronously, but it's more complex than SMTP.

GISB has a built in mechanism for reporting errors that occur after transmission
has completed. The trading partner is notified immediately upon discovery
of such errors.

>Any high-reliability system is going to need to keep a log of 
>transactions, and check to make sure there are no problems, so I 
>don't see that checking for missing reciepts is any extra effort. It's 
>much simpler than getting FTP to work.

> Can't you have a session with http (maybe that is
>> what cookies are used for), in which case http is real-time?

>A cookie "session" is a way to associate separate transactions, 
>which otherwise have no relationship. With EDIINT, Message-Ids 
>are used to establish a relationship. BTW, with FTP there are 2 
>connections made: 1 a "session" control link, and a separate link 
>for each file transmitted.

GISB uses a single HTTP socket connection for sending data/receiving
acknowledgements, no need for cookies.

---

From: Gunther Schadow <[email-masked]>
Date: Fri, 23 Apr 1999 11:54:28 -0500 (EST)

To wrap this up:

(A) I am glad that Gary does agree that FTP is definitely *not* the
better alternative.

(B) I believe it is absolutely wrong to share the following beliefs
with the masses:

	synchronous  = good

	asynchronous = bad

It is not cool to juggle with the terms "synchronous" and
"asynchronous" in that way, since it shows that the juggler didn't
understand much more than the bare definition of those terms.

Both modes of communication have their use cases. It is important to
keep that in mind once and for all. For example:

EX1:

An electronic bidding system: the buyer sends out a multicast message
to all the sellers asking for a price and setting a dead line. The
sellers respond with tehir quote (that might have been manually
determined and approved.)

There is little use for synchronous communication in this example.

As a system designer, you have to ask yourselves:

(1) do I want to be blocked by waiting for the response?

(2) do I want to handle errors here?

(3) do I need any information of the response message in order
    to move on with my job?

If the answer is "no" or "probably not", then your best bet is going
asynchronous. Let your mail transfer agent take care of the queueing
and delivery.  Yes, you have to match your response, but as I showed
earlier last year, there is a simple way to match responses as they
come in.  This is especially useful if you do not have automatic
standard ways of how to correct an error. Resend and timeout are
things that any mail transfer agent has built in. So why do you want
to reinvent the wheel only to be able to say "ours is synchronous
(implying 'better')".  If you alarm and log failures after all retries
and timeouts anyway, you are not better off with HTTP. With e-mail you
can route and filter your bouncing messages much nicer.

And finally, as Carl Hage preaches so often (nobody seems to believe
him), you can use SMTP in synchronous mode with one session for both
request and response messages.

(C) E-mail is not unreliable.  When can we finally sell this message
at least on this list?

  - e-mail can be run as directly and as synchronously as HTTP

  - where e-mail starts relaying, your chances to be able to
    use HTTP drop significantly.  

    If your trading partner sits behind a dialup line and polls his
    e-mail with POP, you won't have much success to get through to
    him with HTTP either. All the same with firewalls, and non-IP
    networks.

In short, AS #1 is the more flexible (shall I say powerful) way to
go. AS #2 is certainly the more popular way to go.  But one thing is
clearly wrong:

> I'm sorry but I just don't buy your argument.  HTTP is the only
> reasonable way to do high volume EDI over TCP/IP.

AS #1 is definitely not unreasonable.

regards
-Gunther

PS: my point in starting this thread in the first place was to collect
success reports and journal article that would defend--what I believed
then--our common position.  But now I realize that popularity of AS #1
has dropped significantly even in this group.

Gunther Schadow ----------------------------------- http://aurora.rg.iupui.edu
Regenstrief Institute for Health Care
1001 W 10th Street RG5, Indianapolis IN 46202, Phone: (317) 630 7960
[email-masked] ---------------------- #include <usual/disclaimer>

---

From: "Bjorn & Lilian Schmid" <[email-masked]>
Date: Mon, 26 Apr 1999 01:15:16 +1000

I may be a little naieve, but I too am a little amazed at the resistance to
smtp delivery and receipting of transaction data.  I am involved in a major
Health Interconnectivity project here in Australia.  Due to the vast
distances and relatively poor telecommunications available in many areas, we
are not considering anything but smtp as a common means of messaging.  It is
ubiqutous, anyone anywhere can have an email address with a minimum of
equipment and connectivity.

Of course there are time-critical or high-volume requirements which may only
be solved by a high capacity intranet or point-to-point connections of some
sort, but these are usually higher-end application users as well.

I subscribed to EDIINT to be involved in solutions like AS1 propose.
I hope to see the final release soon.
Regards,
Bjorn Schmid
-----Original Message-----
From: Gunther Schadow <[email-masked]>
To: [email-masked] <[email-masked]>; [email-masked]
<[email-masked]>; [email-masked]
<[email-masked]>; [email-masked] <[email-masked]>
Cc: [email-masked] <[email-masked]>
Date: Saturday, 24 April 1999 4:07
Subject: Re: We do have a marketing problem ...

---

From: Andrew Ferguson <[email-masked]>
Date: Mon, 26 Apr 1999 23:49:42 +1000

All,

I for some months now have been watching with interest the current debate
on what might be the consensus view  for the delivery of Electronic Data
Interchange or EDI over the Net.

I have been involved in EDI and now ECommerce solutions both as  Consultant
, a Product Manager and more recently running my own business providing
solutions in the market for some 20 years and the issues here in Australia
and New Zealand are I think a little different to those experienced around
the rest of the world, but nevertheless I believe have a practical
application to this current debate.

In Australia, we have for a number of years had large scale EDI deployments
of ten of thousands companie deploying ANSI X.12 and EDIFACT based EDI
messages over in the main X.400 based managed network services. These
services are in daily use by a number of industries where a guarantee of
service capability and audit trail is fundamental to good business practice
i in industries such as the Retail, Automotive, Manufacturing and now
increasingly in the Health sectors. In some cases these services are
mandated. To export all cargo in Australia you have to electronically lodge
your export documentation using EDI or wait up to two weeks before maual
processing can happen (EDI tclearance happens in a matter of minutes, but
you have to have the audit trail to satisfy customs desk audits). However
whilst these services deliver guaranteed performance and service together
with audit trails the downside is that they cost far too much.

Most of these organisations in Australia, and I expect in the main US and
European organisations are seeking a more cost effective of doing exactly
the same over standard Internet based network services.

Interestingly at least one Australian service network vendor is now
offering managed X.400 services over TCP/IP at flat internet rates. 

However the market demands are for ways of delivering these services over
standard TCP-IP services both Messaging (SMTP based over standard networks
ie store and forward) or HTTP based. Of course the underlying issues are
that it must be secure and offer a guaranteed level of service such that
they can trust their business processes to move from the current
"expensive" methods to the cheaper more cost effective methods using EDIINT.

I agree totally with Dick Brooks view that for guarantees of service and
delivery  and imediate acknowledgement, support for large messages etc HTTP
or preferably S-HTTP is the way to go, the simple fact of business today
are that in the main SMTP is used for casual messaging (ie person to
person) and  cannot I believe except on a point to point basis offer a
business based service which can come any where near the levels of service
guarantee needed by industry which will make the move from their "managed"
services to EDI over the Net, even if the security perspective is solved.

I also believe Carl Hage is correct in that SMTP can offer these immediate
benefits, the problem is here in our country, in the main industry does not
implement point to point services, rather that they rely on the store and
forward solutions to guarantee their Just in Time logistics or Export
orientated  organisations stay in business.

Interestingly Bjorn from Australia made some comments about his Health
project. To day in Australia, the major ECommerce networks in the Health
sector uses X.400 based EDI services, however what the industry wants is
Internet based business transactions, but with a guarantee of service.
However when this is provided for life threatening or patient related data,
immediacy and guarantees of service will I believe outweigh the so called
cost benefits of store and forward messaging with its lack of  immediacy
and guarantee of delivery except in "Intranet service".
Not that I am defending X.400 which is long past its use by date, but I
note with interest that most of the European implementations of Business
Document Interchange and Electronic Messaging in the Health sector are
still being rolled out on X.400 based managed services networks admittedly
using TCP/IP as the backbone provision.

So to summarise, in my view it all comes down to Dick's point, the decision
to use HTTP or SMTP should be based on the business requirements. 

------
Software Agencies Australia Pty Ltd
1 Huntingdale Road
Burwood,
Victoria 3125
Australia
ACN 078 025 813 
Tel: +61-3-9808-0522
Fax: +61-3-9888-6019
Mobile: +61-41-222-3940
Email: [email-masked]
URL: www.software-aus.com.au
Your Business Partner in Electronic and Internet Commerce

---

From: Dick Brooks <[email-masked]>
Date: Mon, 26 Apr 1999 12:55:36 -0500 (CDT)

From:           	Carl Hage<[email-masked]>

>> SMTP based solutions are adequate when the following conditions are met:
>>  - immediate acknowledgement is NOT required
>>
>>SMTP provides immediate acks (within 1 second of an HTTP 
>>implementation) provided the TP has the network connection 
>>required for HTTP, and does not use a store-and-forward gateway.

>Please do not equate "queued sendmail + store-and-forward 
>gateway" with "SMTP". SMTP has no queuing and no delay. Use 
>of store-and-forward is also not part of SMTP except an SMTP 
>server can deliver to a store-and-forward system if configured that 
>way.

Carl, technically SMTP could provide immediate acknowledgement using the 
OPTIONAL "TURN" capability. If this is the intended behavior then AS1 should
specify that "TURN" should be used for immediate acknowledgement delivery. 
At present, AS1 does not specify this as a requirement.

>>  - guaranteed timely delivery is NOT required

>SMTP can provide guaranteed delivery in an identical time as 
>HTTP. However, SMTP also works with non-dedicated links where 
>immediate response is not possible.

SMTP can provide this, as an OPTIONAL configuration. The default behavior
of sendmail based systems is store-and-forward with delivery allowed to relay 
hosts.

HTTP also works with non-dedicated links in a push/pull type of mode.

>>  - login/password authorization is NOT needed

>AS1/2 uses digital signatures and certificates, which provide 
>authentication but do not require logins or secret passwords.

You're missing the point here. SMTP has no facility to "BLOCK" unwanted parties
from sending data to a transaction server. I'm not referring to anti-spam type
blocking, I mean user/password type of security controls. GISB also uses
digital signatures to "authenticate" originators of documents as a second
level of security.

>>  - Delays of minutes to days are acceptable

>SMTP has no delay unless the TP is not directly reachable, in 
>which case it still succeeds, even though a TP might not connect 
>for minutes to days.

This is exactly my point, successful delivery could come days later under a
typical MTA delivery mechanism.

>>  - No gateways/relays encountered during transmission to affect the 
>>    integrity of signed multiparts

>SMTP/AS1 works with gateways, including those which cannot 
>confirm delivery immediately.

Carl, you need to read Need Freeds RFC 2480 published 1/99 that identifies 
known problems where gateways can cause problems for signed multiparts.
He specifically identifies the following known problems (which I've taken
directly from the RFC):

   "Difficulties arise, however, in heterogeneous environments involving 
    gateways to environments that don't support MIME. Specifically:

    (1)   Security services have to be applied to MIME objects in
          their entirety. Failure to do so can lead to security
          exposures.

          For example, a signature that covers only object data and not
          the object's MIME labels would allow someone to tamper with
          the labels in an undetectable fashion.  Similarly, failure to
          encrypt MIME label information exposes information about the
          content that could facilitate traffic analysis.

          Composite MIME objects (e.g., multipart/mixed, message/rfc822)
          also have to be secured as a unit.  Again, failure to do so
          may facilitate tampering, reveal important information
          unnecessarily, or both.

    (2)   Gateways that deal with MIME objects have to be able to
          convert them to non-MIME formats.

          For example, gateways often have to transform MIME labelling
          information into other forms. MIME type information may end up
          being expressed as a file extension or as an OID.

          Gateways also have to take apart composite MIME objects into
          their component parts, converting the resulting set of parts
          into whatever form the non-MIME environments uses for
          composite objects. Failure to do so makes the objects unusable
          in any environment that doesn't support MIME. In many cases
          this also means that multi-level MIME structures have to be
          converted into a sequential list of parts.

    (3)   Security services have to be deployed in an end-to-end
          fashion. Failure to do so again can lead to security
          exposures.

          An integrity service deployed at something other than a
          connection end point means a region exists between the point
          where the integrity service is applied and the actual end
          point where object tampering is possible. A confidentiality
          service deployed at something other than a connection end
          point means a region exists where the object is transferred in
          the clear. And worse, distributed private keys are usually
          necessary whenever someone other than the originator applies
          an integrity service or someone other than the recipient
          removes a confidentiality service, which in turn may make
          theft of private key information a possibility.

          All of these issues can be addressed, of course. For example,
          it may be possible to use multiple overlapping security
          services to assure that no exposure exists even though there
          is no end-to-end security per se. And keys can be distributed
          in a secure fashion. However, such designs tend to be quite
          complex, and complexity in a security system is highly
          undesireable.

   The preceeding three requirments are fundamentally in conflict: It is
   possible to satisfy two of them at once, but not all three at once.

   In fact the conflict is even worse than it first appears. In most
   situations of this sort some sort of compromise is possible which,
   while not satisfying any of the requirements completely, does
   optimize some sort of average of all the requirements. Such a
   solution does not exist in this case, however, because many real
   world situations exist where any one of these requirements absolutely
   must be satisfied."

>>  - Messages are reasonably small ( < 20 MB)

>Neither SMTP nor HTTP have length limits unless set as 
>administrative policy.

In EDIINT AS1 in section 5.4.1 Large File Processing, note the risk of large
messages being fragmented by some MTA's. The Pennsylvania Electric utilities
are sending 200-500 MB files. Files this large are vulnerable to
fragmentation during transmission and this could have serious implications
if one of the fragments gets lost/delayed.

>>  - Business process involving the data exchange is NOT time sensitive

>SMTP supports transactions in less than one second, assuming 
>net transmission is <1s and encryption hardware is available.

Yes, here again SO LONG AS NO RELAYS ARE INVOLVED AND COMMUNICATION IS DIRECT.
the truth is most e-mail systems in use by companies employ a store and forward
approach.

>-----

>Note: An SMTP-based EDIINT messaging subsystem can work in 
>a synchronous manner, e.g. 

>status =     
>sendmessage(tp_id,message,max_wait,&reply_buf,&status_buf);

>where status could be
>   0 - succeeded with reply in buf
>   1 - failed with reply in buf
>   2 - succeeded, no reply
>   3 - failed, no reply
>   4 - deferred

>An HTTP-based system would either do the same, or not support 
>the deferred mode.

>Note: The only difference between HTTP and SMTP is that SMTP 
>should normally encode binary data.

Actually there are many differences between HTTP and SMTP:
	- HTTP' MIME specifications differ from SMTP's  ref:section 4 of 
	  EDIINT AS2 revision 4.
	- SMTP does NOT support login/user name access for file uploads
	- SMTP does not support interactive sessions
	- SMTP does not support Java applets
	- and more....

The bottom line is: HTTP is better equipped than SMTP to process
mission critical business transactions.

Dick Brooks                            [email-masked]
Tel. 205-250-8053		       Birmingham, Alabama
Group 8760 LLC                         WWW URL:  http://www.8760.com/
SECURE ELECTRONIC COMMERCE SOLUTIONS FOR HEALTHCARE AND NATURAL GAS INDUSTRIES