Re: A simpler approach to bilateral non-repudiation of EDI trans

[Chuck Shih <chucks@xxxxxxxxxxxxx>]

Gunther,

I'll go over your note in more detail later, but was wondering whether or
not you would be interested in writing an AS#3 that basically dealt with
piggy-backing a response and a receipt -- signed or otherwise? I talked to
Rik D. about this and he is likes the idea.


Gunther Schadow wrote:

> Hi Chuck, Karen, and Carl
>
> This is a lengthy posting, but I intergrated responses to all of
> you. Thank you all for considering my suggestions!
>
> Chuck wrote:
> > I think non-repudiation on a transaction basis is certainly an
> > interesting topic, and one that neither the EDIINT Requirements
> > document, AS#1, or AS#2 directly addresses.  The intent of AS#1 was
> > to define a mechanism for "signed receipts", which is nothing more
> > than a functional message sent from a receiver to a sender with a
> > digital signature.
>
> Thank you Chuck, I feel that you exactky got my point.
>
> > Thus, the sender of the message can not only get delivery
> > notification, but can also verify the returned signature in order to
> > achieve NRR.  AS#1 makes the distinction between the
> > "signed-receipt" and "NRR". NRR is a "legal event" that occurs when
> > the original sender of an EDI/EC message has verified the signed
> > receipt coming back from the receiver. NRR is NOT a functional or a
> > technical concept.
>
> > In some cases as you point out, proving that a party has fulfilled
> > its obligation may require more than just a single message.  It may
> > require that the party prove it has processed a series of messages
> > or a transaction.
>
> And not only that it has processed it, but that it makes a certain,
> complex, domain-dependent statement of what it thinks the results of
> this processing was. E.g. in query mode, answering "What's the time,
> please?" with "O.K. your message has been processed!" is certainly
> incomplete.
>
> > Much of EDI is not built on a request-response model as is HL7. POs
> > are acknowledged by PO Acks or when ASNs are sent back to the
> > original sender of the PO. The correlation back to the origiinal
> > message (for instance the PO) is self-contained in the message going
> > back to the sender (for instance the PO Ack or ASN).
>
> Even though I don't understand your abbreviations "PO" and "ASN", I
> think that security with its notion of a "legal event" makes it
> important to re-think some traditional approaches of EDI (that are
> also prevalent at HL7).
>
> First of all, the policy of "implied application layer acknowledgement
> if no error returned" is difficult. It can not be proofed in a
> positive way: instead this practice effectively needs a
> "non-repudiation of non-origin" of an error message, weird! Thus, a
> proper explicitely positive application layer acknowledgement should
> always conclude an EDI interchange as a "legal event", even if there
> is no information like query results to return. This is a reflection
> of the quest for security that falls back on the EDI standard
> organizations (including HL7) to correct their policies.
>
> Other problematic policies include light-hearted multicasting to
> receivers not known in detail to the sender. This common practice is
> in fact invalidated by the need to send an extra session key to every
> single recipient of a message.
>
> > In essence, AS#1 does indeed provide a mechanism to achieve NRR --
> > though on a message by message basis, and is actually suited to many
> > common use EDI transactions.
>
> What I keep thinking is that the message by message model does not
> meet the very core nature of EDI. EDI is a communication of "legal
> events" be it with security or not. EDI without security is just an
> other level of trust and reliability. But nevertheless, the contract
> according to which I have pay $200.000.000 to you is the "legal event"
> at heart of EDI. In my oppinion, the message by message model comes
> close but does not readily catch it.
>
> An NNR telling "I received and processed your message XYZ" does not
> tell that "I agree with what you said in your message XYZ". Therefore
> an NNR may conclude a message exchange as a "legal event", but it does
> *not* conclude an EDI interchange as a "legal event".
>
> I have the feeling that the EDI community is only about to get this
> right since it cares about technical security. But the mechanisms that
> are prevalent there still need to get raised back onto an abstract
> layer, digested by the EDI standards in order to make them really
> secure and non-repudiateable (not only appear as such).
>
> > However as you point out, using AS#1 for the HL7 request/response
> > model presents two primary problems:
>
> > 1. While there is a secure link between the mail message carrying the
> > EDI message and its respective NRR, the very important link between
> > EDI response message and EDI request message is only very weak. [...]
>
> > 2. Requiring four e-mail messages in order to communicate one
> > transaction is a considerable overhead in e-mail processing and
> > traveling time. Moreover, the probability of failure of the whole
> > transaction due to e-mail delivery problems is twice as high as
> > necessary.
>
> > I think if you defined a multipart/mixed that consisted of the
> > response message as well as the mdn, that this might solve both of
> > these concerns. I think the format for such a "response" is
> > certainly of interest to EDIINT, and would be something that the HL7
> > folks could come up with.
>
> We finally decided to do this in order to be as much conformant with
> EDIINT's AS1 as possible. There are however several alternatives to
> choose from and we are seeking your advice here. The following options
> do exist:
>
> 1. MULTIPART/MIXED
>     the simplest alternative
> 2. MULTIPART/RELATED; protocol="EDI-RESPONSE"
>     this at least allows to make clearly define what this multipart
>     is, i.e. in order to route this mime entity to an edi-response
>     handler.
> 3. MULTIPART/EDI-RESPONSE
>     a consequence of 2. but lifts the protocol up to a higher
>     dignity.
> 4. MULTIPART/REPORT
>     two suboptions:
>
>     1. use APPLICATION/EDI* as the first part instead of the human
>        readable TEXT/PLAIN
>
>     2. use a fourth body part to put the reply APPLICATION/EDI* into.
>
>     Discussion: 1 is more compact, and it is straight forward to put
>     the response in a machine readable format, when machines
>     communicate. However, 1 bends the definition of MULTIPART/REPORT
>     way too much, and an EDI response message is *not* just an other
>     way to express the receipt. Carl suggests to standardize
>     2. However, one can argue that EDI response is not to be subsumed
>     to an MDN.
>
> I tend to the solution number 2 as it is specific enough but respects
> the objective of the MULTIPART/RELATED RFC, which is mainly to not
> extend the namespace of MULTIPART-subtypes too much.
>
> Please if you would comment on this. It is extremely important for us
> to know the direction as we are writing a recommendation to our HL7
> members which should be rather solid in this point.
>
> > I also think that your proposal to add the Received-content-MIC to
> > the MIME headers is not enough -- you must also provide the headers
> > to request receipts, signed or otherwise, and you must also provide
> > the mechanism to convey back error conditions, and not just the MIC
> > of what was received. This is what AS#1 addresses.  However, I think
> > your point of moving some of these headers from the RFC 822 level to
> > the MIME level is a VALID one, and one that in my opinion would not
> > be difficult at this point for the drafts to accomodate. I am
> > curious what the folks on the list think of this.
>
> You might be right that my approach is not yet enough. It might
> therefore be wise to wait, think, and see whether it could be made
> more complete.
>
> > headers to request receipts, signed or otherwise
>
>    This could be implicit. Always do it! but you probably mean that
>    the proper MICALG must be selectable by the initiator?
>
> > mechanism to convey back error conditions
>
>    This is true! I withheld that in order to not earn to heavy
>    rejection. I needed a way to note on the MIME-EDI level whether
>    the response is positive (and thus can conclude the EDI interchange
>    as a "legal event") or negative.
>
>    On the other hand one might fear that this practice eventually
>    unrolls any EDI protocol's message header into the MIME-EDI entity.
>
> And in deed, I now seem to understand Carl's objections better than
> before. The MIME standard as I understand it now, wants to prevent a
> proliferation of different entity headers. Thus it states that only
> those that begin with "Content-" should be guaranteed to pass through
> gateways unchanged. Whether I think this is wise or not (I personally
> doubt that it is), in order to conform to fundamental MIME one should
> not expand the MIME-EDI part in its header area. In fact, the header
> area seems to "belong" to the basic MIME standard and not to the
> standard that might define a MIME media type. Thus a media type
> standard as APPLICATION/EDI* obviously is not allowed to freely define
> new MIME header extension fields. Is this right?
>
> I think it is unfortunate, but it is. However, I doubt that it is a
> better choice to put information about the EDI "objects" into an
> RFC822 header just because this is the only allowed place to define
> new header fields and just because an RFC822 message eventually
> encloses an APPLICATION/EDI entity (somehow, and if not define it as
> a Message-ID in AS2 for HTML). This is not a clean design.
>
> > I also would like to point out, that by moving the headers for
> > "signed-receipt" to the MIME level you still don't truly solve the
> > problem of transaction NRR. For instance as you state in your note,
> > HL7 at a minimum consists of a pair of request-response messages to
> > complete a transaction. By using the mulitpart/mixed we can
> > piggy-back the response and the signed-receipt back to the sender.
> > However, if the "transaction" consisted of 2 request-response pairs,
> > or in perhaps other commerce transactions multiple request-response
> > pairs as well as other independent messages, you still cannot
> > achieve NRR without "piecing together" all of these messsages or
> > message pairs. Again, transaction based NRR is an interesting,
> > though in my opinion slippery subject to deal with in a very general
> > way.
>
> Therefore you should read my draft-schadow-edirev-01 document posted
> later on that tries to approach exactly this. Instead of many words I
> repeat a figure from that document. Does it come close to what you
> mean?
>
>          +------------+      +------------+      +------------+
>          |EDI MESSAGE |      |EDI MESSAGE |      |EDI MESSAGE |
>          |message id  |<--+  |message id  |<--+  |message id  |
>          |replies to  |   +--+replies to  |   +--+replies to  |
>          |transaction |      |transaction |      |transaction |
>          +----------+-+      +------+-----+      +-+----------+
>                     |               |              |
>                     +---------------+--------------+
>                                     |
>                                     v
>                               +-----------+
>                               |TRANSACTION|
>                               |trnsact. id|
>                               +-----------+
>
> This could well be expanded further such that nested transactions
> could be expressed. HOWEVER: it is the very nature of traditional EDI
> to do "coarse grained" communication. That means, multiple nested
> transactions are usually not necessary. One EDI message exchange
> normally is a complete self-consistent transaction. This is very
> different in database transactions, that are usually fine-grained and
> nested. Therefore the term "transaction" is often misunderstood as I
> use it in EDI. I do not mean nesting, two-phase commit etc. at the
> first place. I'd like this term to be interpreted more closely to its
> origin: a "financial transaction", a kind of a "contract".
>
> Karen noted:
>
> > I think moving the receipt specific RFC822 headers to the MIME
> > header level is a good idea.  It allows for receipt requests at the
> > attachment level, and avoids the message/partial confusion inherent
> > in the MDN spec.
>
> Do you think that it is allowed to extend MIME body part headers? I
> agree that it is necessary as it cleans up some anomalies with partial
> messages or multiple APPLICATION/EDI* per RFC822 messages (a batch),
> but I doubt now that it is allowed to extend MIME header areas.
>
> Chuck asks Karen:
>
> > Any opinions on the added complexity in processing?
>
> I think that the processing is even simpler, since you take clean up
> any metaphoric interpretation that EDI makes for RFC822 headers. An
> EDI message would thus be completely independent from the context of
> RFC822 messages. This makes it much easier use MIME-EDI it over HTTP
> without the need to draw Message-IDs and the like from RFC822.
>
> And finally Carl made a statement that I agree so much that I want to
> repeat it in a box :-)
>
>  +--------------------------------------------------------------------+
>  | [...] things that are application specific move to the application |
>  | MIME layers, things that are generic to messaging move up to       |
>  | application independent layers.                                    |
>  +--------------------------------------------------------------------+
>
> And he continues:
>
> > I'm a little confused about what the goals are here with respect to
> > receipts and add-ons to the MIME headers. The issues raised are good
> > ones, but several have been lumped together. Please correct or add to
> > this list:
> >  1 - Including an application level ack (e.g. X12-997FA) in the same
> >      message as the MDN/receipt. (Acks are not ack'd)
> >  2 - Reducing the messaging by including a MDN with a response
> >      message (e.g. Invoice in response to PO) (receipt/ack required)
> >  3 - Application level ID included in a receipt
> >  4 - Connecting an application response to the source message
> >  5 - Multiple application data parts in a single message
> >  6 - Non-repudiation
>
> Thank you, this is a good summary!
>
> I summarize (parts of) what you said by
>
> * achieve goal 2 through 1
>    - 1 can also achieve 4
>
> * the legal event is recorded properly in logs that repeat the
>   original signed messages and the signed receipts.
>    - right, if the reply tracking is secure, and it seems to be as
>      the Original-Message-Id and the Received-Content-MIC within
>      the MDN does it and provided that you *always* piggy-back EDI-
>      responses with the MDNs and sign both at once.
>
> * batch transfer is possible with MULTIPART/DIGEST that encapsulates a
>   number of RFC822 messages.
>    - It becomes clear to me now, that you intend to treat all that is
>      message-like in EDI by means of RFC822 message handling. Thus you
>      argue:
>
> * other multiparts than DIGEST should not be used to paste multiple
>   EDI messages together.
>    - especially no MULTIPART/RELATED; PROTOCOL="EDI-BATCH";
>
>    - however, I think the "Content-Id" mechanism is there to be used
>      by EDI messages.
>
> * you fear that too much of protocol specific issues are unrolled into
>   the general EDI container, and the application message ids should
>   probably not repeated in Mail or MIME headers.
>    - I agree! I think it is better to issue new Message- or
>      Content- IDs on the MIME/EDI messageing system.
>
>    - the problem is that basic MIME seems to forbid defining something
>      like In-Reply-To in MIME headers. Did I really get that right? No
>      way for an MIME-extension-header?
>
> * but you agree to the point that it could make sense to enable the
>   EDI/MIME message handler to some transaction tracking.
>
> > For the application specific ID, I don't understand the precise
> > semantics enough to understand the relationship beteween the
> > generalized application-independent messaging and the application
> > data processing. What is the general concept?
> >
> > I suppose it makes sense to say "this application data is a response
> > to Message-ID XXXX", e.g. an In-Reply-To: header.
>
> For example, I am currently prototyping such a message handler that is
> independent of HL7. There is a functionality that "translates" between
> synchronous messaging that an EDI application might expect to
> asynchronous messaging that E-Mail introduces. Now, what I do is to
> write a transaction-control-record on disk whenever an EDI interchange
> is initiated locally. This lists the Message-ID and Content-MIC and a
> process ID of the process that falls asleep short after the initiating
> message has been sent...
>
> For any incoming message, the message handler looks whether it is a
> piggy-backed MDN. If so, it extracts the Original-Message-ID and
> Received-Content-MIC from the MDN and tries to find a
> transaction-control-record for that response. If it finds one, it gets
> the ID of the sleeping process and then, guess what it does? It saves
> the reply message on disk under the id of the waiting process, and
> then sends the sleeping beauty a kiss SIGIO. That process reads the
> message and returns it to the EDI application, which was blocked all
> the way reading from some bidirectional pipe.
>
> I can do this without In-Reply-To, however, I require that any such
> response message must be piggy-backed on an MDN.
>
> > A relevant issue is the layering with respect to encryption and
> > signatures. I don't think AS#1 allows this, but there is no reason
> > not to allow a sign then encrypt of a multipart/digest. Each part
> > would have a separate Message-ID and receipt request. Is this what
> > you were trying to accomplish? If each application data has an
> > associated Message-ID, then the In-Reply-To works to correlate
> > responses (for logging) in addition to the message disposition
> > receipts.
>
> For "secure logging" it is important to be able to signe
> Message-IDs. So, a normal In-Reply-To field in a RFC822 header is not
> secure. But the MDN's Original-Message-ID is.
>
> > To summarize, I don't see a case where new headers are needed. We can
> > extend AS#1 slightly by allowing cases of already defined MIME
> > headers to be used to reduce excessive messaging. A single signature
> > and receipt should be used for multiparts, except for digests that
> > can be sent to multiple final recipients.
>
> Now that I understand you motivation I can accept this. You want to
> adhere to RFC822 messages as much as the message-nature of EDI
> allows. You do not want a MIME EDI "object" that encapsulates all its
> attributes within the MIME-EDI entity.
>
> I followed that object-oriented interpretation of MIME, which is
> probably not what MIME really is -- therefore I love MIME less than
> before.
>
> > (Note that AS#1 could also suggest the use of the SMTP TURN command,
> > so the receipt/response could be returned in the same TCP session.)
>
> Thank you, I didn't know that.
>
> best regards
> -Gunther
>
> Gunther Schadow-----------Windsteiner Weg 54a, Berlin 14165, FR. Germany
> Dept. of Anaesthesia, Benjamin Franklin Univerity Hospital, Berlin.
> gusw@xxxxxxxxxxxxxxxxxx               http://userpage.fu-berlin.de/~gusw
> ----------------------------------#include <usual/disclaimer>-----------