RE: RFC 1426 problem and solution

[Ned Freed <NED@xxxxxxxxxxxx>]

> RFC 1426 specifically states (page 3):

> "A server which supports the 8-bit MIME transport service extension
> shall preserve all bits in each octet passed using the DATA command."
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

> So any implementation that replies with the EHLO keyword 8BITMIME
> must store the message with the body intact.  If it doesn't, then it
> is broken and shouldn't be used.  The standard is there to force all
> implementations to work correctly.

I'm sorry, but the ability to use 8BITMIME as a means for binary transport
simply does not follow from this. (I speak as the editor of the specification
here.) RFC1426 is dealing with MTA to MTA communications. That's all. We
felt that asking MTAs to preserve the binary contents of messages was a good
idea. But there is much, much more to Internet mail than MTAs. For example, the
process of delivering an 8BITMIME message to an MUA is not covered by any of
this, yet this operation is an indispensible part of the handling of every
message.

An agent, be it MTA or MUA, receiving messages sent with the 8BITMIME option
expects to see line-oriented material with CRLF sequences between the lines.
Failure to adhere to these restrictions will cause all sorts of
interoperability problems. Binary files may contradict both the expectation
that line break will occur at reasonable intervals *and* the expectation that
the message will always end in a CRLF. Both a new option and a new DATA verb
would be needed to transfer binary objects correctly. (The latter has in fact
been proposed and written up but has never been turned into an actual RFC.)

> As long as the standard is adhered to, it doesn't make any difference
> if there are bare LF or CR characters since they will be there when
> you pick up your mail.

If you want to restrict your transfer of binary to the few binary objects that
end in CRLF and always have a CRLF at least once every 1000 octets, you are
free to attempt to use extended SMTP as a transfer protocol. However, this is
so limiting it is for all intents and purposes useless.

And as a purely practical matter you are going to find that CRs and LFs will be
turned into CRLFs and vice versa. The reason for this is simple: A nontrivial
number of SMTP clients on the Internet use bare LF instead of CRLF as a line
terminator. (This has its origins in some improper sendmail configuration files
that were widely promulgated many years ago.) Because of this most other MTAs
accept LF as an alternative terminator to the standard CRLF. The choice is
quite simple: Comply with the specifications and fail to interoperate or
violate the specifications and interoperate quite nicely. In the real world
customers purchase products on the basis of their ability to interoperate. I
can tell you from personal experience trying to sell a product with a
"standard" SMTP server which choice people make and what we ended up doing.

> > and the ability to apply a 
> > content-transfer-encoding when relaying a binary message to a
> > site that didn't support binary transport.

> This is also discussed in RFC 1426.  The sender has the option of
> either returning the message with an error (easy to implement in
> the short term), or performing some kind of MIME compliant trans-
> formation (to base64, etc.) to convert the message to 7bit MIME.

This is presented as an option but RFC1426 doesn't specify how to do it. And
doing it is far from trivial. I've written support for on-the-fly MIME encoding
conversion, so I know what I'm talking about here.

> As more and more sites switch to full 8-bit capability, the con-
> version option will be more desirable to keep communications going.
> And in the future when everybody has 8bitMIME-compatible mail with
> no line length restrictions we will all be much better off.

I think the chances of this occuring are so remote as to be not worth the
bandwidth to discuss. You are talking about implementors somehow heeding a
higher call that appears in no published document anywhere to remove a
restriction that makes implementations substantially simpler? And even if such
a thing were to happen, a single solitary holdout who didn't remove the
restriction can point to the specifcations and say, "I'm compliant and you're
not when you send me this stuff. Stop doing it!"

If you want to propose a negotiated pure binary transfer option together with a
new DATA verb feel free to do so. Who knows? It may even see deployment and
implementation. But don't attempt to overload an existing mechanism that
was never intended for pure binary transfer -- it simply isn't up to the
strain.

				Ned