From: Charles Lindsey (chl@clw.cs.man.ac.uk)
Date: Tue Jan 07 2003 - 17:29:40 CST
In <138AA78F80DCE84B8EE424399FFBF9C904F9DB@exchange.ad.skymv.com> "Dan Kohn" <dan@dankohn.com> writes:
>I wrote:
>>> I have a simple question. What can a UTF-8 subject header
>>> communicate that an RFC 2047 one can't? Other than inelegance,
>>> what's the downside of 2047, when the upside is a huge increase in
>>> backward compatibility?
>Charles Lindsey responded:
>> No, it's just the inelegance. Plus the fact that the backward
>> compatibility issue is nowhere so huge as you imagine. In fact, it is
>> rather small.
>Here, I believe, is the crux of the disagreement between the working
>consensus of usefor and the unanimous agreement of email folks from
>ietf-822.
>Usefor recently conducted a straw poll [1] in which raw UTF-8 headers
>barely won out [2] over using 2047/2231. (Specifically, in the #1 vs.
>#5 vote, the tally was 9 to 8.) The chair, Dave Barr, declared "rough
>consensus" [3] and suggested the group should move forward on that
>basis.
OK, time to give some background as to what has been going on.
1. We conducted a straw poll and the conclusion reached was that we would
proceed on the basis that:
a) We would continue to use UTF-8 in headers, with RFC 2047/2231 as an
alternative (and special considerations for newsgroup-names).
b) Whenever anything had to go via Email, there MUST be transformations so
that the Email is compliant (the transformations would involve RFC
2047/2231 and 5.5.2).
2. As part of the implementation of that decision, I wrote some text on
those transformations and posted it here. There has been no comment here
on that text.
3. Because I wanted to be sure that the transformations would indeed
produce compliant email, I also posted that text to the ietf-822 mailing
list (RFC 2047 is a difficulat beast to comply with, and there were some
awkward corners I had to address).
4. All Hell broke loose, much of it based on gut reaction rather than
careful appraisal of what I was proposing. You need to understand that the
ietf-822 list is peopled by the authors of the MIME-standards, and by
those who claim to have great knowledge of, and influence on, the IESG.
5. And so there were predictions, prophecies and threats that, it we
proceeded on our present course, it would never get past the IESG. There
were essentially three issues:
6. Issue #1. My transformation for the awkward corners (which were
advisory rather than obligatory) could lead to non-compliant email. This
is true, but only in the event of a long chain of improbable happeninggs.
I considered my suggestions to be better, on average, than the alternaitve
which was to drop information on the floor. But I have no problem with
revisiting that part of the proposal.
7. Issue #2. News->Mail gateways that had not upgraded to our standard
would fail as soon as they saw some UTF-8 character, because they would
not know how to do those transformations. I tried to explain that this was
a small and containable problem, because GENERAL-PURPOSE gateways from
news->mail do not exist. By "GENERAL-PURPOSE" I mean a gateway that is set
up to handle input from _every_ group on Usenet and to produce
corresponding email. The gateways that are around (and there are not that
many of them - a few hundred maybe) are all SPECIAL-PURPOSE, being
created to gateway particular newsgroups for particular classes of people
(usually a mailing list). Since the language used by most of them is
English and the charset is ASCII, they are not going to notice. In the
cases where it may matter, they will start to see a small trickle of
articles with strange things in their headers, whereupon they may decide
to tolerate them or else decide that it is time to update the gateway. My
point is that the problem is _containable_. But I completely failed to get
across to them the sifnificance of the phrase "GENERAL-PURPOSE" (I had not
thought it necessary to shout it then) and so my arguments were falling on
deaf ears. But I still think this problem is manageable.
8. Issue #3. This is the real crunch. We have destroyed the purity of
MIME, by permitting UTF-8 characters in two places:
a) in parameters in some of the Content-* headers
b) in the headers of message/rfc822 objects (because we propose to use
that Content-Type rather than the to-be-obsoleted message/news for Netnews
articles that get encapsulated within other Netnews articles).
Note that in both cases, those relaxations are ONLY to be allowed within
Netnews articles - if you want to move that article, or that
message/rfc822 object, into mail, then you MUST transform as above. I have
tried to claim that the existing MIME definitions apply only within Email
(see RFC 2045) and that it is therefore lawful to augment them for use
within Netnews only (and subject to transformation before gatewaying to
mail). I have also pointed out that the HTTP spec was allowed to take
similar liberties with some MIME definitions. But they will have none of
it.
>I suggest to the usefor chair that the group should conduct the poll
>again, based on a new piece of information. Specifically, they've seen
>the applicable area director react in such a viscerally negative fashion
>[4] that it is nearly impossible to imagine anything resembling the
>current usefor approach being approved by the IESG. I.e., the backward
>compatibility issue is nowhere near as small as the group seems to
>believe.
>(Ned, if I'm overstating your viewpoint and its implications, my
>apologies, and please correct my error.)
No, Ned is NOT the applicable Area Director (though he is a member of the
IESG). The AD is Pat Falstrom, who has not yet appeared in the dicussion.
I think a sueful next move would be for our Chairman to speak to Pat.
>Based on this new information, I would suggest that the group consider
>rechartering with a more precise plan and schedule, both approved by the
>IESG, in such a matter that successful publication of a standards-track
>RFC is a likely (or at least possible) outcome.
>Alternatively, if the group decides it's not interested in the IESG
>imprimatur, I'd suggest that the group consider reforming outside the
>aegis of the IETF.
Yes, that is one possibility, but there are many other possibilities.
One is to press ahead, and deal with the IESG when we have to, which is
not yet.
Another would be to write the standard for RFC 2047/2231 only (but leaving
newsgroup-names with 5.5.2) and then immediately to issue an extension
allowing UTF-8 as an Experimental Protocol (the marketplace would then
determine whether or not to join in the experiment).
And there are doubtless many other possibilities.
But the fundamental problem remains this: There is a tension between the
"elegant" solution and the "backward-compatible" solution. The IESG has,
in the past, always favoured the "backward-compatible" solution for fear
of offending some piece of legacy software out there. That might be a fine
thing, but it sure is a bar to progress in the long-term. And usenet, as
an institution, does not have to be 100% perfect. It is much more able to
tolerate the occasional backward-compatibility failure. More so than email
would be, and certainly more so than DNS (which is why we are now faced
with IDNA).
-- Charles H. Lindsey ---------At Home, doing my own thing------------------------ Tel: +44 161 436 6131 Fax: +44 161 436 6133 Web: http://www.cs.man.ac.uk/~chl Email: chl@clw.cs.man.ac.uk Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K. PGP: 2C15F1A9 Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5