From: Bruce Lilly (blilly@erols.com)
Date: Wed Sep 11 2002 - 12:45:26 CDT
Charles Lindsey wrote:
> In <3D7E8948.5020900@alex.blilly.com> Bruce Lilly <blilly@erols.com> writes:
>
>
>
>>Charles Lindsey wrote:
>>Having done some extensive parsing work, I can state that there's
>>no parsing ambiguity. There should be no problem with Complaints-To
>>or Mail-Copies-To -- those are newly defined in the Usefor draft, so
>>they could include MIME-style parameters if there were a need to do
>>so.
>
>
> The ambiguity arises if the 'group' syntax from RFC 2822 is used (it never
> is, of course, but we have to be prepared), beacuse it includes a ';'.
> Yes, I know it can be parsed by a suitable Turing machine, but it is a
> bloody nuisance, which is one of the reasons why we removed the insistence
> on doing it.
There is no ambiguity even with a simple parser -- a named group always
has a colon and semicolon bracketing the grouped mailbox list. If a colon
is encountered (after the one which ends the field name, of course) in a
header which might contain a group, one is "inside" the list, which is
then terminated by the corresponding semicolon.
>>But headers defined in other RFCs should not be [re]defined incompatibly
>>in our draft.
>
>
> There is no reason why headers should not be defined differently in mail
> and news. It is simply a question of wether it is wise or not in each
> particular case.
There are very good reasons. As you pointed out recently:
> OTOH, we have had a policy from the start of removing gratuitous
> differences between news and mail headers, even though we have said "MUST
> accept, SHOULD NOT generate yet" in most cases. That seemed to be the only
> way that things that needed changing would "eventually" get changed.
>
> And I think we have to stick with that policy where we can.
> The news headers are already a superset of the mail ones,
> because we allow UTF-8.
That is a circular argument. Currently, RFC 1036 applies and that
does not permit any 8-bit codes. The 8-bit incompatibilities also
need to be addressed; see separate message posted 10 hours ago.
> Essentially, we took a conscious decision to place
> the burden, if any, on the gateways, in order to achieve a cleaner product
> within the news environment (which is our primary concern).
Gateways are irrelevant to post-and-mail. And the "news environment"
*requires* use of email, so the incompatibility must be resolved.
>>The incompaibility arises whenever such a header might be injected
>>into its native domain, viz. email. I mentioned three such
>>mechanisms earlier, and I now add a fourth; the four are:
>>1. post-and-mail
>>2. incoming gateways from email (also when a proto-article is sent
>> to a moderator via email w/o encapsulation)
>>3. outgoing gateways
>>4. via intermediate files. A usenet article may be saved as a file
>> and subsequently forwarded as email (or some software (e.g. combined
>> news reader/email program) might do so w/o an intermediate file)
>
>
> Yes, these cases all need to be taken care of in the proper places
> (gateways, for example). The draft draws attention to what needs to be
> done. Actually, I don't think there is much problem with (2), and (4) is
> simply a complicated way of going gatewaying. He who hacks around with
> files bears the responsibility for doing it right.
If the MIME-style parameters are permitted in article headers which
are adapted from email, where those parameters are not permitted,
case 2 poses a problem; it requires that *every* injection agent be
capable of parsing and removing the MIME-style parameters when emailing
to a moderator without encapsulation. There *are* injection agents
without that capability, and they cannot all be changed overnight.
Case 4 (w/o intermediate files) arises in combined news/mail user
agents, which usually permit forwarding an article as email.
> The following cases are the ones you have asked to be changed. They need
> to be decided on an individual basis, but so far I have heard no support
> for your request. So hands up, please, anyone who wants to exclude
> MIME-style parameters in these cases (or from anyone who wants to retain
> them, for that matter).
>
>
>>>So there is an arguable case for Sender.
>>
>
>>Sender is defined in RFC 2822, section 3.6.2.
>
>
>>>We have already removed it from Message-ID.
>>
>
>>Hmmm. In response to a request for the full draft, Charles wrote
>>(on Aug 2):
>
>
>>>>Is this draft available in full somewhere?
>>>
>
> The current draft is draft-ietf-usefor-article-08.txt, which is available
> both on the Landfield site and from the usual internet-drafts sources. You
> have been looking at an earlier one.
>
>
>>>There is an arguable case for Date.
>>
>
>>Date is defined in RFC 2822 section 3.6.1.
>
>
>>>I am not convinced there is a serious problem wuth Keywords.
>>
>
>>Keywords is defined in RFC 2822 section 3.6.5.
>
>
>>>And I am not convinced about References.
>>
>
>>References is defined in RFC 2822 section 3.6.4.
>
>
>>And Supersedes is defined in RFC 2156, and has no provision
>>for MIME-style parameters. The Usefor syntax for Supersedes
>>(excepting the paramters) is a subset of what 2156 permits --
>>if parameters are removed from the specification in the draft
>>there won't be any problems in cases 1 and 3 (and presumably
>>incoming gateways and injection agents will deal with any
>>email-originated Supersedes headers which have multiple msg-ids).
>
>
> No, we make it clear in 6.15.that our Supersedes header has no connection
> with that defined in RFC 2156 (which is provided purely for use with
> gateways from X-400 mail). The semantics are totally different. We
> discussed and resolved this issue several months ago.
Saying so doesn't make it so. If the field name is the same (it is),
there is a connection. If Suersedes ever appears in email, it is
interpreted in accordance with RFC 2156. And the semantics are
not "totslly different"; boith provide for the message containing
the header to supersede previous message(s) with the msg-id(s)
given in the header content. If there's no pressing need for
MIME-style parameters, they should be removed from the syntax for
Supersedes in the draft, which will resolve the incompatibility.
>>>My understranding is that it is already not allowed for CTE and any other
>>>MIME headers that do not allow it, since we simply incorporate the
>>>relevant MIME documents, modulo various trimmings we explicitly mention.
>>
>
>>The text of section 4.2.2 seems to imply that MIME-style parameters
>>may be added willy-nilly to these headers.
>
>
> OK, that could be clarified.
>
>
>>A related concern is that 6.21.1 specifically states that those
>>headers which *are* defined such as to accept parameters may be
>>generated in such a way as to be imcompatible with the syntax
>>in the defining standards (specifically UTF-xtra-chars). That
>>should not be done, as the MIME RFCs specifically provide a means
>>of encoding 8-bit characters in parameters (see RFC 2231 and
>>the RFC errata page at http://www.rfc-editor.org/errata.html).
>
>
> It is generally the case that our draft allows either UTF-8 OR RFC
> 2047/2231 in all relevant places (with no bias either way - it is a matter
> for the market place). Of course, in the case of posted and mailed and
> similar situations, the sensible approach might well be to use RFC
> 2047/2231 for both, but that is up to the implementor.
It's not merely a matter of what's "sensible"; if the headers appear
in email, they MUST comply with the email RFCs (2822, MIME, and
relevant extensions) which means no UTF-8. That cannot be left up
to the implementor; it is a matter of compliance with the email
protocols as established by the relevant RFCs.
Moreover, if the draft, as it states, incorporates MIME, then the
MIME standards must be incorporated as they exist, not with
unofficial (w.r.t. the cognizant MIME WG) extensions or exceptions --
it is not within the Usefor WG's charter to tinker with the MIME
standards. And 2231 is clearly part of the MIME standards and
provides a suitable encoding mechanism for 8-bit parameter content
which is not permitted in raw UTF-8 by any of the MIME standards.
There are no proposed uses for the MIME-style parameters in the
draft for the headers mentioned above. Therfore the situation
is that the draft as it now stands introduces a new incompatibility
with email via those parameters *and* imposes requirements on news
software which is known not to be implemented by some (most) current
software, yet yields no real or potential benefit.