From: Bruce Lilly (blilly@erols.com)
Date: Fri Mar 15 2002 - 15:08:44 CST
Charles Lindsey wrote:
>
[...]
> In Section 4, regarding parameter (both ours, and any we inherit from
> MIME)
>
> [Bruce Lilly
> What about RFC 2231 chars used in attributes, esp. with continuation and
> charsets: "*", "'", "%"]
>
> Well does any Mail system actually implement RFC 2231? My feeling is that
> we just don't want to know, so I have written an extra paragraph to follow
> the existing paragraph about RFC 2047 in 4.4.1 (both paragraphs reproduced
> below). I probably need to say something in gateways when I get there.
>
> Where the use of non-ASCII characters, encoded in UTF-8, is permitted
> as above, they MAY also be encoded using the MIME mechanism defined
> in [RFC 2047], but this usage is deprecated within news articles
> (even though it is required in email messages) since it is less
> legible in older reading agents which support neither it nor UTF-8.
> Nevertheless, reading agents SHOULD support this usage, but only in
> those contexts explicitly mentioned in [RFC 2047].
>
> Similar considerations apply to non-ASCII characters within the
> values of parameters (which, according to the syntax, MUST be in the
> form of quoted-strings in order for UTF8-xtra-chars to be
> accomodated). There is NO requirement to support the extensions set
> out in [RFC 2231] for specifying continuations, character sets or
> languages in such values, though reading agents MAY support them.
>
> Is that OK?
>
> [RFC 2231] N. Freed and K. Moore, "MIME Parameter Value and Encoded
> Word Extensions: Character Sets, Languages, and COntinuations",
> RFC 2231, November 1997.
I don't believe it is OK to say there is no requirement to support
RFC 2231; 2231 revises the ABNF of 2045, is a standards-track RFC,
and is listed in the rfc-index as updating RFC 2045 (among others).
Any system supporting MIME must therefore support 2231's extensions.
Any agent, not limited to readers, which must interpret parameters must
be able to correctly parse a header using 2231's extensions. As there
is a catch-all in the draft regarding parameter use with headers in
USENET articles, that essentially means that all agents have to
support all of MIME including the 2231 extensions.
> Section 5 - Path
>
> [Bruce Lilly
> ":.-_" is a valid path-identity Perhaps path-identity should be
> constrained to begin with ALPHA or DIGIT.
>
> An upper bound (presumably no more than 73 characters) on the length of
> path-identity and tail-entry should be specified.]
>
> I agree with the first, and I now have
>
> path-identity = ( ALPHA / DIGIT )
> *( ALPHA / DIGIT / "-" / "." / ":" / "_" )
>
> Any objections?
>
> But I see no point in a limit (short of that 998). This header is for
> machines to read. We do encourage it to be folded (because humans
> sometimes like to peer at it, but that is a secondary use). Fixed numbers
> are, generally speaking, a Bad Thing.
>
> 0.0.1. Adding a path-identity to the Path-header
>
> When an injecting, relaying or serving agent receives an article, it
> MUST prepend its own path-identity followed by a path-delimiter to
> the beginning of the Path-content. In addition, it SHOULD then add
> CRLF and WSP if it would otherwise result in a line longer than 79
> characters.
There already exists the 79 character limit (immediately above). As
there is no provision for line folding before the first path-identity,
the 73 takes into account the Path field name and the following colon
and space.
Other limits are implied: use of a FQDN requires adherence to the DNS
RFCs (1034 and 1035), which limit the length of a component (atom in
RFC 2822 terminology) to 63 characters and the total length of a domain
name to 255 characters.
[...]
> [Bruce Lilly
> RFC 820 cannot be correct. It was obsoleted by RFC 870 which was
> obsoleted by RFC 900 which was obsoleted by .... RFC 1700 which was
> obsoleted by RFC 3232. The correct RFC should appear in the
> references.]
>
> Well RFC 1700 obsoletes 820, 870 and 900, but RFC 3232 does not obsolete
> 1700 (it is only an experimental protocol). But RFC 1700 does not define
> the "dotted-quad" notation (though it clearly uses it). In fact, it seems
> to be one of those things which "everybody knows", because I cannot find
> a formal definition anywhere :-( .
From RFC 3232:
=====================
Network Working Group J. Reynolds, Editor
Request for Comments: 3232 RFC Editor
Obsoletes: 1700 January 2002
Category: Informational
Assigned Numbers: RFC 1700 is Replaced by an On-line Database
======================
I have also suggested RFC 1123, section 2.1 as a reference for IPv4
dotted quad definition.
[...]
>
> [Bruce Lilly
> RFC 1036 section 2.1.6 treats ':' , '_', and whitespace as delimiters.
> Path-identifiers ("systems") consist solely of letters, digits, hyphens,
> and periods. To avoid interoperability problems, a transition period
> should be provided where:
> 1. use of ':' and '_' as delimiters is deprecated (via SHOULD NOT)
> 2. use of ':' or '_' in path-identifiers is discouraged (also via SHOULD
> NOT)
> 3. use of whitespace as sole delimiter is deprecated (but not forbidden).
> 4. whitespace separating path- identifiers MAY be interpreted as a
> delimiter (for backward compatibility)
> 5. ':' and '_' MAY be interpreted as delimiters (for backward
> compatibility) Note should be made that future revisions may require the
> specific delimiters mentioned
>
> No! We discussed this a long way back. It is quite clear that most
> relayers support arbitrary punctuation as delimiters, as required by RFC
> 1036, so our introduction of special meanings for '%', '/' etc does not
> interfere with current practice. OTOH, no relayer that we know of uses
> other than '!' for a delimiter, therefore no harm can arise if we assign
> some of the former delimiters to other uses.
>
> We HAVE to allow ':' within path-identities because IPv6addresses are
> surely going to appear in them (though probably not in the very short
> term). There is the slight possibility that an existing relayer will
> interpret a ':' in an IPv6address as a delimiter, but the worst that can
> lead to is that a site may sometimes be offered an article it has already
> got. We can live with that.
The objective is assured interoperability. If your group is
omniscient and can therefore guarantee that no implementation,
present or in the future up to the official transformation of
the draft into an RFC, uses the options available in the current
official standard (viz. 1036), then fine. Otherwise, a transition
period is highly recommended. N.B. 1036 DOES permit whitespace as
the sole delimiter.
> Section 6.
>
> References:
>
> [Bruce Lilly
> Any restrictions on where an agent may break or unfold a long References
> line?:
> when adding a reference
> when trimming
> when relaying (e.g. if there are very long lines)]
>
> I see no problem. A followup agent may fold the References line anywhere it
> likes (I see no requirement to use the exact folding in the References
> line of the precursor). Of course, a relaying agent MUST NOT refold
> anything.
Even if an old followup agent which does not fold lines supplies
a line longer than 998 characters? If injection agents may not
refold such long lines, then injection agents should probably be
required to reject articles containing such long lines. But then
what about "old" (i.e. existing) injection agents which neither
refold long lines nor reject articles with excessively long lines?
What should relaying agents do when encountering an excessively
long line? The draft has no provision for this in the requirements
for relaying agents. What should they do?
> Subject lines starting with "cmsg"
>
> NOTE: The presence of a Subject-header starting with the string
> "cmsg " and followed by a Control-content MUST NOT be construed,
> in the absence of a proper Control-header, as a request to
> perform that control action (as may have occurred in some legacy
> software). See also section 5.4.
> [Bruce Lilly
> RFC 1036 section 2.2.6 recommends exactly what this draft forbids. An
> overnight change is impractical; a transition period where the cmsg
> interpretation is recommended against (i.e. SHOULD NOT rather than MUST
> NOT) should be provided, with a warning to implemetors that a future
> revision may change that to MUST NOT. And the requirement or
> recommendation itself should be moved from the NOTE into the body of the
> document.]
>
> I think we were agreed that the practice in question was so abhorrent, and
> so likely to be misused by trolls and other maldoers, that its use had to
> be curtailed immediately. No proper current practice is affected because
> no legitimate issuer of control messages in his right mind uses it.
> Moreover, all decent newsreaders make provision for a user to cancel his
> own articles with a proper cancel message, and anyone who might
> legitimately want to issue a 3rd party cancel and doesn't know how
> probably shouldn't be doing it anyway.
The issue here is again interoperability, and all that applies
w.r.t. Path delimiters applies in this case as well. Abhorrent
it may be, but the practice is strongly recommended by the
current standard (1036).
> Supersedes:
>
> header =/ Supersedes-header
> Supersedes-header = "Supersedes" ":" SP Supersedes-content
> *( ";" other-parameter )
> Supersedes-content = msg-id
> [Bruce Lilly
> The original official Internet definition of Supersedes as it appears in
> the defining document (RFC 2156) is: "Supersedes" ":" 1*msg-id That
> uses RFC 822 syntax, so CFWS is permitted (but not required) between
> msg-ids.
>
> This document should be consistent with that definition by permitting
> more than one msg-id.
>
> Since Supersedes was not in RFC 1036, there is no reason to require CFWS
> between msg-ids (as is claimed for References).]
>
> I would not regard RFC 2156 as "the" definition of Supersedes for Netnews,
> though it is for Email. I have added a NOTE:
>
> NOTE: The Supersedes-header defined here has no connection with
> the Supersedes-header that sometimes appears in Email messages
> converted from X.400 according to [RFC 2156]; in particular, the
> syntax here permits only one msg-id in contrast to the multiple
> msg-ids in that Email version.
>
> [RFC 2156] S. Kille, "MIXER (Mime Internet X.400 Enhanced Relay):
> Mapping between X.400 and RFC 822/MIME", RFC 2156, January 1998.
As the introductory parts of the draft correctly point out, for a
considerable time, USENET articles have also been valid text messages
as defined in RFC 822 (now obsoleted by 2822). Because many clients
interpret both email and USENET articles using the same code, it is
highly undesirable that there should be a conflict between RFC 2156
(or any other RFC which defines a header) and this draft. Consider
also mail-to-news gateways.
What is the harm in permitting multiple msg-ids in Supersedes in
USENET articles?
If a header having the functionality of Supersedes is required,
but with a single msg-id, another header field name should be
chosen; "Supersedes" is already taken (tough luck for implementations
that did not follow proper procedures as outlined in RFC 822 sections
4.7.4 and 4.7.5, the latter of which clearly states that user-defined
fields other than extension fields may be preempted; exactly what has
happened with Supersedes).
What should a mail-to-news gateway do if it receives ad article
(as email) with multiple msg-ids in a Supersedes header? What about
injection agents (in case of a message emailed to a moderator)? These
cases do not seem to be covered by the draft.
I suspect that the best course of action, all things considered, is
to use the same syntax (albeit conforming to current 2234/2822 ABNF)
as RFC 2156 for Supersedes.
> The posting-date-parameter:
>
> This parameter identifies the time at which the article was injected
> (as distinct from the Date-header, which indicates when it was
> written). It is in the form of the number of seconds elapsed since
> January 1st 1970, optionally followed by a date-time which MUST
> indicate the same time.
> [Bruce Lilly
> seconds elapsed since Jan 1 1970 IN WHICH TIME ZONE?]
>
> I think that is taking pedantry too far (he will be wanting me to exclude
> those leap seconds next :-) ). If somebody can give me a reference to the
> proper POSIX document, I will mention it. Otherwise, it is a thing that
> "everybody just knows".
There are several issues:
1. In principle, a standard should be as self-contained as practicable.
References should be given where it is not practical to include
precise details in-line. The issue is not pedantry, but precision:
the standard should be unambiguous.
2. Time zone is crucial, since the number of seconds elapsed since
some specified time of day on 1 Jan 1970 in my time zone may
or may not be different from the number of seconds elapsed since
that time of day on 1 Jan 1970 in yours. This is of course
further complicated by Daylight Savings Time (by that or any
other name).
3. 1 Jan 1970 was, like most days, a 24-hour day. If you mean "since
00:00:00 on 1 Jan 1970" you should say so; failing an explicit
specification, one might reasonably assume 12:00:00, i.e. mid-day.
4. If you wish to refer to ISO/IEC 9945-1 (a.k.a. IEEE 1003.1)
section 2.2.2.77, then do so. But beware; there is an error
there in handling leap years (a y2k bug). Also, if using the
POSIX standard as a reference, there should be a NOTE explaining
that use of a POSIX-compliant operating system is NOT required
by the draft. See also 9945-1 sections 2.2.2.24 and B.2.2.2
(headings "Epoch" and "seconds since the Epoch").
5. It's unclear why the number was used rather than a date-time
specification. IMO, a date-time specification (quoted due to
use of tspecials) in the parameter would be easier to use and
more likely to be implemented correctly than the current
specification, even with the ambiguities about time-of-day,
time zone, y2k issues, and leap seconds resolved).
If there is some valid reason for not using a date-time specification,
then all things considered it is perhaps best to include the
necessary details in-line, as they are reasonably brief and that
would avoid the errors in POSIX.1. You need to specify: 00:00:00 1
Jan 1970 UTC, give an algorithm corrected for y2k issues, and note
that leap seconds are intentionally not included in the algorithm.
> MIME:
>
> The following headers, as defined within [RFC 2045] and its
> extensions, may be used within articles conforming to this standard.
>
> MIME-Version:
> Content-Type:
> Content-Transfer-Encoding:
> Content-ID:
> Content-Description:
> Content-Disposition:
> Content-MD5:
> [Bruce Lilly
> What about Content-Language (used in one of the examples!)? Content-
> Duration? Content-Location?]
>
> I'll buy Content-Language (and I will give proper RFC numbers for them
> all). I am open to opinions regarding the other two.
References are:
Content-Language: draft-alvestrand-content-language-03.txt (to replace
RFC 1766, which has been obsoleted without a current approved
definition of Content-Language)
Content-Duration: RFC 2424
Content-Location: RFC 2557 (see also RFC 2110 and Content-Base, which
although deprecated is still permitted by RFC 2557)
> The Content-Type: "multipart/digest" is commended for any article
> composed of multiple messages more conveniently viewed as separate
> entities, thus enabling reading agents to move rapidly between them.
> The "boundary" should be composed of 28 hyphens (US-ASCII 45) (which
> makes each boundary delimiter 30 hyphens, or 32 for the final one) so
> as to enable reading agents which currently support the digest usage
> described in [RFC 1153] to continue to operate correctly.
> [Actually, this conflicts with some present digest usage (such as the
> news.answers rules), but should still be the right way to go. There
> remains the possibility that future MIME-compliant readers could enable
> one to proceed directly to some particular message by clicking on it in
> a table of contents, but that feature is not yet supported by the
> current MIME standards.]
> [Bruce Lilly
> See Content-Location.]
>
> Yes, this was considered (and discussed on the FAQ-Maintainers list). The
> problem is not with using Content-Location (or Content-ID) in each item in
> the Digest, but in referring to that URL in the "Index" section of the
> Digest, which is not in general written in HTML (indeed we strongly
> discourage HTML within Netnews). Yes, I know that many newsreaders can
> "guess" what is a URL when they see one, but no standard mechanism has
> been defined for indicating them in text documents as yet, and I couldn't
> find any system (even one that recognized those URLs) that would pick up
> the Content-Location in another part. So I think that is future work.
There is no reason why a MIME digest could not include a text/html (or
application/html) media type as (part of) the index, and the content of
that type's body could use URIs pointing to the digest messages. Use of
URIs with Content-Location is certainly the intent of RFC 2557. So
technically, as 2557 describes a Content- header, which has special meaning
under MIME, the feature mentioned IS supported by the standards. Implementation
is of course another issue. See also draft-palme-mhtml-info-01.txt and
http://dsv.su.se/jpalme/ietf/mhtml-test/intro.html, the latter is also
accessible via a link from http://dsv.su.se/jpalme/ietf/mhtml.html.
[...]
> Definition of some new Content-Types:
>
> This standard defines (or redefines) several new Content-Types, which
> require to be registered with IANA as provided for in [RFC 2048].
> For "application/news-groupinfo" see 7.2.1.2, for "application/news-
> checkgroups" see 7.2.4.1, and for "application/news-transmission" see
> the following section.
> [Bruce Lilly
> Use "media type[s]" rather than "Content-Type[s]" to avoid confusion
> with the header Content-Type.]
>
> Bruce wants me to change that terminology in lots of places. I will grant
> you that RFC 2045 uses the term "media type" to refer to the entities such
> as "text/plain" of "application/news-transmission", but I remain
> unconvinced. Surely, when we speak of the "Subject" on an article, we mean
> the content of its Subject-header. When we speak of the "Date" of an
> article, we mean the content of its Date-header. So what is wrong with
> referring to whatever is written in its Content-Type-header as its
> "Content-Type"?
1. Note that a Content-Type header also includes the header field name, a
colon, and whitespace. It MAY (in some cases, e.g. for multipart media
types, it MUST) include one or more semicolons and parameters. A media
type, OTOH, does not include those.
2. One might speak of the "subject" or the "Subject"; the latter is
usually understood to mean the header line. Likewise for "date"
vs. Date.
3. This is essentially an editorial issue, and the suggestion is
intended to reduce possible ambiguity.
[...]
> Section 7
>
> Newgroup:
>
> The "newgroup" control message requests that the specified group be
> created or changed. If the request is honoured, or if the group
> already exists on the serving agent, and if the newgroup-flag
> "moderated" is present, then the group MUST be marked as moderated,
> and vice versa. "Moderated" is the only such flag defined by this
> standard; other flags MAY be defined for use in cooperating subnets,
> but newgroup messages containing them MUST NOT be acted on outside of
> those subnets.
>
> NOTE: Specifically, some alternative flags such as "y" and "m",
> which are sent and recognised by some current software, are NOT
> part of this standard. Moreover, some existing implementations
> treat any flag other than "moderated" as indicating an
> unmoderated newsgroup. Both of these usages are contrary to this
> standard.
> [Bruce Lilly
> The penultimate sentence of the NOTE is rather curious. At most one
> newgroup-flag is permitted. The group is moderated if and only if the
> flag is "moderated". So if there is a flag other than "moderated", the
> newsgroup is necessarily unmoderated. How can that be "contrary to this
> standard"?
> If treating a flag other than "moderated" as indicating an unmoderated
> group is contrary to this document, the implication is that the presence
> of ANY flag should be considered an indication of a moderated group.]
>
> No, I think the MUST NOT in the normative text in the first paragraph
> covers the situation. But to be quite sure, I have added to the NOTE
> "and control messages with such non-standard flags should be ignored".
The problem is that the NOTE contradicts the normative text as it stands.
Removing the sentence beginning with "Moreover" from the NOTE and making
a corresponding revision to the last sentence of the NOTE would resolve the
problem.
> Section 8.
>
> Duties of an Injecting Agent
>
> [Bruce Lilly
> Injecting agents are also responsible for ensuring that articles comply
> with other standards, e.g. RFC 2298 (and the draft of its successor,
> draft-vaudreuil-mdnbis-02.txt) strongly recommends that news articles not
> include a Disposition-Notification-To header.]
>
> Hmmm! I think if we start down that route we shall never end. There are
> lots of headers defined in lots of other standards that may appear in
> Netnews, where they may or may not have sensible usages. We have given a
> blanket warning in Section 4 that all such headers MAY appear, but should
> be ignored unless the agent thinks it knows what to do with them. Actually,
> they are usually more useful to human readers who can sometimes garner
> much information about how an article entered the system by perusing some
> of those obscure headers.
Several issues:
1. Because of mail-to-news and news-to-mail gateways, it is important
that USENET articles be valid text (i.e. email) messages. I suppose
one could place the requirement on posting (and followup) agents, but
injecting agents seem like a more logical place. Issues involve things
like number of headers (e.g. no more than one MIME-Version header, and
MIME-Version must be present if any Content- headers are present).
One could also put the onus on news-to-mail gateways rather than
injection agents or posting (and followup) agents, but that may be
far removed from the origin, it's unclear what a gateway should do
(fail to forward the article? attempt to sanitize it? send a message
to the poster?), and there may be multiple news-to-mail gateways
with differing policies.
2. A blanket statement (perhaps similar to the one in RFC 2822) would
suffice for most cases. Disposition-Notification-To has been singled
out because of its enormous potential for disastrous results (see
below).
> Now the Disposition-Notification-To header is used in email to say "please
> send me an automatic acknowledgement when this message arrives at its
> recipient". Clearly, it is a nonsense in Netnews, and the authors of RFC
> 2298 have kindly made it a SHOULD NOT, which is fine.
Actually, what you describe is a Delivery Status Notification (DSN),
whereas Disposition-Notification-To is used with Message Disposition
Notification (MDN), which is a different critter. An MDN indicates
when a message is displayed by a user agent, while a DSN indicates
delivery by a transport agent. Unlike a DSN, an MDN isn't nonsense
in USENET, but due to the possibly huge number of readers, it is at
best unwieldy.
> But that is not going to stop it happening and, occasionally, in an
> article gatewayed into news and then back into email it might even make
> sense. Bruce suggests that injectors and gateways should be instructed to
> remove it when they see it. I think I disagree, again on the grounds that
> if we do it for one we have to do it for all.
Here are my reasons for Disposition-Notification-To. Consider the
following scenarios:
1. A poster writes an article to a moderated newsgroup. He wants to
be informed when the moderator acts on the message, and therefore
he (or his software on his behalf) includes a
Disposition-Notification-To header. If the moderator (and injector)
fail to remove that header when the article is posted by the
moderator, the unfortunate poster may find himself the victim
of a suicide mailbomb. Remember that many news reader clients
are also mail clients and do respond with MDNs (e.g. Pine, Netscape
Messenger, Microsoft Outlook and Outlook Express).
2. A miscreant creates an article, posted to a widely read newsgroup
and containing a Disposition-Notification-To header pointing to
his intended victim. Note that an MDN may include the full original
message, so if the miscreant posts, say, a 5 MB file to one of the
binaries groups, the potential of such a mailbomb makes sendsys
look mild by comparison; and sendsys has been removed in the draft
precisely because of its mailbomb potential.
3. A spammer wants to collect email addresses of gullible and/or
the clueless. He posts a brief non-spam article to a widely
read newsgroup with a Disposition-Notification-To header pointing
to his email address (probably a throw-away address under
hotmail.com or similar domain). He then sits back and waits
for the MDNs, complete with validated email addresses, to
arrive.
I believe that because of the potential for mailbombing in particular,
Disposition-Notification-To should be prohibited from USENET
articles. I have made some suggestions as to how that might be
achieved (viz. removal by injection agents and gateways). If
that's a problem, another method that achieves the same result
would be fine.
I don't think Disposition-Notification-To would be useful post
news-to-mail gatewaying; to whom would the notification go --
the gateway operator?
It might be useful for a mail-to-news gateway to inform the
originator that his article had been gatewayed into USENET
(subject to policy of the gateway operator), but because of
the abuse potential, the header should be elided by the gateway
when injecting the article into USENET.
Note that it is not necessary for injection agents and gateways
to remove other headers as a Comments header (for example) has
no potential for use as a mailbomb. I'm not aware of any other standard
(i.e. defined in an RFC or Internet draft) headers which have potential
for damage like that of Disposition-Notification-To.
In removing a Disposition-Notification-To header, injection agents
and gateways should probably also remove any
Disposition-Notification-Options header which is present, since
the latter is meaningless without the former, though
there is little damage potential if it is left intact. About
the worst that can be said is that it consumes inordinate
amounts of disk space on servers due to its length :-).
> Moreover, if a message is both posted and mailed (and a Posted-And-Mailed
> header is used) then the two versions MUST be identical, according to our
> current draft. I would not want to go to the trouble of declaring
> Disposition-Notification-To to be a variant header.
Remember, "First, do no harm". Which is worse; making a
reasoned exception for Disposition-Notification-To because of
its abuse potential, or the results of Disposition-Notification-To
abuse itself? N.B. the latter may effectively amount to a DoS
attack against ISPs due to the large volume of traffic that may
be generated.
See the NOTE in section 6.9 of the draft for already existing
potential issues regarding differences between posted and mailed
versions.
[...]
Though not previously mentioned, the latest version of my
comments (as of a couple of days ago) also remarks on the
Keywords header. Section 6 of the draft states:
Any header
defined in this (or any other) standard MUST NOT appear more than
once in an article unless specifically stated otherwise.
Section 6.4 (Keywords) contains no statement overriding
section 6.
RFC 2822 section 3.6 explicitly permits multiple Keywords
headers.
Is there any specific reason for forbidding multiple Keywords
headers in USENET articles? If so:
1. those reasons should probably be stated in the draft
2. instructions for gateways and injection agents for handling
articles (received as email) with multiple Keywords headers
should be provided; should the headers be consolidated into one?
should the article be rejected?
If there is no good reason to forbid multiple Keywords headers,
there should be some indication in section 6.4 that multiple
Keywords headers are permitted.
Best regards,
Bruce Lilly