From: Charles Lindsey (chl@clerew.man.ac.uk)
Date: Fri Apr 23 2004 - 15:07:14 CDT
In <4087E374.6040105@erols.com> Bruce Lilly <blilly@erols.com> writes:
>> Comments?
>Our editor should not be introducing controversial syntax which has not
>been discussed by the WG.
It is the editor's job to propose syntax whenever the WG decides to
introduce a new feature. How is the WG supposed to discuss a syntax unless
it is first put before them as a proposal?
Anyway, this thread if about the changes I have proposed to incorporate
the new Injection-Date header. You have raised many issues arising from
texts which are not affected by these changes, and these are OFF
TOPIC for this thread. If you are concerned about them, then please raise
them in separate threads (though I have included brief comments on some of
your OFF TOPIC items, just to clarify things).
>> 5.1. Date
>>
>> header =/ Date-header
>> Date-header = "Date" ":" SP Date-content
>> Date-content = date-time
>A general comment: the draft is already far too long; convoluted ABNF
>such as the above could be replaced by
> Date-header = "Date" ":" SP date-time
OFF TOPIC {The WG agreed to follow that method of presenting the header
syntax a long time ago.}
>> | The date-time MUST NOT be more than 24 hours into the future
>> | (injecting agents will reject such articles, possibly even when the
>> | margin is less than 24 hours - see 8.2.2) and it SHOULD NOT be
>> | inordinately far into the past. Relaying agents MUST NOT modify the
>> | Date-header in transit.
>> [Do you want me to be more specific about that "inordinately"?]
>The whole of the text regarding past dates for this field is unnecessary
>and inappropriate.
Yes, this is an issue which I was uncertain about and which we can
discuss. By "inordinately" I mean months, or even years, out of date. Such
things are clearly undesirable, but we can debate the extent to whether
the point belongs in USEFOR or USEAGE.
>> 5.7. Injection-Date
>>
>> |
>> | header =/ Injection-Date-header
>> | Injection-Date-header
>> | = "Injection-Date" ":" SP Injection-Date-content
>> | *( ";" extension-parameter )
>> | Injection-Date-content
>> | = date-time
>We have not discussed any "parameters". We have agreed to introduce the field
>to convey the correct semantics which differ from the defined semantics for
>the Date field, which the draft had previously used (with *no* "parameters").
The WG decided a long time ago that all headers would have parameters
EXCEPT those which were already defined in other standards (notably RFC
2822), or which contained address syntax.
Neither applies here. This is a new news-only header. There may be
unforeseen problems with it, in which case we might be glad of an
opportunity to use a parameter to resolve them.
>> 6.19. Injection-Info
>>
>> | An Injection-Info-header MUST NOT be added to an article except by an
>> injecting agent. Any Injection-Info-header present when an article
>> | arrives at an injecting agent MUST be removed. In particular if, for
>> | some exceptional reason, an article is being reinjected (8.2.2), the
>> | Injection-Info-header will always relate to the latest injection (to
>> | the extent that this happens, the Injection-Info-header can be
>> | regarded as a variant header).
>As noted elsewhere, "reinjection" has yet to be justified.
This is just a change of terminology. The present text says "injected
twice". It is not a substantive change.
>> 8.2. Duties of an Injecting Agent
>>
>> | NOTE: Examples of some situations in which the policy of an
>> | injecting agent might permit reinjection to occur:
>> |
>> | o an article injected by another injecting agent which was
>> | temporarily unable to contact the relaying agent(s) with
>> | which it normally peered;
>Injecting agents don't "peer" with relaying agents. They are different
>types of agents, not peers.
A valid point. It now says:
o an article from a relaying agent temporarily unable to
contact its usual peer(s) and therefore injected at some
other (non peering) site;
>> | o an attempt to relay by a site which only had permission to
>> | inject (whether due to some misunderstanding between the
>> | sites or as part of some backup relaying mechanism);
>Again, different agents are involved. If a proto-article is involved (i.e.
>a message which has never been injected), then injection, not relaying, is
>the appropriate action. If it is instead an article (i.e. it has already
>been injected), then relaying is the appropriate action -- and is performed
>by a different agent.
Presumably this is a (not proto-) article and is perhaps being offered to
this site as a backup to its normal relaying route (to ensure that it gets
out to the network one way or the other). In that case, there is benefit
in using the IHAVE rather than the POST command (as INN now allows IIUC)
because it saves bandwidth. Some sites prefer IHAVE to be used in this
situation for that reason.
>> | o an article which had been gatewayed out of Netnews into
>> | some other medium, and then back again into the same, or a
>> | different, Netnews network.
>This possibly warrants further discussion, but it seems that the appropriate
>behavior would be rejection by the x->news gateway.
Possibly, but gatewaying can get very complex if there are crossposts to
unexpected places and/or submission to several mailing lists (all
discussed in our section 8.8). Basically, it is up to the administrator of
the gateway to have a very clear idea of what he is trying to achieve and
to set his policies accordingly. In general, (re-)injecting may be the
only option offered to him by his ISP, in which case it is essential that
any pre-existing Injection-Date be retained (particularly since articles
that have taken such tortuous routes are especially likely to be near
their stale limit).
>> 8.2.2. Procedure to be followed by Injecting Agents
>>
>> An injecting agent receives proto-articles from posting and followup
>> agents. ....
>>
>> | Exceptionally, it MAY reinject an article, perhaps as a part of some
>> | complex gatewaying process (in which case it will add a further '%'
>> path-delimiter to the Path-header). It MUST NOT forward an already
>> injected article to a moderator.
>That is inconsistent; posting and followup agents produce proto-articles,
>not articles. Relaying agents do not transfer articles to injecting agents.
>Exactly where is an injecting agent supposed to get an article (N.B. not a
>proto-article)?
Presumably because someone sent the article to it (it is actually quite
common for articles prepared by some automated posting mechanism, e.g. a
FAQ poster, to be sent directly to an injecting agent without passing
anywhere near a posting agent as that term is commonly understood). But
point taken, and the wording can be clarified:
Exceptionally, it MAY reinject an already injected article received
from some other source, perhaps as a part of some complex gatewaying
process (in which case it will add a further '%' path-delimiter to
the Path-header). It MUST NOT forward an already injected article to
a moderator.
>> 1. It MUST remove any Injection-Info- or Complaints-To-header already
>> present (though it might be useful to copy them to suitable X-
>> headers). It SHOULD likewise remove any NNTP-Posting-Host or other
>> undocumented tracing header.
>If the "header" is undocumented, how can it be defined as "tracing", and
>how is an injecting agent supposed to know about it?
Strictly OFF TOPIC. However ...
You would be surprised how clever some injecting agents are :-) .
Actually, the header I had in ind was the X-Trace-header, upon which our
Injection-Info header is based. It now says "remove any NNTP-Posting-Host,
X-Trace or other ...".
>> | 3. It SHOULD reject any article whose Date-header (5.1) is more than
>> | 24 hours into the future (and MAY use a margin less than that 24
>> | hours). It MUST, except when reinjecting, reject any article with
>> | an Injection-Date-header already present (and SHOULD do likewise
>> | with any NNTP-Posting-Date-header). It MAY when reinjecting, but
>> | only if there is no Injection-Date-header present, reject any
>> | article whose Date-header appears to be stale (e.g. more than 72
>> | hours into the past).
>This may be a problem for FAQ and similar articles which are posted
>periodically, often with no change in content (and hence a Date field
>with date-time well in the past).
Oh No! When FAQs are reposted, they always get a new Date-header
(sometimes a Last-Modified pseudo header is added). Anyway, this
particular bit is only intended for the interim period during which
Injection-Date-headers may not routinely be present, and simply carries on
surrent practice based on the Date-header. I could be persuaded that '72
hours' is the wrong number, or even that the sentence should be omitted
entirely, but that is a matter for discussion.
>> | 4. It MUST reject any article that does not have the correct
>proto-article, not article.
No because (see below) that text also covers reinjection.
>> | mandatory headers for a proto-article (or, when reinjecting, all
>> | the mandatory headers other than Injection-Date), or which
>> contains any header that does not have legal contents. It SHOULD
>> reject any article which contains any header deprecated for
>> Netnews (4.2.1).
>"legal" is rather vague. "SHOULD" may be too weak in the case of
>Disposition-Notification-To.
OFF TOPIC {But ...
It used to say "syntactically legal", but you made me change it :-(.
"SHOULD NOT" is what it says in RFC 2298.}
>> 5. If the article is rejected (for reasons given above, or for other
>proto-article, not article.
No, this applies both for primary injection and reinjection.
>> formatting errors or matters of site policy) the posting agent
>> SHOULD be informed (such as via an NNTP 44x response code) that
>> posting has failed and the article MUST NOT then be processed
>> further.
>ITYM 441 rather than 44x.
OFF TOPIC {but ... a 440 response could also occur.}
>> 6. The Message-ID, Date and From-headers (and their contents) MUST be
>> | added when not already present (but that situation could not arise
>> | during reinjection).
>Exactly where is an injection agent supposed to come up with content
>for a From field?
OFF TOPIC {but ... it can easily happen when the injecting agent is on the
same host (or more likely the same intranet) as the posting agent and can
pick up the identity of the poster from the passwd file. This is actually
quite a common setup (or was so some years back before NNTP became the
norm - it is still more efficient for a newsreader to bypass NNTP in such
a setup, but it tends to be only the older newsreaders such as rn, trn and
nn which have retained that capability).}
>> 7. A Path-header with a tail-entry (5.6.3) MUST be correctly added if
>> not already present (except that it SHOULD NOT be added if the
>> | article is to be forwarded to a moderator). During reinjection,
>> | the existing Path-header SHOULD be retained.
>To improve clarity and avoid confusion and awkward wording, it would
>be best to include forwarding via email to moderators in the definition
>of an injecting agent. And the determination of whether the agent is
>forwarding or injecting should be described early. Then the steps taken
>for the two processes can be described independently.
OFF TOPIC {The various Steps in the injecting process have been shuffled
around quite a bit as things have developed, and the present arrangement
seemed the best way to bring the bulk of the moderation process into one
place - whenever you squeeze it, a problem always springs up in another
place. But I am not averse to reviewing this once more once these
present issues under discussion have stabilized.}
>> 8. The path-identity of the injecting agent with a '%' path-delimiter
>> (5.6.2) MUST be prepended to the Path-header; moreover, that
>> path-identity MUST be an FQDN mailable address (5.6.2).
>That warrants examination of the full draft text.
OFF TOPIC {But 5.6 has not changed since the last full draft.}
>> 9. An Injection-Info-header (6.19) SHOULD be added, identifying the
>> trusted source of the article, and a suitable Complaints-To-header
>> (6.20) MAY be added (except that these two headers MUST NOT be
>> added if the article is to be forwarded to a moderator).
>More awkward wording due to lack of early mention of forwarding.
OFF TOPIC {But see above.}
>> 10.The injecting agent MUST NOT alter the body of the article in any
>> | way. It MAY, except when reinjecting, add other headers not
>> | already provided by the poster, but SHOULD NOT alter, delete, or
>> reorder any existing header, with the specific exception of
>> "tracing" headers such as Injection-Info and Complaints-To, which
>> are to be removed as already mentioned.
>Conflicts with numbered paragraph 1.
In what way?
>> 11.If the Newsgroups-header contains no moderated groups, or if it
>> | contains an Approved-header, the injecting agent MUST then add an
>> | Injection-Date-header (5.7) if one is not already present, but it
>> | MUST NOT alter, or remove, an already present Injection-Date-
>> | header (and likewise SHOULD NOT alter, or remove, an already
>> | present NNTP-Posting-Date-header). Finally, it forwards the
>> article to one or more relaying or serving agents.
>This is a big problem in the overall design (or lack thereof); how
>exactly is an injecting agent supposed to be able to determine which
>newsgroups are moderated? Isn't NNTP-Posting-Date one of those undocumented
>tracing "headers" that the injecting agent was supposed to have removed
>in step 1?
1. OFF TOPIC {It is not defined in the document, but it is also recognized
that not all agents will be aware of all moderated groups, which is why
provision is made in 8.3 for every relaying agent to repeat the test. In
practice, unapproved articles to moderated groups do not propagate very
far.}
2. No it is not, because we single out NNTP-Posting-Date for special
mention, both here and elsewhere.
>> 12.Otherwise, when the Newsgroups-header contains one or more
>> moderated groups and the article does NOT contain an Approved-
>See above w.r.t. moderated groups.
OFF TOPIC {No, this is indeed the main place where the moderation
submission process is described in full.}
>> header, the injecting agent MUST forward it to the moderator of
>> the first (leftmost) moderated group listed in the Newsgroups-
>> header via email. There are two possibilities for doing this:
>How does the injecting agent determine the email address of the moderator?
>What happens if there is more than one moderator for a given newsgroup?
OFF TOPIC {With difficulty :-( . See below. And there is only one
moderation submission address for a given newsgroup; if that leads to two
moderators, then that is their problem.}
>> (a) The complete article is encapsulated....
>>
>> (b) The article is sent as an email as it stands....
>>
>> Although both of these methods have seen use in the past, the
>> preponderance of current usage on Usenet has been for method (b)
>> and many moderators are ill-prepared to deal with method (a).
>> Therefore, method (a) SHOULD NOT be used until such time as the
>> majority of moderators are able to accept it.
>Shouldn't there be a statement to the effect that moderators SHOULD
>prepare to accept method (a)? Otherwise "such time" will be approximately
>never.
OFF TOPIC {There IS such a statement, in 8.7.}
>> 13.This standard does not prescribe how the email address of the
>> moderator is to be determined, that being a matter of policy to be
>> arranged by the agency responsible for the oversight of each
>> hierarchy. Nevertheless, there do exist various agents worldwide
>That is a serious weakness in the design.
OFF TOPIC {That was a deliberate choice. There IS a service to fulfil this
need, provided voluntarily by ISC (or, in practice now, by Russ's
group-admin setup). However, it is beyond the possibilities of this
standard to mandate ISC or anyone else to provide that service and, in any
case, there are likely hierarchies for which it does not currently work. I
can imagine than an Informational RFC could cover this at a later date.}
>> (b) The result of these operations is used as the local-part of
>> the mailbox of the agent. For example, articles intended for
>> "news.announce.important" would be emailed to "news-
>> announce-important@forwardingagent.example".
>If an injecting agent is to be permitted to mail to such a forwarding
>agent rather than directly to the moderator, the text in earlier paragraphs
>should so indicate.
OFF TOPIC {Such forwarding is a normal feature of SMTP (see RFC 2821). It
does not need any special menation.}
>> 8.4. Duties of a Serving Agent
>>
>> A serving agent processes articles as follows:
>>
>> | 2. It MUST examine the Injection-Date-header (or, if that is absent,
>> | the Date-header) and reject the article as stale (5.7) if that
>> | predates the earliest articles of which it normally keeps record,
>> | or if it is more than 24 hours into the future (the margin MAY be
>> | less than that 24 hours).
>Do serving agents "reject" articles, or do they simply fail to store
>them?
OFF TOPIC {It comes to the same thing. See identical wording for relaying
agents. Do they "reject" them, or do they simply omit to pass them on?}
>> 8.5. Duties of a Posting Agent
>>
>> A Posting Agent is used to assist the poster in creating a valid
>> proto-article and forwarding it to an injecting agent.
>>
>> Postings agents SHOULD ensure that proto-articles they create are
>> valid news articles according to this standard and other applicable
>proto-articles, not articles.
OFF TOPIC {But I have removed the two words "news articles".}
>> | policies. In particular, they MUST NOT create any Injection-Date-,
>> | Injection-Info- or Complaints-To-header.
>>
>> Posting agents meant for use by ordinary posters SHOULD reject any
>> attempt to post an article which cancels or Supersedes another
>> article of which the poster is not the author.
>Why is "Supersedes" capitalized?
OFF TOPIC {Because it arises from the Supersedes-header, which is
customarily spelt that way.}
-- Charles H. Lindsey ---------At Home, doing my own thing------------------------ Tel: +44 161 436 6131 Fax: +44 161 436 6133 Web: http://www.cs.man.ac.uk/~chl Email: chl@clerew.man.ac.uk Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K. PGP: 2C15F1A9 Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5