From: Russ Allbery (rra@stanford.edu)
Date: Sun Feb 01 2004 - 13:51:18 CST
Bruce Lilly <blilly@erols.com> writes:
> I think the following issues warrant some discussion:
> 1. semantics of the Date header field vs. RFCs 850/1036/2822
It's unfortunately going to be very difficult to change the effective
semantics of Date on Usenet since it's an integral part of the acceptance
algorithms of pretty much every news server out there. That being said,
the ideal would be to preserve the e-mail semantics of Date for
compatibility and introduce a new header, like Injection-Date, which is
used for the accept/reject algorithm that's dominated by history size.
This would also make gatewaying far easier, since one could preserve the
original Date header and just add the new Usenet-specific header.
I'm very skeptical of our ability to get there from here, though.
> 2. Bases for expiration of an article from a storage system.
> Clearly one is the Expires header field, which is an absolute date-time.
> Another is local policy, often on a per-newsgroup basis -- but is
> that/should that be based on:
> a) amount of time the article has been on *that* system
> b) elapsed time since composition/editing
> c) elapsed time since initial injection
This is site policy outside the scope of our standard, I think. INN
offers an option between the date the article was received on the system
and the date in the Date header (whichever of b or c that means). The
former is the default.
> 3. Source of delay between composition/editing, injection, and arrival
> at a given system. Offline composition has been mentioned.
> Obviously, delays in the moderation process also cause delay between
> composition and injection. And clearly there are transport delays --
> it is not unusual for propagation of an article to take several days.
Actually, that is pretty unusual on Usenet. Not unheard of, but you have
to be behind a very slow initial link for it to take anywhere near that
long, and at some point those cases start looking a lot like off-line
composition rather than a true case of slow propagation.
Of course, if the original source of the message is e-mail, that's a
different story. It's much more common to see e-mail delays of hours or
sometimes even days.
I think you've covered the major reasons in off-line composition and
moderation delay.
> Equally clearly, the posting-date parameter of Injector-Info fulfills
> the role attributed to Date by 850/1036.
Parsing Injector-Info in its current form in order to get that information
is, I believe, a non-starter. It's certainly code I have no interest in
writing.
> And the fact of offline/moderation/transport delays makes use of either
> composition date or initial injection date dubious at best for the
> purpose of expiration.
It depends on how you're defining injection. It makes injection date look
quite attractive assuming that you define injection to mean something that
doesn't happen until the article is approved.
> Here are some thoughts regarding an ideal system:
> A. Date should have the same semantics as in 2822, and its content
> should be essentially ignored by all news software, except for syntax
> and semantic checking per RFC 2822 at injection (i.e. for human
> consumption).
Agreed.
> B. Injector-Info posted-date records time of initial injection
I agree with this in principal but think the current header design for
Injector-Info is the wrong way to do it. We need something dead-simple
and easy, since this is a core news protocol component.
> C. Expiration should be based on time an article exists on the system in
> question and local policy, with possible modification based on
> absolute date-time given by Expires field if present.
We have no business specifying this. News administrators can and will
configure their systems to expire based either on arrival time or on delta
from posting time (or injection time) based on their own preferences, and
this does not pose an interoperability issue.
> Whether local arrival time is recorded in something like a non-
> transmitted Date-Received field a la RFC 850, or by some
> implementation-specific unspecified mechanism is an open issue.
Given that all existing news systems that I'm aware of manage just fine
with a local database, I really see no reason to add another header to the
list of headers that can be modified by transit servers.
> D. Given C, Injector-Info and its posted-date are optional (irrelevant
> to expiration, useful for tracing, possibly slightly useful for
> sorting for display).
No. If you change the semantics of the Date header, whatever you replace
it with must be mandatory, since the Date header is an integral component
of the accept/reject decision for an incoming article. Expiration is a
minor additional use that's not nearly as important.
> E. Given C, Date is not strictly required, but should be maintained as a
> mandatory field consistent with 2822s requirements.
Agreed.
> Comments? In particular, data regarding different server
> implementations of expiration policy (item #2) would be useful.
Don't focus on the expiration policy. Focus on the history retention
policy and the accept/reject decision for incoming messages. Remember,
the primary purpose of the Date header on Usenet is to prevent systems
from re-accepting old articles that have expired out of history, and a
vital component of the configuration of any news server is to reject any
incoming article older than the retention period of the history database.
What happens to the article itself after it's been accepted by the server
is a completely separate and not particularly interesting discussion. The
local news administrator may well delete the article immediately by hand
for some reason. The important point is that the accept/reject decision
must be robust against re-accepting the same stale article, regardless of
whether that article is still in the spool or not.
-- Russ Allbery (rra@stanford.edu) <http://www.eyrie.org/~eagle/>