Re: #1416 Reinjection - an attempted summary, and a suggested resolution

["Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>]

In <87ejogm792.fsf@xxxxxxxxxxxxxxxxxxxxx> Russ Allbery <rra@xxxxxxxxxxxx> writes:

>Charles Lindsey <chl@xxxxxxxxxxxxxxxx> writes:

>> But if an injecting agent is aware of the cutoff of its immediately
>> following relaying agent (which is a reasonable assumption, because they
>> are often two aspects of the same agent), then it will also be aware
>> whether said relaying agent understands Injection-Date. In which case
>> that SHOULD should not apply. IOW, what you are proposing should be
>> regarded as only a temporary measure for use when Injection-Date is not
>> observed (i.e.  a continuation of what some agents are currently doing
>> anyway).

>Yes, exactly.  I think that the root of our disagreement is again that I
>believe said transition period until Injection-Date is widely enough
>observed that an article with a very stale Date will propagate usefully is
>on the order of five to ten years.

It only needs to be implemented by those news servers with exceptionally
short history retention times for its benefits to be felt, and the number
of such servers is relatively small.


>I believe that some posting agents deliberately add the Message-ID and
>Date header so that they can inject the article with the same identity to
>multiple servers simultaneously.  I know that at least some proxies have
>that capability.  If each injecting agent then adds their own
>Injection-Date, we break the identity model.

OK, so the issue only arises with agents that multi-inject, and harm will
only arise if those injections are significantly separated in time
(and by a period measured in hours rather than minutes).

>Sure.  Suppose that I'm using a local news proxy which talks to three
>different remote servers for me and presents me with a combined view of
>newsgroups from those servers.  (This is an increasingly common substitute
>for running a full-fledged local news server, and in some ways works much
>nicer.)  Suppose that for maximum propagation (for exactly the same sorts
>of reasons that you do something similar with C News), I have it
>configured to post my messages to all of those remote servers at the same
>time.

>If one of those servers is down at the time of the posting, the safest and
>most reliable behavior, and the one that I believe is currently
>implemented in at least some software of this sort, is to queue the post
>and keep trying.

And that would seem to be the only situation in which a significant time
separation could occur during multi-injection.

And that has to be set against my concern about the man who prepares
articles offline and then injects them much later.

So let us examine the two competing strategies.

I think we are agreed that

. Injecting agents MUST insert Date and Message-ID when absent.
. Injecting agents MUST insert Injection-Date when Injection-Date is
absent AND (either Date OR Message-ID is absent).
. Injecting agents MUST NOT insert Injection-Date if it is already
present.

We disagree on the following two possibilities:

C. Injecting agents MUST insert Injection-Date when it is absent.

R. Injecting agents MUST NOT insert Injection-Date if both (Date AND
   Message-ID are already present).

So now we can examine the consequences of those two rules.

Consequences of Rule C:

Posting agents (or proxies etc) which multi-inject AND where one of the
injections sits around in a queue because a server is down, and which
inserts an Injection-Date when it eventually wakes up, may cause relaying
agents further on which understand Injection-Date to re-accept articles
that they have already accepted before.

But note that this problem is not permanent - it will go away as and when
those multi-injecting agents start inserting Injection-Date themselves
(which they would have to do to claim compliance with our new standard).

OTOH, posting agents which prepare articles offline, adding the Date at
composition time, and then failing to inject until hours/days later, and
who then inject to a single injecting agent (either adding Injection-Date
themselves or letting the injecting agent do it) will find that their
articles propagate well amongst servers that understand Injection-Date. So
their situation will improve as Injection-Date becomes more widely
adopted.

Consequences of Rule R:

Posting agents (or proxies etc) which multi-inject AND where one of the
injections sits around in a queue because a server is down will not cause
articles to be re-accepted.

OTOH, posting agents which prepare articles offline, adding the Date at
composition time, and then failing to inject until hours/days later, and
who then inject to a single injecting agent and do not add Injection-Date
themselves will find that their articles propagate as badly as they do at
present.

Note that this problem is permanent (or at least until those posting
agents learn to insert Injection-Date themselves). Moreover, the problem
will get worse as more and more posting agents accept our recommendation
to add the Date header at composition time.

So now we have to consider which of those scenarios is the more
troublesome. For this, we have to make some guesses:

1. How many multi-injecting agents are there around, and how often will
there be disruption because one of their injectees is down; and how does
this compare with the number of offline posting agents around and how
often their injecting get delayed?

2. How will the deployment of Injection-Date go? My expectation is that
it will be taken up first by those relaying agents which currently have
short expiries on their history files (because they have the most to
gain). It will then be taken up by smaller sites with clueful
administrators, and that probably includes people running proxies and
doing multi-injecting. And finally, it will be taken up by ordinary user
agents, and indeed that will be a _very_ slow uptake.

3. How rapidly will the practice of constructing the Date at composition
time spread? Probably faster than the spread of posting agents that know
how to insert Injection-Date themselves (which I regard as normally
unnecessary, although we seem to disagree there). But this is a guess
where we have some evidence to go on, because this practice is already
widespread.

I performed an experiment on 350 articles in the group uk.misc (where I
would expect to find a pretty average bunch of posters). I compared the
NNTP-Posting-Date with the Date header. I would expect these to be
identical (to the same second) where they were both created on the
injecting agent. Where they were different, they were often surprisingly
close (indicating that lots of people were actually online while they
composed articles, though differences between the two dates were both
positive and negative, indicating differences between the clocks on the
two systems). But the interesting fact to emerge was that nearly 70% of
posting agents were apparently already constructing their own date
headers. OK, the sample was small, and I will happily perform a larger
experiment if people wish. But that 70% was considerably higher than I
would have expected.

So my conclusion is that it will soon be commonplace for Date to be already
present at injection time (indeed we are almost there), and so to make
assumptions on the basis that this indicates a multi-injecting proxy is
unwise. On balance, it sees that rule C is going to provide fewer
problems, especially in the longer term.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: chl@xxxxxxxxxxxxxxxx      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5