Re: #1416 Reinjection - an attempted summary, and a suggested resolution

["Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>]

In <45D2C21C.7090600@xxxxxxxxxxxxx> Harald Alvestrand <harald@xxxxxxxxxxxxx> writes:

>There was some report of an experiment on propagation with different 
>Date: headers, but I've not been able to find that in my archive. Bad 
>choice of keywords to search for, I assume.

Essentially, I established that some injecting agents impose an arbitrary
cutoff (e.g. 4 days) as a matter of site policy, and some will accept any
degree os staleness (but whether they then succeed in relaying/propagating
them is another matter).

Also, it seems to be the case that articles up to 8 days stale, once
accepted by an injecting agent, seemed to propagate, and to do so without
much degradation of speed or by using noticeably different routes. But I
have insufficient data to give a definitive answer as to whether these
observations are typical (now, if Google reported arrival times, or
retained the Path, such experiments would be much easier :-( ).

>Suggested hacks include:
>- Changing Date: at injection to be within a short distance from the 
>current time
>- Allowing posting agent to set Injection-Date:, to ensure it's 
>consistent between injected copies

Yes, Russ suggested that, and I agree - modulo various niggles about what
constitutes a 'posting agent' in this particular context.

>- Drop Injection-Date: from the spec


>--------------------------------------------------------------------------------------------------------------------------
>PROPOSED RESOLUTION:

>- Date: identifies the date of composition. Don't change it, ever.

Yes.

>- Injection-Date: identifies the date of injection, by the last agent 
>that identifies itself as an injecting agent. Proto-articles don't have 
>it.

Almost. It might have been put there by a posting agent (see 'suggested
hacks above) at the time of posting (Note: that does not mean that the
proto-article has it).

> Relaying and serving agents never, ever change it.

Yes. And likewise gateways (if the outgoing medium will allow it).

>- "Reinjection" doesn't exist.

You can say that, but it will not prevent it from happening :-( . And
sometimes it is even sensible. And safe if the Injection-Date is retained
(there _may_ be some special cases where not retaining it might be OK, but
those need careful thinking about).

> There is only multipoint injection. The 
>means by which a proto-article (including message-ID, but NOT including 
>Injection-Date:) gets from wherever it started out to the posting agent 
>is not defined by the protocol; if the protocol happens to be NNTP, 
>that's not an issue for this specification. (we can add warnings...)

Eh? Surely the prime duty of a posting agent is to generate
proto-articles. What the protocol does not define is whether a
posting-agent maintains a queue of stuff waiting to be injected (lots of
them do, and those are the ones which might be allowed to insert
Injection-Date). OTOH, "normal" posting agents will not be doing
multipoint injection; that is more likely to arise from specially written
scripts.

>- When doing multipoint injection, Message-ID: MUST be consistent in all 
>copies. Date: SHOULD be consistent, if it's not, Bad Things Can Happen.

And likewise Injection-Date.

>On the stale article problem:

>- We state that existing implementations use Date: to check for 
>staleness, and that will cause discarding of articles that are too old. 
>Period, end of story, accepted.

Yes.

>- We recommend (require?) that injecting agents refuse posting of "too 
>old" articles by Date: - the exact acceptable Date: range should be a 
>local decision.

Disagree. That is a matter of site policy. It is the relayer that should
make that choice, and hopefully it will be using Injection-Date for that
putpose. OK, if the injecting agent knows that its immediately following
relaying agent is going to reject, then it would be couteous to reject it
itself (but equally well, if could look at Injection-Date itself). As I
said, this is all a matter of site policy.

>- We recommend that the staleness check use Injection-Date: if present, 
>otherwise Date:

Yes. REQUIRE.

>- We state that if stale article rejection is an issue for people, they 
>should:
>  - Ensure that their injecting agent adds Injection-Date:

Or add it themselves (e.g the man who carries his server around on his
laptop).

>  - Put pressure on the discarding agent to check Injection-Date: if present

Yes.

However, I think we should wait until Russ has produced the revised
proposal that he promised a couple of days back.

>Case analysis:
>If a message has Date: on day 0, and is injected on day 7, into a 
>network where anything older than 3 days is considered stale, but the 
>injecting agents accept 14-day-old proto-articles, this will happen:

>- Servers that check Injection-Date: will relay the message. More 
>propagation than today.
>- Servers that check Date: will reject the message. Unchanged from today.
>- Servers that feed from a server that checks Date: will not get the 
>message in the normal feed; they may get it offered from feeds that go 
>via Injection-Date: checking servers.

All correct. Injecting that late is currently a bit dodgy. It will improve
as more servers are upgraded to use Injection-Date.

>If a message has Date: on day 0, gets injected on day 1 into the 
>network, and gets injected again on day 7 into the same network, the 
>following will happen:

>- Servers that check Date: will see the first copy.
>- Servers that check Injection-Date: will relay the message twice, 
>possibly causing users to see it twice.

But not if whoever reinjected on day 7 kept the original Injection-Date
(which is what I claim SHOULD/MUST be the normal practice - though a few
exceptions might be worth looking into).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: chl@xxxxxxxxxxxxxxxx      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5