From: Charles Lindsey (chl@clerew.man.ac.uk)
Date: Fri Apr 02 2004 - 12:29:49 CST
In <406CC294.5090302@erols.com> Bruce Lilly <blilly@erols.com> writes:
>Charles Lindsey wrote:
>>
>> 5.6.4 speaks of "double-injection" which come to the same thing (I would
>> probably change that to "reinjection" as part of the change). Likewise
>> 8.2.1 contains the phrase "injected twice". 8.2.2 uses the term
>> "reinject".
>"[S]peaks of", "contains the phrase", and "uses the term" are
>all different from "defines".
Sure. I was just pointing out that the underlying _concept_ had been
present in the draft for a long time. I am now proposing to _define_ a
term for what was there all along.
We discussed this extensively in January and February. You even used the
term "re-injection" yourself at one point in that thread.
> Neither the term nor reasons
>why the process might legitimately take place are defined.
It is going to happen, whether it is legitimate or not. See
www.landfield.com.2004/Feb/0039.html by Russ, which contains the
following:
>.> D. What happens to articles that get injected twice? It is not supposed
>> to happen, except in some rather rare gatewaying situations, and with
>> some particular BOFHish injectors, but it needs to be covered.
>
>
>Particularly since in practice it happens all the time.
>
>
>> I suspect the 2nd injector needs to overwrite whatever Injection-Date
>> was present before (which is what we currently prescribe for
>> Injector-Info).
>
>
>No, if we're using injection-date for the staleness check, it may not ever
>be overwritten or modified, for the same reason that we require the server
>not modify the Date header. Doing anything else breaks Usenet's loop
>detection algorithm and potentially allows reinjection of stale messages.
>
>
>The choices are accepting the message with the injection-date intact or
>rejecting messages with a user-supplied injection-date.
Note that "it happens all the time".
Anyway, my main concern now is to make sure that the Injection-Date
header, once added to an article, MUST NOT be changed again. Otherwise you
can get looping.
>> . a relaying agent unable to relay an article to its usual peers due to
>> unanticipated communication problems (for example, an agent on a laptop
>> that has been physically disconnected from its usual environment) might
>> attempt instead to reinject it at a site which did not recognize it as
>> a peer;
>If the article has been injected, it should be relayed, not "reinject"ed.
>If the laptop has separate injecting and relaying agents, and the article
>has already been injected, the relaying agent has it and is expected to
>relay it (not inject or reinject it). If it has a combined injection/relaying
>agent and that agent has accepted the article with all of the necessary
>checks associated with injection, but finds that it cannot relay the article,
>it might be reasonable for that agent to convert the article back into a
>proto-article (removing Injection-Date, Injector-Info, etc.) and inject that.
NO! Removing Injection-Date is verboten. This laptop has a complete news
server on it (quite a common setup). It may normally relay to several
sites. Maybe it has already relayed to one (with the original
Injection-Date) before it tried this unusual injection attempt. Maybe it
is going to relay it again when it gets home. We cannot afford to have
multiple copies of the same article wandering around Usenet with different
Injection-Dates (different Injection-Infos are a different thing).
>> . a relaying agent receiving an attelpted relay from a source with
>> which it did not have a peering agreement might choose to reinject it
>> instead, in order to fulfil its obligation to the rest of the network
>> (8.2) to be responsible ...
>According to 8.2, an injection agent is responsible "for the behaviour
>of its posters"; in the case you describe, either the relaying agent is
>unable to fulfill that obligation or it has no reason to reject the
>relayed article. Either way, [re]injection is not warranted.
No, we are talking about the case when an agent B is unwilling to accept
relaying from A, but it is willing to accept injection from A (because
allowing someone to peer with you requires a greater degree of trust in
that someone than does allowing him to inject). So A tries to relay to B
(and perhaps he also relays to another peer C that DOES so allow). A uses the
NNTP IHAVE (so as not to waste time if B has already seen the article via
C). B has not seen it yet via C, so is willing to accept it but, because
it only trusts A as an injector, decides to reinject it (and inserts its
own Injection-Info, in fulfilment of its 8.2 obligations). But it MUST NOT
change the Injection-Date.
An example of a site that does this is uni-berlin, which is one of the
best-respected and best-run sites on the whole of Usenet, and a case which
we discussed in the thread back in February.
>> . there might be a genuine misunderstanding between two sites as to
>> whether the relationship between them was as peers for relaying, or as
>> client/server for injecting.
>How can that happen given the requirement in 8.2 (as above)? An injection
>agent must know its clients quite well.
Would that were true! Usenet is full of dodgy sites, and even more dodgy
clients. It happens all the time, and yet we want Usenet to be robust (in
particular no loops) in spite of the dodgy sites. And it's no use saying
"those sites are dodgy, they must be fixed". It doesn't work like that
(see present thread on the moderators@isc.org list concerning an
unresponsive site that is allowing forged moderator approvals).
>> . an article might be gatewayed out of Netnews into some other medium,
>> and then back again into the same, or a different, Netnews network.
>In that case one or both gateways should take appropriate action (to be
>discussed) to prevent reinjection into the same network, and to remove
>Injection-Date, Path, etc. if entering a different network.
And again, that cannot be enforced. Private networks leak. Sysadmins have
spent years trying to plug leaks, and it just does not work. It takes only
one rogue node on a network to cause a leak. Then the article meanders
back through Usenet to the place where it was first gatewayed out, is
accepted there (because its rewritten Injection-Date looks nice and fresh
and its original copy of the article has long expired), and off it goes
round the loop again.
I can imagine that a gateway whose admin really Really REALLY knew what he
was about might reinject without the original Injection-Date into an
injecting site whose sysadmin really Really REALLY understood why it was
safe, but I would not like to write that into the draft (let them declare
a cooperating subnet if they want to preserve a semblance of legality).
>> In all such cases, it is up to the receiving agent to detect that the
>> actual circumstance is consistent with its site policies for
>> reinjection.
>In all the cases described, an injection agent (or gateway assuming the
>functionality of an injection agent) would need to have sufficient
>information about the source of the article(s) to comply with the 8.2
>obligations. In the specific case of gateways, the x-to-news gateway
>will have to have sufficient information to be able to determine that
>the network that the article will be injected into is in fact different
>(i.e. disconnected from) any news network that the article may have
>been posted to.
Yes, and if they don't not have that information, then they have no
business attempting to reinject. Safer in that case to reject outright any
article with an Injection-Date already present.
>>>>>What about NNTP-Posting-Date? If it is to be ignored, then there is no
>>>>>need to say that it MUST be retained.
>>
>>
>>>If it's not supposed to be generated, then why mandate preservation?
>>>If it's not used for anything, then it is effectively ignored.
>>
>>
>> Because it is generally a Bad Thing to remove any header without good
>> cause. But you may well argue that MUST is a bit strong here. I have made
>> a note to review this when I come to writing that bit of the text.
>There are some (e.g. Xref) that shouldn't be there in the first place,
>and which should always be removed when relaying, gatewaying, etc.
Yes, that is why we invented the category of "variant headers".
> One
>issue with what you wrote is indeed that under the circumstances
>(NNTP-Posting-Date is ignored) "MUST be retained" is to strong. However,
>it is arguable that NNTP-Posting-Date should not be ignored if it is
>present and Injection-Date is not, in which case "MUST be retained" is
>reasonable. Specifically, the date-time taken to be the time of
>injection should probably be:
>1. that specified in Injection-Date, if present, else
>2..that specified in NNTP-Posting-Date, if present, else
>3. that specified in Date, as a last resort
>I.e. NNTP-Posting-Date fills an interim role until Injection-Date becomes
>widely supported.
I don't think I want to write that into the draft (we should not be
telling people to use a header that we are deprecating). But I have no
problem if some sites take that course as an interim measure. Not that it
will make much difference, since NNTP-Posting-Date is not added by the
majority of current injectors. And the draft will certainly tell them to
use Date is that is all that they have got.
-- Charles H. Lindsey ---------At Home, doing my own thing------------------------ Tel: +44 161 436 6131 Fax: +44 161 436 6133 Web: http://www.cs.man.ac.uk/~chl Email: chl@clerew.man.ac.uk Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K. PGP: 2C15F1A9 Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5