[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Protocol changes in draft-allbery-usefor-usepro-00
In <873b7i9b2m.fsf@xxxxxxxxxxxxxxxxxxxxx> Russ Allbery <rra@xxxxxxxxxxxx> writes:
>Charles Lindsey <chl@xxxxxxxxxxxxxxxx> writes:
BTW, you spent a lot of time justifying cases which I had already marked
with [+1], indicating my agreement (though other might want to disagree,
so needed to be made aware of them).
>> 1. [-1] (2.2) <path-identity> - Possibility to use a FQDN not resolvable
>> in the DNS.
>> (This was a suggestion by Harald, & 1 of 2 alternatives in USEPRO)
>Intentional, but I don't object to changing it.
OK. My vote is still [-1].
>The intentional change here is that I added a different preferred first
>option, namely the FQDN of the news server itself.
Which I did not quite understand. Did you mean the A or AAAA record? In
which case please be explicit. But I would prefer to allow CNAMEs and MXes
as well.
>> 2. [00] (3.2) <path-identity>s are case-sensitive.
>> I was always told that some servers treated them one way and some the
>> other, so you could not rely on either. If anything, I would have
>> expected case-insensitive (that is what domain-names are, and what
>> USEFOR declares <diag-keyword>s are).
>Intentional change.
>Declaring them case-sensitive is the conservative choice. If you treat
>them as case-sensitive and they're actually compared case-insensitively,
>everything works fine. The opposite is not true; if your Path identity
>varies in case, servers that compare case-sensitively may add incorrect
>MISMATCH entries or send you articles that you sent them.
OK, I will buy that.
>The one exception is if you use a Path header that varies only in case
>from another site and assume that it will be compared case-sensitively. I
>thought about adding language to specifically say not to do that, but it's
>hard to tell a site how to guarantee that ...
That's the site's problem :-( .
>Anyone doing this is already violating the SHOULD that says path
>identities should be in all lowercase...
Yes, but please add "(since some current usage treats them
case-insensitively)".
>> 3. [-1] (3.3.1,3.4,3.9.2) From not omittable in proto-article
>> There is existing usage where the injecting agent fills in From header
>> (not possible in NNTP, of course)
>Intentional change.
>What injecting agent supports this?
INN apparently (see below).
Newsreaders such as rn, trn, nn, and others of that generation had (and
still have) the capability to interact directly with the newspool (either on
the same host, or more likely NFS mounted from some server) rather than
going via NNTP (which did not exist when they were first written). They
injected articles by calling a program 'inews' which, in the absence of an
explicit From:, assumed the poster was the user who had called 'inews'.
I have been reading and posting to Usenet for around 20 years now, and I
have never yet configured my newseader with what I wanted in my From: line
(except when doing tests with upstart agents such as netscape :-) ). I
don't suppose I am the only one. I suspect that 'inews' still exists on a
lot of systems.
It certainly comes as part of CNews. 'nn' comes with a version of 'inews'
that invokes NNTP with a POST command, but it still checks to see whether
a From: header has been provided, and goes looking in /etc/passwd if not.
And Stan Barber's model implementation of NNTP, when it comes to
implementing the POST command, looks for a program 'inews' on the host it
is running on (though of course it doesn't expect automatic deduction of
From: in that situation).
And now I find that INN seems to have an 'inews'. I just found that the
University offers an 'inews', and I just posted an article with no From:
header using it, and read it back:
kiss% telnet wapping 119
Trying 130.88.198.1...
Connected to wapping.
Escape character is '^]'.
200 cs.man.ac.uk InterNetNews NNRP server INN 2.3.3 ready (posting ok).
group man.test
211 1 1680 1681 man.test
article 1681
220 1681 <em6777$2am$1@xxxxxxxxxxxxxxxxxxxx> article
Path: cs.man.ac.uk!chl
From: chl@xxxxxxxxxxxx (Charles Lindsey)
Subject: testing inews
Date: Mon, 18 Dec 2006 14:08:25 +0000 (UTC)
Organization: School of Computer Science, University of Manchester, U.K.
Lines: 2
Message-ID: <em6777$2am$1@xxxxxxxxxxxxxxxxxxxx>
NNTP-Posting-Host: kiss.cs.man.ac.uk
X-Trace: wapping.cs.man.ac.uk 1166450905 2390 130.88.192.64 (18 Dec 2006
14:08:25 GMT)
X-Complaints-To: news@xxxxxxxxxxxxxxxxxxxx
NNTP-Posting-Date: Mon, 18 Dec 2006 14:08:25 +0000 (UTC)
Xref: cs.man.ac.uk man.test:1681
called /opt/cs/bin/inews from kiss.
No From line, but it was from Charles Lindsey <chl@xxxxxxxxxxxxxxxx>
You will see that it has constructed a suitable From header (that is my
correct email address when I am inside the Manchester firewall).
The wording I wrote in USEPRO was
"... (and even From if the particular injecting agent can derive that
information from other sources)".
That indicates pretty clearly that it would be an unusual possibility,
but fine if you happen to have it. It ain't broke ...
>.... This doesn't allow for injecting agents that replace the From
>header with one containing the authenticated identity of the user, and I
>do know of some injecting agents which do this, but I don't think that's
>something that we want to support.
Sure, and the wording in USEPRO gives no support whatsoever for that
practice.
>> 5. [-1] (3.3.4) SHOULD trim References to keep <= 998: reduced from MUST.
>> This could allow header lines longer than 998 octets, which is known to
>> severely impact interoperability. It is not disputed that trimming when
>> within the 998 limit is a MAY.
>Intentional change, but I'm dithering.
One might solve the problem by just saying "fold it so that no header line
is more than 998". However, that is not robust enough.
There was a case on this list a few years back (yes, that was email, but
the problem is the same for both). Some thread got so long as to run into
the 998 limit. This is essentially a problem for user agents, not servers,
and some people's MUAs were clearly folding the huge References headers
in a sensible manner. But it was apparent that other MUAs were first
unfolding them, adding the extra entry, and then hopefully refolding them
again. And some agent(s) out there was evidently trying to do all that in
a buffer of 998, and when it overflowed it just truncated things at that
point (or somesuch - I forget the precise details). So you ended up with
an invalid References header which was then sent forth to the list. And
when that reached my 'nn' (I gateway this list into news), it positively
refused to display the article at all (so I could not even see what was
wrong). I had to go to my newspool and hand-edit the References header
into somethng legal (and chop a lot out of it at the same time), and then
it was fine.
>> 6. [00] (3.4) Injecting agents reporting rejection to user agent; was
>> SHOULD, now merely "encouraged".
>> All it needs is an NNTP 44x response or similar.
>Intentional change.
>What this means is that the Netnews protocol would be placing a
>requirement on the transport layer that it have a means of reporting
>errors back to a client.
I would regard an injecting agent that was so detached from its clients as
to be unable to report back to its clients as irretrievably broken, hence
making the system hopelessly inadequate for its users (see my remarks in
response to your "philosophy" claims). Surely that is just the sort of
situation that "SHOULD" is designed for (I agree that if you are
communicating with your injecting agent via carrier pigeon, there is no
way to get the bird to fly back, so "MUST" would be too strong).
Count me as [-1] on this.
>> 8. [-1] (3.4) Requirement for injecting agent to forward articles to
>> moderator groups reduced from MUST to SHOULD.
>> If some small latitude is needed for when the moderated group is in a
>> crosspost to some remote and unknown hierarchy, then this should be
>> stated explicitly.
>Intentional change.
>This section says more than just that, and I think the change makes more
>sense with more context:
> 7. If the Newsgroups header contains one or more moderated groups
> and the proto-article does not contain an Approved header field,
> the injecting agent SHOULD forward it to a moderator as
> specified in Section 3.4.1. If the article cannot be forwarded
> to a moderator for some reason, it MUST be rejected.
OK, the wording could be better. How about:
7. If the Newsgroups header contains one or more moderated groups and
the proto-article does not contain an Approved header field, the
injecting agent MUST either forward it to a moderator as specified
in Section 3.4.1 or, if that is not possible, reject it.
>> 9. [-1] (3.4,3.5) Prepending of <path-identity> to Path by injecting
>> agent reduced from MUST to SHOULD.
>> I would accept that prepending the <path-diagnostic> POSTED is a SHOULD.
>And that's exactly what my language says. Here's the text:
> 9. The injecting agent SHOULD then prepend the <path-identity> of
> the injecting agent followed by "!.POSTED", optionally "." and
> the FQDN or IP address of the source, and a further "!" to the
> content of the Path header field. If the injecting agent does
> not support the use of <diag-keyword>, it MUST instead prepend
> its <path-identity> followed by "!"; one or the other of these
> mechanisms MUST be used.
OK, so it's a matter of wording. I think something based on my USEPRO
would be better, such as
9. The injecting agent MUST then prepend to the content of the Path
header field the its own <path-identity> followed by a "!", which
SHOULD then be followed by a ".", the <diag-keyword> "POSTED" and
a further "!".
I just noticed there that you also had an optional FQDN or IP address
after the "POSTED". I don't think that was ever the intention of what we
discussed during USEFOR, but as a matter of wording it could easily be
added to the revised text.
>The language is phrased similarly for relaing and serving agents.
Yes, similar changes are needed there, and I would suggest looking again
at my USEPRO version. The essential difference in those cases is that I
described the stuff to be added from left-to-right (which is how I think
people will tend to read it), so that you do the obligatory
<path-identity> first, and then the various possibilities for the
diagnostics.
>Changing the <path-diagnostic> from a MUST to a SHOULD is an intentional
>change.
Yes, I am happy with that (the latest USEPRO left it open for discussion).
>> 10. [00] (3.4) Folding of Path header when length becomes excessive
>> reduced from SHOULD to MAY.
>Intentional change.
I am neutral on this one. Let others decide.
>> 11. [-1] (3.4) Recommendation that each injecting agent SHOULD use a
>> consistent format for Injection-Info removed.
>> Otherwise, the task of chasing the bad guys becomes harder.
>Intentional change.
>I don't see much purpose served by this requirement and the meaning is
>very murky to me.
USEFOR allows a lot of ways for filling in the Injection-Info. I just
don't want to see a single injecting agent, on successive articles,
hopping around at random between various forms of posting-host,
posting-account and logging-data just in order to force its way past my
killfile.
>> 12. [-1] (3.5) The significance of "world" and "local" in the
>> Distribution header are no longer mentioned.
>> We recently took care to reserve these in USEFOR, but that requires
>> backup here to ensure the protocol fulfils that promise.
>Intentional change.
>Most everything useful to be said about "world" and "local" was already
>said in USEFOR, and "world" in particular SHOULD NOT be used anyway (as
>already stated in USEFOR). Distribution is generally a waste of effort,
>and I'd rather keep the rules related to it to a minimum since in practice
>few sites are ever going to care.
I was more concerned about 'local' than I was about 'world' (which is
unnecessary, though it was seen at one time). If you want to rely on the
wording in USEFOR, then please add something like:
"See [USEPRO] section 3.2.4 for the significance of the reserved
<dist-name>s "world" and "local".
>> 15. [-1] (3.5) Relaying agent MUST reject any article already already
>> sent to it.
>Intentional change.
>Every relayer I've ever heard of keeps a history file and has to to be
>usable in the real world. The protocol breaks down badly if you have
>multiple peers and don't do this. This is a core part of the flood-fill
>protocol.
OK, you have persuaded me. And it is also needed too to check for stale
Injection-Dates, which I _did_ have in USEPRO.
>> 16. [-1] (3.5) Relaying agent SHOULD deal with cancel message (if it
>> chooses to honour it)
>> Yes, USEPRO says the same, but I suspect both are wrong, because it is
>> serving/storage agents that should be doing this (all a pure relaying
>> agent can do is then not to propagate it further).
>Not actually a change, but I think this should stay. Here's what it says:
> 5. It SHOULD reject any article that matches an already-received
> cancel control message or the contents of the Supersedes header
> field of an accepted article, provided that the relaying agent
> chose (on the basis of local site policy) to honor that cancel
> control message or Supersedes header field.
Agreed, but my concern was that this was a matter for serving agents
rather than for relaying agents. If you aren't actially storing the
article, then there is little point in trying to implement cancels. Just
relay it on, and let the agents that _do_ store it decide whether to
honour the cancel or not. Do current relay-only agents currently bother to
do this?
>> 20. [00] (3.5) Relaying agent MAY add a new Xref header.
>> Why might it want to do that?
>Intentional change.
>Because the same piece of software also implements a serving agent, which
>has to add the Xref header, and the same code path is used for all
>articles received by the server whether they are for further relaying or
>local serving.
Well if that is the case it was nominally the serving function that added
the Xref (section 3.6). Now whether such an agent first stores the article
and then relays what it has stored, or whether it relays first and then
stores after is none of our business. So if you were just trying to cover
the first of those cases, where the new Xref might then escape by relaying,
then I think it could be worded better. How about:
Any Xref header, whether present on input or added by an associated
local serving agent, MAY be deleted before relaying.
>> 21. [00] (3.6) Storage of control messages in the (pseudo) hierarchy
>> control(.*) is a SHOULD.
>> Is this any of our business?
>Intentional change. See previous discussion.
I remain neutral [00].
Alert! Duplicated issue number #21!
>> 21. [-1] (3.6) No mention of processing cancel messages by
>> serving/storage agents.
>The same language is here as for relaying agents, but you may have missed
>it because it's combined with another similar point:
> 3. It SHOULD reject any article that has already been successfully
> sent to it or that matches an already-received and honored cancel
> message or Supersedes header field following the same rules as a
> relaying agent (Section 3.5).
OK, point taken, but it might be better split out. Moreover, the first bit
corresponds to step 2 of relaying which is a MUST, as is the matter of
stale articles (which is also a history file matter).
>Yes, there is no specific mention in the duties section of acting on
>control messages for already received messages. This is for the same
>reason that there's no specific mention in the duties section of acting on
>newgroup, rmgroup, or checkgroups control messages.
OK, but people will think it odd that you mention the obscure case of
cancels which arise first here, but make to mention of the common case.
Perhaps an extra step to say that it SHOULD then process any control
messages (including cancels) in acccordance with section 5.
>> 22. [00] (3.7) Removed recommendation that reading agent SHOULD have the
>> capability to display the raw article 'as received'.
>Intentional change.
Frank and Forrest have commented on this. Seems controversial. I remain
neutral for now.
>> 23. [-1] (3.8) Moderators merely "encouraged" (was SHOULD) to retain
>> existing <msg-id>.
>Intentional change.
Seems controversial.
>> 24. [00] (3.2.1) Removed provision (MAY) for outgoing gateway to change
>> Content-Transfer-Encoding.
>I don't consider this to be a change. There's nothing that says that it
>*can't* change the Content-Transfer-Encoding,...
Sure, but it may be helpful to draw attention to it. I remain [00].
>> 26. [-1] (4.2,4.3) Removed requirement that
>> application/news-[groupinfo,checkgroups] MUST NOT be used except with
>> relevant control messages.
>> Application types are inherently dangerous ...
>Intentional change.
>I can't think of an example of how application/news-groupinfo could be
>dangerous. It's nothing but a bit of text giving a newsgroup name and a
>description. In and of itself, it has no force, power, or action
>whatsoever.
Application types are intended to cause applications to be invoked, with
possible side effects. In this case, the side efect is that a line gets
added/changed in the Newsgroups file. Fine when it is part of a newgroup
message that you intend to honour, but not anywhere else without explicit
human intervention. Likewise for application/checkgroups.
>Prohibiting use of these media types in all other situations is overly
>broad and seems unnecessary to me. If I mail a checkgroups for a
>hierarchy to another newsadmin, I don't see any reason why I should be
>prohibited from using application/news-checkgroups for that mail message.
OK, in that case what you need to say is:
This media type is intended to be used in conjunction with the
newgroup control message as described in section 5.2.1. It MUST NOT be
automatically invoked in any other context without explicit human
intervention.
and similarly for checkgroups.
Alert! Duplicated issue number #27!
>> 27. [-1] (5) Nothing stated regarding Newsgroups header of Control
>> messages.
>> Was SHOULD include the groups affected (& possibly others). Without
>> that, control messages won't propagate to the places they are likely to
>> be needed.
>Intentional change, but after looking at the INN code, I was wrong.
OK.
>> 28. [00] (5) Subjects starting with 'cmsg' not accompanied by a Control
>> header MAY be rejected.
>> We discussed this at some stage, but I don't think we ever agreed it one
>> way or the other.
>Intentional change.
Others have commented, so controversial. I remain [00] for now.
>> 29. [00] (5.2) Newsgroup-names MUST be checked against disallowed names
>> before group control message is honoured.
>Intentional change.
>It's just as important for checkgroups as it is for newgroup.
Point taken.
>> 30. [-1] (5.2) Nothing said about content of Approved header.
>> Surely, it SHOULD identify the person/identity/role of the issuer, ...
>Intentional change.
>The content of the Approved header serves no protocol purpose, and USEFOR
>already adequately covers the definition of its content. Control message
>authorization is done on the basis of the Sender or From header
>(preferrably in combination with a digital signature).
It asserts that this is an "authorized" message (just what "authorized"
means is a separate issue which I shall raise later). As such, it is
pretty pointless unless it identifies the entity that is authorized, and
all news servers that I know of can be configured with the identity that
is to be recognized for each hierarchy. Of course, it really needs to be
digitally signed to make it effective, but that again is a separate issue.
I think the WG early on took the view that the Approved header would
simply fall into disrepute if one could always get away with
Approved: kibo
and that it therefore needed some firmer language (digitally signed in due
course). I remain of that view.
>> 31. [-1] (5.2.1) Newgroup message SHOULD include application/group-info.
>> Was MUST ...
>Intentional change.
>Newsgroup descriptions are optional in the protocol and newgroup messages
>without descriptions work fine with existing software.
OK, if absence means that it just creates a Newsgroups file entry with
just the name of the newsgroup, and maybe the TAB, and certainly the
" (Moderated)" if needed. Likewise for checkgroups. So you need something
like:
The application/group-info MUST be included if the group is moderated,
in which case it MUST include a <moderation-flag>. For other groups, it
MAY be omitted if there is no <newsgroup-description> to be provided.
>> 32. [-1] (5.2.1) Newgroup message containing other attachments in
>> addition to news-groupinfo to be multipart/related (was
>> multipart/mixed).
>Unintentional change.
OK.
>> 34. [00] (5.2.3) Checkgroups SHOULD contain sub-hierarchies excluded by
>> chkscope, for backwards compatibility.
>> Fine in principle, but maybe not implementable. If de.alt.* were as
>> chaotic as alt.* (is it?) there would be no way the admins of de.* could
>> include a defnitive list of de.alt.*. So some qualification is needed.
>Intentional change. See previous discussion.
Then it needs to say:
and SHOULD contain ... unless no definitive list for such
sub-hierarchies is available.
which, for a sufficuently disorganized alt-like hierarchy might well be
the case.
>> 36. [00] (5.2.3) Body of checkgroups SHOULD be application/checkgroups,
>> and otherwise SHOULD treat as if it were.
>> Why does that SHOULD differ from the corresponding MUST in newgroups? I
>> would have expected "SHOULD ... and otherwise MUST ...".
>Intentional change.
>What MUST in newgroup? Scanning the body in newgroup messages without a
>MIME part is a MAY. It's raised to SHOULD in checkgroups because unlike
>for newgroups, this is actually necessary to honor the checkgroups.
OK, I seem to have misread a MUST somewhere. Objection withdrawn.
>> 37. [00] (5.3) Body of cancel message MAY contain ...explanatory text.
>> Was SHOULD.
>Intentional change.
Frank seems to disagree. So do I, so controversial I think.
>> 38. [-1] (5.3) No mention of who should be in From/Sender of cancel
>> message.
>> Agreed it is not the person who posted the original article. USEPRO says
>> it "should" be the person who issued the cancel. That should really be
>> "SHOULD".
>Intentional change.
>The From/Sender of a cancel message is no different in definition than the
>From/Sender of any other Netnews message, and as such I think USEFOR
>covers the situation sufficiently. I don't see a reason to call special
>attention to the fields for cancel messages when cancel messages are no
>different than normal articles in this regard.
I think the point is that we have made a change from RFC 1036 here. If you
reworded it as:
Contrary to [RFC1036], the From and Sender fields SHOULD contain the
address of the issuer of the cancel, as opposed to that of the poster
of the cancelled article. That former requirement only ...
then I would be happy. s/SHOULD/should/ if you like.
>> 39. [-1] (5.5) The <relayer-name> in ihave and sendme control messages
>> is now optional.
>> It is required to be present in USEPRO and in Son-of-1036.
>Unintentional change.
>USEPRO offers two different syntaxes, one of which is marked SHOULD NOT,
>and I found that confusing. I tried to combine them and missed this
>distinction. (So, actually, it's not required to be present in USEPRO;
>it's only a SHOULD NOT to omit it.)
I think the syntax you need, if you don't want to do it my way, is
Ihave-command = "ihave" Ihave-argument
Ihave-argument = 1*WSP *( msg-id 1*WSP ) relayer-name
and then say somewhere that , if there is a list of <msg-id>s in the body,
there MUST NOT be any <msg-id>s in the Ihave-argument (ditto for sendme).
The reason I did it the way I did was because it seemed, even at the time
Henry was writing, the practice of putting them in the Ihave-argument was
already long dead, and there was little reason even to continue supporting
it.
>> 40. [-1] (5.5) Format of batched news in response to sendme removed.
>> Note that this was a format 'on the wire' and was intended to be sent on
>> the same 'wire' as the ihave and sendme messages ...
>Intentional change.
>Specifying the behavior of the transport layer is out of the scope of this
>document. Discussion of the UUCP batch format no more belongs here than
>does a discussion of dot-escaping in NNTP.
If we are allowed to specify the format of news articles 'on the wire'
(which is what USEFOR is all about), then why on earth cannot we specify
the format of the only other thing that is ever put on the same wire? It
is not the method of transport we are specifying, but the format of the
data - otherwise there will be no interoperability.
>> 41. [-1] (5.5) The protocol for using the 'to.' newsgroup-name is
>> omitted.
>> Insofar as this protocol may still occasionally be used, it needs to be
>> documented (as with the batch format).
>I have:
> These control messages are normally sent as point-to-point articles
> between two sites and not then sent on to other sites. Newsgroups
> beginning with "to." are reserved for such point-to-point
> communications.
>Son-of has:
> It is conventional to reserve newsgroup names beginning with
> "to." for test messages sent on an essentially point-to-
> point basis (see also the ihave/sendme protocol described in
> section 7.2); newsgroup names beginning with "to." SHOULD
> not be used for any other purpose. The second (and possibly
> later) components of such a name should, together, comprise
> the relayer name (see section 5.6) of a relayer. The news-
> group exists only at the named relayer and its neighbors.
> The neighbors all pass that newsgroup to the named relayer,
> while the named relayer does not pass it to anyone.
And USEPRO has much the same.
I think the minimum that should be said is that, syntactically, this funny
newsgroup-name consists of "to." followed by a <relayer-name> (we have
already said what <relayer-name>s are), and that control messages using
this notation MUST NOT be propagated to any site other than the sending
site and the <relayer-name> specified. So we leave it to the implementor
to devise how he causes it to propagate to the one site it is allowed to
propagate to (but we have already offered a strong hint that this is all
designed for use with UUCP).
>> 42. [-1] (5.6) Obsolete control messages SHOULD NOT be sent/honoured/
>> Was MUST NOT. Cf the MUST NOT for obsolete headers in USEFOR.
>Intentional change.
>MUST NOT is unwarranted. Sending them doesn't break anything.
Except cause mailbombs, which seems like severe breakage to me.
We have forbidden the obsolete headers in USEOR, so why not here?
--
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133 Web: http://www.cs.man.ac.uk/~chl
Email: chl@xxxxxxxxxxxxxxxx Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9 Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5
--
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133 Web: http://www.cs.man.ac.uk/~chl
Email: chl@xxxxxxxxxxxxxxxx Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9 Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5