From: Charles Lindsey (chl@clw.cs.man.ac.uk)
Date: Fri Apr 28 2000 - 11:21:34 CDT
Time to get back to generating text for the draft.
I have taken the draft that Russ submitted re Gatewaying a couple of
months back, and tidied it up, bringing it into line with the house
style, and the usage of terminology elsewhere in the document.
The only change I have made is to remove Russ' section on moderation,
and instead created a new section on "Duties of Moderators". You should
look at this carefully, noting my various pseudo-comments.
Here is the new material, followed by the diffs wrt Russ's text.
The latest text of section 8 will appear on the website shortly as
section_8.03.01.
-------------------------------------------------------------------------------
8.7. Duties of a Moderator
A Moderator receives news articles by email, decides whether to
accept them and, if so, either injects them into the news stream or
forwards them to further moderators.
A moderator processes an article, as submitted to any newsgroup that
he moderates, as follows:
1. He decides, on the basis of whatever moderation policy applies to
his group, whether to accept or reject the article. He MAY do this
manually, or else partially or wholly with the aid of appropriate
software for whose operation he is then responsible. He MAY modify
the article if that is in accordance with the applicable
moderation policy (and in particular he MAY remove redundant
headers and add Comments and other informational headers). He MAY
inform the poster as to whether the article has been accepted or
rejected.
If the article is rejected, then it fails for all the newsgroups
for which it was intended (in particular the moderator MUST NOT
resubmit the article, with a reduced Newsgroups header, to any
remaining groups). If the article is accepted, the moderator
proceeds with the following steps.
2. The Date header MUST be retained, except that if it is stale (5.1)
the article SHOULD be rejected. Any local headers (4.2.2.3) or
variant headers (4.2.2.4) MUST be removed, except that a Path
header MAY be truncated to only its pre-injection region (5.6.3).
[Also remove any Injector-Info header, when we have that.]
[Note several differences from Kent Landfield's 'Moderator's Handbook'.
The original Date and Message-ID are retained.
Any Distribution header is retained.
Any Sender header is retained.
Various other minor headers are retained (though the moderator MAY, of
course, remove them.
]
3. He adds an Approved header (6.12) containing a mailbox identifying
himself (or, if the article already contains an Approved header
from another moderator, he adds that identifying information to
it). He MAY also add further headers to authenticate that the
article has been properly approved.
[That can be strengthened when we have defined proper authentication
mechanisms.]
4. If the Newsgrpups header contains further moderated newsgroups for
which approval has not already been given, he forwards the article
to the moderator of the leftmost such group (which, if this
standard has been followed correctly, will always be the group
immediately to the right of the group(s) for which he is
responsible). However, he MUST NOT alter the order in which the
newsgroups are listed in the Newsgroups header.
5. Otherwise, he causes the article to be injected, having first
observed all the duties of a posting agent (8.5).
NOTE: This standard does not prescribe how the moderator or
moderation policy for each newsgroup is established; rather it
assumes that whatever agencies are responsible for the relevant
network or hierarchy (1.1) will have made appropriate
arrangements in that regard.
It SHOULD be the case that articles will be received by the moderator
encapsulated as an object of Content-Type application/news-
transmission (8.2.2), or possibly encapsulated but without an
explicit Content-Type header. In such a case, the complete article is
immediately available for processing by the moderator.
However, prior to the introduction of this standard, it was more
common for injecting agents to transform proto-articles into mail
messages, mixing the Netnews headers with the Mail headers.
Moderators SHOULD therefore be prepared to accept submission in this
format, although they need then to be aware of the Duties of an
Incoming Gateway (8.8.2) (and, in particular, they SHOULD adopt the
Message-ID and Date headers of the mail message, though they SHOULD
NOT add any Sender header).
8.8. Duties of a Gateway
A Gateway transforms an article into the native message format of
another medium, or translates the messages of another medium into
news articles. Encapsulation of a news article into a message of MIME
type application/news-transmission, or the subsequent undoing of that
encapsulation, is not gatewaying, since it involves no transformation
of the article.
There are two basic types of gateway, the Outgoing Gateway that
transforms a news article into a different type of message, and the
Incoming Gateway that transforms a message from another medium into a
news article and injects it into a Netnews system. These are handled
separately below.
The primary dictat for a gateway is:
Above all, prevent loops.
Transformation of an article into another medium stands a very high
chance of discarding or interfering with the protection inherent in
the news system against duplicate articles. The most common problem
caused by gateways is "spews," gateway loops that cause previously
posted articles to be reinjected repeatedly into Usenet. To prevent
this, a gateway MUST take precautions against loops, as detailed
below.
If bidirectional gatewaying (both an incoming and an outgoing
gateway) is being set up between Netnews and some other medium, the
incoming and outgoing gateways SHOULD be coordinated to avoid
reinjection of gated articles. Circular gatewaying (gatewaying a
message into another medium and then back into Netnews) SHOULD NOT be
done; encapsulation of the article SHOULD be used instead where this
is necessary.
A second general principal of gatewaying is that the transformations
applied to the message SHOULD be as minimal as possible while still
accomplishing the gatewaying. Every change made by a gateway
potentially breaks a property of one of the media or loses
information, and therefore only those transformations made necessary
by the differences between the media should be applied.
It is worth noting that safe bidirectional gatewaying between a
mailing list and a newsgroup is far easier if the newsgroup is
moderated. Posts to the moderated group and submissions to the
mailing list can then go through a single point that does the
necessary gatewaying and then sends the message out to both the
newsgroup and the mailing list at the same time, eliminating most of
the possibility of loops. Bidirectional gatewaying between a mailing
list and an unmoderated newsgroup, in contrast, is difficult to do
correctly and is far more fragile.
Newsgroups intended to be bidirectionally gated to a mailing list
SHOULD therefore be moderated where possible, even if the moderator
is a simple gateway and injecting agent that correctly handles
crossposting to other moderated groups and otherwise passes all
traffic.
8.8.1. Duties of an Outgoing Gateway
From the perspective of Netnews, an outgoing gateway is just a
special type of reading agent. The exact nature of what the outgoing
gateway will need to do to articles depends on the medium to which
the articles are being gated. The operation of the outgoing gateway
is only subject to additional constraints in the presence of one or
more corresponding incoming gateways back from that medium to
Netnews, since this opens the possibility of loops.
It is recommended, however, that the following practices be followed
by all outgoing gateways regardless of whether there is known to be a
related incoming gateway, both as a precautionary measure and as a
guideline to quality of implementation.
1. Only the minimal necessary changes should be made, as stated
above.
2. The message identifier of the news article should be preserved if
at all possible, preferably as or within the corresponding unique
identifier of the other medium, but if not at least as a comment
in the message. This helps greatly with preventing loops.
3. The Date of the news article should also be preserved if possible,
for similar reasons.
4. The message should be tagged in some way so as to prevent its
reinjection into Netnews. This may be impossible to do without
knowledge of potential incoming gateways, but it is better to try
to provide some indication even if not successful; at the least, a
human-readable indication that the article should not be gated
back to Netnews can help locate a human problem.
5. News control messages should not be gated to another medium unless
they would somehow be meaningful in that medium.
8.8.2. Duties of an Incoming Gateway
The incoming gateway has the serious responsibility of ensuring that
all of the requirements of this standard are met by the articles that
it forms. In addition to its special duties as a gateway, it bears
all of the duties and responsibilities of an injecting agent as well,
and additionally has the same responsibility of a relaying agent to
reject articles that it has already gatewayed.
An incoming gateway MUST NOT gate the same message twice. It may not
be possible to ensure this in the face of mangling or modification of
the message, but at the very least a gateway, when given a copy of a
message it has already gated identical except for trace headers (like
Received in e-mail or Path in Netnews) MUST NOT gate the message
again. An incoming gateway SHOULD take precautions against having
this rule bypassed by modifications of the message that can be
anticipated.
News articles prepared by gateways MUST be legal news articles. In
particular, they MUST include all of the mandatory headers and MUST
fully conform to the restrictions on said headers. This often
requires that a gateway function not only as a relaying agent, but
also partly as a posting agent, aiding in the synthesis of a
conforming article from non-conforming input.
Incoming gateways MUST NOT pass control messages (articles containing
a Control header) without removing or renaming that header. Gateways
MAY, however, generate their own cancel messages, under the general
allowance for injecting agents to cancel their own messages (7.5).
If a gateway receives a message that it can determine is a valid
equivalent of a cancel message in the medium it is gatewaying, it
SHOULD discard that message without gatewaying it, generate a
corresponding cancel message of its own, and inject that cancel
message.
Incoming gateways MUST NOT inject control messages other than
cancels. Encapsulation SHOULD be used instead of gatewaying, when
direct posting is not possible or desirable.
NOTE: It is not unheard of for mail-to-news gateways to be used
to post control messages, but encapsulation should be used for
these cases instead. Gateways by their very nature are
particularly prone to loops. Spews of normal articles are bad
enough; spews of control messages with special significance to
the news system, possibly resulting in high processing load or
even e-mail sent for every message received, are catastrophic.
It is far preferable to construct a system specifically for
posting control messages that can do appropriate consistency
checks and authentication of the originator of the message.
[Is that really so bad? I do it regularly.]
If there is a message identifier that fills a role similar to that of
the Message-ID header in news, it SHOULD be used in the formation of
the message identifier of the news article, perhaps with
transformations required to meet the uniqueness requirement of
Netnews. This transformation SHOULD be designed so that two messages
with the same identifier generate the same Message-ID header.
NOTE: Message identifiers play a central role in the prevention
of duplicates, and their correct use by gateways will do much to
prevent loops. Netnews does, however, require that message
identifiers be unique, and therefore message identifiers from
other media may not be suitable for use without modification. A
balance must be struck by the gateway between preserving
information used to prevent loops and generating unique message
identifiers.
Exceptionally, if there are multiple incoming gateways for a
particular set of messages, each incoming gateway SHOULD generate a
message identifier unique to that gateway. Each incoming gateway
nonetheless MUST ensure that it does not gate the same message twice.
[Should that SHOULD be a MAY?]
NOTE: Consider the example of two gateways of a given mailing
list into the world-wide Usenet newsgroups, both of which
preserve the mail message identifier. Each newsgroup may then
receive a portion of the messages (different sites seeing
different portions). In these cases, where there is no one
"official" gateway, some other method of generating message
identifiers has to be used to avoid collisions. It would
obviously be preferable for there to be only one gateway which
crossposts, but this may not be possible to coordinate.
If no date information is available, the gateway MAY supply a Date
header with the gateway's current date. If only partial information
is available (e.g. date but not time), this SHOULD be fleshed out to
a full Date header by adding default values rather than discarding
this information. Only in very exceptional circumstances should Date
information be discarded, as it plays an important role in preventing
reinjection of old messages.
An incoming gateway MUST add a Sender header to the news article it
forms containing the mailbox of the administrator of the gateway.
Problems with the gateway may be reported to this address. The
display-name portion of this mailbox SHOULD indicate that the entity
responsible for injection of the message is a gateway. If the
original message already had a Sender header, it SHOULD be renamed so
that its contents can be preserved.
8.8.3. Example
To illustrate the type of precautions that should be taken against
loops, here is an example of the measures taken by one particular
combination of mail-to-news and news-to-mail gateways at Stanford
University designed to handle bidirectional gatewaying between
mailing lists and unmoderated groups.
1. The news-to-mail gateway preserves the message identifier of the
news article in the generated mail message. The mail-to-news
gateway likewise preserves the mail message identifier provided
that it is syntactically valid for Netnews. This allows the news
system's built-in suppression of duplicates to serve as the first
line of defense against loops.
2. The news-to-mail gateway adds an X-Gateway header to all messages
it generates. The mail-to-news gateway discards any incoming
messages containing this header. This is robust against mailing
list managers that replace the message identifier, and against any
number of mail hops, provided that the other message headers are
preserved.
3. The mail-to-news gateway inserts the host name from which it
received the mail message in the pre-injection region of the Path
(5.6.3). The news-to-mail gateway refuses to gateway any message
that contains the list server name in the pre-injection region of
its Path header. This is robust against any amount of munging of
the message headers by the mailing list, provided that the mail
only goes through one hop.
4. The mail-to-news gateway is designed never to generate bounces to
the envelope sender. Instead, articles that are rejected by the
news server (for reasons not warranting silent discarding of the
message) result in a bounce message sent to an errors address
known not to forward to any mailing lists, so that they can be
handled by the news administrators.
These precautions have proven effective in practice at preventing
loops for this particular application (bidirectional gatewaying
between mailing lists and locally distributed newsgroups where both
gateways can be designed together). General gatewaying to world-wide
newsgroups poses additional difficulties; one must be very wary of
strange configurations, such as a newsgroup gated to a mailing list
which is in turn gated to a different newsgroup.
-------------------------------------------------------------------------------
*** gatewaying.orig Fri Apr 28 16:48:37 2000
--- /tmp/bar Fri Apr 28 17:18:41 2000
***************
*** 1,26 ****
! 8.7. Duties of a Gateway
A Gateway transforms an article into the native message format of
another medium, or translates the messages of another medium into
! news articles. Encapsulation of a news article into a message of
! MIME type application/news-transmission, or the subsequent undoing of
! that encapsulation, is not gatewaying, since it involves no
! transformation of the article.
! There are two basic types of Gateway, the Outgoing Gateway that
transforms a news article into a different type of message, and the
Incoming Gateway that transforms a message from another medium into a
! news article and injects it into a netnews system. These are handled
separately below.
! The primary dictate for a gateway is:
! Above all, prevent loops.
Transformation of an article into another medium stands a very high
chance of discarding or interfering with the protection inherent in
! the news system against duplicate articles. The most common problem
caused by gateways is "spews," gateway loops that cause previously
! posted articles to be reinjected repeatedly into Usenet. To prevent
this, a gateway MUST take precautions against loops, as detailed
below.
--- 1,26 ----
! 8.8. Duties of a Gateway
A Gateway transforms an article into the native message format of
another medium, or translates the messages of another medium into
! news articles. Encapsulation of a news article into a message of MIME
! type application/news-transmission, or the subsequent undoing of that
! encapsulation, is not gatewaying, since it involves no transformation
! of the article.
! There are two basic types of gateway, the Outgoing Gateway that
transforms a news article into a different type of message, and the
Incoming Gateway that transforms a message from another medium into a
! news article and injects it into a Netnews system. These are handled
separately below.
! The primary dictat for a gateway is:
! Above all, prevent loops.
Transformation of an article into another medium stands a very high
chance of discarding or interfering with the protection inherent in
! the news system against duplicate articles. The most common problem
caused by gateways is "spews," gateway loops that cause previously
! posted articles to be reinjected repeatedly into Usenet. To prevent
this, a gateway MUST take precautions against loops, as detailed
below.
***************
*** 27,34 ****
If bidirectional gatewaying (both an incoming and an outgoing
! gateway) is being set up between netnews and some other medium, the
incoming and outgoing gateways SHOULD be coordinated to avoid
! reinjection of gated articles. Circular gatewaying (gatewaying a
! message into another medium and then back into netnews) SHOULD NOT be
done; encapsulation of the article SHOULD be used instead where this
is necessary.
--- 27,34 ----
If bidirectional gatewaying (both an incoming and an outgoing
! gateway) is being set up between Netnews and some other medium, the
incoming and outgoing gateways SHOULD be coordinated to avoid
! reinjection of gated articles. Circular gatewaying (gatewaying a
! message into another medium and then back into Netnews) SHOULD NOT be
done; encapsulation of the article SHOULD be used instead where this
is necessary.
***************
*** 36,40 ****
A second general principal of gatewaying is that the transformations
applied to the message SHOULD be as minimal as possible while still
! accomplishing the gatewaying. Every change made by a gateway
potentially breaks a property of one of the media or loses
information, and therefore only those transformations made necessary
--- 36,40 ----
A second general principal of gatewaying is that the transformations
applied to the message SHOULD be as minimal as possible while still
! accomplishing the gatewaying. Every change made by a gateway
potentially breaks a property of one of the media or loses
information, and therefore only those transformations made necessary
***************
*** 41,58 ****
by the differences between the media should be applied.
! 8.7.1. Duties of an Outgoing Gateway
! From the perspective of netnews, an Outgoing Gateway is just a
! special type of news reader. The exact nature of what the outgoing
gateway will need to do to articles depends on the medium to which
! the articles are being gated. The operation of the outgoing gateway
is only subject to additional constraints in the presence of one or
more corresponding incoming gateways back from that medium to
! netnews, since this opens the possibility of loops.
It is recommended, however, that the following practices be followed
by all outgoing gateways regardless of whether there is known to be a
! related incoming gateway, both as a precautionary measure and as
! quality of implementation guidelines.
1. Only the minimal necessary changes should be made, as stated
--- 41,75 ----
by the differences between the media should be applied.
! It is worth noting that safe bidirectional gatewaying between a
! mailing list and a newsgroup is far easier if the newsgroup is
! moderated. Posts to the moderated group and submissions to the
! mailing list can then go through a single point that does the
! necessary gatewaying and then sends the message out to both the
! newsgroup and the mailing list at the same time, eliminating most of
! the possibility of loops. Bidirectional gatewaying between a mailing
! list and an unmoderated newsgroup, in contrast, is difficult to do
! correctly and is far more fragile.
!
! Newsgroups intended to be bidirectionally gated to a mailing list
! SHOULD therefore be moderated where possible, even if the moderator
! is a simple gateway and injecting agent that correctly handles
! crossposting to other moderated groups and otherwise passes all
! traffic.
!
! 8.8.1. Duties of an Outgoing Gateway
!
! From the perspective of Netnews, an outgoing gateway is just a
! special type of reading agent. The exact nature of what the outgoing
gateway will need to do to articles depends on the medium to which
! the articles are being gated. The operation of the outgoing gateway
is only subject to additional constraints in the presence of one or
more corresponding incoming gateways back from that medium to
! Netnews, since this opens the possibility of loops.
It is recommended, however, that the following practices be followed
by all outgoing gateways regardless of whether there is known to be a
! related incoming gateway, both as a precautionary measure and as a
! guideline to quality of implementation.
1. Only the minimal necessary changes should be made, as stated
***************
*** 59,66 ****
above.
! 2. The Message-ID of the news article should be preserved if at all
! possible, preferably as or within the corresponding unique
identifier of the other medium, but if not at least as a comment
! in the message. This helps greatly with preventing loops.
3. The Date of the news article should also be preserved if possible,
--- 76,83 ----
above.
! 2. The message identifier of the news article should be preserved if
! at all possible, preferably as or within the corresponding unique
identifier of the other medium, but if not at least as a comment
! in the message. This helps greatly with preventing loops.
3. The Date of the news article should also be preserved if possible,
***************
*** 67,113 ****
for similar reasons.
! 4. The message should be tagged in some way so as to prevent it's
! reinjection into netnews. This may be impossible to do without
! knowledge of potential incoming gateways, but it's better to try
to provide some indication even if not successful; at the least, a
human-readable indication that the article should not be gated
! back to netnews can help a human locate problems.
! 5. News control messages should not be gated to other media unless
they would somehow be meaningful in that medium.
! 8.7.2. Duties of an Incoming Gateway
The incoming gateway has the serious responsibility of ensuring that
all of the requirements of this standard are met by the articles that
! it forms. In addition to its special duties as a gateway, it bears
! all of the duties and responsibilities of an injector as well, and
! additionally has the same responsibility of a relaying agent to
reject articles that it has already gatewayed.
! An incoming gateway MUST NOT gate the same message twice. It may not
be possible to ensure this in the face of mangling or modification of
the message, but at the very least a gateway, when given a copy of a
message it has already gated identical except for trace headers (like
! Received in e-mail or Path in netnews) MUST NOT gate the message
again. An incoming gateway SHOULD take precautions against having
! this rule bypassed violated by modifications of the message that can
! be anticipated.
! News articles prepared by gateways MUST be legal news articles. In
particular, they MUST include all of the mandatory headers and MUST
! fully conform to the restrictions on said headers. This often
! requires that a gateway function not only as a relayer, but also
! partly as a posting agent, aiding in the synthesis of a conforming
! article from non-conforming input.
Incoming gateways MUST NOT pass control messages (articles containing
! a Control header) without removing or renaming that header. Gateways
MAY, however, generate their own cancel messages, under the general
! allowance for injectors cancelling their own messages. If a gateway
! receives a message that it can determine is a valid equivalent of a
! cancel message in the medium it is gatewaying, it SHOULD discard that
! message without gatewaying it, generate a corresponding cancel
! message, and inject that cancel message.
Incoming gateways MUST NOT inject control messages other than
--- 84,131 ----
for similar reasons.
! 4. The message should be tagged in some way so as to prevent its
! reinjection into Netnews. This may be impossible to do without
! knowledge of potential incoming gateways, but it is better to try
to provide some indication even if not successful; at the least, a
human-readable indication that the article should not be gated
! back to Netnews can help locate a human problem.
! 5. News control messages should not be gated to another medium unless
they would somehow be meaningful in that medium.
! 8.8.2. Duties of an Incoming Gateway
The incoming gateway has the serious responsibility of ensuring that
all of the requirements of this standard are met by the articles that
! it forms. In addition to its special duties as a gateway, it bears
! all of the duties and responsibilities of an injecting agent as well,
! and additionally has the same responsibility of a relaying agent to
reject articles that it has already gatewayed.
! An incoming gateway MUST NOT gate the same message twice. It may not
be possible to ensure this in the face of mangling or modification of
the message, but at the very least a gateway, when given a copy of a
message it has already gated identical except for trace headers (like
! Received in e-mail or Path in Netnews) MUST NOT gate the message
again. An incoming gateway SHOULD take precautions against having
! this rule bypassed by modifications of the message that can be
! anticipated.
! News articles prepared by gateways MUST be legal news articles. In
particular, they MUST include all of the mandatory headers and MUST
! fully conform to the restrictions on said headers. This often
! requires that a gateway function not only as a relaying agent, but
! also partly as a posting agent, aiding in the synthesis of a
! conforming article from non-conforming input.
Incoming gateways MUST NOT pass control messages (articles containing
! a Control header) without removing or renaming that header. Gateways
MAY, however, generate their own cancel messages, under the general
! allowance for injecting agents to cancel their own messages (7.5).
! If a gateway receives a message that it can determine is a valid
! equivalent of a cancel message in the medium it is gatewaying, it
! SHOULD discard that message without gatewaying it, generate a
! corresponding cancel message of its own, and inject that cancel
! message.
Incoming gateways MUST NOT inject control messages other than
***************
*** 117,163 ****
NOTE: It is not unheard of for mail-to-news gateways to be used
to post control messages, but encapsulation should be used for
! these cases instead. Gateways by their very nature are
! particularly prone to loops. Spews of normal articles are bad
enough; spews of control messages with special significance to
! the news system, possibly resulting high processing load or even
! e-mail sent for every message received, are catastrophic. It is
! far preferable to construct a system specifically for posting
! control messages that can do appropriate consistency checks and
! authentication of the originator of the message.
! If there a message identifier that fills a role similar to that of
! Message-ID in news, it SHOULD be used in the formation of the
! Message-ID of the news article, perhaps with transformations required
! to meet the uniqueness requirement of netnews. This transformation
! SHOULD be designed so that two messages with the same identifier
! receive the same Message-ID.
! NOTE: Message-ID serves a central role in the prevention of
! duplicates, and its correct use by gateways will do much to
! prevent loops. Netnews does, however, require that Message-ID
! be unique, and therefore message identifiers from other media
! may not be suitable for use without modification. A balance
! must be struck by the gateway between preserving information
! used to prevent loops and generating unique Message-IDs.
Exceptionally, if there are multiple incoming gateways for a
particular set of messages, each incoming gateway SHOULD generate a
! Message-ID unique to that gateway. Each incoming gateway nonetheless
! MUST ensure that it does not gate the same message twice.
NOTE: Consider the example of two gateways of a given mailing
! list into world-wide Usenet newsgroups, both of which preserve
! the mail Message-ID. Each newsgroup may receive only 50% of the
! messages. In these cases, where there is no one "official"
! gateway, some other method of generating Message-IDs has to be
! used to avoid collisions. It would obviously be preferable for
! there to be only one gateway which crossposts, but this may not
! be possible to coordinate.
If no date information is available, the gateway MAY supply a Date
! header with the gateway's current date. If only partial information
is available (e.g. date but not time), this SHOULD be fleshed out to
a full Date header by adding default values rather than discarding
! this information. Only in very exceptional circumstances should Date
information be discarded, as it plays an important role in preventing
reinjection of old messages.
--- 135,186 ----
NOTE: It is not unheard of for mail-to-news gateways to be used
to post control messages, but encapsulation should be used for
! these cases instead. Gateways by their very nature are
! particularly prone to loops. Spews of normal articles are bad
enough; spews of control messages with special significance to
! the news system, possibly resulting in high processing load or
! even e-mail sent for every message received, are catastrophic.
! It is far preferable to construct a system specifically for
! posting control messages that can do appropriate consistency
! checks and authentication of the originator of the message.
! [Is that really so bad? I do it regularly.]
! If there is a message identifier that fills a role similar to that of
! the Message-ID header in news, it SHOULD be used in the formation of
! the message identifier of the news article, perhaps with
! transformations required to meet the uniqueness requirement of
! Netnews. This transformation SHOULD be designed so that two messages
! with the same identifier generate the same Message-ID header.
! NOTE: Message identifiers play a central role in the prevention
! of duplicates, and their correct use by gateways will do much to
! prevent loops. Netnews does, however, require that message
! identifiers be unique, and therefore message identifiers from
! other media may not be suitable for use without modification. A
! balance must be struck by the gateway between preserving
! information used to prevent loops and generating unique message
! identifiers.
Exceptionally, if there are multiple incoming gateways for a
particular set of messages, each incoming gateway SHOULD generate a
! message identifier unique to that gateway. Each incoming gateway
! nonetheless MUST ensure that it does not gate the same message twice.
! [Should that SHOULD be a MAY?]
+
NOTE: Consider the example of two gateways of a given mailing
! list into the world-wide Usenet newsgroups, both of which
! preserve the mail message identifier. Each newsgroup may then
! receive a portion of the messages (different sites seeing
! different portions). In these cases, where there is no one
! "official" gateway, some other method of generating message
! identifiers has to be used to avoid collisions. It would
! obviously be preferable for there to be only one gateway which
! crossposts, but this may not be possible to coordinate.
If no date information is available, the gateway MAY supply a Date
! header with the gateway's current date. If only partial information
is available (e.g. date but not time), this SHOULD be fleshed out to
a full Date header by adding default values rather than discarding
! this information. Only in very exceptional circumstances should Date
information be discarded, as it plays an important role in preventing
reinjection of old messages.
***************
*** 165,209 ****
An incoming gateway MUST add a Sender header to the news article it
forms containing the mailbox of the administrator of the gateway.
! Problems with the gateway may be reported to this address. The
! display name portion of this mailbox SHOULD indicate that the entity
! responsible for injection of the message is a gateway. If the
original message already had a Sender header, it SHOULD be renamed so
that its contents can be preserved.
! 8.7.3. Moderators as Incoming Gateways
- This standard requires that submissions to moderators be sent in the
- form of encapsulated articles, but this is a change from existing
- practice before this standard. Previously, news systems transformed
- proto-articles into mail messages and sent them to the moderation
- submission address. Moderators SHOULD therefore be prepared to serve
- as gateways until news systems have been made to conform with this
- standard. A message requiring gatewaying can be identified by the
- lack of an application/news-transmission Content-Type.
-
- Some news systems did encapsulate news articles, but did so without
- MIME labeling. Moderators may therefore want to treat messages where
- the body of the message starts with valid news headers, including a
- Newsgroups header containing the name of the moderated group, as
- encapsulated news messages, particularly if the main mail headers are
- lacking required news headers such as Subject.
-
- It is worth noting that safe bidirectional gatewaying between a
- mailing list and newsgroup is far easier if the newsgroup is
- moderated. Posts to the moderated group and submissions to the
- mailing list can then go through a single point that does the
- necessary gatewaying and then sends the message out to both the
- newsgroup and the mailing list at the same time, eliminating most of
- the possibility of loops. Bidirectional gatewaying between a mailing
- list and an unmoderated newsgroup, in contrast, is difficult to do
- correctly and is far more fragile.
-
- Newsgroups intended to be bidirectionally gated to a mailing list
- SHOULD therefore be moderated where possible, even if the moderator
- is a simple gateway and injector that correctly handles crossposting
- to other moderated groups and otherwise passes all traffic.
-
- 8.7.4. Example
-
To illustrate the type of precautions that should be taken against
loops, here is an example of the measures taken by one particular
--- 188,199 ----
An incoming gateway MUST add a Sender header to the news article it
forms containing the mailbox of the administrator of the gateway.
! Problems with the gateway may be reported to this address. The
! display-name portion of this mailbox SHOULD indicate that the entity
! responsible for injection of the message is a gateway. If the
original message already had a Sender header, it SHOULD be renamed so
that its contents can be preserved.
! 8.8.3. Example
To illustrate the type of precautions that should be taken against
loops, here is an example of the measures taken by one particular
***************
*** 212,238 ****
mailing lists and unmoderated groups.
! 1. The outgoing gateway preserves the Message-ID of the news article
! in generated mail message. The incoming gateway likewise
! preserves the mail Message-ID provided that it is syntactically
! valid for netnews. This allows the news system's built-in
! suppression of duplicates to serves as the first line of defense
! against loops.
! 2. The outgoing gateway adds an X-Gateway header to all messages it
! generates. The incoming gateway discards any incoming messages
! containing this header. This is robust against mailing list
! managers that replace the message ID, and against any number of
! mail hops, provided that the other message headers are preserved.
! 3. The incoming gateway adds the host name from which it received the
! mail message to the Path tail. The outgoing gateway refuses to
! gateway any message that contains the list server name in the Path
! tail. This is robust against any amount of munging of the message
! headers by the mailing list, provided that the mail only goes
! through one hop.
! 4. The incoming gateway is designed to never generate bounces to the
! envelope sender. Instead, articles that are rejected by the news
! server (for reasons not warranting silent discarding of the
message) result in a bounce message sent to an errors address
known not to forward to any mailing lists, so that they can be
--- 202,232 ----
mailing lists and unmoderated groups.
! 1. The news-to-mail gateway preserves the message identifier of the
! news article in the generated mail message. The mail-to-news
! gateway likewise preserves the mail message identifier provided
! that it is syntactically valid for Netnews. This allows the news
! system's built-in suppression of duplicates to serve as the first
! line of defense against loops.
! 2. The news-to-mail gateway adds an X-Gateway header to all messages
! it generates. The mail-to-news gateway discards any incoming
! messages containing this header. This is robust against mailing
! list managers that replace the message identifier, and against any
! number of mail hops, provided that the other message headers are
! preserved.
! 3. The mail-to-news gateway inserts the host name from which it
! received the mail message in the pre-injection region of the Path
! (5.6.3). The news-to-mail gateway refuses to gateway any message
! that contains the list server name in the pre-injection region of
! its Path header. This is robust against any amount of munging of
! the message headers by the mailing list, provided that the mail
! only goes through one hop.
!
!
! 4. The mail-to-news gateway is designed never to generate bounces to
! the envelope sender. Instead, articles that are rejected by the
! news server (for reasons not warranting silent discarding of the
message) result in a bounce message sent to an errors address
known not to forward to any mailing lists, so that they can be
***************
*** 242,246 ****
loops for this particular application (bidirectional gatewaying
between mailing lists and locally distributed newsgroups where both
! gateways can be designed together). General gatewaying to world-wide
newsgroups poses additional difficulties; one must be very wary of
strange configurations, such as a newsgroup gated to a mailing list
--- 236,240 ----
loops for this particular application (bidirectional gatewaying
between mailing lists and locally distributed newsgroups where both
! gateways can be designed together). General gatewaying to world-wide
newsgroups poses additional difficulties; one must be very wary of
strange configurations, such as a newsgroup gated to a mailing list
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Email: chl@clw.cs.man.ac.uk Web: http://www.cs.man.ac.uk/~chl
Voice/Fax: +44 161 437 4506 Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9 Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5