Re: Empty description for moderated newsgroups

[Julien ÉLIE <julien@xxxxxxxxxxxxxxx>]

Hi Harald,

> > fr.test
> > fr.test.y\tA description.
> > fr.test.m\tA description. (Moderated)
> > fr.test.m.without.desc (Moderated)
> >
> > I do not believe the last one is valid.  And even though USEPRO states
> > that "this unusual format is backward-compatible", I think the last one
> > will be considered as an unmoderated newsgroup whose description is
> > "(Moderated)".
>
> Which software currently acts as if the last one is non-moderated?

At least INN currently does.

Example:

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
pouet.test\tTest. (Moderated)
pouet.test2 (Moderated)
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

If I feed it to controlchan (the program INN uses for checkgroups received
from peers), I receive:

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
# The following newsgroups are missing and should be added:
#
# pouet.test
# pouet.test2
#
# You can add them by executing the command(s):

/home/news/bin/ctlinnd newgroup pouet.test2 y
/home/news/bin/ctlinnd newgroup pouet.test m

# The following newsgroups descriptions are missing and should be added:
#
# pouet.test2\t(Moderated)
# pouet.test\tTest. (Moderated)
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Note the tabulations in the descriptions found.



> I can see multiple ways to write a parser for this syntax; some of them will have this problem, other won't.
> (for instance, if the check for "(Moderated)" is done independently of attempting to extract the description, there will not be a 
> problem)

Yes, I understand.  It is possible to fix it in software.
INN just splits with:
 my ($ngname, $ngdesc) = split(/\s+/, $_, 2);
before checking for " (Moderated)".

Note that USEPRO explicitly mentions " (Moderated)" with a *leading* space.


> If it's a real problem, we should be able to cite which existing software has it.

The main problem is that RFC 3977 specifies that the format of LIST
NEWSGROUPS follows that rule:

  The newsgroups list is maintained by NNTP servers to contain the name
  of each newsgroup that is available on the server and a short
  description about the purpose of the group.  Each line of this list
  consists of two fields separated from each other by one or more space
  or TAB characters (the usual practice is a single TAB).  The first
  field is the name of the newsgroup, and the second is a short
  description of the group.

    ; newsgroups
    list-content =/ list-newsgroups-content
    list-newsgroups-content =
          *(newsgroup-name WS newsgroup-description CRLF)
    newsgroup-description = S-TEXT

    WS = 1*(SP / TAB)
    S-TEXT   = U-TEXT
    U-TEXT   = P-CHAR *U-CHAR
    P-CHAR   = A-CHAR / UTF8-non-ascii
    A-CHAR   = %x21-7E
    U-CHAR   = CTRL / TAB / SP / A-CHAR / UTF8-non-ascii


Leading spaces are only separators.  Therefore, the description
is "(Moderated)".  Not " (Moderated)".
Here, USEPRO adds a requirement to parse differently such lines.

A checkgroups was not supposed to have a different format than
the newsgroups file...

Also have a look at Son-of-1036:

   checkgroups-body    = [ invalidation ] valid-groups
                             / invalidation
   invalidation        = "!" plain-component *( "," plain-component ) eol
   valid-groups        = 1*( description-line eol )
   description-line    = newsgroup-name space description
   description         = nonblank-text [ " (Moderated)" ]


In fact, it looks as though newsgroups were required to
have a description.
It is no longer the case with USEPRO:

   newsgroups-line     = newsgroup-name
                              [ 1*HTAB newsgroup-description ]
                              [ *WSP moderation-flag ]


Note that I'm fine with that.  I just wanted to warn that
it may cause problems with existing software.
Parsing is different and, when writing the newsgroups file,
it requires to add "No description.", as it is for instance
done on <ftp://ftp.isc.org/pub/usenet/CONFIG/newsgroups>:
ab.arnet\tNo description. (Moderated)

Not easy for implementors not to walk into the trap!

As far as I am concerned, I will try to cope with that it
in the next release of INN.

--
Julien ÉLIE

« Audentes fortunat iuvat. » (Virgile)