RE: The anti-abuse rDNS check that FTP gave up

["Storz, Michael" <Michael.Storz@xxxxxx>]

> From: Carl S. Gutekunst [mailto:csg@xxxxxxxxxxx]
> Sent: Wednesday, October 05, 2011 8:55 PM
> To: Keith Moore
> Cc: Storz, Michael; Valdis.Kletnieks@xxxxxx; SMTP Interest Group
> Subject: Re: The anti-abuse rDNS check that FTP gave up
> 
> Keith Moore wrote:
> > Just imagine how many wrongly rejected emails aren't reported.
> >
> > Stupid spam filtering mechanisms are a DoS attack on email.
> 
> The problem is nearly all of our anti-spam measures are empirical. We
> all know a lot of people who swear by this, that, or the other check,
> even if the only supporting evidence is that using that particular
> mechanism cut down the number of complaints. And what works on one
> stream fails miserably on another.
> 
> FWIW, I last looked at this problem about three years ago. I
> specifically wanted to know if some form of RDNS checks might be useful
> in cutting down the load on the content-based spam filters. (I was also
> checking effect and utilization of SPF and TLS.) Note that the purpose
> here was not to improve the catch rate. This was on a stream that was
> already filtered by a short-lived (60 minute) IP reputation filter, so
> that would reduce the message count from some known spam sources; and
> most of the recipients were business users.
> 
> As I recall, my sample size was something around 10 million E-mails
> from
> a single MX server in a load balanced cluster over a 24-hour period.
> There was a weak correlation between spam and a simple existence check
> for the PTR record. There was no correlation at all for a stronger
> check, e.g., A record matches; a message that failed the strong check
> was as likely to be judged ham as spam by the content filter.
> 
> <csg>

Hi Carl,

this is really interesting, because we got a totally different result when analyzed our data before we introduced the FCrDNS check. To verify our data, I took all rejected IP addresses which had a PTR record but did not have a matching A record from the log of the last 12 hours from one of our servers and run it against the Spamhaus ZEN DNSBL. The result is 

- 420 different IP addresses not on ZEN
- 61722 different IP addresses are on ZEN = 99,3 % of all rejected IP addresses

The distribution to the different parts of ZEN is:

     1 XBL-NJABL
     6 SBLCSS
    24 SBL
  4825 PBL-ISP
 25608 PBL-Spamhaus
 31258 XBL-CBL

I would say, that means there is a strong correlation between spam and at least the second part of the FCrDNS check.

Michael