Log in

No account? Create an account


Bloody edge

« previous entry | next entry »
7th Dec 2004 | 16:20

More on the same topic as http://www.livejournal.com/users/fanf/29644.html

There are a few problems with the configuration snippet I posted there.

The expansion of the dnsdb lookups happens before the outer list of blacklist domains is separated into items, which means you end up with a list looking like sbl.spamhaus.org/ This means look up in the sbl.spamhaus.org blacklist, then look up $sender_host_address in the blacklist which is nonsense. A corrected ACL clause is:
    message        = The name servers for the domain ${sender_address_domain} \
                     are listed at ${dnslist_domain} ($dnslist_value); \
                     See ${dnslist_text}
    dnslists       = sbl.spamhaus.org/<|${lookup dnsdb {>|a=<|\
                                        ${lookup dnsdb {>|zns=<|\
                                        $sender_address_domain} }} }

In this version, as in other Exim lists, < defines the separator to use when parsing a list. > is a special dnsdb feature to define the separator to use when producing a list. This is a bit cryptic but it has a nice Unixish mnemonic.

So after expansion we get sbl.spamhaus.org/<||
which is the correct result: a list of lookups separated by | to perform against sbl.spamhaus.org. The nested ordering of the dnsdb lookup
expansions means that the inner | lists don't get confused.

A more fundamental problem is that it has too much collateral damage - the SBL is tuned for spammers' outgoing email and if you try to use it for other purposes it tends to assign too much guilt by association. For example, http://www.spamhaus.org/sbl/sbl.lasso?query=SBL16513 is a Spamhaus record for Evesham Technology, who are customers of Pipex/UUnet/WorldCom/MCI and who continue to spam despite MCI's anti-spam AUP. As a result of this various Pipex name servers are in the SBL which could cause problems for a lot of more respectable UK organizations.

It also requires a certain amount of competent DNS administration on the part of the sender, and might be too eager to find a problem and return a 450 "temporary" error. This is much more likely to occur if you apply the zns check to $sender_helo_name. For example, Ebay's outgoing email servers use a HELO name in a zone that has nameservers in I have a patch (not yet integrated into the official source) which allows you to work around this problem by adding a defer behaviour control to dnsdb.

Maybe you can work around the problems using a configuration like the below, but maintaining the exemption list would be too much work for us. It's better to implement the checks in SpamAssassin, which is much less sensitive to false positives in individual checks.
    set acl_c3     = ${lookup dnsdb {>|defer_never,zns=<|$sender_address_domain} }
    set acl_c4     = ${lookup dnsdb {>|defer_never,zns=<|$sender_helo_name} }
    set acl_c5     = ${lookup dnsdb {>|defer_never,zns=<|$sender_host_name} }
    condition      = ${if match{$acl_c3|$acl_c4|$acl_c5}{SKIPDNS_REGEX} }

    message        = The name servers for the domain ${sender_address_domain} \
                     are listed at ${dnslist_domain}; See ${dnslist_text}
    dnslists       = sbl.spamhaus.org/<|${lookup dnsdb {>|defer_never,a=<|$acl_c3} }

    message        = The name servers for the host name ${sender_helo_name} \
                     are listed at ${dnslist_domain}; See ${dnslist_text}
    dnslists       = sbl.spamhaus.org/<|${lookup dnsdb {>|defer_never,a=<|$acl_c4} }

    message        = The name servers for the host name ${sender_host_name} \
                     are listed at ${dnslist_domain}; See ${dnslist_text}
    dnslists       = sbl.spamhaus.org/<|${lookup dnsdb {>|defer_never,a=<|$acl_c5} }

pir asked about other anti-spam checks.

The following is extremely effective and doesn't unduly punish idiots. It's fairly precisely targeted at the signature protocol abuses of known malware.
    message        = Please use your name when saying HELO (not $sender_helo_name)
  ! verify         = helo
    condition      = ${if or{{ eq{$acl_c1}{bad} } \
                             { isip{$sender_helo_name} } \
                             { eq{$sender_helo_name}{$local_part} } \
                             { match{$sender_helo_name}{\N[.][.]|.{55}\N} } \
                             { match_domain{$sender_helo_name}{+our_domains} }} }
    set acl_c1     = bad

Callback verification is also very good, though somewhat more troublesome. Most of our ACLs have clauses along the following lines, which first verifies the sender's email domain then calls an auxiliary ACL to do more thorough checks.
    verify         = sender
    acl            = aux_verify_sender

The more thorough checks are as follows. We have a locally-maintained callout exemption list of domains and email addresses, and we also use the RFC-Ignorant DSN list to avoid attempting callouts to incompetently run domains. We also give the sender the benefit of the doubt if their server isn't reachable; this is mainly to let email from web servers through. It's a sad fact that web developers are the most clueless senders of legitimate email. What is worse is that their cluelessness leads to spammers exploiting their moronic form-mail scripts. Bah! Out, out, demons of stupidity!
    condition      = \
      ${lookup {${lc:$sender_address_domain}} partial-cdb {DB/nocallout.cdb} \
               {yes} {${lookup {${lc:$sender_address}} cdb {DB/nocallout.cdb} \
                               {yes} {no} }} }
    dnslists       = dsn.rfc-ignorant.org/$sender_address_domain
    verify         = sender/callout=4m,maxwait=4m,connect=30s,defer_ok

Another good thing to do is to delay a little at strategic points in order to give Exim's pump-and-dump detection a better chance of working. If the sending host is in any of a number of blacklists, or if it uses an invalid HELO name, I delay for 5 seconds before the initial greeting and before the response to HELO. I could probably add some more criteria to this check (e.g. incorrect recipient address) and it might also be worth delaying at the one remaining synchronization point (in acl_smtp_predata), but I haven't got round to trying it yet (and it would add significant complexity).

The above three checks each deal with about 10% of junk email.

I'm currently testing out a new HELO heuristic, to see if it can plausibly be used in anger. This is to deal with spammers who use HELO names of the form ajfgyfsgjh.com, and with the random-concatenation HELO names that are shorter than 55 characters.
  # A more complicated HELO check. If the top level domain has name servers,
  # but the second and third (if present) level domains do not, reject. We
  # allow missing top level servers because of private addresses.
    log_message    = F=<$sender_address> RCPT=<$local_part@$domain> \
                     HELO DNS check failed for $sender_helo_name
    condition      = \
      ${if match{$sender_helo_name}{([^.]+[.])?([^.]+[.])([^.]+)\$} \
           {${lookup dnsdb {defer_never,ns=$3} \
                     {${lookup dnsdb {defer_never,ns=$2$3} \
                               {no} {${lookup dnsdb {defer_never,ns=$1$2$3} \
                                              {no} {yes} }} }} }} }

| Leave a comment | Share

Comments {3}

from: pir
date: 7th Dec 2004 09:54 (UTC)

THanks for more details...

Looking up the nameservers in dnsbl might be more useful for me with a delay than as an explicit block (again for the pump-and-dump detection)... of course I could just delay all banners by a short while.

Callouts... I was never too fond of, in part because I don't really want to deal with all the exceptions.

The last HELO check is particularly interesting, currently I'm doing;

  deny   message = Your server announcement ($sender_helo_name) \
                     is a single word rather than a FQDN. This is \
                     in breach of RFC2821
       condition = ${if match {$sender_helo_name}{[.]}{no}{yes}}

  deny   message = Your server announces itself \
                     ($sender_helo_name) with my \
                     IP address which is in breach of RFC2821.
       condition = ${if eq {$sender_helo_name}{$interface_address}\

  warn condition = ${if isip {$sender_helo_name}{yes}{no}}
     log_message = Plain IP in HELO - delaying banner message
           delay = 30s

  deny   message = Your server announces itself \
                     ($sender_helo_name) with my name \
                     which is in breach of RFC2821.
       condition = ${if match_domain {$sender_helo_name}\

which does many of the same things as your checks. I was blocking completely on any plain (non []) IP addresses in HELO and then found that a girlfriend's phone company did that so I never got her text pages :/

When are you doing that chunk of verification, though ? At DATA (since you don't have $local_part at HELO) ?

Reply | Thread

Tony Finch

from: fanf
date: 7th Dec 2004 10:04 (UTC)

At RCPT time (you don't have $local_part at DATA time either because there might be more than one recipient). We don't reject before then in order to allow people to get in touch with postmaster.

Rejecting single component HELO names would have too many false positives for us - it's a common misconfiguration and not necessarily the result of malice.

Reply | Parent | Thread

from: pir
date: 7th Dec 2004 10:13 (UTC)

I get too much spam to postmaster so I block that too (but then I'm doing this for a rather different type of site ;)

Single componant HELO a common misconfiguration ? One I've never seen from a legitimate source, then...

Reply | Parent | Thread