assp is good, but ...

I use the Anti Spam SMTP Proxy: http://assp.sourceforge.net/
and I was encountering spam coming in that mailed to postmaster and whomever else was also in "To:".

It turned out that in "Recipients", I had Skip Spam Checks for Postmaster Catchall (sendAllPostmasterNP) -- and abuse checked, which basically precluded the check for "but not if other people are also in 'To:'".

After unchecking this, (and sendAllAbuseNP), the spam dropped.

ASSP with MailArchiva

Oh, this is so cool and fast:

ASSP gets a facelift when coupled with MailArchiva. Set up MailArchiva (separately is ok, and perhaps even the Open Source version) and make certain that the Listen for Exchange/SMTP requests is turned on. Then in ASSP configuration, sendAllSpam to USERNAME@DOMAIN and sendAllDestination to mailarchivamachinename:8091 (or whatever port you're listening to in MailArchiva).

Apply changes, and now ASSP's spam goes to a different location, doesn't pollute your main mail archival, and yet users should be able to self-retrieve "missing" emails. Even still, I'd likely recommend upgrading MailArchiva to Enterprise Edition if only for retention purposes.

ASSP Tweak: Filename in maillog.txt

This is a simple change to ASSP (assp.pl) that places the filename of the email in the maillog.txt
First, find and change the following -- it's around line 2578 in 1.1.0 or 8046 in the latest release, inside sub Maillog. The bold lines were added. You'll need to restart ASSP to see this take effect.

Now you know exactly which file pertains to your email and you don't have to grep for it in /spam or /notspam

$Con{$fh}->{maillogfh}=$FH;
$Con{$fh}->{mailloglength}=0;
binmode $FH;
# logging filenames
mlog($fh, "'$fn'");


Colorize your log files!

I posted my awk colorizer for tail before and the general consensus seemed to be "meh".

If you are greping through monochrome logs, you're missing out on color.

Here's my latest:

grep -l search criteria * | xargs head | awk -f colorit.awk | more

great for spam logs.

awk colorizer for tail.

One of the problems I have with tail -f maillog.txt | grep "search" is that I really did want to watch the noise and not just the signal. Except that I'd like to notice the signal. Here's my printcolor.awk Now I can use tail -f maillog.txt | awk -f printcolor.awk and see the whole tail, with keywords highlighted in *different* colors.

NOTE: ^[ is supposed to be Ctrl-v, Escape; NOT caret, left bracket.


function colorize(word, color)
{
c["red"] = "^[[1;31;40m"
c["green"] = "^[[1;32;40m"
c["yellow"] = "^[[1;33;40m"
c["blue"] = "^[[1;34;40m"
c["magenta"] = "^[[1;35;40m"
if (line ~ word)
{ split (line, a, word)
line=a[1] c[color] (word) "^[[0;37;40m" a[2]
}
}
{line = $0
colorize("whitelist","green")
colorize("Bayesian Spam","red")
print line
}



ETA: If you'd like to be annoyed/beeped at for something that you're looking for, you can add ctrl-v, ctrl-g in the right hand side, eg: "^G^[[1;32;40m" (not caret G)

ASSP - forwarding spam adds to whitelist

I have a specific issue where CompanyA.com split off with SubCompanyA.com and in the process, SubCompanyA.com's users still wanted to forever be reached by their old @CompanyA.com email address. Inside Exchange, I added quite a few Contact efwd emails which worked quite nicely. I have another post ... somewhere about using the exchange-to-csv export and import to help automate this process.

In any case, I noticed a LOT of spam that was resulting from whitelisted email addresses from spammers. It appears that my spam filter, ASSP, figured that all email coming from my Exchange Server is whitelisted mail, no matter how I tried to redlist the domains or users or anything. Why did my Exchange Server whitelist spam?

It turns out that the Exchange Server Contacts forwarded out through my spam filter to SubCompanyA.com. The question comes: how to avoid using my spam filter for SubCompanyA.com bounces? (of course, the other question is why the spam filter isn't kicking these out in the first place, but that's likely because of the whitelist=valid email issue. OK, it's circular reasoning, but let's stop the whitelisting)

This site Configuring and Using an SMTP Connector shows how to add an SMTP connector for problem domains. Ah! a solution! I set one up for the new SMTP server (actually the new SMTP server's spam front end) and now @CompanyA.com's forwardings to SubCompanyA.com's email addresses never touch (read: don't add to the whitelist of) my spam filter. This may cost my Bayesian filter to stop understanding valid emails between CompanyA.com and SubCompanyA.com, but then again, whitelisting between the two companies should already have occurred and the further CompanyA.com/SubCompanyA.com whitelisting is trivial.

Edited to add: Well, maybe not trivial, per se. After realizing what will happen (whitelist expiration of emails@SubCompanyA.com after 90 days), SubCompanyA.com's inbound emails will possibly need to be vetted each inbound time. Well... at least I can noprocess SubCompanyA.com inbound from the spam filter, so that seems to fix that.

ASSP and grep Part 2

I just used this monstrosity because I needed to get the email addresses of things that I knew whitelisted but I wanted to remove and populate the redlist.


grep "whitelist addition" maillog.txt | grep "my expression" | cut -d \> -f 1 | cut 33- > rlist


What does it do?

1. It finds all the "whitelist additions" in maillog.txt
   
2. For all of those, it searches for "my expression" to further limit what I'm searching
   
3. It then cuts the resulting line off at the > delimiter.
   
4. Then, it grabs the email address after position 33
   
5. and dumps it into rlist
   

I probably *should* have used awk to do this, but I didn't have it, and this is the ugly way to get the email list that I wanted. Arguably, I could have used cut -d \< -f 2 instead of the cut 33- but there you have it.

Edited to add: actually, the latter option is better/more flexible. I'd suggest that in the future.

Further edited to add:
grep "whitelist addition" maillog.txt | grep -v "localdomain.com"
This shows me all whitelists that my server made that were based upon bounces (contact-forwards, usually), not from true outbound from local users.

ASSP Spam in Commission. A Search Open.

Some queries that I'm using to parse ASSP spam directory, just in case I need to look it up again:

grep "email@address" * | cut -d: -f1 | xargs grep "content"
Result: in all the spam emails that contain the email address, get the filename and in those files, look for "content"

ls -l | grep "Oct 10" | cut -c57-80 | xargs grep "email@address"
Result: for a specific day (or range of days/month), get the appropriate file names and in those files, search for email (or other criteria)