# Exim SpamAssassin at SMTP time

### What's that?

    mail from: merlin@gandalf
    250 OK
    rcpt to: merlin@gandalf
    250 Accepted
    data
    354 Enter message, ending with "." on a line by itself
    From: merlin@gandalf
    To: merlin@gandalf
    Subject: $$$ Make Money Fast $$$ !!!

    viagra 100% GARANTEE AMAZING FULL REFUND
    This is not spam
    .
    550 Rejected

(logs would show something like this: `2004-03-10 08:27:18 1B16Y8-0001UP-4R SA: Action: permanently rejected message: hits=14.8 required=7.0 trigger=11.0 ( scanned in 2/2 secs | Message-Id: CCQPVENACPQBFLTRLICXWQVEK@gandalf). From <merlin@gandalf> (host=gandalf [127.0.0.1]) for merlin@gandalf`)

An example of teergrube would return this instead

    data
    354 Enter message, ending with "." on a line by itself
    (...)
    body     SEE_FOR_YOURSELF       /See (?:for|it) yourself\b/i
    describe SEE_FOR_YOURSELF       See for yourself

    body ORDER_NOW                  /\border (?:now|soon|fast|quickly|while)\b/i
    describe ORDER_NOW              Encourages you to waste no time in ordering

    .
    451- wait for more output
    451- wait for more output
    451- wait for more output
    (... one line every 10 secs, 15 minutes elapse ...)
    450 Please try again later

The idea here is to stall and waste the resources of the remote sender (BTW teergrube comes from german, and means tar-pitting, or stopping someone in his tracks)

### Why?

SpamAssassin can be run inside exim after the mail has been accepted, as shown here, but if you're not going to use my patch and you just want to run SA as an exim transport, this version is recommended

Now, while this will work, we can do better, hence the reason for my code (just to make things clear, you do not want to run both my code, and dman's transports. It'd work, but you'd be scanning the message twice)

The reason why I wanted SpamAssassin in local scan is that I don't want to accept the damn spam in the first place.

- While my code lets you do that, I don't like to send mails to the bit bucket, so you need to bounce them.
- Once you accept the spam, you can't bounce it half the time, or you bounce it to an innocent whose Email was forged as an envelope sender (some spam even forges the bounce address to you)
- If I refuse spam at SMTP time, it will remove the spam addresses from at least a few lists (they gotta clean their lists eventually otherwise they'd spend more time Emailing dead addresses than good ones)
- I have the option of toying with spammers and stall their connections and waste their resources (see the following page for details on teergrubing

Note that you can also use this code to simply run SA on all your mails (or portion thereof as configured with SAEximRunCond) without having to configure SA in your exim.conf. In other words, this code can be configured to not reject any mails.

### SpamAssassin? What's that?

Ah, you need to visit this page first then

### How does it work, what knobs are there?

You need to configure spamassassin to flags mails as spam after a certain threshold (7 for instance). After that, this code can be configured to

- Pretend to be processing the Email and send continuation lines to the remote server until it gives up (aka teergrubing)
- Accept but not deliver mail with a high threshold (i.e. devnull the mail)
- Reject mail with a lower threshold
- Temporarily reject mail with a still lower threshold (you can then inspect your logs to decide if you want to tweak SA so that next time the mail is sent, you can receive it)
- In all 5 cases, mail can be optionally saved to disk so that you can inspect all the mails you've rejected or /dev/nulled

You can also (and probably should :-)) use the new greylisting support for even better spam control

For more details, you should look at the self-documented config file and you can see some sample rejects and what you get in the logs

### Greylisting you say?

While when sa-exim first came out, its strongest point was being one of the first programs (if not the first) that let you reject Spam at SMTP time, its coolest feature now is adaptive greylisting support
In a nutshell, you get the advantages of greylisting without the disadvantages:

- mails with a low spam score are accepted without delay
- mails with an average spam score are greylisted, and only those are delayed
- mails with high spam scores are rejected regardless (no greylisting)

This method is the best combination I've seen out there so far, and while I've been talking about it for a while, I don't yet know of other programs that implement this method (if you do, please let me know so that I can acknowledge them)
For more details on how this works, check out the greylisting README

### Ok, where's the code? / Downloads

- The latest version is here (browsable tree or tar.gz). You can also get it from sf.net
- The CVS version is here (browsable tree) and you can also get the CVS tree from sf.net
- The latest config file with documentation is here
- Debian packages (source and binary) are here

As explained in the archive, you can either copy `sa-exim.c` over exim's `src/local_scan.c` You need to copy local_scan in src in the exim source tree and rebuild it, or you can build sa-exim as a loadable module (you need to patch exim to support loadable modules though)

You can also browse all my exim files here

### Mailing list

You should probably subscribe to this low traffic mailing list if you download the code to keep apprised of bug fixes and enhancements

### Integration with Exim 4

This code works without anything in the exim conf, but you probably want to use some knobs to disable scanning for some users (like setting `X-SA-Do-Not-Rej` or `X-SA-Do-Not-Run` in the rcpt ACL and removing those headers in the right places).
See my exim4 conf tree and more specifically the exim4.conf file

You can look at the README for more integration details.

### Changelog/Download

- 2006/01/09 - v4.2.1 (sa-exim.tar.gz or local_scan only)
  Security update (reported by Chris Morris)
  - Modified Greylisting.pm not to generate tuplets with spaces, although the cleaning cron job is now safe with regard to whitespace
  - Included Mark Lawrence's perl script to better clean old tuplets
  - The default config now ships with the greylistclean cronjob running as nobody (which spamd should be running as, too)
- 2005/01/17 - v4.2 (sa-exim.tar.gz or local_scan only)
  Do not use, greylistclean is insecure, use 4.2.1 instead
  - Fixed code so that it compiles inside the exim tree too (thanks Jason)
  - Support SA 3.0 new score reporting format (score= instead of hits=)
  - Proper SA 3.0 plugin support
  - Documentation updates
  - greylisting files now also save the last SA score, just in case
  - Saved files are now group read/write too, if you need to shut off group access, do it at the directory level
  - Only rewrite Content-Type/Content-Transfer-Encoding if SARewriteBody is on now (this used to be useful with old SA versions and defang_mime but those options are gone). Suggested by Adam Tilghman.
  - Finally made all score threshold variables exim conditions that are evaluated at runtime
  - SAteergrubecond is deprecated as a result
- 2004/08/16 - v4.1 (sa-exim.tar.gz or local_scan only) Please see the mailing list, or use the CVS version if you are compiling sa-exim inside your exim tree (there is a small mistake in the source which will prevent proper building) / You also need CVS if you are using SA 3.0
  - Deal with being called with the number of recipients is 0 (reported by John Horne)
  - Made hopefully fully 64 bit clean (well, it was written that way, except for that pesky printf in C, which got in the way). Kudos go to Adam D. Barratt for finding and explaining the bug
  - Switched to using strchr instead of index to remove compilation warnings on some systems (reported by John Horne/Jeff Carter)
  - Added SAspamcHost/SAspamcPort at the request of several people
  - Fixed mbox From file to be standards complaint (reported by Cliff Hones)
  - Added SAFEMESGIDCHARS option for safer message-id based filenames
  - New version of the SA patch, with safe file creation (clean env from and rcpt to before using as directory names)
  - IMPORTANT: if you are using greylisting, and applied the previous greylisting patch, it didn't create shell safe filenames, it is therefore highly recommended that you apply the new version
  - Added a quick patch to sa-exim to deal with headers that are longer than 8K (and drop the remaining lines)
- 2004/03/16 - v4.0 (sa-exim.tar.gz or local_scan only)
  - Added SAspamcpath (from Richard Lithvall)
  - Makefile fixes for version.h and dependencies
  - Added exim acl_m trick from Chirik in README
  - New code to read and pass on any X-Spam- header from SpamAssassin
  - Message-Id is now logged in SA log entries to allow you track down a message in other logfiles, like SA's log
  - Add X-SA-Exim-Connect-IP header for greylisting
  - Allow for tempreject on messages not flagged as spam by SA (for greylisting)
  - Added greylisting support (see README.greylisting)
  - Added the following options to sa-exim.conf:
    - SAspamcSockPath
    - SAgreylistiswhitestr
    - SAgreylistraisetempreject
  - Lots of documentation fixes and updates
  - Basic debian tree shipped by default (but real deb package from Sander Smeenk)
- 2003/08/18 - v3.1 (sa-exim tar.gz or local_scan only)
  - Fixed SA: Action: teergrubed sender until full configured duration output
  - spamassassin.conf was renamed sa-exim.conf (bad initial name choice)
  - Changed teergrubing so that it works with exim 4.20 and above
  - Added the much requested SARewriteBody option for SA 2.50's report_safe (courtesy of Richard Lithvall)
  - Added SAaddSAEheaderBeforeSA to deal with SARewriteBody and privacy
  - Added SAPrependArchiveWithFrom option (Richard Lithvall)
  - As announced in the previous version, SAStallSender was removed. Use Teergrubing instead
  - Message-Id is read directly from Exim now that the API allows for that (we used to parse the Message-Id header ourselves) -> except that we don't use this for logging or saving messages: as soon as we can read the real Message-Id from the headers, we use that Message-Id for logging (this is to facilitate tracking of messages from the logs and correlating with something like SA timelog files)
  - Logging improved some more: Message-Id is added to log entries, as well as the full mailinfo whether the message is spam or not
- 2003/04/30 - v3.0 (sa-exim tar.gz or local_scan only)
  - Makefile can generate a short sa-exim.conf (for Brian Kendig)
  - Added a contrib directory with 3rd party scripts
  - Since we already had X-SA-Exim-Rcpt-To: to show the envelope to (disabled by default), I added X-SA-Exim-Mail-From: to show the envelope from (always enabled, but you can delete it in system_filter if you wish). If you can't easily see or parse the envelope sender in your mails, this should definitely help
  - Much improved directory creation and error handling for the reject save directories
  - Added SIGCHLD patch from David Woodhouse
  - Added version header that looks like this: X-SA-Exim-Version: 2.2.x-cvs (built Tue Apr 22 10:28:25 PDT 2003)
  - Rewrote pretty much every log message to be more consistent and grepable (the previous ones were quite bad). Suggested by multiple people including Brian Kendig
  - Made the SMTP error messages finally configurable (suggested by several people). This is now possible since exim lets me output a different message in the log than in the SMTP session. Consequently, new default messages do not show the spam score, this only goes to the exim now.
  - Made SAmaxrcptlistlength and option to control how long of a X-SA-Exim-Rcpt-To header you can output if you want to output it at all
  - Do not output "savemail condition expanded to false" if not in debug mode (as reported by Brian Kendig)
  - Now ships with a sample local_scan.h if we can't find the exim source
  - Change of logic to delete SA headers that were in the original mail but weren't outputted by the SA run (like X-Spam-Flag, as reported by Chad Leigh)
  - Modified local_scan dlopen patch to deal with updates to the exim local_scan API
  - The default location for spamasassin.conf was changed to /etc/exim4 (debian default)
  - Updated README and INSTALL
  - Updated localscan_dlopen.patch to deal with minor/major version numbers
  - Added X-SA-Exim-Version
- 2002/10/28 - v2.2 (sa-exim tar.gz or local_scan only)
  - Fixed a bug that affected all mails bigger than what spamc would accept. Doh! (it not clear why, but spamc would hang and stop reading after it had been fed more than it was willing to accept)
  - Added more debugging code to help track the above problem
  - Depending on SATruncBodyCond will now either not pass a message that's too big to spamc, or will optionally truncate it first
  - Now strips any X-SA-Exim-* headers already present in the message before scanning it
  - Added new X-SA-Exim-Rcpt-To: header (see privacy section in README) after a suggestion from Brian Kendig
  - Teergrube is now spelled correctly (note that the option names in sa-exim.conf changed as a result)
  - Teergrubing has been re-implemented to detect that the other side went away so that exim doesn't stay around for nothing.
  - Added SAteergrubecond so that you don't teergrube your neighbours
  - Small cleanups
- 2002/10/13 - v2.1 (buggy) (sa-exim tar.gz or local_scan only)
  - Default mail config is now in /etc/exim instead of /etc/mail
  - Fixed SAstallsender so that exim doesn't leave spool files behind
  - Fixed comparison for spam values as suggested by Patrice Fournier
  - Returned a header saying whether SA-Exim was able to scan the mail or not (based on a suggestion from Patrice Fournier)
  - Added support for teergrubing as suggested by Robert Strickler (see http://www.iks-jena.de/mitarb/lutz/usenet/teergrube.en.html)
  - We now make sure that we feed spamc a little more data than it's willing to accept. That way, we don't feed a truncated message
  - sa-exim.conf now appends /new on save directories so that they are valid maildir spool and you can run mutt -f dirname to read
  - Small cleanups
- 2002/07/07 - v2.0.1 (sa-exim tar.gz or local_scan only)
  - Removed unneeded ## concatenation in macro
  - Fixed lseek call, as reported by Peter N Lewis
  - Improved Makefile for documentation
  - Suggestion from dman to allow for config options overrides in Makefile
  - Fixed all the compiler warnings that were fixable
  - Handle more kinds of Message-Id as suggested by Patrice Fournier
- 2002/06/14 - v2.0 (sa-exim tar.gz or local_scan only)
  - Allow for stalling SMTP sessions from spammers (caution)
  - Support X-Spam-Status from SA pre 2.30
  - Build patches from Norm <norm@sandbox.org.uk>
  - Documented that spamd shouldn't be run with -c
  - dman's patch to allow for this to be a shared library
  - dman's indirect request for saved files to be 660
- 2002/06/01 - v2.0b1
  This is how 1.0 would have been if I had done it right :-)
  - We now attempt to create directories we write to (suggestion from Patrice Fournier)
  - Log envelope sender, recipients and connecting host on rejected messages (suggestion from Patrice Fournier)
  - We can now save non rejected messages whether they were flagged as spam or not (based on request from Patrice)
  - Each save rule now has its own condition that lets you decide which messages you want to save or not
  - Fixed parsing of Message-Id. May not parse all of them but won't do anything stupid with bad input
  - Fixed a buglet where we removed old headers before knowing whether they were going to be replaced.
  - Added much needed internal alarm so that we don't run too long
  - I added lots of options, but weren't able to test all of them and their combinations, please reports problems you may notice and documentation issues (in config file)
  - Thanks also go to dman for his plentiful feedback
- 2002/05/21 - v1.3
  - We now close the files we create on disk. Duh!
- 2002/05/17 - v1.2.2
  - SA options are now kept in memory. Small cleanups
- 2002/05/13 - v1.2.1
  - v1.2 (unreleased) didn't work right: it tagged messages properly, but failed to see what was marked as spam and couldn't reject messages. Fixed.
  - Stripped newlines in header lines (better for logging)
  - fixed header_add bug if headers contained '%'
- 2002/05/12 - v1.2 (unreleased)
  - According to Craig R Hughes, any X-Spam header can be multiline. Let's parse them accordingly
- 2002/05/08 - v1.1.1
  - Added fake envelope from to mails that we save on disk so that they can be opened with MUAs (idea from dman)
- 2002/05/07 - v1.1
  - Cleaned up and fixed multiline header parsing
  - Added parsing of Subject back from Spamc (suggestion from dman)
  - We now log how long spamc took, and all messages processed by spamc get logged in mainlog so that we know the outcome
  - Added custom header compare function to handle mixed case headers
- 2002/05/06 - v1.0.1
  - Fixed problems with reading Content-Type back from spamc (we used to read whole body too, ooops...)
  - Handle multi-line Content-Type headers.
  - All log entries now start with "SA: "
  - If SA gets run twice, we drop the previous X-Spam headers
  - Support for X-Spam-Prev-Content-Transfer-Encoding
- 2002/05/05 - v1.0
  - *Many* changes. This is now a real program with hopefully most of the configuration options people could need
  - Fully configurable through external config file
  - Condition to run or not run SA against a message
  - 3 Levels of spam handling
  - Option to save messages that are rejected or cause errors
  - Very complete error checking
- 2002/04/17 - v0.9.1
  - Added a few checks for possible failures
- 2002/04/16 - v0.9
  - Pre-release

  More generally, all the files can also be found here

  Feedback is appreciated (but please prefer the use of the sa-exim list)

  ### Acknowledgements

  While I wrote SA-Exim after realizing that I didn't want to accept Spam in the first place, this package would not have been put together without the help and contributions of the following people:
  - Philip Hazel
    We wouldn't have exim without him :-)
  - Justin Mason, Craig R Hughes, Dan Quinlan, and the rest of the SpamAssassin crew
  - Derrick 'dman' Hudson
    Early adopter, feedback, ideas, first integration with localscan_dlopen
  - David Woodhouse
    Excellent original localscan_dlopen patch for exim 4 SIGCHLD patch (was set to ignore by exim)
  - Norm
    Build patches and trailing slash problem for mkdir on netbsd
  - Patrice Fournier
    Several suggestions, including adding the X-SA-Exim-Scanned header
  - Peter N Lewis
    Spotted a buglet in lseek call
  - Robert Strickler
    Suggested that I add teergrube support
  - Tim Jackson
    Narrowed the problem with SA Exim 2.1 to a hang when we feed a message that's too big to spamc and did a lot of research on the bug
  - Brian Kendig
    Suggested the functionality behind X-SA-Exim-Rcpt-To: Noted and requested logging changes
  - Paul Matthews
    Reported string length problem with X-SA-Exim-Rcpt-To:
  - Chad Leigh
    Reported that SA-Exim doesn't delete a X-Spam-Flag header in the original mail if the local config decides that the mail isn't spam
  - Martin Balvers
    Reported that SA: Action: teergrubed sender until full configured duration logging wasn't consistent. Author of http://nossie.addicts.nl/projects/sa-exim-stats/
  - Andreas Metzler
    Has provided invaluable help by being the main maintainer of the exim4 debian package and doing most of the work for the sa-exim package, while patiently helping me with my debian build questions
  - Richard Lithvall
    Added the much requested SARewriteBody option for SA 2.50's report_safe Also added SAPrependArchiveWithFrom option
    Also added SAspamcSockPath for SA 2.60 and better
  - Justin F. Knotzke
    Reported that Dec 31st 1969 isn't a good idea and upsets some MUAs like mutt
  - Norihisa Washitake
    Pointed out that I should be using date -R to override foreign locales
  - Chirik
    Made suggestions on increasing logging consistency Gave example on how you can trigger mail scans without using headers (using internal exim acl variables)
  - Ross Boylan
    Suggestions for documentation tweaks
  - Sander Smeenk
    Made sa-exim show up in Debian. Yeah! :) Major doc diffs
  - Stéphane MANHES
    Pointed out that greylisting requires X-SA-Exim-Rcpt-To:
  - John Horne
    Reported crash on discard, due to the number of recipients being 0
  - Adam D. Barratt
    Found the problem with sa-exim's indirect use of printf and causing a crash in exim on 64 bit architectures, and gave a detailed explanation on what happened and how to fix it
  - Jeffrey D. Carter
    Suggested strchr instead of index for easier compilation.
  - Danilo Lotina F. & Others
    Suggested adding a destination host for spamc (I added a port too)
  - Cliff Hones
    Reported that From mbox line was in an incorrect format
  - Jason John Schwarz
    Reported compile problem inside the exim tree (i.e. without local_scan)
  - Kristopher Austin
    Initial port of the SA patch to an SA 3.0 plugin
  - Adam Tilghman
    Pointed out that I shouldn't be rewriting Content-Type headers if we don't rewrite the body (and defang_mime is dead now)
  - Mark Lawrence
    He is the author of the greylistclean cron job
  - Chris Morris
    Reported the ealier tuplet deletion cronjob in shell as insecure
  - Your name here
    If you contribute or if you did and I forgot your name (sorry, let me know)