Running SpamD on Windows - Overview

This document describes how to get SpamD to run on Windows. ATTENTION: This is a extremely rough solution and not intended to be used in a production environment. It works around the in windows unimplemented fork issues by incorporating the children processes' tasks into the main process. Thus no spin-off processes are needed. Various other issues are not tackled but these result only in warning messages.

Kudos to Arik for providing this document. In NO WAY do I wish to minimize his achievement!

I must however caution folks that this is not really a huge improvement IMO from Klaus Mueller's previous attempt on SA 2.5x. In both cases you will get SpamD hobbling along, and you WILL see some speed improvement, particularly under modest load. But under heavier load, you will likely find any performance advantage disappearing and may experience timeouts. Also, this amount of patching is not fot the novice and will break all too easily with only minor code updates -- MichaelBell

Update for SpamAssassin 3.2.5

Brain2000 has updated this document to also work with SpamAssassin 3.2.5. However, it has only been tested by running the two sample-spam/nonspam files thousands of times simultaneously. After enough runs though, there is a memory leak in the check() routine that slowly builds up and eventually causes an error. The memory leak seems to be traced down to parsing the headers:

      SpamAssassin::parse()      [more time needed to pinpoint further]
      spamd::parse_body()        my $mail = $spamtest->parse(\@msglines, 0);
      spamd::check()             my ($mail, $actual_length) = parse_body($client, $expected_length, $compress_zlib);
      spamd::accept_a_con()      check($method, $version, $start, $remote_hostname, $remote_hostaddr);

Installation

The version used with this description is 3.2.5 and also 3.0.0-pre4. Each step applies to both versions unless a specific version is denoted:

1. Follow the procedures in InstallingOnWindows or on http://www.openhandhome.com/howtosa.html.

2. Copy spamd.raw from the SpamAssassin source directory to C:\Perl\bin.

3. From the already existing spamassassin.bat copy the first few lines that look like: {{{@rem = '--*-Perl-*-- @echo off SET RES_NAMESERVERS=192.168.2.1 SET LANG=en_US if "%OS%" == "Windows_NT" goto WinNT perl -x -S "%0" %1 %2 %3 %4 %5 %6 %7 %8 %9 goto endofperl :WinNT perl -x -S %0 %* if NOT "%COMSPEC%" == "%SystemRoot%\system32\cmd.exe" goto endofperl if %errorlevel% == 9009 echo You do not have Perl in your PATH. if errorlevel 1 goto script_failed_so_exit_with_non_zero_val 2>nul goto endofperl @rem '; #!C:\Perl\bin\perl.exe -T -w #line 15

# <@LICENSE> # Copyright 2004 Apache Software Foundation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # </@LICENSE>

my $PREFIX = 'C:\Perl\site'; # substituted at 'make' time my $DEF_RULES_DIR = 'C:\Perl\site/share/spamassassin'; # substituted at 'make' time my $LOCAL_RULES_DIR = 'C:\Perl\site/etc/mail/spamassassin'; # substituted at 'make' time

use lib 'C:\Perl\site\lib'; # substituted at 'make' time }}}

4. and replace following lines in spamd.raw with it: {{{#!/usr/bin/perl -w -T # <@LICENSE> # Copyright 2004 Apache Software Foundation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # </@LICENSE>

my $PREFIX = '@@PREFIX@@'; # substituted at 'make' time my $DEF_RULES_DIR = '@@DEF_RULES_DIR@@'; # substituted at 'make' time my $LOCAL_RULES_DIR = '@@LOCAL_RULES_DIR@@'; # substituted at 'make' time use lib '@@INSTALLSITELIB@@';}}}

5a. (SpamAssassin version 3.0.0 only) Further down, comment out the following line by adding a # in front of it: use Sys::Syslog qw(:DEFAULT setlogsock);

5b. (SpamAssassin version 3.2.5 only) Further down, change the following line from 'mail' to 'null': {{{# of a specific logfile my $log_facility = $opt{'syslog'} || 'mail';}}} Change to: {{{# of a specific logfile my $log_facility = $opt{'syslog'} || 'null';}}}

6. Even further down comment out the two occurrences of the line: spawn();

7a. (SpamAssassin version 3.0.0 only) Find the beginning of following block: {{{while (1) {

sleep; # wait for a signal (ie: child's death) if ( defined $got_sighup ) {
- if (defined($opt{'pidfile'})) {
  - unlink($opt{'pidfile'}) || warn "Can't unlink $opt{'pidfile'}: $!\n";
  } # leave Client fds active, and do not kill children; they can still # service clients until they exit. But restart the listener anyway. # And close the logfile, so the new instance can reopen it. close(STDLOG) if $log_facility eq 'file'; chdir($ORIG_CWD)
  - || die "spamd restart failed: chdir failed: ${ORIG_CWD}: $!\n";
  exec( $ORIG_ARG0, @ORIG_ARGV ); # should not get past that... die "spamd restart failed: exec failed: "
  - join ( ' ', $ORIG_ARG0, @ORIG_ARGV )
  - ": $!\n";
}
for ( my $i = keys %children ; $i < $childlimit ; $i++ ) {
- spawn();
}

} }}}

Add this code segment immediately in front of it: {{{while (1) {

# use a large eval scope to catch die()s and ensure they # don't kill the server. my $evalret = eval { accept_a_conn(); }; if (!defined ($evalret)) {
- logmsg("error: $@ $!, continuing");
  if ($client) { $client->close(); } # avoid fd leaks
} elsif ($evalret == -1) {
- # serious error; used for accept() failure die("fatal error; respawning server");
} # if we changed UID during processing, change back!
if ( $> != $< and $> != ( $< - 2**32 ) ) {
- $) = "$( $("; # change eGID
  $> = $<; # change eUID if ( $> != $< and $> != ( $< - 2**32 ) ) {
  - logmsg("fatal: return setuid failed"); die; # make it fatal to avoid security breaches
  }
} if ($copy_config_p) {
- while(my($k,$v) = each %msa_backup) {
  - $spamtest->{$k} = $v;
  } # if we changed user, we would have also loaded up new configs # (potentially), so let's restore back the saved version we # had before. #
  $spamtest->copy_config(\%conf_backup, undef) ||
  - die "error returned from copy_config, no Storable module?\n";
} undef $current_user;

} }}}

7b. (SpamAssassin version 3.2.5 only) Find the beginning of following block: {{{while (1) {

if (!$scaling) {
- # wait for a signal (ie: child's death) # bug 4190: use a time-limited sleep, and call child_handler() even # if haven't received a SIGCHLD, due to inherent race condition sleep 10; child_handler();
} else {
- $scaling->main_server_poll($opt{'server-scale-period'});
} ( defined $got_sighup ) and do_sighup_restart();
for (my $i = keys %children; $i < $childlimit; $i++) {
- spawn();
}

} }}}

Add this code segment immediately in front of it: {{{my $i = 0; $backchannel->setup_backchannel_parent_pre_fork(); $backchannel->setup_backchannel_parent_post_fork($$); $0 = 'spamd child'; $children{$$} = 1; if ($scaling) {

$scaling->add_child($$);

} $spamtest->call_plugins("spamd_child_init"); $backchannel->setup_backchannel_child_post_fork(); $backchannel->setup_backchannel_parent_pre_fork(); while (1) {

my $evalret = eval { accept_a_conn(); }; if (!defined ($evalret)) {
- warn("spamd: error: $@ $!, continuing");
  if ($client) { $client->close(); } # avoid fd leaks
} elsif ($evalret == -1) {
- # serious error; used for accept() failure die("spamd: respawning server");
}
$spamtest->call_plugins("spamd_child_post_connection_close"); if ($copy_config_p) {
- # use a timeout! There are bugs in Storable on certain platforms # that can cause spamd to hang -- see bug 3828 comment 154. # we don't use Storable any more, but leave this in -- just # in case. # bug 4699: this is the alarm that often ends up with an empty $@
  my $timer = Mail::SpamAssassin::Timeout->new({ secs => 20 }); my $err = $timer->run(sub {
  - while(my($k,$v) = each %msa_backup) {
    - $spamtest->{$k} = $v;
    } # if we changed user, we would have also loaded up new configs # (potentially), so let's restore back the saved version we # had before.
    $spamtest->copy_config(\%conf_backup, undef) ||
    - die "spamd: error returned from copy_config\n";
  });
  if ($timer->timed_out()) {
  - warn("spamd: copy_config timeout, respawning child process after ".($i+1)." messages"); exit; # so that the master spamd can respawn
  }
} undef $current_user; $i += 1;

} }}}

8. (SpamAssassin version 3.0.0 only) Below find the line: my ( $uid, $gid ) = ( getpwnam('nobody') )[ 2, 3 ]; and replace it with: my ( $uid, $gid ) = 'nobody';

9. (SpamAssassin version 3.0.0 only) Find the block: {{{ my ( $name, $pwd, $uid, $gid, $quota, $comment, $gcos, $dir, $etc ) =

getpwnam($userid);

}}}

Replace it with: {{{ my ( $name, $pwd, $uid, $gid, $quota, $comment, $gcos, $dir, $etc ) =

'nobody';

}}}

10. (SpamAssassin version 3.2.5 only) Find the routine handle_setuid_to_user() and comment out the code in it leaving an empty function: {{{sub handle_setuid_to_user { # if ($spamtest->{paranoid}) { # die("spamd: in paranoid mode, still running as root: closing connection"); # } # warn("spamd: still running as root: user not specified with -u, " # . "not found, or set to root, falling back to nobody\n"); # # my ( $name, $pwd, $uid, $gid, $quota, $comment, $gcos, $dir, $etc ) = # 'nobody'; # # $) = "$gid $gid"; # eGID # $> = $uid; # eUID # if (!defined($uid) || ($> != $uid and $> != ($uid - 2**32))) { # die("spamd: setuid to nobody failed"); # } # # $spamtest->signal_user_changed( # { # username => $name, # user_dir => $dir # } # ); } }}}

11. (SpamAssassin version 3.2.5 only) Find and comment out all the lines that quit if credentials are root: {{{# if ($> == 0) { die "spamd: still running as root! dying"; } }}}

12. (SpamAssassin version 3.2.5 only) Open c:\perl\site\lib\mail\spamassassin\util.pm and change the routine trap_sigalrm_fully() as follows: {{{sub trap_sigalrm_fully {

my ($handler) = @_;

# if ($] < 5.008) {

# signals are always unsafe, just use %SIG $SIG{ALRM} = $handler;

# } else { # # may be using "safe" signals with %SIG; use POSIX to avoid it # POSIX::sigaction POSIX::SIGALRM(), new POSIX::SigAction $handler; # } } }}}

13. Go all the way to the end of the document and append these lines:

__END__
:endofperl

14. Save the file and rename it to spamd.bat.

Running SpamD

Just open a command prompt and type: spamd

Sample output

Output after starting SpamD and checking one spam and one ham message.

{{{failed to setlogsock(unix): Undefined subroutine &main::setlogsock called at C:\ Perl\bin\spamd.bat line 378. reporting logs to stderr 2004-08-08 13:28:31 [6236] i: server started on port 783/tcp (running version 3. 0.0-pre4) 2004-08-08 13:28:53 [6236] i: connection from localhost [127.0.0.1] at port 1481

2004-08-08 13:28:53 [6236] i: handle_user: unable to find user 'Arik Funke'! 2004-08-08 13:28:53 [6236] i: Still running as root: user not specified with -u,

not found, or set to root. Fall back to nobody.

2004-08-08 13:28:53 [6236] i: checking message <20040808152848.000042d1@ariktab>

for Arik Funke:0.

2004-08-08 13:29:04 [6236] i: identified spam (11.3/5.0) for Arik Funke:0 in 10. 7 seconds, 1527 bytes. 2004-08-08 13:29:04 [6236] i: result: Y 11 - DRUGS_ANXIETY,DRUGS_ANXIETY_OBFU,DR UGS_DIET,DRUGS_DIET_OBFU,DRUGS_MUSCLE,RCVD_IN_NJABL_DUL,RCVD_IN_SORBS_DUL,URIBL_ OB_SURBL,URIBL_SBL,URIBL_WS_SURBL scantime=10.7,size=1527,mid=<20040808152848.00 0042d1@ariktab>,autolearn=no 2004-08-08 13:29:52 [6236] i: connection from localhost [127.0.0.1] at port 1510

2004-08-08 13:29:52 [6236] i: handle_user: unable to find user 'Arik Funke'! 2004-08-08 13:29:52 [6236] i: Still running as root: user not specified with -u,

not found, or set to root. Fall back to nobody.

2004-08-08 13:29:52 [6236] i: checking message <20040807050312.29938gmx1@mx029.g mx.net> aka <20040808152948.00006ac2@ariktab> for Arik Funke:0. 2004-08-08 13:29:56 [6236] i: clean message (2.9/5.0) for Arik Funke:0 in 4.5 se conds, 1929 bytes. 2004-08-08 13:29:56 [6236] i: result: . 2 - NO_REAL_NAME,RCVD_IN_NJABL_DUL,RCVD _IN_SORBS_DUL,SUBJ_HAS_UNIQ_ID scantime=4.5,size=1929,mid=<20040807050312.29938g mx1@mx029.gmx.net>,rmid=<20040808152948.00006ac2@ariktab>,autolearn=no}}}

by ArikFunke

CategoryInstall

SpamdOnWindows

Running SpamD on Windows - Overview

Update for SpamAssassin 3.2.5

Installation

Running SpamD

Sample output