Manual Nightly MassChecks

The manual way to do NightlyMassChecks.

Here's more detail on that process, if you don't want to use the "corpus-nightly" script.

Get ahold of http://rsync.spamassassin.org/$VERS-versions.txt, where $VERS is either "nightly" or "weekly". "nightly" is updated a little before 0900 UTC Sunday through Friday. "weekly" is updated at the same time on Saturdays, and is meant to be a net-enabled run. ie: wait until at least 0900 UTC before trying to do a corpus run. The above files are also available via the standard rsync system.

Get a "nightly" rsync account (see 'How?' above).

The format of the above files is a file of "date <tab> revision <LF>", date in YYYY-MM-DD format, revision being the value that comes out of SVN. New lines are added to the bottom of the file.

So... Grab the file, find the right line (you can either grep for the date, or just take the last line of the file), and use the second column to update your corpora version. ie:

REV=`tail -1 nightly-versions.txt | awk '{print $2}'`
cd /path/to/spamassassin-checkout
svn update -r $REV

Alternatively, if you would prefer to pick it up via rsync:

rsync -vrz --delete \
     rsync://rsync.spamassassin.org/tagged_builds/nightly_mass_check .

(replace "nightly" with "weekly" for the weekly builds.)

Then use that build of SpamAssassin to perform a MassCheck , and when that completes, upload the results as per the instructions in http://spamassassin.org/dist/masses/CORPUS_SUBMIT_NIGHTLY .

Note: The result log-files must have an SVN revision line in the output, like so:

# mass-check results from jm@jalapeno, on Mon Nov 21 09:10:15 UTC 2005
# M:SA version 3.2.0-r322462
# SVN revision: 345462
# Perl version: 5.008003 on i386-linux-thread-multi
# Switches: '--progress --tail=20000 -j 4 -f /home/jm/cor/tgts'

If that line isn't present, the rule-QA reporting system cannot correlate the logs with the source revision, and instead ignores them.

If you do not use SVN to retrieve the SpamAssassin source tree, this may not be present, since "mass-check" cannot use "svn info" to get the current revision data. However, there's a workaround. Before running "mass-check", run "svn info" and redirect the output into a file called "svninfo.tmp" in the "masses" directory. Mass-check will read that and use its data for the "SVN revision:" line.

(The version of the tree available at rsync://rsync.spamassassin.org/tagged_builds/nightly_mass_check and .../weekly_mass_check already has this file included.)

ManualNightlyMassCheck (last edited 2011-05-02 18:34:20 by Darxus)