= Rule Promotion and the sa-update Backend =

'sa-update' updates are created on the zone, nightly, by a cron job.
Here are some details as to how this works.

At 0830 UTC, the '''build/mkupdates/run_nightly''' script is run from svn
trunk.  This runs the script '''build/mkupdates/listpromotable''', which
queries the rule-QA web app for the previous day's rule accuracy figures, and
determines which rules from the rules source dirs (rulesrc/core and
rulesrc/sandbox/*) are to be ''promoted'' to the active ruleset, as described in RuleLifeCycle.

== Promotion Criteria ==

You can see the previous day's accuracy figures
[[http://ruleqa.spamassassin.org/?daterev=last-night|at this URL]].
Promotable rules show up in dark text on the RuleQaApp, non-promoted-by-default
rules show up in light grey text.

The promotion criteria are detailed on some other wiki page,
RulesProjPromotion perhaps? (TODO).

Anyway, using those figures, and some other criteria:

 * "tflags nopublish" rules are never published
 * rules with a {{{T_}}} prefix to their names are never published
 * rules without an explicit "tflags publish" line are never published
 * "tflags net", "userconf" or "learn" rules are always published
 * rules that fail lint -- if that can be attributed to the rule! -- are not published
 * rules that require a plugin that '''build/mkupdates/listpromotable''' thinks is not part of the default plugin set, are not published

A proposed new list of 'active' rules is created in the file 'rules/active.list'.

== The Active List ==

The active list, in the form of the file '''rules/active.list''' is checked in to SVN trunk, and is used by '''build/mkrules''' to determine which rules are active.

 * Rules marked active, and their dependencies (if they're meta rules), are written to '''rules/72_active.cf''';

 * Rules which are not in the active list, but which were loaded from a sandbox directory in ''rulesrc/sandbox/*'', are written to '''rules/70_sandbox.cf''', and renamed to always include a {{{T_}}} prefix;

 * Rules which are not in the active list, but which were loaded from a core-rules dir in ''rulesrc/core'', are written to '''rules/70_inactive.cf'''.

More info on these rule states can be found at RuleLifeCycle.

build/mkrules attempts to keep the contents of those files consistent. In
other words, meta subrules for a rule in the active list will always likewise
be copied to the active file -- even if they are sandbox T_ rules. It
will also ignore rules that are inside an {{{ifplugin}}} block for
a plugin that is not available.

'''rules/70_sandbox.cf''' and '''rules/70_inactive.cf''' are used for
development, and for mass-checks, but will not be installed during
{{{make install}}} and are not packaged in sa-update tarballs.
Rules in '''rules/70_sandbox.cf''' are the only ones used in the "bbmass" preflight mass-checks.

== The sa-update Tarball ==

'''build/mkrules''' is run, using the active list to create
'''rules/72_active.cf''', '''rules/70_sandbox.cf''' and
'''rules/70_inactive.cf'''. The latter two files are then
discarded, and {{{spamassassin --lint}}} run to ensure the
ruleset is still in a consistent state. If lint fails,
the update process is abandoned.

A {{{make install}}} is run, to a temporary directory, and the resulting
ruleset is packaged using tar, linted again from that dir, and then signed and
checksummed.

The DNS zone is updated using the SVN revision number, and the
tarball copied into place.

=== Administrivia ===

The cron jobs run as the user ''updatesd''. The crontab looks like:

{{{
30 8 * * * bash /home/updatesd/svn/spamassassin/build/mkupdates/run_nightly > /var/www/buildbot.spamassassin.org/updatesd/mkupdates.txt 2>&1
50 8 * * * bash /home/updatesd/svn/spamassassin/build/mkupdates/run_part2 > /var/www/buildbot.spamassassin.org/updatesd/mkupdatespt2.txt 2>&1
}}}

=== The Other End ===

See RuleUpdates for details on running "sa-update".