Rules Project: Secrecy
(part of RulesProjectPlan)
LorenWilton: 'There is a second thing here that gives me even greater concern. We have discovered that rules can be discussed openly on the users or dev list just fine, even going into some detail on what they do and how they work, and it will not have a noticible effect on how well a rule catches spam.
We have also found that the instant an actual rule is posted on the user's list, it will lose about 80% of its effectiveness, usually within about 16 hours. Within a week it will be virtually useless. Sometimes the rule will regain some effectiveness a few months later, and in rare cases posting a rule will not affect the hit rate. But in general, public posting in a readable forum of a rule body will negate the usefulness of the rule almost instantly.
One can speculate on why this happens, since the rules are there to read on any SA system, and can be trivially downloaded from SA and SARE for casual examination. Evidence shows though that this doesn't have an effect on the effectiveness of the rules. But posting the body of the rule on a mailing list does. Moderately strange, but of moderate concern, also.
I have some concern that a rules project *might* open up new rules to ineffectiveness, similar to posting them in a forum. However, the difficulties (for the average spam tool writer, at least) in using svn may prevent this from being a real problem. But it is worth devoting a few moments thought to the possibility.'
JustinMason: it *is* a problem, but in my opinion there's really nothing that can be done about this – we're an open source project, and the code is visible. while there's downsides, it also brings big benefits as well (as I said, the alternative is working for Brightmail 
. Open development is a requirement of being an ASF project, iirc.
The key factor to fix this problem, we think, is to have fast, fast turnaround on rule publishing – that way when the spammer mutates, if they do, we can keep up. we know we need to get things turning around faster – Theo's "sa-update" script (SaUpdatePlan) is the key to this.
There are other techniques, also, but let's not talk about them here...