The TxRep - Reputation Plugin
Introduction
TxRep was designed as an enhanced replacement of the
AutoWhiteList plugin
TxRep, just like AWL, tracks scores of messages previously received, and adjusts the current message score, either by boosting messages from senders who send ham or penalizing senders who have sent spam previously. This not only treats some senders as if they were whitelisted but also treats spammers as if they were blacklisted. Each message from a particular sender adjusts the historical total score which can change them from a spammer if they send non-spam messages. Senders who are considered non-spammers can become treated as spammers if they send messages which appear to be spam. Simpler told TxRep is a score averaging system. It keeps track of the historical average of a sender, and pushes any subsequent mail towards that average.
The most important difference of TxRep in comparison to AWL is its ability to learn. It can be trained by sa-learn, it has also the auto-learn ability, and old messages can be re-learned anytime to adjust historical records after a revision of rules. There are more differences, though. Below, there is a brief list of features introduced by TxRep to work around some shortcomings of AWL:
- Improved scoring algorithm 2. Learning 3. Auto-Learning 4. Re-learning 5. Record Aging 6. Blacklisting and Whitelisting 7. Complex Sender Identification 8. Message Tracking 9. Dual User and Global Storages 10. Outbound Whitelisting
More details are available on the TxRep POD page.
How Does It Work?
The algorithm works using a local database of entries. Each entry has a key formed by the identificator, and optionally the IP address it originated at, and the DKIM signature. It contains a TOTAL score of messages and a COUNT of messages. The MEAN score is TOTAL/COUNT. Each sender is identified by several IDs: the From email address in combination with the originating IP block, the standalone From email address (without any IP), the domain name of the From address, the full IP address, and the HELO name. Each of these ID types has a configurable weight factor when calculating the overall sender's reputation. The overall txrep_factor can be adjusted in the configuration to adjust the impact of the reputation, which may be useful when starting off.
How do I train spam/ham?
In exactly the same way (and in the same time) as you train spam and ham to the Bayesian SA system:
sa-learn --spam filesa-learn --ham file
It means that if your server is set up to use IMAP folders, webmail, or other tools for training SpamAssassin, the TxRep reputation will be adjusted at senders of all the trained messages anytime you use it. More details about the algorithm are available in TxRep POD.
Additionally, you can also enable the txrep_autolearn option. In such case, when the message scores trigger SpamAssassin's auto-learn process, TxRep will add or subtract additional points to the stored reputation.
How do I whitelist/blacklist someone?
Blacklisting/whitelisting can be done manually through the following command line options of SpamAssassin:
spamassassin --add-addr-to-blacklist=foe@spam.bizspamassassin --add-addr-to-whitelist=friend@ham.org
It is necessary to understand that whitelisting/blacklisting through TxRep is not the same as whitelisting/blacklisting in a cf file, using the whitelist_from or blacklist_from directives. TxRep whitelisting/blacklisting adjusts the reputation of the plain email address by a high score (details can be found in TxRep POD). This blacklisted or whitelisted reputation score can wear out over time, as scores of new messages from the sender are added to the total reputation score.
Besides whitelisting/blacklisting of email addresses, in the same way also domain names, IP addresses, and dot-less HELO names can be whitelisted or blacklisted. For example:
spamassassin --add-addr-to-blacklist=spamming.bizspamassassin --add-addr-to-whitelist=12.123.12.234spamassassin --add-addr-to-whitelist=1234:abcd:1111:12::3spamassassin --add-addr-to-blacklist=foe-pc
Please note that when blacklisting/whitelisting an email address or domain, all records of the address or domain bound to certain IP address, DKIM signature, or an SPF pass, will be removed from the database, and only the plain record (not bound to any specific IP address or signature) is kept.
TxRep can also automatically whitelist all recipients of outgoing email. To enable this feature, set the option txrep_whitelist_out. Please note that this feature can only work when SpamAssassin processes outbound email too, and, of course, it does not work for email sent through 3rd party SMTP servers.
Database Storages & Utilities
TxRep uses the same storage handlers as its predecessor AWL, therefor TxRep Berkeley DB format backend files can be examined, pruned, and manipulated with the same tools as at AWL. See also the AWL page for some more details
- 'sa-awl' - simple tool available in the distribution package
- 'sa-heatu' v1 - external enhanced tool
- 'sa-heatu' v2 - newer, enhanced version
When using the SQL storage type, multitude of SQL tools can be used for the same purpose - for example PhpMyAdmin for MySQL, PhpPgAdmin for Postgresql, etc.
Although requested, there is currently no Redis storage handler available for AWL or TxRep, but MySQL storage tuned with the MEMORY engine, or InnoDB engine with a sufficiently big innodb_buffer_pool parameter, or together with the MySQL memcache plugin, would offer similar performance as Redis, while allowing much better vertical and horizontal scalability (it would work better for both bigger tables and multiple concurrent accesses as well).