Blog spam is completely different from e-mail spam. The objective of the e-mail spammer is for you to read their message and respond quickly. The opposite holds true of the weblog spammer. The spammer needs their comments to remain undetected (or at least undeleted) to boost and maintain the pagerank of the site that they are spamming for. This type of spam is not limited to blogging systems and can easily be expanded to take into account other collaborative portals (e.g., wiki, forums, etc.). In regards to blogs, the main ports of entry for spam are:
While there are difference between e-mail spam and blog spam, SpamAssassin is a strong candidate as a basis for preventing blog spam. There have already been several attempts to integrate SpamAssassin with a blog (WordPress and Moveable Type):
These plugins basically take the content from a blog, tests it with SpamAssassin, and flags it as needing moderation if deemed unsafe.
Collaborative filtering: IronPort maintains a database of e-mail server traffic volumes called SenderBase. Mail servers can use SenderBase to find "traffic spikes" and potentially block e-mail from those servers. Something similar could be done for weblogs. As comments come in, weblogs could report the urls in the comments to a central server. If an URL is sent in too rapidly, it can be added to a list of probable spam urls and weblogs can quarantine or delete comments containing that url.
DNS-based URI Blocklists: SpamAssassin has had great success using Jeff Chan's Spam URI Realtime Blocklists. When an e-mail arrives, SpamAssassin extracts the urls contained within and performs a few DNS TXT queries to find whether the url has been reported in spam. These blocklists can be used for weblogs too. Instead of Jay maintaining a central blocklist that people download and install manually, mt-blacklist could use a DNS-based blocklist that is effectively updated in real time. This would significantly cut down on comment spam because weblog owners would not need to actively maintain their blocklists. The submission process could be streamlined so that it doesn't consume so much of any one person's time.
Sharing its methodology with BlogSpamAssassin under PHSDL GNU.
Project Framework Constraints
- Not to stop search engine Spam
- Stop comments Malware and redirect domains Spam
- Not to stop off topic comments Spam
- Develop a universal API anti Spam filter for different Scripting languages
- API anti Spam Filter must be available for forums, blogs, and book marking services
PHSDL is supported by StopBadware
StopBadware uses Malware API list from Google
BlogSpamAssassin can adopt the StobBadware Google API framework
API Anti Spam Filter can be developed for WordPress, phpBB, vBulletin, and SMF
Avoid Problemetic Honeypots
WikiPedia domain is not Spam (Open relay error)
- False Positives Documented (Human editor error)
Minimize resource consumption
- Structure SBL by domain not by url or sub domain
- Query SBL primary database for domain match
- Utilize push technology vs. pull technology
Adopt Standarized Filtering Techniques
- Primary Filter needs to be based on Malware and cloaking redirect Domains
- Tertiary should be user defined white and black list Filters
BlogSpamAssassin Sub Algorithm PHSDL Filter Test
A mailing list has been created to begin work on this project.
You can subscribe via: