NOTE: This plugin is based on the OcrPlugin written by Maarten de Boer and was extended and improved.
This plugin checks for specific keywords in image/gif, image/jpeg or image/png attachments, using gocr (an optical character recognition program).
This plugin can be used to detect spam that puts all the real spam content in an attached image. The mail itself only random text and random html, without any URL's or identifiable information.
Additionally to the normal OcrPlugin, it can do approximate matches on words, so errors in recognition or attempts to obfuscate the text inside the image will not cause the detection to fail. Another improvement was to move the wordlist into the configuration file so it can be easily extended.
You will need giftopnm, jpegtopnm and pngtopnm (from netpbm) and gocr installed.
Additionally, you will need the perl module
String::Approx |
and giffix (from giflib).
Version 2.0: * Replaced imagemagick with netpbm tools
Save the two files below in your local configuration directory. Open FuzzyOcr.cf and extend the wordlist as you wish.
The scoring is dynamic, more word matches lead to a higher score. The scoring is done as soon as focr_counts_required matches were found. It scores exactly focr_base_score points then. For every additional match, it scores additionally focr_add_score points.
Attention: Do not add a score line to the config file. It will not be used! Scoring is done INTERNALLY and can only be configured with the two parameters described above.
The variable $countreq can be adjusted via the configuration file parameter focr_counts_required and indicates the number of matches that need to be found before any score will be triggered.
The variable $treshold is similarly adjusted with the configuration file parameter focr_treshold. This is a float value between 0 and 1 and indicates the maximum relative edit distance between the wordlist word and the obfuscated version (less means the words need to be more similar, 0 means identical). The default of 0.3 normally does not need any change. Note that this module also matches substrings (see example).
Explanation of the additional options:
focr_tmp_path - String determining the absolute path to a directory where the plugin may write temporary files to (without trailing slash)
focr_verbose - Verbose level (0 - 2). (1 is currently the default)
Lets say you have defined focr_word investor in your configuration. Now you receive an image which, after converted and recognized gives you:
ATTENTION ALL IN\lESTORS AND DAY TRADERS
Then the plugin will find the word investor. It would even succeed if the text was ATTENTION ALL STUPUDIN\lESTORSHAHA or INVSTORSZ etc.
Generally, the plugin follows these rules:
gocr can take up quite a bit of resources, so be careful. But it is only executed for messages that contain gif, png or jpeg attachments.
– Author: Christian Holler, decoder_at_own-hero_dot_net
loadplugin FuzzyOcr FuzzyOcr.pm body FUZZY_OCR eval:check_fuzzy_ocr() describe FUZZY_OCR Mail contains an image with common spam text inside # Here we defined the words to scan for focr_word stock focr_word investor focr_word international focr_word company focr_word money focr_word million focr_word thousand focr_word buy focr_word price focr_word trade focr_word banking focr_word service focr_word kunde focr_word volksbank focr_word sparkasse focr_word software focr_word viagra focr_word cialis focr_word levitra focr_word medicine focr_word legal focr_word medication focr_word click here focr_word penis focr_word growth focr_word drugs focr_word pharmacy # These parameters can be used to change other detection settings # # Detection treshold (see manual) #focr_treshold 0.3 # # This is the score for a hit after focr_counts_required matches #focr_base_score 4 # # This is the additional score for every additional match after focr_counts_required matches #focr_add_score 1 # # Number of minimum matches before the rule scores #focr_counts_required 2 # # Verbosity level (see manual) #focr_verbose 2 # # Path for temporary files #focr_tmp_path "/tmp" |
# FuzzyOcr plugin, version 2.0
# Changelog:
# version 2.0
# Replaced imagemagick with netpbm
# Invoke giffix to fix broken gifs before conversion
# Support png images
# Analyze the file to detect the format without content-type
# Added several configuration parameters
#
#
# written by Christian Holler decoder_at_own-hero_dot_net
package FuzzyOcr;
use strict;
use Mail::SpamAssassin;
use Mail::SpamAssassin::Util;
use Mail::SpamAssassin::Plugin;
use String::Approx 'adistr';
our @ISA = qw (Mail::SpamAssassin::Plugin);
our @words = ( );
# Default values
our $treshold = "0.3";
our $base_score = "4";
our $add_score = "1";
our $countreq = 2;
our $verbose = 1;
our $tmppath = "/tmp";
# constructor: register the eval rule
sub new {
my ( $class, $mailsa ) = @_;
$class = ref($class) || $class;
my $self = $class->SUPER::new($mailsa);
bless( $self, $class );
$self->register_eval_rule("check_fuzzy_ocr");
return $self;
}
sub parse_config {
my ($self, $opts) = @_;
if ($opts->{key} eq "focr_word") {
push(@words, $opts->{value});
} elsif ($opts->{key} eq "focr_treshold") {
$treshold = $opts->{value};
} elsif ($opts->{key} eq "focr_base_score") {
$base_score = $opts->{value};
} elsif ($opts->{key} eq "focr_add_score") {
$add_score = $opts->{value};
} elsif ($opts->{key} eq "focr_counts_required") {
$countreq = $opts->{value};
} elsif ($opts->{key} eq "focr_verbose") {
$verbose = $opts->{value};
} elsif ($opts->{key} eq "focr_tmp_path") {
$tmppath = $opts->{value};
}
}
sub check_fuzzy_ocr {
my ( $self, $pms ) = @_;
my @found = ( );
my $image_type = 0;
my $cnt = 0;
foreach my $p ( $pms->{msg}->find_parts("image") ) {
my ( $ctype, $boundary, $charset, $name ) =
Mail::SpamAssassin::Util::parse_content_type(
$p->get_header('content-type') );
if ($ctype =~ /image/) {
my $firstline = ($p->decode())[0];
my $tempfile = $tmppath . "/" . "spamassassin.$$.focr";
if ($firstline =~ /^\x47\x49\x46/) {
$image_type = 1;
open IMAGE_PROCESSOR, "|/usr/bin/giffix | /usr/bin/giftopnm - |/usr/bin/gocr -i - > $tempfile";
} elsif ($firstline =~ /^\xff\xd8/) {
$image_type = 2;
open IMAGE_PROCESSOR, "|/usr/bin/jpegtopnm - |/usr/bin/gocr -i - > $tempfile";
} elsif ($firstline =~ /^\x89\x50\x4e\x47/) {
$image_type = 3;
open IMAGE_PROCESSOR, "|/usr/bin/pngtopnm - |/usr/bin/gocr -i - > $tempfile";
} else {
$image_type = 0;
print "No compatible file type detected... skipping image...\n";
next;
}
foreach $p ( $p->decode() ) {
print IMAGE_PROCESSOR $p;
}
if ($verbose > 1) {
open DEBUG, ">debug.$$.focr";
print DEBUG "File type: $image_type\n\n"
}
close IMAGE_PROCESSOR;
open OCR_DATA, "<$tempfile";
while (<OCR_DATA>) {
s/[^a-zA-Z ]//g;
$_ = lc;
if ($verbose > 1) {
print DEBUG $_;
}
my $w;
foreach $w (@words) {
$w = lc $w;
my $matched = adistr($w, $_);
if (abs($matched) < $treshold) {
$cnt++;
if ($verbose > 0) {
push(@found, "\"$w\"" . " with fuzz of " . abs($matched));
}
}
}
}
close OCR_DATA;
unlink $tempfile;
if ($verbose > 1) {
close DEBUG;
}
}
}
if ($cnt >= $countreq) {
my $score = $base_score + ($cnt - $countreq) * $add_score;
my $debuginfo = "";
if ($verbose > 0) {
$debuginfo = ("\nWords found:\n" . join("\n", @found) ."\n($cnt word occurrences found)");
}
$pms->_handle_hit("FUZZY_OCR", $score, "BODY: ", $pms->{conf}->{descriptions}->{FUZZY_OCR} . $debuginfo);
}
return 0;
}
1;
|