...
Warning: Enabling this will almost certainly slow down SpamAssassin. Possibly a lot (I haven't checked).
More general advice
SA should log per-message scan times to the system log. From that you should be able to determine a message recipient and message-ID for messages that take a long time to scan.
You may also be experiencing inbound traffic volume spikes that may be overloading your system.
So, if you can track when the processing spikes occur, two things to correlate are the number of messages you're processing at that time (you may have too many spamd children defined, or not enough memory - are you hitting swap?), and any specific messages received at that time that take an unusually long time to scan (they may be exercising weaknesses in the rules).
Older versions have had rules that were known to perform poorly in certain situations and that have been fixed in newer versions.