How can I change the way SpamAssassin marks up messages it considers to be spam?

The best step is to type "man Mail::SpamAssassin::Conf" at a shell prompt – amongst other things, this explains the options available. The online version may not match the version you have installed.

You want to focus on "add_header", "rewrite_header", "report_safe" and a few others. Please try not to use the deprecated options as they will eventually disappear.