Ugh. No matter how much I try and cover all the bases, some little thing always turns what should be a 2-3 hour transition period into a 12 hour ordeal. Everything went pretty smoothly until final testing on re-training false positives with dspam. I had tested it with the existing quarantine, which was created under dspam 3.6.x, and everything worked fine.
So I flipped the switch, and as mail started to filter in after the switchover I tested the training app and, well… telling dspam that a mail it had flagged as spam wasn’t actually spam didn’t work. It’s a long, sordid story, so I’ll cut right to the chase.
This problem only appears if you are using postfix as the MTA, and running dspam in daemon mode via LMTP using the configuration outlined in ./doc/postfix.txt
Here’s the symptoms:
- Attempting to retrain false positives in the quarantine fails. You can select the message for retraining, but when you submit the form the page simply reloads with the message still in quarantine.
- Attempting to retrain false postives from the history page are reported as successful (message is marked as retrained in the history), but the message remains in the quarantine list and is not delivered to the mailbox.
- Attempting to view the message in quarantine results in a blank screen
- ^M characters are in the user’s .mbox file (this is the root cause of the problem)
The problem with 3.8.0 is that it adds <CR><LF> as an EOL to the user’s .mbox file instead of just <CR>. This breaks how dspam.cgi parses things. John Peacock came up with a fix for dspam.cgi that enables it to parse the messages properly, but that addresses the symptom, not the root cause, which is dspam itself. If you’ve had any messages delivered to your users without the dspam fix, you’ll probably want to modify dspam.cgi with the John’s patch until it’s reasonable to think the users have dumped their quarantines (or care about the messages in them).
There is a patch for dspam that corrects the problem, but it is unfortunately not in CVS, nor in the release tarball. The patch was only posted to the dspam dev list, and was a little hard to find until John sent it to me. For reference, this information is available at the mail-archive for dspam-dev under “Patch for ^M in Quarantine / Blank Quarantines”. The patch file is also at the bottom of this post, and you’ll need to patch dspam.c in your source and recompile. This will correct the linefeeds being inserted into the quarantine, and everything will play nice moving forward.
Huge, huge thanks to John Peacock for providing the trail, and for answering mail on a holiday pointing me at the fix. I was kind of banging my head over this from 2am through 8am. Good times.
Patch after the jump:
765c765,770
< copyback = _ds_assemble_message (CTX->message, (USE_LMTP || USE_SMTP) ? "\r\n" : "\n");
---
> if (CTX->result == DSR_ISSPAM && (!(ATX->flags & DAF_DELIVER_SPAM))) {
> copyback = _ds_assemble_message (CTX->message, "\n");
> } else {
> copyback = _ds_assemble_message (CTX->message,
> (USE_LMTP || USE_SMTP) ? "\r\n" : "\n");
> }