XF 1.2 Spam Prevention and Management

While 1.1.4 added some spam prevention tools, XenForo 1.2 takes these further. This is by no means the end of our fight against spam though.

In 1.2, we've mostly focused on adding support for APIs and adaptable spam prevention techniques. Let's look at a few.

Deeper Stop Forum Spam Integration
We actually detailed this a long time ago. While 1.1.4 added support for checking a DNSBL that used the Stop Forum Spam database, 1.2 has a much more configurable integration:

ss-2013-05-28_10-53-35.webp


In addition to simply checking registrations against SFS, you can also get an API key and submit back to them automatically when spam cleaning a user.

If you opt to use the SFS integration to check registrations, the Tornevall DNSBL will not be checked.

Project Honey Pot HTTP Blacklist
In addition to the existing DNSBLs, you can sign up for a Project Honey Pot API key to check their blacklist. While this often overlaps with SFS's database, there are some variations.

ss-2013-05-28_11-03-32.webp


Message Content Checking: Akismet and Spam Phrases
Up until now, we've focused on preventing spam registrations. While that's the ideal situation, it's not always possible to detect spam until you see what they actually try to post.

ss-2013-05-28_11-08-18.webp


Here, we provide a simple "spam phrases" option, which uses simple pattern matching (or regular expressions) to detect spam. For example, here on XenForo.com, the vast majority of the spam we see fits a particular profile that can be matched with "watch * online". Content matching these phrases will be rejected or set to be manually approved. The checks are done on posts, profile posts and conversations. In situations where manual approval isn't possible (such as in conversations), the messages will be rejected.

To limit the potential for false positives, the checks are limited to only the first few messages by a user.

Then there's Akismet. It provides an API that uses various heuristics to tell you whether the message being submitted is spam. Note that depending on your site's purpose, Akismet may not be a free service.

Spam Cleaner Can Delete Sent Conversations
To help deal with conversation spam, the spam cleaner can now (optionally) delete all conversations started by the spammer. This will remove the conversations from the inbox of anyone that received them (and update their unread conversation counts).

For Developers: General Spam Prevention API
There is now a SpamPrevention model that is called in various scenarios. You can simply hook your spam prevention call into the correct position and return what action to take. This is called in the following situations:
  • When a new user registers
  • When a user is spam cleaned (to submit spam users and spam content)
  • To determine if the user posting a message needs to be spam checked
  • When a message is submitted
  • When a message that was moderated because of spam and is approved (submitting "ham")
This system should make it easier to add-on developers to integrate their own spam prevention systems.
 
I think quite a lot of people will be willing to add an API key, once they realise the benefit of reporting spam is a feed back loop for them (the more they report, the more automation spam avoids them)

You can spam clean from a profile page, which would submit it to SFS.

I know that lots of people have liked this comment... but what you just said here might not be good.

Not all spam cleared data is real spam, I do not want to report all things that I clear as spam to StopForumSpam (and hope that this is not default once the SFS key is added),

...so I assume there is an option (which is unticked by default) for submitting to the SFS database, and we do not make the mistake of going down the path of accidentally submitting tonnes of false positives.
... If we went down that path, SFS would almost certainly have reasons to be angry

It should be easy to administrate, yes, but a higher priority should be given to avoid false positives. By adding the reporting method to the core, the responsibility of avoiding false positives submission pushes closer to the XF developers... So are there methods in place to avoid false positive submission

  • unticked option for submission
  • only submit certain accounts
  • If it's an old account, do not submit
  • If over x posts, do not submit
  • If no proof is available, do not submit
  • etc..
Or something along those lines
 
...but an email already arrived about the conversation into the receiver's email address.

There is little that can be done about it after the email has been received.

It's no different to getting a watched thread email and the thread or post is subsequently deleted.
 
There is little that can be done about it after the email has been received.

It's no different to getting a watched thread email and the thread or post is subsequently deleted.
Isn't it possible to check the content of the message before it's being sent?
 
There are tools, they are just not used in XRumer yet (just google OCR)... there are even open source artificial neural networks written in JavaScript, I can write a tool using ANNs, so the technology is there.

What I can't do is identify custom objects within that image (at least not very easily). For example "a pink elephant stepping on a mouse", if the question was "what is the elephant stepping on" I would find it easy to extract the text from the images (most of the time), but quite hard to relate the text to the objects in the image... so I would be left with brute forcing for anything associated with "elephant" (mouse will come up in that list of associations). If I use a big arrow and point to the mouse, and ask, what is the arrow pointing at... my list of brute for attempt becomes very large, making the image CAPTCHA very hard to solve programmatically (yet still very easy for humans)


Oh I understand. That would be a great preventing system in deed. But did it exist already? Because when it doesn't or when Xenforo itself don't have the time to make such a CAPTCHA, I might have some developers who can.
 
I have to agree with this. I have been checking the tools that were and are being used to spam forums, and there is no tool or anything that can automatically solve questions in images.

Agreed. The two things we implemented on our (vB based) forum which stopped spammers pretty much dead was:

1) Questions based on an image at registration
2) Unable to submit a post with links in until X posts reached

Very occasionally a persistent human will complete the registration, only to be immediately frustrated as their script cannot submit the spam posts.
 
For Developers: General Spam Prevention API
There is now a SpamPrevention model that is called in various scenarios. You can simply hook your spam prevention call into the correct position and return what action to take. This is called in the following situations:
  • When a new user registers
  • When a user is spam cleaned (to submit spam users and spam content)
  • To determine if the user posting a message needs to be spam checked
  • When a message is submitted
  • When a message that was moderated because of spam and is approved (submitting "ham")
This system should make it easier to add-on developers to integrate their own spam prevention systems.

^^^THIS is the best part of the announcement. (y)
 
I want to mention that I have 0 spammers since February since I have changed from ReCAPTCHA to Q&A:

"what is the name of our forum domain? the answer is ABC EFG.com, but just without space between the 2 words." correct answers:
- abcefg.com
- abcefg
- abcefgcom

So I don't need any spam improvements on xenforo and my Q & A is easier for my users than an unreadable reCAPTCHA ;)

PS: And my forum ranks well on google, so the spambots know my forum ;)
 
I'm liking the spam phrases option, I could see it being useful in various situations even if the user isn't a spammer/bot. I was wondering before, but none of these produce any spam registration logs right? I guess I'm used to these from addons, kind of interesting to see the list of registrations prevented by who. What might be nice is a log for Q/A. If you could associate it with a bot you can easily see which questions need to be replaced quickly.

Not much of a problem, but I noticed in the first screenshot the sub checkbox lines seem too close to the hint/explain text above it, maybe some extra margin/padding in that case? "The Project Honey Pot Key:", "Action:", "Spam Phrases:" headings also seem a little awkwardly placed, might just be me though...

A HYS on Friday, Monday and Tuesday?

What could it all mean? :cautious:
It could mean this: http://xenforo.com/community/threads/announcement-hys-are-too-invisible.50783/page-4#post-543193 :p
 
Man, this is getting seriously awesome!

Our mods and I always had trouble dealing with human spammers. I hope the new system will make it even easier to prevent spam.
 
1.2 is going to kill a part of my Stop Spam Here addon :D. Nice updates.

IMO, I do not like Captcha, sometime it is hard to fill out or complete, I like to use Spam bot checking and Akismet checking that do not disturb almost real users.
 
I use the Spam prevention add-on by sonnb which does all this and much more, was really helpful in controlling. Though all this is sufficient for me I think. So one less add-on to install.

I think I will have to uninstall half my add-ons after the 1.2 upgrade. Keeping my fingers crossed for a few more though, especially navigation manager and conversation manager. (y)
 
Top Bottom