XF 1.4 I am quite literally having a nightmare with spam :(

[Duncan Wraight]

Morning folks,

I've been using XenForo for just over a year now on www.thetabletennisforum.com.

Past 3 months or so we've been getting hit by spam... incessantly.

I run quite a few WordPress websites and I use CleanTalk.org to moderate those. One of those websites has good Google ranking and pulls in about 30,000 to 40,000 unique visitors per week yet I don't get any spam thanks to the WordPress/CT combination. When I check the reports for CT there's a ton of spam trying to hit the website, but none of it getting through - last month it was 26,000 users/comments.

On my XenForo install I've probably only had about 1,800 spam users/comments but a significant portion of those have gotten through.

Here's what I've got setup, options-wise:

• Check new registrations against CleanTalk
• Check new registrations against the StopForumSpam database (mod when 1 flag, reject when 3 flag, count flags from past 60 days)
• Check DNBSL on registration (Project Honey Pot)
• Manually approve reg if user shares IP used by a banned user
• 10s reg timer
• Check 3 messages
• Akisment API key

Plugin-wise:

• Antispam by CleanTalk 1.0.0
• KeyCAPTCHA 2.2

I've tried Q&A CAPTCHA ("What is table tennis also known as?", "ping pong", "ping pang", "pingpong"), reCAPTCHA and KeyCAPTCHA. I'm now back using Q&A ("What is the four letter acronym of this website?").

I'm no expert with this, at all. I don't know if I'm possibly bamboozling XenForo by using too many different spam-checking services. Therefore I'm here to seek some advice from fellow users or XenForo staff.

I've bullet-pointed my main concerns and would appreciate any advice which addressed them:

• A lot of new registrations are bots. I'd estimate that 50% of them are getting through my spam filtering options. I have ~700 members and I think around ~400 of them are bots. I can't find any way of identifying the bots en masse - I've gotten rid of all .ru mail users but the remaining bots are on @Outlook, @yahoo and @gmail, but so are a lot of our human users.
• Once bots have registered, they tend to put weblinks in their profile. Is there any way I can search for all users with profile posts?
• When the bots start posting, XenForo catches about 80% of the posts and they go into "Moderation". However, this means that for Administrator users, we've got page after page of green-highlighted spam posts (not visible to standard users). To remove these from the forum entirely, I have to manually click through them and delete them. That's pretty much the same as not having any spam filtering!
• The other 20% of posts have to be manually deleted/spam cleaned

Thanks in advance,
Duncan

 

[TPerry]

A LOT of the new spam registrations are actual HUMANs that are able to be hired cheap over in Pakistan/India area and log in and spam.

 

[Jake Bunce]

You have a lot of stuff implemented to handle automated spammers. Tracy might be right... you might have more of a problem with human spammers.

On my forum I used to have a required field on registration where I would ask users why they wanted to join. Then I would review their answer in moderating new registrations. You might consider something like that to deal with humans.

 

[rainmotorsports]

The thing I noticed on our human spammers is most of them are registered as female with 2 names. Sometimes you get a common name between them luvv, Ronnie, Indina. If yours are like that you can spot them at a glance.

With the stop forum spam set the way you have it, I'd expect most of them to go into moderation. The longer the check time the better this goes but the more single flag false positives you will get.

 

[Duncan Wraight]

They could be human? Are you kidding?! Urrgghhh that's quite genuinely upsetting!!

I do have a lot of females, all with two names and mostly "normal" sounding names. Are they bots or human-bots?

 

[Jim Boy]

One thing that helps us is that we dont allow new users to create threads until they have either made 20 replies or been registered for 20 days - you can set that up by using the promotional system

We also ban over 100 email address patterns such as *@guerillamail.org

 

[Duncan Wraight]

Mmm that's an interesting idea, Jim. Won't the hu-bots just start posting in threads instead of creating their own then though?

 

[rainmotorsports]

They could be human? Are you kidding?! Urrgghhh that's quite genuinely upsetting!!

I do have a lot of females, all with two names and mostly "normal" sounding names. Are they bots or human-bots?

Quick check the IPS. You get a lot of Pakistani, Airtel broadband in India and the rest of the spammers will have obvious data center IPS in the US and other countries like leaseweb, rackspace, hilvelocity.

I'm getting told they are human and generally when you have a puzzle captcha it means the ones getting through are human. They pay people to do this stuff.

If you want I'll give you a hand sorting through some but the future ones are on you.

 

[Duncan Wraight]

If you want I'll give you a hand sorting through some but the future ones are on you.

Ah I really appreciate that, thank you, but don't worry - I'll sift through 'em myself one day at work when I'm really bored...

 

[SoManyPosts]

I would try KeyCaptcha – they solved a spam nightmare on a forum I used to host that had 250,000+ views/month. It's very nice.

 

[Duncan Wraight]

I was using KeyCaptcha for the majority of the time, chris. Didn't seem to make a hint of difference

 

[SoManyPosts]

I was using KeyCaptcha for the majority of the time, chris. Didn't seem to make a hint of difference

That's very surprising. It does seem like you are dealing with human spammers as mentioned earlier.

Wild idea, but see if you can use three captchas at once.

Also, sign up for CloudFlare and turn on "I'm under attack" mode.
Get the Pro version if you can, too. You can also email CloudFlare and should be able to help you.

That's my only other advice, except manual approvals and emailing people to verify or something.

 

[Duncan Wraight]

Thanks pal, I'll give that a go.

 

[SoManyPosts]

Thanks pal, I'll give that a go.

No problem! CF should scan the browser for 5 seconds or so to verify it. It should work very well against fake ones.

 

[TPerry]

That's my only other advice, except manual approvals and emailing people to verify or something.

Or just simply doing a block of India & Pakistan if they aren't important to the site for visitors (which for most of mine they aren't). Was really simple to do in the firewall.

 

[Duncan Wraight]

That's a definite option, we don't really have any non-UK or US visitors. Could you PM me the rules you used?

 

[TPerry]

I'm using ConfigServer Firewall and the CC_DENY ability

CC_DENY = "IN,PK"

This is dependent on being on a VPS or dedicated server.

 

[rainmotorsports]

That's a definite option, we don't really have any non-UK or US visitors. Could you PM me the rules you used?

In case anyone suggests it I tried TACs stop country spam with PK set and it was 100% in effective on blocking any Pakistan traffic. Were talking well known IP ranges with pk as the domain in the hostname. Maybe I missed something but its definitely easier to pull off in the firewall at that point.

Some sites out there will generate rules for you after telling it what firewall you use or worst case it can be done via htaccess as well.

 

[SoManyPosts]

CloudFlare also makes it easy. You'd just type the countries in the block box and it denies them all haha.

This way, they never even hit your server nor do they know where it is located, as with all CF traffic.

 

[Radiola]

A LOT of the new spam registrations are actual HUMANs that are able to be hired cheap over in Pakistan/India area and log in and spam.

What Tracy said. In addition there are also a lot in China and the Philippines. Check the IP addresses.