XF2 Anti-Spam Needs Upgrading

[Anthony Parsons]

XF has a decent anti-spam system for robots, but it sucks for human spam. It is seriously cheap to employ people in India and such countries to do nothing but copy and paste spam. They can do it at incredible rates, and once human registered for multiple accounts, they let the software do its work.

XF2 needs real url control limiting by the admin, where the admin can set defined limits to when a user can post any link, including embed, image, etc, incorporate a status structure to maintain link posting of x likes to x posts, a ratio.

I use a system like this now... I have zero issue. Like many developers though, development is no more for it. XF2 really needs to be taking care of total spam control at the core, not leaving it for additional developers to fix.

I have near zero spam issue by limiting links to a minimum 50 posts, and then requiring a constant ratio to be maintained at all times to continue posting links (like to posts). It really throws the dedicated spammers off... thinking they can hang around for 20, 30, 40 posts... to find even at 50 they still can't post because their posts are crap and not liked enough to upgrade their user group.

IMHO... I hope XF2 is catering this at the core. No link control in any field, anywhere, across the entire software, at discretion of the admin by usergroup control.

 

[Brent W]

I agree. Spam would be much more controlable with a simple permission to control whether a user can post links in posts, website etc.

Until then we just moderate all registrations from India, China, Russia and Pakistan.

 

[ŽivaAkcija]

i agree there is HUGE problems with human spam and nick names like wer2345, sirt654 and others, xenforo 1.5.x failed on human spam.

 

[Amaury]

XenForo's current anti-spam measures are honestly excellent. Look at how rarely spam gets through on here. You just need to set up your anti-spam settings properly.

 

[ŽivaAkcija]

XenForo's current anti-spam measures are honestly excellent. Look at how rarely spam gets through on here. You just need to set up your anti-spam settings properly.

VAU realy, please tell us how to configure properly.

 

[Amaury]

There are plenty of guides here. If you don't set them up properly, that's on you, not XenForo.

 

[Brent W]

XenForo's current anti-spam measures are honestly excellent. Look at how rarely spam gets through on here. You just need to set up your anti-spam settings properly.

I see spam on here a few times a week. Spam that would have never been made public with a simple permissions check. I also see attempts to post spam almost hourly on our largest forum now that we moderate countries.

 

[Fred.]

I totally agree! A usergroup permission to post links would be the first thing to do. And make a seperate one for PC's please!

 

[Divvens]

We use the inbuilt spam check to check first message of all users and if it contains a link its put in mod queue. Anyone above 1 approved message/legit message is never troubled by this. Spam messages have not made it into public eye for over 7 months since we took this approach. This however does not help dormant bot accounts.

 

[Anthony Parsons]

We use the inbuilt spam check to check first message of all users and if it contains a link its put in mod queue.

Yer... but human spam is far beyond that now. They register multiple accounts, they post general comments onto general content, thus otherwise legit posts, then they push. I found link limitations didn't work, because I had human spammers register accounts, input the details into bots, then the bots posted general content to meet x threshold, then spammed. Moving to a ratio stopped it all. Members must have a higher like ratio than posts to continue posting links.

Don't get me wrong, moderators and members have active roles in my communities, the ability to report content and place it directly into moderation at the member level, and we have mods globally located so there is near always someone online to quickly catch any human attempts. We get a fair share of human spam that is caught in our system... very little ever makes it public, but if we didn't have link permissions via user group promotions and a ratio filter being applied to every member for like / post to ensure quality content is posted and obtaining likes, not just ****e... it would be a huge battle to control it.

I believe the core product needs more robust anti-spam measures... leaving admins and third parties to develop features and interesting products for use... not having to sort basic spam problems.

 

[Alpha1]

We get 4000 new members per month and barely get any spam because of the solutions we created for my vbulletin big board and manual review of all new accounts. We are now automating the process for XF1.
The main problem is not automated spammers but human spammers. These are:

1. paid human spammers
2. representatives of websites and organisations who want to promote their site/product/organisation/service

We are quite far in the development of a system for XF1 that provides the functionality to eliminate human spam. The functionality is based upon our experience in the last 10 years and reviewing all new accounts and analysing which factors are at play with flagging a human spammer account. We have identified similarities in what human spammers will do, will not do and cannot do.

At registration we need criteria for default fields and non default fields, including username, website. profile fields, date of birth.

• We need to be able to set which fields are mandatory (default and non default fields)
• We need to be able to set which fields to show and hide on registration page (default and non default fields)
• We need to be able to easily add new criteria to moderate an account or not.
• We also need to be able to add separate criteria to refuse registration or not.

Criteria can relate to keywords in aforementioned fields, but also the composition/content of the fields. This will allow the admin to set minimum characters, minimum different characters, keywords that must be in the field, keywords that should not be in the field, URLs or email in fields, or add other algorithms.

The beauty of criteria is that every admin can use unique values.

Email whitelisting and blacklisting is important. Only allow registrations from whitelisted email domains. No more spammer@pakistanpharmacy.com or spammer@disposable-email.com

IP range flagging is important. Does an IP range get many spam registrations on your site or in a SFS/PH blacklist in the last 1/2 months, then always moderate the registration or if the value is really high then just automatically block the range for a few months.

The above are some examples of what can be done at registration. Most of the above is already implemented in an XF1 addon we built.

Then there is a need to stop human spammers that make it trough registration. Basically the same applies:
We need criteria to moderate suspect posts and accounts that are applied to the posts of new users. If posts and profile fields of new users meet criteria then send it to the moderation queue. Example criteria:

• New user posts link or email.
• New user posts have received X reports.
• New user posts have received X downvotes. *

* For posts we also need downvotes. This would not only fulfil our legal requirement to counter illegal content, but also allows our members to flag spammers with one click post ratings or reputation. More than X downvotes sends to moderation queue.

Multiple account detection is essential. We cannot do without it. We need criteria for:

• IP's, ranges, static vs dynamic
  
• profile similarity (fields, DoB, username, simple spelling algorithms)
  
• browser fingerprinting
• cookies

These are just some of the many signals to consider. It should be easy to identify if two accounts are operated by the same person. Once again this needs criteria to either moderate or block the account.

As human spammers and other malicious users use scripts and bots after registration these bots need to be blocked during normal website use. Solutions like Bad Behavior are useful for this. @DragonByte Tech is creating an integration for it in their DBT Security addon.

 

[Teapot]

One thing I've done on my site (which is actually in core in XF2 as they're not there by default anymore) is to remove the occupation and home page fields from the site – our community doesn't use them, so they were only used by spammers. We've also set up a "Verified" usergroup, and an automatic promotion to put new users into it after some legitimate activity on the site. This restores the ability to use links in signatures and start private conversations, which aren't granted to users in the standard Registered group. This, along with the excellent "TPU: Detect Spam Registrations" mod and a blacklist for disposable e-mail addresses, means we very rarely get any automated spammers, and those that get through are unable to actually post anything interesting.

In general, all I'd want to see in XF2 core is more nuanced spam management tools at registration and post time – the DSR mod we use gives us very fine-grained control over many useful functions, and its functionality is something I'd love to see make its way into core.

One thing my system doesn't solve, however, is the practice of human spammers registering and then copying content from other forums – I don't know exactly why they bother, as there weren't any links in the post or their profiles, but it's still a thing that happens. It's far less of an issue, though, and not one that can easily be countered mechanically.

I also agree that multiple account detection is a feature I'd really appreciate in XenForo, but I think that's a topic for a different suggestion

 

[Anthony Parsons]

At registration we need criteria for default fields and non default fields, including username, website. profile fields, date of birth.

• We need to be able to set which fields are mandatory (default and non default fields)
• We need to be able to set which fields to show and hide on registration page (default and non default fields)
• We need to be able to easily add new criteria to moderate an account or not.
• We also need to be able to add separate criteria to refuse registration or not.

How is this going to stop human spammers?

XF already has a good honeypot system built-in, you can do a fair amount with custom fields already.

IP range flagging is important.

I find IP's unreliable, especially with the increased use of VPN's.

A system to detect sneaky URL's, any form or effort to create a URL in near any context, email, phone, so forth... that pretty much removes you from spam, as spammers get nothing if they can't direct you somewhere, to something.

 

[pegasus]

A high proportion of the spam on my XenForo board posts URLs directly as thread titles. It seems like it would be simple just to block posts like this, as there is almost no situation where a fully-qualified URL would be an acceptable thread title. Such a measure would block close to 50% of the content I have to manually disapprove on a daily basis.

When deleting spam content, it would also be nice to have an inline moderation option to:
- Ban or fully delete the posting user's account
- Automatically delete all other posts by the same user

I don't like leaving 1000s of spam accounts, many with fake email addresses, in the database. Once the spam content is removed, it becomes difficult to mass identify and select the spam accounts. And when you have to delete hundreds of spam per day, individually altering the accounts as you remove each post is too time consuming.

 

[Alpha1]

How is this going to stop human spammers?

95% of human spammers can be flagged based upon how they fill in mandatory profile fields, including username.
Obviously I cannot share in public what parameters we use to identify human spam accounts.

I find IP's unreliable, especially with the increased use of VPN's.

True but that's not the point. IP ranges are important when it comes to IP blocks that are almost exclusively used by spammers. It happens regularly that a mass of spammers keep coming from the same heavily abused IP range and the IP range is not used by any legitimate users. In such case it makes sense to block the IP range as an additional measure.

For example: some time ago we kept getting hit by spammers from Cameroon on an IP range that was never used on my site in the last 13 years. The same IP block was also heavily reported on by Stop Forum Spam with hundreds of spammer IP's. As SFS works currently, fresh IP's are not blacklisted if they are not in the SFS database and thus allowing spammers to create new accounts. As the chance of any legitimate users from thsi IP block is practically zero, it makes sense to block all registrations from this block for some time. I think most admins will just IP ban such range without looking back if they find out that they have a spam attack from a specific IP block.

 

[Anthony Parsons]

IP ranges are important when it comes to IP blocks that are almost exclusively used by spammers.

True... to be honest, I block the entire of China from my servers. The majority of its traffic is spam and naughty stuff... I think I've had one actual English speaking user from China over the past 10 years.

 

[Anthony Parsons]

A high proportion of the spam on my XenForo board posts URLs directly as thread titles.

If XF2 incorporated the discussed proposal here for URL's, then this would be a none issue, as it would block it the moment the field detected a URL structure, or any attempted type of URL structure:

• https://xenforo.com
• x e n f o r o . c o m
• xenforo . com

So forth. Sneaky ones are the work in progress with such systems, but you eventually get 99% of the sneaky types covered adequately.

I mean ALL fields, username, full profile, any input field needs to be blocked by permission control.

 

[Fred.]

I can't block countries because I have a travel forum, people connect to my forum from all over the world.
No problems so Far.
I just have this in banned email *@*.cn, *@*.ru