Improving the user profile. Adding permissions and anti-spam for fields & overhauling the presentation

[Stuart Wright]

This is quite a broad suggestion, but I feel it's needed because there are some significant shortcomings with user profiles.

Spam checking
Firstly we have a big problem with people using the legacy profile fields Website, Occupation and About for spamming. Mainly Website.
These fields are carry-overs from vBulletin in a lot of instances.
I'm running this SQL:

SELECT CONCAT('[URL='https://www.avforums.com/spam-cleaner/name.%27,xf_user.user_id,%27/?no_redirect=1%27),website,location,snog_flag,message_count,about']https://www.avforums.com/spam-cleaner/name.',xf_user.user_id,'/?no_redirect=1'),website,location,snog_flag,message_count,about[/URL]
FROM xf_user_profile,xf_user 
WHERE xf_user_profile.user_id = xf_user.user_id and length(website) > 0
and user_state='valid'
and is_banned = 0
order by xf_user.user_id desc

which returns nearly 11,000 rows.
As well as the Website field, I'm noticing that the About field is being used with links to spam also, so the first suggestion is to put all user profile text entry fields through the usual spam checking routine whenever they are changed. And maybe block links from the About field also.

Use custom user fields
But I actually think we need more than that. It seems to me that the best way to address these and some other profile fields (Location, Gender) is to turn them in to custom user fields (predefined in new installations to make it easy for new admins) so that admins have more control over them. Including deleting them altogether (which is what I'd do to the Website field).

Custom user field permissions (and sort order button)
But I actually think we need more than that. We need more permissions for the custom user fields so that admin can control what usergroups can edit and view them. With separate permissions to view in the user profile and view in the postbit.
This would give lots more flexibility for the custom user fields so that they are unavailable to new members, for example, and only become available to more active members. Perhaps we want some user fields available for members to view but not guests or search engines. Perhaps we want to put specific fields in the message postbit as well as the user profile about tab.
There's also a reasonable argument for having view permissions for system fields like Join date and Birthday.
Maybe we also need a permission that puts edits in to the moderation queue so we can deal with particularly problematic custom fields like Website.
Oh and can we have a sort button so we can drag and drop the display order of custom user fields please?


User profile customisation
But I actually think we need more than that. With custom user field permissions (for viewing), we'd already be customising the contents of the user profile about tab. But I think we need to be able to have usergroup permissions for which tabs are shown on the user profiles as well.


We could go even further and allow the customisation of the user profile tabs and allocation of custom user fields to those tabs.

It's 42 years since I started coding on computers so I can make an educated guess that making the above changes in Xenforo core won't be trivial. But I'm guessing (hoping) they won't be huge projects, either. The benefits in flexibility will be significant, though.

Xenforo guys please correct me if I've made mistakes in my assumptions about what is and isn't already possible.

So folks reading this, if you think it's time there was a revamp of the user fields and user profile, please upvote this suggestion (using the little up arrow top-right of the post).
Maybe we can have the guys shoehorn it in to 2.3? Cheers

 

[digitalpoint]

I can make an educated guess that making the above changes in Xenforo core won't be trivial.

It’s about an hour of work. I did all that (and a lot more in a couple hours).

 

[Alpha1]

As long as we can still allow spammers to fill in the website and about fields, I am happy. These are great honeypots and as such it saves a lot of work.

 

[digitalpoint]

As long as we can still allow spammers to fill in the website and about fields, I am happy. These are great honeypots and as such it saves a lot of work.

Yep agreed. In my Profile add-on, I don't go so far as to automatically ban the user, but it does log to the error log in the admin area if there's an "extra" field submitted by the user. So the error log shows who the user was and what the value was.

if (!$visitor->hasPermission('general', 'editWebsite'))
{
   if ($this->filter('website', 'str'))
   {
      \XF::logError(\XF::phrase('extra_field_x_in_request', ['field' => 'website', 'value' => $this->filter('website', 'str')]));
   }
   $request->set('profile.website', '');
}

At some point I should make it an option to automatically spam-clean/ban the users, but haven't gotten around to it (initially I was thinking I should let it run like that in case there was some false-positives somehow... which there hasn't been).

 

[Marcus]

some other profile fields (Location, Gender)

These should be autofilled by the system on AI guesses, the location from geoip, gender, interest etc. from google analytics (dont know if GA actually back-shares its "findings"), active times, activity times (like who posts every 35 hours like clockwork) etc. Lots can be autofilled by AI.

 

[Stuart Wright]

As long as we can still allow spammers to fill in the website and about fields, I am happy. These are great honeypots and as such it saves a lot of work.

That's true, but it's still plenty of work to deal with them if you have lots of daily registrations.
I've hidden the Website field now with a template edit, but really I would rather people didn't try to spam because they knew it would fail either because there are no fields they can use to spam or their spam gets automatically detected and put in the moderation queue so a moderator can allow it or spam clean them.

 

[Alpha1]

In my Profile add-on, I don't go so far as to automatically ban the user

Me neither, but getting those in moderation queue is awesome. I explain it in this post:

I embrace that the website and about fields are spam magnets. Its an awesome way to highlight spam registrations and immediately ban those.

Signup abuse detection and blocking

From (simple) multiple accounts detection to isp/connection fingerprinting with score-based moderating/rejecting logic. These are very effective low-hanging fruit at reducing spam. Supports migration configuration from the following XF1 add-ons...

xenforo.com

@Xon has built triggers for this into his Signup Abuse addon for me. It works like a charm. Spammers cannot resist to put spam links and keywords in there. So the registrations go to moderation and then are spam cleaned.
So my staff doesnt have to go hunting for spammers. They will just announce themselves.
Members will never see the accounts. I hide the website field anyway. Only the member itself can see it on profile.

That's true, but it's still plenty of work to deal with them if you have lots of daily registrations.
I've hidden the Website field now with a template edit, but really I would rather people didn't try to spam because they knew it would fail either because there are no fields they can use to spam or their spam gets automatically detected and put in the moderation queue so a moderator can allow it or spam clean them.

Without these fields the same spammers would probably register. But instead the accounts would be used to spam the forum, conversations or other content types. Finding spammers who use conversations or who start posting after a few months, is harder and more work.

My preference is to stop trouble at the door so that members are not confronted with it. In my current setup we only have to check the moderation queue to spam clean the spammers who fall for this.

 

[digitalpoint]

Me neither, but getting those in moderation queue is awesome. I explain it in this post:

Is there ever a situation in your setup where someone posts to a non-existent field and you keep that user? Just wondering because I’m tossing around the idea of just automatically spam cleaning/banning them rather than sending them to moderation queue.

 

[eL_]

How many pro votes this suggestion need to have to be implemented?

 

[digitalpoint]

How many pro votes this suggestion need to have to be implemented?

There is not a certain number that would force developers to add it against their will if that’s what you are asking. It will be added if developers think it’s worthwhile. The number of votes just help it get in front of their eyes for consideration.

 

[eL_]

There is not a certain number that would force developers to add it against their will if that’s what you are asking. It will be added if developers think it’s worthwhile. The number of votes just help it get in front of their eyes for consideration.

Thank you, @digitalpoint

 

[Alpha1]

Is there ever a situation in your setup where someone posts to a non-existent field and you keep that user? Just wondering because I’m tossing around the idea of just automatically spam cleaning/banning them rather than sending them to moderation queue.

I use hidden fields as well. There are a few issues that occur:
Some users use the tab key to go to the next field to fill in and then type.
Users with autofill scripts active. (Chrome)
These will trip such setup. We get occasional support tickets about this. Which probably means there there are more who get denied registration but do not contact support. If I had to guestimate, I'd say users that are caught by hidden fields are 95%+ spam bots.

IMHO hidden fields are a useful layer in a multi-layered approach.

 

[digitalpoint]

Ya, I was more talking about fields that aren't hidden (rather than ones that don't exist in the form they are submitting). For example if I have a setup where "website" isn't a valid field (it's not hidden and doesn't exist anywhere in the form itself). But then a user submits the form with a value for "website". Just tossing around the idea of automatically spam cleaning/banning them without needing to bother to send them to moderation queue.

 

[Max Taxable]

I love your solution Mr. Stuart. Exactly what my inspiration was for the Spaminator products, 12 years ago!

 

[digitalpoint]

I use hidden fields as well. There are a few issues that occur:
Some users use the tab key to go to the next field to fill in and then type.
Users with autofill scripts active. (Chrome)
These will trip such setup. We get occasional support tickets about this. Which probably means there there are more who get denied registration but do not contact support. If I had to guestimate, I'd say users that are caught by hidden fields are 95%+ spam bots.

IMHO hidden fields are a useful layer in a multi-layered approach.

We'll see how it goes...

 

[Alpha1]

If the form has no fields for these 3 (hidden or visible), then how can users submit content for those non-existent fields?

 

[digitalpoint]

If the form has no fields for these 3 (hidden or visible), then how can users submit content for those non-existent fields?

Exactly… some spam bots know what fields should be there normally.

 

[Alpha1]

Ah, yes. Some spam bots also inject content for the expected fields into the custom fields.

 

[Stuart Wright]

Without these fields the same spammers would probably register. But instead the accounts would be used to spam the forum, conversations or other content types. Finding spammers who use conversations or who start posting after a few months, is harder and more work.

My preference is to stop trouble at the door so that members are not confronted with it. In my current setup we only have to check the moderation queue to spam clean the spammers who fall for this.

I totally get what you are saying and I agree that the ideal solution is automatic prevention rather than mitigation once they have registered.
My issue is that we should be using a designed method to detect and prevent spammers that doesn't rely on the use of these legitimate user profile fields.

 

[Forsaken]

Hidden fields that are accessible with tab are a nightmare for screenreaders... Sadly a lot of bots utilize the same method as screenreaders to bypass honeypot fields, which creates a usability nightmare.