Decline of Forums/UGC in Google

Alfa1 said:
IPS took notice and hired renowned SEO company Yoast for a SEO update and also did a speed update.
Google just keeps on trucking with their new SEO rules. First it was health/medic only. Then money and products. Probably it will roll out further and hit more niches.
Click to expand...
That's pretty cool they implemented both, how does Xenforo compare to these?
 
We don’t currently acknowledge that there are any significant shortcomings in our SEO approach.

It’s all good and well saying “make things better” but without any context as to what that is, it’s difficult to translate that into implementable features.

Most metrics we see, even some of Google’s own Lighthouse metrics specifically rank XF 2.1 as 100% for SEO.

We’ve consistently been ahead of the curve in structured microdata, performance and responsive design.

What’s left? Answers on a post card, please. Or more specifically, specific and actionable suggestions in the correct forum.

Unfortunately the reality is, if Google changes their algorithms there’s realistically very little we can do to manipulate that. If you think a change in code here and a tweak in code there is going to reverse anything, then I’m afraid that’s unlikely. You’re more likely to be at the mercy of Google’s decision making than our feature set.

If there are any specific and actionable ideas of course we will consider them (if posted in the correct place) but I suspect anything people suggest will be microoptimisations at best. Happy to be proved wrong.
 
Chris D said:
Happy to be proved wrong.
Click to expand...
If you are really serious about this, then contact Yoast. Talk to them. They are right around the corner here and I know people working for them. I am certain that a review by them will give you 100 point list of things to improve in XF that includes tweaks and structural changes and it will make a very large difference. Also consider to sit down with Google. Google has done the same with WordPress and others.

XF advertises its SEO on its homepage. You have seen and will be seeing more and more posts from admin that have lost 50-90% of their traffic. Because SEO has changed. IMHO it would be really welcome if XF would get with the times and adjust to the changes.
I honestly think that considering SEO improvements are 'microoptimisations at best' is really underestimating it. I think the opposite is the case: SEO improvements needed are such a massive order that it cannot possibly fit in one release.

If you think SEO updates are trivial, then consider that Google now holds specific niches to higher SEO standards. i.e. medical and financial sites can get thrown out of google's index for SEO issues that do not matter to other sites. My XF1 big board scores 100 for SEO on Lighthouse. Yet, Google punishes us for various SEO issues and Google Search Console lists various errors.

Chris D said:
We don’t currently acknowledge that there are any significant shortcomings in our SEO approach.
Click to expand...
Consider that there is not much difference anymore between SEO and UXD. Or SEO and abusive content moderation. I have been beating the drum on this for years now. Nor is there much difference between speed and SEO. You mentioned lighthouse audit by Google which gives 45 out of 100 for speed on this very thread. 74 for accessibility and 80 for best practices. That gives significant room for improvements.

I did post various bug reports, suggestions, the links above to what IPS has done. I also posted a lengthy document by google in my suggestion about abusive content moderation which exposes a lot of area's of improvement for which functionality can be added.

Thin and scattered Content
Thin content is an issue with XF. If you have addons that use the same approach as the RM then you get a lot of pages without content. Example:
xenforo.com

xF1-Espanol-(ESP)-Resource Manager [Deleted]

elvamo submitted a new resource: xF-Espanol-(ESP)-Resource Manager - xF, Resource Manager, 1.2.4, Español: --------------------------------- Traducción de este portal, para poder usarlo necesitas descargar el addon de su respectivo dueño. English: ---------------------------------...
xenforo.com xenforo.com
At the time of writing this page has only 2 automatically generated posts. There is nothing of value in that page. The sole reason this page exists is because non-forum content does not display in new-posts. This is essentially a UXD issue that has SEO impact.

XF & XFRM modeled addons have to create a lot of pages that have no content.
There is no functionality in XF to only expose rich content.

There are other UXD issues you are well aware of that impact SEO. The way addons are integrated with core is more of a bolted on approach than tightly integrated. There is no central area per topic. People have to click on dozens of tabs to get anywhere. To make that more clear: lets say you have a gaming site, with categories for:
  1. fortnite
  2. grand theft auto
  3. spiderman
So if a user wants to access threads, resources, media about fortnite then they need to first go to the relevant content type and then to the relevant content type. If there are addons then they need to find it among a dozen navigation tabs while there is only room for a few tabs on a phone. This approach doesn't work anymore in a mobile age. Especially not if the site covers a lot of topics.
What would be good SEO-wise as well as UXD is to centralize topics, so that all threads, resources, media and addon content about fortnite is accessible from one location, where there is no content about GTA or Spiderman.
Solve problem of overly complex forum structure and users not finding the right forum

Just like people do, Google wants to see fresh content. Give Google one page to find all new content:
'All New Content' tab on find-new (new posts)
Better than a tab would be just one page with filters.

Language Issues
The absence of language functionality causes Google to not only ignore content in different languages, but also punish such sites as the content in different languages basically becomes bad content that sabotages the overall quality.
Define language per node
Language Nodes
Separate Tags in different languages
Use hreflang markup

What it does to tagging is really problematic as well. For example: lets say xenforo.com adds a German forum and the German customers start adding German tags. Then the English customers accidentally add German tags because words are similar and XF has autocomplete for tags. Consider what that would do for SEO.

XF does not have functionality to handle dead links, 404s and redirects for such. Yet this is a massive issue for forums. Especially sites that exist for decades.
Reports for dead links through Google Search Console API

Microdata
In regards to microdata there is plenty to be done. But you really need to talk to an expert about this, because otherwise I may miscommunicate it. What the Google document above shows is that there is a need for an 'about us' page with the appropriate microdata. A lot more microdata can be added to display more in google search results. For example use structured data for rich cards or display posts and other rich snippets:
xenforo.com

Lack of interest - Follow permalink posts - google snippets

I removed nofollow from the permalink and after that i got the amount of posts and authors very quickly in search. And also the nice snippets below you can see on mobile (Meer=More=post). When i added nofollow these were removed. So i suggest leaving 1 #post url per post because it could be...
xenforo.com xenforo.com

Speed
In regards to speed improvements: I am sure that you are well aware of things that can be done and are probably planning this. You already did a lot in XF2.0 and xf 2.1. I'm hoping that we will see a lot more in 2.2. Personally I am hoping to see integration with CloudFlare caching happen.
Implement Lazy Loading...
Log Slow Queries in XF

Mobile First
In regards to mobile first, there are top voted suggestions like AMP, Progressive Web App.

Content Quality
Content is king. Yet, XF does not yet have adequate means to identify thin content, abusive content, low quality, copied content and then manage it. Various governments are pushing Google to crack down on bad content. So the content quality review does not come as a complete surprise.
Thread Reviews: Quality Control & SEO.
Better functionality to report and moderate illegal hate speech
Post Checking System - a hugely important piece of functionality
Automatically moderate, report or delete content rated with specific Reactions
Make post hide after X number of report
Moderation - users can select a recommended action

Not only does XF need functionality to identify and get rid of problematic content, it also needs ways to identify, expose, highlight and surface quality content. Weighted reactions is one step closer to identifying such content.

Social Media
SMO is an area that has become important for SEO. The google document above makes that clear when it comes to EAT.
I believe there is a suggestion to auto-post quality content to Social Media.
Another important function would be to get your members to post reviews to social media. Recommend site on Facebook.

Media Gallery
Having this addressed in the media gallery would be nice:
Google reports lacking canonical tag on /media/1234/preview
Fix 'Duplicate, Google chose different canonical than user' for media with same name
Use full url for comment permalink in datetime
Review structured data in CreativeWork

The media gallery on my site and on many others I have reviewed result in a major mess. Not much content gets categorized well. File names are often gibberish. People don't know where to post (albums or categories) or where to find relevant content. An Instagram like setup with topics to follow and collections to compile would be much more effective. The unstructured mess is not good for SEO. Its really great that XF 2.1 has added video upload to threads. Now members will just discard the gallery. What would be great is if there would be auto-categorization. Reuse of media uploaded to threads by putting it into media categories and albums. So that media is really integrated with forum.

For example, if I post in a forum about the game GTA , then the media that I add could be added to my album and the GTA media category, And conversely when I browse threads in the GTA forum, then display media that has been uploaded to the GTA gallery category.
Sidebar management and automatic sidebar content creation
Similar Content Widgets (for any content type)

Centralizing content based on topic and tightly integrating content types to form rich content is a much better approach than bolting on content types. At least IMHO.

Adding image alt tags would be nice:
Add a title + alt option to images


In regards to centralizing content: the same goes for the resource manager and content types from other addons.
Resource updates and support url should be disabled if the function is not used, because its irrelevant content. if you use the XFRM for documents or other things that do not have a url, then people will just enter irrelevant links there.
Option to disable Resource Updates
Category option: Disable 'Support URL'
Do not state a Resource Update timestamp, when there has never been an update.

Content filtering is another thing that would help users access the content they need. And making the filtering results accessible to search engine bots. Because that exposes grouped related content to bots.
Access to "faceted" search please!

As GDPR name changes do not apply to quotes and mentions, users report the site to google which removes such pages from its index.
Apply Name Change to quotes & user mentions (GDPR)

Sitemaps can still contain content that is blocked trough robots.txt It should be easy to add a check and exclude such content from sitemaps.

Error pages can should display why the user cannot access the page and give options about what to do next. I reported this 8 years ago because error messages are still the same as forums had in the previous century:
Tell users why they do not have permission to access a page.
Please check the google document to see their examples of bad error pages and good error pages. Consider that a XF big board has its error page hit a massive number of times per month.

As you know I can go on and on about improvements and much has already been posted in the suggestions forums.
While there is a mass of things to improve, I do want to acknowledge the hard work that the XF team is putting into XF. The fact that the internet and Google is changing rapidly and this has major impact on our sites, poses a significant challenge for both admins and XF to get ahead of the curve again.
 
At first glance the issues you raise aren’t exactly unique to this forum software or most website platforms in general.

There's a good portion of what you've mentioned that, despite your assertions, will have absolutely zero impact on whether Google will index your pages or not. I'd actually argue that most of those things, while nice to have and definitely targets for the future, will have little to no impact at all on SEO.

But what I mostly take exception to is people being “wowed” by “innovation” in other forum software and treating us like we’re some sort of underdog.

It’s great that IPS worked closely with Yoast there. But let’s be real, most of the stuff in the pages you linked to is stuff we already have. Specifically the highest impact ones that are mentioned are:
  • Pagination - we've had this since XF 1.x and made it even easier to implement in XF 2.x.
  • Canonical Tags - we've had this since XF 1.x.
  • Improving non-image attachments - we've had this since XF 1.x.
  • All the letters - since these were implemented in XF 2.0 they were already implemented in the most performant way possible - they are 100% pure HTML and CSS
If that's all Yoast had to recommend to IPS then the majority of the stuff you highlighted likely doesn't apply in the opinion of Yoast, because IPS has a bunch of those issues you highlight too.

And similarly, although we might well consider many of the suggestions you highlighted, I think people will be sorely disappointed to see that, when/if we do add them, they'll have very little impact.

Not all Google search algorithm change impacts can be reversed with changes such as the ones you highlighted; that's my primary point.

We'll keep working towards as many improvements as we think are necessary, of course, but sometimes when Google changes things, it is the way it is and not anyone can do a thing about it.
 
There are a ton of things admins can do themselves with editing the style if they become educated in SEO. It's not all backend issues.
 
Chris D said:
But what I mostly take exception to is people being “wowed” by “innovation” in other forum software and treating us like we’re some sort of underdog.
Click to expand...
That's not the perception that I want to convey at all. In fact the very reason I choose XF over IPS was their past decisions over SEO. And XF2.1 has certainly wowed me.
There is however an issue with SEO changes which has very significant impact on our sites. Loosing millions of users due to higher SEO standards leaves us scrambling about the way to go for the future. XF2? other forum software? WordPress? Stick to XF1 until we sort the SEO issues? I dont know. Maybe even call it quits?

My reason for my reply is that I am hoping that the XF team is interested to review the possibilities for improvement in this area. An effort in this regard is something that I would consider of very high value and appeal.
 
Chris D said:
But what I mostly take exception to is people being “wowed” by “innovation” in other forum software and treating us like we’re some sort of underdog.
Click to expand...
XF is not an underdog, but the notion of a forum platform is industry wide becoming an underdog. It's been pretty obvious for the past several years that forum platforms are under threat and declining in use. We don't want to become like the old usenet news groups. This is the pressure we feel as admins when we see numbers generally decline because of not keeping pace with other platform types, changes in technology and how people use the internet in general. That is the heart of our anxiety.

For me mobile usability is everything. Kids these days barely use laptops, if only to write a paper. That generational impact will be felt more quickly than we think. Whole countries now are basically mobile first and people are communicating in a twitter style. The age of long form messaging is dying.
 
Alfa1 said:
That's not the perception that I want to convey at all. In fact the very reason I choose XF over IPS was their past decisions over SEO. And XF2.1 has certainly wowed me.
There is however an issue with SEO changes which has very significant impact on our sites. Loosing millions of users due to higher SEO standards leaves us scrambling about the way to go for the future. XF2? other forum software? WordPress? Stick to XF1 until we sort the SEO issues? I dont know. Maybe even call it quits?

My reason for my reply is that I am hoping that the XF team is interested to review the possibilities for improvement in this area. An effort in this regard is something that I would consider of very high value and appeal.
Click to expand...
But the whole point of this thread was to highlight that a competitor has “taken notice” implying that we haven’t.

Well, all they did is implement a bunch of things we already have, because those things are the highest impact to SEO.


My main point is, we will of course listen to suggestions and trends in that area. The comments should be made where relevant though too with specific context, e.g. specific suggestions about specific SEO features, and not random threads in the OT forum that don’t really relate to anything specific.

dethfire said:
XF is not an underdog, but the notion of a forum platform is industry wide becoming an underdog. It's been pretty obvious for the past several years that forum platforms are under threat and declining in use. We don't want to become like the old usenet news groups. This is the pressure we feel as admins when we see numbers generally decline because of not keeping pace with other platform types, changes in technology and how people use the internet in general. That is the heart of our anxiety.

For me mobile usability is everything. Kids these days barely use laptops, if only to write a paper. That generational impact will be felt more quickly than we think. Whole countries now are basically mobile first and people are communicating in a twitter style. The age of long form messaging is dying.
Click to expand...
If anything mobile usability is likely to have more impact than any micro SEO optimisation can be thought up. And I don’t mean it from the context of SEO either. I think in terms of SEO our mobile usability should be classed as sufficient in terms of being “mobile first” etc.

Obviously there’s more to do there. PWA is definitely something to keep an eye on. But rest assured the driver for doing that is a better user experience and that’s what will have the impact. Any impact on search rankings as a result of such things would be minor at best.
 
Chris D said:
If anything mobile usability is likely to have more impact than any micro SEO optimisation can be thought up. And I don’t mean it from the context of SEO either. I think in terms of SEO our mobile usability should be classed as sufficient in terms of being “mobile first” etc.

Obviously there’s more to do there. PWA is definitely something to keep an eye on. But rest assured the driver for doing that is a better user experience and that’s what will have the impact. Any impact on search rankings as a result of such things would be minor at best.
Click to expand...
My reply was not really in response to SEO although, it wouldn't hurt to reach out to a top SEO firm like Yoast and get consult because it is important and you guys are top shelf in programming and etc, but you can't be expert in everything.

And furthermore my response above was not really being critical of XF. We're in this together. It's really really tough to figure out where things are headed but a vision must be set or we'll be playing catch up and losing ground here on out.
 
Chris D said:
But the whole point of this thread was to highlight that a competitor has “taken notice” implying that we haven’t.
Click to expand...

We're definitely not saying you haven't Chris, far form it. XF might still be doing great or industry leading in terms of SEO, but audits from those that live and breath that stuff day in and day out could shed light and highlight even small tweaks that could improve it further. SEO is one of the most complex and fast changing aspects that effect every website and community currently on the web.

Yoast has been controversial in terms of bloat and other issues in the WP add on that made them famous, but the fact that IPB went out and sought outsider help and feedback from experts in the field definitely demonstrates willingness to learn and improve (and spend money on experts to gain this knowledge quickly to implement it into their product) based on feedback from those external audits. It's a great selling point, even if it's a display of confirmation of what's already implemented and being on the right track. It might not be a bad idea to show willingness in the same vein, then digest and implement the feedback that you see fit to ensure your product is still in the top tier in that regard.

Heck, it might even demonstrate the strength of this platform compared to wordpress and their analogous add on ecosystem if they decide it's worth their time and investment to release a paid 3rd party add on to enhance the SEO potential of forums and online communities even further past their current WP offering.

If XF is to continue to be a power house and continued leader in it's space then perhaps information from data backed SEO audits, security audits and any other kind of specialty featured audits could bring to light and highlight anything else that could be improved or needs attention in this space and make the platform even more appealing.

Sometimes it's not about exactly what the small team has done incredibly well together in silence behind the scenes, but the perception to it's current and potential customers that you continue to do everything within you're power to ensure that you're still leaders in the spotlight.

You might not actually need the audit, SEO, security, or otherwise. IPB might not have needed it either, and they perhaps have figured it out themselves, but it's great marketing either way and has the potential to instil a great amount of confidence in current and potential customers when considering the platform for their next community.
 
Last edited:
dethfire said:
We don't want to become like the old usenet news groups.
[...]
The age of long form messaging is dying.
Click to expand...
Ultimately, this is what will happen and unless this usage pattern changes again there pretty much isn't anything we as Forum owners can do.
 
RobinHood said:
We're definitely not saying you haven't Chris, far form it. XF might still be doing great or industry leading in terms of SEO, but audits from those that live and breath that stuff day in and day out could shed light and highlight even small tweaks that could improve it further. SEO is one of the most complex and fast changing aspects that effect every website and community currently on the web.
Click to expand...
There could certainly be structural items to optimize but I'll reiterate that admins can do A LOT themselves with their styles and templates.
 

Similar threads

Sal Collaziano
  • Question Question
XF 2.2 New reason prevents pages in a sitemap from being indexed on Google
Replies
5
Views
112
Miri
M
R
Errors in Google Search Console
Replies
2
Views
62
Rycky
R
I
  • Question Question
XF 2.3 Curious issues with google indexing of 'Whatsnew Posts' page
Replies
1
Views
43
ForumDevs
ForumDevs
Leeroy Brown
Marking forums thread titles, posts as rel="ugc"
Replies
0
Views
470
Leeroy Brown
Leeroy Brown
rrlevering
  • Suggestion Suggestion
Implemented Markup improvements for Google
6 7 8
Replies
141
Views
19K
rrlevering
rrlevering
Back
Top Bottom