Decline of Forums/UGC in Google

[nrep]

I know that many large site owners here will have seen a drop in search engine referral traffic, particularly starting in November 2012. It seems to be a common story for large UGC sites, using all sorts of different forum software. I'm sure many sites weren't affected (or benefited) - so this thread isn't going to apply to all forum owners here.

I was wondering how people who had sites that were hit are fairing now? Did you do anything to help the recovery? What worked and didn't work?

I've spend a lot of time listening the comments by John Mueller (Google Employee) regarding UGC, and tried some of them out to disastrous effect. He suggested no-indexing threads with questions and no answers, low quality off-topic threads, no-indexing new threads until moderated, etc.... I didn't think it made much sense at the time, but this is someone working on the team that deployed this update - so I listened to it and tried their suggestions. Traffic declined even more, so we just reversed the changes after 6 months.

It seems that Google does not think highly of forum content in general and now prefers large established Q&A sites like StackExchange (and their sub sites). There is no doubt that SE is a good site, but high-quality forums definitely have plenty of worth.

P.S - @dethfire I've tagged you in this, as I spotted your name on another forum where this problem was discussed a few years ago - so thought you may be interested in the results.

 

[Mr Lucky]

no-indexing new threads until moderated, etc....

Surely threads waiting for moderation would not be indexed anyway? (unless one of your moderators is Google)

 

[nrep]

Surely threads waiting for moderation would not be indexed anyway? (unless one of your moderators is Google)

Sorry, bad wording on my part. No-indexed until a moderator has likely seen the thread and know's it's ok - not part of the moderation queue. i.e. wait 24h after new thread is posted then allow indexing (as if it's still there, it's a legit thread). Not something I think is a good idea, but it was a suggestion from a Google representative for forum content.

 

[Mr Lucky]

Sorry, bad wording on my part. No-indexed until a moderator has likely seen the thread and know's it's ok - not part of the moderation queue.

Ok, so theoretically everything is no-indexed on creation, even if visible to all groups and guests. Mods have ability to un-noindex - presumably via an addon such as @Rivmedia https://seoforums.uk/resources/rivmedia-seo-meta-xf2.1/

 

[Alpha1]

Basically what is needed is an evaluation of thread content quality. For large forums its not possible to have moderators do that. Members need to do so.
Then delete or noindex threads with low quality. Suggestion here:
https://xenforo.com/community/threads/thread-reviews-quality-control-seo.152696/

There also is the issue of toxic content. Suggestion and discussion here:
https://xenforo.com/community/threa...port-and-moderate-illegal-hate-speech.127828/

 

[Mr Lucky]

At one stage I think I deleted all threads with one post.

I also use @AndyB addon similar threads which encourage's people not to start a new thread on same topic which means there are fewer but more contenty threads on that topic. It also flags threads that can merged to achieve the same end (also helps with duplicate content and tighter forum search results)

 

[Alpha1]

That addon slowed down my site. I am using Xon's enhanced search addon which has similar threads.

It also flags threads that can merged to achieve the same end

Is this a moderators function?

At one stage I think I deleted all threads with one post.

I've done it as well but you do loose high quality essays that way. Quantity and quality are related but different things.

 

[Mr Lucky]

That addon slowed down my site.

No such problem for me.

Is this a moderators function?

Not automatic, it just gives you an indication to check and merge manually.

I've done it as well but you do loose high quality essays that way. Quantity and quality are related but different things.

Yes, some manual checking is a good idea first.

 

[RobinHood]

The index and no index attribute could be linked to post ratings and change from no index to index if it reaches a certain reaction or like count.

This could perhaps be first post only, or cumulative for all thread posts depending on the thread or forum style.

I believe this is how reddit handles it, with threads only getting indexed if they receive a certain number of upvotes

 

[Alpha1]

Post ratings on the first post do not translate to thread ratings. A stupid question can lead to great answers and conversely.
Reddit counts downvotes as well as upvotes. Without negative ratings you cannot measure quality. Likes just don't work for content quality.

 

[RobinHood]

I agree, that’s why I said certain reactions, for those they have multiple ratings of different sentiment

 

[dethfire]

There was a documented Google algorithm update in late 2014 that devalued UGC big time.

Google has a hard time understanding what is quality and what isn't. That is precisely why up/down vote sites like StackExchange has done so well. Google trusts it because there is a quality mechanism. Q&A is much easier to frame. Here is the question, here is the best answer. Done. Next. Forums with wandering 100 page threads is rough. How the hell is anyone supposed to evaluate that much less a computer algorithm. Google doesn't trust forums. There are too many bad ones.

No forum software I've seen is keeping with the times of SEO (I do SEO for a living btw). Part of that is not their fault, it's time consuming and they are programmers, not marketers. I'd say Xenforo is a good 5-6 years behind SEO best practices. It's especially hard with large forums who need tools to control content better. At a min we badly need something like a Yoast plugin.

 

[Neutral Singh]

Xenforo is a good 5-6 years behind SEO best practices.

care to explain!

 

[dethfire]

care to explain!

Off the top of my head, and granted I'm still on 1.5, XF gives no index controls. This is a massive issue for big board. If XF thinks it's okay to dump 500k thin content member pages, tag pages and threads with no replies then that is shocking. Furthermore, these pages could actually offer valuable content, but they are designed poorly, so 95% don't.

 

[RobinHood]

Some Yoast like features would be welcomed. You can’t even do basic stuff such as set the URL slug different to the thread title, the SEO title different to the thread title (and therefore the threads’s H1 tag), set a meta description different from displayed the on page forum description.

This isn’t just for threads, this is for subforum titles and indexes.

You don’t want every single sub forum index to necessarily display the word ‘Forum’ in the displayed H1 title, as it looks ludicrous and repetitive at when listed on the forum index, yet you may want it included on the sub forum itself for when users land on it, or only in the underlying code so it appears that way in SERPS so when a user searches for a topic that you have a subforum dedicated to it actually shows up in a nice readable and descriptive format encouraging search engine uses to click.

Considering SEO is one of the biggest selling points for forums compared to all the other social network and groups options available these days, it does seem incredibly poorly addressed.

 

[Mr Lucky]

I just had a look at my two main sites stats over the years since Google first rolled out Panda. The first is my wordpress site https://TamingtheSaxophone.com, the second is my forum https://cafesaxophone.com (which migrated from vbulletin to xenforo in Jan 2015

Tamingthesaxophone (wordpress):

(The SEO work coincided with switching content from a html static site to Wordpress) Most of the SEO was content improvement as opposed to backlink strategy and I found Yoast very useful for this). Along with the wordpress also the site became mobile friendly.


Cafesaxophone (vbulletin to xenforo):

As you can see, the Google algorhythm updates impacted the wordpress site (as did my SEO work) but had more or less no effect on the forum. Nor did xenforo it seems.

Forum SEO has obviously been way less as there isn't much that can be done. I think what SEO efforts I have done on the forum may have stopped its decline a bit due to the normal things, ie FB and increased competition in the niche.

 

[RobinHood]

Forum SEO has obviously been way less as there isn't much that can be done.

Yeah, there's so much room for improvement and better forum SEO tools, which evident from the success from your Wordpress site which is highly optimised for SEO out of the box and has a lot of plugins to help users optimise it further.

From the basic stuff such as offering the ability to customise various important forum pages for SERPS, to better schema markup to classify and rate posts or threads, which can then result increased visual impact in serps, such as stars or simply better titles and descriptions, thereby providing a better CTA to click. Better control of follow and no follow link distribution based on thread and post quality scores.

The ability to better structure forum posts, especially long informative 1st posts in threads so that you can use H2, H3 tags etc. to make them more article like and give the posts better content structure for indexing.

The ability to optimise images from some kind of attachment manager so you can edit the actual file name of images to make them more descriptive and add alt text to them. This may enable more of your original attachment content to get better indexed and show up in SERPs as it's own item.

Better built in media management tools to store and optimise uploaded attachments in conjunction with the original media in order to improve page load times. There's loads of wordpress plugins that do this, such as https://imagify.io

 

[nrep]

Some really interesting points here, thank you all for chiming in with your thoughts and ideas .

Interestingly, it doesn't look like you've had much turbulence in traffic on your forum, @Mr Lucky. Is that graph showing search engine referrals only, or does it include regulars? My traffic drops off a cliff (40%) after some of the updates.

There was a documented Google algorithm update in late 2014 that devalued UGC big time.

Do you happen to know which update this was? There's a great list of updates here (https://moz.com/google-algorithm-change), but I can't see one that mentions UGC - I'd be really interested to know which one it may have been if you can remember, as it'll give me a chance to read up on what happened.

 

[Mr Lucky]

Interestingly, it doesn't look like you've had much turbulence in traffic on your forum, @Mr Lucky. Is that graph showing search engine referrals only, or does it include regulars? My traffic drops off a cliff (40%) after some of the updates.

That's just the Audience overview (all users) so I presume it shows every visit.

I found it interesting over the same period the other site is up and down, but the forum has stayed very steady. Like a lot of forums we tend to have a lot more guests online than members, about 12:1.

I think for me it's a good argument for improving SEO for the forum.

 

[Amphicar770]

My site, www.amphicar770.com is the largest source of information related to Amphicar's on the planet. For many years, it consistently showed up in the top 3 for Google on a search for Amphicar. These days it is somewhere around page 15 after just about anything (commercial) with the word Amphicar in it. Clearly, the SEO is no longer working. Is this just with Xenforo?