Automated no-index of light pages (zero replies, thin content)

If anyone needed more confirmation that we need to something collectively as forum owners, this post from Google's John Mueller speaks volumes. I'd be willing to help fund a plugin if someone is willing and able.

"Another thing you didn't mention, but which is common with forums is low-quality user-generated content. If you have ways of recognizing this kind of content, and blocking it from indexing, it can make it much easier for algorithms to review the overall quality of your website."
 
cmeinck said:
If anyone needed more confirmation that we need to something collectively as forum owners, this post from Google's John Mueller speaks volumes. I'd be willing to help fund a plugin if someone is willing and able.

"Another thing you didn't mention, but which is common with forums is low-quality user-generated content. If you have ways of recognizing this kind of content, and blocking it from indexing, it can make it much easier for algorithms to review the overall quality of your website."
Click to expand...
Also, that's very direct!
 
@cmeinck I'd be willing to help fund an add-on. If we can specify it's exact functionality maybe we can then get a quote and crowdsource it's development?
 
Live Free said:
@cmeinck I'd be willing to help fund an add-on. If we can specify it's exact functionality maybe we can then get a quote and crowdsource it's development?
Click to expand...

We should look @jdeg has done with his for vBulletin. There is another one which isn't supported, but worth reviewing the feature set. Is there a place on XenForo to recruit developers?
 
I would also be interested to help fund something like this. Having seen a site suffer under Panda, and then recover after dealing with thin content I know how important it is.
 
Andre Daub said:
I am interested too.

Would be perfect if it works with the new sitemap feature Form 1.4. Putting noindex to Threads with 0 replies and remove them from the sitemap if included.

Interesting Link: http://www.webmasterworld.com/google/4641876-2-30.htm

How do you handle OffTopic Forums? Did you make them not visible for guests?
Click to expand...

I use @Jake Bunce code for no-indexing threads in Off Topic.

Getting back to that topic, I'm wondering how I would go about no-indexing pages within that forum. Google still indexes Off Topic Page 1,2, etc.

Thoughts?
 
Live Free said:
Do we know that there's any real benefit from no-indexing thin content? So having thin content hurts quality content on the same url?
Click to expand...
I doubt there is any concrete proof, but a there is a lot of anecdotal circumstantial evidence. After getting rid of thin content a site I had that suffered from Panda made a reasonable recovery.

Until then I had assumed that only pages were hit by algorithm changes, not whole sites, but I now think differently.

As no indexing thin content wouldn't do any harm, it would make sense to me that being able to do it would be a useful option, hard evidence or not.
 
I would also be happy to contribute towards the development of an add-on to noindex thin content. Something along the lines of - IF there are less than x posts in a thread AND the thread is over x days old AND the total characters in the thread is < x THEN add noindex.

I think the character count would be useful as it would prevent high quality threads that don't contain many posts from being noindexed. For instance we have a number of stickies that provide a lot of information but are locked with one post to avoid clutter.

I don't know anything about developing add-ons but perhaps it would be possible to have a cron task that runs daily to check for threads that may fit the criteria, then a database table with a list of the noindexed threads. I suppose there would also need to be a way of checking existing threads that have already been noindexed in case further content gets added.

@Chris D don't suppose this would interest you at all?

BTW would like to say thank you to @Jake Bunce for his code that we use to noindex our entire general chat section, along with member profiles.
 
TW89 said:
Interesting, thanks for the update. Looks like there are plenty of options to customise the way it works which is great.
Click to expand...

I had a few additional requests that did not make it to the final build. Still, once it's working properly, it can be incredibly useful. Right now, I'm concerned as it seems to be inefficient at determining word count for a thread. @******* has been responsive and working on getting it sorted out.

It seems to work great on threads with zero replies. Once a reply is detected, it removes the no-index.
 
cmeinck said:
I had a few additional requests that did not make it to the final build. Still, once it's working properly, it can be incredibly useful. Right now, I'm concerned as it seems to be inefficient at determining word count for a thread. @******* has been responsive and working on getting it sorted out.

It seems to work great on threads with zero replies. Once a reply is detected, it removes the no-index.
Click to expand...
When I first read that the default was noindex I was a bit surprised but that makes a lot of sense actually.

I'm still a little cautious as to whether to actually go ahead and noindex all the thin content because there's got to be a risk of losing traffic but at the same time I think that many people underestimate the risk of leaving it in place for Google to index and so I'll almost certainly be giving it a go.

Word count is quite an important feature IMO so I'll probably hang on until that's sorted but I'll purchase it then. Thanks :)
 
My issues with the add-on were addressed with the latest release. It had been counting words in the first post and not the entire page of a thread. This latest release seems to do just that. I'd recommend caution when using this, as it can seriously impact your site based on your settings. You may have some pages that are performing well, but are set to noindex based on the parameters you set. Good luck everyone!
 
@cmeinck, I've enjoyed reading your posts about this for literally over a year, and I think they deserve to be seen more widely. I don't have time to go through the process of investigating it right now, and I suspect I'm not alone in this. Is there any chance that you would put together a guide, tying everything together? I don't think it's much harder than starting a thread, and it might help some people.

https://xenforo.com/community/resources/categories/xenforo-tips-and-guides.34/
 
ForestForTrees said:
@cmeinck, I've enjoyed reading your posts about this for literally over a year, and I think they deserve to be seen more widely. I don't have time to go through the process of investigating it right now, and I suspect I'm not alone in this. Is there any chance that you would put together a guide, tying everything together? I don't think it's much harder than starting a thread, and it might help some people.

https://xenforo.com/community/resources/categories/xenforo-tips-and-guides.34/
Click to expand...

Thanks, I appreciate the kind words. I've done an extensive amount of research and changes over the past few years. I do feel strongly that sites should actively monitor what gets indexed and there are so many things you can do to improve the quality of your forums. That being said, I've yet to experience a great deal of search engine success as a result of changes I've made. I think results are something which is necessary for me to be sufficiently qualified to offer a guide on the subject. If you have any specific questions, feel free to send me a PM and I'd be happy to help as best I can.
 
You must log in or register to reply here.

Similar threads

LaxmiSathy
  • Question Question
No indexing thin content forum pages - Before or After Migration
Replies
1
Views
750
Chris D
Chris D
Top Bottom