Automated no-index of light pages (zero replies, thin content)
- Thread starter cmeinck
- Start date
cmeinck
Well-known member
For those wondering, can Panda hit me, here are some threads from Google's Webmaster forums:
http://productforums.google.com/forum/#!searchin/webmasters/site$20hit$20like$20daniweb$20and$20hitchhiker.$20not$20sure$20what$20animal$20is$20hurting$20me|sort:relevance/webmasters/2J_m5FyfRXo/s_ogMvmr90gJ
http://productforums.google.com/forum/#!topic/webmasters/HiGVM9h2ioE[1-25-false]
This is from Google's John Mueller:
http://productforums.google.com/d/msg/webmasters/T-4i0yB7CrQ/CUil4bGP0XwJ
This quote in particular is important to note:
When it comes to search, what you provide to be indexed is often what users first see when they see your site in the search results. Putting your best foot forward there is something that - imo - isn't just done for the "algorithm," but really primarily for your new users. Sometimes that involves technical measures, other times that involves finding ways to recognize and feature fantastic content created by your community. There's no "one size fits all" solution here, but just putting everything up for search, and hoping that new users consider that perhaps there's more to "a seemingly confusing site" (taking the general UGC site, definitely not specific to yours) and take the time to dig for the treasure, is probably not the best approach.
If we are indexing all of our content, there will be posts that end up indexed, that isn't representative of our sites. For example, let's say someone asks a question in 2012 and it never gets answered. Fast forward to September, 2013. If a user with the same problem finds that thread, there is no answer to their question. They end up back in Google, looking for another site that may have the answer. That makes for a poor user experience, one which I believe Mueller is discussing above.
Ideally, an no-index add-on would do the following:
http://productforums.google.com/forum/#!searchin/webmasters/site$20hit$20like$20daniweb$20and$20hitchhiker.$20not$20sure$20what$20animal$20is$20hurting$20me|sort:relevance/webmasters/2J_m5FyfRXo/s_ogMvmr90gJ
http://productforums.google.com/forum/#!topic/webmasters/HiGVM9h2ioE[1-25-false]
This is from Google's John Mueller:
http://productforums.google.com/d/msg/webmasters/T-4i0yB7CrQ/CUil4bGP0XwJ
This quote in particular is important to note:
When it comes to search, what you provide to be indexed is often what users first see when they see your site in the search results. Putting your best foot forward there is something that - imo - isn't just done for the "algorithm," but really primarily for your new users. Sometimes that involves technical measures, other times that involves finding ways to recognize and feature fantastic content created by your community. There's no "one size fits all" solution here, but just putting everything up for search, and hoping that new users consider that perhaps there's more to "a seemingly confusing site" (taking the general UGC site, definitely not specific to yours) and take the time to dig for the treasure, is probably not the best approach.
If we are indexing all of our content, there will be posts that end up indexed, that isn't representative of our sites. For example, let's say someone asks a question in 2012 and it never gets answered. Fast forward to September, 2013. If a user with the same problem finds that thread, there is no answer to their question. They end up back in Google, looking for another site that may have the answer. That makes for a poor user experience, one which I believe Mueller is discussing above.
Ideally, an no-index add-on would do the following:
- Set posts with 0 replies with X number of days to noindex
- If someone replies, the noindex tag is removed
- Allow for manual 'ignore URL' to allow admins to bypass noindex on informative threads, which may not be open for replies
I think what you want is possible with BD Widget, using it's awesome Expression field.
@xfrocks What you think?
I manage to detect a thread without a rely, or a thread with only 1 reply.
But adding the days condition, might be tricky.
xfrocks
Well-known member
It's possible to check for replies and days but I'm more concern about how to add the noindex meta tag to the header. Ideally, you can use template widget with a template that uses <xen:container /> but that's kinda complicated. Making an add-on to do this is rather simple and it will be much easier to use IMHO.
jdeg
Active member
Thanks for posting links to those posts cmeink. This actually explains a lot in regards to a larger vbulletin site I run. I took a big hit in Nov 2012 just like a lot of those thread posters. I'm a little relieved to hear that I'm not the only one.
But back to the subject - I'd like to see a modification like this as well since I've now moved some of that content to it's own XF site.
But back to the subject - I'd like to see a modification like this as well since I've now moved some of that content to it's own XF site.
cmeinck
Well-known member
Sorry to hear that. It is somewhat of a relief to see other stories. I just hope that changes will be sufficient to get forums that were hit back in the good graces of Google.
DRaver
Active member
Is there anyone here who could program it?
Member Profiles without activities and threats with zero reply should be set to noindex,follow until activities it there.
I do not know why people always take at XenForo noindex without follow. This is certainly not a good solution.
With noindex, follow the internal links will be strengthened.
Member Profiles without activities and threats with zero reply should be set to noindex,follow until activities it there.
I do not know why people always take at XenForo noindex without follow. This is certainly not a good solution.
With noindex, follow the internal links will be strengthened.
Jaxel
Well-known member
I personally think all this SEO stuff is snake oil. These algorithms change all the time, without warning, and entire industries pop up in order figure out how to game the systems. SEO is not going to drive new people to your website; its a waste of time and energy. You can spend hours fixing this stuff, and then google changes it and now you have to start over. If anything needs to be fixed, its google's algorithms, not the websites.
Just stop bothering with this crap and just let it be.
Just stop bothering with this crap and just let it be.
tajhay
Well-known member
Are other people noindexing RSS feeds? Basically i use the feeder options in xenforo to basically add new news stories into a separate forum. 99% of the time noone replies, but its good for ppl to know the news stories occurring around the forum niche. Good idea to noindex that? Especially seeing as its duplicated content?
cmeinck
Well-known member
If you experience a 70% or greater decline in traffic over the span of a year, you have a choice. Either work within the guidelines of Google's algorithms or go out of business. I've heard from more than a few sites, also forums, who have seen similar issues. Google represents a major part of my traffic, so for me, it's important. Making your content and website the best it can be for Google isn't in any way trying to game the system. Having spent the last seven or so months working on this project, user generated content over time can create some very real issues for forum owners. Poorly worded, thin-content being one of them. Thus, I think there is a real need for an add-on which tackles this issue and welcome any developer willing to take it on.
SelfSufficientMe
Well-known member
Sounds more like a problem Google should be solving not us. Why should we be collectively house keeping our sites just because Google can't create an algorithm to sort the husks from the seeds.
I'm all for spring cleaning to make a forum or site less cluttered, more relevant, and a better experience for visitors/users. However, I hate having a gun held to my head by a search engine monopoly who threatens to make my website irrelevant unless I tow the line. Classic example of how too much power breeds bullies.
I hope over time other avenues (like social media) will get better at sending traffic to relevant sites and we can all be less dependant on Google.
I'm all for spring cleaning to make a forum or site less cluttered, more relevant, and a better experience for visitors/users. However, I hate having a gun held to my head by a search engine monopoly who threatens to make my website irrelevant unless I tow the line. Classic example of how too much power breeds bullies.
I hope over time other avenues (like social media) will get better at sending traffic to relevant sites and we can all be less dependant on Google.
cmeinck
Well-known member
They are bullies and unfortunately, they have massive influence. Forums that have been hit by the algo are typically large and likely those with large problems. I believe thin content removal is only one element of recovery. I can say that during my clean-up, I found all sorts of issues that I had no idea existed. For me, clean-up has become a full-time job. I could ignore Google's signals, but that would result in my having to find new employment. Honestly, that might happen regardless, but waiting and ignoring these signals has done nothing but result in a gradual decline in traffic. In my case, I believe the forums and UGC 'thin content' is adversely affecting the ranking of my articles.
Why not set articles with zero replies (after x days) to noindex? It keeps content on your forum, but cleans things up a bit for Google. To me, that seems like a win, win.
To help better understand the implications, I'll share this image which shows traffic from November 2012 through November 2103. I'm in a competitive niche, was the first one there and it remains an incredibly popular topic for forums.
SelfSufficientMe
Well-known member
I guess so, but there could be situations in forums where a single post with no reply is relevant and even content rich. More parameters need to be avail before I would employ such a measure like post size, keywords, date, etc.
Your graft showing a decline in traffic is certainly disappointing, My forums are barely 12 months old and small (slow positive growth but it's hard to decline from zero lol); however, I have a blog which took a sizable hit in traffic at exactly the same time as your example. The fall has stabilised now but I'm not seeing any real growth recovery in visitor numbers as yet. Interestingly, my revenue has not fallen with the numbers and in fact has increased slightly - could it be the people arriving at my site are better targeted? BTW, I don't believe any of my content is "thin" either rather it mustn't be good enough in the critical eyes of Googs to warrant sending me the traffic it was 6 months ago.
My blog isn't quite 3 years old so I haven't been in this game for too long, still, for what my 2 cents is worth, online publishers of forums. blogs, or whatever, need to diversify and optimise not just for search engines but other potential traffic generators equally so not all the eggs are in one basket. As far "cleaning up" goes, it's probably not a bad thing to scan through your own site and do some maintenance on a scheduled basis to make it easier for bots and people to read and that's something I will do from the start but one thing I won't be doing is second guessing Google for the rest of my online career...
ForestForTrees
Well-known member
Thanks for bringing up this important topic, @cmeinck. SEO is absolutely crucial to the growth of my forum, so it's something I take very seriously. The links you provided about Panda were very helpful.
The addon you are requesting doesn't seem like it would be that hard to write. Perhaps you could approach a possible developer directly? Another technique that might work would be using a crowdfunding site to gather cash.
You might be interested in a thread I just started. Personally, I'm just starting to learn about the SEO implications of the noindex meta tag:
http://xenforo.com/community/threads/noindex-meta-tag-on-tag-pages.65656/
I'd be interested in hearing if you've come across any new sources of information since your Sept 16 update.
The addon you are requesting doesn't seem like it would be that hard to write. Perhaps you could approach a possible developer directly? Another technique that might work would be using a crowdfunding site to gather cash.
You might be interested in a thread I just started. Personally, I'm just starting to learn about the SEO implications of the noindex meta tag:
http://xenforo.com/community/threads/noindex-meta-tag-on-tag-pages.65656/
I'd be interested in hearing if you've come across any new sources of information since your Sept 16 update.
cmeinck
Well-known member
@Jake Bunce Is there a way to set noindex meta on specific pages that I've found to have low user engagement?
cmeinck
Well-known member
I've been able to do that for categories and members. What if I wanted to do it for specific thread IDs? For example, let's say I'm able to pinpoint 10 or 20 pages where users are not engaged. I could delete those pages, but I'd much rather retain them and have Google no-index those specific pages.
Jake Bunce
Well-known member
In thread_view:
Code:
<xen:if is="in_array({$thread.thread_id}, array(80,81))">
<xen:container var="$head.robots">
<meta name="robots" content="noindex" /></xen:container>
</xen:if>jdeg
Active member
Thanks for posting! I'm surprised that it was said so directly. He wouldn't have said you should no-index your thin or no response questions if it wasn't something that definitely works for a lot, but not all, of forums. I'm thinking they literally put something in the algorithm in 2012 (when one of my sites started to decline, and a lot of others' sites) that says if the site is a forum, and there are a lot of threads where content is < length and posts = 1, this site's content overall isn't so great. BUT, if the webmaster is aware of this and has a no-index on thin content, lets rank the site based on the index-able content instead.
FYI, I saw a recent small boost in traffic that brought back some of my old keywords. I was surprised at this, and I think it means Google is refining whatever forum specific algorithm changes they made in the past.
