noindex meta tag on tag pages

[ForestForTrees]

Hi everyone,

I was surprised to learn recently that many people recommend using the noindex meta tag on tag pages like this one: https://xfrocks.com/tags/widget+framework/

It surprised me because one of the reasons that I think tags are so important is that I have been amazed by the SEO performance of a WordPress site I manage. The best explanation I could find was the extensive tagging built into WordPress. I think of WP essentially having four tagging systems built in: "tag," category, date, and author, and this must provide terrific anchor text and internal linking.

However, some people seem to believe that tag pages should be marked "noindex." The best source is a study published in Search Engine Journal in 2012, where they saw a 20%+ increase in organic traffic after setting tag pages to noindex and also optimizing post pagination. A blogger reported a 300% increase in traffic from noindexing the tags page. A Moz article on setting Wordpress up for SEO success also recommended it. Several forum threads also discuss it: MozQ&A1, MozQ&A2, StackExchangeWebmaster.

I'd be curious to hear what others think of this. Because XenTag doesn't use snippets, it may avoid a duplicate content problem that WordPress has with tag pages. (Most of the best practices posts about tagging focus on WordPress.) However, if duplicate content problems were the issue with WordPress, why aren't people advocating decreasing the size of the snippets rather than using noindex?

I'm a big fan of Xentag. I wonder if it would be hard to add a "noindexed tag pages" option to XenTag. Setting aside all other concerns, it is easy to argue that tag pages make bad landing pages for organic searches, so admins might want to noindex them anyway.

Interested to hear people's thoughts...

 

[Alpha1]

What do you mean that XenTags does not use snippets? Do you mean rich snippets or thread previews?

 

[ForestForTrees]

What do you mean that XenTags does not use snippets? Do you mean rich snippets or thread previews?

I should have been more precise. XenTag does use snippets, but in my experience, they are so very short that they wouldn't cause duplicate content problems the way that WordPress tags might. Briefly reviewing a WordPress site I manage, the software seems to be including entire posts, much more than the around 150 characters that XenTag includes.

Thanks for the correction.

 

[Carlos]

It surprised me because one of the reasons that I think tags are so important is that I have been amazed by the SEO performance of a WordPress site I manage. The best explanation I could find was the extensive tagging built into WordPress. I think of WP essentially having four tagging systems built in: "tag," category, date, and author, and this must provide terrific anchor text and internal linking.

I agree. I tried to empathise the importance of meta tags/tags (in general they're petty much the same thing), and I found that if you run a niche-traffic website (like Call of Duty for example) you can eat the proverbial pie set by the niche. I've already done this on a few websites thus far, one of my most successful sites took off and ate 5-10% of the market. If xenForo could do that and then some more, I'd be happy.

But no matter how hard I try to state my case to anyone here at xenForo.. my notions or my experience to them just goes through from one of their ears to the other.

I'm about to launch another like-minded blog next year (or two) in preparation of that particular niche... and I'm going to try and outdo my old site.

However, some people seem to believe that tag pages should be marked "noindex." The best source is a study published in Search Engine Journal in 2012, where they saw a 20%+ increase in organic traffic after setting tag pages to noindex and also optimizing post pagination. A blogger reported a 300% increase in traffic from noindexing the tags page. A Moz article on setting Wordpress up for SEO success also recommended it. Several forum threads also discuss it: MozQ&A1, MozQ&A2, StackExchangeWebmaster.

I've always thought that "noindex" doesn't have an effect on anything with SEO. I always thought that's only reserved for being afraid of getting penalized for having spam and whatnot.

 

[Alpha1]

I should have been more precise. XenTag does use snippets, but in my experience, they are so very short that they wouldn't cause duplicate content problems the way that WordPress tags might. Briefly reviewing a WordPress site I manage, the software seems to be including entire posts, much more than the around 150 characters that XenTag includes.

Thanks for the correction.

I see that you mean thread previews, not rich snippets ( microdata tags)
I do not think there is any problem with including thread preview snippets. As you say these are short. I have had marvelous results with thread tags, including short thread previews.
It's easy to imagine why long previews can cause duplicate content issues. However, that should be solved with the correct microdata tags.

 

[Carlos]

I should have been more precise. XenTag does use snippets, but in my experience, they are so very short that they wouldn't cause duplicate content problems the way that WordPress tags might. Briefly reviewing a WordPress site I manage, the software seems to be including entire posts, much more than the around 150 characters that XenTag includes.

Thanks for the correction.

Then if this is a problem on wordpress for you, I guess you should install All in One to fix those duplicate posts 'problem' since it allows you to edit each wordpress post(s) to include your own snippet.

 

[ForestForTrees]

Thanks for your feedback. Like I said, I'm really amazed at how well our WordPress site does SEO-wise. I only brought up duplicate content issues because I was surprised at why so many people would recommend using meta noindex on tag pages for WordPress. We do in fact use All in One, and don't seem to be having any problems on our own site. The real question is, why does it seem like several smart SEOs (links in the original post) are recommending to "noindex" tag pages?

It's easy to imagine why long previews can cause duplicate content issues. However, that should be solved with the correct microdata tags.

Do you mean rel="canonical"? How would microdata be used to fix duplicate content issues?

I thought the following Matt Cutts video was relevant because the concerns about site maps being nonsticky or looking spammy also apply equally well to tag pages: (perhaps I'm actually convincing myself that I like this idea of noindexing tag pages. Originally I just brought it up to get feedback.)

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.

Accept third party cookies

​

I agree. I tried to empathise the importance of meta tags/tags (in general they're petty much the same thing), and I found that if you run a niche-traffic website (like Call of Duty for example) you can eat the proverbial pie set by the niche.

Hi Carlos, this sounds interesting. Am I right in understanding that you are saying that the tagging software helped you take over your niche? I'd love to hear more about that.

I have had marvelous results with thread tags, including short thread previews.

I'd really love to hear more about that, Alfa1. My team is planning on putting around 60 hours of work into revamping our tagging system (we are hard core and plan to come up with a list of official tags and then tag some of our best old threads). Hearing about your or Carlos's successes and what you've learned from them would be very helpful.

 

[Carlos]

Thanks for your feedback. Like I said, I'm really amazed at how well our WordPress site does SEO-wise. I only brought up duplicate content issues because I was surprised at why so many people would recommend using meta noindex on tag pages for WordPress. We do in fact use All in One, and don't seem to be having any problems on our own site. The real question is, why does it seem like several smart SEOs (links in the original post) are recommending to "noindex" tag pages?

Oh, okay. I understand what you're saying now that you explained it. A lot of software add-ons don't really give you the option to edit the snippets like All-In-One does. So, you create posts that start out with the same wording as before (I mean the first paragraph - because the first paragraph is the first thing the snippet "grabs"), and ends up looking like a duplicate to let's say... google. This is where the early google code was designed to do - catch you when you're making duplicates. A lot of website owners do it - and likewise, website owners pull(s) your posts (from RSS, XML, etc) and ends up looking like there are a bunch of duplicates in the google indexes. I guess this is why All-In-One or smilar add-ons designed it to prevent that - telling google - "THIS ONE is the real deal. That's a crook trying to steal our crap. He's trying to steal from us!"

But like I said, a lot of programs out there don't really do this - and not the way that All-In-One does. Forums is different in this aspect. You can't really steal the same posts unless you have an indepth - and robust system to really steal your "full" posts. Facebook and other programs have a hard time pulling your posts even in new upgrades.

Do you mean rel="canonical"? How would microdata be used to fix duplicate content issues?

I thought the following Matt Cutts video was relevant because the concerns about site maps being nonsticky or looking spammy also apply equally well to tag pages: (perhaps I'm actually convincing myself that I like this idea of noindexing tag pages. Originally I just brought it up to get feedback.)

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.

Accept third party cookies

​

As much as this is from google themselves, this is their marketing "speech" to prevent everyone from using the same tactics. "noindex" means that your links are not followed by search engines - so even if you have 1,000 links in the comments section (and that's where the spammers are trying to hit), and you have "noindex" on each comments' links whether it's in their "website" field, or the actual post itself... Your SEO doesn't take a hit because it's not being indexed by google - it's being ignored.

Canonical links in general are the "native" links within' a software or a website - For example, those navigation links at the top of this page? Where "Home" "Forums" "Members" are? Those are canonical links. It's hardcoded into the software and won't change in the near future until the developer changes the phrase or link.

Hi Carlos, this sounds interesting. Am I right in understanding that you are saying that the tagging software helped you take over your niche? I'd love to hear more about that.

Yes, I keep seeing keywords that are similar to what I'm writing, or at least what I'm trying to target (with tags, I meant). The only time that I felt that metatags didn't work as much as I thought it would do... is those "pages" in wordpress. Annoying, really. I launched "Multiplayer," "KillStreaks" and other popular keywords for Call of Duty as pages [here, take a look] - and it didn't pull in those keywords - even though they're supposedly popular. On CODForums I was able to pull in some traffic from those keywords because xenForo's SEO is just killer.

But having on-page SEO isn't enough, I need more traffic and tags are usually good for this aspect. It worked for blogs, so, why can't forums have the same kind of result?

I'd really love to hear more about that, Alfa1. My team is planning on putting around 60 hours of work into revamping our tagging system (we are hard core and plan to come up with a list of official tags and then tag some of our best old threads). Hearing about your or Carlos's successes and what you've learned from them would be very helpful.

Good luck with your project!

 

[ForestForTrees]

Hi Carlos, thanks for your support and congratulations on CODForums. I just googled and you are the first ranked independent COD forum, which is pretty cool. I know from hard personal experience how incredibly much work goes into getting to number one. We are a tough breed, us forum admins.

Canonical links in general are the "native" links within' a software or a website - For example, those navigation links at the top of this page? Where "Home" "Forums" "Members" are? Those are canonical links. It's hardcoded into the software and won't change in the near future until the developer changes the phrase or link.

That's interesting, when I think of canonical links, I think of <link rel="canonical"> in the head of the HTML file rather than a class of links in the body like you are describing.

"noindex" means that your links are not followed by search engines - so even if you have 1,000 links in the comments section (and that's where the spammers are trying to hit), and you have "noindex" on each comments' links whether it's in their "website" field, or the actual post itself... Your SEO doesn't take a hit because it's not being indexed by google - it's being ignored.

I think you may be mixing up"noindex" and "nofollow." The "nofollow" meta tag is what tells search engines to ignore all of the links in the page, whereas "noindex" tells the search engine not to include the page in it's index and therefore, presumably, not to return it as a search result. "Nofollow" is, of course, more frequently applied to individual links rather than as a meta tag and would be the one that you would apply to avoid being penalized for links to low quality sites.

 

[Carlos]

Hi Carlos, thanks for your support and congratulations on CODForums. I just googled and you are the first ranked independent COD forum, which is pretty cool. I know from hard personal experience how incredibly much work goes into getting to number one. We are a tough breed, us forum admins.

Yep. Thanks. Trouble is, I need to push more, to get as big as the other COD sites. Fortunately, I have a plan to do that. But it will take some time, and patience to do so...

That's interesting, when I think of canonical links, I think of <link rel="canonical"> in the head of the HTML file rather than a class of links in the body like you are describing.

That might be on the HTML side. But in general, canonical links are those links at the top of the page, like I described. Let me put it this way: Think of it as "Deep Links." For another example, xenForo shows "Home > Forums" and then category x, and node x - right? Those are deep links.

I think you may be mixing up"noindex" and "nofollow." The "nofollow" meta tag is what tells search engines to ignore all of the links in the page, whereas "noindex" tells the search engine not to include the page in it's index and therefore, presumably, not to return it as a search result. "Nofollow" is, of course, more frequently applied to individual links rather than as a meta tag and would be the one that you would apply to avoid being penalized for links to low quality sites.

Nope, not confused or mixed up. Nofollow is just a page HTML code that tells google and other search engines not to follow that particular page, or link. Noindex means that; don't index this page. That's what I meant by what I said.

So, in other words, those 1,000 comments with links on them, if I tell search engines not to index those links - they won't do it, it's ignored. If I tell it not to follow, search engines in general won't try to find the links and "follow" it to see if it's a trusty link... You see what I'm saying?

In either case, google (and/or other search engines) won't slap your hand, or penalize your site because spammers are trying to hurt your site or take away your traffic.

 

[ForestForTrees]

Thanks for clarifying that, Carlos. Now I understand what you are talking about better.

For anyone interested in whether "noindex follow" is appropriate for tag pages, I just found another Matt Cutts video weighing in on it. In this one, he responds to the question, "Do tag clouds help or hinder SEO?" He says that there is a tradeoff in that they could make Google think that you are keyword stuffing but that they can have a good use. The takeaway for me is that if you don't make them look like keyword stuffing, they could be very helpful. The site that I help run that uses Tags does amazingly well, SEO-wise, given it's low pagerank, and I'm pretty sure that that has a lot to do with anchortext and pagerank benefits of tagging. We certainly don't keyword stuff, though.

Here's the video:

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.

Accept third party cookies

​

 

[dethfire]

Did you ever figure out how to noindex xentag? It's very much needed. The tag pages are thin.

 

[ForestForTrees]

Nope, I just haven't had time. It seems like it should be an easy template edit.

 

[cherylferraro]

@ForestForTrees Did you ever figure out how to add the noindex follow meta tags to the tags pages? I tried the following method on both tinhte_xentag_tag_list and tinhte_xentag_search_result_page but the meta tags do not show up in the page source.

Find the root template of the page:

http://xenforo.com/community/threads/1-0-0-b1-how-to-identify-the-root-template-of-a-page.5591/

And add this to the top:

<xen:container var="$head.robots">
    <meta name="robots" content="noindex" /></xen:container>

 

[ForestForTrees]

Nope. Unfortunately, I've been busy. Sorry!

 

[cherylferraro]

@ForestForTrees

Just an fyi, the previously posted method does work on the main tag page. The page must have been cached because the meta is displaying in the page source now.

 

[ProCom]

Digging up this old thread. An SEO firm I'm contracting said:

"Several /tags/ URLs are showing up in Google's Search Index causing "index bloat"
All tag pages must be made "not-indexable" by way of meta robots tags.

Reference: https://developers.google.com/search/reference/robots_meta_tag

Place the following code into the <head> of all /tags/ URLS:

<meta name="robots" content="noindex" />

Important:
Only add this code to tag pages. Adding this anywhere else could result pages not indexing.
Be sure it is only "noindex"
Make sure the snippet is found in the <head>"

What do you guys think? Is this worthwhile?

Either way, how would this be accomplished?

Thanks!

 

[cdub]

Yes I've noticed that google displays my tag pages in results that should show some of the great posts or articles for that keyword. Have you made that noindex meta change to the tag pages?