XF 2.1 Duplicate content Tags

Agentos

Member
Since I installed xenforo 2, big problems with content duplication have started to appear. How should I fix it? 98% tags is duplicate
 
Last edited:

djbaxter

Well-known member
Minimally if at all. People don't search for tags and tags don't help with ranking. The only thing tags might help with is crawling but sitemaps do a much better job of that. Tags may easily become a spider trap.

People search for content and search engines index content.

I haven't used tags on any site, forums, blogs, or anything else, for years. It's an SEO dinosaur but like many persistenjt SEO myths people still seem to believe that it's helping in some way.
 

djbaxter

Well-known member
It should yes.

Using the robots.txt for Xenforo.com, just add /tags/ at the bottom of the list:

Code:
User-agent: *
Disallow: /community/whats-new/
Disallow: /community/account/
Disallow: /community/attachments/
Disallow: /community/goto/
Disallow: /community/posts/
Disallow: /community/login/
Disallow: /community/admin.php
Disallow: /community/tags/
Allow: /

Sitemap: https://xenforo.com/community/sitemap.xml
Make sure you remove the /community part if your forum is in the root directory, or change it to whatever name you have for the forum subdirectory.

Also make sure that your robots.txt file is in your root folder, even if the forum is in a subfolder:


Note: It may take Google a while to catch up but those will eventually stop appearing in your Google Analytics page.
 

djbaxter

Well-known member
Because it IS duplicate content since many different threads/pages will use the same tag. It also looks like you are using some sort of addon to automatically generate tags based on content of posts, which just magnifies the problem.

The easiest thing to do is just turn off tags:

AdminCP >> Setup >> Options >> Content tagging >> Enable tags = unchecked
AdminCP >> Setup >> Options >> Advanced Search >> Enable tags = unchecked

Let me turn this around: Why are you using tags at all? What do you think all those tags are doing for your forum?
 

djbaxter

Well-known member
I cant delete. Guest and members sometimes using search tags
And if they weren't there they would search using content terms.

But if you feel you must keep using tags on your forum, then use the robots.txt solution I posted above to stop Google listing them, as duplicate content.

And if you don't want to do even that, just ignore the duplicate content messages, but know that having googlebot follow all those tags is probably using up a lot of your crawl budget.
 

Agentos

Member
Something is wrong with Siteliner. I checked other pages on it, does it show almost entire pages as duplicate content? Is there any other audit where you can check duplicate content?
 
Top