XF 2.1 Robots, noindex question

JoyFreak

JoyFreak

Well-known member
Looking for a quick confirmation, if I add this to a template:
<xf:head option="robots"><meta name="robots" content="noindex" /></xf:head>
Click to expand...

Does it stop robots from indexing that page?

Thanks
 
Sort by date Sort by votes
Yes.

developers.google.com

Robots Meta Tags Specifications | Google Search Central | Documentation | Google for Developers

Learn how to add robots meta tags and read how page and text-level settings can be used to adjust how Google presents your content in search results.
developers.google.com developers.google.com
Using the robots meta tag
The robots meta tag lets you utilize a granular, page-specific approach to controlling how an individual page should be indexed and served to users in Google Search results. Place the robots meta tag in the <head> section of a given page, like this:

Code: 
<!DOCTYPE html>
<html><head>
<meta name="robots" content="noindex" />
(…)
</head>
<body>(…)</body>
</html>
The robots meta tag in the above example instructs search engines not to show the page in search results. The value of the name attribute (robots) specifies that the directive applies to all crawlers. To address a specific crawler, replace the robots value of the name attribute with the name of the crawler that you are addressing. Specific crawlers are also known as user-agents (a crawler uses its user-agent to request a page.) Google's standard web crawler has the user-agent name Googlebot. To prevent only Googlebot from crawling your page, update the tag as follows:

<meta name="googlebot" content="noindex" />
Click to expand...
 
Upvote 0 Downvote
palhanow said:
Just for curiosity, if you permit my question:

Whats the benefice in blocking the search crawlers/trackers? Is not good for SEO, right?
Click to expand...
Best of my knowledge,
All website have a search engine budget to crawl your site. Some pages are just not too important, so I block them, allowing room for search engines to find compelling content instead.
 
Upvote 0 Downvote
You must log in or register to reply here.

Similar threads

K
Inconsistent flagging of noindex pages
Replies
0
Views
175
Kirby
K
dethfire
  • Question Question
XF 2.2 Does xenforo add 'X-Robots-Tag' noindex http header to sitemap files in certain cases?
Replies
5
Views
376
James
J
4
  • Question Question
XF 2.2 How to Remove "Robots" from Members Online?
Replies
7
Views
816
Charlie G
Charlie G
Mr Lucky
  • Suggestion Suggestion
Lack of interest Batch update threads to noindex
Replies
0
Views
662
Mr Lucky
Mr Lucky
C
  • Question Question
XF 2.2 Notice conditionals question
Replies
0
Views
79
CTS
C
Top Bottom