XF 2.2 How can I stop members from getting indexes?

[Garfield™]

Users are not checked in sitemap options, the following commands are added in robots txt, but it still gets indexed.

Disallow: /kullanicilar
Disallow: /kullanicilar/

https://www.iyiforumlar.net/robots.txt

 

[AndyB]

Only way would be to not allow the Unregistered / Unconfirmed user group to view the members list and profiles:

 

[Chromaniac]

Yeah. Google has been ignoring everything and indexing members pages for me too. They are disabled for access in usergroup settings. They are also blocked with nofollow meta tag. AND they are also blocked in Robots.txt. Of course they cannot read the content on these pages so they get raw results.

No page information in search results - Search Console Help

Google actually says this on this support page:

You have a rule in your site's robots.txt file* that prevents Google from reading the page (which is how we generate a description in search results). However, this doesn't prevent Google from showing the page in Search results. Therefore if the page appears in Search results, it will appear without any description.

Google further says that you should use noindex on pages rather than robots file if you want to prevent indexing.

Use "noindex" on your page. If using noindex, you must also remove the robots.txt rule that blocks the page to search engines. Sounds strange, but we need to be able to read the page in order to see your "noindex" instruction.

This post might help.

XF 2.0 - Noindex members pages

I would like to noindex all the members pages. How can I do this in 2.x? Thanks!

xenforo.com

 

[Mr Lucky]

Only way would be to not allow the Unregistered / Unconfirmed user group to view the members list and profiles:

Yes that will work, but do you know, in that case, why there is this conditional in member_view?

<xf:if is="!$user.isSearchEngineIndexable()">
    <xf:head option="metaNoindex"><meta name="robots" content="noindex" /></xf:head>
</xf:if>

This implies there is a condition under which user profiles are either indexed or not indexed.

 

[M@rc]

Yes that will work, but do you know, in that case, why there is this conditional in member_view?

<xf:if is="!$user.isSearchEngineIndexable()">
    <xf:head option="metaNoindex"><meta name="robots" content="noindex" /></xf:head>
</xf:if>

This implies there is a condition under which user profiles are either indexed or not indexed.

See this:

XF 2.2 Post in thread 'What affects the !$user.isSearchEngineIndexable() conditional?'

Dec 31, 2020

At this time, this is mostly a placeholder function in the code. There aren't any switches that control it, but it serves as a point where an add-on could expand functionality, for example. (It was added primarily to mirror functionality added to threads which does define criteria over what threads are indexable.)

• Mike