Request to block indexing of URLs containing search and member!

[Chromaniac]

I fail miserably using the internal search feature of XF so I end up using Google Site Search.

But Google seems to be indexing pages related to cached search results and member profiles.

samples from google serps:
xenforo.com/community/search/88582/?page=4
xenforo.com/community/members/sushubh-mittal.1705/

If the developers can block access to these pages using the htaccess file (or whatever they are using!), it would be nice as search results would become less cluttered overall.

I do not think blocking these two parameters is going to harm the community as such.

I am also trying to keep an eye on how nicely this forum is indexed by Google so this would be very helpful.

Cheers.

 

[Carlos]

Um. Why would you want to block member profiles - am I reading this right?

 

[Chromaniac]

Only from indexing by Google (and other search engines). In my personal opinion, member profiles does not add anything worthy to your indexed content.

You can check the Google Cache of my own profile: http://webcache.googleusercontent.c...ittal.1705/&aq=f&aqi=&aql=&oq=&gs_rfai=&pbx=1

It does not have anything useful!

 

[James]

You could use a robots.txt file to do so possibly (on your own installation). I wouldn't want to enable this by default though as people may want their user profiles to be indexed.

 

[Chromaniac]

Well I just wanted it to be blocked around here so that searching the forum through Google Site Search can become more meaningful. Of course, developers would have a better idea if my request is logical or not.

 

[Shadab]

Search results have the noindex meta tag added to the document head (check the html source). But since this was added quite late, Google had already indexed lots of search result pages. It will drop those pages from the index eventually.

<meta name="robots" content="noindex" />

 

[Chromaniac]

Ooh. Yeah. Just noticed. That's good. Member profiles would help too! Thanks...

 

[Mike]

Was about to post what Shadab mentioned, but he beat me to it.

I disagree with not indexing member profiles though; there is information there that's not available anywhere else. If you want to make profiles non-indexed, you can add that same meta tag.

 

[erich37]

A user should be able to decide whether he wants his XF-Profile-Page to be indexed by Google (and other search-engines) or not.

This is a matter of user-privacy and should be build into XF.

 

[Chromaniac]

ah well i can always play around with advance parameters to exclude 'members' keyword in URL field in my google site searches. so all is well.

about the above two posts... i would like to mmm repeat that i was not looking for a feature in XF. i just wanted it to be implemented here to clean up the google site search results for xenforo.com...

just that. nothing more. as an administrator of my own forum, i can always play around with the robots.txt file to achieve that

 

[Mike]

A user should be able to decide whether he wants his XF-Profile-Page to be indexed by Google (and other search-engines) or not.
This is a matter of user-privacy and should be build into XF.

http://xenforo.com/community/settings/privacy

First option in "people who may..."

 

[erich37]

so by selecting "Members Only", Search-Engines will not be able to index the Profile-Page ?

 

[Chromaniac]

you can always check out by using incognito mode in chrome or private browsing mode in other browsers!

i just made my profile private. and if i check through incognito mode in chrome i get this on my own profile page:

XenForo Community - Error​

This member limits who may view their full profile.​

I think the mission in accomplished!

Now, if only I can change my name on this forum. It picked it up from my Facebook profile and I missed changing it to something not related at the time of account creation. I am not sure if sending a private message to the admins would help out. I do not want to annoy them.

 

[Carlos]

Only from indexing by Google (and other search engines). In my personal opinion, member profiles does not add anything worthy to your indexed content.

You can check the Google Cache of my own profile: http://webcache.googleusercontent.c...ittal.1705/&aq=f&aqi=&aql=&oq=&gs_rfai=&pbx=1

It does not have anything useful!

I am not trying to make you or the OP sound stupid... but I have always thought that having a member URL being index'ed by google or other SE's is good for SEO, because it would mean that they crawl the threads you started, posts you made, and your status.

 

[Chromaniac]

But on Xenforo that data is fetched using ajax and is unlikely to be served to search spiders my user profile page indexed by google shows no relevant data.

This post was only made to suggest that indexing should be stopped for member pages so that searching on this forum through Google is easier. Not for the SEO point of view.