1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Request to block indexing of URLs containing search and member!

Discussion in 'XenForo Pre-Sales Questions' started by OperaManiac, Sep 7, 2010.

  1. OperaManiac

    OperaManiac Well-Known Member

    I fail miserably using the internal search feature of XF so I end up using Google Site Search.

    But Google seems to be indexing pages related to cached search results and member profiles.

    samples from google serps:

    If the developers can block access to these pages using the htaccess file (or whatever they are using!), it would be nice as search results would become less cluttered overall.

    I do not think blocking these two parameters is going to harm the community as such.

    I am also trying to keep an eye on how nicely this forum is indexed by Google so this would be very helpful.

    erich37 likes this.
  2. Carlos

    Carlos Well-Known Member

    Um. Why would you want to block member profiles - am I reading this right?
  3. OperaManiac

    OperaManiac Well-Known Member

  4. James

    James Well-Known Member

    You could use a robots.txt file to do so possibly (on your own installation). I wouldn't want to enable this by default though as people may want their user profiles to be indexed.
  5. OperaManiac

    OperaManiac Well-Known Member

    Well I just wanted it to be blocked around here so that searching the forum through Google Site Search can become more meaningful. :) Of course, developers would have a better idea if my request is logical or not.
  6. Shadab

    Shadab Well-Known Member

    Search results have the noindex meta tag added to the document head (check the html source). But since this was added quite late, Google had already indexed lots of search result pages. It will drop those pages from the index eventually.

    <meta name="robots" content="noindex" />
    Walter likes this.
  7. OperaManiac

    OperaManiac Well-Known Member

    Ooh. Yeah. Just noticed. That's good. Member profiles would help too! Thanks...
  8. Mike

    Mike XenForo Developer Staff Member

    Was about to post what Shadab mentioned, but he beat me to it.

    I disagree with not indexing member profiles though; there is information there that's not available anywhere else. If you want to make profiles non-indexed, you can add that same meta tag.
    Walter and Shadab like this.
  9. erich37

    erich37 Well-Known Member

    A user should be able to decide whether he wants his XF-Profile-Page to be indexed by Google (and other search-engines) or not.

    This is a matter of user-privacy and should be build into XF.
    Sushubh Mittal likes this.
  10. OperaManiac

    OperaManiac Well-Known Member

    ah well i can always play around with advance parameters to exclude 'members' keyword in URL field in my google site searches. so all is well.

    about the above two posts... i would like to mmm repeat that i was not looking for a feature in XF. i just wanted it to be implemented here to clean up the google site search results for xenforo.com...

    just that. nothing more. as an administrator of my own forum, i can always play around with the robots.txt file to achieve that :)
  11. Mike

    Mike XenForo Developer Staff Member


    First option in "people who may..."
    Been Told and erich37 like this.
  12. erich37

    erich37 Well-Known Member

    so by selecting "Members Only", Search-Engines will not be able to index the Profile-Page ?
  13. OperaManiac

    OperaManiac Well-Known Member

    you can always check out by using incognito mode in chrome or private browsing mode in other browsers!

    i just made my profile private. and if i check through incognito mode in chrome i get this on my own profile page:

    XenForo Community - Error​
    This member limits who may view their full profile.​

    I think the mission in accomplished!

    Now, if only I can change my name on this forum. It picked it up from my Facebook profile and I missed changing it to something not related at the time of account creation. I am not sure if sending a private message to the admins would help out. I do not want to annoy them. ;)
    erich37 likes this.
  14. Carlos

    Carlos Well-Known Member

    I am not trying to make you or the OP sound stupid... but I have always thought that having a member URL being index'ed by google or other SE's is good for SEO, because it would mean that they crawl the threads you started, posts you made, and your status.
  15. OperaManiac

    OperaManiac Well-Known Member

    But on Xenforo that data is fetched using ajax and is unlikely to be served to search spiders :) my user profile page indexed by google shows no relevant data.

    This post was only made to suggest that indexing should be stopped for member pages so that searching on this forum through Google is easier. Not for the SEO point of view. :)

Share This Page