1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

XF 1.1 How do I stop Google from indexing profile pages?

Discussion in 'Troubleshooting and Problems' started by jauburn, Dec 18, 2011.

  1. jauburn

    jauburn Well-Known Member

    How do I keep google from indexing profile pages? I'm using the sitemap generator on this site and I have this in my robots.txt file

    User-agent: *
    Disallow: /forum/find-new/
    Disallow: /forum/account/
    Disallow: /forum/attachments/
    Disallow: /forum/goto/
    Disallow: /forum/posts/
    Disallow: /forum/login/
    Disallow: /forum/admin.php
    Allow: /

    Thanks for any help.
     
  2. Brogan

    Brogan XenForo Moderator Staff Member

  3. jauburn

    jauburn Well-Known Member

    Thanks I already saw that thread, but there are *so many* robots text file examples there, how do I know which one to use? Also the first one listed seems to have all kinds of things that aren't relevant to a stock xen installation (wiki? tera-tweet?)
     
  4. Brogan

    Brogan XenForo Moderator Staff Member

    The fifth post has an entry for Disallow: /members/
     
  5. jauburn

    jauburn Well-Known Member

Share This Page