1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

XF 1.4 Google bots, and others ....

Discussion in 'XenForo Questions and Support' started by gajinoz, Feb 18, 2015.

  1. gajinoz

    gajinoz Member

    How do I stop bots like google and others from seeing a particular category and/or forum? I'm happy for Google to index most of my site but not some private areas.
  2. gajinoz

    gajinoz Member

    I think I may have found it. I have the category permissions set to NO for unregistered users. I'm assuming this also includes bots, is this correct?

    Also, if I set NO to the category, does this apply to all forums under that or do I need to set each forum individually?

  3. Brogan

    Brogan XenForo Moderator Staff Member

    Bots are the same as guests, so yes, they will not have access if you restrict it for guests.

    Permissions cascade down, but can be overridden in child nodes by being explicitly set.
  4. Newt

    Newt Active Member

    So if a thread content view is revoked for guests, is a bot following a thread link also redirected to the login/registration page? Or does it get an error message (like a 404)?

    Thread content restriction might entice people to register, but does anybody knows how it is handled by a bot, for instance, if seeing titles but not content has some positive or negative effects on SEO (other that not reaching content prevents from indexing it)?
  5. Brogan

    Brogan XenForo Moderator Staff Member

    The standard error_with_login template would be shown, which is a 403 (forbidden).
    Newt likes this.

Share This Page