1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

XF 1.5 Keeping one sub-forum's threads unindexed by search engines

Discussion in 'Troubleshooting and Problems' started by TomCann, Apr 27, 2016.

  1. TomCann

    TomCann Member

    I have found various threads on here about robots.txt and robots=noindex, but have been unable to work out what to do for my situation. I run a forum with half a dozen sub-forums, and one of these contains information that a significant number of members do not want to be indexed by search engines as it contains financial information. Because it is such an issue I tried to block that specific sub forum with a robots.txt disallow on that sub-thread, but that clearly is not working - threads within it are still being indexed.

    Is there a way to allow indexing of 5 sub-forums and their threads, while blocking indexing of 1 sub-forum and its threads, and if so, can someone describe how it is done?

    Many thanks for your help
  2. Martok

    Martok Well-Known Member

    Can't you just make that forum available to registered users only? That would solve the problem.
  3. TomCann

    TomCann Member


    Thanks, but that doesn't stop search engines indexing as far as I can see? This particular forum already requiress registered users to apply for an extra permission, but seems to still be indexed by Google and other search engines.
  4. Brogan

    Brogan XenForo Moderator Staff Member

    Search engines can't index content they have no access to.

    Removing access for guests, as Martok suggested, would achieve it.
  5. TomCann

    TomCann Member

    Many thanks. OK, it looks like I must have permissions set incorrectly for this forum, so will try that first.
  6. TomCann

    TomCann Member

    Brogan, can you say exactly what you mean by "removing access for guests"? In permissions for this node I had allowed all registered AND unregistered users to view it, but to post or reply you have to be registered and ask to be added to special sub-group that has permission to post. If I now change this to deny view access to the 'unregistered/uncomfirmed' group, will that acheive want I want? Or do I have to do something else? Many thanks again.
  7. Brogan

    Brogan XenForo Moderator Staff Member

    Click on the Permissions link in the node tree for that forum.

    Revoke the main 'View node' permission for the unregistered user group.
  8. TomCann

    TomCann Member

    Great - that's what I'd figured, so thanks for confirming.

Share This Page