1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

XF 1.2 Robots.txt recommendations

Discussion in 'Troubleshooting and Problems' started by leemy, Oct 10, 2013.

  1. leemy

    leemy Member

    I noticed that google is recording each post URL, for example Post #60 permalink, post #72 permalink. On my SMF site, I disallowed individual posts, so each page is captured.

    Anyone have recommendations for a good robots.txt for XF?

    Thanks!
     
  2. Adam Howard

    Adam Howard Well-Known Member

    http://xenforo.com/robots.txt

    Of course you should change the path to match your own site
     
  3. leemy

    leemy Member

    Thanks!
     

Share This Page