XF 1.2 Robots.txt recommendations

[leemy]

I noticed that google is recording each post URL, for example Post #60 permalink, post #72 permalink. On my SMF site, I disallowed individual posts, so each page is captured.

Anyone have recommendations for a good robots.txt for XF?

Thanks!

 

[Adam Howard]

I noticed that google is recording each post URL, for example Post #60 permalink, post #72 permalink. On my SMF site, I disallowed individual posts, so each page is captured.

Anyone have recommendations for a good robots.txt for XF?

Thanks!

http://xenforo.com/robots.txt

Of course you should change the path to match your own site

 

[leemy]

Thanks!