XF 1.2 Robots.txt recommendations

leemy

Member
I noticed that google is recording each post URL, for example Post #60 permalink, post #72 permalink. On my SMF site, I disallowed individual posts, so each page is captured.

Anyone have recommendations for a good robots.txt for XF?

Thanks!
 
Top Bottom