robots.txt

[Deleted member 745]

Hi,

Will there be a robots.txt included with XF in the near future or maybe in the next beta package?

 

[Trip]

I'm sure you have already seen this, but just in case somebody else who is reading this discussion "needs this now". There is a good example of one within one of the forum sections...
http://xenforo.com/community/threads/my-robots-txt-file.6752/

 

[Deleted member 745]

Seen it but not sure if its a good one as on IPB they block links (/*app=core&module=usercp) as well as direct links of folders. (/cache/)