• This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.

XF 1.1 Define Bot access

MagnusB

Well-known member
#2
You shouldn't allow google access to content guest users aren't allowed to see. Rather add Disallow *: /attachments/ in your robots.txt. That way Google wont index your attachments.
 

abdfahim

Active member
#3
You shouldn't allow google access to content guest users aren't allowed to see. Rather add Disallow *: /attachments/ in your robots.txt. That way Google wont index your attachments.
you mean I should write following in my robot.txt?
Code:
Disallow *: /attachments/