Is there a good reason to Disallow: /attachments/ in robots.txt?

ozzy47

Well-known member
Just a guess, maybe some admins don’t want their images indexed by google, or they are afraid of the bots using up bandwidth.
 

Kevin

Well-known member
On the basis that it's a good thing if Google can index attachments on your site - predominantly images - then why would you Disallow: /attachments/ ?
Do you allow guests to view attachments? Remember, the Google bot is a guest and if you have attachment viewing for guests turned off then the bot will be running into permission errors when trying to retrieve the attachment URLs.
 

Chris D

XenForo developer
Staff member
There is no bots usergroup by default in XF and @Kevin's previous post was accurate - bots are treated the same as guests.
 
Top