Is there a good reason to Disallow: /attachments/ in robots.txt?

Ozzy47

Well-known member
Just a guess, maybe some admins don’t want their images indexed by google, or they are afraid of the bots using up bandwidth.
 

Kevin

Well-known member
On the basis that it's a good thing if Google can index attachments on your site - predominantly images - then why would you Disallow: /attachments/ ?
Do you allow guests to view attachments? Remember, the Google bot is a guest and if you have attachment viewing for guests turned off then the bot will be running into permission errors when trying to retrieve the attachment URLs.
 

Chris D

XenForo developer
Staff member
There is no bots usergroup by default in XF and @Kevin's previous post was accurate - bots are treated the same as guests.
 

jgaulard

Member
On the basis that it's a good thing if Google can index attachments on your site - predominantly images - then why would you Disallow: /attachments/ ?
The actual attachments path is /data/attachments/. The clickable path to open the attachment is just /attachments/. Two different paths. By blocking /attachments/ in the robots.txt file, you're not blocking any uploaded attachments, just the clickable paths to them. It's set up a little weird. The actual page (lightbox) that the image shows in is actually considered a page in itself.
 
Top