Robots.txt blocking access to attachments. Good or bad?

[Stuart Wright]

I'm wondering what the reason is behind blocking access to attachments in the robots.txt file.
What are the disadvantages of allowing the indexing of attachments?

 

[whynot]

What are the disadvantages of allowing the indexing of attachments?

Maybe because attachments might disappear if they are not accessed?

 

[Steve F]

If attachments are not viewable by guest then it can lead to crawl errors so blocking that would be a good idea.

 

[agison]

If attachments are not viewable by guest then it can lead to crawl errors so blocking that would be a good idea.

If it is not viewable to guest, blocking or not doesn't effect by Goolge

 

[Steve F]

If an attachment is in a post viewable by a bot (or guest) but the URL to that attachment is behind a permission to view (or not view by guest) then it will certainly show an error in google webmaster tools (crawl errors).

 

[Stuart Wright]

If an attachment is in a post viewable by a bot (or guest) but the URL to that attachment is behind a permission to view (or not view by guest) then it will certainly show an error in google webmaster tools (crawl errors).

That's not the case with us. So is there any other reason to block them?

 

[Alpha1]

Only if bandwidth/resources is a consideration.

 

[Haltech]

I don't block attachment crawls on any of my sites. The reason being, i've never seen anyone successfully link to an attachment where guest view permissions have been disabled from the usergroup permissions side. I have also been told by one of Google's Dev Engineers, their robots by default no longer crawl forums for other than the topic thread meta and body post meta (May 2016). Makes sense as i used to be able to find people by username through Google Search. Now, you have to be more far more specific with your long-tail search.

I do digital marketing by trade and spend a lot of time working with GA Pro, Search Console and Adobe Analytics among many other tools. I do not see any crawl errors and have not for the last 15 or so years with many of my personal sites. Why would you want to give a guest those benefits anyhow? In many cases, they are basically taking without permission for their own personal use, if not... maybe even as a competitor. If they cannot be bothered to even become a member of a community which has provided them a bounty, why support it? This has always been a personal pet peeve of mine

The permissions have always worked for me and never had to the disable in robots. Your mileage may vary of course...

 

[Brent W]

People continue to over think SEO.

User-agent: *
Allow: /

User-agent: BoardReader
Disallow: /

Across all of our sites.

Google is smart. Just let it do its thing.

 

[Live Free]

@Haltech what if you wanted to hide attachments from guests but they haven’t been to this point? Would you make them noindex rather than block with robots due to them potentially being already indexed and maybe in image search results?