• This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.

For those not blocking attachments, data in robots.txt

cmeinck

Well-known member
#1
How many of you are not blocking your attachments and data folders with robots.txt. If you're among this group, I'm curious if Google is indexing these within the regular index.

If you do a site:yoursite.com inurl:/attachments/

What are the results?
 

Mr. Goodie2Shoes

Well-known member
#2
Just wanted to know: is there any benefit (except for saving bandwidth) to block Google to index attachments where most would be images?
Wouldn't it help reach some visitors who were searching images?
 

cmeinck

Well-known member
#3
Just wanted to know: is there any benefit (except for saving bandwidth) to block Google to index attachments where most would be images?
Wouldn't it help reach some visitors who were searching images?
My concern would be attachment pages looking like actual pages -- thus creating thousands of thin-content pages. I agree with you, if Google handles the images correctly. I'm curious if they show in the index.
 

melbo

Well-known member
#7
I should also note that we allow guest downloads of all file attachments so Google may be displaying these more richly as a result.
 

Ridemonkey

Well-known member
#8
We allow guest viewing of attachments, and Google's results are similar to my fellow Monkey-site owner above. Only attachments that have actual indexed content appear in search results, otherwise you have to do a Google Image search to see the rest.
 

Donny

Active member
#10
I made attachments public a month ago and haven't seen any traffic bump
Wondering if google has now indexed these and whether you're getting any fresh traffic from them?

I have 14000 images in my sitemap in my google webmasters account but it doesn't say they're indexed yet. Wondering why, and if I can find some way to help google index them.
 

dethfire

Well-known member
#11
Wondering if google has now indexed these and whether you're getting any fresh traffic from them?

I have 14000 images in my sitemap in my google webmasters account but it doesn't say they're indexed yet. Wondering why, and if I can find some way to help google index them.
I'm about 6 months in and have noticed no increase in traffic
 

dethfire

Well-known member
#13
Thanks.

Has google indexed the attachments and images and whatnot?
How many approx does google have in their index now compared to before, is that something you know?
site: physicsforums.com/attachments

That doesn't appear to include images and if I switch over to images it doesn't give a count