1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

For those not blocking attachments, data in robots.txt

Discussion in 'Forum Management' started by cmeinck, Mar 12, 2015.

  1. cmeinck

    cmeinck Well-Known Member

    How many of you are not blocking your attachments and data folders with robots.txt. If you're among this group, I'm curious if Google is indexing these within the regular index.

    If you do a site:yoursite.com inurl:/attachments/

    What are the results?
     
  2. Mr. Goodie2Shoes

    Mr. Goodie2Shoes Well-Known Member

    Just wanted to know: is there any benefit (except for saving bandwidth) to block Google to index attachments where most would be images?
    Wouldn't it help reach some visitors who were searching images?
     
  3. cmeinck

    cmeinck Well-Known Member

    My concern would be attachment pages looking like actual pages -- thus creating thousands of thin-content pages. I agree with you, if Google handles the images correctly. I'm curious if they show in the index.
     
  4. Anthony Parsons

    Anthony Parsons Well-Known Member

    My page results were nothing, images displayed only in image search... so no issue.
     
  5. melbo

    melbo Well-Known Member

    Only pdf's, Images are in Google Images

    site:survivalmonkey.com inurl:/attachments/


    upload_2015-3-12_14-30-3.png
     
  6. cmeinck

    cmeinck Well-Known Member

    This was very helpful. Thanks for the feedback.
     
  7. melbo

    melbo Well-Known Member

    I should also note that we allow guest downloads of all file attachments so Google may be displaying these more richly as a result.
     
  8. Ridemonkey

    Ridemonkey Well-Known Member

    We allow guest viewing of attachments, and Google's results are similar to my fellow Monkey-site owner above. Only attachments that have actual indexed content appear in search results, otherwise you have to do a Google Image search to see the rest.
     
    melbo and cmeinck like this.
  9. dethfire

    dethfire Well-Known Member

    I made attachments public a month ago and haven't seen any traffic bump
     
  10. Donny

    Donny Active Member

    Wondering if google has now indexed these and whether you're getting any fresh traffic from them?

    I have 14000 images in my sitemap in my google webmasters account but it doesn't say they're indexed yet. Wondering why, and if I can find some way to help google index them.
     
  11. dethfire

    dethfire Well-Known Member

    I'm about 6 months in and have noticed no increase in traffic
     
  12. Donny

    Donny Active Member

    Thanks.

    Has google indexed the attachments and images and whatnot?
    How many approx does google have in their index now compared to before, is that something you know?
     
  13. dethfire

    dethfire Well-Known Member

    site: physicsforums.com/attachments

    That doesn't appear to include images and if I switch over to images it doesn't give a count
     

Share This Page