1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Why does GWT show errors for domain.com/posts/<number> ?

Discussion in 'Server Configuration and Hosting' started by TheBigK, Feb 28, 2014.

  1. TheBigK

    TheBigK Well-Known Member

    My robots.txt file looks like this:

    User-agent: *
    Disallow: /find-new/
    Disallow: /account/
    Disallow: /goto/
    Disallow: /posts/
    Disallow: /login/
    Disallow: /admin.php
    Allow: /

    ...and yet; GWT reports hundreds of errors for URLs like:

    Should I be worried? Is there anything that needs a fix?
  2. CyclingTribe

    CyclingTribe Well-Known Member

    What is the nature of the error? 404 - 301/2 - 500 - or something else?

    And have you made any changes to your robots.txt recently in relation to /posts/ or changed your site structure in any way?
  3. TheBigK

    TheBigK Well-Known Member

    I did not make any changes. There is no error code associated with those errors! That's why I'm curious to know what's really happening.

    I found that GWT now allows you to download a bit of your total errors. The downloaded CSV shows no error code or type against these errors.
  4. CyclingTribe

    CyclingTribe Well-Known Member

    Could you provide a screenshot of the error text as it is displayed in GWT or perhaps explain what section of GWT the errors are in?
  5. TheBigK

    TheBigK Well-Known Member

    This isn't very useful, I guess :-

    Screen Shot 2014-02-28 at 2.17.58 pm.png

    WMT shows only top 1000 errors so these errors can only be seen after downloading the error log.
  6. CyclingTribe

    CyclingTribe Well-Known Member

  7. TheBigK

    TheBigK Well-Known Member

    Yep - they do redirect properly. So there should not be any issue. However, Google reporting them as error got me worried. My bigger question is: Why is Google ignoring our robots.txt.

Share This Page