• This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.

Why does GWT show errors for domain.com/posts/<number> ?

TheBigK

Well-known member
#1
My robots.txt file looks like this:

User-agent: *
Disallow: /find-new/
Disallow: /account/
Disallow: /goto/
Disallow: /posts/
Disallow: /login/
Disallow: /admin.php
Allow: /

...and yet; GWT reports hundreds of errors for URLs like:
http://www.domain.com/posts/300709/


Should I be worried? Is there anything that needs a fix?
 

CTXMedia

Formerly CyclingTribe
#2
What is the nature of the error? 404 - 301/2 - 500 - or something else?

And have you made any changes to your robots.txt recently in relation to /posts/ or changed your site structure in any way?
 

TheBigK

Well-known member
#3
What is the nature of the error? 404 - 301/2 - 500 - or something else?

And have you made any changes to your robots.txt recently in relation to /posts/ or changed your site structure in any way?
I did not make any changes. There is no error code associated with those errors! That's why I'm curious to know what's really happening.

I found that GWT now allows you to download a bit of your total errors. The downloaded CSV shows no error code or type against these errors.
 

CTXMedia

Formerly CyclingTribe
#4
Could you provide a screenshot of the error text as it is displayed in GWT or perhaps explain what section of GWT the errors are in?
 

TheBigK

Well-known member
#5
Could you provide a screenshot of the error text as it is displayed in GWT or perhaps explain what section of GWT the errors are in?
This isn't very useful, I guess :-

Screen Shot 2014-02-28 at 2.17.58 pm.png

WMT shows only top 1000 errors so these errors can only be seen after downloading the error log.
 

TheBigK

Well-known member
#7
Yep - they do redirect properly. So there should not be any issue. However, Google reporting them as error got me worried. My bigger question is: Why is Google ignoring our robots.txt.