XF 1.4 Site map

bsdinsight

Active member
Hi All

I am not sure how long Xenforo Site Map submit to Google, but I can see my site from Sep 15, 2014 - can you help me to fix it?


forum.webp
 
You have something in your robots.txt file that disallows Google from crawling, yet you apparently are submitting links in sitemap to crawl that they aren't allowed to due to this.

I've checked your robots.txt file and can't see anything obvious but you've not given full details of the URLs that are blocked - we can only see part of them.
 
Thanks for your information.

My robots.txt file is:

User-agent: Mediapartners-Google
Disallow:

User-agent: *
Disallow: /account/
Disallow: /cdn-cgi/
Disallow: /cgi-bin/
Disallow: /conversations/
Disallow: /doubleclick/
Disallow: /draft/
Disallow: /find-new/
Disallow: /goto/
Disallow: /login/
Disallow: /posts/
Disallow: /unanswered/

Sitemap: http://forums.bsdinsight.com/sitemap.php

what information do you need more, please tell me and I can upload for you to help me to fix it.
 
Is it reporting that member profiles are blocked then? Are there other URLs besides these that it reports as blocked? I ask because in your screenshot it shows 3144 problems, yet you only have 1580 members.

Have you edited the robots.text file recently? Is Google still reporting these URLs as blocked, or only back in September?
 
Is it reporting that member profiles are blocked then? Yes
Are there other URLs besides these that it reports as blocked? No, I can see only 3 links.

Have you edited the robots.text file recently? No
Is Google still reporting these URLs as blocked, or only back in September? Only in Sep

Do you want to logon into my Google account? I can give you.
 
I would note that this is not related to when Google last read the sitemap; that will be listed elsewhere. It's related to specific errors it found on the sitemap at some point in time. It's possible those errors have since been rectified, but it certainly seems to show them for historical purposes.
 
Top Bottom