1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

XF 1.4 Sitemap warnings

Discussion in 'Troubleshooting and Problems' started by Stuart Wright, Oct 19, 2014.

  1. Stuart Wright

    Stuart Wright Well-Known Member

    Folks, Google webmaster tools is telling me
    Sitemap contains urls which are blocked by robots.txt. Count: 39,915
    Example: Sitemap:www.avforums.com/sitemap.php?c=34 Value:https://static.avforums.com/data/avatars/l/111/111117.jpg?1299678671

    But my sitemap, copied from Xenforo's, does not block /data/avatars
    Here is my robots.txt:
    Confused. Where is the problem, please?
  2. Chris D

    Chris D XenForo Developer Staff Member

    It's technically correct.

    The robots.txt it is referring to is at static.avforums.com:


    That disallows everything in terms of the static. subdomain. It may be ok to just leave it. Or adjusting the robots.txt on the static subdomain to allow /data/avatars/ may be necessary.

    It's worth noting, by the way, that your robots.txt is wrong. Most of those URLs don't exist on your site. XenForo is installed in the root. Should be /find-new/ /account/ /attachments/ etc.
    HWS likes this.

Share This Page