1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Crawl Errors Rising

Discussion in 'Troubleshooting and Problems' started by lazer, Jun 15, 2013.

  1. lazer

    lazer Well-Known Member

    Hey :)

    So, I was looking at GWT today and noticed a steady increase in crawl errors. The weird thing is that the URL's it is "failing" on, haven't been physically present on my server for over a year (converted from vB to xF in April 2012).

    Also, since installing the Resources add-on, we closed off the forum that was doing the job up until then. Google is still trying to index that forum but is listing 1,000's of "Access Denied" errors, as the forum is no longer open to the public.

    Can you spot when we installed RM and closed the related forum? :p

    Why is Google still looking at these URL's (and subsequently returning a Soft404 or "access denied" errors) and how do I tell them to stop?

  2. Jake Bunce

    Jake Bunce XenForo Moderator Staff Member

  3. lazer

    lazer Well-Known Member

    Hm. Can I actually do that though?

    The errors are for locations such as this...

    My "live" board has its URL's redirected from www.xxxx.com/xxxx to www.xyz.com/xf/xxxx

    There's no "directory" as such.
  4. CyclingTribe

    CyclingTribe Well-Known Member

  5. lazer

    lazer Well-Known Member

    Hey Shaun
    Although, they haven't done so already, after over a year.
    The figure is around 2,000 URL's. Thanks for the link, I already knew about that but was hoping to find something a little less "manual" :)
    CyclingTribe likes this.
  6. CyclingTribe

    CyclingTribe Well-Known Member

    I've got 404's for page links that are 2 years older and more - when you click on the link (in Crawl Errors) to analyse them, you get a tab which says "Linked from" - presumably these links back to the URL from other site are what keep encouraging Google to try and index it?

    If it's a soft/hard 404 issue it might be worth opening a discussion here on XF regarding the presentation of 404's to SEs - maybe something could be done to make it a more "hard" 404 error and encourage them to drop the links sooner?
    lazer likes this.

Share This Page