1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

XF 1.3 XF from WP gave me 7000+ crawl errors in Google. Fix?

Discussion in 'Troubleshooting and Problems' started by drastic, Mar 16, 2014.

  1. drastic

    drastic Well-Known Member

    I moved my site from wordpress to XF.

    I have 1000+ crawl errors from missing urls...like:

    domain.com/tag/?
    domain.com/videos/?
    etc...

    What's the best way to get rid of those errors?

    Is there any easy htaccess fix I can use that will cover at least most of these errors?
     
  2. Jake Bunce

    Jake Bunce XenForo Moderator Staff Member

    Presumably 404 errors (not found)?

    You can choose to ignore it since 404 is correct since the content doesn't exist anymore.

    Or some people prefer to redirect that "removed content" to their forum index. That isn't correct from a content perspective but it gets rid of the 404s. You would use this in your .htaccess in the web root:

    Code:
    RewriteEngine On
    
    RewriteRule ^(tag|videos)/.*$ http://www.yoursite.com/? [R=301,L]
    
     
  3. drastic

    drastic Well-Known Member

    I think the option to ignore sounds pretty easy!

    Where can I update my 404 page template? I want to link to some of the forums so that there's content on it when those users arrive.
     
  4. craigiri

    craigiri Well-Known Member

    You can also update your robots.txt file to tell google to stay out of any particular URLs....those with the member and conversation beginnings, etc....

    like this:
    User-agent: *
    Disallow: /your-forum/conversations/
    User-agent: *
    Disallow: /your-forum/conversations/*
    Disallow: /your-forum/conversations/add?to=*
    Disallow: /your-forum/bookmarks/*
    Disallow: /your-forum/members/*
     
  5. drastic

    drastic Well-Known Member

    Where's the template for the 404 page?

    I want to put links to the most popular sections of my site on that page. makes a better/useful transition from an error to a few pageviews.

    nvm - found it under the phrases section.
     

Share This Page