1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Won't Fix Searches should return proper HTTP 404 code if nothing found

Discussion in 'Resolved Bug Reports' started by karll, Jun 5, 2013.

  1. karll

    karll Well-Known Member

    ... or so Google webmaster tools seems to think. It complains that my site returns "soft 404" pages (i.e. 200 OK) when it's searching for stuff it can't find.

    You can reproduce the issue here on xenforo.com with a terminal command similar to:
    Code:
    wget 'http://xenforo.com/community/search/search/?keywords=Soyuz&type=forums'
    (Don't forget the quotes around the URL.)

    This will give you a "200 OK" code rather than a 404 code. (Obviously replace the search term "Soyuz" with something else not likely to be found as that search will now probably find this very page!)
     
  2. Mike

    Mike XenForo Developer Staff Member

    I can't say I agree. The RFC says:
    We have found something matching the request -- we've actually done the search, it just hasn't matched anything. A "we couldn't find anything for that search" seems different to me than "this page doesn't exist". It's roughly the equivalent to an empty forum.
     
    Walter likes this.
  3. CyclingTribe

    CyclingTribe Well-Known Member

    I would hope so too - we don't want hard 404's when people use the search feature.

    Search isn't page (indexable) content either, not in terms of SE's, so what you're better off doing is excluding your XF site search feature from being crawled in the first place using robots.txt - E.g.;

    Code:
    User-agent: *
    Disallow: /account/
    Disallow: /admin.php
    Disallow: /ajax/
    Disallow: /attachments/
    Disallow: /chat/
    Disallow: /conversations/
    Disallow: /find-new/
    Disallow: /goto/
    Disallow: /help/
    Disallow: /login/
    Disallow: /logos/
    Disallow: /logstats/
    Disallow: /lost-password/
    Disallow: /members/
    Disallow: /mobiquo/
    Disallow: /online/
    Disallow: /posts/
    Disallow: /recent-activity/
    Disallow: /register/
    Disallow: /search/
    
    Cheers,
    Shaun :D

    [Ninja'd :)]
     
    karll likes this.
  4. karll

    karll Well-Known Member

    I suppose what they're saying is that the page "http://xenforo.com/community/search/search/?keywords=Soyuz&type=forums" doesn't really exist. (If it did, then similar pages for all sorts of random search terms also exist, which means you have a lot of pages ...!)

    Hm, that makes a lot of sense. Thank you!
     
    CyclingTribe likes this.

Share This Page