1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

XF 1.4 sitemap major issues

Discussion in 'Troubleshooting and Problems' started by Echelon, Sep 21, 2014.

  1. Echelon

    Echelon Active Member

    Is there somebody who can check my sitemap consistency on our platform?

    got more then 800 urls but the sitemap only "sees" 190

    this is too difficult for me to get a grip on it so it would be nice if someone can check this for me
  2. Chris D

    Chris D XenForo Developer Staff Member

    It's quite simple, really, there's not much that can go wrong.

    How are you calculating 800 URLs?

    What content types do you have checked to include in the Sitemap in Admin CP > Options > XML Sitemap?

    Do you have any forums that are not visible to guests?
    Echelon likes this.
  3. Echelon

    Echelon Active Member

    yes. and they are not indexed

    does this has something to do with route options because the normal routes are in English and i translated them into Dutch
    i.e. see visual below

  4. Hoffi

    Hoffi Well-Known Member

    There are many mor URL's on your Forum as th system writes in the sitemap.xml - anything is OK.
    Echelon likes this.
  5. Echelon

    Echelon Active Member

    So what you say is that the sitemap structure is in good order and i miss no important things?
    i do not need to fill in extra urls and xenforo sitemap recognise the routechanges i made?
  6. Chris D

    Chris D XenForo Developer Staff Member


    This is where the confusion lies.

    The idea of a sitemap isn't necessarily to tell a search engine about every single URL on your site down to the finest detail, and providing a sitemap doesn't necessarily say to the search engine that they are your ONLY URLs.

    If your sitemap contains 100 URLs and Google indexes 1000; there's no problem.

    There's a few factors to consider:
    • The Sitemap generation does not include every page of every thread (or every page of every other piece of content), it only includes the URL to the first page.
    • The Sitemap won't include content you have hidden from guests using permissions.
    • The Sitemap won't include URLs from add-ons unless those add-ons have specifically added support for the Sitemap feature, e.g. you have a lot of "tags" pages indexed in Google, but these won't appear in the Sitemap unless that add-on adds support for that.
    • Search engines are typically more than capable of finding their own way around. They will often find URLs you may not have explicitly included in the Sitemap, hence why in the example above it has indexed tags pages even though you don't have a sitemap for them.
    • Search engines may index pages blocked using robots.txt, and may crawl pages specified as "noindex" but exclude the content from search results.
    So, with all that considered, your sitemap is normal, your presence in Google is already good.

    Correct. All is good and you do not need to do anything extra.
    Echelon likes this.
  7. Chris D

    Chris D XenForo Developer Staff Member

    All that being said, your Sitemap does contain some duplicate URLs and I'm not sure why it would.

    Those URLs both go to the same page but the only difference is one is the original route, and one is the route you've changed that original route to with a route filter.
    Echelon likes this.
  8. Echelon

    Echelon Active Member

    Wow! Thanks a lot Chris for debunking my confusion and uncertainty about this sitemap issue. Now I'm satisfied.
    Chris D likes this.
  9. Echelon

    Echelon Active Member

    Well...as it seems the first is the translated one from resources [US] to Databank [Dutch]
    How to solve this?
  10. Chris D

    Chris D XenForo Developer Staff Member

    I don't know, it's not a behaviour I would expect to see.

    It's not a big problem because Google would understand what's going on as one 301 redirects to the other.

    If you go to Admin CP > Tools > Rebuild Caches > Rebuild XML Sitemap, it will rebuild the Sitemap for you. Then I'll check again.

    I can't reproduce this myself by adding a Route Filter...
    Echelon likes this.
  11. Echelon

    Echelon Active Member

    Ok. I just rebuild my sitemap.
  12. Chris D

    Chris D XenForo Developer Staff Member

    Still there...

    @Mike might need to answer this because I don't think that's expected but also I can't see where in the code it would be duplicating that.
  13. Echelon

    Echelon Active Member

    Would be nice if this can be solved...i miss the knowledge to solve it myself. Thanks again Chris for sorting things out
  14. Mike

    Mike XenForo Developer Staff Member

    Have you added URLs manually to the sitemap? The order seems completely inconsistent as well, which is odd given that the sitemap is built by content type and with increasing IDs.

    Any other add-ons enabled? Try disabling them and rebuilding the sitemap. Can you list all of the route filters you have?
    Chris D likes this.
  15. Echelon

    Echelon Active Member

    No. I did not added any other url's. Before i used a sitemap addon but deinstalled it.

    got 35 routefilters...how do i list them since i cannot select the text
  16. Chris D

    Chris D XenForo Developer Staff Member

    That crossed my mind too, but they have identical lastmod dates so I assumed they were coming from the same sitemap handler.

    Screenshot, or, alternatively, if you're comfortable running queries on the database, run this query and copy and paste (or screenshot) the output:

    SELECT * FROM xf_route_filter WHERE enabled = 1
    Echelon likes this.
  17. Echelon

    Echelon Active Member

    2.png 1.png
  18. Chris D

    Chris D XenForo Developer Staff Member

    One point you missed from Mike's post is the question about add-ons. Clearly you have a lot of add-ons.

    As a temporary measure, please disable all add-ons except XenForo Resource Manager and generate the Sitemap again.

    As soon as you have generated the sitemap, you can re-enable all add-ons.
    Echelon likes this.
  19. Echelon

    Echelon Active Member

    yes....84 addons minus the ones we don't need anymore because it's integrated in xenforo 1.4 ;-)

    Ok. i will disable all the addons except resource manager tonight,
    then generate a new sitemap,
    then activate all the addons again

    sorry for the fuzz but im not technical
    Chris D likes this.
  20. Echelon

    Echelon Active Member

    at this moment i disabled all addons and created a new sitemap....what to do next?

Share This Page