1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

XF 1.5 Sitemap :no individual pages ?

Discussion in 'XenForo Questions and Support' started by Weppa333, Dec 15, 2015.

  1. Weppa333

    Weppa333 Active Member


    I just noticed on my install, but also on this website, that the sitemaps do not include individual pages (above the first page obviously).
    When sitemap was an external add-on, it was the case (all the pages were included).

    Basically, a lot of canonical urls are missing from XF sitemaps if they don't include pages ; why is it so ?
  2. Chris D

    Chris D XenForo Developer Staff Member

    Ultimately the benefit of including page URLs is limited. Once a search engine finds the thread, there's plenty of hints from there as to what other content should be crawled, including metadata in the source itself:
    <link rel="next" href="threads/my-forums-getting-lots-of-spam.35195/page-2" />
  3. Weppa333

    Weppa333 Active Member

    Well it's your opinion :)
    I'm not sure many people realized this.

    Also, in the google sitemap tools, you see the difference between "submitted urls" and "crawled urls". With your way of doing things, this tools makes no sense at all anymore since you don't submit every URL.

    I'll edit the sitemap deferred task to include all the pages, and I'm sure many people here expect this behaviour (they can like my post if they do :) )
  4. Chris D

    Chris D XenForo Developer Staff Member

  5. Weppa333

    Weppa333 Active Member

    I suggest we agree to disagree.

    You absolutely do not know how google handles sitemaps. Neither do I.
    What you describe over here is the way google bot works without sitemaps (it discovers page after page), or with a partial sitemap.

    With a full sitemap officially submitted in GWT, it's likely google "jumps" directly to "not indexed content". I could prove this to you but I'm not in the business of creating forum software and I suggest you reproduce this in a lab to make a proper decision.

    Also, as I said, you've got stats at GWT that make no sense with your way of doing sitemaps (which none else does this way)
    And finally, your website looks less "active" to google GWT since your sitemap only changes when a new thread is submitted.

    It's a bad decision.

Share This Page