1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

XML Sitemap for XenForo 1.3 [Not needed, included in 1.4]

Discussion in 'Add-on Releases' started by Rigel Kentaurus, Dec 27, 2010.

  1. Rigel Kentaurus

    Rigel Kentaurus Well-Known Member

  2. Rigel Kentaurus

    Rigel Kentaurus Well-Known Member

    • Updated for XenForo 1.1
    • Removed thread redirects from sitemap
    • Added configuration options in AdminCP (enable threads, forums, members)
    • Added configuration options for total URLs per sitemap, and sitemap directory
    • Added the option to include member profile pages in sitemap
    • Added to option to ping Google and Bing automatically after generating the sitemap
    • Enhanced security, to make sure that only the threads and forums that are public are included
    • Generates the Index file for the sitemap
    • Generates the sitemap for forums
    • Generates the sitemap for threads
    • Supports gzip compression of the generated file
    Core Freedom, eod, moutaz and 3 others like this.
  3. Rigel Kentaurus

    Rigel Kentaurus Well-Known Member

    There are several ways to make Google notice your Sitemap

    One is going to Google WebMaster tools, registering your site, and adding your sitemap manually. This works.

    The other one is to just update the robots.txt entry, and add a line containing Sitemap: url
    for example, my robots.txt looks like this

    Sitemap: http://fanficslandia.com/sitemap/sitemap.xml.gz
    (I guess the other one is just making the Sitemap generator ping Google, but what would be the fun in that?) :)
  4. yavuz

    yavuz Well-Known Member

    Thank God, finally someone made a sitemap addon. I'm not sure why anyone made one until now. Thank you a million Rigel. I'll try it out and get back with the results.
    DRE likes this.
  5. yavuz

    yavuz Well-Known Member

    These features, I belive would help improve the sitemap:

    - Max URLs in Sitemap Files
    - Include - Exlude Pages, Member Profiles* etc.
    - Sitemap Update Frequency (For elements such as threads, pages, forums etc)
    - Be Able to set up sitemap priorities
    - Auto-Notify Search Engines
    - Be Able to change the sitemap directory.
    - Crawler Report. Be able to see the search engine & bot activity.

    Some of them can be excluded via robots.txt but it would be nice if we could determin what we like to include or exlude inside de sitemap addon.

    This is which I came up to in short notice. Thank you again for this addon Rigel.
    Jeffin and Perspektif like this.
  6. Romchik®

    Romchik® Well-Known Member

    Try Unlimited Sitemap Generator.
  7. Kaiser

    Kaiser Well-Known Member

    I have always been confused of what a sitemap is. Anyone care to explain?
  8. Brogan

    Brogan XenForo Moderator Staff Member

  9. Rigel Kentaurus

    Rigel Kentaurus Well-Known Member

    You can either wait for Google to crawl your site, or you can submit a sitemap in advance with all the URLs that your site provides (all the threads, forums, etc). Doing a good sitemap can help you get more traffic from search engines. For example, Google had missing 6400 out of 10000 URLs in my sitemap, I can expect eventually to have all of them indexed
  10. DSF

    DSF Well-Known Member

    The minimun request for this nice add-on is to use the guest-setting.
    Sorry, but I will not public all the hidden forums like as admin, mod and special users.
    Andy.N and ragtek like this.
  11. Rigel Kentaurus

    Rigel Kentaurus Well-Known Member

    This should be taken care of now, it kind of was in my to-do list. The sitemap will only include threads and forums that are visible as a guest.
    DSF likes this.
  12. Rigel Kentaurus

    Rigel Kentaurus Well-Known Member

    I added some of this
    • Max URLs now defaults to 10,000 and can be configured
    • There is now an option in the AdminCP to select whether to include threads, forums and members
    • Auto-Notify search engines is ready
    • Sitemap directory defaults to sitemap, and can be changed (it does not validate that it is writable as of yet, but I assume if someone changes that he knows what he's doing :))

    I am not sure about the crawler report .. it is way too much work. Especially because from the Google Webmaster Tools console you can see your sitemap, and see the last time it was accesed, parsed, and the number of links that are already in the index. It seems kind of redundant to have it again.

    Setting up sitemap priorities is something that I want to look into, probably I'll check that next. I am still not sure whether to give priority to new content (that makes sense), or if for example forums should have higher priority than threads (and higher than member pages), or if all that should just be configurable. I guess I will have to do some research on the best practice.
    laztrix likes this.
  13. Kaiser

    Kaiser Well-Known Member

    Oh ok, yea googles always crawling my site :) I have no need for it.
  14. DSF

    DSF Well-Known Member

    Is it a good idea to zip the sitemap's?
  15. Weiyan

    Weiyan Well-Known Member

    Its better to configure which forum should appear in the sitemap. For some post, I like the user can search it, but not allowed to view untill they registered.
  16. yavuz

    yavuz Well-Known Member

    Amazing, and it such a short notice, I've installed it and it works smoothly. Thanks Rigelfor the hard work.
    Techhubsolutions likes this.
  17. Puntocom

    Puntocom Well-Known Member

    Thanks for the mod. When I click Run I get this page (black rectangle added by me to hide url)


    Anyway it works fine!
  18. fattony69

    fattony69 Well-Known Member

    I was waiting for this! Very awesome. Thank you very much!
    Techhubsolutions likes this.
  19. Weiyan

    Weiyan Well-Known Member

    After half day's work on finding bug of other mod, install this mod. Since other use tested it, I feel confidence to use it.
    Techhubsolutions likes this.
  20. Weiyan

    Weiyan Well-Known Member

    As I see the generating log, it excluded private forum. Does this means threads and subforums also excluded?
    For public forums, does it iterate in the order of latest replied thread first? That means for a very old thread, if there is a reply, then it will appear at the top of the sitemap?
    dathoquoc1912 likes this.

Share This Page