Rigel Kentaurus
Well-known member
Moved to the resources section:
http://xenforo.com/community/resources/sitemap-for-xenforo.67/
http://xenforo.com/community/resources/sitemap-for-xenforo.67/
XML Sitemap for XenForo 1.3 [Not needed, included in 1.4]
- Thread starter Rigel Kentaurus
- Start date
Rigel Kentaurus
Well-known member
v1.1
- Updated for XenForo 1.1
- Removed thread redirects from sitemap
- Added configuration options in AdminCP (enable threads, forums, members)
- Added configuration options for total URLs per sitemap, and sitemap directory
- Added the option to include member profile pages in sitemap
- Added to option to ping Google and Bing automatically after generating the sitemap
- Enhanced security, to make sure that only the threads and forums that are public are included
- Generates the Index file for the sitemap
- Generates the sitemap for forums
- Generates the sitemap for threads
- Supports gzip compression of the generated file
Rigel Kentaurus
Well-known member
There are several ways to make Google notice your Sitemap
One is going to Google WebMaster tools, registering your site, and adding your sitemap manually. This works.
http://www.google.com/webmasters/
The other one is to just update the robots.txt entry, and add a line containing Sitemap: url
for example, my robots.txt looks like this
(I guess the other one is just making the Sitemap generator ping Google, but what would be the fun in that?)
One is going to Google WebMaster tools, registering your site, and adding your sitemap manually. This works.
http://www.google.com/webmasters/
The other one is to just update the robots.txt entry, and add a line containing Sitemap: url
for example, my robots.txt looks like this
Code:
Sitemap: http://fanficslandia.com/sitemap/sitemap.xml.gz(I guess the other one is just making the Sitemap generator ping Google, but what would be the fun in that?)
yavuz
Well-known member
These features, I belive would help improve the sitemap:
- Max URLs in Sitemap Files
- Include - Exlude Pages, Member Profiles* etc.
- Sitemap Update Frequency (For elements such as threads, pages, forums etc)
- Be Able to set up sitemap priorities
- Auto-Notify Search Engines
- Be Able to change the sitemap directory.
- Crawler Report. Be able to see the search engine & bot activity.
Some of them can be excluded via robots.txt but it would be nice if we could determin what we like to include or exlude inside de sitemap addon.
This is which I came up to in short notice. Thank you again for this addon Rigel.
- Max URLs in Sitemap Files
- Include - Exlude Pages, Member Profiles* etc.
- Sitemap Update Frequency (For elements such as threads, pages, forums etc)
- Be Able to set up sitemap priorities
- Auto-Notify Search Engines
- Be Able to change the sitemap directory.
- Crawler Report. Be able to see the search engine & bot activity.
Some of them can be excluded via robots.txt but it would be nice if we could determin what we like to include or exlude inside de sitemap addon.
This is which I came up to in short notice. Thank you again for this addon Rigel.
Romchik®
Well-known member
Rigel Kentaurus
Well-known member
You can either wait for Google to crawl your site, or you can submit a sitemap in advance with all the URLs that your site provides (all the threads, forums, etc). Doing a good sitemap can help you get more traffic from search engines. For example, Google had missing 6400 out of 10000 URLs in my sitemap, I can expect eventually to have all of them indexed
Rigel Kentaurus
Well-known member
Rigel Kentaurus
Well-known member
I added some of this
- Max URLs now defaults to 10,000 and can be configured
- There is now an option in the AdminCP to select whether to include threads, forums and members
- Auto-Notify search engines is ready
- Sitemap directory defaults to sitemap, and can be changed (it does not validate that it is writable as of yet, but I assume if someone changes that he knows what he's doing )
I am not sure about the crawler report .. it is way too much work. Especially because from the Google Webmaster Tools console you can see your sitemap, and see the last time it was accesed, parsed, and the number of links that are already in the index. It seems kind of redundant to have it again.
Setting up sitemap priorities is something that I want to look into, probably I'll check that next. I am still not sure whether to give priority to new content (that makes sense), or if for example forums should have higher priority than threads (and higher than member pages), or if all that should just be configurable. I guess I will have to do some research on the best practice.
Kaiser
Well-known member
Oh ok, yea googles always crawling my site
I have no need for it.
yavuz
Well-known member
I added some of this
I am not sure about the crawler report .. it is way too much work. Especially because from the Google Webmaster Tools console you can see your sitemap, and see the last time it was accesed, parsed, and the number of links that are already in the index. It seems kind of redundant to have it again.
- Max URLs now defaults to 10,000 and can be configured
- There is now an option in the AdminCP to select whether to include threads, forums and members
- Auto-Notify search engines is ready
- Sitemap directory defaults to sitemap, and can be changed (it does not validate that it is writable as of yet, but I assume if someone changes that he knows what he's doing
Setting up sitemap priorities is something that I want to look into, probably I'll check that next. I am still not sure whether to give priority to new content (that makes sense), or if for example forums should have higher priority than threads (and higher than member pages), or if all that should just be configurable. I guess I will have to do some research on the best practice.
Amazing, and it such a short notice, I've installed it and it works smoothly. Thanks Rigelfor the hard work.
Weiyan
Well-known member
As I see the generating log, it excluded private forum. Does this means threads and subforums also excluded?
For public forums, does it iterate in the order of latest replied thread first? That means for a very old thread, if there is a reply, then it will appear at the top of the sitemap?
For public forums, does it iterate in the order of latest replied thread first? That means for a very old thread, if there is a reply, then it will appear at the top of the sitemap?
Similar threads
- Question
