XML Sitemap for XenForo 1.3 [Not needed, included in 1.4]

[Stuart Wright]

@Rigel Kentaurus would $ persuade you to make this work on a forum with 20 million posts?

 

[Phases]

Is this the only real sitemap option for xenforo and if so, we may or may not be in that same boat - does this not work for larger boards? Please let me know if there is a known limit. We are about to convert a 6 million post board.

We've already converted in dev so, I can just download this and give it a shot, will do that later today, but in the meantime if there's anything I should know....

Thanks

 

[psx]

There's an error

Error Info:
ErrorException: Undefined index: HTTP_HOST - library/XfAddOns/Sitemap/ControllerPublic/Robots.php:55
Generated By: Unknown Account, Today at 4:57 PM

Stack Trace:
#0 /var/www/forum/library/XfAddOns/Sitemap/ControllerPublic/Robots.php(55): XenForo_Application::handlePhpError(8, 'Undefined index...', '/var/www/forum/...', 55, Array)
#1 /var/www/forum/library/XfAddOns/Sitemap/ControllerPublic/Robots.php(37): XfAddOns_Sitemap_ControllerPublic_Robots->getSitemapLocation()
#2 /var/www/forum/library/XenForo/FrontController.php(337): XfAddOns_Sitemap_ControllerPublic_Robots->actionIndex()
#3 /var/www/forum/library/XenForo/FrontController.php(134): XenForo_FrontController->dispatch(Object(XenForo_RouteMatch))
#4 /var/www/forum/robots.php(27): XenForo_FrontController->run()
#5 {main}

Request State:
array(3) {
["url"] => string(45) "http://XXXXXXXXX.comindex.php?xfa-robots/"
["_GET"] => array(0) {
}
["_POST"] => array(0) {
}
}

 

[ForestForTrees]

Hi @Rigel Kentaurus,

I use both this addon and @Chris Deeming's Add-on Install and Upgrade 1.1.0 and love both of them. Unfortunately, this add-on shows up in the add-on manager as always in need of upgrade. Apparently, there is a very quick fix for this. All you need to do is to change the version string in the add-on XML to match that of the Resource Manager.

References:
http://xenforo.com/community/threads/add-on-install-upgrade.35211/page-26#post-639265
http://xenforo.com/community/threads/add-on-install-upgrade.35211/page-26#post-639268

To some degree this doesn't matter at all because I can just ignore it. But it's a bit like Chinese water torture for everyone who hasn't figured out how to handle it (it took me several weeks ). Would you be up for either releasing a new new version that fixes this (so no one else has to go through the process of debugging/researching it) or committing to maintaining the version string in future releases?

Thanks for a great add-on.

 

[Chris D]

As much as I do feel that where possible the version string in RM should match the version that is installed (it makes more sense on many levels) there are valid cases where this may not happen.

That's why when you click on the update details in my add-on you can click on "Ignore this update" as one of the posts you linked to explains.

 

[ForestForTrees]

Fair enough, Chris. I just thought it was worth informing @Rigel Kentaurus in case there is a quick fix. I've already been using the "Ignore this update" solution.

 

[jauburn]

Google says "Sitemap contains urls which are blocked by robots.txt"

Is there a way for this sitemap plugin not to generate URLs which should not be indexed, like member pages, login, silly stuff?

 

[brookelyn]

I am a bit confused by this sitemap and why my site map folder has multiple sitemaps inside. With vB, I uploaded one sitemap.xml.gz to google webmasters and it constantly updated without me having to do anything more. Are you saying I need to add multiple sitemaps on a regular basis to stay up to date? I have over 1 million posts. Is this the right tool for me?

 

[ForestForTrees]

I am a bit confused by this sitemap and why my site map folder has multiple sitemaps inside. With vB, I uploaded one sitemap.xml.gz to google webmasters and it constantly updated without me having to do anything more. Are you saying I need to add multiple sitemaps on a regular basis to stay up to date? I have over 1 million posts. Is this the right tool for me?

Hi brookelyn, there is one master sitemap that refers to the other sitemaps, so search engines only need to be notified of the master sitemap. This is consistent with the sitemap standard, so it is not a problem. Also, bing and Google are automatically notified whenever a new sitemap is generated, which happens automatically according to XenForo's built in cron too. See Options > Sitemap for Xenforo and Tools > Cron for details. I hope this helps and think that this is a great tool. As far as I can tell, "it just works."

 

[ngannguyen]

Hi,
I used https://www.google.com/webmasters/tools?hl=en&authuser=0 to upload the sitemap, but Google informed: "Your Sitemap does not contain any URLs. Please validate and resubmit your Sitemap."
Please help me to solve this case.
Thanks,
Ngan.

 

[Phases]

Did you upload the xml file or the HTML file? Upload the xml.gz file.

 

[ngannguyen]

Did you upload the xml file or the HTML file? Upload the xml.gz file.

Here are my procedures to upload output files to https://www.google.com/webmasters/tools/sitemap-list?hl=en&siteUrl=:
1. Admincp --> tools --> Cron Entries --> Rebuild sitemap --> Run (I also scheduled to daily run) --> I checked the output files at "public_html/xx/sitemap":

2. https://www.google.com/webmasters/tools --> Crawl --> Sitemap --> Add/Test sitemap --> fill in the URL "http://mysite.com/xx/sitemap" --> Google informed the errors: "Your Sitemap does not contain any URLs. Please validate and resubmit your Sitemap."
I don't know what do next (I use ftp software to check sitemap folder, there are files inside, but if I use browser to access http://mysite.com/xx/sitemap, there are nothing inside)

Brs,
Ngan.

 

[tafreehm]

Here are my procedures to upload output files to https://www.google.com/webmasters/tools/sitemap-list?hl=en&siteUrl=:
1. Admincp --> tools --> Cron Entries --> Rebuild sitemap --> Run (I also scheduled to daily run) --> I checked the output files at "public_html/xx/sitemap":

2. https://www.google.com/webmasters/tools --> Crawl --> Sitemap --> Add/Test sitemap --> fill in the URL "http://mysite.com/xx/sitemap" --> Google informed the errors: "Your Sitemap does not contain any URLs. Please validate and resubmit your Sitemap."
I don't know what do next (I use ftp software to check sitemap folder, there are files inside, but if I use browser to access http://mysite.com/xx/sitemap, there are nothing inside)

Brs,
Ngan.

you have to submit complete filename instead of directory. for example submit: "http://yoursite.com/xx/sitemap/sitemap.threads.1.xml.gz" and all other files

 

[ngannguyen]

OK, let me try again.
Thanks,
Ngan.

 

[jauburn]

If I add the URL to the photopost gallery attached to the forum, will this sitemap index the gallery images?

 

[ngannguyen]

If I add the URL to the photopost gallery attached to the forum, will this sitemap index the gallery images?

Dear Jauburn,
I can not get your points, please make clear...

 

[jauburn]

Dear Jauburn,
I can not get your points, please make clear...

That's okay. Never mind. I see that Photopost includes its own sitemap maker.

 

[Stuart Wright]

I manually ran the process and after about 4 minutes, got this error:
Request Timeout
This request takes too long to process, it is timed out by the server. If it should not be timed out, please contact administrator of this web site to increase 'Connection Timeout'.

Our connection timeout is already set high enough.

The sitemap process should really be a deferred type process so that this doesn't happen.

@Rigel Kentaurus any chance this could be addressed please?

 

[tommydamic68]

@Rigel Kentaurus when going to www.mysite.com/robots.txt i just get directed back to my site. is there something additional i have to do to redirect to the robots.txt, i didn't see it in your original instructions for this add on.

 

[tommydamic68]

@Rigel Kentaurus - can you please explain this? Do i add something to my .htaccess file? I cannot view my robots.txt - im not sure if this add on is creating this problem.

You can have XenForo generate the robots.txt file for you. Please note for this to work, you need to do a mod rewrite that goes:


robots.txt => index.php?xfa-robots/index