XML Sitemap for XenForo 1.3 [Not needed, included in 1.4]

Wrestled with this mod quite a bit tonight. I had noticed that my sitemaps appeared to be failing on google webmaster tools for the past few weeks. It would connect, but no pages would be indexed. I tried to download the sitemap data myself via my browser, and it was scrambled. Even after GUNZIP on my PC, it was still encoded in a way that was unreadable. But then I downloaded the same file from the same server, using WinSCP instead of my browser, and the files were perfectly readable. I traced it to mod_deflate on my Apache server; it was gzipping these files twice, and browsers were barfing on it. Once I fixed the mod_deflate to not gzip XML pages, it downloaded fine via browser or WinSCP, and I expect that google will now be happy with it. I think the change I made a few weeks ago wasn't only an upgrade to this mod, but adding some more mod_deflate parameters in my httpd.conf, and it appears to have affected this.

This is different behavior than the vbseo sitemaps on vbulletin, which work perfectly well gzipped twice and then downloaded via browser or WinSCP, so there is something different about the file properties or gzip commands on this side. So - I'd suggest looking into the gzip process to see what might be done to make the files more robust, but also to provide an option to not gzip the files through the mod. Plus, a description about how mod_ deflate / mod_gzip might affect things for some server configs.

Will post up in a few days to confirm that all is well with the Google submissions/processing.
 
  • Like
Reactions: HWS
Wrestled with this mod quite a bit tonight. I had noticed that my sitemaps appeared to be failing on google webmaster tools for the past few weeks. It would connect, but no pages would be indexed. I tried to download the sitemap data myself via my browser, and it was scrambled. Even after GUNZIP on my PC, it was still encoded in a way that was unreadable. But then I downloaded the same file from the same server, using WinSCP instead of my browser, and the files were perfectly readable. I traced it to mod_deflate on my Apache server; it was gzipping these files twice, and browsers were barfing on it. Once I fixed the mod_deflate to not gzip XML pages, it downloaded fine via browser or WinSCP, and I expect that google will now be happy with it. I think the change I made a few weeks ago wasn't only an upgrade to this mod, but adding some more mod_deflate parameters in my httpd.conf, and it appears to have affected this.

This is different behavior than the vbseo sitemaps on vbulletin, which work perfectly well gzipped twice and then downloaded via browser or WinSCP, so there is something different about the file properties or gzip commands on this side. So - I'd suggest looking into the gzip process to see what might be done to make the files more robust, but also to provide an option to not gzip the files through the mod. Plus, a description about how mod_ deflate / mod_gzip might affect things for some server configs.

Will post up in a few days to confirm that all is well with the Google submissions/processing.
Strange, i downloaded the xml.gz file using my browser and i can open it with my win rar.
 
Strange, i downloaded the xml.gz file using my browser and i can open it with my win rar.

Right - mine now can as well; but I had to make sure that mod_deflate / gzip was turned off for XML files for it to work. With it enabled, it fails on http browser downloads. You can check to see whether your sitemap (or any other url on your server) is being sent with gzip enabled or not at this site:

http://www.gidnetwork.com/tools/gzip-test.php
 
Right - mine now can as well; but I had to make sure that mod_deflate / gzip was turned off for XML files for it to work. With it enabled, it fails on http browser downloads. You can check to see whether your sitemap (or any other url on your server) is being sent with gzip enabled or not at this site:

http://www.gidnetwork.com/tools/gzip-test.php
I have tested, i think it is not compressed for my sitemap?

Web page compressed? No
Compression type? none
Size, Markup (bytes) 184
Size, Compressed (bytes) 0
Compression % 0.0
 
Right - that's why you're not seeing an issue with it. But test the rest of your site, and if it is also uncompressed, you may want to look into it. It greatly speeds up page load time, makes the site feel much faster, and can save server resources. Doing it within Apache (or equivalent webserver) is more efficient than doing it within the forum software.
 
You say that's compatible with 1.2 but I've tried to install on my 1.2.5 XF and an error appeared telling me that's compatible only with 1.3...
 
Hi,

it seems like there is no sitemap for the visuals. is this on purpose? and how can i add a visuals sitemap?
 
Top Bottom