Partial fix Google search console: sitemap pending due to query string param

[dutchbb]

If i am correct this should be the new files. Backup FrontController.php and Sitemap.php first. SitemapRenderer.php is a new file.

Put FrontController.php and SitemapRenderer.php in library/XenForo

And sitemap.php in index. Then rebuild sitemap i presume.

Edit: sitemap.php file updated.

 

[dutchbb]

And it works! Sitemap has been accepted by google webmaster. Nothing is pending anymore.

Edit: Sitemap.php was not working at first, but now it does with new file.

 

[Mike]

sitemap.php should still work -- you'll need to resubmit the sitemap for it to get picked up, but I didn't change sitemap.php to sitemap.xml here and it resolved the issue immediately. It appears to just be the query string part.

 

[ehd]

My sitemap.php is empty, too. But changing to sitemap.xml is not the worst...

WebmasterTools has already got all new sitemaps.

 

[Mike]

Ahh the sitemap.php from @dutchbb is missing code from the diff.

 

[dutchbb]

Ahh the sitemap.php from @dutchbb is missing code from the diff.

Ok was thinking those should be put somewhere.

 

[AzzidReign]

Looks to be working on my end! Thanks for the fix!

 

[CTXMedia]

Just updated, rebuilt and it's working for CycleChat.

Thanks Mike. Much appreciated.

Will this be added to the next XF1 update or will it need reapplying manually for the time being until you find a 'catchall' fix?

 

[Robru]

Is there any solution for XF2.0?

 

[cosmo]

Google search console has accepted the sitemaps. Thanks
But google still can not index the urls and 14 sitemaps have pending status now

 

[dutchbb]

Google search console has accepted the sitemaps. Thanks
But google still can not index the urls and 14 sitemaps have pending status now

View attachment 161116View attachment 161117

Also no indexing here, but could take a few days? On the other hand most url's in it should be indexed already and google shows this pretty fast.

 

[CTXMedia]

The important thing is that they are no longer "Pending" - Google can see the total number of URLs/Images etc. and will presumably update the numbers in a few days time.

 

[cosmo]

After 24 hours not even 1 url is indexed. That's not normal.

@ Mike
I would like to split the sitemaps in 20,000 or 30,000 urls like vB

Maybe 50,000 urls is too big for google?

 

[Chris D]

Google follows this spec. 50,000 is fine and there doesn't need to be a configuration option:

https://www.sitemaps.org/faq.html#faq_sitemap_size

 

[Sim]

Ahh - that's much better!



... this had been frustrating me for a while and I couldn't work out what the problem was.

None of my other sites had problems (less than 50,000 pages), so I didn't even think it might have been a XenForo issue.

 

[Kevin]

Another one here with the problem, the files by @Mike & @dutchbb took care of it.

 

[Mouth]

After successfully applying the .diff and using cron to Rebuild Sitemap, should the resulting .xml's be giving 404 errors?

sitemap.php:

<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<sitemap>
<loc>https://mysite.com/sitemap-1.xml</loc>
[...]

https://mysite.com/sitemap-1.xml:
404 Not Found

Friendly URL's is active. /admin.php?logs/sitemap shows successful rebuild after applying .diff. No sitemap*.xml files exist within XF root dir.

 

[Mike]

I visited your site and saw the 404, which is coming from Nginx. You likely have a rule in your Nginx config that is conflicting. (Nginx only matches 1 location block per request.)

 

[Mouth]

You likely have a rule in your Nginx config that is conflicting.

Ahh, yes. Thanks. Had to ...

#       location ~*  \.(jpe?g|png|gif|svg|svgz|webp|css|js|ico|swf|flv|xml|pdf)$ {
        location ~*  \.(jpe?g|png|gif|svg|svgz|webp|css|js|ico|swf|flv|pdf)$ {
                expires 30d;
                add_header Pragma public;
                add_header Cache-Control "max-age=2592000, stale-while-revalidate=86400, stale-if-error=604800, public, must-revalidate, proxy-revalidate, immutable";
                add_header 'Access-Control-Allow-Origin' '*';
                add_header Strict-Transport-Security $hsts_header always;
                access_log off;
                log_not_found off;
                gzip on;
                gzip_static on;
        }

 

[dutchbb]

Still no pages indexed from the sitemap, so that is a little but strange.