XF 2.2 GSC only reads in the first 9 sitemap files...

[benFF]

Bit of an odd one this, but it's basically as the title says.

My total sitemap size is about 730,000 pages - so this is split up into 15 sitemap files (sitemap-1.xml > sitemap-15.xml) however when Google search console reads in the main sitemap.xml index file, it only picks up the first 9 - totalling 450,000 pages.

There doesn't seem to be any rhyme or reason for this - just wondered if anyone else had come across this before or any ideas how to solve it?

I did try adding the sitemap again as the .php version just to see if that made a difference, but nope, still just the first 9...

(And yes, you can access the sitemap files manually)

 

[Chris D]

There have been a few mentions of sitemap related issues from a few customers, some of which are noted in "Similar threads" at the bottom of this page.

So I don't think you're alone in this but no one really knows why this would be the case. The sitemap is produced according to the official specification, and Google's own documentation. It very much seems to me like there may be issues on Google's side processing these. It doesn't appear to be exclusive to people who have multiple pages of sitemaps either.

If the sitemaps are accessible in the browser then from a software perspective we've done all we can.

Hopefully this can be reported to Google for them to provide comment.

 

[benFF]

Thanks Chris - I did have a look at existing but couldn't see anything really similar.

As a test, I've submitted the missing sitemaps manually - I'll give it a day or so for them to be accepted (or not) and report back

 

[nodle]

@benFF you can check this thread:

Not a bug - Sitemap Problem?

To be clear, the only URL that needs to be submitted as a sitemap is sitemap.xml I would love to but Google will not allow it, so what do we do now?

xenforo.com

There is something strange going on with Sitemaps, I am now sitting on day 24 and Google still won't read it. It flat out refuses to take the .XML version at all. The .PHP version will accept but won't read it after accepting it. I have everything proper in my robots and according to my sitemap logs it is creating the sitemaps properly. It's like Google is flat out rejecting Xenforo's sitemap now.

 

[Doughtyorg]

Thanks Chris - I did have a look at existing but couldn't see anything really similar.

As a test, I've submitted the missing sitemaps manually - I'll give it a day or so for them to be accepted (or not) and report back

I've had the exact same issue as you and would love to know the results We have 30 sitemaps and only 9 showing

 

[benFF]

Thanks Chris - I did have a look at existing but couldn't see anything really similar.

As a test, I've submitted the missing sitemaps manually - I'll give it a day or so for them to be accepted (or not) and report back

Ok reporting back.

Manually submitting them as single sitemaps didn't work either - Google just gave the same don't fetch error.

However, I was pondering this again today and wondered if it really could be just the fact the number has 2 digits.

So easy test, convert everything over 9 into hex (10 = A, 11 = B, 12 = C etc) and submitted again - low and behold it works!

Now I can submit the last few sitemaps.

I didn't want to muck around with writing an addon for this, so a simple htaccess rule can automate this (and I didn't want to write a map either, coz it's just 5 lines)

# Sitemap rewrite for 10+
RewriteRule ^sitemap-a\.xml$ https://YOURSITEURL/sitemap-10.xml [NC,L,P]
RewriteRule ^sitemap-b\.xml$ https://YOURSITEURL/sitemap-11.xml [NC,L,P]
RewriteRule ^sitemap-c\.xml$ https://YOURSITEURL/sitemap-12.xml [NC,L,P]
RewriteRule ^sitemap-d\.xml$ https://YOURSITEURL/sitemap-13.xml [NC,L,P]
RewriteRule ^sitemap-e\.xml$ https://YOURSITEURL/sitemap-14.xml [NC,L,P]

@Chris D - any idea why this might work?

 

[Evan Burck]

Ok reporting back.

Manually submitting them as single sitemaps didn't work either - Google just gave the same don't fetch error.

However, I was pondering this again today and wondered if it really could be just the fact the number has 2 digits.

So easy test, convert everything over 9 into hex (10 = A, 11 = B, 12 = C etc) and submitted again - low and behold it works!

Now I can submit the last few sitemaps.

I didn't want to muck around with writing an addon for this, so a simple htaccess rule can automate this (and I didn't want to write a map either, coz it's just 5 lines)



View attachment 280159

@Chris D - any idea why this might work?

Is there a... "how to automate this for dummies" version of this somewhere? Would this be something I'd add to an htaccess file or something along those lines?

 

[benFF]

Is there a... "how to automate this for dummies" version of this somewhere? Would this be something I'd add to an htaccess file or something along those lines?

Hey Evan,

Yup, those are just lines in my site roots htaccess file.

 

[dethfire]

I've read on some reddits that this can happen with high processing dynamic sitemaps. @Jeremy P

I've been plagued by this issue for a couple of years and tried this method. It's a bit strange. The main screen still says "couldn't fetch" but if I click into the sitemap it returns the URL count successfully and says it's a success. wtf is going on?