Lack of interest Better Sitemap Magement

[BT10]

While it's great that XenForo includes the ability to build and submit forum's sitemap, I think improvements can be made. I recently came across this article by Oliver H.G. Mason, where he talks about optimizing his XML Sitemap.

A great read from Oliver H.G. Mason on his findings on splitting XML sitemaps into smaller chunks. Instead of a few XML Sitemaps capped at 50,000 URLs the magic number Google recommends, he decided to go with many XML Sitemaps containing 10,000 URLs.

Some SEOs recommend 1,000 URL Sitemaps so they can get around Search Console limits so it can be used to monitor which URLs aren’t getting indexed.

I have found that limiting sitemaps to only 10,000 URLs leads to more thorough levels of indexing. I’m not sure why – I suspect that smaller lists of URLs are easier for Google to process and crawl – but it’s been proven time and again that smaller sitemaps lead to higher degrees of indexing.

To my knowledge, in XenForo, all of our URLs are being included in a single sitemap.php file.

If you are an owner of a big forum, how do you manage your sitemap?

Suggestion: each forum and nodes will have its own sitemap link?

 

[markoroots]

Lack of interest - Better Sitemap Magement

While it's great that XenForo includes the ability to build and submit forum's sitemap, I think improvements can be made. I recently came across this article by Oliver H.G. Mason, where he talks about optimizing his XML Sitemap. A great read from Oliver H.G. Mason on his findings on splitting...

xenforo.com

Hi Arn, can you please tell us what change and where, to do that?

 

[arn]

Hi Arn, can you please tell us what change and where, to do that?

it wasn’t a setting change. I made a script that generated and split the list of threads into multiple site maps.

https://forums.macrumors.com/mr-sitemap/sitemap-threads-index.xml

our indexing increased but I dont think it helped our google traffic so not sure it’s definitely worth pursuing.

arn

 

[loadcell]

You really believe that Google's crawlers need help from webmasters to discover their pages these days? )
They start watching you as soon as you register a new domain. ))
I have a site with 20k pages. Without a sitemap at all. All the pages are listed.

Yeah, the crawlers will follow the links. The only reason to have a site map is for SEO value. Some say your site gets a mark against it for not having a site map. It’s also nice to tell them to not index or crawl specific pages.

 

[arn]

so I think it did bump up our index numbers, but didn't help our traffic. I'm gonna end this experiment because I want to make more changes to our sitemap, and prune it more. But I do think it increased our indexing.

View attachment 231768

fwiw we are at 1.6 million valid pages now. I kept the split site map. Cant say it did it or not. But fyi

 

[nrep]

fwiw we are at 1.6 million valid pages now. I kept the split site map. Cant say it did it or not. But fyi

Looks like an interesting experiment! Would you consider releasing the script for this please, as it would be interesting to see if others can replicate this.