Sitemap for 2nd and next pages too

For the most part, generating sitemap entries for additional pages is pretty much superfluous. One objective of the sitemap is to tell search engine's where your important pages are, even if they aren't sufficiently linked together. The kind of pages that Google might have difficulty in finding via crawling in the normal way.

With that in mind, with the way content is structured in forums, sitemaps probably have limited benefit anyway, though they are certainly more useful for a new site or a site that has just migrated from somewhere else.

Though, forgetting that point, if Google can find your threads (be that via crawling or via sitemap submission) then it should be able to very easily find each page of the thread. There's plenty of metadata in the HTML and links to find it.
 
I think you might be missing an important factor: the last page of a thread has the newest posts. By including last pages you inform google that threads have new content, which is an important purpose of sitemaps. Because it shows the need to crawl again. And because fresh content is an important ranking signal.
 
Chris D said:
. One objective of the sitemap is to tell search engine's where your important pages are, even if they aren't sufficiently linked together. The kind of pages that Google might have difficulty in finding via crawling in the normal way.
Click to expand...

Not important pages. That is true:
The kind of pages that Google might have difficulty in finding via crawling in the normal way.
Click to expand...

So what is better to find for Google? The first site of a thread or page 19?
It is very often that is not the best answer to a question on the first page. That's the reason why all pages should be in the sitemap.
 
-It can be an option with checkbox for all threads pages, so your costumers can choose to enable it or not.

More than 50% of the content on many boards are in this pages (2,3,4...)
so it is a big +1 for me it have to be an option in core
 
I am testing this now because page 300, 500 or 800 of a thread is a deeplink and probably harder to get for google. Could be some interesting stuff on it to index. For large forums it could be a plus and easy to implement.

With 9mil posts on my forum the xf sitemap contains 270K threads and 190K of those indexed. With all pages of threads included 580K are send, so not much more, and 240K indexed. This means 50K pages of threads are already indexed and 340K not.

I am sending this in sitemaps of 10k a piece, makes it easier to open/check. After a few days i am seeing a small increase indexed pages, after a few weeks i will have more relevant data.

Interesting thread
xenforo.com

XF 1.5 - Sitemap :no individual pages ?

Hello I just noticed on my install, but also on this website, that the sitemaps do not include individual pages (above the first page obviously). When sitemap was an external add-on, it was the case (all the pages were included). Basically, a lot of canonical urls are missing from XF sitemaps...
xenforo.com xenforo.com
 
Last edited:

Similar threads

bzcomputers
  • Suggestion Suggestion
Lack of interest Add ability to limit the number of tag pages created and added to sitemap
Replies
1
Views
622
RichDevman
RichDevman
M
  • Suggestion Suggestion
Add 'Previous' and 'Next' title attributes to XFMG media item < and > links
Replies
0
Views
481
marquisite
M
bzcomputers
Not a bug Incorrect og:url and lack of sitemap inclusion of canonical url for secondary thread pages - affecting SEO
Replies
1
Views
644
Mike
Mike
bzcomputers
  • Suggestion Suggestion
Lack of interest Reduce likelihood of sites being banned for serving ads on inaccessible and no content pages
Replies
4
Views
1K
djbaxter
D
Alpha1
Awaiting feedback Pagination appears when there is no content to see. And empty pages for ignored content
Replies
2
Views
2K
sHORTYWZ
S
Back
Top Bottom