• This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.

XF 1.5 Sitemap :no individual pages ?

Weppa333

Active member
#1
Hello

I just noticed on my install, but also on this website, that the sitemaps do not include individual pages (above the first page obviously).
When sitemap was an external add-on, it was the case (all the pages were included).

Basically, a lot of canonical urls are missing from XF sitemaps if they don't include pages ; why is it so ?
 

Chris D

XenForo developer
Staff member
#2
Ultimately the benefit of including page URLs is limited. Once a search engine finds the thread, there's plenty of hints from there as to what other content should be crawled, including metadata in the source itself:
Code:
<link rel="next" href="threads/my-forums-getting-lots-of-spam.35195/page-2" />
 

Weppa333

Active member
#3
Well it's your opinion :)
I'm not sure many people realized this.

Also, in the google sitemap tools, you see the difference between "submitted urls" and "crawled urls". With your way of doing things, this tools makes no sense at all anymore since you don't submit every URL.

I'll edit the sitemap deferred task to include all the pages, and I'm sure many people here expect this behaviour (they can like my post if they do :) )
 

Weppa333

Active member
#5
I suggest we agree to disagree.

You absolutely do not know how google handles sitemaps. Neither do I.
What you describe over here is the way google bot works without sitemaps (it discovers page after page), or with a partial sitemap.

With a full sitemap officially submitted in GWT, it's likely google "jumps" directly to "not indexed content". I could prove this to you but I'm not in the business of creating forum software and I suggest you reproduce this in a lab to make a proper decision.

Also, as I said, you've got stats at GWT that make no sense with your way of doing sitemaps (which none else does this way)
And finally, your website looks less "active" to google GWT since your sitemap only changes when a new thread is submitted.

It's a bad decision.
 

Crazy-Achmet

Well-known member
#6
I know this thread is very old. I've used the external Sitemap AddOn (with all pages of threads) for a long time. I've switched to use the internal sitemap create almost 2 years ago (February 2016). Around that time, i also changed a lot in my navigation.

I've around 65% of traffic in 3 weeks. I paniced and i've undone all the changes (except sitemap cause i've never ever thought this could have caused the problem cause i've thought the internal sitemap create works same as the AddOn) but nothing changed. In december 2016 (still very concerned about the lost traffic) i've stumbled around the sitemap and realized that the new sitemap includes way less URLs then the old sitemap from the backup. I disabled the internal sitemap creation and used the AddOn again. My traffic went up within weeks.

1517131978969.png
Screenshot of searchmetrics SEO visibility of that timeframe

Why am i posting this now? I would like to get rid of the AddOn. Less Addons are better, specially when you have the same functions build in. I would like to include a lot of other pages of AddOns in the sitemap (like Showcase,...) but that's not possible with the external AddOn. At least not for me.

So please consider adding the option to include all pages of a thread (and maybe also forum). In the meantime, maybe there is a developer out there who could create a custom modification for the XF1 sitemap for me. :) This is SOO important for me!
 

dutchbb

Well-known member
#7
I suggest we agree to disagree.

You absolutely do not know how google handles sitemaps. Neither do I.
What you describe over here is the way google bot works without sitemaps (it discovers page after page), or with a partial sitemap.

With a full sitemap officially submitted in GWT, it's likely google "jumps" directly to "not indexed content". I could prove this to you but I'm not in the business of creating forum software and I suggest you reproduce this in a lab to make a proper decision.

Also, as I said, you've got stats at GWT that make no sense with your way of doing sitemaps (which none else does this way)
And finally, your website looks less "active" to google GWT since your sitemap only changes when a new thread is submitted.

It's a bad decision.
This would be a lot of pages to send with a 10mil posts forum for example. Also pages in between are not necessary then, only first and last page i think. Google just needs to detect when a thread has been updated.
 

dutchbb

Well-known member
#8
I know this thread is very old. I've used the external Sitemap AddOn (with all pages of threads) for a long time. I've switched to use the internal sitemap create almost 2 years ago (February 2016). Around that time, i also changed a lot in my navigation.

I've around 65% of traffic in 3 weeks. I paniced and i've undone all the changes (except sitemap cause i've never ever thought this could have caused the problem cause i've thought the internal sitemap create works same as the AddOn) but nothing changed. In december 2016 (still very concerned about the lost traffic) i've stumbled around the sitemap and realized that the new sitemap includes way less URLs then the old sitemap from the backup. I disabled the internal sitemap creation and used the AddOn again. My traffic went up within weeks.

View attachment 167624
Screenshot of searchmetrics SEO visibility of that timeframe

Why am i posting this now? I would like to get rid of the AddOn. Less Addons are better, specially when you have the same functions build in. I would like to include a lot of other pages of AddOns in the sitemap (like Showcase,...) but that's not possible with the external AddOn. At least not for me.

So please consider adding the option to include all pages of a thread (and maybe also forum). In the meantime, maybe there is a developer out there who could create a custom modification for the XF1 sitemap for me. :)This is SOO important for me!
Which add-on are you using?
 

dutchbb

Well-known member
#10
Last edited:
Top