XF 1.4 SiteMap

[New Joe]

I was just wondering how does the Sitemap work?

Does it just do everything automatically?
Nothing to set up?

 

[Paul B]

You need to add any custom URLs as required.
Also, if you block access to member profiles using robots.txt/permissions, uncheck the Users option.

Otherwise there's nothing you need to do.

 

[New Joe]

You need to add any custom URLs as required.

Such as what?

using robots.txt/permissions

I'm interested in this, what kind of things do you add in a robots.txt file?

 

[Paul B]

Such as what?

Anything custom other than nodes, threads and users.

what kind of things do you add in a robots.txt file?

https://xenforo.com/robots.txt

 

[New Joe]

https://xenforo.com/robots.txt

Why would you disallow posts?
Disallow: /community/posts/

 

[Paul B]

https://xenforo.com/community/threads/posts-urls-being-indexed-by-google.48729/page-2#post-660930

 

[Mouth]

Couple of questions ...
1. Does sitemap refer to /robots.txt and not include URL's that have been disallowed within robots.txt?
2. Is there a method to not have page nodes appear within sitemap, when you're not showing them on your site?

Thanks.

 

[Paul B]

There is no dynamic link between robots.txt and the sitemap.
If you block something in robots.txt then you shouldn't add it to the sitemap.

when you're not showing them on your site

Can you clarify?
If there is no view permission then they won't be indexed.

 

[Mouth]

Can you clarify?
If there is no view permission then they won't be indexed.

I have a page node, with no HTML, that is a parent node/category. It's permissions are set/ticked for it to be a private node. When I check Analyze Permissions on the node for guests and a member, View Node is No. Yet, the URL is still be included in the sitemap.

 

[Paul B]

Do you have any add-ons installed which may be interfering?

Private nodes aren't indexed.

 

[tommydamic68]

If I changed my URL to https: - should I also change my sitemap info in my robots.txt?

 

[tommydamic68]

If I changed my URL to https: - should I also change my sitemap info in my robots.txt?

Never mind, I guess you would.