XF 2.0 Restricting Access without affecting Crawl Bots

[ElgrandOC]

At the moment my forum is public. Everyone can see everything.

I am looking to restrict access to my forum in the below manor:
Guests - see nothing
Registered - see something
@10+ posts - see most of everything
Paid - see everything

What I do NOT want is for this to affect Google Bots etc.

If I start changing permissions for user tiers etc (I assume bots are classed as guests/unregistered users), how can I do so and retain the bots crawling the forum?

What I want is for people to be able to search for an answer and see that the answer is on my forum, but for them to then become part of the community, rather than just being able to take what they want without joining in.

Is this possible?

 

[Martok]

You shouldn't restrict your forum to guests and allow access to bots. It is frowned upon by Google, it is considered a violation of their webmaster guidelines.

https://support.google.com/webmasters/answer/66355?hl=en

 

[ElgrandOC]

Okay, so even though the information is still there, and available to anyone who registers, that's still classed as "cloaking"?

I read from that, that the cloaking refers to sending different information to search engines and human users, which in reality I am not doing as that information is there, they just need to register to get it?

 

[ElgrandOC]

Just asked on the Google webmaster forum about this, and was given an example of "Flexible Sampling" which should cover me from accusations of "cloaking"...

Anyone have any experience of this or how I could implement it?

 

[Martok]

I'm sure you've already seen this on "Flexible Sampling" https://support.google.com/webmasters/answer/7532484?hl=en

From that, there's developer documentation: https://developers.google.com/search/docs/data-types/paywalled-content

From what I can see, this isn't something that you are going to be able to just tweak using the core software, I expect an add-on would need to be developed for this. I'm not aware of any add-ons that currently do this (though do check in the resources section).