Not a bug Google can't index consent protected content

[Kirby]

Affected version

2.2.12

I am not 100% sure, but as far as I can see Google (or any other crawlers) are not able to view / index consent protected content (like embedded videos) if advanced cookie consent is being used:

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.

Accept third party cookies

In most cases that I have looked at we can crawl and index the content normally.
So waht is important for us with any kind of interstitial or anything taht you use as a pop up or as a banner on a page, is that he HTML version of the page, when we crawl it we can find the information that you whnt to have indexed for that page

Unfortunately, this Google requrement is not a case for the implementation XenForo choose - if the user has not consented to 3rd party cookies the HTML does not contain the embedded content (like a YouTube Video) but instead only the following message:

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.
Accept third party cookies

Googlebot (or other crawlers) most certainly won't have the necessary xf_consent cookie as getting this cookie requires the client to perform a XHR POST (and at least I have never seen Googlebot perform a POST?).

There also doesn't seem to be an override so crawlers are excempted from cookie consent.

This not only causes embedded content to not be indexed (which IMHO shouldn't be a that big issue), it also causes duplicate shingles (the above message) on many pages - which migh thave a negative impact on SEO (keyword densitiy?).



This must be changed so that either
The full HTML for embedded content is included in the HTML but not active until consent is given (Preferred)
or
There is an override so crwalers are not asked for consent at all and just get the full HTML

 

[Kirby]

This should probably be marked as "Not a bug" (after some testing); in fact the content is included with a template tag, so Google might be able to index the content (not sure if this is being done though).