Fixed IndexNow: link to content restricted to guests being sent out

[Chromaniac]

Affected version

2.3 Beta 5

Was going through IndexNow panel on Bing Console and noticed that it was getting profile post links from XenForo. The one I noticed was for a comment on a profile post. Now the user profiles are blocked from guest usergroup at my end so maybe they should be filtered out based on the same settings that are used for sitemap files?

Also, XenForo is sending post links which a lot of us block through robots.txt file (Disallow: /community/posts/ is part of robots.txt file here.). Should this be removed from Robots file by admins who want to benefit from this functionality? I suppose they would just follow the redirect and reach the actual thread page which should not be blocked? Google seems to suggest that content restricted by robots.txt can get followed by their spiders so maybe Bing is the same.

I also decided to check another thing based on above findings. Made a post in a node restricted to guests. And XenForo sent the post link of that to IndexNow as well.

 

[XF Bug Bot]

Thank you for reporting this issue, it has now been resolved. We are aiming to include any changes that have been made in a future XF release (2.3.0 Beta 6).

Change log:

Only queue content for IndexNow if …

There may be a delay before changes are rolled out to the XenForo Community.