XF 1.5 XML Sitemap generator - including non-guest pages

Zimply

Member
Lately I found hundreds of link in the Google Webmaster Tools report showing pages which are blocked for unregistered users but included in the sitemap.

I am not technical enough to know if its a sitemap generation issue OR I need some other steps to block these URLs from robots.txt.

The links are all pointing to the front-end page in this URL structure:
forum.com/threads/thread-title.123/edit-prefix

The edit-prefix page is only accessible to logged in users who have permission to edit thread prefixes. But, google is trying to view these pages and reports a permission error warning.

To check this on your forum, try to view a thread as a GUEST and add this at the end of the URL /edit-prefix and view the page, you should most likely get a login page to continue.

If this is not the sitemap issue, sorry for posting here, please move my thread to appropriate location.
And if anyone has an idea how I can block robots.txt from accessing just the /edit-prefix type of URLs please advice.

Thanks everyone
 
/edit-prefix is not actually a URL we use in the core software so I think this must be coming from an add-on. I've also checked your sitemap, and there are no URLs there with edit-prefix, so that is nothing to do with the sitemap generation.

All thing's considered, it would probably be best to identify which add-on that URL belongs to and speak with the author of that first of all.
 
@Chris D Thanks for the prompt reply, sorry if this is the wrong section to have posted this.
I've isolated the culprit add-on causing the issue. Prefix Actions add-on by Snog is creating this form on the front-end to change a post's prefix. I will get in touch with the developer to see what can be done.

Thanks and appreciate it!
 
Back
Top Bottom