XF 1.5 Crawler Errors

[thenashy]

I've been getting this issue with Google recently:



How do I stop Google looking for these, as they don't seem to be in use on our site at all.

This is the address to the forum referenced above:



Thanks in advance for any assistance.

 

[Ozzy47]

Do you have any type of redirects in your .htaccess file or any route filters set up?

 

[thenashy]

Do you have any type of redirects in your .htaccess file or any route filters set up?

I wouldn't have edited anything myself, so I'm not really sure. There are certainly re-writes from XF, my htaccess is below.

RewriteEngine On
RewriteCond %{SERVER_PORT} 80
RewriteRule ^(.*)$ https://broncoshq.com/$1 [R,L]
RewriteRule ^[^/]+/([0-9]+)-[^\.]+\.html$ /showthread.php?t=$1 [R=301,L]

#    Mod_security can interfere with uploading of content such as attachments. If you
#    cannot attach files, remove the "#" from the lines below.
#<IfModule mod_security.c>
#    SecFilterEngine Off
#    SecFilterScanPOST Off
#</IfModule>

ErrorDocument 401 default
ErrorDocument 403 default
ErrorDocument 404 default
ErrorDocument 405 default
ErrorDocument 406 default
ErrorDocument 500 default
ErrorDocument 501 default
ErrorDocument 503 default

<IfModule mod_rewrite.c>
    RewriteEngine On

    #    If you are having problems with the rewrite rules, remove the "#" from the
    #    line that begins "RewriteBase" below. You will also have to change the path
    #    of the rewrite to reflect the path to your XenForo installation.
    #RewriteBase /xenforo

    #    This line may be needed to enable WebDAV editing with PHP as a CGI.
    #RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]

    RewriteCond %{REQUEST_FILENAME} -f [OR]
    RewriteCond %{REQUEST_FILENAME} -l [OR]
    RewriteCond %{REQUEST_FILENAME} -d
    RewriteRule ^.*$ - [NC,L]
    RewriteRule ^(data/|js/|styles/|install/|favicon\.ico|crossdomain\.xml|robots\.txt) - [NC,L]
    RewriteRule ^.*$ index.php [NC,L]
</IfModule>