Jake B.
Well-known member
library/Zend/Http/UserAgent/Bot.php I believe.
Anything to identify visitor can be spoofed. No responsible or mainstream bot tries to spoof/hide itself.
Where ware the "robots" defind and how? useragent? can be spoofed...
I can put the bad robots in htaccess you are right.
I think you're saying that people could change their UserAgent to match up to a Google bot to get past this limitation. While this is true there really isn't much you can do except possibly block useragents that aren't from a valid IP for that bot, but that would be an extreme amount of maintenance to keep those lists updated compared to the amount of people that would go through the trouble of changing their UserAgent and possibly breaking site functionality on other sites.