Community Killers to Avoid at All Costs

Some interesting user agents I found today from spammers that signed up
  • Mozilla/1.22 (compatible; MSIE 2.0; Windows 95)
  • (compatible; MSIE 5.01; Windows 95; MSIECrawler)
  • Mozilla/4.0 (compatible; MSIE 5.5; Windows 95)
 
Baidu is one of the most aggressive crawlers on the web, block them by robots.txt if you don't want them scanning your content because they have a vast IP range.
 
Top Bottom