• This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.

can i ban these sites

Tracy Perry

Well-known member
#2
they are customizing a lot of resources

Robot: Baidu
Robot: Sogou
If you are using nginx, it's easy enough to do.
Code:
    if ($http_user_agent ~* (Baiduspider|Sogou|Yandex) ) {
        return 444;
    }
placed in the location / segment will drop all connections from the listed robots. It's one of the side benefits of nginx as since the return 444 will allow the connection to be dropped as soon as it's seen to be the listed spider/bot.
Apache does not have the ability to DROP the connection and I'm sure that there are some guides to doing basically the same with Apache that can be found via Google.