1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

can i ban these sites

Discussion in 'Server Configuration and Hosting' started by aiman.h.kallaf, Aug 3, 2013.

  1. aiman.h.kallaf

    aiman.h.kallaf Active Member

    they are customizing a lot of resources

    Robot: Baidu
    Robot: Sogou
  2. Tracy Perry

    Tracy Perry Well-Known Member

    If you are using nginx, it's easy enough to do.
        if ($http_user_agent ~* (Baiduspider|Sogou|Yandex) ) {
            return 444;
    placed in the location / segment will drop all connections from the listed robots. It's one of the side benefits of nginx as since the return 444 will allow the connection to be dropped as soon as it's seen to be the listed spider/bot.
    Apache does not have the ability to DROP the connection and I'm sure that there are some guides to doing basically the same with Apache that can be found via Google.
    aiman.h.kallaf likes this.

Share This Page