1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

XF 1.4 Blockng Spiders/Robots

Discussion in 'Styling and Customization Questions' started by New Joe, Oct 13, 2014.

  1. New Joe

    New Joe Active Member

    What do you guys use to block them?
    The Baiduspider is one of the ones I want to stop
  2. Mouth

    Mouth Well-Known Member

    try "robots.txt" in search for lots of topics on this question.
  3. iGloo'

    iGloo' Member

    As Mouth said, you just need to create a "robots.txt" and edit it as you want. You can find good informations here or on Google...
    Every website should have a "robots.txt". :D

Share This Page