1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

XF 1.4 Blocking bots by user agent

Discussion in 'XenForo Questions and Support' started by ophy, Apr 13, 2015.

  1. ophy

    ophy Member

    I used to have an add on that allowed me to input bots and it would block them directly from the software from accessing the site. This made the stress on the server much much less and I was wondering how I could accomplish this on Xenforo? I would like to do this manually, could I do this with robots.txt for example? This is an example of my spider list that I used to block, it's 50+ spiders in the blocklist overall that I had

    Code:
    SemrushBot
    SeznamBot
    http_requester
    BIXOCRAWLER
    KomodiaBot
    QACC
    Plukkie
    botje
    'Mozilla
    CompSpyBot
    ScreenerBot
    Chrome/15
    python-requests
    ZumBot
    
     

Share This Page