1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Be wary of IP banning users...

Discussion in 'General XenForo Discussion and Feedback' started by Chris D, May 19, 2013.

  1. Chris D

    Chris D XenForo Developer Staff Member

    We don't make a habit of IP banning users. Mainly because it's too easy to circumvent. But, we occasionally do. I need to speak to the Admin involved to rationalise the reason behind the action, but recently we IP banned a user.

    It's not usually a problem though.

    So I get alerted to the fact today that our site is pretty much invisible to Google. It's nowhere to be seen in search results.

    We find hundreds of thousands of 403 errors on Google Webmaster Tools and we investigate a bit further.

    Turns out that one of our IP bans (the aforementioned user) had been spoofing his IP address. The IP address he was spoofing with was Google's.

    We had IP banned one of Google's crawlbots.

    This has now been rectified. And we will never ban an IP address again.

    Be wary :)
     
    Alfa1, Lisa and Veer like this.
  2. FredC

    FredC Well-Known Member

    What am i missing? I fail to see how that couldnt have been avoided with a little homework before the hammer came down on that particular IP? Sorry serious question. I guess ive always made a habit of fully investigating any IP that comes with a bad apple. But then as a disclaimer my site doesn't get hundreds of registrations a week either. And even though i have a large membership the active user base has never been more then a hundred or two at any given point in its history.. Making those couple extra queries not to troublesome.
     
  3. Chris D

    Chris D XenForo Developer Staff Member

    I couldn't agree more.

    I'm pretty sure we briefed the moderators to not use IP banning or discouragement in the first place which is why I need to work out why that decision was made and ensure we don't make that mistake again.
     
  4. Biker

    Biker Well-Known Member

    Weary or wary? ;)
     
    Chris D likes this.
  5. Chris D

    Chris D XenForo Developer Staff Member

    Doh.

    Requested it to be changed, thanks.
     
  6. Biker

    Biker Well-Known Member

    Although, I get tired (weary) of banning them as well. :D
     
  7. AlexT

    AlexT Well-Known Member

    Uhm. How would it be possible to spoof a Google IP? You don't just "spoof" an IP address unless you somehow gained access to a Google machine and/or datacenter (which I doubt happened here). You can fake IP addresses in certain http headers, which, however should never be used for anything such as banning (and I doubt xenForo uses them in such a way). So whatever happened, to me it looks more like one of your team members accidentally added the wrong IP to the ban.
     
  8. Tracy Perry

    Tracy Perry Well-Known Member

    I guess it could be both?
    Being wary to not ban an IP, and weary from banning a wrong one and then having to figure out why Google is getting so many 403 errors. :p
     
  9. Mike

    Mike XenForo Developer Staff Member

    What AlexT said.

    Unless you're setting the REMOTE_ADDR to something like HTTP_X_FORWARDED_FOR -- that's actually the reason we don't do anything like that. The proxy IP should only be trusted in very specific scenarios. The REMOTE_ADDR is the actual address that did the TCP handshake so, as far as I understand, it can't be spoofed (unlike UDP traffic, hence things like DNS reflection attacks).
     
    tenants likes this.
  10. Chris D

    Chris D XenForo Developer Staff Member

    One way or another, it definitely happened. This was not human error.

    We IP banned the user. The user had Google's IP address.

    Here's a look up of the IP address:
    [​IMG]

    Here's a look at the user's IP addresses:
    [​IMG]

    And here's a ping resolving to the hostname:
    [​IMG]
     
  11. AlexT

    AlexT Well-Known Member

    My guess would be some addon messing with your user sessions or the xF IP Model.
     
  12. Biker

    Biker Well-Known Member

    I've seen similar things with spoofed Google headers. For example:

    95.211.77.144 # lfd: (sshd) Failed SSH login from 95.211.77.144 (NL/Netherlands/crawl-66-249-71-83.googlebot.com): 5 in the last 300 secs - Thu Nov 17 16:46:32 2011

    Chris, have you checked your server logs to see if there's something similar?
     
    tenants likes this.
  13. erich37

    erich37 Well-Known Member

    is it possible to "whitelist" IP's of crawlers?
    So that those will not get banned, even if you ban it in ACP ?
     
    CyclingTribe likes this.
  14. Adam Howard

    Adam Howard Well-Known Member

    You don't need to spoof Google. You can use Google as a "poor mans proxy". The same IP address uses for Good Translate is also rotated for other services including Google Bot.
     
  15. Adam Howard

    Adam Howard Well-Known Member

    I didn't clear my cookies or flush out my dns and a few other things I could have done before doing this "demo"

    But I'm fairly sure if @Mike checks the IP address on this post, it would match Google's

    And I would have to do is this
    http://www.google.com/translate?hl=&sl=en&tl=nl&u= Change domain name

    You can do the same thing with Yahoo the last I checked.
     
  16. Adam Howard

    Adam Howard Well-Known Member

  17. AlexT

    AlexT Well-Known Member

    Not true.

     
    SneakyDave likes this.
  18. Adam Howard

    Adam Howard Well-Known Member

    Disagree based upon actual use.

    We banned the IP address used for Google Translate for this very reason. Doing so also banned Google Bot.

    Your source is also dated 2006 .... 7 years ago.
     
  19. AlexT

    AlexT Well-Known Member

    This is the reverse DNS - it resolves an IP address to a domain name. That domain name can be freely chosen, thus faked (not spoofed). To verify that it is not a fake domain name, you'd have to try FCrDNS verification. See here:

    http://en.wikipedia.org/wiki/Forward_Confirmed_reverse_DNS
     
  20. AlexT

    AlexT Well-Known Member

    Sorry for quoting an outdated source (who the heck is Matt Cutts anyways?). You banned "the IP address"? :) Well, there might be more than one IP address. And Google doesn't publish the IP addresses/ranges for their services (since they can change at any time). And I say it again: Googlebot and Google Translate do not share IP addresses, simple because otherwise their recommended way (rDNS on the IP) of verifying Googlebot wouldn't work.
     
    SneakyDave likes this.

Share This Page