1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

VPS Compromised?

Discussion in 'Server Configuration and Hosting' started by silence, Aug 2, 2014.

  1. silence

    silence Well-Known Member

    Alright so my VPS started randomly issuing attacks on IPs and I couldn't figure out where it was coming from.
    My host has disabled us for the second time and I really need some insight on what I can do in order to resolve this.
    I did malware scans with ClamAV and chkrootkit but found nothing.

    The first notice I got was the following:
    Code:
    Your VDS has been running large DoS attacks, containing junk packets consistent with a rudimentary attack tool. The most recent one:
    
    13:06:37.512586 IP (tos 0x0, ttl 64, id 6096, offset 0, flags [DF], proto UDP (17), length 795)
        xx.xx.xx.xx.33118 > 114.141.72.225.80: UDP, payload 767
    0x0000:  4500 031b 17d0 4000 4011 8959 c0df 1a5b  E.....@.@..Y...[
    0x0010:  728d 48e1 815e 0050 0307 7b49 457a 031a  r.H..^.P..{IEz..
    0x0020:  0000 4000 ff11 e1af c0df 1a5b 728d 48e1  ..@........[r.H.
    0x0030:  0000 0050 0306 62e9 0000 0000 0000 0000  ...P..b.........
    0x0040:  0000 0000 0000 0000 0000 0000 0000 0000  ................
    0x0050:  0000                                     ..
    13:06:37.512615 IP (tos 0x0, ttl 64, id 51589, offset 0, flags [DF], proto UDP (17), length 797)
        xx.xx.xx.xx.33118 > 114.141.72.235.80: UDP, payload 769
    0x0000:  4500 031d c985 4000 4011 d797 c0df 1a5b  E.....@.@......[
    0x0010:  728d 48eb 815e 0050 0309 7b47 457a 031c  r.H..^.P..{GEz..
    0x0020:  0000 4000 ff11 e1a3 c0df 1a5b 728d 48eb  ..@........[r.H.
    0x0030:  0000 0050 0308 62db 0000 0000 0000 0000  ...P..b.........
    0x0040:  0000 0000 0000 0000 0000 0000 0000 0000  ................
    0x0050:  0000                                     ..
    13:06:37.512652 IP (tos 0x0, ttl 64, id 27986, offset 0, flags [DF], proto UDP (17), length 799)
        xx.xx.xx.xx.33118 > 114.141.72.236.80: UDP, payload 771
    0x0000:  4500 031f 6d52 4000 4011 33c8 c0df 1a5b  E...mR@.@.3....[
    0x0010:  728d 48ec 815e 0050 030b 7b45 457a 031e  r.H..^.P..{EEz..
    0x0020:  0000 4000 ff11 e1a0 c0df 1a5b 728d 48ec  ..@........[r.H.
    0x0030:  0000 0050 030a 62d6 0000 0000 0000 0000  ...P..b.........
    0x0040:  0000 0000 0000 0000 0000 0000 0000 0000  ................
    0x0050:  0000                                     .. 
    I reinstalled the entire VPS, put XenForo on it and everything seemed fine for a few days.
    Then I got another event log:
    Code:
     Your VDS is back to using tons of bandwidth all the time. This time, it looks like the issue may be your Cloudflare settings and not an attack that your VDS is running. You will need to investigate and resolve this. 
    I have issued a support request previously about the first event log since it looked like CloudFlare was somehow using up all my bandwidth but I'm not sure if it's there end since I resolve all CF IPs properly.

    Now the third event log:
    Code:
    Your VDS has continued to launch attacks.
    
    It is clear that you have just been reloading the same software back on and it is just being re-compromised. You need to actually investigate and resolve it this time. If you can't find how this person is gaining control of the VDS, you will need to shut it down entirely.
    
    09:11:00.569949 IP (tos 0x0, ttl 64, id 33332, offset 0, flags [DF], proto UDP (17), length 1041) xx.xx.xx.xx.42971 > 103.26.180.35.80: UDP, payload 1013
    0x0000:  4500 0411 8234 4000 4011 2c6c d834 94c9  E....4@.@.,l.4..
    0x0010:  671a b423 a7db 0050 03fd d4c7 457a 0410  g..#...P....Ez..
    0x0020:  0000 4000 ff11 6e35 c07e 2d71 671a b423  ..@...n5.~-qg..#
    0x0030:  d757 0050 03fc 1721 0000 0000 0000 0000  .W.P...!........
    0x0040:  0000 0000 0000 0000 0000 0000 0000 0000  ................
    0x0050:  0000                                     ..
    09:11:00.569952 IP (tos 0x0, ttl 64, id 33333, offset 0, flags [DF], proto UDP (17), length 1043) xx.xx.xx.xx.42971 > 103.26.180.35.80: UDP, payload 1015
    0x0000:  4500 0413 8235 4000 4011 2c69 d834 94c9  E....5@.@.,i.4..
    0x0010:  671a b423 a7db 0050 03ff d4c6 457a 0412  g..#...P....Ez..
    0x0020:  0000 4000 ff11 6e32 c07e 2d72 671a b423  ..@...n2.~-rg..#
    0x0030:  d758 0050 03fe 171b 0000 0000 0000 0000  .X.P............
    0x0040:  0000 0000 0000 0000 0000 0000 0000 0000  ................
    0x0050:  0000                                     ..
    09:11:00.569964 IP (tos 0x0, ttl 64, id 33334, offset 0, flags [DF], proto UDP (17), length 1045) xx.xx.xx.xx.42971 > 103.26.180.35.80: UDP, payload 1017
    0x0000:  4500 0415 8236 4000 4011 2c66 d834 94c9  E....6@.@.,f.4..
    0x0010:  671a b423 a7db 0050 0401 d4c5 457a 0414  g..#...P....Ez..
    0x0020:  0000 4000 ff11 6e2f c07e 2d73 671a b423  ..@...n/.~-sg..#
    0x0030:  d759 0050 0400 1715 0000 0000 0000 0000  .Y.P............
    0x0040:  0000 0000 0000 0000 0000 0000 0000 0000  ................
    0x0050:  0000                                     .. 
    Can someone give me some help with this, as I have no idea what to do and cannot figure out what is launching these attacks.

    Thanks!
     
  2. Slavik

    Slavik XenForo Moderator Staff Member

    So, when you re-imaged the VPS and put the software (XenForo and others) back on, were those clean uploads, or the backups you took?

    Most likely one or more of your web files contains a shell script in there.
     
    silence and Floren like this.
  3. silence

    silence Well-Known Member

    I did not reimage the VPS, the software of XF was a backup though.
    Is it possible it could be XF though?
    The software I was running was the following:
    Go-Camo: https://github.com/cactus/go-camo
    Minotar: https://github.com/minotar/minotar
    XenForo (including 50 addons, all from XF and purchased legally)
    SourceBans: http://www.sourcebans.net/

    How can I detect this shell script so I can get rid of it and learn how it got there to prevent any future instances like this? Could it have been uploaded to my web server through XenForo?
     
  4. Floren

    Floren Well-Known Member

    @Slavik, or an user logs in... running fancy things. They are all UDP packets. :)
     
    Slavik and silence like this.
  5. silence

    silence Well-Known Member

    How would I investigate I'm new to investigating a VPS running rogue things D:
     
  6. Slavik

    Slavik XenForo Moderator Staff Member

    This should be the first thing to do. Start from a clean base, lock it down, use SSH keys or 2 factor auth (@Floren has a guide on using Googles authenticator for this), then start adding things in checking the files (or re-uploading a clean copy from the customer area) for anything out of place.
     
    Floren likes this.
  7. silence

    silence Well-Known Member

    No I meant I wiped the VPS and did a clean install of Gentoo. (sorry nevermind read your answer I'm a mess atm haha)
    And SSH shows that I'm the only one that logged in ever.
     
    Last edited: Aug 2, 2014
  8. Slavik

    Slavik XenForo Moderator Staff Member

    And get yourself onto a decent OS like CentOS :)
     
    Floren likes this.
  9. silence

    silence Well-Known Member

    Yeah although I absolutely adore Gentoo, CentOS seems like a good idea for now until I can figure out what's going on.
    I have to admit I really want to figure out where this is coming from, so does anyone have a good guide on what I can do to investigate and discover where this is happening at?
     
  10. Slavik

    Slavik XenForo Moderator Staff Member

    I haven't used Gentoo so apart from generic check logs type suggestions I cant really point you in any direction sorry.
     
    silence likes this.
  11. silence

    silence Well-Known Member

    Alright well thanks for the help :)
     
  12. Tracy Perry

    Tracy Perry Well-Known Member

    Or Debian... either one are fairly easy to secure - I use keys AND Google 2 factor authentication (for when I'm not where I can install a key on the computer). :p


    Gentoo is nice - for a desktop OS, but you need to really be using something that is targeted more towards server use.... centOS, Debian and the like.... I'm not even crazy about Ubuntu in some aspects, even though it is a Debian derivative.
    Looks like most of your additional stuff you use are related to MineCraft? Welcome to the world of kiddies that get twisted when they get banned. Odds are one of those is your attack vector.
     
  13. Floren

    Floren Well-Known Member

    What is Gentoo? :giggle:
     
  14. Tracy Perry

    Tracy Perry Well-Known Member

    A somewhat nice desktop version of Linux. :ROFLMAO:
     
    Floren likes this.
  15. Floren

    Floren Well-Known Member

    What is Desktop? :D
     
  16. Floren

    Floren Well-Known Member

    Is probably a UDP backdoor you have installed, plenty of them available on Google. Time to start from scratch with a clean OS, I recommend you to use CentOS and secure properly your server with Selinux/firewalld and iptables. I have no idea why people disable Selinux, I would rather die then turn it off in any of my servers.

    I would also secure those /tmp directories, as well get rid of 0777 insanity in XenForo. :)
    The devs provided us with proper tools to secure the directories/files, yet everyone uses 0777 and 0666 as their bible... no wonder you get hacked. :)
    http://xenforo.com/community/threads/php-fpm-configuration-encountered-and-error.79759/#post-803113
     
    Last edited: Aug 3, 2014
  17. Sheratan

    Sheratan Well-Known Member

    I tought you need to manually compile kernel/etc to run Gentoo?

    You, my friend, are the first person I know to use Gentoo in production server. All my friend use it in development only.
     
    Fufu likes this.
  18. WSWD

    WSWD Well-Known Member

    If you're new to VPS and have no idea what you're doing, the first thing you should do is hire a management/administration team to secure and harden the VPS for you.

    As has been mentioned you need to start over from scratch. Change any root password or admin password you previously used, and make it random and impossible to guess.
    Reload the OS from scratch
    Reload Xenforo from scratch and simply import the database.

    Then go from there. You can't keep loading backups to your VPS, as it's likely the backups have the compromised files. You literally need to start from scratch and load things one at a time from scratch...freshly downloaded, verified files, that are not from your backups. Again, this is after wiping your VPS completely clean and reinstalling the OS from scratch. To echo what the others said...Gentoo? No...
     
  19. motowebmaster

    motowebmaster Active Member

    There are several service providers that can help create a clean and secure server for you. I remember Gentoo, but haven't touched it in years.

    I don't use them but thought it would be worth sharing:
    https://serverpilot.io
     
  20. silence

    silence Well-Known Member

Share This Page