1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

My server has been compromised.

Discussion in 'Server Configuration and Hosting' started by jgas, Jul 10, 2013.

  1. jgas

    jgas Active Member

    What should I do to understand what's caused the problem and how to fix it?

    Hi, recently my server starter to suffer from "massive amounts of outbound udp packets", just from time to time. When it happens, my hosting company turns the server down for a day, then it re-enable it and for a couple of weeks everything seems ok, but after the problem happens again.

    The hosting company is doing nothing to help me because it's an unmanaged server, and since I'm not a programmer or technician (I'm doing marketing!) I didn't know that I could encounter such big problems like these.
    If they are going to solve the problem in my place, they want me to pay for their technical services that will cost me hundreds of pounds, so the best thing for me will just be to change host...

    So, I started to learn from scratch the basics of ssh and with the help of some tutorial I installed fail2ban (that ban an ip for example after 3 times it tries to connect to ssh) and rkhunter , that search for maliciuous scripts in my server.

    Both were not so useful... rkhunter only pointed out these warnings:

    Warning: Application 'httpd', version '2.2.3', is out of date, and possibly a security risk.
    Warning: Application 'openssl', version '0.9.8e', is out of date, and possibly a security risk.
    Warning: Application 'php', version '5.2.10', is out of date, and possibly a security risk.
    Warning: Application 'sshd', version '4.3p2', is out of date, and possibly a security risk.

    Do you think that they REALLY need to be updated?
    I asked my hosting company to update them, but they are REALLY SLOW to answer me. Imagine that to have my server online again I had to CALL THEM in the uk and wait 20 minutes for the customer service to answer...

    So, in the end, my question is: I know that in the future someone will try to launch an attack from my server again, so how can I track what he is doing, and finally understand which is the hole in the server?

    Many thanks!
  2. Tracy Perry

    Tracy Perry Well-Known Member

    Not sure what the current version is, but looking at the others it is probably way out of date also.

    This is WAY out of date. 5.4.xx is current and 5.5 is right around the corner - so yes, it DOES need to be upgraded.

    Considering that this is what you SSH via, it is vitally important that it be kept up to date.
    I believe that the current version is around 6.2 (again, you are WAY out of date).

    Uh, yeah - like yesterday.
    If you are going to get an unmanaged system then you need to either do 1 of 2 things
    1) Hire and administrator for it
    2) learn to administer it yourself

    The most important thing is keeping the system up to date with the latest versions that are stable.
    WSWD and Jeremy like this.
  3. Slavik

    Slavik XenForo Moderator Staff Member

    Fixing an already comprimised and outdated server is akin to using band-aids to fix a stab wound.

    If your current company is providing such terrible service, the first thing i'd do is look to find a new host.

    Then, get a new server get it all up to date, secure and running and ready for your site (If you cant do this, then hire someone who can), and then start transfering your data over, checking it through for any scripts or malicious code that may have been put in your web files allowing the attacker access.
  4. Tracy Perry

    Tracy Perry Well-Known Member

    From what he indicated from his current hosting comments - I'd be running as fast as I could to find another hosting solution.
    If he's going to maintain it himself then he REALLY needs to brush up on administration of a Linux server (I'll be the first to say it's not for the faint of heart and not for everybody).
  5. Biker

    Biker Well-Known Member

    I'm sorry, but this is one of those cases where the OP needs to bring someone on board that knows what they're doing.

    Don't get me wrong, I'm all for folks learning how to do things on their own. But you don't do it on a live, production server that talks to the world. All that does is create another compromised box that everyone else has to worry about.

    If the OP wants to learn Linux and how to run a secure server, do it on a home network. THEN apply those skills to a live server.
    hellreturn and Tracy Perry like this.
  6. Tracy Perry

    Tracy Perry Well-Known Member

    Agreed 100%. That's how I taught myself (and am still teaching - up now to 7 flavors of Linux installed on computers here in the computer room - 6 desktops and 2 actual servers).
    Gregory Lynn likes this.
  7. jgas

    jgas Active Member

    Thank you guys! I understand that learning everything will be to much for me... I just don't have time to do that.
    Maybe, the easiest solution will be to find a fully managed virtual server, and that's it, as you clearly pointed out ;)
    Any recommendation for a virtual managed server with 1 gb ram, 20 gb webspace, 500 gb of bandwidth, 1 core of at least 2 gb? (most of the traffic will be from Italy and Europe).
  8. Slavik

    Slavik XenForo Moderator Staff Member

    I believe nimbus hosting are fully managed servers, and are quite highly reccomended around here.
    hellreturn and Tracy Perry like this.
  9. jgas

    jgas Active Member

    Nimbus can't take me onboard (I already tried...) because due to a legal restriction that is in place between Nimbus and my hosting, Nimbus can't take me as a client. That's what Nimbus customer care wrote me...

    What about a small orange?

  10. Tracy Perry

    Tracy Perry Well-Known Member

  11. Slavik

    Slavik XenForo Moderator Staff Member

    Jake Bunce uses them and says their fantastic.
    jgas likes this.
  12. dbembibre

    dbembibre Active Member

    I dont think that the hole can be in yours outdated binaries, maybe you have the hole in the web, review your /tmp partition to see if anyone upload an script there that is using your server with any malware or something similar. Maybe a network misconfiguration is generating a lot of broadcast packages and flood the routers.

    Take a look to this web http://apache-range-exploit.com/
    and this web too http://www.incapsula.com/ddos/ddos-attacks
    I remember a apache script (ping of death) that flood your server with a lot of udp outgoing packages.

    You can prevent an udp flood with a easy iptables command. Anyway i recommend that install a firewall like apf

    /sbin/iptables -A OUTPUT -p udp -m state --state NEW -j ACCEPT
    /sbin/iptables -A OUTPUT -p udp -m limit --limit 50/s -j ACCEPT
    /sbin/iptables -A OUTPUT -p udp -j DROP
    Excuse my broken English
    Last edited: Jul 10, 2013
    jgas likes this.
  13. SneakyDave

    SneakyDave Well-Known Member

    Make sure all of your web applications: blog, forum, etc. And their addons are up to date.
  14. WSWD

    WSWD Well-Known Member

    You can give Knownhost a try, though I don't believe they have any servers in Europe.
  15. Tracy Perry

    Tracy Perry Well-Known Member

    Here is just a little sample of why you want to make sure everything is updated and as secured as possible. Over 700 invalid attempts at SSH on one of my servers.
    p4guru likes this.
  16. p4guru

    p4guru Well-Known Member

    +1 i see alot of these in CSF firewall logs as well but CSF firewall would of never allowed 700+ failed attempts i don't think unless they're all different ips ? Of course great case for changing default SSH port 22 to something else :)

    Reason all my CentOS servers have CSF Firewall installed http://configserver.com/cp/csf.html :)

    I also use Centmin Mod and that automatically installs CSF as well http://centminmod.com/csf_firewall.html

    Also install Malware Detect (maldet) as well http://www.rfxn.com/projects/linux-malware-detect/

    There's also free AVG Antivirus for Linux that Centmin Mod posted at http://centminmod.com/avg_antivirus_free.html.

    While AVG is way faster at scanning that Maldet, I have had problems with AVG virus definition updates failing to update on some servers while on others it works fine. So i'd use AVG and Maldet in conjunction to complement each other (if AVG works for you).

    How much speed difference ? On a server with 190,000 files and directories to scan, AVG took around 30-45 mins versus Maldet took just under 12 hrs !

    Maldet and AVG both managed to detect malware on one of my clients' servers although AVG picked up a few email message I-Worm/Netsky, I-Worm/Bagle.BDE and HTML/Framer.FA in addition to what both reported for some PHP/backdoor/PHP Cmdshell hits.

    p.s. AVG has debian and rpm and source downloads available at http://free.avg.com/us-en/download.prd-alf and was one of the first malware/anti-virus able to detect SSHD Rootkit exploit discussed at http://www.webhostingtalk.com/showthread.php?p=8569541#post8569541
    Last edited: Jul 11, 2013
    jgas likes this.
  17. Biker

    Biker Well-Known Member

    That title needs to be changed as it was NOT an SSHD rootkit. In the early hours and days, it was thought it might be, but if you read that entire thread, you'll see that it wasn't related to SSHD at all.
    p4guru likes this.
  18. p4guru

    p4guru Well-Known Member

    Yeah i know it turned out it wasn't just keeping it consistent with the thread linked :)
  19. Tracy Perry

    Tracy Perry Well-Known Member

    Yeah, and normally fail2ban will stop it also (there were over 200 from two different Chinese IP's - they are in the drop list now). Fail2ban was not reading the auth.log and banning the ip's. Got that issue resolved and that's why you see them drop to almost zero after that point.
    p4guru likes this.

Share This Page