How to best determine requirements when selecting 'best' server?

[cmeinck]

I currently run my forum and Wordpress blog on a dedicated server at SoftLayer. They've been great and overall I'm pleased with site update, etc. In addition to costs associated with my server, I pay for a support team to 'manage' my server. On some months, they don't much at all. I find they are good for setting up new domains or if I ask something specifically. For backups, I'm using Jungledisk ($10 per month) to Amazon Web Services. Anyway, I'm thinking that perhaps there is a better deal out there for me and also don't want to get too much server.​

​

What's the best way to determine my usage? Actual storage I'm using, bandwidth and needs based on concurrent traffic?​

​

Here's my current rig, which costs $464 per month.​

​

​

Server: Dual Processor Quad Core Xeon 5345 - 2.33GHz (Clovertown) - 2 x 8MB cache​

​

​

$309.00​

$0.00​

$0.00​

$0.00 ​

​

Second Processor: Quad Core Xeon 5345 - 2.33GHz (Clovertown) - 2 x 8MB cache​

$0.00​

$0.00​

$0.00​

$0.00 ​

​

Uplink Port Speeds: 10 Mbps Public & Private Networks​

$0.00​

$0.00​

$0.00​

$0.00 ​

​

Public Network Port: 10 Mbps public uplink​

$0.00​

$0.00​

$0.00​

$0.00 ​

​

Public Bandwidth: 3000 GB Bandwidth​

$0.00​

$0.00​

$0.00​

$0.00 ​

​

Primary IP Addresses: 1 IP Address​

$0.00​

$0.00​

$0.00​

$0.00 ​

​

Public Secondary IP Addresses: 4 Public IP Addresses​

$0.00​

$0.00​

$0.00​

$0.00 ​

​

Disk Controller: SA-SCSI Disk Controller​

$0.00​

$0.00​

$0.00​

$0.00 ​

​

First Hard Drive: 73GB SA-SCSI 10K RPM​

$30.00​

$0.00​

$0.00​

$0.00 ​

​

Second Hard Drive: 73GB SA-SCSI 10K RPM​

$0.00​

$0.00​

$0.00​

$0.00 ​

​

Operating System: CentOS 5 (32 bit)​

$0.00​

$0.00​

$0.00​

$0.00 ​

​

Control Panel Software: cPanel/WHM with Fantastico and RVskin​

$25.00​

$0.00​

$0.00​

$0.00 ​

​

Ram: 8 GB FB-DIMM Registered 533/667​

$100.00​

$0.00​

$0.00​

$0.00 ​

​

​

Network Attached Storage: 20 GB NAS​

$10.00​

​

Depending on the month, my site can do around 3 million page views with 2 million unique visitors.​

​

I was offered the following for $334 per month, which again, might be too much server or just right.​

​

Base Hardware​

Intel Xeon E5-2620 (6-Core, 2.0GHZ, 15MB Cache, HT)​

16GB DDR3 ECC​

​

Primary Disk:​

120GB Intel 330 SSD​

​

Secondary Drive​

120GB Intel 330 SSD​

​

Additional Options​

Route Optimized Bandwidth : 10TB Premium Transfer​

Operating System : CentOS Linux 64-bit + cPanel/WHM​

Chassis : Standard SuperMicro Rackmount​

Public Port Speed : 100Mbs Public​

Private Port Speed : 100Mbs Private​

IP Addresses : 8 IP Addresses​

Control Panel : cPanel / WHM​

​

Support Services​

Fully Managed​

24x7 Phone and Helpdesk​

24x7 Proactive Monitoring​

Proactive Updates​

ServerShield Hardening​

 

[Ghan_04]

Whoa whoa whoa here. 3 million views a month is all? And you're paying for that server?
That makes me sad. I run about 17 different domains with over 500,000 requests total (over all 17) per DAY on a Linode VPS with 768 MB of RAM. So on the surface, that kind of server hardware seems completely absurd to be paying for. However, there might be more to it than that. There are wordpress plugins out there that can be really resource intensive. What I would do is this:

- Find out the load on your system's CPUs during peak times.
- Find out how much RAM is in use at any given time.
- Find out how big your databases are.
- Find out how much space total you need.

If you have any significant load on the CPU (anything over 1.00 might be worth looking into) then check to see what processes are using the CPU.
It would also help to see what kind of configuration you have for MySQL and/or PHP. Things like the my.cnf file, php.ini file, and loaded PHP extensions would be helpful.
Also, what webserver are you using? I use Litespeed, and it is great for resource usage and completely compatible with Apache, so that might be something to consider as well.

Hope this helps. Also, I'd be happy to help you look at other server options or even help you set up stuff if your management guys charge too much. (Just a forum and a wordpress install is really easy - I have worked and currently work with Xenforo and Wordpress in server environments.)

 

[p4guru]

It's all about concurrency not total cumulative numbers.

There's a huge difference in server hardware requirements for the following scenarios of 3 million visitors per month

• 100,000 visitors per day x 30 days
• 1,000 visitors per day for 29 days + 2,971,000 visitors all in 1 day

Both examples will show 3 million visitors per month. The latter example is the extreme case but it illustrates the point. The server hardware required to service the 2nd example for that one peak 1 day will be alot higher than the 1st example
You need to gather daily and maybe hourly server usage statistics over a longer period of time to see trends of what your peak times are to see what hardware is required to service such peak times.

 

[Ghan_04]

It's all about concurrency not total cumulative numbers.

There's a huge difference in server hardware requirements for the following scenarios of 3 million visitors per month

• 100,000 visitors per day x 30 days
• 1,000 visitors per day for 29 days + 2,971,000 visitors all in 1 day

Both examples will show 3 million visitors per month. The latter example is the extreme case but it illustrates the point. The server hardware required to service the 2nd example for that one peak 1 day will be alot higher than the 1st example

You need to gather daily and maybe hourly server usage statistics over a longer period of time to see trends of what your peak times are to see what hardware is required to service such peak times.

I realize that, which I why I suggested to gather statistics about peak usage times.

 

[cmeinck]

Thanks for the feedback. I've pinged my support team and asked for the information mentioned. I can tell you that my XF db is 521MB and my WordPress db is 166MB. From what I could tell in WHM, my storage usage is about 36GB.

Let me add more to my dilemma. I have about 5 or 6 sites that I run on another server, which I'm paying $374. Those sites see 3-4k per day if I'm lucky. Some are basically older and won't see any growth.



BTW, here's one sample configuration offered by WiredTree. They have coupons running that double the hard drive. I'd have to see if 240GB would be sufficient, as my 2nd server has 300GB drives. I'll have a better idea once I get usage numbers.

Base Hardware

Intel Xeon E5-2620 (6-Core, 2.0GHZ, 15MB Cache, HT)
16GB DDR3 ECC

Primary Disk:

120GB Intel 330 SSD

Secondary Drive

120GB Intel 330 SSD

Additional Options

Route Optimized Bandwidth : 10TB Premium Transfer
Operating System : CentOS Linux 64-bit + cPanel/WHM
Chassis : Standard SuperMicro Rackmount
Public Port Speed : 100Mbs Public
Private Port Speed : 100Mbs Private
IP Addresses : 8 IP Addresses
Control Panel : cPanel / WHM


Would it make sense to plan on moving all of the sites to one server or when I make my move, keep them separated. My monthly server costs are about $858 now. If I cut that in half, I'd be thrilled. The server above prices out at $334 or so. Their coupons double the RAM and the SSD size. I'd also be adding a 1000GB drive for backups.

 

[Ghan_04]

I'll wait until I see the usage numbers to say for certain, but it sounds like you might be able to cut more than half of your budget. Perhaps as much as $550 or so.
One question, though. If you have a support team to manage your box, why are you looking at WiredTree, which is a fully managed provider? I'd be considering someone unmanaged like WebNX.

 

[cmeinck]

Here's the information provided by my support team. BTW, the reason I'm going to managed is to save on support. I pay $130 per month total, which covers my existing (2) servers. Basically, with servers and support, I'm spending $1k.

redmond :

Filesystem Size Used Avail Use% Mounted on
/dev/sda2 270G 166G 90G 65% /
/dev/sdb1 275G 117G 145G 45% /home2

starbucks :

Filesystem Size Used Avail Use% Mounted on
/dev/sda5 9.7G 8.8G 440M 96% /
/dev/sda7 35G 9.2G 24G 29% /home
/dev/sda3 9.7G 7.1G 2.2G 77% /usr
/dev/sda2 9.7G 8.1G 1.2G 88% /var

> RAM usage
used free
server a 1143MB 2907MB
server b 1024MB 7091MB

> Size of databases

server a 125G
server b 3.2G

> Load on CPU during peak times

Please let us know peak hours for the servers .

> my.cnf file

redmond
===========================
[mysqld]
port = 3306
socket = /var/lib/mysql/mysql.sock
skip-locking
key_buffer_size = 256M
max_allowed_packet = 10M
table_open_cache = 256
sort_buffer_size = 1M
read_buffer_size = 1M
read_rnd_buffer_size = 4M
myisam_sort_buffer_size = 64M
thread_cache_size = 8
query_cache_size= 16M
thread_concurrency = 8
max_connections=1000
log-bin=mysql-bin
binlog_format=mixed
server-id = 1
[mysqldump]
quick
max_allowed_packet = 16M
[mysql]
no-auto-rehash
[myisamchk]
key_buffer_size = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M
[mysqlhotcopy]
interactive-timeout
==============================


starbucks
============================
[mysqld]
port = 3306
socket = /var/lib/mysql/mysql.sock
skip-locking
key_buffer = 256M
max_allowed_packet = 16M
table_cache = 256
sort_buffer_size = 1M
read_buffer_size = 1M
read_rnd_buffer_size = 4M
myisam_sort_buffer_size = 64M
thread_cache_size = 8
query_cache_size= 16M
thread_concurrency = 8
max_connections=2000
server-id = 1
[mysqldump]
quick
max_allowed_packet = 16M
[mysql]
no-auto-rehash
[isamchk]
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M
[myisamchk]
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M
[mysqlhotcopy]
interactive-timeout
========================

There are some issues with server a mySQL numbers. I think there are some very old, unused databases taking up space. I'm going to clean those out today to get a better idea of actual usage.

 

[Luke F]

Whoa whoa whoa here. 3 million views a month is all? And you're paying for that server?
That makes me sad. I run about 17 different domains with over 500,000 requests total (over all 17) per DAY on a Linode VPS with 768 MB of RAM. So on the surface, that kind of server hardware seems completely absurd to be paying for. However, there might be more to it than that. There are wordpress plugins out there that can be really resource intensive. What I would do is this:

- Find out the load on your system's CPUs during peak times.
- Find out how much RAM is in use at any given time.
- Find out how big your databases are.
- Find out how much space total you need.

If you have any significant load on the CPU (anything over 1.00 might be worth looking into) then check to see what processes are using the CPU.
It would also help to see what kind of configuration you have for MySQL and/or PHP. Things like the my.cnf file, php.ini file, and loaded PHP extensions would be helpful.
Also, what webserver are you using? I use Litespeed, and it is great for resource usage and completely compatible with Apache, so that might be something to consider as well.

Hope this helps. Also, I'd be happy to help you look at other server options or even help you set up stuff if your management guys charge too much. (Just a forum and a wordpress install is really easy - I have worked and currently work with Xenforo and Wordpress in server environments.)

Huge +1 to this

With the right software optimisation, I have run 150 million pageviews /month on a fraction of that server power (never more than 25% of an i7-2600k). I think you're overspending by quite a large margin.

 

[p4guru]

That is some beefy hardware so could be overkill but without proper logging of usage and stats for server we all have no idea. As you use WHM control panel download and install mysqlmymonlite.sh bash script http://mysqlmymon.com/ - it gathers heaps of server usage and configuration stats in less than 12 seconds which you can provide to web hosts, your hired system admins or forum users to give you advice on.

Cpanel support forum thread on mysqlmymonlite.sh http://forums.cpanel.net/f189/mysqlmymonlite-sh-server-stats-gathering-tool-268011.html along with how to install video.

 

[Ghan_04]

Ok, well it sounds like jumping to managed would be good idea, then. WiredTree has great support - I've used their VPS product in the past.
I would swap out the SSDs for the 1 TB HDDs instead - it looks like you may need the space. Also, the E3 1230 CPU should be enough for what you're doing. WiredTree is running a Black Friday special through Monday that gives you 20% off new dedicated servers for the life of the account. So as long as you don't have any crazy usage spikes that require more power, I'd say go for it.

 

[cmeinck]

As it turns out, my server was configured for mySQL binding for replication. It was burning up 130GB worth of space. I also did some sprint cleaning, removing older Joomla/vB files and unused databases. I've also deleted a few older sites that weren't used. The sites are now lean. I think I should be able to get them within a usable range for the 240GB RAID1.

As for order of importance, is this correct?
RAM
Hard Drives
Processor
Software & Optimization

I've seen a few people here and elsewhere recommend the E3 1230. Wired Tree has the 1270 for only $25. That's even less considering the 20% off coupon. Would it be overkill to opt for a faster processor?

I'm looking at these options. If I cut the RAM down to 16GB, I save $32 off these prices.

Base Hardware

• Intel Xeon E3-1270 (4-Core, 3.5GHZ 8MB Cache, HT)
• 32GB DDR3 ECC

Primary Disk: Hardware Raid-1

• 2x240GB Intel 330 SSD

Secondary Drive

• 1000GB 7.2k RPM HDD

Additional Options

• Hardware RAID Controller : Adaptec 6405e - 128MB Cache - 4 Ports
• Tertiary Disk : 1000GB 7.2k RPM HDD
• Route Optimized Bandwidth : 10TB Premium Transfer
• Chassis : Hot-Swappable SuperMicro Rackmount [Included]
• Public Port Speed : 1000Mbs Public
• Private Port Speed : 1000Mbs Private

$403
Base Hardware

• Intel Xeon E5-2620 (6-Core, 2.0GHZ, 15MB Cache, HT)
• 32GB DDR3 ECC

Primary Disk: Hardware Raid-1

• 2x240GB Intel 330 SSD

Secondary Drive

• 1000GB 7.2k RPM HDD

Additional Options

• Hardware RAID Controller : Adaptec 6405e - 128MB Cache - 4 Ports
• Route Optimized Bandwidth : 10TB Premium Transfer
• Operating System : CentOS Linux 64-bit + cPanel/WHM
• Tertiary Disk : 1000GB 7.2k RPM HDD
• Chassis : Hot-Swappable SuperMicro Rackmount [Included]
• Public Port Speed : 1000Mbs Public
• Private Port Speed : 1000Mbs Private

$455


Base Hardware

• Intel Xeon E3-1230 (4-Core, 3.3GHZ 8MB Cache, HT)
• 32GB DDR3 ECC

Primary Disk: Hardware Raid-1

• 2x240GB Intel 330 SSD

Secondary Drive

• 1000GB 7.2k RPM HDD

Additional Options

• Hardware RAID Controller : Adaptec 6405e - 128MB Cache - 4 Ports
• Tertiary Disk : 1000GB 7.2k RPM HDD
• Route Optimized Bandwidth : 10TB Premium Transfer
• Chassis : Hot-Swappable SuperMicro Rackmount [Included]
• Public Port Speed : 1000Mbs Public
• Private Port Speed : 1000Mbs Private

$383

 

[cmeinck]

Thanks everyone for your help. This has been invaluable.

BTW, the costs above are roughly $25 less. Wired Tree is upgrading port speed to 1000Mbs for free.

Apparently this is a very rare deal and it doesn't expire. The deal I make today will stay for as long as I keep my server.

 

[Ghan_04]

Based on the stats you posted, 16 GB of RAM should be more than enough.
You don't really need the 1270 CPU - the 1230 will probably be sleeping most of the time anyway.
Hardware RAID 1 is great - definitely recommend.
Consider upgrading to Litespeed as the webserver. It's fully compatible with Apache config options and runs much faster. WiredTree fully supports it and it sounds like you are not hurting on your budget.

I would take the last config and cut the RAM to 16 GB. As you showed here:

> RAM usage
used free
server a 1143MB 2907MB
server b 1024MB 7091MB

The existing servers don't even have 16 GB of RAM between them. You should be golden then.
Also, I have some experience tuning MySQL memory usage, so if you'd like someone to take a look at your my.cnf file besides the standard WiredTree support stuff, I'd be happy to.

 

[cmeinck]

Based on the stats you posted, 16 GB of RAM should be more than enough.
You don't really need the 1270 CPU - the 1230 will probably be sleeping most of the time anyway.
Hardware RAID 1 is great - definitely recommend.
Consider upgrading to Litespeed as the webserver. It's fully compatible with Apache config options and runs much faster. WiredTree fully supports it and it sounds like you are not hurting on your budget.

I would take the last config and cut the RAM to 16 GB. As you showed here:

> RAM usage
used free
server a 1143MB 2907MB
server b 1024MB 7091MB

The existing servers don't even have 16 GB of RAM between them. You should be golden then.
Also, I have some experience tuning MySQL memory usage, so if you'd like someone to take a look at your my.cnf file besides the standard WiredTree support stuff, I'd be happy to.

I definitely would once I get moved over. I have a Litespeed license, which they won't support. Would love to use it, if I could. I did see some great comparisons between E-1230 and E-1270. From what I've read, the 1230 is great and not far off from the 1270.

Just checking with WT on their latest promotions to quadruple RAM and include free RAID controller.

 

[Ghan_04]

A battery backup for the RAID controller wouldn't be a bad idea, either. That way you can use write caching. Depends on your disk usage. It's probably not a big deal with SSDs.

 

[cmeinck]

A battery backup for the RAID controller wouldn't be a bad idea, either. That way you can use write caching.

Is that worth $40? I'd have to upgrade the controller for $30 and it's $20 for the battery backup (minus 20%).

 

[Ghan_04]

Eh, with SSDs in the box it's probably not that big of a deal. It helps increase performance on writes, but I doubt you will be doing a lot of heavy writing to disk.

 

[cmeinck]

I want to thank everyone, especially Ghan_04, who took the time to help with my server selection. I greatly appreciate the help and advice. I can happily report that I ordered my server from Wired Tree. I'm just waiting on them to confirm the order and then it's migration time. Here are the specs for my server. On a semi-annual agreement, this breaks out to $326 per month. My old servers: $858/month. Server support $130/month. Jungledisk backups: $10/month. Amazon Web Services $5/month. This all translates to a monthly savings of $677.

Hardware
Intel Xeon E3 - Socket 1155
32GB DDR3 ECC
Primary Disk:Hardware Raid-1
2x240GB Intel 330 SSD
Backup Drive: 1000GB 7.2k RPM HDD

Additional Options
CPU : Intel Xeon E3-1230 (4-Core, 3.3GHZ 8MB Cache, HT)
Control Panel : cPanel / WHM
Public Port Speed : 1000Mbs Public
Private Port Speed : 1000Mbs Private
Route Optimized Bandwidth : 10TB Premium Transfer
Chassis : Hot-Swappable SuperMicro Rackmount [Included]
Hardware RAID Controller : Adaptec 6405e - 128MB Cache - 4 Ports
IP Addresses : 8 IP Addresses

 

[Ghan_04]

Looks excellent. Good luck on getting everything migrated.

 

[cmeinck]

On day 2 of the migration. Not as smooth as I would have liked, but it's progressing. They are now installing Litespeed.

Any recommendations on what I should be asking them to install? The server runs SuPHP, which by itself doesn't support APC? They did say that with Litespeed, I could get APC/xCache installed. Is that the direction I should go in?

I had a number of issues. My javascript/data was served on a cookie-less domain that was not transferred during the migration. I pulled that from my configuration and it's flying. I cannot believe this is going to go faster. Not sure how that's possible.