Looking For a New Host for Large Forum

Can you do "top" in your server's terminal and see how much memory/CPU are you currently taking (it's actually better doing it during peak times)? on what server are you right now? XF takes basically much less resources than vB, so I wouldn't go on way better server than whatever you have right now (although I would change to SSD if you are currently on HDD).
 
Yes, that import is going to be ... a pain (to put it mildly).

1) I am located Oregon (USA). The majority of our users are also in USa, but we do have a worldwide audience.

2) Well, that really depends on what is needed -- I would LOVE to stay as cheap as possible obviously for what could handle our traffic with a solid uptime as well as fast speeds. I would say the absolute high-end of the budget would be 800-900 a month? But lower would obviously be a nice change of pace.

3) I am not completely comfortable with server management. If something was constantly going wrong, that would probably be outside of my expertise. But I can do a lot of the basic stuff myself, so I was hoping to find some kind of solution that included some kind of management as well when it's needed. I do not have someone in mind already, and would absolutely be open to suggestions in that front as well. I'd love to find someone that could, at least, help with the set-up process and be able to help intial troubleshooting of load times and making sure everything can run smoothly. Then be available for future questions and help if growth makes the current situation untenible.

Thats a huge budget. And realy does open your options up to the more premium providers, having said that, do shop around a little and contact them about the pricing. I for example consider wiretree to be very expensive by the time youve added in a few basic options and upgrades, however, they are fully managed servers IIRC so, you do get that peace of mind.
 
Does Lightspeed restart hourly when it's logs are processed like Apache does? If you run Xcache / APC with Apache in Cpanel for example, you need to enable piped logging to stop it restarting.

Thanks again Matt - piped logging solved the problem (and probably others as well). The only downside is that now I have to increase my Variable Cache - it gets a lot larger over time than it did with the frequent restarts. OTOH, the Opcode cache stays more or less the same size over time. :D
 
Do you have a suggestion for someone that does support Xenforo hosting, or what the best way to look into developing a server/admin strategy of my own would be? Those really feel like the next steps -- given my current time table looks like I may have to:

1) move the current VB website to the new server,
2) upgrade to the new version of the website on xenforo when it is complete.

But I am REALLY hoping to just start with the new site, and have the migration/import already done before that step.

I'm no good at recommending hosting services, but Mike Edge appears to offer a hosting service that others like.

Some hosts, like WiredTree and MediaTemple, can help make things easier for new server admins by offering control panels and firewall/backup options. Managed Servers was something I used to subscribe to years ago, and there's nothing wrong with doing so today but I wouldn't have the patience for it.

A few years ago, I sold one of my former sites to a company that had BSD admins on staff. They were hardcore, and believed in running everything on commodity hardware. I bring up this example to express that server management is a bit of an art, and everyone has an opinion on how it should be done. I didn't agree with everything those BSD-focused were doing, but I wouldn't be any easier to convince either ;)

I see a couple users on this site offering solid VPS Hosting services as well, if you're limited on budget. There are lot of VPS options out there, but very few providers do it properly.
 
I'm considering shifting my site (20M+ posts) to AWS, anybody had experience running large sites on them?
 
I haven't managed or run it (and it really isn't database sites), but I believe my employers use AWS for all our client sites. Not sure how it'd work for a dynamic application tho.
 
Well Reddit, Netflix, Spotify and Foursquare all seem to cope pretty well running on AWS, plus I have a small site running on AWS already. AWS are the big players in cloud.

No my query is about very large sites- as in top 5 biggest xenforo sites, which can scale servers up and down with demand (the site is unpredictable spiky). Its going to require app servers, maybe memcached, the CDN should be utilised etc. Like anywhere, configre the infrastructure incorrectly and you'll either have a system that underperforms or one where you are paying far too much.

The sort of things I need to know are what are optimal database configurations for RDS, for example, how much provisioned IOPS should be used, maybe a figure per concurrent user (1?) and other items which are perculiar

Would be interested in others perspectves on such things as impact on the site during database snapshots.
 
OVH, if you know what you are doing and properly setup the server from scratch.
Their default image is crap. Running my site on the $49/month package:
Code:
top - 02:25:02 up 7 days,  3:18,  1 user,  load average: 0.00, 0.02, 0.00
Tasks: 167 total,   1 running, 166 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.5%us,  0.2%sy,  0.0%ni, 99.3%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:  7770.422M total, 1094.016M used, 6676.406M free,  148.105M buffers
Swap: 8191.992M total,    0.000k used, 8191.992M free,  409.656M cached

Mem: 7770.422M total, 1094.016M used

Edit: This is a very small site but I simply wanted to show what you can do with a proper setup.
 
Last edited:
Do you have a suggestion for someone that does support Xenforo hosting, or what the best way to look into developing a server/admin strategy of my own would be? Those really feel like the next steps -- given my current time table looks like I may have to:

1) move the current VB website to the new server,
2) upgrade to the new version of the website on xenforo when it is complete.

But I am REALLY hoping to just start with the new site, and have the migration/import already done before that step.

I'd recommend just migrating. Install XenForo on the new server, take backups on the old one, and then perform the import. Save yourself the hassle of having to replicate the vB environment first and then having to ditch it when you're done, though with the size of your forum, I'd also look into having a test-import done first. In case of any issues, that gives you the flexibility of being able to tackle them prior to doing the live upgrade.



As for a good host, we have been hosting with NimbusHosting for years, and are extremely satisfied with them (both their support as well as their up-time).
 
I'm no good at recommending hosting services, but Mike Edge appears to offer a hosting service that others like.

Some hosts, like WiredTree and MediaTemple, can help make things easier for new server admins by offering control panels and firewall/backup options. Managed Servers was something I used to subscribe to years ago, and there's nothing wrong with doing so today but I wouldn't have the patience for it.

A few years ago, I sold one of my former sites to a company that had BSD admins on staff. They were hardcore, and believed in running everything on commodity hardware. I bring up this example to express that server management is a bit of an art, and everyone has an opinion on how it should be done. I didn't agree with everything those BSD-focused were doing, but I wouldn't be any easier to convince either ;)

I see a couple users on this site offering solid VPS Hosting services as well, if you're limited on budget. There are lot of VPS options out there, but very few providers do it properly.

But what happens if something happens to @Mike Edge? A one-person company disappears if the owner isn't around anymore or he decides to leave the business. At least with a larger company, you know it's going to continue and your site won't go down when the host company does or if the owner is in an accident or dies or something. I'm not saying this would happen. Hopefully it never does. But it is a valid concern. Larger companies seem to be more stable, but don't offer the personal level of service that a smaller provider can. Maybe Mike has a contingency plan in place? Everyone here raves about his site speed and service, so I am sure the reputation is well-earned and worth considering as a hosting provider.
 
5 biggest xenforo sites, which can scale servers up and down with demand (the site is unpredictable spiky). Its going to require app servers, maybe memcached, the CDN should be utilised etc. Like anywhere, configre the infrastructure incorrectly and you'll either have a system that underperforms or one where you are paying far too much..


Well, let's start there. How big do you think the 5 biggest XF sites are? Which are they?

I know this is not totally accurate, but:
http://trends.builtwith.com/cms/XenForo

says only two XF sites are in the quantcast top 10K.

We are one of them, Digitalpoint is the other (granted, this MAY be USA users, but still close enough).
https://www.quantcast.com/hearth.com

We host on a 3 years old server with 8GB ram and nothing fancy at all in terms of disk, cache, CDN, etc....load has never hit the number of CPU's we have. Google rates our pagespeed and everything else as 4 or 5 stars...or high percentages.

So, the question becomes if you are overthinking some of this?

DP uses a number of servers - overkill, IMHO, but he's into that stuff. If he wasn't having so much fun putting together the ultimate system(s), he might be able to get by with less.

But, realistically, how many XF are up in that range? Based on the numbers I see, it's like about what you could count on one hand or less.
 
DP uses a number of servers - overkill, IMHO, but he's into that stuff. If he wasn't having so much fun putting together the ultimate system(s), he might be able to get by with less.
You could easily run the XenForo portion of our site on a single server.

95% of our server resources go to non-XenForo stuff. We have a lot of things running on our cluster that take a decent amount of resources that are not related to XenForo. For example publishers using Digital Point Ads (we serve up ads in realtime for every page view of every one of those sites), we have the world's largest search engine position tracker (~100k users tracking 2.5M+ keyword/URL combos for search engine rankings), Geo Visitors tool also also requires a query and HTTP hit for every page view of every site using it (somewhere around 400,000 sites), we have a search engine spider constantly spidering sites 24/7, etc. There are a couple additional things in the works that will be very resource intensive as well... All our "interesting" stuff is outside our "forums" subdomain. :)

Even with all that, our setup is overkill to the extreme... I hate upgrading servers... it's an annoying process that I like to minimize. I also don't like to have any single point of failure. Right now, any piece of equipment can fail completely (hard drive, entire server, etc.) and end users wouldn't notice. We use MySQL Cluster (ndbcluster) storage engine for our tables... and our setup can handle somewhere around 15,000,000 SQL reads concurrently with ~5,000,000 SQL writes per second.

An overall load average of 256.00 would mean our cluster is saturated, and as you can see even with all that stuff running on it, it's pretty stable, and pretty low (no where remotely close to what it can handle... which again would be 256.00 load average).

upload_2013-11-3_10-7-12.webp
 
OK, guys, so there you go!

Two of the 5 largest XF sites could run on a single server.
So maybe 10 people in the world need to think about all the balancing and other stuff.

And, of course, adult sites which often don't show up in the stats...

I upgrade about every 3 years - sometimes 3 1/2 or 4. But I don't have to do all that crazy custom stuff that DP does, so my upgrade is almost painless. My ISP sets it up so both sites mirror each other constantly (old and new) and then we move. I usually have about 90 minutes of work with some gory details.

My site is pretty much stock XF.....with some old parts (Gallery 2) still hanging around. If I had to do the amt of work that DP did in terms of custom, I'd want to get 5-8 years out of my servers...or more! When my site first started, it shared a 90 MHZ Pentium with many other sites!
 
When my site first started, it shared a 90 MHZ Pentium with many other sites!

Most people here probably don't remember those days. We used to offer hosting on Pentium IVs and Celerons (as did everyone else) and there were never any issues. Sure, sites have gotten a little more intensive (i.e. Wordpress and such), but the ultimate thing driving all these high-end servers is the fact that the host can put thousands of clients on them, instead of the couple hundred on a P4, especially when you add things like CloudLinux.

I'd be willing to bet that 99% of the sites here running dedicated servers could run on an old P4 with 8GB of RAM or such without any difficulty at all.
 
Most people here probably don't remember those days. We used to offer hosting on Pentium IVs and Celerons (as did everyone else) and there were never any issues.

And a ton of sites on the server too to try to make the cost of the very expensive T1.
 
Top Bottom