Is it true that having x00 sections (node type forum) will cause performance problems?

[sbj]

Hey,

once an add-on developer told me that I would have performance issues if I want to create hundreds of sections.
Not that I don't believe him, I just want to be sure.

Are here any people who have hundreds of sections? Can you tell your experience from your forums?
Of course, it depends on the servers.
But I am not talking about big servers. I am a simple customer of a shared hosting company.

Many forums have millions of postings and thousands of threads. So I thought having hundreds of forums (categories) will not be a problem. So, how is it?

Can you enlighten me, please?

 

[Chris D]

https://www.avforums.com

They have over 300 nodes.

That does have a hefty farm of servers behind it.

It isn't necessarily the existence of hundreds of nodes that is the problem. It is typically the process of rebuilding caches related to nodes and node permissions that is the problem. There have been improvements over the last couple of years in this area, but if you are going to experience performance issues they will be when you do something related to nodes in the Admin CP, e.g. deleting nodes or adding nodes. Though, typically, this shouldn't affect the general usage of your site while it happens as long as your site can handle it. It may just be that it seems to take a while to complete the rebuilds.

 

[sbj]

Either this (having hundreds of nodes) or a custom add-on which surpasses this issue.
But nobody is willing to do this.
I guess a simple forum will have big problems with having hundreds of nodes. I am not talking about 300, this is the lowest case I would have. I am talking about 1000+ up to 2000.

Thank you for answering @Chris D.

I am doomed.

 

[Chris D]

Where in my post did you get the suggestion that you would be doomed?

I'm pretty much saying there would not be any performance issues at runtime but you may find some processes take longer at save time. The latter should not adversely affect the experience of your users.

 

[sbj]

You said that they have a hefty farm of servers behind them. Also you said this shouldnt affect the general usage of my site while it happens as long as my site can handle it.
So the conclusion for me is that a simple site with a simple hosting will have problems. It won't be able to handle it.

How long will be those some processes which takes longer at save time?

I am doomed because I can't find a solution to pass over this.

 

[Chris D]

They have a hefty farm of servers because they have a hefty number of visitors.

If you aren't going to have a hefty number of visitors then you should be ok but then I wouldn't understand why you would need so many nodes.

You would most likely need to test this yourself to know how it would perform on the resources you have available.

 

[ge66]

Why have so many nodes?
Can it be solved with prefixes instead? I have removed almost all my sub-forums and I’m using prefixes instead.
I should say that I don’t know what impact a lot of prefixes would have on a server.

 

[sbj]

I wish prefixes would help me. But what do you think about hundreds of prefixes? Is this a solution?
At least having subforums help users to find easily the correct place.
But 1 place with hundreds of prefixes will be a messy place. No structure and chaos. You can't find anything you are looking for.

 

[x3sphere]

I ran into significant performance issues with around 1000 nodes or so. This was on a dual E5 2620 with 64GB RAM too. Problems I noticed:

Index page speed rendering slowed down a lot, an initial page view went from around 150ms to 800ms.
Changing permissions on any nodes would take upwards of 3-5 seconds
Editing prefixes would fail because the dropdown box which contains all your nodes was so huge, caused some issue with the JS

I don't think XenForo is currently built to handle so many nodes. It certainly can if you have a beefy server, but performance is far from optimal, and you end up running into a lot of usability issues on the admin side, as the interface doesn't scale to so many nodes.

A good solution might be to create "virtual" nodes, in that you have a single node with set permissions, but multiple entry points. All threads would be stored in a single node, but filtered by these entry points. Obviously though this would only work if you could deal with having the same permissions for these virtual nodes. I might do something like this eventually as it'd work for my use-case.

 

[sbj]

Thank you so much for this feedback @x3sphere . Ok, with this information I can forget the idea of having so many nodes...

How did you solve your problem?

A good solution might be to create "virtual" nodes, in that you have a single node with set permissions, but multiple entry points. All threads would be stored in a single node, but filtered by these entry points. Obviously though this would only work if you could deal with having the same permissions for these virtual nodes. I might do something like this eventually as it'd work for my use-case.

With this?

 

[x3sphere]

That's what I'm thinking of implementing, haven't done it yet. It's for a project that is still in progress. I don't have an immediate need for it so I may hold off until 2.0 is released.

 

[Claudio]

I ran into significant performance issues with around 1000 nodes or so. This was on a dual E5 2620 with 64GB RAM too.

Using SSD or hard disk?

 

[x3sphere]

Using SSD or hard disk?

Just a regular old 7200 RPM HDD. I don't think that would have been a bottleneck though considering the entire DB fit in RAM.

 

[Claudio]

Just a regular old 7200 RPM HDD. I don't think that would have been a bottleneck though considering the entire DB fit in RAM.

If the database is in the same HDD as the files you should really consider SSD. There is a HUGE speed difference between both.

 

[Mike]

In terms of display performance, it's really just likely to have a linear slowdown where all nodes need to be shown/considered. The forum list being the main example here as it is loaded frequently and generates differently based on permissions (counts/last posts will be different, regardless of what's shown). This does come up in other places, but that's the most common. It has more significant effects on permission calculation for example, but that is done very infrequently (compared to the number of reads).

I don't think an SSD will make a difference as a well tuned MySQL will have the data in RAM anyway.