Can I... (have thousands of forums)

[Birchy]

xenForo comes a lot closer to what I need to have in a Forums module on one of my commercial sits. The biggest obstacle for all I've considered, is that it needs to be able to handle, in addition to a handful of general forums, at least 30,000 town forums, browsable by state, and having about six forums per town.
Let's try a diagram:


General Forums
-stuff
-more stuff
-even more stuff

Location Forums
-Alaska
---Fairbanks
------Town stuff
------more Town stuff
------even more Town stuff
------yep, more Town stuff
---Nome
------Town stuff
------more Town stuff
------even more Town stuff
------yep, more Town stuff

etc

So, this is taxing most forum packages. Can xenForo be modified to do this?

I save the user data integration into other software question for after this thread.

 

[tajhay]

You dont need that many forums. Use prefixes.

 

[Mike Edge]

Why so many towns and why 6 per? I'd do it more like http://www.city-data.com/forum/ imo..

 

[Pierce]

On another note I don't think anybody's computer/ phone could render 30,000 nodes on the front page.

Are you already doing this somewhere?

 

[Mike]

I would certainly expect performance issues with 30,000 forums (I think you're actually implying 180,000 with 6 forums per town). A forum is a pretty big unit itself, as it can have distinct permissions, so there's a fair amount of overhead to have one.

 

[Sim]

This is something I'm going to have to deal with shortly. I've been putting off migrating my ZooChat forums across from vBulletin until I have had a chance to work out how to structure things.

My current structure is to have one forum per country (around 115 I think) and then use prefixes for each zoo around 2,000 zoos in total.

But then I've modified the forums to use threads for photo gallery (PhotoPost) comments, and so there is another forum for each photo gallery, so we have well over 2,000 forums on the current site. And the CPU runs at about 150% constantly (on a 2 core machine with 4GB RAM and caching turned on)

I haven't started testing on XenForo yet to see how well a similar structure performs, but I think I'm going to have to try a different approach anyway.

 

[Mike]

The issue isn't so much when you view a particular forum. The problem comes on pages where we work with a full node list. The main instance of this is the forum list page, but it does also apply to things like moving a thread. There's also challenges in terms of cache building, as we precalculate permissions for every forum for each permission combination (roughly each permutation of user groups applied to users).

On the forum list page, we need to check permissions for every node. To do this, we load the permission cache which will have a bunch of entries per node. Of course we also need all of the node data (general and type specific) which we "walk" over to get correct last post information, for example. Loading and processing all this data adds up.

Hence using prefixes or other somewhat "lighter" options can make the system run better overall.

 

[Kintaro]

Hence using prefixes or other somewhat "lighter" options can make the system run better overall.

the only problem I see here Mike is that prefix are not in the url so they lose some SEO. I added two additional examples HERE right now.

 

[Sim]

the only problem I see here Mike is that prefix are not in the url so they lose some SEO. I added two additional examples HERE right now.

Well yes, but neither are forum titles in the URL, so Mike's suggestion doesn't actually change that.

Of course a related but separate issue is that prefixes don't really get their own unique landing page (it's a filter for the thread list page), so can't be used for SEO benefit, so your point is still kind of valid in that there is likely going to be a SEO hit from using prefixes rather than forums.

I haven't thought through the implications fully yet to consider whether there is a workable alternative.

 

[Amaury]

but neither are forum titles in the URL

Forum titles are in the URL... For example: https://xenforo.com/community/forums/xenforo-pre-sales-questions.5/

 

[Sim]

Forum titles are in the URL... For example: https://xenforo.com/community/forums/xenforo-pre-sales-questions.5/

Forum titles are in the forum URL, they aren't in the thread URL. Prefixes are associated with threads, not with forums.

But yes, the point still remains that there isn't a URL landing page which contains the prefix in the title and in the URL like we get for forums.

 

[Mr Lucky]

I wouldn't even worry about what's in the url for SEO.

 

[Birchy]

Thanks for all the replies! I don't understand most of them, but the most important takeaway seems to be the issue of the package wanting to be able to list all the forums in one index page. And that's the biggest problem with the package that I'm using now (aside from the fact that a server move last year broke some parts that I haven't had time to fix, since I'm abandoning it anyway, and it's a feature that isn't critical to my site yet). I'm trying to avoid having to write a forum section from scratch, or heavily modifying a free or low cost package. I don't like the details of a lot of forums and their packages, but when I saw xenForo, most of what I want in features and details is already there. I need to disable the page that will try to show all the forums. I just need to link to the six forums for each town (and some general topic forums that aren't town-specific) from the non-forum parts of my existing site and return to the main site, as well as manage the user login and registration. I will need to assign admins to specific town forums and the six children forums for each town, as well as site admins permissions. Visitors need to be able to view everything, but must be users to post. I also need to have media galleries for each town, with multiple albums per town.

 

[James]

need to disable the page that will try to show all the forums.

Don't forget this applies to more than just the forum list. It's loaded during inline moderation and the quick navigation etc.

 

[Birchy]

Why so many towns and why 6 per? I'd do it more like http://www.city-data.com/forum/ imo..

That site is similar, but with more towns and with 6 forums per town. But at least it lets the user drill down one level. I just need to drill down one more level and handle a lot more towns, not just the major metroplexes. Some day, I hope to break the larger towns into neighborhoods, much like NextDoor.com does. But by that time, I would hope that I could afford to write the package from scratch.

 

[Birchy]

I suppose my "six forums per town" could be implemented as five "pinned/sticky threads", but I'd lose some granular statistical reporting (perhaps?). Still, it would be a good compromise for the short term. As long as pinned/sticky threads can be used by visitors to post etc as any other thread, just using the pinning to force the pre-determined sub-topics to the top of the list, as opposed to using them as announcements/FAQs/Rules etc.

 

[Kintaro]

@Birchy another way to enhance granularity are "thread tags" maybe with a modified view of "thread tags" and "prefixes" you can archive what are you looking for without the need to create too many nodes (sub-forums).

 

[Sim]

I posted some initial findings from our migration last week over here: https://xenforo.com/community/threads/zoochat.122322/

In short:

• we've cut our node list from over 2,200 down to 160 - so no problems there, but my test server seemed fine even when we had over 2,200 nodes in the list - although there was no load on that server
• 1,859 thread prefixes (one per zoo, grouped by country) is no problem, although the filter in the admin view is pretty slow. No problems on public side though.
• 2,216 media categories also causes the filter in the admin view to be very slow, but no problems with the category navigation on the public side
• Need to limit the number of recent days of media displayed on media home page
• Need to remove or modify the recent comments query in the media sidebar (16 seconds to complete query with 280,000 photos in the gallery!)

Overall, it's a lot better than I had expected. See my ZooChat thread linked above for more details about our server environment and other things I found with the migration.

If I find anything else which is causing issues performance-wise, I'll update things.

 

[Alpha1]

Your site works well indeed. Nice site btw!