Sim
Well-known member
We migrated ZooChat last week from a heavily modified vBulletin v3.8 + PhotoPost and went live on Friday (also the site's 13th birthday - we're now a moody teenager!).
ZooChat :: Zoo and Animal Conservation Community
This initial post is to address some of the early findings relating to the "Can I have thousands of forums" thread, which was of concern to us before we migrated - Can I... (have thousands of forums)
I'll post more about what we've done on the new site later.
We originally had something like 2,200 forums because we had one per country (for discussions) and one per zoo (for photos), but those per zoo forums were removed when we migrated the photo gallery to XFMG as the photo discussion threads are now media comments (which introduced heap of new issued I'm going to need to fix because of a lack of functionality in XFMG).
So we're down to 160 nodes, which isn't causing any issues at all - but we've got 1,859 thread prefixes (again, one per zoo) which makes the thread prefix filter in the admin area a bit slow but I've not found any issues with the prefix filtering on the public side.
I did add a new thread prefix selector to the top of the thread list (the one at the bottom is useless - members don't even know it's there!). Thread prefixes are grouped by country and only those prefixes for the country we're in are displayed, so it works well.
We have 2,216 media categories which again makes the category filter pretty slow in the admin area (I've got a fast/powerful computer, so it's not too bad), but the public navigation works pretty well - I have the category navigation set to "Collapsible" style and it works well although the zoo lists for some countries is very long!
Because we've grouped the zoos by country and then by region, it's not too bad to navigate so many of them. A filter on the public interface would be nice here, but again would probably struggle for performance because of the number of categories.
With 280,000 photos in the gallery we found that loading the main media page was very slow because it tried to paginate every photo, so we used the "Only show media added within the last X days (Media Home)" option set to 30 days to improve performance there.
We also found the Recent Comments block in the gallery sidebar was incredibly slow (as in 16 seconds to complete the query!) because it was joining in just about every table and several of those joins (namely those referencing albums) were doing full table scans - and with 280,000 photos that was really slow.
So I had to rewite the query which generates the recent comments (basically I removed the album joins) and adjust the
getCommentsForBlock callback which generates the block.
We're not happy with the functionality of that comments block anyway, so I will likely be rewriting it from scratch to be much more useful. We have a LOT of comments in the gallery (345,000 comments for 280,000 photos), and in many ways, commenting on media is more important to a lot of our members than forum posts. My users are pretty disappointed with how unimportant comments seem to be in the gallery section now, so I'll be making a lot of suggestions for improvements in later versions of the product.
I'm going to have to do a lot of modification to the galleries to get the functionality our users want anyway - but none of that is especially related to performance or the large numbers of categories we have.
I should note that we had to migrate to a 12GB Linode (6 CPU cores) to fit all the galleries in (over 100GB disk space required) - so we'll be looking for a solution to move the media back to S3 (I had to pull everything back from S3 from our customised PhotoPost to be local ready for the migration), and we have a separate 8GB Linode (4 CPU cores) running as our database server (3.2GB data in the database, 4.5GB InnoDB cache allocated) and we're running a separate 2GB Linode (1 CPU core) for ElasticSearch. We also use APCu + Zend OPcache on the web server, so we've done quite a bit to help with performance. Even so, I noticed a significant increase in CPU usage on both the web server and database server, but they are still basically idling along nicely.
If I can fix the disk usage issues, I'll be looking to downgrade the web server to an 8GB Linode or perhaps even 4GB - will be interesting to see what happens to CPU load then.
Overall, the users are very happy with the migration so far and I'm surprised (and very happy) with how well things are performing for users. We only have 10,000 members, but they are a very passionate and vocal bunch (nearly 800,000 posts) and we get over 900k pageviews and 90,000 unique visitors per month.
I'll post more later about the new site.
ZooChat :: Zoo and Animal Conservation Community
This initial post is to address some of the early findings relating to the "Can I have thousands of forums" thread, which was of concern to us before we migrated - Can I... (have thousands of forums)
I'll post more about what we've done on the new site later.
We originally had something like 2,200 forums because we had one per country (for discussions) and one per zoo (for photos), but those per zoo forums were removed when we migrated the photo gallery to XFMG as the photo discussion threads are now media comments (which introduced heap of new issued I'm going to need to fix because of a lack of functionality in XFMG).
So we're down to 160 nodes, which isn't causing any issues at all - but we've got 1,859 thread prefixes (again, one per zoo) which makes the thread prefix filter in the admin area a bit slow but I've not found any issues with the prefix filtering on the public side.
I did add a new thread prefix selector to the top of the thread list (the one at the bottom is useless - members don't even know it's there!). Thread prefixes are grouped by country and only those prefixes for the country we're in are displayed, so it works well.
We have 2,216 media categories which again makes the category filter pretty slow in the admin area (I've got a fast/powerful computer, so it's not too bad), but the public navigation works pretty well - I have the category navigation set to "Collapsible" style and it works well although the zoo lists for some countries is very long!
Because we've grouped the zoos by country and then by region, it's not too bad to navigate so many of them. A filter on the public interface would be nice here, but again would probably struggle for performance because of the number of categories.
With 280,000 photos in the gallery we found that loading the main media page was very slow because it tried to paginate every photo, so we used the "Only show media added within the last X days (Media Home)" option set to 30 days to improve performance there.
We also found the Recent Comments block in the gallery sidebar was incredibly slow (as in 16 seconds to complete the query!) because it was joining in just about every table and several of those joins (namely those referencing albums) were doing full table scans - and with 280,000 photos that was really slow.
So I had to rewite the query which generates the recent comments (basically I removed the album joins) and adjust the
getCommentsForBlock callback which generates the block.
We're not happy with the functionality of that comments block anyway, so I will likely be rewriting it from scratch to be much more useful. We have a LOT of comments in the gallery (345,000 comments for 280,000 photos), and in many ways, commenting on media is more important to a lot of our members than forum posts. My users are pretty disappointed with how unimportant comments seem to be in the gallery section now, so I'll be making a lot of suggestions for improvements in later versions of the product.
I'm going to have to do a lot of modification to the galleries to get the functionality our users want anyway - but none of that is especially related to performance or the large numbers of categories we have.
I should note that we had to migrate to a 12GB Linode (6 CPU cores) to fit all the galleries in (over 100GB disk space required) - so we'll be looking for a solution to move the media back to S3 (I had to pull everything back from S3 from our customised PhotoPost to be local ready for the migration), and we have a separate 8GB Linode (4 CPU cores) running as our database server (3.2GB data in the database, 4.5GB InnoDB cache allocated) and we're running a separate 2GB Linode (1 CPU core) for ElasticSearch. We also use APCu + Zend OPcache on the web server, so we've done quite a bit to help with performance. Even so, I noticed a significant increase in CPU usage on both the web server and database server, but they are still basically idling along nicely.
If I can fix the disk usage issues, I'll be looking to downgrade the web server to an 8GB Linode or perhaps even 4GB - will be interesting to see what happens to CPU load then.
Overall, the users are very happy with the migration so far and I'm surprised (and very happy) with how well things are performing for users. We only have 10,000 members, but they are a very passionate and vocal bunch (nearly 800,000 posts) and we get over 900k pageviews and 90,000 unique visitors per month.
I'll post more later about the new site.
Last edited:
