Not a bug Potentially poor performance with lots of [GALLERY] BB code embeds

4

420

Active member
Hi Chris,

I'd like to chime in here to determine if Jake resolved this issue or not, since he imported our gallery photos via his own created import tool, instead of yours. We are still experiencing the same exact slow load times since his migration, yet he claims the problem should be resolved and that the issue is now your responsibility, after performing several self built scripts to perform certain changes. Pages can sometimes load perfectly fine, quick and speedy, then at random times page loads take up to ten minutes per page. We have lost 80% of our traffic since he performed the migration and everyone is complaining, both staff and members alike. Unfortunately, I am stuck in the middle and need to figure out who is the one to get us through this obstacle, before my 25 year old business is destroyed. How shall we proceed, please tell me whatever data I can supply, to get us all on the same page.

Thank you.
 
All of the items have been converted back into albums after the import, I haven't seen any more performance issues since that was completed
 
Here is what we have gathered from our team and members thus far, none of which was happening before the gallery migration.

  • Page load issues: (posts carried over 2 weeks and 30 pages - attempting to summarize, it’s a bit of a ramble)
    • Starting a week or two ago numerous members were reporting huge wait times for pages to load.
    • Is worst with image laden pages but happens on pages with only a few images too.
    • The conversation went back and forth nearly all weekend. Many having this issue.
    • Sometimes the loading takes so long that some images never land at all.
    • While this is happening replying, liking, composing, quoting all get hung up.
    • Most of us were holding off reporting this until the image gallery background processing was happening but are under the impression that has completed now and we are still having the issue.
    • Friday night and into Saturday (EST) was a particular time of slow site response all around FYI (folks were having trouble just posting responses, replies etc. and images were not loading into pages at all for many of us).
    • This all continued into Saturday evening and Sunday.
    • It should be noted that within this, on Saturday evening sometime, one or two folks reported blazingly fast browsing where images loaded instantly on even media rich pages (I’ve not ever had this experience since the site migration - but our member GT did).
    • Meanwhile, (so at the same time) many others of us were still having dialup type experiences - while browsing anywhere else except our site was perfectly fine. (MJ ran a web page test and linked it in this post (Sent to Conversation) and then DD ran one on a media rich page (Sent to Conversation).
    • Folks have identified big spikes in data usage, perhaps this supports the idea being tossed around that the site is not delivering device adapted media in terms of the size that’s being delivered (not uploaded - this is about what the site is ‘serving’ to us.)
    • I don’t think the developers can legitimately demand individual reports on the tech setup of each user reporting this experience when it’s an intermittent and widespread problem like this - and while all of us with the issue are having no trouble on any other site (I’ve checked via different devices and networks too and found the problem was the same regardless).
    • Some have reported that cancelling the page load and reloading fixed the issue - that doesn’t work for others though (myself included).
    • There were some issues for some logging into the site at various times during this period too. Main request coming out of this is for the site to properly resize the images for delivery like the previous site used to. (My data usage has tripled since the upgrade and that’s without even getting to see most of the pics I’m trying to load - I’m being patient with it but I imagine there are those out there seeing red over this circumstance).
    • I am linking here to the post where GT reported things getting faster (blazingly) to give you an idea of the time stamp - because of where i am I can’t always report accurately what time these things were happening without working out time differences - which I’m not going to do! (Sent to Conversation) It seemed to continue well for GT into Sunday. I can say though that at that exact same time others were still having serious issues.
    • Update on this today - it continues to be problematic for me (on multiple devices and different networks while all other usage is fine). Page 16 of my journal still does not load and while it's true there’s lots of images there, that’s not been an issue before the upgrade, and at the same time I have had similar loading problems on pages with only 2-3 images. I am usually waiting over a minute for only about 20% of images to land, if that. Sometimes I walk away and come back to find that the page load gave up.
    • Folks have stopped reporting here over the last few days, mostly because there has been no change and they are leaving for other sites - for me the forum experience is still completely hamstrung as far as image viewing goes.
 
In terms of the gallery, everything seems to be functioning very quickly at this moment in time. That isn't really indicative of any sort of inherent performance issue as they would tend to be fairly consistent.

The only thing I can recommend as an optimisation in terms of the gallery is further reducing the date limit (which looks to be around 180 days at the moment) and the next update has a further optimisation in terms of that applying to the gallery view pages too.

If this is a more general performance issue then realistically, this needs to be approached in a top-down way, likely at the server level, monitoring for which process/es are using resources, which resources are being used, how that compares to the amount of traffic at the time and that kind of thing. Having slow queries logged might help identify optimisations that could be made from our end. This is generally stuff your host or server admin will help you with.

If you can use that performance monitoring to track down when and where the bottlenecks are happening, that might lead to some solutions.
 
Chris D said:
In terms of the gallery, everything seems to be functioning very quickly at this moment in time. That isn't really indicative of any sort of inherent performance issue as they would tend to be fairly consistent.

The only thing I can recommend as an optimisation in terms of the gallery is further reducing the date limit (which looks to be around 180 days at the moment) and the next update has a further optimisation in terms of that applying to the gallery view pages too.

If this is a more general performance issue then realistically, this needs to be approached in a top-down way, likely at the server level, monitoring for which process/es are using resources, which resources are being used, how that compares to the amount of traffic at the time and that kind of thing. Having slow queries logged might help identify optimisations that could be made from our end. This is generally stuff your host or server admin will help you with.

If you can use that performance monitoring to track down when and where the bottlenecks are happening, that might lead to some solutions.
Click to expand...
The problem as reported, has been sporadic, but frequent.

Submitting a support ticket to our Systems Admin now to begin gathering this data for you.

Thank you.
 
Just clicked on a link to a thread from an email notification, it's been loading for 4 minutes, with just "Waiting for XXX.com" at the bottom left of browser.
 
Actually, I can see some poor performance and of course now fully read your previous message (which I didn't read until after I submitted mine) and I note you mention "image heavy" pages - would that be specifically like the second example link you sent through in your PM? The one on page 124 of a thread?

Admittedly you have an insane amount of media loading on that particular page, but I would like to be seeing it load quicker than that.

I'm going to have to get my thinking hat on and figure out how we can improve that for you (and likely fairly quickly) as I can see how problematic it must be.

Would it be an issue for you if the images being loaded were actually thumbnail size (squares), rather than the 500px ones that are loading at the moment?
 
Chris D said:
Would it be an issue for you if the images being loaded were actually thumbnail size (squares), rather than the 500px ones that are loading at the moment?
Click to expand...
Unfortunately, we need to keep the standard of what we've had for decades of full size images embedded into posts, otherwise we are being asked to go back in time like Tapatalk tried, forcing us into attachments. We are trying to evolve, not devolve, hence getting out of VB and into XF after all these years. If we can't at least have what we had before, then what's it all worth anyway? Doesn't make sense, we really need to hustle to get this resolved, please I beg of you.

Thank you
 
Chris D said:
Actually, I can see some poor performance and of course now fully read your previous message (which I didn't read until after I submitted mine) and I note you mention "image heavy" pages - would that be specifically like the second example link you sent through in your PM? The one on page 124 of a thread?
Click to expand...
I just sent you the paragraphs with the working links. Sorry, playing middle man with a thousand angry people is overwhelming and confusing.
 
Don't worry about trying to feed bits and bobs back to me. I can see there's issues, let's just focus on at least a temporary workaround until we can implement something more robust.
 
Chris D said:
I mostly meant as a temporary workaround. You know, so your site actually starts working again ;)
Click to expand...
No sir, I want to fix this problem without devolving. We already look like a bunch of amateur losers to our community, that's enough. Thank you. ;)
 
FWIW, I'm temporarily suggesting that we change the image embeds so they look just like thumbnails.

I know that's not a permanent solution for you, but if what I think is happening is happening, then this should drastically improve loading times and performance across the entire forum.
 
Last edited:
420 said:
No sir, I want to fix this problem without devolving. We already look like a bunch of amateur losers to our community, that's enough. Thank you. ;)
Click to expand...
Frankly, that's not going to happen in a matter of hours or potentially even days. It's up to you whether you accept the temporary workaround, or not, but it seems silly to refuse something that would get the site up and running properly potentially immediately vs leaving it in its current state and barely working for people.
 
Chris D said:
Frankly, that's not going to happen in a matter of hours or potentially even days. It's up to you whether you accept the temporary workaround, or not, but it seems silly to refuse something that would get the site up and running properly potentially immediately vs leaving it in its current state and barely working for people.
Click to expand...
With all due respect, I have 25 years of experience managing, building and directing forums since the very first days of VBulletin, and even DCScripts before that and BBS before that. So please don't call my decisions "silly", as that appears to be offensive. After spending $20000 over the last 6 moths preparing for our evolution from VB to Xenforo, only to be given slow load pages and thumbnails in threads, is simply not acceptable and we need to work together to find a solution, without insulting each other. We ditched Tapatalk because they forced us into attachments with thumbnails already, so let's do what we can to get through this obstacle quickly and pain free as possible for us all. I am at your service for whatever you need, feel free to jump on a conference call with us and your team, if desired . Thank you for your support, we are truly grateful.
 
Chris D said:
FWIW, I'm temporarily suggesting that we change the image embeds so they look just like thumbnails.

I know that's not a permanent solution for you, but if what I think is happening is happening, then this should drastically improve loading times and performance across the entire forum.
Click to expand...
Please send me that in our conversation, so I can see and reply. Thank you.
 
I meant no offence by it, I just wanted to highlight the point that it makes a lot of sense to implement a temporary workaround in lieu of a permanent solution which, if you want it to be correct, will take some extra time. In the time we've spent discussing it, the workaround could have been implemented (which may or may not have confirmed if my theory about the problem was correct) and I might have even been able to start work on a permanent solution already.

Your images currently look roughly like this:

1520456422804.webp

The temporary workaround would make them look like this, instead:

1520456485975.webp

In addition to the file size of the images being drastically smaller, and therefore quicker to load, these thumbnail images come directly from the file system, rather than being loaded via a PHP script which may be adding some overhead, especially when there's lots of images on the page. That overhead is likely having a compound effect and generally slowing the rest of the site down (whether or not the pages have lots of images).

At the very least, I'd say it's worth a try and it will help confirm if my current line of thinking is correct.
 
Top Bottom