So much AI, so many add-ons... and so many bugs
- Thread starter PhineasD
- Start date
I have no idea what the above means, can you explain in plain English?
By amalgamating I presume you mean merging similar threads? I can do that myself as I often curate material for SEO and searchable, though am very wary of doing it in a way that does not frustrate members.
But I’m not sure what is meant by level-up or contextualizing.
Sure - that's my plain English tho; words all chosen by me as I typed the post. To simplify:
AI is better at ... triangulating... (these are good words and applicable lol)... let me try again.
AI can assemble vast amounts of data faster and better than humans can. My forum has about 220,000 posts. Not nearly as many as some of the more successful forums on here, but it's still more content and context than I can easily recall or associate in my own mind when trying to respond to a thread to help a member by responding to their post.
In seconds, AI can conversationally point a user to multiple related threads in direct response to a fresh question. Moreover, if the answer isn't available within the provided data set, or.if more context is required or prudent, it can fall back on the chosen language model (Claude in my case, but could be chatGPT, etc), which opens up a world of context to ensure the member gets a fully coherent and informed response. (ForumBot handles all this by the way; forum owners don't need to hook in their own AI accounts or licenses, and they can define their own agent instructionsso that it follows your forum's guidlines, mission, tone, etc. etc).
That's better than any human can do, at minimum by measure of the clock. The 1:1 chat nature of ForumBot allows your members to do this kind of discovery and "leveling-up" without having to appear as a "newbie" on the public forums. I think this is good for a strong community. I'm not saying "no newbies," I'm saying more newbies will be converted to knowledgeable members when infusing AI in your forums.
I'd argue a smarter community leads to a more successful and more active one.
To be clear: by "level up" I mean, making your membership smarter about your forum's subject matter. AI can answer a user's direct questioning on a topic and in a 5 or 10 minute session they can become much smarter about a topic than they would by browsing Similar Threads.
So in a non-AI enabled forum, a user might browse forums, use Seach to hunt for good threads using keywords, and might even post an open-ended question. Or, probably more likely, move on because they prefer to lurk instead, because they don't like to post on forums. They'll accept the knowledge gap.
Ultimately, the user of the AI-enabled forum is better informed (leveled-up) more quickly and richly when compared to using traditional forum tools and methods.
Together, the forum format can still thrive. To the point in my prior post, the only reason CBS and The New York Times are still relevant (barely, in some respects) in the age of the Internet is because they found ways to ride the Internet wave rather than shun it entirely. I'd argue forum owners should do the same with AI. In this context: Can't ignore AI.
AI is better at ... triangulating... (these are good words and applicable lol)... let me try again.
AI can assemble vast amounts of data faster and better than humans can. My forum has about 220,000 posts. Not nearly as many as some of the more successful forums on here, but it's still more content and context than I can easily recall or associate in my own mind when trying to respond to a thread to help a member by responding to their post.
In seconds, AI can conversationally point a user to multiple related threads in direct response to a fresh question. Moreover, if the answer isn't available within the provided data set, or.if more context is required or prudent, it can fall back on the chosen language model (Claude in my case, but could be chatGPT, etc), which opens up a world of context to ensure the member gets a fully coherent and informed response. (ForumBot handles all this by the way; forum owners don't need to hook in their own AI accounts or licenses, and they can define their own agent instructionsso that it follows your forum's guidlines, mission, tone, etc. etc).
That's better than any human can do, at minimum by measure of the clock. The 1:1 chat nature of ForumBot allows your members to do this kind of discovery and "leveling-up" without having to appear as a "newbie" on the public forums. I think this is good for a strong community. I'm not saying "no newbies," I'm saying more newbies will be converted to knowledgeable members when infusing AI in your forums.
I'd argue a smarter community leads to a more successful and more active one.
To be clear: by "level up" I mean, making your membership smarter about your forum's subject matter. AI can answer a user's direct questioning on a topic and in a 5 or 10 minute session they can become much smarter about a topic than they would by browsing Similar Threads.
So in a non-AI enabled forum, a user might browse forums, use Seach to hunt for good threads using keywords, and might even post an open-ended question. Or, probably more likely, move on because they prefer to lurk instead, because they don't like to post on forums. They'll accept the knowledge gap.
Ultimately, the user of the AI-enabled forum is better informed (leveled-up) more quickly and richly when compared to using traditional forum tools and methods.
Together, the forum format can still thrive. To the point in my prior post, the only reason CBS and The New York Times are still relevant (barely, in some respects) in the age of the Internet is because they found ways to ride the Internet wave rather than shun it entirely. I'd argue forum owners should do the same with AI. In this context: Can't ignore AI.
I think that works in a fact-oriented area. In something like religion, the subject of my forum, a lot is subjective and the discussion is more about what each member BELIEVES rather than hard, objective facts. No one can prove "God" or anything about "God", just state what they think based on scripture and other sources. Sure, there's times when being able to find specific bible passages or the ideas of specific theologians/philosophers has merit, but even those discussions are rarely "asking a question to find an answer" but more about using those to show where one's belief(s) originated. So I think the degree to which AI is useful in the way described varies depending on forum, community, and subject matter. Not disagreeing with you, @tonmo, just thinking that we shouldn't assume AI has the same value and impact for all communities. I already have a fundamentalist, evangelical frequent poster weaponizing AI to find support for their rigid, intolerant faith positions. Last thing I need is all my members doing that or an AI being skewed by that person's posts.
On this topic: I am sure you've noticed your CTR crashing since Gemini now prominently intercepts all Google Searches with AI responses. In the case of my forum, just a year or two ago, a question like "How many hearts does an octopus have" would have generated a bunch of clicks. Those clicks show a stark downward trajectory. I'm very specifically citing clicks: impressions have actually increased in some cases.
The answer to that "how many hearts" question is fact-based (three), and that's just a simple example. For a long time now, Google would convert your cups to liters and whatnot, but now it can do all that on any topic, including all the topics being covered by forum admins around the globe. That's an enormous change for forums.
However, AI still cannot replace human connection, and there are obviously many things humans can do that AI cannot do, and so I believe forums have an opportunity to make a big comeback.
It's funny that we essentially posted at the same time, and there's overlap in our discussion on fact-based material.
Without getting into it: covering "cephalopods" as a content area is actually incredibly deep and broad, and yes it does cover facts (three hearts) but it goes far beyond that (home aquariums, ethics, emerging technology, species identification (AI not there, yet) regulations, art, etc. etc.)
I'd imagine there are some fact-based questions that are pertinent to your community as well, but I get your point about the essence of your content. With ForumBot, I instruct it how I want it to respond to various topics, and I keep it all confined to the scope of my forum.
I also believe this is possible, but maybe it won’t happen while so many people believe ai is the be-all and end-all and is always correct.
One problem I see is that there are a lot of egotistical “influencers” out there who spout myths or unproven theories as fact, then get scraped by ai to produce a lot of the slop we see.
Luckily we don’t have too many questions like that in my area, but this example got me thinking that one big loser in that regard may be Wikipedia, which is sad because over time it has become a good resource.
I'm guilty of that. I'm trying to minimize though but some of my addons are custom and absolutely needed.Yes - custom addons pay for my site via user upgrades.
AMAZING IDEA - this would be a perfect solution @Slavik
The idea is that the developer pays for this to become verified.
So the plan would be to get the dependable, and trusted, developers to pay for our security? And they're going to be paying enough to cover the cost of one of the XF staff to take the time to inspect their add-ons to make sure they're certifiable?
And many malicious users will also use AI to create add-ons for popular software packages. I saw this in other projects already. Suddenly someone that just registered post his new add-on for everyone to install.
This is a good reason to have a vetted system for add-ons.
So the plan would be to get the dependable, and trusted, developers to pay for our security? And they're going to be paying enough to cover the cost of one of the XF staff to take the time to inspect their add-ons to make sure they're certifiable?
This is how it works in every software store around the world. Given that Xenforo is the main directory for new add-ons and doesn’t take a percentage of sales.
The only option is to charge for a code review and a “verified” status, and give as much visibility as possible to those who pay for this status.
With featured directories, spotlights, and so on. It’s basic “community management” on top of the technical aspect of code review.
It’s certainly not a perfect solution. But directing 90% of the community toward 5 or 6 developers, I think, that’s a huge help in terms of sales.
Are there definitely problems? Yes. Like updates (do they need to be reviewed or not?) and how does code review work? (I have no idea; I’m not a dev).
What I’ve limited myself to saying is that:
- If I were a platform, I wouldn’t abandon my customers.
- I’d try to leverage my directory power as much as possible.
- I’d try to protect my developer ecosystem, given that I currently “offload” quite a bit of work to them since my platform is versatile but far from complete.
It’s certainly a business direction that needs to be discussed and one that can’t be reversed on a whim.
I would definitely create a verified directory, even with Xenforo Cloud on my hands.
Discourse did the same by integrating the most widely used add-ons into their software.
IPB decided to close their store. But they’ve taken a business direction focused on selling a “360-degree complete solution” with add-ons implemented “in-house.”
They aren’t interested in extending their platform as much as possible. They’re interested in selling a complete package at a B2B price point.
Slavik asked for an opinion, and I gave it.
I don’t work for Xenforo, but if I did, I would make these points.
For example, I would conduct much more analysis on my store. I would try to see which add-ons are the most downloaded. I would contact the developers. I would figure out if I could integrate them into the core (as has already been done with other add-ons...).
There are various business decisions i would make related to the developer landscape. This would be just one of many.
There is hardly any developer today who is not using AI to check or optimize code.
AI significantly boosts the productivity of experienced developers and can greatly improve code quality. Few developers would want to miss that advantage.
At the same time, AI enables inexperienced developers with little coding knowledge to create poor-quality add-ons. AI is not the magic solution many people believe it to be. Without proper guidance and expertise, it often generates just bad code.
So both statements are true: AI can make code better, and AI can make bad code.
The real difference is the person using AI. That should be an important factor when deciding what to buy and what to avoid.
For anyone interested, @eva2000 has documented his AI journey and shared practical examples of how AI can be used to improve code quality and development workflows:
ai.georgeliu.com
AI significantly boosts the productivity of experienced developers and can greatly improve code quality. Few developers would want to miss that advantage.
At the same time, AI enables inexperienced developers with little coding knowledge to create poor-quality add-ons. AI is not the magic solution many people believe it to be. Without proper guidance and expertise, it often generates just bad code.
So both statements are true: AI can make code better, and AI can make bad code.
The real difference is the person using AI. That should be an important factor when deciding what to buy and what to avoid.
For anyone interested, @eva2000 has documented his AI journey and shared practical examples of how AI can be used to improve code quality and development workflows:
ai.georgeliu.com
I know we're generally speaking of AI here, but different models can give different outputs. This was from ChatGPT, and Claude picked it up right away:
Wondering if a human reviewing 1000 lines of code could recognize the bug there, as it does work 90%+ of the time. It might not be found until reported, and when pressing ChatGPT to ask what the issue could be, it wanted to debug JavaScript, JSON, etc., before it even recognized that at all.
I suppose it's not a huge issue, but if glanced over by a human and they say it's okay, then what else could be wrong?
Surely, a developer can see it as it's isolated here. But the premise of the exercise is whether they would pass it with 1000+ lines or not.
PHP:
if (
strlen($value) > strlen((string) PHP_INT_MAX)
|| strcmp($value, (string) PHP_INT_MAX) > 0
)I suppose it's not a huge issue, but if glanced over by a human and they say it's okay, then what else could be wrong?
Surely, a developer can see it as it's isolated here. But the premise of the exercise is whether they would pass it with 1000+ lines or not.
Highlights a few things
- Which models you use and how you use them can make a difference - context management and prompting and code hardness tools/skills used
- Never rely on a single AI model - use multiple AI models
- Verification and testing/review of code matters
- Learn how to properly use AI
Each AI model picked up something uniquely that other AI mdoels didn't so synthesized conclusion is more well rounded and accurate
Last edited:
Thanks for that analysis, @eva2000. Quite expensive to find a bug (~$0.60) in just the provided line. I'm assuming your tool could find issue with it in 1000+ lines, but would the cost exponentially increase?
That'd be an issue with verification, if so. Each update, verified, costing as much as what 1 license may cost.
The argument is also vetting. Would AI vet, a human, or a human interpretation of an AI scan? I don't know by your answer if you saw the issue before running it through AI. If it's up to AI vetting, and there are conflicting reports, if 3/4 models pass, is it deemed safe?
Surely, and hopefully, exploits would be caught, but maybe not XenForo specific permission "errors".
However, that one simple line could be breezed over by a human, and not discovered for a while on production/live as it works 90% of the time.
That'd be an issue with verification, if so. Each update, verified, costing as much as what 1 license may cost.
The argument is also vetting. Would AI vet, a human, or a human interpretation of an AI scan? I don't know by your answer if you saw the issue before running it through AI. If it's up to AI vetting, and there are conflicting reports, if 3/4 models pass, is it deemed safe?
Surely, and hopefully, exploits would be caught, but maybe not XenForo specific permission "errors".
However, that one simple line could be breezed over by a human, and not discovered for a while on production/live as it works 90% of the time.
Well cost is relative, GLM 5.1 is on my $129.60/yr coding plan and Deepseek costs do seems misreported - will need to get on that as it's costs are https://api-docs.deepseek.com/quick_start/pricing/ - DeepSeek V4 Pro is $0.435 per 1m input tokens and $0.87 per 1m output tokens. My entire month DeepSeek V4 Pro costs so far is only $0.55 total!
haha AI! fixing bug now
Cost argument can go both ways - cost of letting a mistakes overlooked spread through an entire code base + the cost of fixing the mistakes versus cost of getting it right the first time
Last edited:
Exactly. But who bears the cost? That would be a trust factor, too. If it goes back on the developer, we're right back at square one. If on XenForo, that could open them up to liability (the license agreement would still allow for error), if not a trust decline in the system altogether.Cost argument can go both ways - cost of letting a mistakes overlooked spread through an entire code base + the cost of fixing the mistakes versus cost of getting it right the first time
From what I understand from this thread is that there is a problem identified — AI — and from responses, no real solution can be reached.
Mentioned earlier, there was a dev that lost reputation, banned, and came right back. An endless loop of that? Vetting devs one by one as "trusted/verified" by XenForo (the liability issue again, or loss of trust and confidence in XenForo if that user eventually failed)? Close Resources completely? A third party check? Or leave it as is for users to decide?