So much AI, so many add-ons... and so many bugs
- Thread starter PhineasD
- Start date
Yeah, that works when there are thousands of developers footing the bill in large ecosystems. However, when there are less than 50 active developers here (maybe less than that who are actually "certifiable") is it worth it to dedicate a staff member's time to examining code and certifying developers? Of course this depends on how much the developers would have to pay for this certification, but I'm not so sure it can be sustainable on this small of a scale. It's a great idea, in theory, as I've said before, and if the staff could just certify a few add-ons at the start of each developers certification and be done with it, I might have a different opinion, but I'd imagine this would have to be an ongoing thing. They'd need to continue to monitor all of the certified developers to make sure they're still putting out quality work that's worthy of that certification. What happens when XF certifies a developer then they get lazy, burnt out, or just want to start pushing out more add-ons more quickly to make a quick buck? Of course we all know which developers wouldn't do this, but there are some who would. Then we're right back to where we are now except with "certified" developers. Not only that, but now XF's reputation would be tied to all of these developers. Regardless of what their ToS says, their reputation would still take a hit as soon as one of these "certified" developers puts out something that causes issues with someone's site or server. I'd also expect their own ticket system to see an uptick in reports from issues with developers just because they've put their stamp of approval on them. To me, it just creates a logistical nightmare for a small company.
I don't mind anyone using ai to create add-ons. Nobody can denie how incredible smart AI has become. AI is being used by more and more IT companies and even replace junior positions left and right. But when you're selling a product there should be proper understanding of the code and all the testing being done for it to function as it should and it be safe for production.
I wish there would be a code check by staff before any resource can be uploaded to Xenforo but I don't think any staff would have the time for that.
If you can't stop it you probably have to help to make it better. So perhaps if Xenforo or a helpful community member to really dive deep into Xenforo and create guidelines and instructions for proper add-on development using AI and testing instructions for after development. You really can't stop AI so what do you do?
I wish there would be a code check by staff before any resource can be uploaded to Xenforo but I don't think any staff would have the time for that.
If you can't stop it you probably have to help to make it better. So perhaps if Xenforo or a helpful community member to really dive deep into Xenforo and create guidelines and instructions for proper add-on development using AI and testing instructions for after development. You really can't stop AI so what do you do?
Most don't do it because it gives them money, it's more to avoid anonymous or unknown developers from posting add-ons, so those app stores mostly request a payment from the developer as sort of extra verification step.
One way Xenforo could do it is just allowing valid licenses holders to publish add-ons, since its not free software in the first place, they can already leverage their existing account system to vet who is posting add-ons.
Reviewing every add-on would be cost prohibitive and still miss things giving users a false sense of security. Just let the community do that work. Users will quickly spot any bugs and issues which can be read by others, if its malicious they will report it quickly.
It's impossible to know if something is AI written today. And that is also not what I would suggest. Using AI does not mean an add-on is automatically bad. AI is just a tool. But AI allows many people which don't understand code posting sloppy add-ons which don't work properly, or worse, could be malicious or by accident introduce security issues.
If an add-on is AI or not is irrelevant. The add-on just needs to be:
1. Vetted, only valid license holders can post on the add-on section
2. Review system with comments from others, there you can spot bugs or read technical issues
3. Report the add-on (malicious) so they take it down
A banned developer can certainly come back. No system is perfect, and it's not meant to be. It's just putting additional roadblocks on purpose.
Example: If you close the account of the malicious developer, they can come back with a fake account, but need to buy a new license again to publish to add-on section. That increases the cost for a malicious actor.
By malicious I mean someone that is intentionally creating an add-on to steal information or hack into xenforo users. Not someone that just posted a sloppy add-on that has bugs.
Last edited:
That's the system now, so the solution in place is fine?
The only way this might work is by putting restrictions on the actual XenForo customer account that ties back into the forum, as you can add multiple forum users to a XenForo account that'd appear as a "licensed customer" here. That would force them to buy a new XenForo license under a different alias if they ever wanted to weasel their way back, or plead their case in private (support) to come back.
That's what github does for Obsidian.MD addons.
Back to the OP of the thread:
I personally wouldn't mind it if a custom field is added to the XF resources section called "AI Assisted". If the check box for AI Assisted is ticked, then in the area where
And as for my 2c:
At this point in the game, AI is here to stay, and whether we like it or not. There is no denying that it has helped (for better or for worse) with code/development. I've used various AI providers over the past few years to sanity check code I've made - sometimes it made a disaster of the code, other times it improved/simplified the code. In another instance, I've had Grok code a tiny 'web app' that integrates servo control up to 16 channels, and two real time cameras, and all settings to be stored into a SQLite DB. It legitimately functions and works... however................
However, the massive drawback is fairly simple with AI (maybe this will change some day): The code generated is only as good as you describe it. You have to know what the outputted AI-code does in the first place, otherwise you're going to be in for a headache. If you don't have a lick of an idea how python or php 'works', you're in for a bad time.
Telling AI to "Create a PHP script that outputs time when accessed" is one thing, as it shouldn't be any more difficult than it adding just this single liner:
I personally wouldn't mind it if a custom field is added to the XF resources section called "AI Assisted". If the check box for AI Assisted is ticked, then in the area where
Compatible X versions, License, Visible Branding` is located on the resource display, it would add a small notice stating "This resource utilized Artificial Intelligence to assist in development." Nothing more.And as for my 2c:
At this point in the game, AI is here to stay, and whether we like it or not. There is no denying that it has helped (for better or for worse) with code/development. I've used various AI providers over the past few years to sanity check code I've made - sometimes it made a disaster of the code, other times it improved/simplified the code. In another instance, I've had Grok code a tiny 'web app' that integrates servo control up to 16 channels, and two real time cameras, and all settings to be stored into a SQLite DB. It legitimately functions and works... however................
However, the massive drawback is fairly simple with AI (maybe this will change some day): The code generated is only as good as you describe it. You have to know what the outputted AI-code does in the first place, otherwise you're going to be in for a headache. If you don't have a lick of an idea how python or php 'works', you're in for a bad time.
Telling AI to "Create a PHP script that outputs time when accessed" is one thing, as it shouldn't be any more difficult than it adding just this single liner:
<?php echo date("Y-m-d H:i:s");. But, telling it "Create a php script that is an addon for XenForo 2 that will display the current time to the user at the top header hook, with the full date-time being able to be configured in the user control panel". The AI will have not a clue how to handle that correctly. As you're leaving out install/uninstall instructions, SQL Table modifications, etc.
That won't be the issue in future, it's whether the better and more expensive AI models are out of reach of everyday users and restricted to those that can afford it.
Absolutely. Given that loads of companies that switched their development to AI (and got rid of hoards of expensive developers in the process) burned through their Token budget for the whole year in just one quarter and at the same time we are still in the phase were AI companies want to gain market traction and therefor don't make profits, burn capital fast (that's why they are going to the stock market right now) and sell their tokens/access cheap and below cost the future will clearly be different. Good models will be very expensive and even now it turns out: An experienced developer earns more money, but needs way less tokens to produce an outcome. A cheap and unexperienced developer burns through tokens like a hot knife through butter, generating excessive cost for tokens, and still delivers less and lower quality output.
We've adopted AI usage pretty well at work (luckily no you must use it for everything, but with how we want to use it) and yeah it costs a lot.
Our yearly budget that was setup around March/April was set to $250,000/yr. We were on track to hit $600,000 for our first "year" just by May lol. Now we all have $1000 monthly limit usages for Claude mainly. ChatGPT is still on a sub plan iirc.
We'll be able to see just how expensive everything will be soon though. Anthropic is working on their IPO so we can finally see their actual cash flow and at that point it's time to start sub pricing maxxing by the companies.
Our yearly budget that was setup around March/April was set to $250,000/yr. We were on track to hit $600,000 for our first "year" just by May lol. Now we all have $1000 monthly limit usages for Claude mainly. ChatGPT is still on a sub plan iirc.
We'll be able to see just how expensive everything will be soon though. Anthropic is working on their IPO so we can finally see their actual cash flow and at that point it's time to start sub pricing maxxing by the companies.
wow, bill shock anyone? 
Seriously doubt AI labs will make back their investments in the next 5-10yrs. AI labs maybe made of venture funded $$$$ but most of their customers aren't and don't have endless supplies of $$$$
Last edited:
I think AI coding is amazing. I think its more the people trying to cheap skate their way through coding with sub-par products and zero analysis. @eva2000 recently highlighted to me the importance of having multiple agents look at the same code, as they have their strengths and weakness. That works.
My wife's company uses MYOB heavily for clients, MYOB hasn't done anything with their time billing component in 5 years, because that part of their software is really a support mechanism for bookkeepers and accountants, not the broader client. Simply, its crap nowadays and no longer does the job needed.
So she asked me if I could build her something custom for internal use. I thought it would be a good challenge - and AI has helped me solve many problems, a custom system to suit her company and employees, Python backend, React frontend. Now I guess if I relied on cheap AI, then it would be a coding mess, likely nightmare and unusable, instead I rely on the expensive end of the market, looking through code and finding issues, helping me code entire components and even pages with me just reading over it. More than 95% of the time, I give AI coding agents a big tick. Again, upper end stuff, not the free end. It will cost me nothing more than a few hundred bucks for the entire system, perfect, security focused, locked down in every aspect outside of its use. End to end encryption. Meets all financial system regulations.
I wouldn't have been able to do it without AI - I'm not that good. I probably don't have the patience either for that level of coding and required perfection. AI coding is extremely good.
I think the issue being experienced here is, free AI coding agents to pump out useless code in an attempt to grab some $$$. Like said already here, these people aren't testing the addons, they aren't fault finding them, and they likely can't read the code.
If you get your system setup locally with XF installed, quality AI agents integrated, I think you could develop just about anything you could dream up, all with AI. It will cost you, but the higher level agents could absolutely do a better job than the best coders in the world. When Linus Torvel admits AI coding has surpassed him, this isn't rubbish, its just the user and their effort IMO. Top coders in the world have made admissions that AI agents are doing it better and faster than they could. That doesn't mean all AI coding systems... which is probably the issue with the people making rubbish for XF's resource area.
My wife's company uses MYOB heavily for clients, MYOB hasn't done anything with their time billing component in 5 years, because that part of their software is really a support mechanism for bookkeepers and accountants, not the broader client. Simply, its crap nowadays and no longer does the job needed.
So she asked me if I could build her something custom for internal use. I thought it would be a good challenge - and AI has helped me solve many problems, a custom system to suit her company and employees, Python backend, React frontend. Now I guess if I relied on cheap AI, then it would be a coding mess, likely nightmare and unusable, instead I rely on the expensive end of the market, looking through code and finding issues, helping me code entire components and even pages with me just reading over it. More than 95% of the time, I give AI coding agents a big tick. Again, upper end stuff, not the free end. It will cost me nothing more than a few hundred bucks for the entire system, perfect, security focused, locked down in every aspect outside of its use. End to end encryption. Meets all financial system regulations.
I wouldn't have been able to do it without AI - I'm not that good. I probably don't have the patience either for that level of coding and required perfection. AI coding is extremely good.
I think the issue being experienced here is, free AI coding agents to pump out useless code in an attempt to grab some $$$. Like said already here, these people aren't testing the addons, they aren't fault finding them, and they likely can't read the code.
If you get your system setup locally with XF installed, quality AI agents integrated, I think you could develop just about anything you could dream up, all with AI. It will cost you, but the higher level agents could absolutely do a better job than the best coders in the world. When Linus Torvel admits AI coding has surpassed him, this isn't rubbish, its just the user and their effort IMO. Top coders in the world have made admissions that AI agents are doing it better and faster than they could. That doesn't mean all AI coding systems... which is probably the issue with the people making rubbish for XF's resource area.
Sure does multi-AI in action within Claude Code
Ah that is on my list to do too
I audited several normal addons (not vibe coded) with Claude Code and sent the reports to the developers.
This resulted in this security release:
The other audit reports may lead to similar releases for other addons, if the developers find the reports useful.
This resulted in this security release:
The other audit reports may lead to similar releases for other addons, if the developers find the reports useful.
I’m sure it is, but that doesn’t get in the way of the basic premise of this thread that in the wrong hands it is very dangerous. Luckily we see very few amateur heart surgeons.
A fool with a tool is still a fool. Has been like that in the past, will be like that in the future.
