XenForo AI roadmap?
- Thread starter back
- Start date
Okay but what does it do with AI? saying you integrated AI leaves that question.
Also because AI can get really expensive, the solution should work with localhosted models.
I can run something 90% as good as Deepseek R1 on a single 5090 ( bytedance Seed OSS 36B ) as of today, i don't see why i'd want to give a copy of my forum to a third party provider if i can just run a GPU and get inference for a lot less.
I described what it does in the quoted text; basically I have setup a user (octobot) which responds when tagged by users who are in the Supporter user group (or Staff).
No argument that it could be done locally for self-use. Not sure how pragmatic it would be to set it up to have it function for your community's users, but it'd involve internet traffic so I think yours is a different use case from what I described.
Xpertloop is a startup comprised of people I had worked with for many years. As for my data, their service ingests it and is used solely for my purposes. The data they have is essentially what has been scrape-able by Google and other such robots (ie. public posts).
Another feature that I didn't mention in my other post is that when Octobot is tagged, the last N posts in the thread are sent along to the API so that the system has adequate context on the discussion and can respond intelligently (along with the post itself which contained the tag and query, of course). I think I've set it to 5 posts.
I'm not a developer but I essentially created my part of the script through vibe-coding (using mostly Grok but Xpertloop does offer its admins it's own private instance for such tasks as well). Stuff like, only respond when Octobot is tagged by a Supporter, ignore quoted text, and send the last 5 posts in the thread for any context, etc. It's a cloudflare worker script currently (also using XF API and webhooks) but I'd like to see if I can make it an add-on for others to use (vs CF worker). By making it an add-on, I could help others with Xpertloop onboarding, etc.
Xpertloop typically works with authors and other such content owners; I am working with them as it relates to forum discussion systems. Personally I believe this is the right kind of AI stepping stone needed for forums.
No argument that it could be done locally for self-use. Not sure how pragmatic it would be to set it up to have it function for your community's users, but it'd involve internet traffic so I think yours is a different use case from what I described.
Xpertloop is a startup comprised of people I had worked with for many years. As for my data, their service ingests it and is used solely for my purposes. The data they have is essentially what has been scrape-able by Google and other such robots (ie. public posts).
Another feature that I didn't mention in my other post is that when Octobot is tagged, the last N posts in the thread are sent along to the API so that the system has adequate context on the discussion and can respond intelligently (along with the post itself which contained the tag and query, of course). I think I've set it to 5 posts.
I'm not a developer but I essentially created my part of the script through vibe-coding (using mostly Grok but Xpertloop does offer its admins it's own private instance for such tasks as well). Stuff like, only respond when Octobot is tagged by a Supporter, ignore quoted text, and send the last 5 posts in the thread for any context, etc. It's a cloudflare worker script currently (also using XF API and webhooks) but I'd like to see if I can make it an add-on for others to use (vs CF worker). By making it an add-on, I could help others with Xpertloop onboarding, etc.
Xpertloop typically works with authors and other such content owners; I am working with them as it relates to forum discussion systems. Personally I believe this is the right kind of AI stepping stone needed for forums.
My bad for not reading the quoted text - didn't expect details to be there.
Okay, i get it. This is kind of like how you can tag Grok on X. But users have to pay for it.
How is uptake?
Okay, i get it. This is kind of like how you can tag Grok on X. But users have to pay for it.
How is uptake?
Last edited:
exactly, like the Grok feature. I originally had a button next to the share icons in each post but just tagging octobot works best.
Users are generally skeptical, but accepting. As mentioned, I don't mind the slow adoption. It's definitely experimental. Getting the agent right is crucial. I'm thinking it's in a good place right now but need to test further without creating noise. I can test outside of my production site of course, but thread context and all that is a bit tedious to recreate, so I'm just casually working it in when it feels natural to do so.
Users are generally skeptical, but accepting. As mentioned, I don't mind the slow adoption. It's definitely experimental. Getting the agent right is crucial. I'm thinking it's in a good place right now but need to test further without creating noise. I can test outside of my production site of course, but thread context and all that is a bit tedious to recreate, so I'm just casually working it in when it feels natural to do so.
Just furthering the discussion on potential for AI in xenforo -- I've implemented this feature set (through a separate partnership with Xpertloop); here's how it works:
1) I have "robot"-type user which I've enabled to handle RSS feed ingestion (i.e., news on my forum's subject matter) and now, AI responses.
2) There's a dedicated nav item which goes to a Page node for one-on-one chats with the Xpertloop API (which is customized to my forum's content and agent instructions). Chats are strictly limited to my forum's subject matter. Registered users get up to three queries per day. Supporters get "unlimited" queries. Guests must register to use. The UI is simple; essentially just a query prompt and a "clear chat" button. When the user submits, there's a "please wait" message while it works on a response.
3) The Grok-like feature is implemented as well; here's a screenshot of the icon (robot fontawesome). Users who click it go to the Page node, with the post contents dynamically sent to the API for a response (along with the last 5 posts in the thread, for additional context for the AI's response).
4) The Supporter usergroup can tag the robot user in any post, which will result in the robot providing its AI response to the thread publicly.
Thoughts on these features / use cases? I think it will be helpful to gel this kind of information summoning with the forum user experience, and helps users understand when they're interacting with real people vs. an AI robot.
1) I have "robot"-type user which I've enabled to handle RSS feed ingestion (i.e., news on my forum's subject matter) and now, AI responses.
2) There's a dedicated nav item which goes to a Page node for one-on-one chats with the Xpertloop API (which is customized to my forum's content and agent instructions). Chats are strictly limited to my forum's subject matter. Registered users get up to three queries per day. Supporters get "unlimited" queries. Guests must register to use. The UI is simple; essentially just a query prompt and a "clear chat" button. When the user submits, there's a "please wait" message while it works on a response.
3) The Grok-like feature is implemented as well; here's a screenshot of the icon (robot fontawesome). Users who click it go to the Page node, with the post contents dynamically sent to the API for a response (along with the last 5 posts in the thread, for additional context for the AI's response).
4) The Supporter usergroup can tag the robot user in any post, which will result in the robot providing its AI response to the thread publicly.
Thoughts on these features / use cases? I think it will be helpful to gel this kind of information summoning with the forum user experience, and helps users understand when they're interacting with real people vs. an AI robot.
There are many things under the hood, being discussed since a long time.. But one cannot ignore AI today. Since the downfall of forums, AI brings a new hope to forum owners today.
Almost all big competitors are moving towards embedding AI system into there forum software and few of them are very actively working and implementing one major AI feature after other.
Delaying this, may lead to moving to other options and/or becoming obsolete.. IMO
Almost all big competitors are moving towards embedding AI system into there forum software and few of them are very actively working and implementing one major AI feature after other.
Delaying this, may lead to moving to other options and/or becoming obsolete.. IMO
That's good. But let's think seriously about the cost of AI features.
Per function that AI performs ( versus just using regular programming ), the resource cost is >1000x more
The more you use it, the more your operations cost increases.
At my dev shop, we found out how much hardware we need to provide coding assistance with good intelligence and speed to 8 programmers & handle two simultaneous requests.
We spent about $14k to get a RTX PRO 6000, 5090, and computer capable of running both GPUs at x8
Due to thermals i had to relocate the 1500w PSU to where the CD ROM drives would usually be, and leave the case open.
This runs a newer 197B AI model which provides equal capability and speed to what was commercially the middle to upper tier 12 months ago.
What does a Xenforo sized version of this look like? it's probably a single 5090 with a ~30b model
But it has the same constraints of only being able to run a few requests at a time.
Meanwhile our forum can run acceptably on a 5 years old laptop and use a fraction of the electricity to do so.
Enormous differential in how miuch hardware/electricity is needed to serve a function!
Now if you don't wanna spend $5k for the smaller version of this, you will need to rent services.
The problem is that this cost you more money.. so if you have a high scale... it can be the wrong choice.
Another problem is that a lot of these providers have random downtime... and you will see that in prod and your software needs to account for the other end being unreliable. This is what caused me to buy hardware in the end.
Even still, you have to design all kinds of safeguards so that you don't end up with a enormous API bill when someone starts abusing your AI functions. if your software doesn't account for that, you are in for a surprise megabill
So my thoughts on AI support for forum software are..
It's nice to have but not feasible, cost wise, for a lot of people.
I run a development/service shop so i can afford it.. but what % of forum administrators are willing to pay the cost of AI service?
The cost needs to go way down for it to be a mainstream option.
Per function that AI performs ( versus just using regular programming ), the resource cost is >1000x more
The more you use it, the more your operations cost increases.
At my dev shop, we found out how much hardware we need to provide coding assistance with good intelligence and speed to 8 programmers & handle two simultaneous requests.
We spent about $14k to get a RTX PRO 6000, 5090, and computer capable of running both GPUs at x8
Due to thermals i had to relocate the 1500w PSU to where the CD ROM drives would usually be, and leave the case open.
This runs a newer 197B AI model which provides equal capability and speed to what was commercially the middle to upper tier 12 months ago.
What does a Xenforo sized version of this look like? it's probably a single 5090 with a ~30b model
But it has the same constraints of only being able to run a few requests at a time.
Meanwhile our forum can run acceptably on a 5 years old laptop and use a fraction of the electricity to do so.
Enormous differential in how miuch hardware/electricity is needed to serve a function!
Now if you don't wanna spend $5k for the smaller version of this, you will need to rent services.
The problem is that this cost you more money.. so if you have a high scale... it can be the wrong choice.
Another problem is that a lot of these providers have random downtime... and you will see that in prod and your software needs to account for the other end being unreliable. This is what caused me to buy hardware in the end.
Even still, you have to design all kinds of safeguards so that you don't end up with a enormous API bill when someone starts abusing your AI functions. if your software doesn't account for that, you are in for a surprise megabill
So my thoughts on AI support for forum software are..
It's nice to have but not feasible, cost wise, for a lot of people.
I run a development/service shop so i can afford it.. but what % of forum administrators are willing to pay the cost of AI service?
The cost needs to go way down for it to be a mainstream option.
To be clear:
I think AI could do a lot of interesting things on a forum..
..but the costs are nuts right now..
..we really need the hardware to improve 2-3x in efficiency/cost. ( i say this as i sit next to a 50lbs AI box that can draw 900w from the wall and make the office into a sauna if a programmer is using it too much! )
I think the next generation of Nvidia hardware is where it starts to become economical for the average joe forum operator.. but that's in 2027
I think AI could do a lot of interesting things on a forum..
..but the costs are nuts right now..
..we really need the hardware to improve 2-3x in efficiency/cost. ( i say this as i sit next to a 50lbs AI box that can draw 900w from the wall and make the office into a sauna if a programmer is using it too much! )
I think the next generation of Nvidia hardware is where it starts to become economical for the average joe forum operator.. but that's in 2027
Similar threads
