AI generated Thread Summary, Tags, Prefix, SEO Meta, Thread cover, etc.

[Alpha1]

AI / LLM's can be used to make threads and other content more useful and better organized. For better usability, SEO, content discovery and readability.

One of the challenges of using forums is reading through mega-threads with thousands of replies. Especially if high quality replies and useless comments are mixed. For this reasons there have been many requests for a summary feature over the years. Now with the advent of AI the technology to create good summaries for large amounts of text is possible.

Some other challenges for forums are:

1. Correcting useless thread titles. i.e. 'HELP ME NOW!!!' or 'Dumb question'.
2. Optimal SEO meta tags.
3. Useful thread tags.
4. Correct thread prefix.
5. Correct thread type. i.e. a Question posted as a Discussion Type.
6. Making forum threads look better with images.
7. Making unreadable posts readable. When users post broken English, chat speak, mobile t9 autocomplete suggests wrong words, ALL CAPS, no punctuation, no line breaks wall of text, no formatting, then it can make posts completely unreadable to others.
8. Link to related content from other content types for better content discovery. For example if XFRM has a resource that addresses the topic in a thread.

These can be daunting tasks and many admins don't even bother because of the scale of fixing all that.
Some forum platforms like Discourse and Quora have already implemented some LLM solutions. For example:

Discourse AI Topic Summary 🤖

ℹ Summary Uses a remote AI language model to prepare and post a summary of a topic. 🛠 Repository Link GitHub - merefield/discourse-ai-topic-summary: Uses a remote AI language model to prepare and post a summary of a Topic 📖 Install Guide How to install plugins in Discourse ❤ Sponsorship...

meta.discourse.org

Discourse AI

:discourse2: Summary Integration between AI features and Discourse 🌐 Website Discourse AI Features | Discourse - Civilized Discussion 🛠 Repository Link GitHub - discourse/discourse-ai 📖 Install Guide How to install plugins in Discourse Discourse AI Discourse AI is our one-stop...

meta.discourse.org

Once an LLM understands the meaning of the thread, the LLM can create the following with the data:

1. A thread summary. (widget)
2. A better thread snippet.
3. Suggest a better thread title.
4. SEO meta tags: Meta title, Meta Description, Meta Keywords, etc.
5. Thread tags.
6. Select Thread prefix.
7. Select Thread Type.
8. Thread cover, caption and thumb. This would make forum content look much better.
9. Making unreadable posts readable. (correct spelling mistakes)
10. Display related content. (Widget to display related content from all content types)

And the same for all content types. (XFRM, XFMG, Add-ons)
Please consider to add LLM functionality to let AI / LLM suggest the above.

As there are costs for the use of AI / LLM, it would require various limits on when analysis takes place. For example generate x,y,z only after x posts have been made. Or add a budget for AI credits.
As LLM generated data can be erroneous, methods of correct are needed like community editing of summaries or admin view of new tags.

 

[Sal Collaziano]

Would this plugin be able to talk to a local model via the ollama, openai, or lmstudio protocols?
I insist that we don't transit our data to a third party. So it's a dealbreaker for me if it can't do that.


Also you asked for ideas, here's mine.

LLMs are very expensive to run. If i was designing this, i'd only update a summary for a thread after a user settable number of posts in that thread has been made since the last summary regeneration. By default, this should be 1 page ( use xenforo's setting )

Only ~1% of posts are active at any time, so the daily re-generations would be very minimal.

I would also let the user set a maximum number of summary generations per daily batch job. I'd set this to something conservative like 10 threads per night so that out of the box, someone doesn't rack up a surprise bill with their AI provider.


Another way to make money on this is to resell inference. Allow the user to pay a per token fee to you for convenience, or connect to their local provider.

We have deployed our own LLM using Ollama. We will offer options to use your own local model.

We are taking the potential for the feature to be abused and/or used too much into consideration. There will be options and measures in place to prevent that.

 

[Sal Collaziano]

I think the idea of using AI to write thread titles can be extremely useful for SEO and onsite searches. I also think if implemented correctly it will be very well received by most users.

I think by default the thread title box should be locked. The user should just input their thread message and when the user chooses to post, AI automatically creates a title for the thread, giving the user an option to override it. I think most users will accept the AI generated title, and for those that really want to create a personalized one they can easily override it just before it posts.

Something like this would probably work best for new users. In my experience, they need more help than members who've been around a while and have properly "learned the ropes". In my mind, something like:

"Based on your message, I'd like to suggest the following title for your discussion: [suggested_title]"

With two buttons above or next to the submit button: USE THIS TITLE / KEEP MY TITLE

 

[ProCom]

LLMs are very expensive to run. If i was designing this, i'd only update a summary for a thread after a user settable number of posts in that thread has been made since the last summary regeneration. By default, this should be 1 page ( use xenforo's setting )

Only ~1% of posts are active at any time, so the daily re-generations would be very minimal.

Yup, at scale on a really big / busy forum, this could get extremely expensive really fast. It would also be tricky to quantify the ROI on implementing a feature like this.

One thing we considered (assuming the costs to generate the summaries became exorbitant) was to limit the volume of summaries to only cases where people wanted it. Maybe by providing a "generate summary button"... either for all members, or just for our premium members.

 

[Sal Collaziano]

Yup, at scale on a really big / busy forum, this could get extremely expensive really fast. It would also be tricky to quantify the ROI on implementing a feature like this.

One thing we considered (assuming the costs to generate the summaries became exorbitant) was to limit the volume of summaries to only cases where people wanted it. Maybe by providing a "generate summary button"... either for all members, or just for our premium members.

That's how we're going about this:

I yr anniversary

Next month is my 1st anniversary lol. Just wondering if people are happy with there Seltos? My first impressions were very high tech,fuel consumption great. Now still feel the same but still freaked out when it takes over steering. Was sceptical about no rear aircon vents,but rear seats never...

seltosforums.com

...a Generate Summary button.

 

[Digital Doctor]

Generate Summary

The summary ideally is a page. Yes the page could be located at the top of the thread.
It should be editable via BBCODE (not .html only). It should allow Likes. I think it should also allow commenting.
It should have all the features a xenforo post has - can be followed / watched. It ideally is prioritized in Xenforo searches (over posts, etc)
It ideally is the landing page for Google searchers and the summary is the key information shared with Google NOT the thread.

Ideally it follows "Xenforo's content handler system" and therefore the content is accessible to other parts of Xenforo
see here. @Jaxel did it for XenCarta.

Ideally the summary can link to the key parts of the thread where the content comes from.

while at it, release a paid addon where this Summary Area doesn't need AI but everything is done manually.

 

[ebikes.com]

Yup, at scale on a really big / busy forum, this could get extremely expensive really fast. It would also be tricky to quantify the ROI on implementing a feature like this.

One thing we considered (assuming the costs to generate the summaries became exorbitant) was to limit the volume of summaries to only cases where people wanted it. Maybe by providing a "generate summary button"... either for all members, or just for our premium members.

That's why i think some cronjob that automatically pre-generates X number of summaries based on thread activity is the only way to fly.

Whatever pea shooter grade hardware your typical LLM experimenter is gonna have on hand is easily overloaded if end users have the ability to put a load on your LLM hardware.

Imagine implementing a feature in software where a user clicks a button and the response happens in 10's of seconds, to minutes. A spinner is not adequate, you need a progress bar at this point because if something is 'spinning' for over a minute, most users are going to assume the thing is broken.. maybe even refresh the page and try again, lol.

 

[Digital Doctor]

@Sal Collaziano - you'll also want to target fora as a premium purchaser of your addon.

Thread 'A quick peek at fora.com 's modified Xenforo community software'

Aug 23, 2024

(1) INTRODUCTION to Fora.com community software

As discussed here in 2021, https://xenforo.com/community/threads/presale-question-fora-platform-including-xenforo.197555/ ...

Fora is software used by Verticalscope owned communities. Their stock ticker has fora in it.

https://finance.yahoo.com/quote/FORA.TO/

It is a fairly customized version of Xenforo. I'm not sure of all the actual differences but let's review what it looks like vs. stock xenforo.

(from 2021) The Fora platform is a network of sites or a hosting service (not entirely clear) owned or managed...

• Digital Doctor
• Replies: 37
• Forum: Off topic

• 

charge an arm and a leg

 

[Sal Collaziano]

View attachment 325455

The summary ideally is a page. Yes the page could be located at the top of the thread.
It should be editable via BBCODE (not .html only). It should allow Likes. I think it should also allow commenting.
It should have all the features a xenforo post has - can be followed / watched. It ideally is prioritized in Xenforo searches (over posts, etc)
It ideally is the landing page for Google searchers and the summary is the key information shared with Google NOT the thread.

Ideally it follows "Xenforo's content handler system" and therefore the content is accessible to other parts of Xenforo
see here. @Jaxel did it for XenCarta.

Ideally the summary can link to the key parts of the thread where the content comes from.

while at it, release a paid addon where this Summary Area doesn't need AI but everything is done manually.

Okay, so gathering information here. Instead of simply displaying the generated summary at the top of a page, have the system create a separate page (like a topic of it's own) that can be displayed above the discussion like a widget. The generated summary can be liked (and disliked - which can also teach the AI), commented on, shared, etcetera - just like a thread/post.

This would basically split off a new topic based on the original topic. I guess that's what's wanted? I wonder if maybe that should be an option as I'm not sure everyone would want that functionality. But I do see the merits of it...

Prioritizing generated summaries over topics in search... That would mean changes to the way the XenForo search feature works - and I imagine that could be manipulated with the add-on - or with another...

We're already working on linking the summary to the posts it pulled the data from - just like a press release that has numbers in the text [1], [2], etcetera - so the reader can click the number to find that specific spot in the discussion.

Would you please elaborate on the last item you mentioned? About an option for the Summary Area not needing AI but being generated manually... I'm not following (yet)...

That's why i think some cronjob that automatically pre-generates X number of summaries based on thread activity is the only way to fly.

Whatever pea shooter grade hardware your typical LLM experimenter is gonna have on hand is easily overloaded if end users have the ability to put a load on your LLM hardware.

Imagine implementing a feature in software where a user clicks a button and the response happens in 10's of seconds, to minutes. A spinner is not adequate, you need a progress bar at this point because if something is 'spinning' for over a minute, most users are going to assume the thing is broken.. maybe even refresh the page and try again, lol.

Maybe we'd want to only give access to product summaries manually to established users - and otherwise have a cronjob that generates summaries automatically for each discussion as time and server load allows...

As for hardware, my assumption is that - in the very beginning - this won't be very server intensive as it's just getting off the ground. But as the add-on gets used more and more, yes - better hardware will be needed.

I do like the idea of a progress bar...

@Sal Collaziano - you'll also want to target fora as a premium purchaser of your addon.

Thread 'A quick peek at fora.com 's modified Xenforo community software'

Aug 23, 2024

(1) INTRODUCTION to Fora.com community software

As discussed here in 2021, https://xenforo.com/community/threads/presale-question-fora-platform-including-xenforo.197555/ ...

Fora is software used by Verticalscope owned communities. Their stock ticker has fora in it.

https://finance.yahoo.com/quote/FORA.TO/

It is a fairly customized version of Xenforo. I'm not sure of all the actual differences but let's review what it looks like vs. stock xenforo.

(from 2021) The Fora platform is a network of sites or a hosting service (not entirely clear) owned or managed...

• Digital Doctor
• Replies: 37
• Forum: Off topic

• 

charge an arm and a leg

...yes, and that would definitely require impressive hardware - though I imagine we could have it run on their own hardware - which they probably have just about unlimited financial resources to obtain.

 

[GCT2007]

@Sal Collaziano - you'll also want to target fora as a premium purchaser of your addon.

Thread 'A quick peek at fora.com 's modified Xenforo community software'

Aug 23, 2024

(1) INTRODUCTION to Fora.com community software

As discussed here in 2021, https://xenforo.com/community/threads/presale-question-fora-platform-including-xenforo.197555/ ...

Fora is software used by Verticalscope owned communities. Their stock ticker has fora in it.

https://finance.yahoo.com/quote/FORA.TO/

It is a fairly customized version of Xenforo. I'm not sure of all the actual differences but let's review what it looks like vs. stock xenforo.

(from 2021) The Fora platform is a network of sites or a hosting service (not entirely clear) owned or managed...

• Digital Doctor
• Replies: 37
• Forum: Off topic

• 

charge an arm and a leg

VS doesn't use 3rd party addons and I doubt they'd be able to run this anyway. I think they're on a very very hacked 2.1.

 

[Digital Doctor]

This would basically split off a new topic based on the original topic. I guess that's what's wanted?

It's a new AREA, not a new topic/thread. It logically would reside just above the thread (before the first post) that it is summarizing.

I wonder if maybe that should be an option as I'm not sure everyone would want that functionality. But I do see the merits of it...

What's the alternative to an area above the thread ?

Prioritizing generated summaries over topics in search... That would mean changes to the way the XenForo search feature works - and I imagine that could be manipulated with the add-on - or with another...

This is fine tuning and likely shouldn't be a part of an initial release.

Would you please elaborate on the last item you mentioned? About an option for the Summary Area not needing AI but being generated manually... I'm not following (yet)...

The idea is to create the BBCODE editable (no HTML knowledge needed) page I outlined and that it can be used WITHOUT AI. It is essentially a post (has all the features of a post) and it can be used by forum members to summarize threads manually. Many forums will find it to laborious and will default to your AI summarizing service. But keep in mind, your service may be inferior to manual editing in many instances - so manual editing is an essential feature.

This is similar to my First Post Footnotes idea.

Essentially what I am suggesting is you divide your thread summary idea into two parts
(1) the bbcode editable page container I suggested.
and (2) your AI thread summary idea that needs a container otherwise.

I think both could be extremely useful addons in their own right. (1) should be useful to every forum and (2) would be a good idea for people that can afford it.

If you get (1) launched, and shouldn't it be easy ? you can get people making manual summaries and see what they would be willing to pay for. You also want some "structure" in the thread summary page. Subtitles, Lists, etc. The manual editors will help pave the way.

It is also possible, your AI model can learn how forum owners WANT their threads summarized by seeing how they do it themselves.

 

[Digital Doctor]

VS doesn't use 3rd party addons and I doubt they'd be able to run this anyway. I think they're on a very very hacked 2.1.

Fora uses use Bob's showcase for sure.
The thread I started and linked to here has the details.

 

[Sal Collaziano]

It's a new AREA, not a new topic/thread. It logically would reside just above the thread (before the first post) that it is summarizing.

What's the alternative to an area above the thread ?

This is fine tuning and likely shouldn't be a part of an initial release.

It sounds like a topic within a topic - being that it has comments. Or am I just not seeing this clearly? Are the comments relating to how accurate the summarize is? Not sure if that's what you're referring to - or comments on the content of the summary...

The idea is to create the BBCODE editable (no HTML knowledge needed) page I outlined and that it can be used WITHOUT AI. It is essentially a post (has all the features of a post) and it can be used by forum members to summarize threads manually. Many forums will find it to laborious and will default to your AI summarizing service. But keep in mind, your service may be inferior to manual editing in many instances - so manual editing is an essential feature.

Yes, it'll probably be some time until AI can generate perfect summaries - better than what humans can do... So the AI created summary would use the same format as a post into a new 'area'. We can display it at the top of a page via widget. We can set user groups to have the ability to edit the summary... Or, if desired, create their own summary for the topic. Am I following? See my response below before answering.

This is similar to my First Post Footnotes idea.

Essentially what I am suggesting is you divide your thread summary idea into two parts
(1) the bbcode editable page container I suggested.
and (2) your AI thread summary idea that needs a container otherwise.

...so two parts in one add-on or two separate add-ons? One AI-powered and one human-powered? If two parts in one add-on, we could give the user the opportunity to post their own summary (and then certain user groups would have the ability to, I guess, over-write that summary) - or click "Generate Summary" to have one created for them/the community.

I think both could be extremely useful addons in their own right. (1) should be useful to every forum and (2) would be a good idea for people that can afford it.

If you get (1) launched, and shouldn't it be easy ? you can get people making manual summaries and see what they would be willing to pay for.

 

[ProCom]

This would basically split off a new topic based on the original topic. I guess that's what's wanted? I wonder if maybe that should be an option as I'm not sure everyone would want that functionality. But I do see the merits of it...

Ya, I'm one that wouldn't want it split-off of the thread. As a user, I VERY much appreciate when I go to an article (often news) and see a bulleted highlights list at the top.

We're already working on linking the summary to the posts it pulled the data from - just like a press release that has numbers in the text [1], [2], etcetera - so the reader can click the number to find that specific spot in the discussion.

Fantastic!!! When I'm working with the various LLMs, I love when they link back to the source material for me to do a deeper dive / do validation. This should really help the user find the info they need quickly and potentially drive more engagement.

I do like the idea of a progress bar...

Ya, if the summary isn't already generated, and it is "user initiated" with a button... then some visual progress/activity would be good. Hopefully we're using LLMs that aren't super slow (we're not doing advanced PHD deep research), so I'd hope most modern LLMs would be fast enough that even a simple spinner would suffice for the 20 - 40 seconds needed? As a test, I just dumped a 100 post thread into Gemini 2.5 Pro (smart, but not super fast) and it gave me a full summary in 32 seconds.

(BTW, assuming if a user generates a summary it is cached for that thread... maybe with a note if needed "Thread updated with new posts since summary was created. Do you want to refresh the summary?")

 

[Sal Collaziano]

Ya, I'm one that wouldn't want it split-off of the thread. As a user, I VERY much appreciate when I go to an article (often news) and see a bulleted highlights list at the top.

That's what I'm thinking but I have a feeling I'm not fully grasping the idea being presented.

Fantastic!!! When I'm working with the various LLMs, I love when they link back to the source material for me to do a deeper dive / do validation. This should really help the user find the info they need quickly and potentially drive more engagement.

Yes, this allows a viewer to jump directly to a point in the discussion that's important to them. And, of course, it allows the owner(s) of the community to benefit from extra page views and interactions with those posts...

Ya, if the summary isn't already generated, and it is "user initiated" with a button... then some visual progress/activity would be good. Hopefully we're using LLMs that aren't super slow (we're not doing advanced PHD deep research), so I'd hope most modern LLMs would be fast enough that even a simple spinner would suffice for the 20 - 40 seconds needed? As a test, I just dumped a 100 post thread into Gemini 2.5 Pro (smart, but not super fast) and it gave me a full summary in 32 seconds.

A 100 post thread shouldn't be too much trouble to generate. I just wonder about the 100 PAGE thread - and how that should work. It's one of the reasons a page-level summary might be better in some instances. But having one big beautiful AI Generated Summary for an entire discussion would be the best...

(BTW, assuming if a user generates a summary it is cached for that thread... maybe with a note if needed "Thread updated with new posts since summary was created. Do you want to refresh the summary?")

I do like this idea and think it makes sense... Or we could probably just have the AI re-generate the summary after every PAGE of posts...

 

[ProCom]

@Sal Collaziano you seem like you're on the right track! Of course with any app-dev, you'll have 1,000 users with 5,000 different ideas on how it should be built / customized... most believing their way is the best, LOL!

Good luck with the process of managing # of features, options, and variables vs. too much complexity!

 

[Sal Collaziano]

@Sal Collaziano you seem like you're on the right track! Of course with any app-dev, you'll have 1,000 users with 5,000 different ideas on how it should be built / customized... most believing their way is the best, LOL!

Good luck with the process of managing # of features, options, and variables vs. too much complexity!

Thank you very much! We'll be trying to keep this add-on as simple as possible - to get the job done with the least amount of complexity. We can always add features moving forward. We're working on several add-ons, but this one will be the first one ready for beta testing.

 

[GCT2007]

Fora uses use Bob's showcase for sure.
The thread I started and linked to here has the details.

I'm very well aware of what the use lol. They bought Bobs addon when they were using XF1 and it barely functions.

 

[GCT2007]

They won't use external AI tools when they're clearly building their own.

 

[Digital Doctor]

It sounds like a topic within a topic - being that it has comments. Or am I just not seeing this clearly? Are the comments relating to how accurate the summarize is? Not sure if that's what you're referring to - or comments on the content of the summary...

"topic within a topic - being that it has comments"

... that's fair. So then just scrap my "Thread Summary" comments. It's not key anyway. You can "hide" thread summary comments in a tab like wikipedia does. Regardless ... it isn't of primary importance.

Yes, it'll probably be some time until AI can generate perfect summaries - better than what humans can do... So the AI created summary would use the same format as a post into a new 'area'. We can display it at the top of a page via widget. We can set user groups to have the ability to edit the summary... Or, if desired, create their own summary for the topic. Am I following? See my response below before answering.

Yes. that is it exactly.

This question and answer thread type could easily work as I was suggesting.
... more in a minute.

 

[tajhay]

Happy to do beta test