AI generated Thread Summary, Tags, Prefix, SEO Meta, Thread cover, etc.

[Alpha1]

AI / LLM's can be used to make threads and other content more useful and better organized. For better usability, SEO, content discovery and readability.

One of the challenges of using forums is reading through mega-threads with thousands of replies. Especially if high quality replies and useless comments are mixed. For this reasons there have been many requests for a summary feature over the years. Now with the advent of AI the technology to create good summaries for large amounts of text is possible.

Some other challenges for forums are:

1. Correcting useless thread titles. i.e. 'HELP ME NOW!!!' or 'Dumb question'.
2. Optimal SEO meta tags.
3. Useful thread tags.
4. Correct thread prefix.
5. Correct thread type. i.e. a Question posted as a Discussion Type.
6. Making forum threads look better with images.
7. Making unreadable posts readable. When users post broken English, chat speak, mobile t9 autocomplete suggests wrong words, ALL CAPS, no punctuation, no line breaks wall of text, no formatting, then it can make posts completely unreadable to others.
8. Link to related content from other content types for better content discovery. For example if XFRM has a resource that addresses the topic in a thread.

These can be daunting tasks and many admins don't even bother because of the scale of fixing all that.
Some forum platforms like Discourse and Quora have already implemented some LLM solutions. For example:

Discourse AI Topic Summary 🤖

ℹ Summary Uses a remote AI language model to prepare and post a summary of a topic. 🛠 Repository Link GitHub - merefield/discourse-ai-topic-summary: Uses a remote AI language model to prepare and post a summary of a Topic 📖 Install Guide How to install plugins in Discourse ❤ Sponsorship...

meta.discourse.org

Discourse AI

:discourse2: Summary Integration between AI features and Discourse 🌐 Website Discourse AI Features | Discourse - Civilized Discussion 🛠 Repository Link GitHub - discourse/discourse-ai 📖 Install Guide How to install plugins in Discourse Discourse AI Discourse AI is our one-stop...

meta.discourse.org

Once an LLM understands the meaning of the thread, the LLM can create the following with the data:

1. A thread summary. (widget)
2. A better thread snippet.
3. Suggest a better thread title.
4. SEO meta tags: Meta title, Meta Description, Meta Keywords, etc.
5. Thread tags.
6. Select Thread prefix.
7. Select Thread Type.
8. Thread cover, caption and thumb. This would make forum content look much better.
9. Making unreadable posts readable. (correct spelling mistakes)
10. Display related content. (Widget to display related content from all content types)

And the same for all content types. (XFRM, XFMG, Add-ons)
Please consider to add LLM functionality to let AI / LLM suggest the above.

As there are costs for the use of AI / LLM, it would require various limits on when analysis takes place. For example generate x,y,z only after x posts have been made. Or add a budget for AI credits.
As LLM generated data can be erroneous, methods of correct are needed like community editing of summaries or admin view of new tags.

 

[ProCom]

In any event, we're working on making this a fully functional add-on. You can see the first stages of this at an old forum of mine that never really took off:

@Sal Collaziano this is pretty exciting to hear!!

A few things I'd add to my perfect "thread summary" addon:

1. Reference links (maybe a linked superscript number) that links to the post(s) where the bits of information in the summary were gathered. This would ideally drive more engagement and pageviews.
2. Group-level permissions on a) who can see them, and b) who can generate / "refresh" them.
3. A way to control which threads these are generated for. A super busy forum might get overwhelmed if it's generated for every thread. Obviously caching would be necessary. Another idea is to have them be created on demand (maybe a "Create Summary" button?) and then cached.

 

[Sal Collaziano]

@Sal Collaziano this is pretty exciting to hear!!

A few things I'd add to my perfect "thread summary" addon:

1. Reference links (maybe a linked superscript number) that links to the post(s) where the bits of information in the summary were gathered. This would ideally drive more engagement and pageviews.
2. Group-level permissions on a) who can see them, and b) who can generate / "refresh" them.
3. A way to control which threads these are generated for. A super busy forum might get overwhelmed if it's generated for every thread. Obviously caching would be necessary. Another idea is to have them be created on demand (maybe a "Create Summary" button?) and then cached.

Thank you for the input. I'm taking notes moving forward. Any features people need, please let me know. Not sure every idea and request will end up in the same add-on or not, but the goal is to make this one robust.

 

[GCT2007]

Sorry if this was already covered but will this scan an entire thread, the first page, specifici chunks, only the OP, etc?

 

[Sal Collaziano]

Sorry if this was already covered but will this scan an entire thread, the first page, specifici chunks, only the OP, etc?

Right now it's set to scan the entire thread. We're planning on having it scan individual pages as an option. I think most admins will still want people reading through the discussions - and if an entire 100 page discussion is summarized in a few paragraphs, among other issues. people will inevitably miss things that may not have been summarized...

 

[GCT2007]

I would think doing large threads would eat up tokens really fast. For me, I think being able to summarize the OP's post would be big, as I think most on my site would use this add-on to catch up on member builds for cars, etc.

 

[Sal Collaziano]

I would think doing large threads would eat up tokens really fast. For me, I think being able to summarize the OP's post would be big, as I think most on my site would use this add-on to catch up on member builds for cars, etc.

That could be an option as well. Only summarize the first post. I imagine you may want to activate the feature only if that post is really long...

 

[GCT2007]

Oh woops, I meant all the OP's posts. Not just the first.

 

[Digital Doctor]

The question, still, after that, as a visitor landing there (miraculously from Google), would you be inclined to go past page 2 or deeper than 2 pages from where you landed?

almost no one.

The reddit approach is what people want or more importantly what Google thinks people want.

TOPIC:

• most popular reply ---- and the replies to that.
• second most popular reply --- and the replies to that.
• etc

You show the Googler the MOST IMPORTANT information.

The Google lander needs to see the SUMMARY FIRST. Not the thread and have the SUMMARY in the Tab.

Note: Xon's Threadmarks could be seen as the OP picking out the most important replies.

Ultimately, Xenforo should switch to threaded threads if they want to take Information seriously.

When you put that into perspective, even a thread with 750 words to be summarized down to 100 would cost roughly 55 cents.

that's not useful.

 

[Sal Collaziano]

Ultimately, Xenforo should switch to threaded threads if they want to take Information seriously.

Yikes, I hate that layout - but it seems to be where the internet has gone... It's the reason I chose UBB over the other options at the time.

 

[Digital Doctor]

hate that layout

There are options that could work that aren't one or the other.

Having 9 pages of chit chat replies is mildly entertaining to a few forum rats and TOTALLY useless to the other 8 billion people.

UBB

I think you could toggle back and forth between threaded and non-threaded threads.

 

[Sal Collaziano]

There are options that could work that aren't one or the other.

Having 9 pages of chit chat replies is mildly entertaining to a few forum rats and TOTALLY useless to the other 8 billion people.


I think you could toggle back and forth between threaded and non-threaded threads.

Yes, I do remember that - being able to go back and forth... I think... I know there was also UBB.Threads... But, yeah, I think UBB had a setting for either way...

And I know what you mean about the 9 pages of chit chat... I guess this is where a good summary view would work well - even page by page. Instead of trying to put all the pertinent information of 100+ pages of discussion into a single summary on page one - it may work better on each page. Or not. We'll have to see...

 

[ProCom]

That could be an option as well. Only summarize the first post. I imagine you may want to activate the feature only if that post is really long...

Regarding how much of the thread to summarize: I think this is going to have to be an option because it's going to be impossible to have one addon setting all users will agree on.

Some admins might want to have the entire thread summarized, some just the first page, some just the last page, etc.

Definitely tricky because there are some threads that are super long and chock-full of great information, while others start off relevant and then go totally off topic and everything in between.

I'd also be curious if there is a way to insert our own LLM and API keys. I doubt everyone will agree on which LLM they want to use and which model.

NOTE: Over the last few months I've actually been experimenting with this type of "addon" manually. I export the contents of a thread... specifically the user ID, post number, and message contents. Then I have the LLM create a summary based on a handful of examples that I had written and curated so it could get a feel for the type of summary I want written. Then I have it create the summary with subscript links back to the individual posts of the content that was referenced in the summary. It's been an interesting experiment to see the different LLMs and how they each summarize using the same exact prompt and data.

 

[Seeker-Smith]

Regarding how much of the thread to summarize: I think this is going to have to be an option because it's going to be impossible to have one addon setting all users will agree on.

Some admins might want to have the entire thread summarized, some just the first page, some just the last page, etc.

Definitely tricky because there are some threads that are super long and chock-full of great information, while others start off relevant and then go totally off topic and everything in between.

I'd also be curious if there is a way to insert our own LLM and API keys. I doubt everyone will agree on which LLM they want to use and which model.

NOTE: Over the last few months I've actually been experimenting with this type of "addon" manually. I export the contents of a thread... specifically the user ID, post number, and message contents. Then I have the LLM create a summary based on a handful of examples that I had written and curated so it could get a feel for the type of summary I want written. Then I have it create the summary with subscript links back to the individual posts of the content that was referenced in the summary. It's been an interesting experiment to see the different LLMs and how they each summarize using the same exact prompt and data.

The LLM solution is to use OpenRouter.

 

[Sal Collaziano]

Regarding how much of the thread to summarize: I think this is going to have to be an option because it's going to be impossible to have one addon setting all users will agree on.

Some admins might want to have the entire thread summarized, some just the first page, some just the last page, etc.

Definitely tricky because there are some threads that are super long and chock-full of great information, while others start off relevant and then go totally off topic and everything in between.

I'd also be curious if there is a way to insert our own LLM and API keys. I doubt everyone will agree on which LLM they want to use and which model.

NOTE: Over the last few months I've actually been experimenting with this type of "addon" manually. I export the contents of a thread... specifically the user ID, post number, and message contents. Then I have the LLM create a summary based on a handful of examples that I had written and curated so it could get a feel for the type of summary I want written. Then I have it create the summary with subscript links back to the individual posts of the content that was referenced in the summary. It's been an interesting experiment to see the different LLMs and how they each summarize using the same exact prompt and data.

There will definitely be various options to choose from relating to which content to summarize.

The LLM solution is to use OpenRouter.

We're currently using OpenRouter, but there will be options for you to use whichever service you want via API Keys - and we'll also be developing our own in house LLM...

 

[bzcomputers]

I think the idea of using AI to write thread titles can be extremely useful for SEO and onsite searches. I also think if implemented correctly it will be very well received by most users.

I think by default the thread title box should be locked. The user should just input their thread message and when the user chooses to post, AI automatically creates a title for the thread, giving the user an option to override it. I think most users will accept the AI generated title, and for those that really want to create a personalized one they can easily override it just before it posts.

 

[DavidB]

Some info on how Invision are doing it. E.g posts with most shares, time spent reading etc.

https://administrata.net/topic/4205-live-chat-with-invision-matt-m/page/3/?&_rid=1268#findComment-27137

 

[AjayJunkies]

So, if you had a summary, that would be good. However, people come to forums for human connection, too. Some want to read every reply to learn what that person has dealt with and how they dealt with it, and perhaps follow up with them 1-on-1 in another reply.

If you summarize everything, you've removed the human element from a forum, making it essentially a blog (or wiki) with multiple contributors that lacks the human depth or emotion a community should offer with different writing styles, tones, etc., all coming from people of different backgrounds and experiences.

That aside, a summary = 1 page view while a thread with 1000 replies = 50 potential page views (provided defaults)... It might be low on CPM, but there are 50 chances that a proper ad may show up, and with a CTR of 2%, that's a chance of 1 click that could be $0.10 to $10 ($25+ in some niches).

The summary might pull the greatest ads to make that point moot, but if you are running ad contracts, they want views, too. However, they could have a higher CTR with 10 impressions than 1000 impressions, so they might stick around.

In other words, this sounds like turning a community into a wiki, essentially, that would cite which post that section of the summary came from (hopefully). But, the question still remains: Where is the line between interaction and just a page view when we're speaking in terms of a forum and communication? A user might as well fire up ChatGPT to give them the answer.

There's also the ranking factor. Would the summary even rank higher than threads do? It's AI-generated content that may or may not be accurate because the data it's summarizing is all over the place. If Google sees that your summary is inaccurate, it might bury the page completely because of 1 user's "misinformation" summarized into the facts.



This isn't always a good thing either as if not prompted right, wit ill just make a bland AI response that sounds robotic, turning a lot of people off.

A lot I could agree with, though, such as prefixes. If the LLM understood the content and there was a thread prefix helper set, it could automatically prefix it how the admin set the prefixes.

Tags are another good thing, same as titles to fit the 140-160 character length.

But, a summary is where I'm at a loss.

I don't know who needs to hear this and this is not addressed to any one person, but AI summarization of our posts is already happening at scale. With or without your input.

Google AI Overviews, Perplexity, and ChatGPT search are all summarizing our content.

Whether or not to show summaries, or spend the energy to generate summaries, is a moot point. It's already happening and it's the future. The question now is a battle to control where it is happening - on our communities that we control, or on the AI search engine platforms. Defending your traditional layout because youre worried about your ad that brings in ten cents while Google drops millions (billions?) in passthrough clicks is like debating about one tree while the forest burns around us.

 

[StarArmy]

I think the idea of using AI to write thread titles can be extremely useful for SEO and onsite searches. I also think if implemented correctly it will be very well received by most users.

I think by default the thread title box should be locked. The user should just input their thread message and when the user chooses to post, AI automatically creates a title for the thread, giving the user an option to override it. I think most users will accept the AI generated title, and for those that really want to create a personalized one they can easily override it just before it posts.

This would never work on my forum. People write RPG scenarios and often have a very specific title format to clearly show what the thread is and how it fits into the larger story campaign, and what time it took place, like:

[RP: YSS Resurgence] Mission 20, Part 2: Hellgrinder (YE 47.5)

I suspect my forum isn't alone and there's many people who are very deliberate about thread titles.

 

[bzcomputers]

This would never work on my forum. People write RPG scenarios and often have a very specific title format to clearly show what the thread is and how it fits into the larger story campaign, and what time it took place, like:

[RP: YSS Resurgence] Mission 20, Part 2: Hellgrinder (YE 47.5)

I suspect my forum isn't alone and there's many people who are very deliberate about thread titles.

Then technically the only difference is they would fill in the title after they fill in the thread message. Not seeing an issue.

Edit: I also assume this would be an "option" that can be easily turned off (possibly by node or universally) for those that find no use for it. Which is how most AI functions/options will go, because there will likely be some cost involved with API use.

 

[ebikes.com]

Would this plugin be able to talk to a local model via the ollama, openai, or lmstudio protocols?
I insist that we don't transit our data to a third party. So it's a dealbreaker for me if it can't do that.


Also you asked for ideas, here's mine.

LLMs are very expensive to run. If i was designing this, i'd only update a summary for a thread after a user settable number of posts in that thread has been made since the last summary regeneration. By default, this should be 1 page ( use xenforo's setting )

Only ~1% of posts are active at any time, so the daily re-generations would be very minimal.

I would also let the user set a maximum number of summary generations per daily batch job. I'd set this to something conservative like 10 threads per night so that out of the box, someone doesn't rack up a surprise bill with their AI provider.


Another way to make money on this is to resell inference. Allow the user to pay a per token fee to you for convenience, or connect to their local provider.