Tag Essentials - with AI auto tagging and XFRM support [Paid] 2.6.8

[Stuart Wright]

You could possibly implement an auto-manual feature using a user's main ChatGPT "credits" (i.e., not using OpenAI API... or even free services), Grok, or other AI services.

Export N number of thread X + thread/post 1 thread title/message in JSON format, the user uploads that file and prompts it "for each entry, examine the thread_title and thread_message of the post; give me X thread_tags that closely relate to it no longer than 3 words in length and each word no longer than 10 characters in length; try to use up to 25% of the tags that are hot topics on the subject found in thread_message if they can relate".

Hypothetically you should have a new JSON file with a new "thread_tags" line populated with "this is a, this tag, about something...". That file could be used to import and circumvent OpenAI credits on batch processes.

I'd have to play with how many threads it could tag per process because anything under 100 would take way too much time. But, if it could process 500 for "free" (unsure unsubscribed ChatGPT) with 4o or another model (or even with Grok), a batch update wouldn't take any OpenAI credits to do for as long as the file stays in the same format: "thread_id" (for easier importing), "thread_title", "thread_message", "thread_tags" (added).

The add on would export all the JSON files and would be zipped so that the user could unzip them and process them 1 by 1 until done. Then zipped back up and then decompressed/applied to all threads when done (as ChatGPT can throttle if you do too much, so it might take a day to get done on 50,000+ threads, but it'd be free, unlike with the OpenAI credits).

Too complicated. Any solution has to be easy for non-technical admins to accomplish.

 

[frm]

Too complicated. Any solution has to be easy for non-technical admins to accomplish.

Free vs. $100s might spark their interest to learn.

Download this zip file. Upload 0001.json to ChatGPT with this prompt. Download 0001_fixed.json. Upload 0001.json to import. Repeat for 0002.json until ChatGPT throttles your account, and continue on when ChatGPT account services resume.

500 threads tagged for free as opposed to normal API operating costs.

 

[Stuart Wright]

Free vs. $100s might spark their interest to learn.



500 threads tagged for free as opposed to normal API operating costs.

I think they wouldn't bother. But that's just my opinion.
And 500 threads is nothing. I did a small test on AVForums, and tagging threads using one specific keyword would involve updating 43,000 threads.

 

[frm]

A CSV would be more practical because they could manually edit the tags in Excel or something before importing with find/replace tags that don't fit.

Here is the selection of 1 thread/post, but of course, it would have to chunk it out in 100-500 posts for ChatGPT to process.

How much time would it take to process for free vs. how much money would be spent on the API calls? Some might prefer the free option. I'm fine with paying, but it could be tens to hundreds of dollars to process an entire forum when it could be chunked out and done day by day for free as the admin sees fit (even quicker if they have trusted mods with ChatGPT).

SELECT threads.thread_id, threads.title, posts.message
FROM xf_thread AS threads
JOIN xf_post AS posts
  ON threads.thread_id = posts.thread_id
WHERE threads.thread_id = 309
  AND posts.post_id = (
    SELECT MIN(post_id)
    FROM xf_post
    WHERE thread_id = 309
  );

Probably best to loop XF:Finder than direct DB queries, but it gives you an idea of linking the post to the thread to get tags from both the title and content (message).

This gives me threads.csv (renamed to txt as the file format doesn't work here).

I dragged and dropped that into the free ChatGPT box and gave it this command:

Add a new column "tags". For each row in this CSV read both the title and the message columns. From these columns generate 5 to 7 tags to be added to the "tags" column. The tags must not be more than 45 characters in length and must be 1 to 3 words most relating to both the title and message that most describe the title and message; use popular SEO search terms if they can relate to the title and message better and fit the word count and length if possible. Comma separate the tags before you add it to the tags column. Save it as a new CSV file.

It gave me threads_with_better_seo_tags.csv (renamed to txt) in the blink of an eye.

It suggested tags (to be added or applied) to thread id 309 as:

forum, caravan, caravan forum, practical, practical caravan, practical caravan forum, discussion

You could do a direct import and apply those tags to thread_id 309, or, the user can polish the tags in Excel before importing as it's more visually appealing than the previous JSON suggestion.

It's not that technical, I'm sure everyone has worked with Excel (even Google Sheets would work) or would just use what's suggested in the first place (as they're pretty good tags from the content provided).

That said, I only did it for 1 thread and have not tested the maximum number of threads I can do at any given time before 1 hour throttling. But, if you could get 500 at a time done for free, it would save a hell of a lot of money from paying for API calls.

Something to consider.

Note: I did this on a separate database as an example. I am not licensed to run Tag Essentials on the forum I pulled data from for this example. I am licensed for 1 forum and use it for 1 forum only, and I highly suggest any admins thinking of running Tag Essentials to purchase more than 1 copy if they intend on using it for multiple forums to support the production and updates of the add on.

 

[Alpha1]

It suggested tags (to be added or applied) to thread id 309 as:
forum, caravan, caravan forum, practical, practical caravan, practical caravan forum, discussion

This is exactly the issue I would like to prevent. Some of these tags seem useless as these will not yield you any traffic from google. (forum, discussion, practical) Nor are these tags useful for your users. Good tags need to be specific terms people actually are searching for. Tags that users will click on and tag pages that will come up in Google search results. The main focus for adding tags should be Search Intent/Keyword Intent. The rest is just pollution and will do the opposite of SEO. More info here:

What Is Search Intent? How to Identify It & Optimize for It

Search intent is the goal behind a user‘s query. Creating content that meets it is vital to ranking on Google.

www.semrush.com

I have done a lot of SEO with auto-tagging scripts in the past. It can yield good traffic, but it can also do a lot of damage if auto-tagging results in mass bad tags.

 

[Stuart Wright]

This is exactly the issue I would like to prevent. Some of these tags seem useless as these will not yield you any traffic from google. (forum, discussion, practical) Nor are these tags useful for your users. Good tags need to be specific terms people actually are searching for. Tags that users will click on and tag pages that will come up in Google search results. The main focus for adding tags should be Search Intent/Keyword Intent. The rest is just pollution and will do the opposite of SEO. More info here:

What Is Search Intent? How to Identify It & Optimize for It

Search intent is the goal behind a user‘s query. Creating content that meets it is vital to ranking on Google.

www.semrush.com

I have done a lot of SEO with auto-tagging scripts in the past. It can yield good traffic, but it can also do a lot of damage if auto-tagging results in mass bad tags.

Certain commonly used useless words could be blacklisted like forum and discussion.

 

[Alpha1]

Sure, but how do prevent the AI from selecting less common useless tags? Isn't the whole point of using air to get accurate useful tags? I'm sure it is possible to have AI generate and add accurate, useful tags that will enhance the threads SEO instead of degrading it. That would likely require a well thought out prompt. And maybe not allowing it to create new tags. Or to moderate all new tags manually.

 

[frm]

This is exactly the issue I would like to prevent. Some of these tags seem useless as these will not yield you any traffic from google. (forum, discussion, practical) Nor are these tags useful for your users.

I quickly made that prompt. You can make a prompt that fits your forum well and get 1-2 high competition keywords or phrases, 2-3 medium, and 3-4 low competition (beforehand, you can get top ranking keywords on your forums niche [1000+ keywords] and instruct it to use 1 or 2 if it fits the message, but if it can't, don't use any from your list and make a better one).

You make the prompt, so it's even better than a batch update because you can't prompt it differently each time.

If you could batch export to CSV, you could even include something along the lines of "These keywords are for threads in AmazingTech's off-topic movie discussion. While we focus on tech, these tags should more accurately reflect tags that would be related to movies, streaming services, etc."

If you check the data that was used, there was not a lot to work off of, either.

This thread is for the general discussion of the forum Practical Caravan Forum. Please add to the discussion here.

I'm sure if I wanted to run a practical test on this, it would spit out the greatest reflecting tags, and tags that are exact match low competition (to rank for some searches that would get higher traffic on those because it's easier to rank for).

Certain commonly used useless words could be blacklisted like forum and discussion.

Since it's auto-manual, you can add that to the prompt without running a new batch update after 50,000 threads have processed and you notice that 1-5 tags were repetitive and don't really fit your forum. You could 1) manually remove them, or 2) get the maximum number of tags that reflect your site the first time.

Edit 2: In the case of the data used, I would want forum and discussion used as tags because it's for a directory of forums. So, I wouldn't have blacklisted them.

But, best of all, you can rerun a batched export with a tweaked prompt that can get you even better keywords for import over and over until you're satisfied. With the API batch update, it's one and then done for most people's budgets.

Edit: You could also add a "matrix" to the prompt and upload all files at once. Blacklist "forum" and the obvious, but something like "Give me 1 keyword or tag that can be used to cross-reference up to 15% of the messages if possible. But, don't force a tag into a message if it doesn't fit just to make the quota. Only use tags that fit the forum's niche and message and make sense to cross-link the content if the discussions are more than 40% similar".

Then, you won't get the "forum" output, but you might get 2000 threads with "television" if the messages across different forums discuss something on television to cross-link it better.

 

[Stuart Wright]

The prompt has to be very carefully constructed to return the most useful tags in the correct format.
We plan to allow admin to edit it.
Putting new tags in the mod queue is an option. It would create a lot of work in a busy forum.

 

[frm]

The prompt has to be very carefully constructed to return the most useful tags in the correct format.
We plan to allow admin to edit it.
Putting new tags in the mod queue is an option. It would create a lot of work in a busy forum.

If not auto-manual process of export/import as proposed (and free as opposed to OpenAI credit usage), I hope that you can set a test run on specific thread ids to see the results before letting it drain the OpenAI credit without being able to tweak and modify prior to letting it rip on all threads.

(Kind of like XF's 'test outbound email')

 

[🔥Iggy🔥]

im easing into the thing still here, but as far as moderation, IMHO it doesnt need to be officially built in....thats what ive been doing already but its simple enough just using the "last used" filter tool...

 

[frm]

im easing into the thing still here, but as far as moderation, IMHO it doesnt need to be officially built in....thats what ive been doing already but its simple enough just using the "last used" filter tool...

I presume this is for ChatGPT tags made on creation and not a batch update.

With a batch update, there would have to be a test run, or a way to stop it from completing the full run, or you will drain 10s or 100s of dollars (maybe upwards of 1000 depending on if you're sending the full message to be examined) if you can't stop the process and see junk tags in the last tag used (if it will update from a batch run).

A test run on 10 threads would be best to be able to refine the command if you are receiving irrelevant tags or tags you see you want to blacklist (and they shouldn't be added as it's a test).

Select 10 random threads, send them through with the command, output a table like this:

Test Job	Title	Message	Suggested Tags
1	[Link:Thread Title]	Message snippet...	tag1, tag2, tag3, tag4, tag5, tag6, tag7, tag8, tag9, tag10
...	...	...	...

Then you can go thread by thread to see if the tags are appropriate and blacklist tags that are irrelevant or are just random words like "where", "can", etc. if not in a phrase like "where is California" for instance.

Even though you could blacklist forum from the beginning, imagine 1 of the 5-10 tags you request always being forum (or another word you could've spotted during a test run), on a batch run of 100,000 threads. That's a mighty lot of credits...

There should either be an auto-manual process (for free tag creation) or a test run before you let it loose on all the threads as it could be a costly mistake if you don't refine your prompt good enough from the start, and there's no way of telling that, unless you did an import (free) or test run.

 

[Alpha1]

Yes, that will do,

It seems to me that it would be a good idea to have a way to add preferred tags into the mix. Similar to how we have blocked tags. Either by allowing us to upload a csv with the preferred tags or to have a field in admincp to list preferred tags. Another method (but more limiting method) would be to allow us to batch import tags, let us whitelist these so they don't get deleted and only allow the LLM to use existing tags.

I think this LLM auto-tagging addon for Discourse has a nice approach:

Discourse AI Topic Summary 🤖

ℹ Summary Uses a remote AI language model to prepare and post a summary of a topic. 🛠 Repository Link GitHub - merefield/discourse-ai-topic-summary: Uses a remote AI language model to prepare and post a summary of a Topic 📖 Install Guide How to install plugins in Discourse Enjoying this...

meta.discourse.org

It analyses the complete thread and then takes a number of actions based upon that data. This includes adding tags. The benefit of this is that the data is sent to the LLM only once and therefore the cost is incurred only once. And it then the addon takes a number of actions that require an understanding of the thread content, including thread summary, thread cover, thread tags. I think such functionality could be expanded to selecting the correct prefix, forum, SEO meta tags.

This is likely be beyond the scope of the current addon, but I do think that the approach of first analyzing & summarizing the whole thread and only then intelligently generating tags makes sense.
I posted a related suggestion here that proposes a more holistic approach:

Thread 'AI generated Thread Summary, Tags, Prefix, SEO Meta, Thread cover, etc.'

Mar 16, 2025

AI / LLM's can be used to make threads and other content more useful and better organized. For better usability, SEO, content discovery and readability.

One of the challenges of using forums is reading through mega-threads with thousands of replies. Especially if high quality replies and useless comments are mixed. For this reasons there have been many requests for a summary feature over the years. Now with the advent of AI the technology to create good summaries for large amounts of text is possible.

Some other challenges for forums are:

1. Correcting useless thread titles. i.e...

• Alpha1
• artificial intelligence
• Replies: 52
• Forum: XenForo suggestions

• 
• 

Some interesting settings in that discourse AI auto-tagging addon are:

• Only auto-tag threads with at least x replies.
• Recreate after x new replies have been posted.
• Only use existing tags for auto-tagging.

 

[frm]

The thing with ChatGPT, though, is that it knows Google's keywords (high, medium, low) and search volume. So, with the right prompt, you can get 1 high competition keyword in, 2 medium, and 3 low to try and rank easier for lesser searched topics, then 2-3 more that it thinks also relates to the content to matrix/cross-link threads/etc. between your existing tags (which you would add to the prompt).

The right prompt might bring old threads out of the graveyard and up into at least the top 25 for lower searched terms.

 

[Stuart Wright]

I've had an idea about how to potentially improve SEO using Tag Essentials.
Some tags have synonyms, right? It's a feature of Tag Essentials which helps focus on the correct (canonical) version of a tag when people might use the wrong version.
Example.

This guy is Spider-man. Note the dash in his name.
But people more often type Spiderman.
So the canonical tag would be spider-man with a synonym of spiderman. Admin have to set up synonyms.

In the schema.org output of a page, the keywords element is made of a list of tags. View the source of any thread with tags and you'll see.
If we were to include the synonyms for each of the tags, the SEO would be better, wouldn't it?

 

[alternadiv]

Question. Should the url of a synonym tag automatically redirect to the canonical url of the parent tag? For example, should /tags/dogs automatically redirect to /tags/dog?