Tag Essentials - with AI auto tagging and XFRM support

Tag Essentials - with AI auto tagging and XFRM support [Paid] 2.6.8

No permission to buy (£35.00)
Overview Updates (48) Reviews (5) History Discussion
got this error while running Rebuild Empty Wiki Tags

XF\Db\Exception: MySQL query error [1406]: Data too long for column 'tagess_wiki_description' at row 1 in src/XF/Db/AbstractStatement.php at line 230
  1. XF\Db\AbstractStatement->getException() insrc/XF/Db/Mysqli/Statement.php at line 207
  2. XF\Db\Mysqli\Statement->getException() insrc/XF/Db/Mysqli/Statement.php at line 83
  3. XF\Db\Mysqli\Statement->execute() in src/XF/Db/AbstractAdapter.phpat line 96
  4. XF\Db\AbstractAdapter->query() in src/XF/Db/AbstractAdapter.php at line 323
  5. XF\Db\AbstractAdapter->update() in src/XF/Mvc/Entity/Entity.php at line1575
  6. XF\Mvc\Entity\Entity->_saveToSource() in src/XF/Mvc/Entity/Entity.phpat line 1300
  7. XF\Mvc\Entity\Entity->save() in src/XF/Mvc/Entity/Entity.php at line 1370
  8. XF\Mvc\Entity\Entity->saveIfChanged() insrc/addons/AVForums/TagEssentials/Job/WikiFixup.php at line 36
  9. AVForums\TagEssentials\Job\WikiFixup->rebuildById() insrc/XF/Job/AbstractRebuildJob.php at line 42
  10. XF\Job\AbstractRebuildJob->run() in src/XF/Job/Manager.php at line275
  11. XF\Job\Manager->runJobInternal() in src/XF/Job/Manager.php at line205
  12. XF\Job\Manager->runJobEntry() in src/XF/Job/Manager.php at line 121
  13. XF\Job\Manager->runByIds() insrc/XF/Admin/Controller/ToolsController.php at line 146
  14. XF\Admin\Controller\ToolsController->actionRunJob() insrc/XF/Mvc/Dispatcher.php at line 362
  15. XF\Mvc\Dispatcher->dispatchClass() in src/XF/Mvc/Dispatcher.php at line 264
  16. XF\Mvc\Dispatcher->dispatchFromMatch() insrc/XF/Mvc/Dispatcher.php at line 121
  17. XF\Mvc\Dispatcher->dispatchLoop() in src/XF/Mvc/Dispatcher.php at line 63
  18. XF\Mvc\Dispatcher->run() in src/XF/App.php at line 2777
  19. XF\App->run() in src/XF.php at line 798
  20. XF::runApp() in admin.php at line 15
 
Sumanta0018 said:
got this error while running Rebuild Empty Wiki Tags

XF\Db\Exception: MySQL query error [1406]: Data too long for column 'tagess_wiki_description' at row 1 in src/XF/Db/AbstractStatement.php at line 230
  1. XF\Db\AbstractStatement->getException() insrc/XF/Db/Mysqli/Statement.php at line 207
  2. XF\Db\Mysqli\Statement->getException() insrc/XF/Db/Mysqli/Statement.php at line 83
  3. XF\Db\Mysqli\Statement->execute() in src/XF/Db/AbstractAdapter.phpat line 96
  4. XF\Db\AbstractAdapter->query() in src/XF/Db/AbstractAdapter.php at line 323
  5. XF\Db\AbstractAdapter->update() in src/XF/Mvc/Entity/Entity.php at line1575
  6. XF\Mvc\Entity\Entity->_saveToSource() in src/XF/Mvc/Entity/Entity.phpat line 1300
  7. XF\Mvc\Entity\Entity->save() in src/XF/Mvc/Entity/Entity.php at line 1370
  8. XF\Mvc\Entity\Entity->saveIfChanged() insrc/addons/AVForums/TagEssentials/Job/WikiFixup.php at line 36
  9. AVForums\TagEssentials\Job\WikiFixup->rebuildById() insrc/XF/Job/AbstractRebuildJob.php at line 42
  10. XF\Job\AbstractRebuildJob->run() in src/XF/Job/Manager.php at line275
  11. XF\Job\Manager->runJobInternal() in src/XF/Job/Manager.php at line205
  12. XF\Job\Manager->runJobEntry() in src/XF/Job/Manager.php at line 121
  13. XF\Job\Manager->runByIds() insrc/XF/Admin/Controller/ToolsController.php at line 146
  14. XF\Admin\Controller\ToolsController->actionRunJob() insrc/XF/Mvc/Dispatcher.php at line 362
  15. XF\Mvc\Dispatcher->dispatchClass() in src/XF/Mvc/Dispatcher.php at line 264
  16. XF\Mvc\Dispatcher->dispatchFromMatch() insrc/XF/Mvc/Dispatcher.php at line 121
  17. XF\Mvc\Dispatcher->dispatchLoop() in src/XF/Mvc/Dispatcher.php at line 63
  18. XF\Mvc\Dispatcher->run() in src/XF/App.php at line 2777
  19. XF\App->run() in src/XF.php at line 798
  20. XF::runApp() in admin.php at line 15
Click to expand...
What version of Xenforo and what version of Tag Essentials please?
 
After a few years and daily reports of many new queries from Wikipedia, I feel like I've ended up with something like 9.5 bazillion entries for my tags. Now I'm asking myself, what's the point of this? The topic of the site is cars, but my tag database also contains, for example, a few hundred cities, colors, and a million other terms. This probably doesn't help Google much, and I doubt users are actually looking at it. Manually reviewing which keywords to keep and which to discard would likely take half a year. So, I'll be turning it off today. Is there a function or any information on how I can clear the database, or do I need to dig into it myself?
 
Robert9 said:
After a few years and daily reports of many new queries from Wikipedia, I feel like I've ended up with something like 9.5 bazillion entries for my tags. Now I'm asking myself, what's the point of this? The topic of the site is cars, but my tag database also contains, for example, a few hundred cities, colors, and a million other terms. This probably doesn't help Google much, and I doubt users are actually looking at it. Manually reviewing which keywords to keep and which to discard would likely take half a year. So, I'll be turning it off today. Is there a function or any information on how I can clear the database, or do I need to dig into it myself?
Click to expand...
I have a similar experience. Perhaps we can work together to improve the quality of our tags.
I have some thoughts and plans on this going forward, Robert.
Firstly, I plan to alter the ChatGPT prompt when auto tagging to have it return tags more useful for content discovery.
Secondly I plan to have a batch process which uses ChatGPT to remove less useful tags.
Plural versions of words, for example. There are often two versions of essentially the same tag. With Car and Cars I would remove Cars.

I also plan on removing certain word types. Or at least making it an option to. So, I think that nouns have the most use as tags.
  • Engine
  • Transmission
  • Chassis
  • Exhaust
  • Dashboard
Some verbs may also have a use
  • Accelerate
  • Brake
  • Steer
  • Shift
  • Drift
While some may not
  • Be
  • Have
  • Do
  • Say
  • Get
Ditto, adjectives. While some are useful
  • Turbocharged
  • Aerodynamic
  • Fuel-efficient
  • Manual
  • Sleek
Some are not
  • Good
  • New
  • First
  • Last
  • Long
Which really points to limiting verbs and adjectives tags to those relevant to your core subject.

While we can definitely block all
  • Adverbs
  • Prepositions
  • Determiners
  • Pronouns
  • Conjunctions
  • Interjections
Refining the ChatGPT prompt to process all the tags in one go will require some effort to get right because it could be an expensive process to run.

I also have an idea to add the unwanted word types to the blacklist. This would then require ChatGPT for checking all new tags. Or running a batch process every day or so to remove the unwanted ones which have been added recently.

I'd be interested to know what you think about these ideas.
 
  • I'm not sure if these tags are even useful. Do they help with Google? This week my client supposedly had 10,000 clicks per day from Google. I don't want to mess that up, especially if I don't know whether the tags play a role in that.

  • I don't find the site's own wiki for tags useful in this case, at least the users don't seem to use it. Does it help with Google? I don't know. There are a few tens of thousands of pages indexed, and even more pages that aren't. I would need to investigate this more thoroughly.

    My question is, how do I turn this off? Through an option that stops pulling in information. And how do I clear a specific table?

  • I'd like to use tags as a true index for linking content together. But maybe with only a few dozen tags. For that, I would need to disable everything automatic and handle it manually. I’m considering building a test forum for this. What I find missing in XenForo is a way to not just link similar topics but manually connect them—no, not with tabs! Not based on a similar title, but with specific keywords that aren't used in the typical way.

    Sometimes I think we need a second version of tags, which could be called “manual indexes” to connect a variety of topics. I still need to think more about this, as it's something that's been bothering me for a while.

    To explain better: For an article about Mercedes, I don't want every topic with the Mercedes tag, but only selected topics. The only way to do that would be to delete "Mercedes" wherever I don’t want a connection. Or by introducing another field like "Important" and "Not important" for a topic list.

    Or you could introduce tags like idx_volkswagen; but then you’re essentially back to a second version of the tag function. I hope this makes sense. The goal is to have manually curated lists of topics that belong together but aren't simply tied to the "Mercedes" tag or the "Mercedes" prefix. I once tried this with an additional field in xf_thread, allowing me to say: Fetch all related topics with current_thread.field. But to make this work, you'd need an advanced management system where you could search, find, and connect topics.

  • I’ll read your text again, or have it translated. To optimize tags using AI, you'd need to give the AI very specific instructions: fetch everything related to cars that's also in the text, but no standalone verbs or attributes, "red Mercedes" is fine, but "red" alone is not, and so on. This could turn into quite a long list, but if you keep telling the AI what to do, you'll get good results eventually. I currently have a test forum where I send things to the AI, and it generates posts. However, I don’t have much time to continue with it right now.
 
Ok, you want to approach this by word types. My 10 cents:
  1. My language comprises more than 220,000 words.
    For good tags on the topic of cars, I might need maybe 500?
    Blacklisting everything else won't work.

  2. Of course, you can filter out auxiliary verbs like "to be" and "to have", but sometimes you might still need these in combination.

  3. I believe, as mentioned above, that you need to give the AI a longer, handmade, and repeatedly refined instruction. That’s exactly what I have been doing manually so far, resulting in lists with SQL statements. But of course, this needs to be handled by a cron job. Send a topic with a=b every x minutes, then tell the AI: Hey, you! I want this, this, and this, but not that and not that.
In my test forum, as mentioned, it works with manual input, and I think it will also have to work as a cron job, provided the interface allows me to do that. I've spent a few hours/days on 20,000 topics already. Without AI, it would take me years or I wouldn't even attempt it.
 
Robert9 said:
Blacklisting everything else won't work.
Click to expand...
Why not?
Robert9 said:
Of course, you can filter out auxiliary verbs like "to be" and "to have", but sometimes you might still need these in combination.
Click to expand...
They might only be suggested as part of a tag phrase when ChatGPT considers it a relevant tag. Tag relevance is what is important, I think.
Robert9 said:
I believe, as mentioned above, that you need to give the AI a longer, handmade, and repeatedly refined instruction.
Click to expand...
We were discussing allowing admins to edit the prompt, but they'd have to be very careful, of course.
 
I just had a quick brainstorm earlier and here's my suggestion.
On mass remove all instances of tag, remove it on any resource it is tagged on and add to blacklist.
Basically for admins, there should display a moderation menu of sorts, those are a couple of ideas.

Also I think adding description on hovering over the tag (like threads on forumview) would be beneficial too.

If I'm not mistaken, isn't it the preview-tooltip thats responsible for that?
 

Attachments

  • Screenshot (254).webp
    Screenshot (254).webp
    94.9 KB · Views: 12
  • Screenshot (255).webp
    Screenshot (255).webp
    60.5 KB · Views: 12
Last edited:
I tried using AI for a day, but my purchased plan was probably used up quickly, resulting in a log full of errors. I'm turning it off for now. Maybe a cron job would be needed to gradually retrieve tags?
 
Robert9 said:
I tried using AI for a day, but my purchased plan was probably used up quickly, resulting in a log full of errors. I'm turning it off for now. Maybe a cron job would be needed to gradually retrieve tags?
Click to expand...
How many new threads? Did they have a lot of text on average? Are you using the simpler tag from title auto-suggest?
That often populates threads with enough tags that ChatGPT isn't used?
Our tag threshold is 3 to 6 tags.
And what plan are you on?
We used $0.28 in the last month. 28 cents for 2,500 threads. Crazy small.
 
Furthermore, tried on Safari, Firefox and mobile Safari. On mobile, it opens the editor box, but there is no editor. When I disable the add-on, it works normally.
 
You must log in or register to reply here.
Back
Top Bottom