Tag Essentials - with AI auto tagging and XFRM support [Paid] 2.5.7
- Thread starter Stuart Wright
- Start date
Alpha1
Well-known member
@Stuart Wright
Does the addon also automatically remove and replace irrelevant tags?
If not, then please consider to add this feature.
Another suggestion:
Please add integration with Google Trends API so that the most popular Tags are added to the thread. i.e. so that Tags that most people are searching for on Google are added or get priority to be added. This should result in more traffic from Google.
Example: Lets say the site has a maximum of 5 tags per thread. ChatGPT identifies 30 relevant tags and has to pick the 5 most relevant out of those 30. Lets say that 3 tags currently have high search volume in Google Trends, then those 3 should be prioritized in adding tags.
Additionally feature request:
Display Google Search volume for tag suggestions, so that the admin knows which tags are useful to add for SEO.
Does the addon also automatically remove and replace irrelevant tags?
If not, then please consider to add this feature.
Another suggestion:
Please add integration with Google Trends API so that the most popular Tags are added to the thread. i.e. so that Tags that most people are searching for on Google are added or get priority to be added. This should result in more traffic from Google.
Example: Lets say the site has a maximum of 5 tags per thread. ChatGPT identifies 30 relevant tags and has to pick the 5 most relevant out of those 30. Lets say that 3 tags currently have high search volume in Google Trends, then those 3 should be prioritized in adding tags.
Additionally feature request:
Display Google Search volume for tag suggestions, so that the admin knows which tags are useful to add for SEO.
Stuart Wright
Well-known member
No. You’re talking about processing the tags in the box at the point the thread is created. How would the system know whether a tag is irrelevant or not?
This is an interesting idea. I would say that the most important factor is that the tags are relevant to the thread. Because if you force ChatGPT to create a lot of tags (30 may be an awful lot if the first post only has a few words in it), then some may be more relevant than others and you may be choosing those based on them trending in Google rather than being relevant to the thread. Then you could have people arriving at the thread and being disappointed because the thread is not about the topic they expected. Something to consider.
I don’t know anything about Google trends or search volume. But on the surface, it seems worthy of exploration.
Alpha1
Well-known member
This can be decided by ChatGPT for existing threads by evaluating thread content. it would be useful if thread tags can be evaluated by chatgpt and irrelevant tags removed or replaced.
If ChatGPT can select highly relevant tags only and then rank those by search volume, and select the top 5, then that should be a good result.
Our most popular threads have quite a lot of tags as our members added many of different levels of relevance. We used to allow 20 tags and later lowered the maximum.
Sometimes when I look in our SEO analytics suite, it suggests to add specific terms to certain threads because that is what people are searching for in Google and the thread is highly relevant. When I make that change, then it indeed increases traffic to that thread.
It seems to me that if this can be automatically done, then this will result in higher organic traffic influx.
I'd love to have all our tags optimized by ChatGPT. My site is decades old. Tags added decades ago may be a different lingo than terms people are using in Google nowadays. It would cost quite a bit of API fees, but it would be well worth it.
Stuart Wright
Well-known member
Ah so you're asking for the ability to re-AI-tag existing threads. The reason I didn't add this initially is because of the potential monetary cost.
If you were to batch update thousands of threads with ChatGPT, the cost could grow to quite a lot.
May was our first full month using Tag Essentials with ChatGPT on AVForums and MGEVs combined and the cost per thread works out at $0.00038266.
If you assume that the average number of characters in the first post of threads is the same on your forum, and you were to auto tag 10,000 threads, that's $3.83 spent instantly. 100,000 threads becomes $38.27. Although probably not instantly because...
Additionally, ChatGPT occasionally fails to return results because of a server error on their side. ChatGPT is busy and it appears to be too busy sometimes. If we were to hit ChatGPT with lots of fast requests, we might also face some issues. So it would require a fair amount of testing to ensure that we're not breaking ChatGPT.
Batch auto tagging with ChatGPT could potentially take a while.
I have sent you a private message to discuss this and your Google trends idea further.
Last edited:
Alpha1
Well-known member
I think cost could be controlled if it would be done in a process similar to batch update threads. i.e. by having criteria like node, word count, reply count, thread age, views, etc. If I only select threads/content that have at least 5 replies, visible to guests/google and in main content forums, then it reduces the number of threads to re-AI-tag by a factor of 7.
If I also factor in thread quality and post quality parameters to filter out low quality threads, then the number of threads goes down further very significantly. But that is something I do not yet have for XF2.
I think it would be valuable to be able to include more posts besides the first one as it would lead to better results. Especially since replies often add a lot of valuable information and new topics/keywords.
The cost would increase with the number of posts per thread analyzed, so being able to limit it would be needed. Some of our threads have thousands of replies, so analyzing replies without limit is probably not optimal.
If I also factor in thread quality and post quality parameters to filter out low quality threads, then the number of threads goes down further very significantly. But that is something I do not yet have for XF2.
I think it would be valuable to be able to include more posts besides the first one as it would lead to better results. Especially since replies often add a lot of valuable information and new topics/keywords.
The cost would increase with the number of posts per thread analyzed, so being able to limit it would be needed. Some of our threads have thousands of replies, so analyzing replies without limit is probably not optimal.
Stuart Wright
Well-known member
The best chance you have to creating relevant tags is by analysing the combination of forum name, thread title and first post text. That's what we do. There is no guarantee that any of the replies are relevant at all.I think cost could be controlled if it would be done in a process similar to batch update threads. i.e. by having criteria like node, word count, reply count, thread age, views, etc. If I only select threads/content that have at least 5 replies, visible to guests/google and in main content forums, then it reduces the number of threads to re-AI-tag by a factor of 7.
If I also factor in thread quality and post quality parameters to filter out low quality threads, then the number of threads goes down further very significantly. But that is something I do not yet have for XF2.
I think it would be valuable to be able to include more posts besides the first one as it would lead to better results. Especially since replies often add a lot of valuable information and new topics/keywords.
The cost would increase with the number of posts per thread analyzed, so being able to limit it would be needed. Some of our threads have thousands of replies, so analyzing replies without limit is probably not optimal.
And although you intend to be careful about which threads you batch re-AI-tag, that's not going to be the case for all admin.
I am actually interested in having batch update auto-AI-tag existing threads. But I'm nowhere near recouping the development cost in adding AI tag to Tag Essentials, yet, so it's not something I would consider right now. I'm open to other members contributing to the funding of it, though.
Last edited:
hello2029
Active member
Dear Stuart Wright,
Thank you for developing such an excellent add-on.
I am currently experiencing an issue as follows, and I'm not sure if it is related to your add-on.
However, after disabling your add-on, I no longer encounter the error.
Thank you for looking into this matter.
Server error log
#1 src/XF/Repository/Tag.php(84): preg_match('[a-zA-Z0-9_-]{3...', 'khu trượt kh...')
#2 src/addons/AVForums/TagEssentials/XF/Repository/Tag.php(56): XF\Repository\Tag->isValidTag('khu trượt kh...')
#3 src/XF/Entity/Tag.php(32): AVForums\TagEssentials\XF\Repository\Tag->isValidTag('khu trượt kh...')
#4 src/XF/Mvc/Entity/Entity.php(798): XF\Entity\Tag->verifyTag('khu trượt kh...', 'tag', 5, Array)
#5 src/XF/Mvc/Entity/Entity.php(643): XF\Mvc\Entity\Entity->_verifyValueCustom('khu trượt kh...', 'tag', 5, Array)
#6 src/XF/Mvc/Entity/Entity.php(577): XF\Mvc\Entity\Entity->set('tag', 'khu trượt kh...')
#7 src/XF/Repository/Tag.php(138): XF\Mvc\Entity\Entity->__set('tag', 'khu trượt kh...')
#8 src/XF/Service/Tag/Changer.php(314): XF\Repository\Tag->createTag('khu trượt kh...')
#9 src/XF/Service/Thread/Creator.php(503): XF\Service\Tag\Changer->save(true)
#10 src/addons/DigitalPoint/Analytics/XF/Service/Thread/Creator.php(9): XF\Service\Thread\Creator->_save()
#11 src/addons/OzzModz/ScheduledContent/XF/Service/Thread/Creator.php(19): DigitalPoint\Analytics\XF\Service\Thread\Creator->_save()
#12 src/XF/Service/ValidateAndSavableTrait.php(42): OzzModz\ScheduledContent\XF\Service\Thread\Creator->_save()
#13 src/XF/Pub/Controller/Forum.php(920): XF\Service\Thread\Creator->save()
#14 src/addons/AVForums/TagEssentials/XF/Pub/Controller/Forum.php(24): XF\Pub\Controller\Forum->actionPostThread(Object(XF\Mvc\ParameterBag))
#15 src/addons/Snog/Movies/XF/Pub/Controller/Forum.php(25): AVForums\TagEssentials\XF\Pub\Controller\Forum->actionPostThread(Object(XF\Mvc\ParameterBag))
#16 src/addons/Snog/TV/XF/Pub/Controller/Forum.php(34): Snog\Movies\XF\Pub\Controller\Forum->actionPostThread(Object(XF\Mvc\ParameterBag))
#17 src/addons/Truonglv/Groups/XF/Pub/Controller/Forum.php(112): Snog\TV\XF\Pub\Controller\Forum->actionPostThread(Object(XF\Mvc\ParameterBag))
#18 src/addons/SV/Threadmarks/XF/Pub/Controller/Forum.php(145): Truonglv\Groups\XF\Pub\Controller\Forum->actionPostThread(Object(XF\Mvc\ParameterBag))
#19 src/XF/Mvc/Dispatcher.php(352): SV\Threadmarks\XF\Pub\Controller\Forum->actionPostThread(Object(XF\Mvc\ParameterBag))
#20 src/XF/Mvc/Dispatcher.php(259): XF\Mvc\Dispatcher->dispatchClass('XF:Forum', 'PostThread', Object(XF\Mvc\RouteMatch), Object(SV\RedisCache\XF\Pub\Controller\Forum), NULL)
#21 src/XF/Mvc/Dispatcher.php(115): XF\Mvc\Dispatcher->dispatchFromMatch(Object(XF\Mvc\RouteMatch), Object(SV\RedisCache\XF\Pub\Controller\Forum), NULL)
#22 src/XF/Mvc/Dispatcher.php(57): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#23 src/XF/App.php(2487): XF\Mvc\Dispatcher->run()
#24 src/XF.php(524): XF\App->run()
#25 index.php(20): XF::runApp('XF\\Pub\\App')
#26 {main}
Thank you for developing such an excellent add-on.
I am currently experiencing an issue as follows, and I'm not sure if it is related to your add-on.
However, after disabling your add-on, I no longer encounter the error.
Thank you for looking into this matter.
Server error log
- ErrorException: Error with tag validation regex: [E_WARNING] preg_match(): Unknown modifier '{'
- src/XF/Repository/Tag.php:84
Stack trace
#0 [internal function]: XF::handlePhpError(2, '[E_WARNING] pre...', '/home/almalinux...', 84)#1 src/XF/Repository/Tag.php(84): preg_match('[a-zA-Z0-9_-]{3...', 'khu trượt kh...')
#2 src/addons/AVForums/TagEssentials/XF/Repository/Tag.php(56): XF\Repository\Tag->isValidTag('khu trượt kh...')
#3 src/XF/Entity/Tag.php(32): AVForums\TagEssentials\XF\Repository\Tag->isValidTag('khu trượt kh...')
#4 src/XF/Mvc/Entity/Entity.php(798): XF\Entity\Tag->verifyTag('khu trượt kh...', 'tag', 5, Array)
#5 src/XF/Mvc/Entity/Entity.php(643): XF\Mvc\Entity\Entity->_verifyValueCustom('khu trượt kh...', 'tag', 5, Array)
#6 src/XF/Mvc/Entity/Entity.php(577): XF\Mvc\Entity\Entity->set('tag', 'khu trượt kh...')
#7 src/XF/Repository/Tag.php(138): XF\Mvc\Entity\Entity->__set('tag', 'khu trượt kh...')
#8 src/XF/Service/Tag/Changer.php(314): XF\Repository\Tag->createTag('khu trượt kh...')
#9 src/XF/Service/Thread/Creator.php(503): XF\Service\Tag\Changer->save(true)
#10 src/addons/DigitalPoint/Analytics/XF/Service/Thread/Creator.php(9): XF\Service\Thread\Creator->_save()
#11 src/addons/OzzModz/ScheduledContent/XF/Service/Thread/Creator.php(19): DigitalPoint\Analytics\XF\Service\Thread\Creator->_save()
#12 src/XF/Service/ValidateAndSavableTrait.php(42): OzzModz\ScheduledContent\XF\Service\Thread\Creator->_save()
#13 src/XF/Pub/Controller/Forum.php(920): XF\Service\Thread\Creator->save()
#14 src/addons/AVForums/TagEssentials/XF/Pub/Controller/Forum.php(24): XF\Pub\Controller\Forum->actionPostThread(Object(XF\Mvc\ParameterBag))
#15 src/addons/Snog/Movies/XF/Pub/Controller/Forum.php(25): AVForums\TagEssentials\XF\Pub\Controller\Forum->actionPostThread(Object(XF\Mvc\ParameterBag))
#16 src/addons/Snog/TV/XF/Pub/Controller/Forum.php(34): Snog\Movies\XF\Pub\Controller\Forum->actionPostThread(Object(XF\Mvc\ParameterBag))
#17 src/addons/Truonglv/Groups/XF/Pub/Controller/Forum.php(112): Snog\TV\XF\Pub\Controller\Forum->actionPostThread(Object(XF\Mvc\ParameterBag))
#18 src/addons/SV/Threadmarks/XF/Pub/Controller/Forum.php(145): Truonglv\Groups\XF\Pub\Controller\Forum->actionPostThread(Object(XF\Mvc\ParameterBag))
#19 src/XF/Mvc/Dispatcher.php(352): SV\Threadmarks\XF\Pub\Controller\Forum->actionPostThread(Object(XF\Mvc\ParameterBag))
#20 src/XF/Mvc/Dispatcher.php(259): XF\Mvc\Dispatcher->dispatchClass('XF:Forum', 'PostThread', Object(XF\Mvc\RouteMatch), Object(SV\RedisCache\XF\Pub\Controller\Forum), NULL)
#21 src/XF/Mvc/Dispatcher.php(115): XF\Mvc\Dispatcher->dispatchFromMatch(Object(XF\Mvc\RouteMatch), Object(SV\RedisCache\XF\Pub\Controller\Forum), NULL)
#22 src/XF/Mvc/Dispatcher.php(57): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#23 src/XF/App.php(2487): XF\Mvc\Dispatcher->run()
#24 src/XF.php(524): XF\App->run()
#25 index.php(20): XF::runApp('XF\\Pub\\App')
#26 {main}
Stuart Wright
Well-known member
This is because the regex you've put into the stock XF option "tagValidation" is not valid for your version of php
Stuart Wright
Well-known member
Stuart Wright updated Tag Essentials - now with AI auto tagging with a new update entry:
Auto tagging for XFRM resource manager plus batch updating resources
Read the rest of this update entry...
Auto tagging for XFRM resource manager plus batch updating resources
Auto tagging XFRM resource tags
Note that I do not use or have XFRM, so I have not tested the new features on AVForums.
- A major new feature to auto tag new resources in the same way that threads are auto tagged from titles and AI.
- A batch update resources facility allow the batch update of resources to add tags from the resource titles (not AI).
thanks to
for funding this update.
vape.to
xenforo.com
Read the rest of this update entry...
Stuart Wright
Well-known member
Note to any Tag Essentials license holders who use the XFRM, please feed back how well this is working as I haven't seen it in action.
Thanks
Thanks
hello2029
Active member
I haven't seen it in action, so I can't provide any feedback on how well it's working.
I believe that this add-on is not functioning properly for XFRM.
vape.to
Active member
auto-tagging works well in XFRM, batch update function still not working for me. Hoping someone else can try batch update and confirm.
vape.to
Active member
after some troubleshooting, can confirm it's working as designed. Very happy with the addon.
Stuart Wright
Well-known member
Thanks for confirming.
hello2029
Active member
@Stuart Wright I am experiencing an error like this, and I'm not sure if it is related to your add-on or not.
Server error log
#1 src/XF/Repository/Tag.php(19): utf8_strtolower(Array)
#2 src/XF/Repository/Tag.php(208): XF\Repository\Tag->normalizeTag(Array)
#3 src/addons/AVForums/TagEssentials/XF/Service/Tag/Changer.php(52): XF\Repository\Tag->getTags(Array, Array)
#4 src/addons/AVForums/TagEssentials/XF/Service/Thread/Creator.php(115): AVForums\TagEssentials\XF\Service\Tag\Changer->_addTags(Array)
#5 src/addons/Snog/Movies/XF/Service/Thread/Creator.php(9): AVForums\TagEssentials\XF\Service\Thread\Creator->finalSetup()
#6 src/addons/Snog/TV/XF/Service/Thread/Creator.php(9): Snog\Movies\XF\Service\Thread\Creator->finalSetup()
#7 src/XF/Service/Thread/Creator.php(446): Snog\TV\XF\Service\Thread\Creator->finalSetup()
#8 src/addons/OzzModz/ScheduledContent/XF/Service/Thread/Creator.php(14): XF\Service\Thread\Creator->_validate()
#9 src/addons/SV/Threadmarks/XF/Service/Thread/Creator.php(87): OzzModz\ScheduledContent\XF\Service\Thread\Creator->_validate()
#10 src/XF/Service/ValidateAndSavableTrait.php(19): SV\Threadmarks\XF\Service\Thread\Creator->_validate()
#11 src/XF/Pub/Controller/Forum.php(901): XF\Service\Thread\Creator->validate(NULL)
#12 src/addons/AVForums/TagEssentials/XF/Pub/Controller/Forum.php(24): XF\Pub\Controller\Forum->actionPostThread(Object(XF\Mvc\ParameterBag))
#13 src/addons/Snog/Movies/XF/Pub/Controller/Forum.php(25): AVForums\TagEssentials\XF\Pub\Controller\Forum->actionPostThread(Object(XF\Mvc\ParameterBag))
#14 src/addons/Snog/TV/XF/Pub/Controller/Forum.php(93): Snog\Movies\XF\Pub\Controller\Forum->actionPostThread(Object(XF\Mvc\ParameterBag))
#15 src/addons/Truonglv/Groups/XF/Pub/Controller/Forum.php(112): Snog\TV\XF\Pub\Controller\Forum->actionPostThread(Object(XF\Mvc\ParameterBag))
#16 src/addons/SV/Threadmarks/XF/Pub/Controller/Forum.php(145): Truonglv\Groups\XF\Pub\Controller\Forum->actionPostThread(Object(XF\Mvc\ParameterBag))
#17 src/XF/Mvc/Dispatcher.php(352): SV\Threadmarks\XF\Pub\Controller\Forum->actionPostThread(Object(XF\Mvc\ParameterBag))
#18 src/XF/Mvc/Dispatcher.php(259): XF\Mvc\Dispatcher->dispatchClass('XF:Forum', 'PostThread', Object(XF\Mvc\RouteMatch), Object(SV\RedisCache\XF\Pub\Controller\Forum), NULL)
#19 src/XF/Mvc/Dispatcher.php(115): XF\Mvc\Dispatcher->dispatchFromMatch(Object(XF\Mvc\RouteMatch), Object(SV\RedisCache\XF\Pub\Controller\Forum), NULL)
#20 src/XF/Mvc/Dispatcher.php(57): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#21 src/XF/App.php(2487): XF\Mvc\Dispatcher->run()
#22 src/XF.php(524): XF\App->run()
#23 index.php(20): XF::runApp('XF\\Pub\\App')
#24 {main}
Server error log
- TypeError: mb_strtolower(): Argument #1 ($string) must be of type string, array given
- src/vendor/dokuwiki/utf8/utf8.php:334
- Generated by:
- 15 July 2023 at 6:08 PM
Stack trace
#0 src/vendor/dokuwiki/utf8/utf8.php(334): mb_strtolower(Array, 'utf-8')#1 src/XF/Repository/Tag.php(19): utf8_strtolower(Array)
#2 src/XF/Repository/Tag.php(208): XF\Repository\Tag->normalizeTag(Array)
#3 src/addons/AVForums/TagEssentials/XF/Service/Tag/Changer.php(52): XF\Repository\Tag->getTags(Array, Array)
#4 src/addons/AVForums/TagEssentials/XF/Service/Thread/Creator.php(115): AVForums\TagEssentials\XF\Service\Tag\Changer->_addTags(Array)
#5 src/addons/Snog/Movies/XF/Service/Thread/Creator.php(9): AVForums\TagEssentials\XF\Service\Thread\Creator->finalSetup()
#6 src/addons/Snog/TV/XF/Service/Thread/Creator.php(9): Snog\Movies\XF\Service\Thread\Creator->finalSetup()
#7 src/XF/Service/Thread/Creator.php(446): Snog\TV\XF\Service\Thread\Creator->finalSetup()
#8 src/addons/OzzModz/ScheduledContent/XF/Service/Thread/Creator.php(14): XF\Service\Thread\Creator->_validate()
#9 src/addons/SV/Threadmarks/XF/Service/Thread/Creator.php(87): OzzModz\ScheduledContent\XF\Service\Thread\Creator->_validate()
#10 src/XF/Service/ValidateAndSavableTrait.php(19): SV\Threadmarks\XF\Service\Thread\Creator->_validate()
#11 src/XF/Pub/Controller/Forum.php(901): XF\Service\Thread\Creator->validate(NULL)
#12 src/addons/AVForums/TagEssentials/XF/Pub/Controller/Forum.php(24): XF\Pub\Controller\Forum->actionPostThread(Object(XF\Mvc\ParameterBag))
#13 src/addons/Snog/Movies/XF/Pub/Controller/Forum.php(25): AVForums\TagEssentials\XF\Pub\Controller\Forum->actionPostThread(Object(XF\Mvc\ParameterBag))
#14 src/addons/Snog/TV/XF/Pub/Controller/Forum.php(93): Snog\Movies\XF\Pub\Controller\Forum->actionPostThread(Object(XF\Mvc\ParameterBag))
#15 src/addons/Truonglv/Groups/XF/Pub/Controller/Forum.php(112): Snog\TV\XF\Pub\Controller\Forum->actionPostThread(Object(XF\Mvc\ParameterBag))
#16 src/addons/SV/Threadmarks/XF/Pub/Controller/Forum.php(145): Truonglv\Groups\XF\Pub\Controller\Forum->actionPostThread(Object(XF\Mvc\ParameterBag))
#17 src/XF/Mvc/Dispatcher.php(352): SV\Threadmarks\XF\Pub\Controller\Forum->actionPostThread(Object(XF\Mvc\ParameterBag))
#18 src/XF/Mvc/Dispatcher.php(259): XF\Mvc\Dispatcher->dispatchClass('XF:Forum', 'PostThread', Object(XF\Mvc\RouteMatch), Object(SV\RedisCache\XF\Pub\Controller\Forum), NULL)
#19 src/XF/Mvc/Dispatcher.php(115): XF\Mvc\Dispatcher->dispatchFromMatch(Object(XF\Mvc\RouteMatch), Object(SV\RedisCache\XF\Pub\Controller\Forum), NULL)
#20 src/XF/Mvc/Dispatcher.php(57): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#21 src/XF/App.php(2487): XF\Mvc\Dispatcher->run()
#22 src/XF.php(524): XF\App->run()
#23 index.php(20): XF::runApp('XF\\Pub\\App')
#24 {main}
Stuart Wright
Well-known member
What is dokuwiki?
hello2029
Active member
I don't use any add-ons with that name. I think dokuwiki is part of your add-on.
After disabling your add-on everything works normally.
