Thread Tagging Improvements

Thread Tagging Improvements 1.0.7 Patch Level 6

No permission to download
Overview Updates (14) Reviews (6) History Discussion
mazzly updated Thread Tagging Improvements with a new update entry:

Bugfix release

  • Don't call the tag-suggest endpoint if tagging is not allowed, should fix @Faust problem
  • Safer limits when collecting potential words, hopefully this fixes @vwts problem on the larger forum
    • Batch load 50k thread titles instead of 100K (doubt this is the problem)
    • Batch the bulk insert of potential words 5k at a time, before could do one bulk insert of 50K items.
Click to expand...

Read the rest of this update entry...
 
Installed the new version.
There are manual rebuild jobs awaiting completion. Continue running them.
Processing…
Rebuilding... Potential Threads for Tag (52... 103... 192... 3753)
Worked 5 minutes: Error at the same place - 3753 (any wrong tag in the database?)
An exception occurred: [XF\Db\Exception] MySQL statement prepare error [2006]: MySQL server has gone away in src/XF/Db/AbstractStatement.php on line 230
  1. XF\Db\AbstractStatement->getException() in src/XF/Db/Mysqli/Statement.php at line 198
  2. XF\Db\Mysqli\Statement->getException() in src/XF/Db/Mysqli/Statement.php at line 41
  3. XF\Db\Mysqli\Statement->prepare() in src/XF/Db/Mysqli/Statement.php at line 56
  4. XF\Db\Mysqli\Statement->execute() in src/XF/Db/AbstractAdapter.php at line 96
  5. XF\Db\AbstractAdapter->query() in src/XF/Mvc/Entity/Finder.php at line 1397
  6. XF\Mvc\Entity\Finder->fetchOne() in src/XF/Repository/UpgradeCheck.php at line 32
  7. XF\Repository\UpgradeCheck->getLatestUpgradeCheck() in src/XF/Admin/App.php at line 254
  8. XF\Admin\App->renderPageHtml() in src/XF/App.php at line 2281
  9. XF\App->renderPage() in src/XF/Admin/App.php at line 138
  10. XF\Admin\App->renderPage() in src/XF/Mvc/Dispatcher.php at line 404
  11. XF\Mvc\Dispatcher->render() in src/XF/Mvc/Dispatcher.php at line 60
  12. XF\Mvc\Dispatcher->run() in src/XF/App.php at line 2483
  13. XF\App->run() in src/XF.php at line 524
  14. XF::runApp() in admin.php at line 13

Manually:
Rebuild caches ->
Potential Tags Helper: Rebuild Potential Threads for Tags - Cache rebuilt successfully. (worked 5 minutes)
Potential Tags Helper: Rebuild Potential Words - error [2006] (immediately error, started with 3753)

Thanks!
 
Last edited:
vwts said:
Installed the new version.
There are manual rebuild jobs awaiting completion. Continue running them.
Processing…
Rebuilding... Potential Threads for Tag (52... 103... 192... 3753)
Wworked 5 minutes: Error at the same place - 3753 (any wrong tag in the database?)
An exception occurred: [XF\Db\Exception] MySQL statement prepare error [2006]: MySQL server has gone away in src/XF/Db/AbstractStatement.php on line 230
  1. XF\Db\AbstractStatement->getException() in src/XF/Db/Mysqli/Statement.php at line 198
  2. XF\Db\Mysqli\Statement->getException() in src/XF/Db/Mysqli/Statement.php at line 41
  3. XF\Db\Mysqli\Statement->prepare() in src/XF/Db/Mysqli/Statement.php at line 56
  4. XF\Db\Mysqli\Statement->execute() in src/XF/Db/AbstractAdapter.php at line 96
  5. XF\Db\AbstractAdapter->query() in src/XF/Mvc/Entity/Finder.php at line 1397
  6. XF\Mvc\Entity\Finder->fetchOne() in src/XF/Repository/UpgradeCheck.php at line 32
  7. XF\Repository\UpgradeCheck->getLatestUpgradeCheck() in src/XF/Admin/App.php at line 254
  8. XF\Admin\App->renderPageHtml() in src/XF/App.php at line 2281
  9. XF\App->renderPage() in src/XF/Admin/App.php at line 138
  10. XF\Admin\App->renderPage() in src/XF/Mvc/Dispatcher.php at line 404
  11. XF\Mvc\Dispatcher->render() in src/XF/Mvc/Dispatcher.php at line 60
  12. XF\Mvc\Dispatcher->run() in src/XF/App.php at line 2483
  13. XF\App->run() in src/XF.php at line 524
  14. XF::runApp() in admin.php at line 13

Manually:
Rebuild caches ->
Potential Tags Helper: Rebuild Potential Threads for Tags - Cache rebuilt successfully. (worked 5 minutes)
Potential Tags Helper: Rebuild Potential Words - error [2006] (immediately error, started with 3753)

Thanks!
Click to expand...
Do you have access to the mysql logs wheb it goes down? 🤔
 
2023-01-05T19:01:49.798694Z 11110234 [Note] Aborted connection 11110234 to db: 'xxx' user: 'xxx' host: 'localhost' (Got a packet bigger than 'max_allowed_packet' bytes)

When using other addons, this error does not occur.
 
Last edited:
My forum, with almost 120,000 users and a large active database from the year 2006 and I have not had any problems, everything works correctly.

Great job!!
 
202,000 users from 2004 year.
The first version of the addon worked well...

Removed the addon and installed again - same error.
The other small forum works well, hosted on the same server with the same MySQL settings.
 
Last edited:
vwts said:
202,000 users from 2004 year.
The first version of the addon worked well...

Removed the addon and installed again - same error.
The other small forum works well, hosted on the same server with the same MySQL settings.
Click to expand...
Can you increase the value of 'max_allowed_packet' and the 'max_execution_time'?
I think it may be due to too low a value for such a large process and from my observations your DB seems to be even larger.
 
mazzly said:
@vwts try that release and put the Bulk Insert Chunk size to something like 500, if that doesn't work try 50... If that doesn't work we've got something fishy going on 🙈
Click to expand...
Thanks! Tried... Old error (2006).

Some ideas:
I changed the option: Minimum word length - 3 (was 4)
And the position at which the addon stops working has changed!

It was 3753, now it's 3089.
Therefore, I draw the following conclusion: in the process of work, the addon gets to some wrong value.
This is where the error occurs. It does not accumulate in the course of work. An error occurred in the value of a field in a table in the database.
 
vwts said:
Thanks! Tried... Old error (2006).

Some ideas:
I changed the option: Minimum word length - 3 (was 4)
And the position at which the addon stops working has changed!

It was 3753, now it's 3089.
Therefore, I draw the following conclusion: in the process of work, the addon gets to some wrong value.
This is where the error occurs. It does not accumulate in the course of work. An error occurred in the value of a field in a table in the database.
Click to expand...
You could try setting the Potential tags max word count to 1 and see if that helps... It is not a viable solution ofc, but could help pinpoint the issue...

I had an idea that maybe I forgot to put limits on the max length of the word that are inserted into the words columns, but I apparently had done that also :D

Could you run the following in your SQL shell and post the output here? show VARIABLES like '%max_allowed_packet%';
On our forums it outputs:
Code: 
+--------------------------+------------+
| Variable_name            | Value      |
+--------------------------+------------+
| max_allowed_packet       | 83886080   |
| slave_max_allowed_packet | 1073741824 |
+--------------------------+------------+
 
Code: 
show VARIABLES like '%max_allowed_packet%';
+--------------------------+------------+
| Variable_name            | Value      |
+--------------------------+------------+
| max_allowed_packet       | 4194304    |
| slave_max_allowed_packet | 1073741824 |
+--------------------------+------------+
2 rows in set (0,00 sec)

But I don't think that's the reason.
The first versions of the addon worked well, on the same database, with the same SQL values.
The latest version that worked built a list of tags.
I looked at the list of tags in the admin panel.
I saw a bad tag and erased it NOT in your addon!
Or in the admin panel or in the post.
And immediately your addon crashed with an error.
The addon didn't work anymore.

Maybe this will help...
Thanks!
 
mazzly said:
You could try setting the Potential tags max word count to 1 and see if that helps... It is not a viable solution ofc, but could help pinpoint the issue...
I had an idea that maybe I forgot to put limits on the max length of the word that are inserted into the words columns, but I apparently had done that also :D
Click to expand...
With this value the addon started working.
Set value to 2. Addon stopped working again...
 
Black Horse Off Road said:
Version 1.04 was the best after trying them all, as newer versions fail to generate tags from titles
Click to expand...
Huh that is weird...

Were there any errors?

Did you try the rebuild caches also?

I don't remember exactly which version, but 1.0.4 had the potential tags stored in simplecache while newer ones store it in its own db table...
 
mazzly said:
Huh that is weird...

Were there any errors?

Did you try the rebuild caches also?

I don't remember exactly which version, but 1.0.4 had the potential tags stored in simplecache while newer ones store it in its own db table...
Click to expand...
no errors but the newer versions aren't working properly, tags wouldn't be generated from titles and only the previously tagged ones will be assigned if presented in the title
 
Last edited:
You must log in or register to reply here.

Similar threads

Freelancer
  • Question Question
XF 1.5 Automatic thread tagging with RSS feeds & automatic threads not always working
Replies
2
Views
1K
Freelancer
Freelancer
Moshe1010
Why thread tagging is important?
Replies
15
Views
1K
Stuart Wright
Stuart Wright
N
Other Auto Thread Tagging for Every Old Threads
Replies
9
Views
4K
Daniel Hood
Daniel Hood
CTXMedia
Thread tag management - how are you doing it?
Replies
9
Views
2K
CTXMedia
CTXMedia
Top Bottom