XF 1.2 Processing... Deleting... Threads (28/$s)

[Live Free]

I'm in the process of an import from vBulletin to Xenforo. Was almost completed. However, when updating counters/importing blogs/installing mods I get this:



The number 32 keeps changing... can anyone give some insight into this process? The Url says: /admin.php?tools/run-deferred

 

[Mike]

That may be a bug, but that's completing a thread deletion task. The only standard method of triggering it is to delete a forum.

You'll just need to let it run to clean up the threads that were in the forum you removed.

 

[Live Free]

That may be a bug, but that's completing a thread deletion task. The only standard method of triggering it is to delete a forum.

Okay, I'm very confused. I had completed the import and was rebuilding the caches, left the page in the middle of a rebuild. Was going to go back to it after importing the vB blogs to Better Blogs (sucessful).

I didn't delete a forum or any threads so I'm really confused. I hope this isn't deleting my imported data. It's taken 5-6 hours to complete this process.

I can find no way to cancel - when I click the cancel button the process starts again if I try to install an addon, add a resource, etc.

This is a clean install, clean db.

Any other info? Advice? I thought I was close to bringing my board back online.

It's in the 400s now...

Edit: some additional info - possibly unrelated, I experienced the issue in the first post before trying below

I was also attempting to install the widget framework plugin and it seems to have only partially been completed:



My sidebar disappeared at the same time.

Really at a loss.

Do I need to restart the import?

 

[Live Free]

Okay, it looks like some of my forums were somehow deleted. I'm missing half my posts. Is there any way I can reimport the forum and thread data only, or do I have to do the entire process over? (5 hours)

 

[Mike]

You would need to do a fresh install to redo any part of an import. You should look at the admin log to see when a forum was deleted. An admin must have done it.

 

[Jon W]

This add-on might allow you to do a partial import:
http://xenforo.com/community/resources/import-tools-by-waindigo.1255/

Have never tried doing it after deleting nodes/threads though. It might only work on things that were never imported in the first place.

 

[Live Free]

You would need to do a fresh install to redo any part of an import. You should look at the admin log to see when a forum was deleted. An admin must have done it.

I'm the only admin and I definitely didn't delete any forums or threads.

This add-on might allow you to do a partial import:
http://xenforo.com/community/resources/import-tools-by-waindigo.1255/

Have never tried doing it after deleting nodes/threads though. It might only work on things that were never imported in the first place.

Thanks, I appreciate it. I already started a new import though, I'll just have to push through. Definitely having a celebratory drink once this is finished!

 

[Rigel Kentaurus]

The only instance that I could fine that triggers the ThreadDelete is the one that Mikes mention, when a Forum is deleted and _postDelete() is invoked



Since this is a Defered task it can be trigger by ... well, anything

 

[Rigel Kentaurus]

I suggest doing the following to hunt down the culprit

On your files, open with a text editor library/XenForo/Deferred/ThreadDelete.php

look for

  public function execute(array $deferred, array $data, $targetRunTime, &$status)
   {

change it to

  public function execute(array $deferred, array $data, $targetRunTime, &$status)
   {
     throw new Exception("ThreadDelete is not allowed");

That will DENY any deferred job of that type to be created, and it will end in your Server Errors Logs, so we would be able to see what is attempting to add that, and if it's even comming from the Forum DataWriter postDelete

 

[Live Free]

I went ahead and did as you said. I replaced the vBulletin.php file and the File Health no longer showed it. I also added the above code to deny ThreadDelete.

I then reuploaded Better Blogs and the .xml file. The file installed fine. Then, I went and filled in the database details for the import. Clicking Configure gave the following error:

An exception occurred: ThreadDelete is not allowed in /home/admin/public_html/demo/library/XenForo/Deferred/ThreadDelete.php on line 7

[LIST=1]
[*]XenForo_Deferred_ThreadDelete->execute() in XenForo/Model/Deferred.php at line 197
[*]XenForo_Model_Deferred->runDeferred() in XenForo/Model/Deferred.php at line 320
[*]XenForo_Model_Deferred->_runInternal() in XenForo/Model/Deferred.php at line 273
[*]XenForo_Model_Deferred->run() in XenForo/ViewRenderer/Abstract.php at line 352
[*]XenForo_ViewRenderer_Abstract::hasManualDeferredToRun() in XenForo/ViewRenderer/HtmlAdmin.php at line 50
[*]XenForo_ViewRenderer_HtmlAdmin->renderRedirect() in XenForo/FrontController.php at line 583
[*]XenForo_FrontController->renderView() in XenForo/FrontController.php at line 158
[*]XenForo_FrontController->run() in /home/admin/public_html/demo/admin.php at line 13
[/LIST]

Line 7 of ThreadDelete.php is of course the code added, mentioned above. I took at the error files, but I don't really know what I'm looking for.

Edit: on this demo board, when I go back to import external data for the blogs, it gives an import in progress message. I click continue import and it's now letting me import (seemingly) properly.

It seems to me that it has something to do with configuring the import database details.

Edit: The error I posted above is displayed at: demo/admin.php?import/config

 

[Rigel Kentaurus]

I went ahead and did as you said. I replaced the vBulletin.php file and the File Health no longer showed it. I also added the above code to deny ThreadDelete.

I then reuploaded Better Blogs and the .xml file. The file installed fine. Then, I went and filled in the database details for the import. Clicking Configure gave the following error:

An exception occurred: ThreadDelete is not allowed in /home/spider7/public_html/demo/library/XenForo/Deferred/ThreadDelete.php on line 7

[LIST=1]
[*]XenForo_Deferred_ThreadDelete->execute() in XenForo/Model/Deferred.php at line 197
[*]XenForo_Model_Deferred->runDeferred() in XenForo/Model/Deferred.php at line 320
[*]XenForo_Model_Deferred->_runInternal() in XenForo/Model/Deferred.php at line 273
[*]XenForo_Model_Deferred->run() in XenForo/ViewRenderer/Abstract.php at line 352
[*]XenForo_ViewRenderer_Abstract::hasManualDeferredToRun() in XenForo/ViewRenderer/HtmlAdmin.php at line 50
[*]XenForo_ViewRenderer_HtmlAdmin->renderRedirect() in XenForo/FrontController.php at line 583
[*]XenForo_FrontController->renderView() in XenForo/FrontController.php at line 158
[*]XenForo_FrontController->run() in /home/spider7/public_html/demo/admin.php at line 13
[/LIST]

Line 7 of ThreadDelete.php is of course the code added, mentioned above. I took at the error files, but I don't really know what I'm looking for.

Edit: on this demo board, when I go back to import external data for the blogs, it gives an import in progress message. I click continue import and it's now letting me import (seemingly) properly.

It seems to me that it has something to do with configuring the import database details.

The deferred task is there, and it is being triggered by Configure, but it was not added by the Blogs Import process.

Sadly, the Stack Trace is telling me that it is trying to run a ThreadDelete, but it does not tell me which add-on, file, or process added it.

Could you do this change. On that same ThreadDelete.php file

Find:

class XenForo_Deferred_ThreadDelete extends XenForo_Deferred_Abstract
{

Replace with:

class XenForo_Deferred_ThreadDelete extends XenForo_Deferred_Abstract
{
   
   public function __construct()
   {
     parent::__construct();
     throw new Exception("Adding a ThreadDelete is not alowed");
   }

I am hoping after changing the file, and trying again, the exception we get in the Server Logs will have the exact line in PHP code that is attempting to schedule the Thread Delete

 

[Live Free]

Sorry, the error I posted was what I saw on-screen - blank white page except the error. Here is the actual Server Error Log:

Error Info
Exception: ThreadDelete is not allowed - library/XenForo/Deferred/ThreadDelete.php:7
Generated By: Daniel, 31 minutes ago

Stack Trace
#0 /home/admin/public_html/demo/library/XenForo/Model/Deferred.php(197): XenForo_Deferred_ThreadDelete->execute(Array, Array, 8.43266201019, '')
#1 /home/admin/public_html/demo/library/XenForo/Model/Deferred.php(320): XenForo_Model_Deferred->runDeferred(Array, 8.43266201019, '', false)
#2 /home/admin/public_html/demo/library/XenForo/Model/Deferred.php(273): XenForo_Model_Deferred->_runInternal(Array, 8.43266510963, '', false)
#3 /home/admin/public_html/demo/library/XenForo/ViewRenderer/Abstract.php(352): XenForo_Model_Deferred->run(true, 8.43266510963)
#4 /home/admin/public_html/demo/library/XenForo/ViewRenderer/HtmlAdmin.php(50): XenForo_ViewRenderer_Abstract::hasManualDeferredToRun()
#5 /home/admin/public_html/demo/library/XenForo/FrontController.php(583): XenForo_ViewRenderer_HtmlAdmin->renderRedirect(4, 'http://www.writ...', NULL, Array)
#6 /home/admin/public_html/demo/library/XenForo/FrontController.php(158): XenForo_FrontController->renderView(Object(XenForo_ControllerResponse_Redirect), Object(XenForo_ViewRenderer_HtmlAdmin), Array)
#7 /home/admin/public_html/demo/admin.php(13): XenForo_FrontController->run() #8 {main}

Request State
array(3) { ["url"] => string(57) "http://www.site.org/demo/admin.php?import/config" ["_GET"] => array(1) { ["import/config"] => string(0) "" } ["_POST"] => array(3) { ["config"] => array(3) { ["db"] => array(7) { ["host"] => string(9) "localhost" ["port"] => string(4) "3306" ["username"] => string(15) "dbusername" ["password"] => string(11) "dbpass" ["dbname"] => string(21) "dbname" ["prefix"] => string(2) "prefix" ["charset"] => string(0) "" } ["blogsAttachmentPath"] => string(6) "images" ["retain_keys"] => string(1) "1" } ["importer"] => string(38) "XfAddOns_Blogs_Importer_BlogsVbulletin" ["_xfToken"] => string(8) "********" } }

Searching Deferred.php doesn't return any results for "ThreadDelete"

Edit: there are 26 entries for ThreadDelete in the database table xf_deferred.

 

[Live Free]

With both edits to ThreadDelete.php I get this error when configuring the import (white screen error, not logged in the admincp):

Parse error: syntax error, unexpected T_IF, expecting T_FUNCTION in /home/admin/public_html/demo/library/XenForo/Deferred/ThreadDelete.php on line 12

Beginning of ThreadDelete.php:

<?php

class XenForo_Deferred_ThreadDelete extends XenForo_Deferred_Abstract
{
  
   public function __construct()
   {
     parent::__construct();
     throw new Exception("Adding a ThreadDelete is not allowed");
   }
    public function execute(array $deferred, array $data, $targetRunTime, &$status)
   {
     throw new Exception("ThreadDelete is not allowed");
        if (!isset($data['conditions']))
        {
            $data = array('conditions' => $data);
        }

 

[Live Free]

Okay, so I tried it two more times.

The first time it was before I implemented your latest edit. I got the first error, went back to import external data, and was able to import it. However, after the import 1/2 the forum posts were missing (seemingly from ThreadDelete, but there was no additional log and the edit should have prevented that). I haven't tried to replicate this yet, but I probably will while monitoring the tables in xf_deferred.

The second time I implemented both ThreadDelete.php edits and got the error in the above post (without it logged in the admincp). I was still able to import the data. I did that, and it imported successfully, without any missing posts, seemingly done correctly.

Two differences: one time has with the first ThreadDelete.php edit and the other was with boh edits.

The second difference is that I clicked import attachments when there was no attachment to import (dummy directory), the second time I skipped the import attachments button (as I should in this case).

Could the attempted import of blog attachments, when they don't exist, be linked to the main cause? That would explain this particular success, though not when the problem occurs with the configuration (unless the use of a dummy attachment directory in configure is also tied to the issue).

 

[Rigel Kentaurus]

Okay, so I tried it two more times.

The first time it was before I implemented your latest edit. I got the first error, went back to import external data, and was able to import it. However, after the import 1/2 the forum posts were missing (seemingly from ThreadDelete, but there was no additional log and the edit should have prevented that). I haven't tried to replicate this yet, but I probably will while monitoring the tables in xf_deferred.

The second time I implemented both ThreadDelete.php edits and got the error in the above post (without it logged in the admincp). I was still able to import the data. I did that, and it imported successfully, without any missing posts, seemingly done correctly.

Two differences: one time has with the first ThreadDelete.php edit and the other was with boh edits.

The second difference is that I clicked import attachments when there was no attachment to import (dummy directory), the second time I skipped the import attachments button (as I should in this case).

Could the attempted import of blog attachments, when they don't exist, be linked to the main cause? That would explain this particular success, though not when the problem occurs with the configuration (unless the use of a dummy attachment directory in configure is also tied to the issue).

The fact that there are 26 entries on that xf_deferred table means that they will run, sometime.

You notice them being started when you do the Blogs Import, but they could've been started by another action instead, they could be started anytime, no matter if you are in configure, attachments, or something else.

I would just delete all entries from that deferred table. Especially the ones programmed to delete threads at a later date.

 

[Live Free]

The fact that there are 26 entries on that xf_deferred table means that they will run, sometime.

You notice them being started when you do the Blogs Import, but they could've been started by another action instead, they could be started anytime, no matter if you are in configure, attachments, or something else.

I would just delete all entries from that deferred table. Especially the ones programmed to delete threads at a later date.

The entries in the deferred table where there on the test I did when I mentioned it, but they didn't show up in the database during the last two attempts (with the edits to ThreadDelete.php).

 

[Rigel Kentaurus]

The entries in the deferred table where there on the test I did when I mentioned it, but they didn't show up in the database during the last two attempts (with the edits to ThreadDelete.php).

Well, the edits to ThreadDelete are preventing anything new from being added ...

It would be nice to know what is adding them, though. I sent you the edited ThreadDelete.php file in a private conversation. That would give you an entry in the Error Log whenever something tries to add a defered task to delete threads.

Blogs does not delete Forums or Threads ... what else do you have installed?

 

[Live Free]

Swapped the file with the one you sent me and tried it again (those it wasn't a clean install, had already imported it, this would be a duplicate import).

After entering the config details I got this error:

Error Info
Exception: Adding a ThreadDelete is not alowed - library/XenForo/Deferred/ThreadDelete.php:9
Generated By: Daniel, 1 minute ago

Stack Trace
#0 /home/admin/public_html/demo/library/XenForo/Deferred/Abstract.php(26): XenForo_Deferred_ThreadDelete->__construct()
#1 /home/admin/public_html/demo/library/XenForo/Model/Deferred.php(59): XenForo_Deferred_Abstract::create('ThreadDelete')
#2 /home/admin/public_html/demo/library/XenForo/Application.php(1246): XenForo_Model_Deferred->defer('ThreadDelete', Array, 'threadDelete_6', true, NULL) #3 /home/admin/public_html/demo/library/XenForo/DataWriter/Forum.php(93): XenForo_Application::defer('ThreadDelete', Array, 'threadDelete_6', true)
#4 /home/admin/public_html/demo/library/XenForo/DataWriter.php(1767): XenForo_DataWriter_Forum->_postDelete()
#5 /home/admin/public_html/demo/library/XenForo/Model/Node.php(1148): XenForo_DataWriter->delete()
#6 /home/admin/public_html/demo/library/XenForo/DataWriter/Node.php(245): XenForo_Model_Node->deleteChildNodes(Array, false)
#7 /home/admin/public_html/demo/library/XenForo/DataWriter.php(1767): XenForo_DataWriter_Node->_postDelete()
#8 /home/admin/public_html/demo/library/XenForo/ControllerAdmin/Import.php(73): XenForo_DataWriter->delete()
#9 /home/admin/public_html/demo/library/XenForo/FrontController.php(337): XenForo_ControllerAdmin_Import->actionConfig()
#10 /home/admin/public_html/demo/library/XenForo/FrontController.php(134): XenForo_FrontController->dispatch(Object(XenForo_RouteMatch)) #11 /home/admin/public_html/demo/admin.php(13): XenForo_FrontController->run()
#12 {main}

Request State
array(3) { ["url"] => string(57) "http://www.url.com/demo/admin.php?import/config" ["_GET"] => array(1) { ["import/config"] => string(0) "" } ["_POST"] => array(3) { ["config"] => array(3) { ["db"] => array(7) { ["host"] => string(9) "localhost" ["port"] => string(4) "3306" ["username"] => string(15) "dbusername" ["password"] => string(11) "dbpassword" ["dbname"] => string(21) "dbname" ["prefix"] => string(2) "prefix" ["charset"] => string(0) "" } ["blogsAttachmentPath"] => string(6) "images" ["retain_keys"] => string(1) "1" } ["importer"] => string(38) "XfAddOns_Blogs_Importer_BlogsVbulletin" ["_xfToken"] => string(8) "********" } }

 

[Live Free]

The fact that there are 26 entries on that xf_deferred table means that they will run, sometime.

You notice them being started when you do the Blogs Import, but they could've been started by another action instead, they could be started anytime, no matter if you are in configure, attachments, or something else.

I would just delete all entries from that deferred table. Especially the ones programmed to delete threads at a later date.

If I delete them though, that wouldn't stop them from reoccurring, right?

Other things installed - but disabled:

Resource Manager
Better Blogs
[bd] Widget Framework
[Tinhte] XenTag
[RT] Online Status Ribbon
Thread Quota Based on Posts 1.0 (custom coded)
Thread Icons 1.0 (switches thread avatars to "classic" vbulletin indicators) (custom coded)
Recent Tweets Widget (custom coded)

After the test import of the blogs I'm also getting several errors like this:

Error Info
ErrorException: Undefined offset: 33687 - library/XfAddOns/Blogs/Model/BlogKey.php:44
Generated By: Unknown Account, 11 minutes ago
Stack Trace
#0 /home/admin/public_html/demo/library/XfAddOns/Blogs/Model/BlogKey.php(44): XenForo_Application::handlePhpError(8, 'Undefined offse...', '/home/admin/p...', 44, Array)
#1 /home/admin/public_html/demo/library/XfAddOns/Blogs/Cron/CreateBlogKeys.php(43): XfAddOns_Blogs_Model_BlogKey->getBlogKey(Array)
#2 /home/admin/public_html/demo/library/XfAddOns/Blogs/Cron/CreateBlogKeys.php(17): XfAddOns_Blogs_Cron_CreateBlogKeys->processKeys()
#3 [internal function]: XfAddOns_Blogs_Cron_CreateBlogKeys::runBlogKeys(Array)
#4 /home/admin/public_html/demo/library/XenForo/Model/Cron.php(356): call_user_func(Array, Array)
#5 /home/admin/public_html/demo/library/XenForo/Deferred/Cron.php(20): XenForo_Model_Cron->runEntry(Array)
#6 /home/admin/public_html/demo/library/XenForo/Model/Deferred.php(197): XenForo_Deferred_Cron->execute(Array, Array, 9.99999904633, '')
#7 /home/admin/public_html/demo/library/XenForo/Model/Deferred.php(320): XenForo_Model_Deferred->runDeferred(Array, 9.99999904633, '', false)
#8 /home/admin/public_html/demo/library/XenForo/Model/Deferred.php(273): XenForo_Model_Deferred->_runInternal(Array, NULL, '', false) #9 /home/admin/public_html/demo/deferred.php(15): XenForo_Model_Deferred->run(false) 
#10 {main}
Request State
array(3) { ["url"] => string(46) "http://www.site.com/demo/deferred.php" ["_GET"] => array(0) { } ["_POST"] => array(3) { ["_xfRequestUri"] => string(6) "/demo/" ["_xfNoRedirect"] => string(1) "1" ["_xfResponseType"] => string(4) "json" } }

Also am getting this server errror:

rror Info
ErrorException: Undefined variable: forum - library/WidgetFramework/WidgetRenderer.php(452) : runtime-created function:1
Generated By: ledeUnsuddy, Today at 2:52 PM
Stack Trace
#0 /home/admin/public_html/library/WidgetFramework/WidgetRenderer.php(452) : runtime-created function(1): XenForo_Application::handlePhpError(8, 'Undefined varia...', '/home/admin/p...', 1, Array)
#1 [internal function]: __lambda_func(Array)
#2 /home/admin/public_html/library/WidgetFramework/WidgetRenderer.php(456): call_user_func('?lambda_1', Array)
#3 /home/admin/public_html/library/WidgetFramework/WidgetRenderer.php(542): WidgetFramework_WidgetRenderer->_executeExpression('in_array($forum...', Array)
#4 /home/admin/public_html/library/WidgetFramework/Core.php(335): WidgetFramework_WidgetRenderer->render(Array, 'forum_list', Array, Object(XenForo_Template_Public), '??????????????<...')
#5 /home/admin/public_html/library/WidgetFramework/Core.php(246): WidgetFramework_Core->_renderWidgetsFor('forum_list', Array, Object(XenForo_Template_Public), '????????????<!-...')
#6 /home/admin/public_html/library/WidgetFramework/Listener.php(51): WidgetFramework_Core->renderWidgetsFor('forum_list', Array, Object(XenForo_Template_Public), Array)
#7 [internal function]: WidgetFramework_Listener::template_post_render('forum_list', '????????????<ol...', Array, Object(XenForo_Template_Public))
#8 /home/admin/public_html/library/XenForo/CodeEvent.php(58): call_user_func_array(Array, Array)
#9 /home/admin/public_html/library/XenForo/Template/Abstract.php(195): XenForo_CodeEvent::fire('template_post_r...', Array, 'forum_list')
#10 /home/admin/public_html/library/XenForo/Template/Public.php(110): XenForo_Template_Abstract->render()
#11 /home/admin/public_html/library/XenForo/ViewRenderer/HtmlPublic.php(119): XenForo_Template_Public->render()
#12 /home/admin/public_html/library/XenForo/FrontController.php(604): XenForo_ViewRenderer_HtmlPublic->renderContainer(Object(XenForo_Template_Public), Array)
#13 /home/admin/public_html/library/XenForo/FrontController.php(158): XenForo_FrontController->renderView(Object(XenForo_ControllerResponse_View), Object(XenForo_ViewRenderer_HtmlPublic), Array)
#14 /home/admin/public_html/index.php(13): XenForo_FrontController->run() #15 {main}
Request State
array(3) { ["url"] => string(29) "http://www.url.com/" ["_GET"] => array(0) { } ["_POST"] => array(0) { } }

 

[Rigel Kentaurus]

Now we are on to something



That retain_keys flag is explicitly deleting a Forum and Category

... When you import the blogs, do NOT select the "retain keys" option.
If you do, that is when the "Delete Forum" kicks in.

I am not sure if I can fix that and still reuse the import logic in vBulletin (I just added a new Importer, and that seems to be in the main controller). I wonder if that will also be a problem with people trying to import "2 forums into 1" (but then again, when doing that retain keys is a bad idea anyway)