XF 2.2 XF API Deletion Issues w/ Attachments

[IbrahimIMC]

Hello!

So I've been using the API to post threads for sales items and delete them after an expiration. So the threads get posted, attachments get attached, and threads get removed. No problem!

But then I noticed my disk space isn't freeing up... the attachments AND the posts are not removed. The threads are. So the user has a TON of attachments (and posts that still exist in the database) but when you click the 'view linked post' in the attachment viewer, the post is a 404. The attachment, however, is still alive and well and also not marked as 'unlinked' or anything.

I'm going to change my XF API Call to delete the posts first and then the thread (maybe even attachments first) but that doesn't solve the 100k attachments I need deleted now.

I've made a list of all attachment_ids (as well as linked post_ids) and if they are suitable to be deleted or not based on their original thread_id being deleted from the database or not.

I think had a script run through and try to delete the posts, but I got 403 and then 500 errors as I tried different keys and permission setups (I had the main API key I've used, but also set up a super-user level key to try to delete these but neither worked.)

All I want is to flag the attachment_ids in a way that will get them deleted from my disk. I'm willing to edit the DB directly, but would prefer either API calls or... some other way.

import csv
import requests
# Configuration
csv_file_path = 'deleted_posts.csv'
xenforo_api_url = 'https://tractorbynet.com/forums/api/posts/'
api_key = '_Mmf_HxPv6ZIdjr5YxO4sNPFC04RkYEX'
def delete_post(post_id):
    url = f"{xenforo_api_url}{post_id}/"
    headers = {
        'XF-Api-Key': api_key,
        'XF-Api-User': '397225'  # Assuming user ID 1 is an admin
    }
    response = requests.delete(url, headers=headers)
    if response.status_code == 200:
        print(f"Successfully deleted post with ID {post_id}")
    else:
        print(f"Failed to delete post with ID {post_id}. Status code: {response.status_code}")
        print(f"Response content: {response.content}")  # Add this line to print the response content
def main():
    with open(csv_file_path, mode='r') as file:
        csv_reader = csv.reader(file)
        next(csv_reader)  # Skip header row if there is one
        for row in csv_reader:
            post_id = row[1]
            delete_post(post_id)
if __name__ == "__main__":
    main()

The error is: "code": "do_not_have_permission",\n "message": "You do not have permission to view this page or perform this action.",

Thanks!

Also I discovered this API attempt also logged server errors:

UPDATE  `xf_post` SET `position` = ? WHERE `post_id` = 6874656
------------

#0 src/XF/Db/Mysqli/Statement.php(198): XF\Db\AbstractStatement->getException('MySQL query err...', 1264, '22003')
#1 src/XF/Db/Mysqli/Statement.php(78): XF\Db\Mysqli\Statement->getException('MySQL query err...', 1264, '22003')
#2 src/XF/Db/AbstractAdapter.php(96): XF\Db\Mysqli\Statement->execute()
#3 src/XF/Db/AbstractAdapter.php(323): XF\Db\AbstractAdapter->query('UPDATE  `xf_pos...', Array)
#4 src/XF/Mvc/Entity/Entity.php(1362): XF\Db\AbstractAdapter->update('xf_post', Array, '`post_id` = 687...')
#5 src/XF/Entity/Post.php(765): XF\Mvc\Entity\Entity->fastUpdate('position', -1)
#6 src/XF/Entity/Post.php(547): XF\Entity\Post->postHidden()
#7 src/XF/Mvc/Entity/Entity.php(1282): XF\Entity\Post->_postSave()
#8 src/XF/Entity/Post.php(882): XF\Mvc\Entity\Entity->save(true, false)
#9 src/XF/Service/Post/Deleter.php(81): XF\Entity\Post->softDelete('', Object(xenMade\TPM\XF\Entity\User))
#10 src/XF/Api/Controller/Post.php(164): XF\Service\Post\Deleter->delete('soft', '')
#11 src/XF/Mvc/Dispatcher.php(352): XF\Api\Controller\Post->actionDelete(Object(XF\Mvc\ParameterBag))
#12 src/XF/Api/Mvc/Dispatcher.php(26): XF\Mvc\Dispatcher->dispatchClass('XF:Post', 'Delete', Object(XF\Api\Mvc\RouteMatch), Object(XF\Api\Controller\Post), NULL)
#13 src/XF/Mvc/Dispatcher.php(115): XF\Api\Mvc\Dispatcher->dispatchFromMatch(Object(XF\Api\Mvc\RouteMatch), Object(XF\Api\Controller\Post), NULL)
#14 src/XF/Mvc/Dispatcher.php(57): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Api\Mvc\RouteMatch))
#15 src/XF/App.php(2487): XF\Mvc\Dispatcher->run()
#16 src/XF.php(524): XF\App->run()
#17 index.php(16): XF::runApp('XF\\Api\\App')
#18 {main}

Is this because they've already been "deleted" and it's trying to shift things around? I only ever sent the delete thread commands over the API endpoints, so I'm not sure why it ignored the posts under the thread, but how do I fix this?

 

[Paul B]

If a thread is only soft deleted then the posts and attachments are also just soft deleted.

If a thread is hard deleted, all posts and attachments are also permanently deleted.

 

[IbrahimIMC]

The thread was hard deleted via the API but the posts and attachments were not deleted. The posts exist in xf_post and the attachments exist in xf_attachment, xf_attachment_data, and xf_attachment_view. I can view all the attachments still when I go to the Attachment browser and view by user.

What I need is a way to delete them now, whether that's through xf, db editing, or something else.

 

[IbrahimIMC]

Here is the python code that is used to delete threads:

def delete_thread(thread_id, lot_id, hard_delete=True):
    if not thread_id:
        if LOGLEVEL >= 1:
            logger.warning("Thread ID is empty, skipping delete.")
        return
    
    # Set the value of hard_delete based on the input
    delete_type = 'yes' if hard_delete else 'no'
    
    logger.info(f"Deleting thread ID {thread_id} with hard_delete={delete_type}")
    
    # Modify the API request to include the hard_delete parameter
    url = f"{XENFORO_API_ENDPOINT}threads/{thread_id}?hard_delete={delete_type}"
    headers = {"XF-Api-Key": XENFORO_API_KEY, "XF-Api-User": XF_API_USER}
    
    response = requests.delete(url, headers=headers)
    
    if LOGLEVEL >= 2:
        logger.debug(f"Response status code: {response.status_code}")
    if LOGLEVEL >= 3:
        logger.debug(f"Response content: {response.content}")
    if response.status_code == 400:
        log_error(response)
    
    response.raise_for_status()

    # If the thread is successfully deleted, mark the lot as deleted in the database
    update_was_deleted(lot_id)

The threads are successfully deleted, but the attachments are not. Also this has been going on for over 1 month. Threads are deleted, posts and attachments are NOT.

 

[IbrahimIMC]

Oh, additionally when I try to delete an attachment from the attachment browser I get the following error:

ErrorException: [E_WARNING] Attempt to read property "title" on null in src/XF/ModeratorLog/Post.php at line 61

    XF::handlePhpError() in src/XF/ModeratorLog/Post.php at line 61
    XF\ModeratorLog\Post->setupLogEntityContent() in src/XF/ModeratorLog/AbstractHandler.php at line 69
    XF\ModeratorLog\AbstractHandler->log() in src/XF/ModeratorLog/Logger.php at line 149
    XF\ModeratorLog\Logger->log() in src/XF/Logger.php at line 37
    XF\Logger->logModeratorAction() in src/XF/Attachment/Post.php at line 49
    XF\Attachment\Post->onAttachmentDelete() in src/XF/Entity/Attachment.php at line 329
    XF\Entity\Attachment->_postDelete() in src/XF/Mvc/Entity/Entity.php at line 1664
    XF\Mvc\Entity\Entity->delete() in src/XF/Admin/Controller/Attachment.php at line 127
    XF\Admin\Controller\Attachment->actionDelete() in src/XF/Mvc/Dispatcher.php at line 352
    XF\Mvc\Dispatcher->dispatchClass() in src/XF/Mvc/Dispatcher.php at line 258
    XF\Mvc\Dispatcher->dispatchFromMatch() in src/XF/Mvc/Dispatcher.php at line 115
    XF\Mvc\Dispatcher->dispatchLoop() in src/XF/Mvc/Dispatcher.php at line 57
    XF\Mvc\Dispatcher->run() in src/XF/App.php at line 2487
    XF\App->run() in src/XF.php at line 524
    XF::runApp() in admin.php at line 13

So either I called the API wrong, or the API thread-delete function isn't working properly. But I still need a solution to deleting attachments.

 

[IbrahimIMC]

Would it be feasible to take my list of attachments_ids I want deleted, and to edit the xf_attachment table to change the 'unassociated' column to '1'?

I did that on one of the attachments and was able to delete it in the attachment browser.

If anyone has a really good reason not to do that, let me know, but it doesn't seem to have any downside since the threads are already gone and the attachments are not viewable from the forum right now.

 

[Paul B]

If the attachments are not associated with any content then those records can be removed from the DB.

It shouldn't really matter how you do it but doing it via the UI (after changing the column to 1) is likely better as any associated data (if there is any) should also be cleaned up.

You're also going to have to deal with the orphaned posts.
The easiest way to do that may be to create a new thread, associate all the posts to it, then delete the thread.

 

[IbrahimIMC]

If the attachments are not associated with any content then those records can be removed from the DB.

It shouldn't really matter how you do it but doing it via the UI (after changing the column to 1) is likely better as any associated data (if there is any) should also be cleaned up.

Okay, thank you!

You're also going to have to deal with the orphaned posts.
The easiest way to do that may be to create a new thread, associate all the posts to it, then delete the thread.

That was going to be where my next question was going, but that sounds like the best way! Thanks!