• This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.

Implemented Allow searching of soft deleted threads/posts/profile posts

digitalpoint

Well-known member
#1
We use search extensively to find spammers, people slinging the same crap from the same websites, etc. It was quite a shock when I realized that soft deleted threads, posts and profile posts aren't searchable by mods/admins. Digging a bit, I realized the DataWriters actually delete the soft deleted items from the search index completely.

I ended up changing that and now posts, threads and profile posts that are soft deleted are searchable by mods/admins (the system already filters them out of search results for users who can't see them normally).

But it got me thinking... Why does XF by default go out of it's way to make soft deleted items not searchable for mods/admins?
 

Sadik B

Well-known member
#2
Probably because they thought that once deleted by an admin / mod, under normal circumstances, admins won't want to see them again.

It could be an ACP search option. Search Results to show soft deleted content for mods/admins (yes / no).
 

MagnusB

Well-known member
#4
What I have found is that the only way to find soft deleted threads are by going through the moderator log or finding them by accident. It is annoying, to say the least.
 

CTXMedia

Formerly CyclingTribe
#5
I hit this problem recently when a moderator accidentally used the Spam Cleaner on a long-term member.

After manually restoring all threads they'd started I had to use a database SET query to make the posts 'visible' again because they wouldn't come up in a search - I then had to rebuilt my ES search index too.

I agree with others - please keep soft-deleted items indexed/searchable, but only for mods/admins. (y)

Cheers,
Shaun :D
 

Rudy

Well-known member
#6
I agree with others - please keep soft-deleted items indexed/searchable, but only for mods/admins. (y)
This was one thing my staff asked me about--if they could search for deleted items. I would think it's a simple thing to add--searching on the "visible" attribute.
 

digitalpoint

Well-known member
#7
It's not quite that easy because I don't think the search index stores the message_state (visible, deleted, moderated, etc.)

The thing I did (just block it from deleting things from the index on a soft delete) was fairly simple and just makes it work like vBulletin does... if you have access to view the post, you will see it in the results, if you don't, you won't.
 

Rudy

Well-known member
#8
It's not quite that easy because I don't think the search index stores the message_state (visible, deleted, moderated, etc.)
Good point--I'd considered adding that data to the search index, but blocking the removal of soft-deleted items from the index would work just as well IMHO. Seeing the presence of soft-deleted threads and posts is permissions-based already, so it's not a stretch to have them show up among all search results for mods and admins.
 

digitalpoint

Well-known member
#12
It's a little complicated to just zip up since I have a "Digital Point Search" addon that does a whole lot more than just allow searching of soft deleted posts... but here are some relevant parts...

For posts:
PHP:
<?php

class DigitalPointSearch_DataWriter_DiscussionMessage extends XFCP_DigitalPointSearch_DataWriter_DiscussionMessage
{
	/**
	 * Updates the search index for this message.
	 * 
	 * Index regardless of soft deletion
	 * 
	 */
	protected function _indexForSearch()
	{
		$this->_insertOrUpdateSearchIndex();
	}
}
For threads:
PHP:
<?php

class DigitalPointSearch_DataWriter_Discussion extends XFCP_DigitalPointSearch_DataWriter_Discussion
{
	/**
	 * Updates the search index for this message.
	 * 
	 * Index regardless of soft deletion
	 * 
	 * @param array $messages List of messages in this discussion. Does not include text!
	 */
	protected function _indexForSearch(array $messages)
	{
		if ($this->get('discussion_state') == 'visible')
		{
			if ($this->getExisting('discussion_state') != 'visible')
			{
				$this->_insertIntoSearchIndex($messages);
			}
			else if ($this->_needsSearchIndexUpdate())
			{
				$this->_updateSearchIndexTitle($messages);
			}
		}
	}
}
 

Rudy

Well-known member
#13
Even if there were a way to temporarily modify things so soft-deleted posts/threads would show up in searches would help us out--not having this ability (which we had in vB) is seriously hampering the staff's ability to research past issues with members who have lengthy histories with us. (We've been online 10 years now, so that can amount to a lot of history.) In fact, that is one of the top three complaints the staff has about XF.

Being on ES, I could easily regenerate the search indexes if a change were made.
 

Rudy

Well-known member
#14
It's a little complicated to just zip up since I have a "Digital Point Search" addon that does a whole lot more than just allow searching of soft deleted posts... but here are some relevant parts...
With all the buzz about XF 1.2, the staff is again asking that I make the deleted posts searchable. But given the code above, and the fact that we're using XFES, I have no idea where I'd even find similar code to look at in the system. I would think it's a simple matter to keep the deletions from getting removed from the index, but where to do this is what I'm stuck at.

Has anyone else tried this yet?
 

Rudy

Well-known member
#15
I did some digging to try to get to the bottom of this. These files are from XenForo 1.2.0.

First of all, in /library/XenForo/DataWriter/Discussion.php, at line 611 is the protected function _indexForSearch(). Here's the block of code:

PHP:
    /**
    * Updates the search index for this discussion.
    */
    protected function _indexForSearch()
    {
        if ($this->get('discussion_state') == 'visible')
        {
            if ($this->getExisting('discussion_state') != 'visible')
            {
                $this->_insertIntoSearchIndex();
            }
            else if ($this->_needsSearchIndexUpdate())
            {
                $this->_updateSearchIndexTitle();
            }
        }
        else if ($this->isUpdate() && $this->get('discussion_state') != 'visible' && $this->getExisting('discussion_state') == 'visible')
        {
            $this->_deleteFromSearchIndex();
        }
    }
And in library/XenForo/DataWriter/DiscussionMessage.php, line 730, the same function:

PHP:
    /**
    * Updates the search index for this message.
    */
    protected function _indexForSearch()
    {
        if ($this->get('message_state') == 'visible')
        {
            if ($this->getExisting('message_state') != 'visible' || $this->isChanged('message'))
            {
                $this->_insertOrUpdateSearchIndex();
            }
        }
        else if ($this->isUpdate() && $this->get('message_state') != 'visible' && $this->getExisting('message_state') == 'visible')
        {
            $this->_deleteFromSearchIndex();
        }
    }
I'm mentally exhausted as I type this (programming logic eludes me at the moment :D ), but would it be as simple as getting rid of both "else if" statements? Or, could we cleanly do away with anything having to do with message_state != 'visible' or == 'visible'?

In order to make all of the soft-deleted messages appear in search, then, would I need to rebuild the search indexes? It makes me wonder if I would have to modify something else in order to get the forum to rebuild the indexes but with the soft-deleted threads and posts intact.

My only concern after these edits: if we decide to keep these soft-deleted posts/threads in the index, will forum permissions prevent these from appearing for non-staff members? I am thinking they'll be OK, but wanted to check...