1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Allow searching of soft deleted threads/posts/profile posts

Discussion in 'XenForo Suggestions' started by digitalpoint, Dec 1, 2012.

  1. digitalpoint

    digitalpoint Well-Known Member

    We use search extensively to find spammers, people slinging the same crap from the same websites, etc. It was quite a shock when I realized that soft deleted threads, posts and profile posts aren't searchable by mods/admins. Digging a bit, I realized the DataWriters actually delete the soft deleted items from the search index completely.

    I ended up changing that and now posts, threads and profile posts that are soft deleted are searchable by mods/admins (the system already filters them out of search results for users who can't see them normally).

    But it got me thinking... Why does XF by default go out of it's way to make soft deleted items not searchable for mods/admins?
     
    we_are_borg, otto, Fethi.dz and 85 others like this.
  2. Sadik B

    Sadik B Well-Known Member

    Probably because they thought that once deleted by an admin / mod, under normal circumstances, admins won't want to see them again.

    It could be an ACP search option. Search Results to show soft deleted content for mods/admins (yes / no).
     
  3. digitalpoint

    digitalpoint Well-Known Member

    I'd think if that was the reasoning and a mod/admin never wanted to see it again, they would hard delete it.
     
  4. MagnusB

    MagnusB Well-Known Member

    What I have found is that the only way to find soft deleted threads are by going through the moderator log or finding them by accident. It is annoying, to say the least.
     
    HWS likes this.
  5. CyclingTribe

    CyclingTribe Well-Known Member

    I hit this problem recently when a moderator accidentally used the Spam Cleaner on a long-term member.

    After manually restoring all threads they'd started I had to use a database SET query to make the posts 'visible' again because they wouldn't come up in a search - I then had to rebuilt my ES search index too.

    I agree with others - please keep soft-deleted items indexed/searchable, but only for mods/admins. (y)

    Cheers,
    Shaun :D
     
  6. Rudy

    Rudy Well-Known Member

    This was one thing my staff asked me about--if they could search for deleted items. I would think it's a simple thing to add--searching on the "visible" attribute.
     
  7. digitalpoint

    digitalpoint Well-Known Member

    It's not quite that easy because I don't think the search index stores the message_state (visible, deleted, moderated, etc.)

    The thing I did (just block it from deleting things from the index on a soft delete) was fairly simple and just makes it work like vBulletin does... if you have access to view the post, you will see it in the results, if you don't, you won't.
     
    Fethi.dz, CyclingTribe and Rudy like this.
  8. Rudy

    Rudy Well-Known Member

    Good point--I'd considered adding that data to the search index, but blocking the removal of soft-deleted items from the index would work just as well IMHO. Seeing the presence of soft-deleted threads and posts is permissions-based already, so it's not a stretch to have them show up among all search results for mods and admins.
     
  9. Luxus

    Luxus Well-Known Member

    Soft-deleted posts should land in a trash can within the yet to be developed mod-cp, as simple as that.
     
  10. Mendo

    Mendo Member

    I think it's a good idea to have some way of displaying the soft-deleted threads.
     
  11. Mouth

    Mouth Well-Known Member

    Would you share that change with us, please?
     
  12. digitalpoint

    digitalpoint Well-Known Member

    It's a little complicated to just zip up since I have a "Digital Point Search" addon that does a whole lot more than just allow searching of soft deleted posts... but here are some relevant parts...

    For posts:
    PHP:
    <?php

    class DigitalPointSearch_DataWriter_DiscussionMessage extends XFCP_DigitalPointSearch_DataWriter_DiscussionMessage
    {
        
    /**
         * Updates the search index for this message.
         * 
         * Index regardless of soft deletion
         * 
         */
        
    protected function _indexForSearch()
        {
            
    $this->_insertOrUpdateSearchIndex();
        }
    }
    For threads:
    PHP:
    <?php

    class DigitalPointSearch_DataWriter_Discussion extends XFCP_DigitalPointSearch_DataWriter_Discussion
    {
        
    /**
         * Updates the search index for this message.
         * 
         * Index regardless of soft deletion
         * 
         * @param array $messages List of messages in this discussion. Does not include text!
         */
        
    protected function _indexForSearch(array $messages)
        {
            if (
    $this->get('discussion_state') == 'visible')
            {
                if (
    $this->getExisting('discussion_state') != 'visible')
                {
                    
    $this->_insertIntoSearchIndex($messages);
                }
                else if (
    $this->_needsSearchIndexUpdate())
                {
                    
    $this->_updateSearchIndexTitle($messages);
                }
            }
        }
    }
     
    CyclingTribe likes this.
  13. Rudy

    Rudy Well-Known Member

    Even if there were a way to temporarily modify things so soft-deleted posts/threads would show up in searches would help us out--not having this ability (which we had in vB) is seriously hampering the staff's ability to research past issues with members who have lengthy histories with us. (We've been online 10 years now, so that can amount to a lot of history.) In fact, that is one of the top three complaints the staff has about XF.

    Being on ES, I could easily regenerate the search indexes if a change were made.
     
    feldon30 likes this.
  14. Rudy

    Rudy Well-Known Member

    With all the buzz about XF 1.2, the staff is again asking that I make the deleted posts searchable. But given the code above, and the fact that we're using XFES, I have no idea where I'd even find similar code to look at in the system. I would think it's a simple matter to keep the deletions from getting removed from the index, but where to do this is what I'm stuck at.

    Has anyone else tried this yet?
     
  15. Rudy

    Rudy Well-Known Member

    I did some digging to try to get to the bottom of this. These files are from XenForo 1.2.0.

    First of all, in /library/XenForo/DataWriter/Discussion.php, at line 611 is the protected function _indexForSearch(). Here's the block of code:

    PHP:
        /**
        * Updates the search index for this discussion.
        */
        
    protected function _indexForSearch()
        {
            if (
    $this->get('discussion_state') == 'visible')
            {
                if (
    $this->getExisting('discussion_state') != 'visible')
                {
                    
    $this->_insertIntoSearchIndex();
                }
                else if (
    $this->_needsSearchIndexUpdate())
                {
                    
    $this->_updateSearchIndexTitle();
                }
            }
            else if (
    $this->isUpdate() && $this->get('discussion_state') != 'visible' && $this->getExisting('discussion_state') == 'visible')
            {
                
    $this->_deleteFromSearchIndex();
            }
        }
    And in library/XenForo/DataWriter/DiscussionMessage.php, line 730, the same function:

    PHP:
        /**
        * Updates the search index for this message.
        */
        
    protected function _indexForSearch()
        {
            if (
    $this->get('message_state') == 'visible')
            {
                if (
    $this->getExisting('message_state') != 'visible' || $this->isChanged('message'))
                {
                    
    $this->_insertOrUpdateSearchIndex();
                }
            }
            else if (
    $this->isUpdate() && $this->get('message_state') != 'visible' && $this->getExisting('message_state') == 'visible')
            {
                
    $this->_deleteFromSearchIndex();
            }
        }
    I'm mentally exhausted as I type this (programming logic eludes me at the moment :D ), but would it be as simple as getting rid of both "else if" statements? Or, could we cleanly do away with anything having to do with message_state != 'visible' or == 'visible'?

    In order to make all of the soft-deleted messages appear in search, then, would I need to rebuild the search indexes? It makes me wonder if I would have to modify something else in order to get the forum to rebuild the indexes but with the soft-deleted threads and posts intact.

    My only concern after these edits: if we decide to keep these soft-deleted posts/threads in the index, will forum permissions prevent these from appearing for non-staff members? I am thinking they'll be OK, but wanted to check...
     
  16. DRE

    DRE Well-Known Member

    Make an addon for this plse lol
     
  17. Rudy

    Rudy Well-Known Member

    It's out of my league...I'm done coding for a living. ;)
     
  18. digitalpoint

    digitalpoint Well-Known Member

    I've made it into an addon. The addon also does many other interesting things with search like add some searchable content types (users, conversations & reports).

    At some point after I get my site migrated to 1.2, I'll be wrapping that up into something installable.
     
    Alfa1, Lucas, CyclingTribe and 3 others like this.
  19. DRE

    DRE Well-Known Member

  20. digitalpoint

    digitalpoint Well-Known Member

    Ended up rolling this functionality into the Search addon I released today...

    https://marketplace.digitalpoint.com/digital-point-search.1772/item
    [​IMG]
     
    Jeremy likes this.

Share This Page