XF 2.2 Customizing "Similar threads"

Anatoliy

Well-known member
I use xenForo's "Similar threads" add-on and I like it. However there is a problem with it as it takes in consideration keyword relevance only, and doesn't (well, because it just can't) consider pages "authority". So I exported a list of top performing pages (bring most organic traffic) from Google Search Console, and my idea is to create "topic related clusters" of 11 pages, so each of them will have a widget with 10 links to others, and those 10 Similar threads will be not just keyword related, but also with the biggest "authority".

It was not difficult to create a first cluster, a widget with links to pages, and a conditional !$xf.visitor.user_id && in_array($xf.reply.contentKey, ['thread-x', 'thread-y', ...', 'thread-z']). And I added a conditional to the XFES: Similar threads widget so the pages that have the widget with my manually selected related links would not display the XFES: Similar threads widget. !$xf.visitor.user_id AND !in_array($xf.reply.contentKey, ['thread-x', ..., 'thread-z']). Everything seems to work as expected.

So here is a question that bothers me. To "similar link" all those 1000 best performing pages I will have to create 100 widgets. (And that's fine, I can handle it. )
So the conditional in XFES: Similar threads widget will include an array with 1000 :eek: elements.

Will it slow down page load speed for all the threads as php now will have to go through that huge array to see if XFES: Similar threads widget should be displayed?
 

Anatoliy

Well-known member
I completely forgot about thread urls. I moved to designing the widget template and realized that I have to somehow transform thread titles in urls.
Is there some built in function or I have to write something in my widget class?
 

nocte

Well-known member
How can I do that? I tried to find the info about adding an index in the tutorial but didn't find it.
PHP:
    public function installStep1()
    {
        $this->schemaManager()->alterTable('xf_thread', function(Alter $table)
        {
            $table->addColumn('av_lt_keyword', 'varchar', 255);
            $table->addKey('av_lt_keyword');
        });
    }

Is there some built in function or I have to write something in my widget class?
of course. I assume you will do this in your template. So you will use:

HTML:
{{ link('threads', $thread) }}
($thread contains the object representing a single thread)
 

Anatoliy

Well-known member
Thank you!
PHP:
    public function installStep1()
    {
        $this->schemaManager()->alterTable('xf_thread', function(Alter $table)
        {
            $table->addColumn('av_lt_keyword', 'varchar', 255);
            $table->addKey('av_lt_keyword');
        });
    }
Do I need also to modify somehow uninstallStep1 to delete that key thing?

of course. I assume you will do this in your template. So you will use:

HTML:
{{ link('threads', $thread) }}
($thread contains the object representing a single thread)
nice)

It looks like I'm ready to build-release and test it on my production forum. 🕺
 

Anatoliy

Well-known member
So I "keyworded" 300+ threads (about 50 clusters of 5-6 threads with the same keyword), and I noticed some duplicated thread titles, which is kind of cool as I can join those threads and it will probably give slightly higher serp. So by following "let's build an add-on" I was able to create a dashboard in the ACP that simply fetches all keyworded threads. What I want next is to add sublink that will display duplicated thread titles only (from keyworded threads, not all of threads). But I have no idea how. To chec each title to the rest 300 of titles? it will make 90000. Probably there is some built in function find_duplicates()? Please advise.

just in case, that;s what I have to display my dashboard

Code:
<?php

namespace AV\LinkedThreads\Repository;

use XF\Mvc\Entity\Finder;
use XF\Mvc\Entity\Repository;

class LinkedThread extends Repository
{
    /**
     * @return Finder
     */
    public function findLinkedThreadsForAdminView()
    {
        $finder = $this->finder('\XF:Thread');
        $finder
            ->setDefaultOrder('av_lt_keyword')
            ->where('av_lt_keyword', '!=', '');
        return $finder;
    }
}

Code:
<?php

namespace AV\LinkedThreads\Pub\Controller;
use XF\Pub\Controller\AbstractController;
class MyController extends AbstractController
    {
        public function actionIndex()
        {
            /** @var \AV\LinkedThreads\Repository\LinkedThread $repo */
            $page = $this->filterPage();
            $perPage = 20;

            $repo = $this->repository('AV\LinkedThreads:LinkedThread');

            $finder = $repo->findLinkedThreadsForAdminView()
            ->limitByPage($page, $perPage);

            $viewParams = [
                'data' => $finder->fetch(),
                'total' => $finder->total(),
                'page' => $page,
                'perPage' => $perPage
            ];

            return $this->view('AV\LinkedThreads:MyController\Index', 'av_lt_index', $viewParams);           
        }
    }
 

nocte

Well-known member
I think the easiest way to do this is to fetch all threads with "keyword" as well and pass them to the template. Then go for each thread (only 20 per page) through all other threads and check for threads with different id's and same title (in lower case).

with 300 threads, this is 6k checks per page. I think, that's o.k.

Your controller:

PHP:
       public function actionIndex()
        {
            /** @var \AV\LinkedThreads\Repository\LinkedThread $repo */
            $page = $this->filterPage();
            $perPage = 20;

            $repo = $this->repository('AV\LinkedThreads:LinkedThread');

            $finder = $repo->findLinkedThreadsForAdminView()
            ->limitByPage($page, $perPage);

            $allKeywordThreads = $repo->findLinkedThreadsForAdminView()->fetch();

            $viewParams = [
                'data' => $finder->fetch(),
                'allKeywordThreads' => $allKeywordThreads,
                'total' => $finder->total(),
                'page' => $page,
                'perPage' => $perPage
            ];

            return $this->view('AV\LinkedThreads:MyController\Index', 'av_lt_index', $viewParams);           
        }
    }

And in your template, foreach thread ($thread) you do:

HTML:
<xf:if contentcheck="true">
    <!-- print a warning, that there a threads with the samel title -->
    <xf:contentcheck>
        <xf:foreach loop="$allKeywordThreads" value="$otherThread">
            <xf:if condition="$thread.thread_id != $otherThread.thread_id && $thread.title|to_lower == $otherThread.title|to_lower">
                 <!-- print thread $otherThread as similar thread / same title -->
            </xf:if>
        </xf:foreach>
    </xf:contentcheck>
</xf:if>

btw, I think you should add ->where('discussion_state', 'visible') to your repo, so you don't return deleted threads.
 

Anatoliy

Well-known member
I think the easiest way to do this is to fetch all threads with "keyword" as well and pass them to the template. Then go for each thread (only 20 per page) through all other threads and check for threads with different id's and same title (in lower case).

with 300 threads, this is 6k checks per page. I think, that's o.k.

Your controller:

PHP:
       public function actionIndex()
        {
            /** @var \AV\LinkedThreads\Repository\LinkedThread $repo */
            $page = $this->filterPage();
            $perPage = 20;

            $repo = $this->repository('AV\LinkedThreads:LinkedThread');

            $finder = $repo->findLinkedThreadsForAdminView()
            ->limitByPage($page, $perPage);

            $allKeywordThreads = $repo->findLinkedThreadsForAdminView()->fetch();

            $viewParams = [
                'data' => $finder->fetch(),
                'allKeywordThreads' => $allKeywordThreads,
                'total' => $finder->total(),
                'page' => $page,
                'perPage' => $perPage
            ];

            return $this->view('AV\LinkedThreads:MyController\Index', 'av_lt_index', $viewParams);          
        }
    }

And in your template, foreach thread ($thread) you do:

HTML:
<xf:if contentcheck="true">
    <!-- print a warning, that there a threads with the samel title -->
    <xf:contentcheck>
        <xf:foreach loop="$allKeywordThreads" value="$otherThread">
            <xf:if condition="$thread.thread_id != $otherThread.thread_id && $thread.title|to_lower == $otherThread.title|to_lower">
                 <!-- print thread $otherThread as similar thread / same title -->
            </xf:if>
        </xf:foreach>
    </xf:contentcheck>
</xf:if>
I'll try it, but it looks like this code will do the same what I have now - lists all keyworded (it's probably not a real word, but it's handy) threads + will highlight a thread title that has duplicates. However it's not really what I need. Right now I have a list of 16 pages (20 per page), and very soon I'm planning to make the list 3 times bigger.

1.jpg

So If I use your code I guess I will have to go through all 16 pages to find duplicated titles. What I have in mind is "Duplicated Threads" submenu under "Linked Threads" and when I go there it shows only duplicates. And after I merge duplicated threads or edited titles, there will be something like "You don't have duplicated titles. Good job, Anatoliy!" )))

btw, I think you should add ->where('discussion_state', 'visible') to your repo, so you don't return deleted threads.
thanks! I can see that there are 3 states - visible, moderated, deleted. Maybe I should use ->where('discussion_state', '!=', 'deleted') ? So I don't filter away moderated. (If moderated means visible and was edited).
 

Anatoliy

Well-known member
What’s the full code to achieve what you have achieve ? If you don’t mind me asking. A bit off topic .
It will be not easy. I'm balancing on the edge between "I don't understand it" and "I don't understand it at all". )
Later, when I will be able to reproduce what I have done, I will be able to explain what I did.

P.S. and there is no any evidence yet that it will increase the traffic. )
 

Faust

Well-known member
It will be not easy. I'm balancing on the edge between "I don't understand it" and "I don't understand it at all". )
Later, when I will be able to reproduce what I have done, I will be able to explain what I did.

P.S. and there is no any evidence yet that it will increase the traffic. )
Just realised you are using XF similar add-on as well which I'm not using it. Just Andys one.
 

Anatoliy

Well-known member
I noticed that link('threads', $dat) that worked fine in a widget template, in admin dashboard template links to /anmin.php?threads/ instead of /threads/blah-blah-blah.xxx/.

Also the template I have now:
Code:
<xf:title>Linked Threads</xf:title>

<xf:pagenav page="{$page}" perpage="{$perPage}" total="{$total}" link="linked-threads" wrapperclass="block" />

<xf:foreach loop="$data" value="$dat">
        <div class="block" style="margin-bottom: 3px">
            <div class="block-container">
                <div style="padding:10px;">
                    <a href="{{ link('threads', $dat) }}"><h3 style="margin:0">{{$dat.title}}</h3></a>
                    <p style="margin:0">{{$dat.av_lt_keyword}}</p>
                </div>
            </div>
        </div>
</xf:foreach>

<xf:pagenav page="{$page}" perpage="{$perPage}" total="{$total}" link="linked-threads" wrapperclass="block" />

So with your code it should be probably like this?

Code:
<xf:title>Linked Threads</xf:title>

<xf:pagenav page="{$page}" perpage="{$perPage}" total="{$total}" link="linked-threads" wrapperclass="block" />

<xf:foreach loop="$data" value="$dat">
    
    <xf:if contentcheck="true">
    <!-- print a warning, that there a threads with the samel title -->
    <xf:contentcheck>
        <xf:foreach loop="$allKeywordThreads" value="$otherThread">
            <xf:if condition="$thread.thread_id != $otherThread.thread_id && $thread.title|to_lower == $otherThread.title|to_lower">
                 <!-- print thread $otherThread as similar thread / same title -->
            </xf:if>
        </xf:foreach>
    </xf:contentcheck>
</xf:if>
    
        <div class="block" style="margin-bottom: 3px">
            <div class="block-container">
                <div style="padding:10px;">
                    <a href="{{ link('threads', $dat) }}"><h3 style="margin:0">{{$dat.title}}</h3></a>
                    <p style="margin:0">{{$dat.av_lt_keyword}}</p>
                </div>
            </div>
        </div>
</xf:foreach>

<xf:pagenav page="{$page}" perpage="{$perPage}" total="{$total}" link="linked-threads" wrapperclass="block" />

but when I'm trying to save, it throughs an error Line 13: Tag must be a valid conditional using an is attribute or content checking. - Template name: admin:av_lt_index

🤷‍♂️
 

nocte

Well-known member
I replaced $thread.thread_id with $dat.thread_id
yeah, if your loop uses $dat instead of $thread, you have to change this.

and <xf:if condition= with <xf:if is=
sorry, my fault..

and it works!
cool!

thanks! I can see that there are 3 states - visible, moderated, deleted. Maybe I should use ->where('discussion_state', '!=', 'deleted') ? So I don't filter away moderated. (If moderated means visible and was edited).
Moderated threads are threads, that are in the Approval queue. Once they get approved, the state changes into "visible".
 

Anatoliy

Well-known member
Moderated threads are threads, that are in the Approval queue. Once they get approved, the state changes into "visible".
Gotcha. Thanks!

Look how cool is that. It shows me 2 duplicated thread titles with just 4 and 8 replies. And both threads bring me organic traffic. I bet serp just has to get higher after I merge those threads. 🕺

1.jpg

Still can't figure out how to proper link titles. Now they link to admin.php?threads/ .
 
Top