Add-on Forum Intent - Recommendations (teaser)

[Adrian Schneider]

This is a sneak peak about a commercial product I'm working on. But first, a little background...

I'm a software developer / technical forum administrator for the past 10 years. I've worked on many hundreds of forums, and have gained some insight on some key problems (or opportunities) for community growth. I feel this is an area that is very under-serviced, if you compare it with the greater software industry. We're operating blind, so my new product company Forum Intent is going to be releasing some products that specialize in developing Community Intelligence. Rather than chase around porting the typical blog, downloads, widgets, tweaks, etc. I'm going to venture out a bit further and try and better the forum industry. This is the first of many products.

--

Recommendations - a recommendations engine to identify and notify users of interesting content.

Why do I need this?
Forums are notorious for having a poor signal to noise ratio. Lot's of fluff, some amazing content, but nobody has the patience to spend all day to find it. The larger the forum, the more this hurts. How much time do we waste navigating through endless lists of threads. We know what sort of things we're interested in, but it's still hard to find.

Individual Relevancy
Everyone has individual tastes of what they want to talk about, and with whom. Popular threads, or even featured threads are not always relevant.

I know me, personally, I know which specific members pique my interest in a thread. I know which ones I like interacting with. I know which topics I always have something to say about. Combine this with a little bit of popularity, and suddenly relevant content is more obvious.

Help them find it
Respect their time and encourage more activity by suggesting threads that you know they'd be interested in. They can then focus their time on creating content, rather than sifting through it. This has two large benefits: members will post more, because they're in threads they care about, and secondly, they'll enjoy using your site.

How does it work?
The plugin continuously analyzes your member activity for patterns and builds profiles on what users are interested in seeing. With this information, it can automatically highlight those threads, show them in a widget, or directly notify the user. It works on confidence, so you can control how relevant content has to be in order for the system to take certain actions.

--

Unlike other ambitious products, I'm footing the bill for developing this. Infact, it is nearly done. I hope to release it within a few weeks for both XenForo and vBulletin 3.8 shortly after. The price is still undetermined. It will depend on the amount of interest shown in the product.

The more interest I gather, the more time I can spend improving the actual recommendations engine.We can do a lot of amazing analysis the more we spend on it. Think how much behavioral data is sitting in your forum that you can use to serve them better content.

Let me know what you guys think. I'm really excited about this one. Once you're done that, we both know we're going back to the new posts button to endlessly search through content with the hopes of finding our next good discussion

 

[TheBigK]

I'm curious to know how you measure the total impact. But if the system works, consider me in (provided the price fits our budget).

 

[tajhay]

Definitely interesting. Would love to see some screenshots

 

[Adrian Schneider]

When you say total impact, are you referring to the website? That is referring to the flagship intelligence service, and is a little out of scope for this thread. Each event, and derived / resultant event is scored and aggregated. The way they generated, and how they are processed contains a ton of secret sauce, and an enormous amount of processing power to do it at scale. I'll open up another soon to get into that more. It's a huge can of worms, I must say. Pricing will be a low monthly cost, based on the size of your forum.


If I'm mistaken, and you are talking about this recommendations system, then it's much simpler... it looks at the types of posts your users are engaging with, and starts to build recommendations based on that information. It's meant to be somewhat self-learning, rather than forcing users to tell you what they want to see.


I'll work on getting some screenshots together, but essentially it would be a typical thread list widget, or a highlighted row. The real product is a background service that enhances your forum with little user interaction.

 

[tajhay]

So from what i gather:
1) require an ongoing monthly subscription.
2) require 'ernormous amount of processing power'

cant change my vote on here, but i think ill pass

 

[Adrian Schneider]

That's a different product than what this thread is about.

This one is a downloadable plug-in for a fixed cost.

Cheers

 

[TheBigK]

@Adrian Schneider - I should have been clearer. I want to understand how the system analyses users and makes recommendations about relevant posts and threads. I think the system would mostly rely on keywords used by users and the threads they have been previously active on to figure out what they like and what they may not like.

Is there any possibility that the users will continue to be more active in similar types of threads than participate in variety of topics? Would love to see results of a pilot program.

 

[Adrian Schneider]

There is a pilot lined up, and I'll be sure to post some results once it's been running for a little while. Using keywords, like you may be hinting at, might force users into specific areas of discussion. But, remember, this is not ever going to filter results, just enhance and highlight. I know some advertising / search programs have that problem - they show you what they think you want to see, and you eventually get into a bubble of content that is not what the average user sees.

Keyword & Forums are a part of it yes, but also more subtle things like

• Who posted in those threads
  
• Who you follow, who follows you
  
• What posts you like, who likes your posts
  
• Who you communicate with over PM or VMs
• Subscriptions / watched threads
  
• Who started thread
• Thread tags

All of these things, and more to come later, will be scored against each active user/thread.

 

[Mouth]

The plugin continuously analyzes your member activity for patterns and builds profiles on what users are interested in seeing. With this information, it can automatically highlight those threads, show them in a widget, or directly notify the user. It works on confidence, so you can control how relevant content has to be in order for the system to take certain actions.

This, if done properly, is very much of interest.

Keyword & Forums are a part of it yes, but also more subtle things like

• Who posted in those threads
  
• Who you follow, who follows you
  
• What posts you like, who likes your posts
  
• Who you communicate with over PM or VMs
• Subscriptions / watched threads
  
• Who started thread
• Thread tags

All of these things, and more to come later, will be scored against each active user/thread.

Yes! Exactly what I seek, direct and intelligent engagement. Profiling users whom have similar thinking and interests at the time, and helping and encouraging them to engage together more actively But, also guiding new (or long-term) joiners with little engagement towards interest areas to get them engaging. Weighting likes, conversation participants, topic/forum watches that are recent higher than older ones, etc.

 

[Adrian Schneider]

Beta testing is starting this week, and I'm hoping to open it up as a discounted public beta sometime next week.

Progress!

 

[Kevin]

Sounds interesting! From the description I interpret it as a "Recommended for you..." type widget/display similar to how Netflix makes recommendations to you based on your interests & prior activity. With that in mind I'm curious as to the performance hits.

For example, are the recommendations done in real time per page load or stored in tables? If in tables, on a large site how much space per user is it expected to use to generate each individuals recommendations? Will the users be able to fine the recommendations that are presented to them (eg: a sci-fi fan gets a recommendation for a "Twilight" thread and wants to mark it as 'not interested' or be able to say that they 'like Star Wars' and have recommendations based upon that)?

With a growing number of popular 3rd-party add-ons that have their own content areas, like XenGallery / XenMediaGallery, any plans to also dig into those content types?

 

[Adrian Schneider]

Hey Kevin,

Yes, it's similar to something like Amazon or Netflix. As you can imagine, there is an enormous amount of room for improvement here; these companies probably have entire teams dedicated to that engine.

Recommendations are processed continuously in the background as new content comes in. As threads are posted to, it asynchronously creates jobs to process that thread for every active user. There are a few different job queues that keeps the whole thing running, and effectively provides real-time data. There is very little extra load on the actual webpage. It just joins a table, looking for a flag for that user.

My vision is to treat the whole product like a framework. It's easy to add more content sources, and it comes with an admin tool to analyze recommendations to try and improve the system. It's really going to take both myself and many of you to make this an amazing tool. For now, the infrastructure is in place to start learning and tweaking.

At this point in time, there is no public user interface. Threads are highlighted inline, or you can receive alerts if the relevance is high enough. I'll be adding a basic widget or recommended threads page soon as well. However, whether we have a full interface where they can browse their collected information and remove/tweak stuff, or whether they can just X out a bad recommendation, that's really a discussion I'd like to have as a community.

Cheers

 

[Forsaken]

I did something similar around when vB4 first came out, however it was mostly to show similar topics across multiple sites.

I think I told Waindigo about it, though I'm not sure.

 

[Adrian Schneider]

Oops, I missed a few points here...

If in tables, on a large site how much space per user is it expected to use to generate each individuals recommendations?

There are various caches of indexed data per source. I'll have to do some more testing, but I'll probably limit X to something like 20 or 50 records per user per source. Actual recommendations are thread/user/score records, so they are small. The indexed data is replaced periodically as a user's interests change naturally over time (or could), and the actual recommendations will likely be capped as well. Maybe something like the top 100 threads are saved at any given time.

All of the recommendations are controlled for usergroups, so, if you really wanted, you could have it as a paid feature. By default, it's set up for all usergroups, requiring at least 10 (configurable) posts, so there is a large enough sample size to make recommendations. It also only generates recommendations for active users, defaulting to 30 days. So if 80% of your users are stale, they won't bog down the system. And the old data is eventually pruned.

A lot of care went into making this performant and useful all the way up to big boards, because that's where I'm used to working.

With a growing number of popular 3rd-party add-ons that have their own content areas, like XenGallery / XenMediaGallery, any plans to also dig into those content types?

Some of the built in sources will scan all content types if they use some of the XenForo built-in tables. For example, likes are not specific to posts, so they'd be counted everywhere. Adding a new source is ~50 lines of code, so they are pretty trivial to add.

 

[Adrian Schneider]

We've been running tests on a fairly large site for the past few days, and things seem to be going well. We're starting to focus more on the quality of the recommendations, and how they are presented.

What would you prefer, a highly qualified list, or a larger list that you can filter through and play with? Sort by score, or date?

Thanks