Veritas AI Moderation

Veritas AI Moderation 1.3.2

No permission to download
Overview History Discussion
ThemeHouse

ThemeHouse

Well-known member
ThemeHouse submitted a new resource:

Veritas AI Moderation - AI Moderation for Spam and Toxicity

Veritas AI is a content moderation add-on for XenForo designed to make community management easier. Using a sophisticated multi-model AI system, it automatically detects and flags harmful content like spam and toxicity with a high degree of accuracy. Instead of spending hours on manual review, moderators can shift their focus from policing to actively engaging with and growing the community.

Disclaimer on Pricing: While this add-on is free to download, and a free trial API key is...
Click to expand...

Read more about this resource...
 
When a new post is created on the forum, the contents of the post are scored based on how likely the post is to be spam content or toxic content.
Click to expand...
Is that a real time call as the post is created or are they scanned in a queued batch?
 
Does this work on all areas of Xenforo content?

Ie, posts, profile posts, DMs, resource manager reviews

Will it work on addons? Such as Bobs AMS etc
 
Kevin said:
Is that a real time call as the post is created or are they scanned in a queued batch?
Click to expand...
They are sent in a batch every 1 second. Originally we intended for real-time, but realized that was creating an unnecessary barrier for people to post. So now we run it immediately afterwards instead :)

🔥Iggy🔥 said:
what is the benefit of this over perspective api which is free?

regards
Click to expand...
Great question! There are a few things:
1. Perspective is only for toxicity, whereas this includes spam detection as well. We also intend to add some additional future functionality (working on AI generated content detection, for example).

2. We originally used Perspective, and realized that it, like most models we've tested, generate too many false positives. So instead of relying on a single model, we run several models, and then compare the data from all of them before deciding if something is spam or toxic. The result is a significant decrease in false positives from any single model.

Basenotes said:
Does this work on all areas of Xenforo content?

Ie, posts, profile posts, DMs, resource manager reviews

Will it work on addons? Such as Bobs AMS etc
Click to expand...

It currently supports posts, profile posts, and DMs. It wouldn't support Bobs AMS out of the box, but we can add support for additional content types, particularly of a customer of Veritas needs it for their site :)
 
For the last several weeks we’ve utilized this tool and watched as our Moderators had fewer and fewer tasks to perform. In fact, when compared to last year, we’ve cut our daily Moderator actions in 1/2.

Before this plugin we would have a human Moderate every members first post, mainly to prevent Spam. Since we installed this and did away with that task our Staff have been able to spend more time inside of the community, and less time behind the magic curtains……

In the early days we did see a large amount of false positives but Ian and Audentio was great at tinkering with the model. His willingness to work together has allowed us to adjust things where now we almost don’t even notice it’s running.
 
ThemeHouse said:
They are sent in a batch every 1 second. Originally we intended for real-time, but realized that was creating an unnecessary barrier for people to post. So now we run it immediately afterwards instead :)
Click to expand...
Is the batch time configurable? I definitely prefer the submissions being done in batch but every 1 second would be an overkill for us.

ThemeHouse said:
(working on AI generated content detection, for example)
Click to expand...
A few questions about this part. :) (1) Is this actively being worked on or is planned? (2) Does content include images or just text? (3) If it is being actively worked on and if it includes images, are you looking for any sites that handle a lot of images?
 
how about triggering that only after someone has reported it ?

Can you also check with a currently logged in member if the AI conclusions are correct ?

workflow:
someone reports it (spam, offensive), AI checks it, if the post fails, ask a logged in member to verify - is this offensive/spam ? If they say YES, then put the post in moderation and lock it.
 
Kevin said:
Is the batch time configurable? I definitely prefer the submissions being done in batch but every 1 second would be an overkill for us.


A few questions about this part. :) (1) Is this actively being worked on or is planned? (2) Does content include images or just text? (3) If it is being actively worked on and if it includes images, are you looking for any sites that handle a lot of images?
Click to expand...

Apologies, I mean every 1 minute*. It is not configurable.

Regarding AI detection, Veritas already returns a prediction for whether the text is AI generated or not. However, the add-on does not currently support doing anything with this information. So, semi-working on it :)

Digital Doctor said:
how about triggering that only after someone has reported it ?

Can you also check with a currently logged in member if the AI conclusions are correct ?

workflow:
someone reports it (spam, offensive), AI checks it, if the post fails, ask a logged in member to verify - is this offensive/spam ? If they say YES, then put the post in moderation and lock it.
Click to expand...

It is triggered when a post is created. However, you can setup multiple actions for it. For example:

1. If a post from a trusted member has toxic content or spam content, create a report. The mods then decide what action to take, if any.
2. If a post from a new member is flagged, send to the approval queue. Because they are a new user, it's less likely to need a moderator to look at it before removing it. It still requires a moderator to confirm (by either approving or denying), but won't be public while it's waiting for approval/removal.
 
You must log in or register to reply here.
Back
Top Bottom