Other Google Perspective API for troll posts - Add-on request

T

tim330i

Member
Google has released an API that ranks the 'toxicity' of a sentence. This would be perfect for forums, I'd love to see this baked into 2.0 but for now an add-on will work fine. Ideally it would show the poster the toxicity of their posts as they write it. From an article on the subject most people who are offensive are simply having a bad day so a ranking might snap them out of it. Additionally the ability to flag for moderation when the post is above toxicity limit would be great.

Jigsaw and its partners on Google’s Counter Abuse Technology Team released a new piece of code called Perspective, an API that gives any developer access to the anti-harassment tools that Jigsaw has worked on for over a year. Part of the team’s broader Conversation AI initiative, Perspective uses machine learning to automatically detect insults, harassment, and abusive speech online. Enter a sentence into its interface, and Jigsaw says its AI can immediately spit out an assessment of the phrase’s “toxicity” more accurately than any keyword blacklist, and faster than any human moderator.
Click to expand...

https://www.wired.com/2017/02/googles-troll-fighting-ai-now-belongs-world/

Official website -
http://www.perspectiveapi.com/
 
tim330i said:
Google has released an API that ranks the 'toxicity' of a sentence. This would be perfect for forums, I'd love to see this baked into 2.0 but for now an add-on will work fine. Ideally it would show the poster the toxicity of their posts as they write it. From an article on the subject most people who are offensive are simply having a bad day so a ranking might snap them out of it. Additionally the ability to flag for moderation when the post is above toxicity limit would be great.



https://www.wired.com/2017/02/googles-troll-fighting-ai-now-belongs-world/

Official website -
http://www.perspectiveapi.com/
Click to expand...

Would be interested in helping fund this.
 
Hello, everyone!

We see this as a very interesting idea and very useful for many communities. We have decided to develop the add-on and release it for free here on XenForo.com

The initial version of the add-on will have the two features mentioned - optional showing the toxicity rank of the message to the user and optional moderation if the rank is higher than the limit defined in the Admin Panel.

We see a great potential in this, so any suggestions for possible features to include are welcome.

We will start working on this asap and you can expect the add-on to be released within 2-3 days.

Thank you!
 
Amazing! Some way to track a users overall toxicity would be nice as well. I'm sure we all have those users that only post semi/offensive comments. Being able to identify and reach out to them or remove them would be great.

Please make sure this add-on will work for large forums!
 
tim330i said:
Amazing! Some way to track a users overall toxicity would be nice as well. I'm sure we all have those users that only post semi/offensive comments. Being able to identify and reach out to them or remove them would be great.

Please make sure this add-on will work for large forums!
Click to expand...

I am looking at this right now, trying to identify if there is a possibility to use client-side API to check the message toxicity, to avoid overloading the server with many requests and the need to make too many API requests. This will ensure the performance for any size of board basically.

Having overall toxicity calculation could slow down the things, as it would require rebuilding user's rating based on their current posts, and will force us to use server-to-server API requests. This could definitely be optimized and we will look into it in this for further versions of the add-on.
 
AddonsLab said:
Hello, everyone!

We see this as a very interesting idea and very useful for many communities. We have decided to develop the add-on and release it for free here on XenForo.com

The initial version of the add-on will have the two features mentioned - optional showing the toxicity rank of the message to the user and optional moderation if the rank is higher than the limit defined in the Admin Panel.

We see a great potential in this, so any suggestions for possible features to include are welcome.

We will start working on this asap and you can expect the add-on to be released within 2-3 days.

Thank you!
Click to expand...

Fantastic work, looking forward to this.
 
I've also signed up for developer API access. I've got specific things I would like to target to potentially reduce the moderator workload, and link to automatic reported content.

@AddonsLab would you be willing to release your version on GitHub as a collaboration?
 
Looks awesome.
I really like the demo @ http://www.perspectiveapi.com/. The ability for a user to define their acceptable toxicity level within their account preferences, and only posts <= the toxicity level chosen by the user will be shown to them looks a terrific idea.
 
Xon said:
I've also signed up for developer API access. I've got specific things I would like to target to potentially reduce the moderator workload, and link to automatic reported content.

@AddonsLab would you be willing to release your version on GitHub as a collaboration?
Click to expand...

Sure, we can do that! Collaboration information will be available in the release notes.

Thank you!
 
Excited about this. Honestly thinking it should be part of the core product (and I'm bloat averse). Will be watching development closely.

I didn't see this as an existing "Xenforo Suggestion" so I've created one here:

https://xenforo.com/community/threa...chine-learning-post-user-toxicity-api.126556/

Please visit and Like and/or respond if you support (or have an argument against) including this with the core.

I don't at all think that suggesting it as an official addition to the core stifles or otherwise makes third-party development of a Perspective bridge irrelevant -- so as I said above I'm excited about the efforts and discussions here and will watch it closely.
 
Last edited:
Looking forward to this as well. My forum is unique as in we are okay with some toxicity and a little edginess, I think our posts would be quite useful for a baseline of how not to post.

Going to request access now to be ready for the addon release.
 
You must log in or register to reply here.

Similar threads

dvsDave
Automated Image Recognition for inserting Alt Tags
Replies
9
Views
1K
Forsaken
Forsaken
Mike Fara
REQUEST: Please add native API support for Mailgun, Sendgrid, etc
Replies
2
Views
2K
Mike Fara
Mike Fara
wcbryant
  • Suggestion Suggestion
Implement Google's Perspective (moderation;machine learning post/user toxicity API)
Replies
0
Views
1K
wcbryant
wcbryant
Mr. Goodie2Shoes
Add-on Crowd-funding proposal for classifieds add-on.
3 4 5
Replies
82
Views
13K
Mr. Goodie2Shoes
Mr. Goodie2Shoes
Top Bottom