[TAC] Any Spam API

[TAC] Any Spam API 1.0.9

No permission to download

tenants

Well-known member
This is included in Both
i) Free (Branded) Tac Anti Spam Collection
ii) Paid (unbranded) Tac Anti Spam Collection

For the API usage, please consider donating to the owners of the APIs that you use (not my self, I have only provided you the ability to add your API logic, the API's do all of the work ;) )

tenants submitted a new resource:

AnyApi - Use Any Api On Registration (anti spam, anti fraud, any api thing) (version 1.0.2b) - Anti-spam... Anti Anything... define APIs you want, they way you want to use them

Quite a few times, I've come across users that want APIs added to an add-on. In particular, certain APIs added to FoolBotHoneyPot

This add-on allows you to define any API, build up the rules applied to the API response , and define the logic of the rules

So now, if there is an API you wish to use on registration that isn't available with any add-on, you can simply use AnyApi and define it your self

This add-on comes preconfigured will the following APIs
  • Project Honey Pot Http:BL...

Read more about this resource...
 
AnyApi Rules

For the most part, you wont need to play with these, you can just tick the default APIs on/off, or fiddle with the values a little.
However, if you want to add your own api definitions for a new API, then the below describes how the API rules and logic can be used

The rules section can be defined by you (the forum admin)

The following has been added to make your life easier:

On the left hand side, this is always the response, or response array
For instance, if StopForumSpam returned:
{"success":1,"email":{"lastseen":"2012-12-20 01:31:08","frequency":3064,"appears":1,"confidence":95.85}}
Then the value of lastseen would be {$response[1]} since indexes start from 0, and last seen is the second index

So for the API rules, the left hand side might be something like
{$response[1]}

We can then compare this values with operators, the following operators are allowed:

= (For numeric only. Equals, return true if the left hand side equals the right hand side)
> (For numeric only. Greater than, return true if the left hand side greater the right hand side)
< (For numeric only. Less than, return true if the left hand side less the right hand side)
contains (For strings only, this returns true if the left hand side contains the right hand side value)
!contains (For strings only, this returns false if the left hands ide contains the right hand side value)

So, for something like Spam Busted, we can use:
Rule1: {$response} contains 'Yes'
Which means, if the reponse contains Yes, return True.
We can then define in the logic section
Prevent Registration If: (Rule1)
... So, if the response from Spam Busted conatains 'Yes', then prevent registration... it's that simple


A second example, for something like StopForumSpam, we can use:
Rule4: {$response['ip']['lastseen']} > $timenow - 1814400 // 3 weeks
Which means, if the response for the index['ip']['lastseen'] is smaller (further in the past) than 3 weeks ago, return false (the IP had to have been seen within the last 3 weeks for this to return true)
We can then use this in part of out logic
Prevent Registration If: (Rule1 AND Rule2 AND Rule3) OR (Rule4 AND Rule5 AND Rule6)

For more examples, just look through the default values in the post below

AnyApi Logic
The logic section combines your rules in a fashion that you wish the APIs to prevent registration

The logic section can contain
Operators: OR AND
Parantheses: (, )
Your Rules: Rule1, Rule2, Rule3 .. etc

Please note, you will have to define the rules and save the API before you can add them to your logic section
 

Default settings for default APIs
Settings For StopForumSpam
API Name: Stop Forum Spam
API URL: http://www.stopforumspam.com/api?ip={$ip}&email={$email}&f=json
API Key:
Returned Data Type:
JSON
API Rules
Rule1:
{$response['email']['lastseen']} > 0 // bot emails banned forever, you dont need this line
Rule2: {$response['email']['confidence']} > 90.01 // more than 90.01 % confidence
Rule3: {$response['email']['appears']} > 0 // appears once or more, you dont need this line
Rule4: {$response['ip']['lastseen']} > $timenow - 1814400 // 3 weeks
Rule5: {$response['ip']['confidence']} > 90.01 // more than 90.01 % confidence
Rule6: {$response['ip']['appears']} > 0 // appears once or more, you dont need this line
Logic Rules
Prevent Registration If: (Rule1 AND Rule2 AND Rule3) OR (Rule4 AND Rule5 AND Rule6)​


Settings For FSpamlist
API Name: FSpamlist
API URL: http://www.fspamlist.com/api.php?key={$key}&spammer={$email},{$ip}&json
API Key: {YOU MUST REGISTER AND GET THIS FROM HERE!}
Returned Data Type:
JSON
API Rules
Rule1:
{$response[0]['isspammer']} = true // email
Rule2: {$response[1]['isspammer']} = true // ip
Rule3: {$response[1]['lastseen']} > $timenow - 1814400 // ip less than 3 weeks old
Logic Rules
Prevent Registration If:
(Rule1) OR (Rule2 AND Rule3)


Settings For Bot Scout
API Name: Bot Scout
API URL: http://botscout.com/test/?multi&mail={$email}&ip={$ip}
API Key:
Returned Data Type:
Plain Text
API Rules
Rule1:
{$response} contains 'Y|MULTI'
Rule2: {$response} !contains 'MAIL|0'
Rule3: {$response} !contains 'IP|0'
Logic Rules
Prevent Registration If: (Rule1) AND (Rule2 OR Rule3)

Settings For Spam Busted
API Name: Spam Busted
API URL: http://www.spambusted.com/api.php?email={$email}&ip={$ip}
API Key:
Returned Data Type:
Plain Text
API Rules
Rule1: {$response} contains 'Yes'
Logic Rules
Prevent Registration If: (Rule1)

Settings Project Honey Pot Http:BL
API Name: Project Honey Pot Http:BL
API URL: {$key}.{$reverse_octet_ip}.dnsbl.httpbl.org
API Key: {YOU MUST REGISTER AND GET THIS FROM HERE!}
Returned Data Type:
Octet
API Rules
Rule1:
{$response[0]} = 127 // all Http:BL start with 127 for the 1st octet
Rule2: {$response[1]} < 21 // last seen less than 21 days ago
Rule3: {$response[2]} > 20 // must have a threat score of over 20
Logic Rules
Prevent Registration If: (Rule1 AND Rule2 AND Rule3)


Settings Block Disposable Email
API Name: Block Disposable Email
API URL: http://check.block-disposable-email.com/easyapi/json/{$key}/{$emaildomain}
API Key: {YOU MUST REGISTER AND GET THIS FROM HERE!}
Returned Data Type:
JSON
API Rules
Rule1:
{$response['domain_status']} = 'block'
Logic Rules
Prevent Registration If:
(Rule1)
 
Last edited:
Installed this yesterday and did some testing. It appears that it's blocking ALL registrations.

Checked the error log, and it shows:

XenForo_Exception: Cannot load class using XFCP. Load the class using the correct loader first. - library/XenForo/Autoloader.php:108

I can PM you the backtrace if necessary.
 
oh... pants!

Yes, it's due to moving all of the folders around (The Tac folder didnt exists before), I'll fix this now

Strange, I registered during testing... I wonder if it's just using it as a stand alone that the bug exists

Yes, I just tested the versions from here:
http://xenforo.com/community/resources/tac-tenants-anti-spam-collection-anti-spam-free-version.1474/
http://xenforo.com/community/resour...ollection-anti-spam-complete-collection.1469/

and they work (they both did the API checks correctly, and both allowed registration)

Can you PM me the entire error
 
oh... I know why, when FBHP is used in combination with AnyApi, it uses it's versions of the public register.. so bypasses the error, that means the error is in and only in the registration php file.. and indeed it is

The class name is:
AnyApi_ControllerPublic_Register extends XFCP_AnyApi_ControllerPublic_Register
But should be
Tac_AnyApi_ControllerPublic_Register extends XFCP_Tac_AnyApi_ControllerPublic_Register

Impossible to reproduce, unless it's used as a standalone (without FBHP)

A simple fix, I'll re-upload this in 5 minutes.

Indecently, why not use this with FoolBotHoneyPot (they're available in either of the 2 collections in the links above)

But, I'll fix the stand-alone now
 
How about extending AnyApi to allow applying to forum posts, as well as registrations?

Several reasons.

1) I've found with a lot of the services that AnyApi uses, given IP's / emails haven't yet made it in to those services before they register on my site. It would nice to be able to catch posts from those folk, when their details get added to the service(s).

2) On one of the sites I'm having spam issues with, I use a bridge for handling registration, so my XF doesn't actually do registration, that's handled in Joomla, and the user is inserted into XF through the back door, by direct table manipulation, without going through any of the XF registration API. So again, I'd like to be able to prevent posts from AnyApi flagged users.

-- hugh
 
I can have a look at it, but it's not high on my priority right now

It's set up to use the API for registration, this is where pretty much anyone that needs an API will use it.
I generally don't like the idea of using APIs for posting (at the time of posting or even after), it's takes up a lot more resources, and would slow posting down.

I try to avoid anything that can affect performance, including keeping the query count down. Depending on the API, calling an API at the time of posting could take 1-30 seconds (which would really be unacceptable)

There aren't as many API that allow you to do this (there would be far more requests), so it would be designed for just a handful of APIs.

I could call the API as a cron entry, but then there would be a period that this spam would exists (and it could exists for hours), so multiple cron entries are not going to be suitable

So, if I did this, I would need to
  • Create a 2nd API rule area for post registration in the Admin control Panel
  • Do something that may slow the system down if used (since API requests are time consuming)
  • re-design a fair chunck of it, for which very few people would need
I'm not saying I wont add this as an option, but I have quite a few other things that are taking priority.
 
Thanks for the response, and for giving this consideration.

I'm interested enough to look at putting something together, if you'd be interested in any contributed code. Just FYI, I've been fighting the spam fight for several decades, first on Usenet, then as 'postmaster' and network admin at a regional ISP. When I got out of that game, about 4 years ago, our small ISP was handling over 20 milion SMTP delivery attempts per day, of which we were rejecting a fraction under 99% as spam. I hate spam. I like killing spam. It makes me happy. :)

I 100% agree about the performance issues, and keeping processing to a minimum during posting, especially if it involves calls to remote 3rd party API's. But this isn't something I'd imagine being applied site wide. I'm only wanting it for my one forum which needs to be open for guest access, running as the last line of spam defense. StopHumanSpam will get most of it, without AnyApi getting involved. And I'd probably code in time limits on the remote API access during posting, so if all the designated API's haven't responded within that timeout, just mark the post for moderation and move on.

I'd also like it as a stopgap for when I get some new spam that manages to slip through, like the "bbcode in URL tag" thing we just fixed. I find there's usually something once or a month or so which manages to get through, one way or another, otherwise known as the Spam Du Jour. And it'd be nice to have a stopgap measure like AnyApi I could enable on all affected forums while I track and fix the SdJ, as I'm fairly sure at least one of the RBL's out there would catch most of them.

I also like your idea of a cron job, although it might be better to just have a simple 'toolkit' backend interface for manually kicking off AnyApi batch runs. It'd be nice to be able to specify (say) a node / forum and a date range, and have AnyApi run through those posts, applying the posting rules retroactively, to help clean up after an SdJ outbreak, once we're out of Zero Day and the perps are in the RBL's. Likewise, it might be good to be able to do the same thing to the user database, specifying a date range and applying the registration rules, to help fix any outbreaks of Registration Hack Du Jour.

This is obviously all just my $0.02, but it just seems like a natural progression for AnyApi. Essentially making AnyApi an API itself. Your code is already structured so it would be very easy to extract the existing registration controller, so AnyApi itself just becomes the library and overall settings for the remote API's. Then things like AnyApiRegistration, AnyApiPost, AnyApiToolkit, etc. could be split out in to separate little addons that just add the controller shim and some application specific settings.

If you took that approach a little further, it would be awesome to have your complete range of spam products built the same way, as libraries with a consistent internal API, which can be called through an Uber API. So someone can just call UberAPI::isSpam($data, $options), where $options includes a list of the spam checks they want done (human, country, anyapi, etc), and $data is a 'superset' data structure with things like ip, content, username, email, etc. that covers everything the various spam libraries need by way of data. You seem to be working towards "bundling" your products anyway, and again this just feels like part of a natural progression for your work.

Anway, I've ranted on for long enough. As I said, I'd be happy to work on building an AnyApiPost as a separate little addon, with absolute minimal changes to your existing code, so all it really is, is a shim with a small controller and a few settings which calls the AnyApi model methods. If you like it, all well and good. If not, it's something I would still use on my sites. And if all you folded in to your code was any minimal changes needed to allow other addons to drive AnyApi, I'd be a happy man.

-- hugh
 
BTW, apologies if I seem a little overbearing or aggressive. I'm just VERY enthusiastic about killing spam. It's basically the next best thing to killing the spammers, without the legal repercussions.

Have I mentioned I don't like spam?

-- hugh
 
I'm just VERY enthusiastic about killing spam. It's basically the next best thing to killing the spammers, without the legal repercussions. Have I mentioned I don't like spam? -- hugh
:LOL: I thought about quoting you in my sig

Feel free to extend AnyApi to suite your needs.... It's a free plugin, and I don't mind extension to this plugin being added and even released (as long as they are kept free). But until I have more time on my hands, AnyApi it's self can only be applied to registration...

There is not a large enough demand for anything else right now. Since, FBHP kills 100% of bots on registration, StopHumanSpam Stops any Human spammers, and then the core uses a couple of mechanism. This was originally only released for those that wanted extra anti-spam API's with FBHP to stop the human spammers (before StopHumanSpam was released). Right now there isn't that much demand for AnyApi (in a couple of months, there might be ... once the xf core mechanisms are no longer working)

I'm also frantically writing personal plugins over the next few months, so I'm really only tweaking existing add-on until I'm finished with my own project
 
Thanks for the response.

Right now there isn't that much demand for AnyApi (in a couple of months, there might be ... once the xf core mechanisms are no longer working)

I guess that's kind of my point. I don't have much of a need for AnyApi on forum posts ... right now ... but then there's the "Spam Du Jour" phenomenum, where you wake up one morning to your forums all stunk up with spam, because something broke, or the spammers found another vector, whatever. At which point, it would be great to be able to flip the switch and have AnyApi start vetting posts, until the new hole can be plugged in a more efficient way.

So I see it as a potentially very useful item in my Spam Killing toolkit, that I may not use very often, but like a strap wrench or a bearing puller in my real world tool kit ... when I do need it, it's a life saver.

Anyway ... I'll chip away at my ideas. If I find I need a couple of small changes to AnyApi to be able to drive the core of it from a separate "AnyApiPost" addon, I'll ask nicely if you wouldn't mind folding them in to your code in a future release.

And once again, thanks for your work on these addons. Much appreciated.

-- hugh
 
there is something seriously wrong with Stop Forum Spam API with any api

i ran many tests even with the default xenforo spam settings off i noticed when Stop Forum Spam is on on any api no one can make and account even using there real names or full names real emails. i even tested with using my real name and every thing valid real emails and still i could not.

but when i turned it off the Stop Forum Spam then it worked
 
StopForumSpam doesn't need to be used via AnyAPI any more, since it's used in the core.

However, I still use StopForumSpam on 2 of my forums via AnyAPI, since it provides me additional information (I do not have issues when people try to register)

Could it be that you have StopForumSpam turned on via both core and AnyAPI?

Also, check your anyApi Logs:
admin.php?anyapi/logs

And click on the row to see what and why you are getting a response from AnyAPI, it should say something like this:

For a blocked user:

Log Info​
Stop Forum Spam Blocked User From Registering
Today at 9:12 AM
generated by username attempt: TaibiaPar
generated by email attempt: dolphinhxak%40gmail.com
IP Address: 190.207.171.171

1) API Request String
http://www.stopforumspam.com/api?ip=***********&email=************&f=json

2) API Response​
{"success":1,"email":{"lastseen":"2013-12-18 09:05:41","frequency":134,"appears":1,"confidence":96.75},"ip":{"lastseen":"2013-12-18 08:10:28","frequency":43,"appears":1,"confidence":90.53}}

3) API Rules​
Rule1 => {$response['email']['lastseen']} > 0 // bot emails banned forever, you dont need this line
Rule2 => {$response['email']['confidence']} > 90.01 // more than 90.01 % confidence
Rule3 => {$response['email']['appears']} > 0 // appears once or more, you dont need this line
Rule4 => {$response['ip']['lastseen']} > $timenow - 1814400 // 3 weeks
Rule5 => {$response['ip']['confidence']} > 90.01 // more than 90.01 % confidence
Rule6 => {$response['ip']['appears']} > 0 // appears once or more, you dont need this line

4) API Logic​
(Rule1 AND Rule2 AND Rule3) OR (Rule4 AND Rule5 AND Rule6)

5) Evaluated API Logic​
(true && true && true) || (true && true && true) = true // registration blocked

You can see what the rules are above, and that in this case rules 1 - 6 all evaluate to true. So
(true && true && true) || (true && true && true) = true // registration blocked

This log information tells you:
1) Exactly what the API request string is
2) The exact API response
3) The rules (logic) that are then applied to that response
4) How the boolean responses are combined
5) The final outcome of the boolean: true // registration blocked

With this level of information, you can tell exactly why / where users are stopped. You can go in and also update the logic, here:
admin.php?anyapi/logic

(but you shouldn't need to)

Maybe you have played around with the logic already, if so, the default settings can be found here:
http://xenforo.com/community/thread...m-anti-fraud-any-api-thing.45358/#post-486798
 
Last edited:
If you want the API "Block Disposable Email" added, you can either follow the example here and add it to the list of your APIs (after upgrading to AnyApi 1.0.4):
http://xenforo.com/community/thread...m-anti-fraud-any-api-thing.45358/#post-486798

Or you can un-install and re-install this add-on, and it will be added as one of the default examples

... If you do un-install, don't forget to copy your API keys somewhere before un-istalling

"Block Disposable Email" requires an API key, so you will need to register (this was a requested API)

If you know a similar free service, you should be able to set up AnyApi to use it, if not, let me know about the API and I will tell you what AnyApi settings you need to add
 
Last edited:
Great update. Xenforo could consider to add the same api's that arn't in it yet to fight spam.
 
Back
Top Bottom