Learnings: Identifying and getting rid of unwanted traffic

[ENF]

Why? I don't have an issue with spam oder spammer's signing up successfully on my forum. These kind of annoyances is handled 100% successfully by the spaminator series of @Ozzy47 on my forums, as I lined out earlier. So what exactly would Xon's add on be good for in the topic of this thread?

To be fair, it's our primary tool to filter out various ASN's upon registration. We found that ASN blocks are not 100% in some cases where data is (or is not) updated at the IP level. However, using the ASN configuration in Xon's Addon, we catch 100% of the ASN's we have on our absolutely-do-not-want list. We update data as we find holes, but Xon's tool(s) are the icing on the cake to prevent anyone from registering from a blocked ASN. Anyway, we elected to add his tool to the arsenal because it employed ASN-level exams and rules.

We have blocks at the OS/Network Layer, Border (ingest) & App layers (xf and other business apps where needed). It's basically set at the level of severity. The major issue we came across was people who were traveling and using temp sim or esims... these nonsensical providers route those temp cards through various dirty connections on some of the blocked ASN's. (I know, I got myself blocked by using a temp sim while in Australia and a few other countries...)

 

[smallwheels]

Xon's tool(s) are the icing on the cake

Absolutely, at many occasions.

Anyway, we elected to add his tool to the arsenal because it employed ASN-level exams and rules.

I may do that as well, but only later and as a bonus upon the multiaccount feature that I'm somewhat after.

The major issue we came across was people who were traveling and using temp sim or esims...

Traveling forum members are the main reason why I am not even more rigid in blocking areas.

 

[smallwheels]

I may do that as well, but only later and as a bonus upon the multiaccount feature tha

Well, later was early. I got curious and bought and installed the add on. Impressive. Looks like a swiss army knife.

 

[smallwheels]

Here's a list of the top ASNs that were blocked by my firewall settings - the vast majority of this blocked traffic did indeed come from that Microsoft Datacenter ASN:

As I dive deeper and deeper into the unknown the more noise has been blocked successfully more and more of those come to the surface thad managed to hide successfully until now. Being aware of your screenshot I was curious if I would find the same players - and I did for the most part: There is a little bit of Hetzner in my logs, but not much. However, Huawei Cloud is a big and very nasty one. Someone from there seems to slowly but steadily suck down my forum contents, using a huge range of different IPs and as huge set of different useragents, claiming to be normal browsers:


Whoever does this is not in a hurry and rather tries to hide himself by acting very slowly, distributed and hiding the fact that a bot is used. So next job will be to block out the Huawei cloud IPs at scale as good as possible - sounds like a lot of manual work (the more, as the whois database is inaccurate and missing to display wider parts of these IPs correctly. Arin says they would have been delegated to ripe and ripe says "we don't know a thing", so it is manual work involved as bigger parts of the IP ranges have fallen in kind of a black hole.

 

[smallwheels]

Time for the next update.

There is a little bit of Hetzner in my logs, but not much. However, Huawei Cloud is a big and very nasty one.

While Huawei keeps being a steady annoyance the ongoing blocking efforts clearly show results - less and less requests come through. However: Hetzner has changed dramtically. There has been a continuous stream of requests coming from their cloud, but to a somewhat limited amount.
On Monday and Tuesday this week this changed: I discovered a massive scraping attempt with around 35.000 requests on Monday and 105.000 requests on Tuesday coming from Hetzner Cloud, using more than 480 different IP addresses from a huge amount of ranges and hiding behind an immense amount of different generic user agents.

As a result, a huge range of IPs from Hetzner have now been blocked - the attempts are continuing, but slower and unsuccessful. To be able to handle that effectively and more efficiently I was forced to revitalize my shell scripting abilities - sed, awk and regex have been pulled back to life from the darker and hidden areas of my brain where forgotten knowledge is preserved. A very special area of fun. In the end I have now come one step closer to a semiautomatic workflow for blocking bad actors.

Out of curiosity and due to the massive amount of malice traffic coming from a provider of my own country I also contacted the abuse desk of Hetzner to see if and how they will react - atm I am still waiting for an answer.

Overall, my manual blocking approach is clearly way too time consuming to follow it forever. For the moment it is still interesting enough in terms of learning to continue with it.

 

[thumped]

Lessons.

 

[smallwheels]

Out of curiosity and due to the massive amount of malice traffic coming from a provider of my own country I also contacted the abuse desk of Hetzner to see if and how they will react - atm I am still waiting for an answer.

Update on this: I got a reply a couple of days later - it was not a DOS alike incident, so while not brilliant this is ok. Unfortunately, the content was just the opposite. Despite having been provided with all necessary information including the logfiles in txt-format they asked me to provide the logfiles - interestingly enough they did event send them back to me as a quote in their answer, as whatever tool they may use is utterly broken - the whole quoting inside their mail was a total mess. They asked for the logs to be provided not as attachment but as txt inline in the mail - a bit of a weird approach given that we are talking about 46MB of txt-logfiles of the requests of their customer which, zipped down are just 3MB. So an abuse desk that cannot deal with txt files and is unable to send proper mails and unable to use proper quoting. Interesting.

They did not read my abuse report properly as they were stating I had complained about one IP (whereas I had in fact complained about a distibuted scraping attempt from more than 480 IPs from their network) and were not willing to tell me the identity of their customer for reasons of data protection laws (clearly an artificially made up statement w/o foundation, given the evidence I had sent them) but were asking if it was ok if they would give my identity and my logfiles to their customer. Else they would not be able to do anything about the incident they said.

So basically a somewhat disturbing answer and in fact the most unprofessional answer I ever got from an abuse desk in decades. I replied to them but did not get any answer back.

As a consequence, while I do not consider Hetzner to be a rogue provider or a black-hat provider they are at least a grey hat, given their behaviour. Probably not purposefully but due to broken processes, a bad attitude and incompetent staff. Their abuse desk acts unprofessionally and irresponsibly, contacting them seems simply a waste of time. Blocking Hetzner's networks completely from accessing your forums is faster and easier and more effective on top of that - and well deserved.

The scraping attempts from their networks continue, but to a smaller, slowed down extent. I don't care as they are provided with a 403 now.

 

[smallwheels]

Another two months later there are again a lot of learnings to be shared. Unfortunately I'm lacking the the time for a proper write up atm, so this will follow later. For the moment ust one thing: Recently there have been a lot of reports about a sudden apperance of loads of guest users on various forums as well as massive amount of spam registrations.

Thread 'Spambots on the rise.'

Mar 5, 2025

I'm getting bombarded by spambots, they seem to be 'legit' users but if you look further you can tell they're spambots. I have a question at registration and they all use the same nearly identical answer. They also use a slew of weird email addresses.

What can be done to prevent this going forward? I have a slew of email addresses/IP addresses banned, I have bot protection turned on in cloudflare.....

• DarkGizmo
• Replies: 15
• Forum: Forum management

Thread 'Spam is Killing my Site'

Jan 8, 2022

I've been getting a ton of spam lately and I'm not sure how to mass kill all of it? I went to the Spam Management tool and tried some things, but I have no clue how to prevent this stuff and also how to run it in a batch process. My host provider keeps shutting down my site due to this spam. My site is pretty dead, but I use it a lot for another reason and it needs to be kept running.

Can anyone help?

Madden NFL Football

Discuss Madden NFL Football thread topics in these forums at the SimHeads.

www.simheads.com

Here's a screenshot of some spam examples I'm receiving...

• kingjavo
• Replies: 16
• Forum: Forum management

A

Thread 'Spam management'

Jul 15, 2025

While those features used to work acceptably well, they no longer work on my forum. I'm getting buried in bot mess. Their postings don't see the light of day, but it sure makes a mess on our forum for the moderators.

Any add ons people like? My bot registrations are pretty huge compared to actual traffic.

• Aloft
• Replies: 5
• Forum: Forum management

E

XF 2.2 Thread 'Too many Guests'

Jan 8, 2025

Hi all, happy 2025!

Our forum is small and never has over 20 registered users logged in, but recently it's been getting stuck due to 3000+ "Guests"/unregistered visitors arriving at the same time, which eventually triggers this error:

• MySQL query error [1114]: The table 'xf_session_activity' is full

and the forum content becomes unavailable and displays a standard message on the front end about a problem generating content. Then it seems to resolve itself and the content comes back up, but this is happening more and more often - a few times a day now. I suspect content scrapers...

• emaw
• Replies: 23
• Forum: Troubleshooting and problems

O

XF 2.2 Thread 'Huge amount of guests'

Jul 7, 2025

Hello
We have an audio forum. We normally see 3K online as guests, and 700 logged in.

The last few weeks we started seeing 4.5K as guest, then 6K, and today we are at 14.5K guests.

If I click the 'members online' and then the Robots page, we have 18 page, not a problem. If I click 'Guests' I see 645 pages!

Has anyone seen a snap or DDS attack coming up as Guest online. The threads these Guests are viewing is genuine threads, and different threads, so not an obvious script / attack.

Any help would be great.

• oneobgynmd
• Replies: 6
• Forum: Troubleshooting and problems

etc.

I did not see any of those on my forum - unfortunately I cannot tell if this is the result of my efforts or ust because the bots ignored my forum. However: As a small and fast aid for those who are affected by spambots I grabbed the IPs of all spambots that were caught on my forum by the registration and login spaminator by @Ozzy47. I installed them eight months ago, not a single successful attempt since then and not a single false positive. The IPs are from all over the world and consist of VPN endpoints, dedicated servers used for spamming, open relays, hacked servers and some dialups. About 1070 IPs in total that made more than 27.000 logged registration attemps and more than 12.000 login attempts on my forum. In reality way more b/c I have blocked a lot of them consistently over the last months after they appeared, w/o that it would be many more b/c many IPs turned out to be notorious and to return on a regular basis.

I prepared two text files with those IPs, one is already prepared to copy the contents into your .htaccess to block all of those IPs. The other ist a simple list of the same IPs that you may use for whatever you want to use it to block the bots. I.e. I did not aggregate the IPs into network blocks for those lists despite some of them coming from the same network.

However: You can block them w/o any danger or harm - these are proven spambots. Hopefully it helps those a bit suffering from them. The list will NOT solve the problem, as it is obviously not comprehensive. Blocking those IPs will also help against scraping to a (small) degree as it turned out that some not only try to register but also scrape - possibly their intention is not spamming but scraping contents behind the registration wall.

Have fun and good luck.