Need Help: Unusually High Traffic on Niche Forum

7

7emper5i

Member
Hello everyone,

I'm reaching out to seek some advice regarding an issue I've been facing with my forum recently. My forum, which is a niche motorcycle forum, has been experiencing an unusually high amount of traffic that I can't seem to trace back to any specific cause.

Here are some details:

  • The forum has been running smoothly for about 20 years with an average of 200 users per day.
  • Up until September 2021, everything was stable and the traffic was normal.
  • Since then, the traffic has increased dramatically. In the last 4 days alone, the forum has generated 34.40 GB of traffic.
  • This is not an image or video-heavy forum. While users do occasionally post pictures and videos, it's primarily a text-based forum.
  • I've tried using analytics software like Matomo, but it hasn't provided any useful insights.

Has anyone here experienced something similar? Are there any tools or features within Xenforo that could help me get more detailed information on what's causing this spike in traffic?

Any suggestions or advice would be greatly appreciated!

Thank you in advance!

Best regards!

Bildschirmfoto 2024-09-04 um 23.29.13.webpBildschirmfoto 2024-09-02 um 21.43.23.webpBildschirmfoto 2024-08-31 um 13.07.35.webp
 
Sort by date Sort by votes
It's almost guaranteed to be a bot of some kind. You can do tracking at the webserver level with your access.log --- or go behind Cloudflare and let them analyze the traffic and you can decide what to block.

Some bots respect robots.txt and some do not. Those that do not you'll be able to block with your webserver or via Cloudflare.
 
Upvote 0 Downvote
i know that i was having a lot of traffic on my new site from ai bots.
the site i am staff on also had the same issue around the same time.
tracy blocked most of them at cloudflare.
 
Upvote 0 Downvote
Adding the following directives to your .htaccess files allows you to control which bots are denied and allowed to access your forum.

pic001.webp

Code: 
    #   Deny and Allow bots by User-Agent
    SetEnvIfNoCase User-Agent "bot|crawler|fetcher|headlesschrome|inspect|search|spider" bad_bot
    SetEnvIfNoCase User-Agent "duckduckgo|googlebot|yahoo" good_bot
    Deny from env=bad_bot
    Allow from env=good_bot
 
Upvote 0 Downvote
If you would like to add additional bad bots, you add them on this line. Keep in mind that by having "bot" already entered, that will cover any bot with the work "bot" in the user agent.

SetEnvIfNoCase User-Agent "bot|crawler|fetcher|headlesschrome|inspect" bad_bot

Just add the | symbol followed by the name of the bad bot.

If you would like to add good bots, you add them on this line.

SetEnvIfNoCase User-Agent "bingbot|duckduckgo|googlebot|yahoo" good_bot

Just add the | symbol followed by the name of the good bot. Note that I prefer to remove the bingbot as I consider it a bad bot.
 
Upvote 0 Downvote
To deny IP addresses, follow this example:

Code: 
#   Deny and Allow bots by User-Agent
SetEnvIfNoCase User-Agent "bot|crawler|fetcher|headlesschrome|inspect|search|spider" bad_bot
SetEnvIfNoCase User-Agent "bingbot|duckduckgo|googlebot|yahoo" good_bot
Deny from env=bad_bot
Deny from 47.76.
Allow from env=good_bot

Notice how only the first two octets are used followed by a period. In this example 47.76. will deny the following range:

47.76.0.0 to 47.76.255.255.
 
Upvote 0 Downvote
You must log in or register to reply here.
Back
Top Bottom