Is your forum being hammered by Bots?

Arty said:
Then you have something wrong with your server. Cloudflare shouldn't add that much.
Click to expand...

That's not my server, that's HIS server. There's nothing wrong with my server.

Test this: http://api.iconify.design/fa-home.svg?height=512 and change height to something random to make sure its not cached. Loads in about 100ms for me. Not cached, so goes through Cloudflare to my origin server.
Click to expand...

Loading a static resource is much different from loading from PHP.

In Cloudflare settings make sure you are connecting to origin server with HTTP, not HTTPS. That reduces page loading speed because when CF connects to your server it doesn't need to do handshake step.
Click to expand...

Loading via an origin server through HTTP negates the entire reason people use HTTPS. If the end user is connected to an HTTPS server, submits registration data and then its submitted in the clear via the origin, in my opinion, its a breach of trust to users who are led to believe their data isn't being sent over the Internet unencrypted.

CDNs make a lot of sense for static resources, or for non-static if the site is the type to attract DDOS attacks. But not for most dynamic resources.
 
My site seems to be fine as far as speed for me. After I added Cloudflare the speed seemed to increase for me personally. Even new threads load quicker.

The bots were my problem.
 
MySiteGuy said:
Loading a static resource is much different from loading from PHP.
Click to expand...
Its not a static resource. It is generated dynamically on origin server, not cached on Cloudflare.

I'm just pointing out that its not Cloudflare's fault. If page takes 800ms to load, without Cloudflare it would probably be 700-750ms.
 
Arty said:
Its not a static resource. It is generated dynamically on origin server, not cached on Cloudflare.

I'm just pointing out that its not Cloudflare's fault. If page takes 800ms to load, without Cloudflare it would probably be 700-750ms.
Click to expand...

The image file is dynamic, and not a fixed file on the server? Are you converting an original PNG, JPG or GIF to svg on the fly?

As to the overhead, Cloudflare loves to push that increasing Time To First Byte isn't a big deal, and have been called out on it. I trust the opinion of one of Google's Web Performance Engineers on the matter:
plus.google.com

New community features for Google Chat and an update on Currents

Note: This blog post outlines upcoming changes to Google Currents for Workspace users. For information on the previous deprecation of Googl...
plus.google.com plus.google.com

For years Google has used TTFB as part of their algorithm. I don't know for certain since they do not reveal all the details of how they rank, but it's reasonable to believe TTFB is even more important since they rolled out the mobile first index this year.

Properly set up with a CDN for static resources, and direct to the server for dynamic resources, a site will almost always be faster. Especially if a site is properly handling encrypted connections by having them on the origin server as well.
 
You are right, but that's a 6 years old post. Things have changed since then. They have changed a lot. CF is completely wrong in that blog, but they have improved their algorithms.
MySiteGuy said:
The image file is dynamic, and not a fixed file on the server? Are you converting an original PNG, JPG or GIF to svg on the fly?
Click to expand...
Yep, its a dynamic image generated by server script. Its a dynamic API script that generates SVG on fly and original source is not SVG. So it can be compared to PHP page. The difference is its not HTML, so there are no other external resource to take into account, but it allows to measure CF performance.
 
CloudFlare has a mass of firewall settings, custom rules, page rules. Many of which you see after clicking 'advanced'. After taking 30 minutes to adjust all settings, keep an eye on blocked bots and blocked members and keep tweaking. You may want to challenge all users from the US. If you do that, then after a few days the firewall log will have a list of all users that were challenged. Most likely this will expose the bots you need to block and what they are doing. This allows you to block their IP ranges and add page rules to block specific behaivor.

Unfortunately CF does not have xenforo specific settings.
 
MySiteGuy said:
Properly set up with a CDN for static resources, and direct to the server for dynamic resources, a site will almost always be faster. Especially if a site is properly handling encrypted connections by having them on the origin server as well.
Click to expand...
I'd say that pretty much depends on your audience. If your traffic is globally distributed, then yes - a CDN will most certainly improve things.

For us, approx. 80% of our traffic is from Germany (which makes sense as our forums are german with .de-domains), so unless a CDN has multiple edge locations in Germany there is not much to gain by using a CDN - in fact the overhead for yet another DNS-Lookup and TLS-Handshake can outweigh any benefit, especially if used for static resources only and taking HTTP/2 push into account.
 
Last edited:
I would not do that. There are some default settings/packages that hinder xf members. For example: packages meant for other software. IIRC QWASP: Joomla & PHPBB.

For your conveinence, here are the Rule sets:
removed
 
Last edited:
Kirby said:
I'd say that pretty much depends on your audience. If your traffic is globally distributed, then yes - a CDN will most certainly improve things.

For us, approx. 80% of our traffic is from Germany (which makes sense as our forums are german with .de-domains), so unless a CDN has multiple edge locations in Germany there is not much to gain by using a CDN - in fact the overhead for yet another DNS-Lookup and TLS-Handshake can outeigh any benefit, especially if used for static resources only and taking HTTP/2 push into account.
Click to expand...

I agree with this. I was speaking in general terms. For a smaller geographic location, even with edge locations it may not be worth it, since at most a CDN might gain a few tens of milliseconds over a direct connection, and that would be negated by DNS lookup, resources blocking rendering while the browser waits on them, etc.
 
So I would like to add my input to this thread. After having about 80 bots constantly being on my site at all time, I set out to see what I could do about it. I really did not want to use my robots.txt file since some bad bots will ignore it anyways. After clicking on my 'Current members' tab and checking out who was visiting most were coming from OVH network. Now I can't find a user agent for this so I had to figure out what to do. Since I use Cloudflare I went to Firewall then to IP access Rules selected FR for France for the country (because that is where the OVH bots comes from) and selected 'Challenge', just in case someone from France wants to visit my site it won't totally block them. Once applied in a few minutes they started dropping like flies. But I was left with another 40 bots of Semrush. So it was time to take care of them as well. I went under Cloudflare once again and Firewall. This time I created a Firewall rule and called it "Bad Bots". Then added the bad bots I wanted to get rid of like in the image posted below. Low and behold I killed them all. You can add other bad bots as well. I hope this helps someone in the future with the same problem. 🤓

195219
 
You must log in or register to reply here.

Similar threads

A
Lack of Google bots
Replies
2
Views
634
Nicolas FR
N
DiscussionBucks
Forum Time?
2
Replies
21
Views
2K
bzcomputers
bzcomputers
P
Problem with SEO, indexing and getting google to crawl pages
Replies
13
Views
2K
Miri
Miri
A
Extremely annoying member
2 3
Replies
58
Views
4K
Suzanne O
Suzanne O
Vercingetorix
Activity Summary emails are the digital equivalent of junk mail.
Replies
3
Views
1K
Vercingetorix
Vercingetorix
Top Bottom