XF 1.3 ROBOTS

Alhidayah

Member
Hi...

My website is showing over 170 robots online due to which i think server cpu usage reaches high whoch then stops website from opening up and gives error saying the website cannot be displayed, database error.

I have used a plugin in called robots, configured it but its still showing over 170 robots.

Is there anyway i can stop this?

Thank You.
 

Attachments

  • Screenshot_20201214-185950_Samsung Internet.webp
    Screenshot_20201214-185950_Samsung Internet.webp
    32.1 KB · Views: 15
Which addon is that? Does the addon actually block the bots? or does it just identify them?

Assuming it just identifies them, you still need to block the worst offenders (some bots are very aggressive and they can eat up bandwidth) using robots.txt exclusions for those that obey robots.txt and .htaccess for those that don't.
 
Which addon is that? Does the addon actually block the bots? or does it just identify them?

Assuming it just identifies them, you still need to block the worst offenders (some bots are very aggressive and they can eat up bandwidth) using robots.txt exclusions for those that obey robots.txt and .htaccess for those that don't.
Hi.

This is the addon

Yes i think its just identify them and not block them.

Yes you are right thats wots happening with my bandwidth, as you can see the attachment on my post theres not alot of members but when i visit my hosting control panel the cpu usage rech sky high and when it reacjes to 100 percent the error show up on website and it dosnt open up.

I dont no how to use robots.txt if you can hrlp me get thru it, i woud appreciate it as i am not so good woth database.

Thank you.
 

Attachments

  • Screenshot_20201215-220031_Samsung Internet.webp
    Screenshot_20201215-220031_Samsung Internet.webp
    6.7 KB · Views: 7
Last edited:
Hi.

This is the addon

Yes i think its just identify them and not block them.

Yes you are right thats wots happening with my bandwidth, as you can see the attachment on my post theres not alot of members but when i visit my hosting control panel the cpu usage rech sky high and when it reacjes to 100 percent the error show up on website and it dosnt open up.

I dont no how to use robots.txt if you can hrlp me get thru it, i woud appreciate it as i am not so good woth database.

Thank you.

Can’t be that addon, that is for XF2 not XF1
 
According to the description of that addon, it does block bots, although you'd have to check with @AndyB as to how it does that.

Disallows robots from indexing your site unless explicitly allowed.

But not all bots respect robots.txt. If your forum is hosted on a linux/apache site you can block them in your .htaccess file once you know the user-agent string for the bot. However, it's easy to kill your entire site if you mess up editing .htaccess. Even using the wrong editor can do that. You would be better off asking someone to do it for you.
 
What is the error you are receiving? Server optimization is recommended. It could be a PHP setting like max connections or you could use a caching solution etc. Because, what happens if you block 170 bots but get 170 real members? Same!
 
Back
Top Bottom