ElgrandOC
Member
Guys,
I had my website account suspended due to excessive resource usage yesterday. I've just been in and checked my cPanel account today and CPU usage is constantly over 80% and bandwidth consumed is 22.6gb in 6 days (yes, GB not MB).
When I've looked into it, it's all been consumed by Googlebot.
I've been into my "Recent Visitors" log to see what IP addresses have been accessing what web pages on my server, and I'd say 95% of the hits are like this one "/index.php?find-new/1915589/posts&page=6"
Can I use robots.txt to disallow /index.php?find-new/ ???
I've had to block the IP of all Googlebots for now to stop it consuming my bandwidth or earning me another account suspension but I don't want to keep Google blocked eternally.
If the above would not work, what can I do to stop it crawling the "new posts" URL as this is obviously going to be constantly changing.
I had my website account suspended due to excessive resource usage yesterday. I've just been in and checked my cPanel account today and CPU usage is constantly over 80% and bandwidth consumed is 22.6gb in 6 days (yes, GB not MB).
When I've looked into it, it's all been consumed by Googlebot.
I've been into my "Recent Visitors" log to see what IP addresses have been accessing what web pages on my server, and I'd say 95% of the hits are like this one "/index.php?find-new/1915589/posts&page=6"
Can I use robots.txt to disallow /index.php?find-new/ ???
I've had to block the IP of all Googlebots for now to stop it consuming my bandwidth or earning me another account suspension but I don't want to keep Google blocked eternally.
If the above would not work, what can I do to stop it crawling the "new posts" URL as this is obviously going to be constantly changing.