VisEntities
Active member
Noticed a sudden spike of 400 guests all at once, should I be worried, or is this just normal traffic fluctuation? Could this be some kind of attack?
Unusual Traffic Spike
- Thread starter VisEntities
- Start date
VisEntities
Active member
Is that something like Google, or is it a human crawling my site for example?
chillibear
Well-known member
Odds are more like google type system. FWIW we've seen a huge uptick over various sites from AI crawlers over the last year or so, seems everyone want data to train their models on and of course forum conversations make for good pickings! They tend to be a bit more aggressive than the search engine bots - I'm guessing the former have managed to find a middle-ground between doing their thing and being blocked.
Tail your web logs and see who is busy viewing the site!
Tail your web logs and see who is busy viewing the site!
chillibear
Well-known member
I'm always a little reluctant to go and point people to their underlying server and command lines if they are less than familiar with it. So caveat emptor and so forth.
However if you're not familiar with whatever operating system (odds are some *NIX flavour or a *NIX clone like Linux) you are running your forum on then even knowing probably wouldn't help you that much. I mean I'd look and pull out some addresses maybe check where they are being advertised from and might be blocking single addresses or subnets, there isn't really "one way" of doing any of that stuff, all very dependent on what firewalls you are running on the server or in your larger network. I'd suggest having a bit of a gentle play and doing some reading and learning around *NIX command line administration before hand so you have bit of a feel for command line fun and games.
I guess you could look at your the members page under guests to give the rough list and if lots are viewing errors you could block some IPs in XF itself. Not ideal, but maybe you'll see a very obvious IP pattern for the guests? Otherwise if your only server admin is via some kind of control panel there may well be ways within that to view/tail log files which might work for you. You could ask your host I guess as they will be familiar with the setup they are running!
Sorry to not be of more help. However unless your server is struggling the worst they are probably doing is searching/indexing and borrowing your content!
tail is a unix command to show the end (ie the tail end) of a file. Typically it shows a set number of lines, so a command like tail /etc/hosts would show the end of the /etc/hosts file. The command optionally takes a -f flag to put it into follow mode, in this mode it constantly reads and outputs the end of a file. So if you have log file you can use this mode to in essence watch the webserver log in real (somewhat dependent on the server config) time, so something like tail -f /var/log/nginx.access. You'd normally see all the various file accesses. Typically if you've got someone busy scraping then you'd see their traffic pattern doesn't match other entries. So me requesting this page would be showing as a request for the page and CSS files and images and so forth. Bots are often only interested in the text content so you'd perhaps see the same IP requesting page after page and generally at unreasonable speed.However if you're not familiar with whatever operating system (odds are some *NIX flavour or a *NIX clone like Linux) you are running your forum on then even knowing probably wouldn't help you that much. I mean I'd look and pull out some addresses maybe check where they are being advertised from and might be blocking single addresses or subnets, there isn't really "one way" of doing any of that stuff, all very dependent on what firewalls you are running on the server or in your larger network. I'd suggest having a bit of a gentle play and doing some reading and learning around *NIX command line administration before hand so you have bit of a feel for command line fun and games.
I guess you could look at your the members page under guests to give the rough list and if lots are viewing errors you could block some IPs in XF itself. Not ideal, but maybe you'll see a very obvious IP pattern for the guests? Otherwise if your only server admin is via some kind of control panel there may well be ways within that to view/tail log files which might work for you. You could ask your host I guess as they will be familiar with the setup they are running!
Sorry to not be of more help. However unless your server is struggling the worst they are probably doing is searching/indexing and borrowing your content!
Chromaniac
Well-known member
in the end, it is a personal choice. if these are not affecting your site's performance and you do not mind getting your content indexed by random (ai and others) spiders all around the world, these can be safely ignored. if not, your choice is to identify as many spiders as you can (use the addon that adds a lot more bot identifications to xenforo's own list) and monitor them and block the ones you do not find worthy of indexing your portal.
Similar threads
