• This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.

Spider recognition

Dream

Active member
#1
I worked a while back on the main spiders database for vBulletin, which is now being run by Mosh.

Does xenForo have a spider recognition system that uses a xml? If it has, I could talk to Mosh so the list he is maintaining also supports the xenForo xml format. It has 500 or so spiders.
 
F

Floris

Guest
#4
I think Cho is just trying to have a bit of fun, but it's not the offtopic forum :)

When I go to 'online users' here, I do not see the IP, but I also dont' see search engine bots specified.
 

James

Well-known member
#6
Feel free to make a suggestion, I'm sure a lot of forum owners would like to know how many bots have crawled their forum over a period of time!
 
#7
Feel free to make a suggestion, I'm sure a lot of forum owners would like to know how many bots have crawled their forum over a period of time!
I think this is more of an admin thing than a user thing as a user don't really care about spiders etc. With that being said, Google Analytics shows you how much spiders crawls the website and xenForo has Google Anal integration so i don't think searching each user's IP to determine if it's a spider or not is such a good idea as the feature is already there.
 

James

Well-known member
#8
I think this is more of an admin thing than a user thing as a user don't really care about spiders etc. With that being said, Google Analytics shows you how much spiders crawls the website and xenForo has Google Anal integration so i don't think searching each user's IP to determine if it's a spider or not is such a good idea as the feature is already there.
Bad abbreviation

Yeah, this should be an admin feature rather than a user one. Users don't care about spiders, they just want more users.

Google Analytics has this yeah, would be awesome if we could access it via the XF ACP though!

 
#9
Bad abbreviation

Yeah, this should be an admin feature rather than a user one. Users don't care about spiders, they just want more users.

Google Analytics has this yeah, would be awesome if we could access it via the XF ACP though!
Im pretty sure The Analytics API has some kind of method to pull in the info as i've seen it done on another forum. Ill have to verify that though.
 

Sador

Well-known member
#11
Yeh, you've got to keep track of the spiders on your forum, or the forumchicks will start complaining about them. ;)

It's one of the more basic functions of all forumsoftware, so I'd be surprised if XF wouldn't get it.
 

Kuma

Active member
#16
For me, it's knowing who is a human guest (lurker) and who's not.
Do you really spend all of your time watching who's online? If I know who the major SE's are, I really do not care about random tiny search engines.

It is easy to spoof a user-agent anyway.
 
F

Floris

Guest
#17
I used to idle on who's online just to see where traffic would come from. I've learned over the years it doesn't benefit or improve my day :)
 

Dream

Active member
#19
Just a question of knowing who are humans, nothing so complicated as to live inside the WOL page. But I won't die of suffocation if this feature isn't there.
 

ijeffers

Active member
#20
Im pretty sure The Analytics API has some kind of method to pull in the info as i've seen it done on another forum. Ill have to verify that though.
This is possible, I'm using a wordpress plugin which does it at the moment. It displays the visitors graph, site usage, top pages, top referrers and top searches on the dashboard which is pretty hand to have.