1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Spider recognition

Discussion in 'XenForo Pre-Sales Questions' started by Dream, Aug 21, 2010.

  1. Dream

    Dream Active Member

    I worked a while back on the main spiders database for vBulletin, which is now being run by Mosh.

    Does xenForo have a spider recognition system that uses a xml? If it has, I could talk to Mosh so the list he is maintaining also supports the xenForo xml format. It has 500 or so spiders.
    Alfa1 and Forsaken like this.
  2. chousho

    chousho Well-Known Member

    Their software works pretty much like this:


    Not a Spider:
    Dragonfly and Tyler like this.
  3. Dream

    Dream Active Member

    Sorry is this a trolling post? :confused: lol
  4. Floris

    Floris Guest

    I think Cho is just trying to have a bit of fun, but it's not the offtopic forum :)

    When I go to 'online users' here, I do not see the IP, but I also dont' see search engine bots specified.
  5. Dream

    Dream Active Member

    I see, must be something that will be looked into at a later date. Thanks Floris.
  6. James

    James Well-Known Member

    Feel free to make a suggestion, I'm sure a lot of forum owners would like to know how many bots have crawled their forum over a period of time!
  7. Brandon_R

    Brandon_R Guest

    I think this is more of an admin thing than a user thing as a user don't really care about spiders etc. With that being said, Google Analytics shows you how much spiders crawls the website and xenForo has Google Anal integration so i don't think searching each user's IP to determine if it's a spider or not is such a good idea as the feature is already there.
  8. James

    James Well-Known Member

    Bad abbreviation

    Yeah, this should be an admin feature rather than a user one. Users don't care about spiders, they just want more users.

    Google Analytics has this yeah, would be awesome if we could access it via the XF ACP though!

  9. Brandon_R

    Brandon_R Guest

    Im pretty sure The Analytics API has some kind of method to pull in the info as i've seen it done on another forum. Ill have to verify that though.
  10. Peggy

    Peggy Well-Known Member

    Yep, I always kept track of spiders on my forum.
  11. Sador

    Sador Well-Known Member

    Yeh, you've got to keep track of the spiders on your forum, or the forumchicks will start complaining about them. ;)

    It's one of the more basic functions of all forumsoftware, so I'd be surprised if XF wouldn't get it.
  12. DaKat

    DaKat Well-Known Member

    I like to keep an eye on which ones crawl my site, too.
  13. DSF

    DSF Well-Known Member

    What for I need this information?
    Help me
  14. Kuma

    Kuma Active Member

    Never understood why people were so worried about what spiders were browsing their site.
  15. Dream

    Dream Active Member

    For me, it's knowing who is a human guest (lurker) and who's not.
  16. Kuma

    Kuma Active Member

    Do you really spend all of your time watching who's online? If I know who the major SE's are, I really do not care about random tiny search engines.

    It is easy to spoof a user-agent anyway.
  17. Floris

    Floris Guest

    I used to idle on who's online just to see where traffic would come from. I've learned over the years it doesn't benefit or improve my day :)
  18. KURTZ

    KURTZ Well-Known Member

  19. Dream

    Dream Active Member

    Just a question of knowing who are humans, nothing so complicated as to live inside the WOL page. But I won't die of suffocation if this feature isn't there.
  20. ijeffers

    ijeffers Active Member

    This is possible, I'm using a wordpress plugin which does it at the moment. It displays the visitors graph, site usage, top pages, top referrers and top searches on the dashboard which is pretty hand to have.

Share This Page