• This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.

Making spider names visable and allow spider access as Registered users.

tenants

Well-known member
#1
Is it possible to make the spiders:
Bing [Bot], Google [Bot], Google Adsense [Bot], and Yahoo [Bot]
Show up as members/bots (instead of guests)

Also, is it possible to give these bots access to "registerd" only areas, so they can index the registerd areas (affectively registering them)

I've seen this possible with some forums (phpBB and VB I think), is it possible with XenForo?

* Right now, if a bot craws my site, it is diplayed as "guest" on the RHS (so I dont know whats a person and whats a bot)
* Also, this "guest" wont have access to some of the restricted areas of the forum
 
R

ragtek

Guest
#3
Also, is it possible to give these bots access to "registerd" only areas, so they can index the registerd areas (affectively registering them)
why would you want this?

If somebody don't wants to register, he'll still not register, but he will be able to read all the "hidden" stuff, because he can read the thread with help of the google cache (for example: http://webcache.googleusercontent.com/search?q=cache:O0Oa4QMfq_cJ:xenforo.com/community/threads/xenforo-1-0-1.14603/ http://xenforo.com/community/threads&cd=1&hl=de&ct=clnk&gl=at&client=firefox-a&source=www.google.at )
2. goggle doesn't like this, they call this cloaking and worst scenario is => loosing a good position)
 

tenants

Well-known member
#4
hmm, okay... I guess there's not much point in indexing areas that guest cant view.

I was thinking it would pull move vistors from the search engines, but you're right, this does sound like cloaking. I'm sure I've heard other sites alowing access for bots to registerd areas
[I'll look it up]

It would still be handy for the bot names to appear.. looking foward to 1.1
 

Brogan

XenForo moderator
Staff member
#5
You are correct that it is possible to register bots in phpBB3.

It's against Google's T&C's though so you run the risk of having your site delisted.