• This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.

how to give google authorization

MagnusB

Well-known member
#2
I dont understand what you mean, AFAIK bots are handled just as any other unregistered user in xF. If you want to reduce the 403 errors, you might look to remove Google's access to those pages via robots.txt. A good place to start would be removing access to /members/ and /attachments/ deny them to index attachments and member profiles.
 

MagnusB

Well-known member
#4
Where did you find that? I have not seen that before, and I just took a look in webmaster tools, and I did not see an option to do that.

If you get allot of 403 errors, I suggest you update your robots.txt file, Google respects that:
Code:
User-agent: *
 
Disallow: /account/
Disallow: /admin.php
Disallow: /ajax/
Disallow: /attachments/
Disallow: /conversations/
Disallow: /data/
Disallow: /forums/-/
Disallow: /forums/tweets/
Disallow: /goto/
Disallow: /help/
Disallow: /internal_data/
Disallow: /js/
Disallow: /library/
Disallow: /login/
Disallow: /lost-password/
Disallow: /misc/contact/
Disallow: /members/
Disallow: /online/
Disallow: /recent-activity/
Disallow: /register/
Disallow: /posts/
Disallow: /search/
Disallow: /styles/
Allow: /
That is what I have, these will block access to most areas you don't want indexed by Google, like registration forms, search pages, member profiles, attachments and some system folders. It will also prevent allot of double links, as many of the xF links are just "redirects", like the posts links will redirect to a specific post within a thread etc etc.

These assume you have full friendly URL's enabled though, I am not sure if they will work without them.
 

CTXMedia

Formerly CyclingTribe
#7
In the Restricted Directory or URL put the URL of the page/forum etc. that you want Adsense to be able to crawl.

Next add the login URL (I've pasted mine, just adjust it to suit your site/domin): http://www.cyclechat.net/index.php?login/login

Login method: POST

Parameters:
login = Username (this is the forum username for the account you've setup for Google to use - I've called mine GoogleSpider)
password = ******** <the password for the user account above>
cookie_check = 1

Hope this helps. (y)

Cheers,
Shaun :D