Diana
Active member
I don't know much about crawlers or SEO, and I don't understand how google bots operate.
Watching the behavior of google bots on my forum it appears that they're stuck in some kind of netherland. All it shows is that they're "viewing unknown page." Once in awhile they view the "events calendar." I've never seen them view a thread, and there are quite a lot of them now.
Can someone help me understand this process better, and is there anything I can do to optimize how google crawls my forum?
btw, I do have a robots file.
User-agent: *
Disallow: /forum/find-new/
Disallow: /forum/forums/-/
Disallow: /forum/account/
Disallow: /forum/attachments/
Disallow: /forum/goto/
Disallow: /forum/posts/
Disallow: /forum/login/
Disallow: /forum/admin.php
Allow: /
Watching the behavior of google bots on my forum it appears that they're stuck in some kind of netherland. All it shows is that they're "viewing unknown page." Once in awhile they view the "events calendar." I've never seen them view a thread, and there are quite a lot of them now.
Can someone help me understand this process better, and is there anything I can do to optimize how google crawls my forum?
btw, I do have a robots file.
User-agent: *
Disallow: /forum/find-new/
Disallow: /forum/forums/-/
Disallow: /forum/account/
Disallow: /forum/attachments/
Disallow: /forum/goto/
Disallow: /forum/posts/
Disallow: /forum/login/
Disallow: /forum/admin.php
Allow: /