1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

How can I block google from crawling the events calendar

Discussion in 'Troubleshooting and Problems' started by Diana, Jul 8, 2012.

  1. Diana

    Diana Active Member

    As far as I can see, google never crawls anything on my site but the events calender.

    this is my robots.txt file.

    User-agent: *

    Disallow: /test/
    Disallow: /forum/find-new/
    Disallow: /forum/account/
    Disallow: /forum/attachments/
    Disallow: /forum/conversations/
    Disallow: /forum/events/birthdays/
    Disallow: /forum/events/monthly
    Disallow: /forum/events/weekly
    Disallow: /forum/forums/-/
    Disallow: /forum/goto/
    Disallow: /forum/posts/
    Disallow: /forum/login/
    Disallow: /forum/events/
    Disallow: /forum/profile/
    Disallow: /forum/calendar/
    Disallow: /forum/misc/
    Disallow: /forum/search/
    Disallow: /forum/members/
    Disallow: /forum/register/
    Disallow: /forum/online/
    Disallow: /forum/recent-activity/
    Disallow: /forum/lost-password/
    Disallow: /forum/admin.php
    Disallow: /forum/help/
    Allow: /

    What am I doing wrong?

    Meanwhile Baidu is all over the place and completely ignores events. I guess that makes me popular in China?
     
  2. Jake Bunce

    Jake Bunce XenForo Moderator Staff Member

    Disallow: /forum/events/

    I wouldn't worry about it. Bots congregating on a specific page is not your problem to fix. And I doubt it implies anything about your Google links. The only reason to disallow a page is if you don't want that page to show up on Google.
     
  3. Diana

    Diana Active Member

    Ah, I get it now.

    Thank you Jake :)
     

Share This Page