1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

XF 1.2 Text for robot.txt

Discussion in 'Installation, Upgrade, and Import Support' started by easiii, Jul 31, 2013.

  1. easiii

    easiii Active Member

    Hello,
    Just bought, and downloaded Xenforo. I have not seen an inlcuded robot.txt file.... Is there one available ??

    Thank you!
     
  2. Andy.N

    Andy.N Well-Known Member

  3. easiii

    easiii Active Member

    Thank you! Is there anything else that I should add?
     
  4. Andy.N

    Andy.N Well-Known Member

    It's up to you really.
     
  5. CyclingTribe

    CyclingTribe Well-Known Member

    Here's what I have in mine (if you want other examples for reference):

    Code:
    User-agent: *
    Disallow: /account/
    Disallow: /admin.php
    Disallow: /ajax/
    Disallow: /attachments/
    Disallow: /conversations/
    Disallow: /find-new/
    Disallow: /goto/
    Disallow: /help/
    Disallow: /login/
    Disallow: /lost-password/
    Disallow: /members/
    Disallow: /mobiquo/
    Disallow: /online/
    Disallow: /posts/
    Disallow: /recent-activity/
    Disallow: /register/
    Disallow: /search/
    
     
    easiii likes this.
  6. easiii

    easiii Active Member

    Thank you! This was what I was looking for! :D
     

Share This Page