1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

XF 1.4 Robots and SEO Urls

Discussion in 'XenForo Questions and Support' started by squashlover, Oct 20, 2014.

  1. squashlover

    squashlover New Member

    Hi There,

    I have recently migrated a forum from VB and all appears to be going good guns aside from a few minor niggles.

    We have the VB url redirect in place so that the OLD vb urls will get directed to the correct content. However, I've been asked to block bots from viewing member profiles and a few other things. I can find info that suggests this format for the robots.txt file:

    Code:
    Disallow: /account/
    Disallow: /admin.php
    Disallow: /ajax/
    Disallow: /attachments/
    Disallow: /conversations/
    Disallow: /data/
    Disallow: /forums/-/
    Disallow: /forums/tweets/
    Disallow: /goto/
    Disallow: /help/
    Disallow: /internal_data/
    Disallow: /js/
    Disallow: /library/
    Disallow: /login/
    Disallow: /lost-password/
    Disallow: /misc/contact/
    Disallow: /members/
    Disallow: /online/
    Disallow: /recent-activity/
    Disallow: /register/
    Disallow: /posts/
    Disallow: /search/
    Disallow: /styles/
    Allow: /
    However, I assume this is for sites that use SEO urls as the format of my urls is /index.php?members/ etc?

    Are the SEO htaccess rules available? Can it be used in conjunction with the VB4 redirect?

    Thank you!
     
  2. SnoSheriff

    SnoSheriff Active Member

    How did you get this done? I want to do the same but my prev software wasn't VB.
     
    Last edited: Oct 21, 2014
  3. squashlover

    squashlover New Member

    I just found the redirect plugin for VB 4 by searching the XF site. I would not know how to achieve this for a different forum software. Sorry.
     

Share This Page