1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Google Crawl and 'No Follow' Settings

Discussion in 'XenForo Questions and Support' started by Core Freedom, Dec 1, 2011.

  1. Core Freedom

    Core Freedom Well-Known Member

    I have a question, if I may. One of my members just sent me the following message:

    "XenForo tags all user posted links as "nofollow" which means their is no search engine benefit"

    Yesterday I found a page in the Google webmaster tools area where it said that my site is not currently getting crawled. I downloaded and then uploaded a robot.txt file onto the 'highest level file' of my site but not knowing what the highest level file is I tried placing it into the public_html folder but that didn't work; then I tried placing it 'next to' the public_html folder but that didn't work either. And now I can't find that page on Google webmaster anymore. :-(

    Obviously I do want my pages and posts get crawled by Google. And I do want to set my XF tags to be set as 'follow.' Does anyone know what I'm doing wrong?
     
  2. Brogan

    Brogan XenForo Moderator Staff Member

    Core Freedom likes this.
  3. Core Freedom

    Core Freedom Well-Known Member

    oh cool, I didn't know that their links were a no-follow. Does that include their links in their signatures?

    I'm not sure what the domain root is. I thought that's where I placed it, but apparently not. When I have Filezilla open in the right window I see various files, including the public_ftp, then public_html, and somewhere further below I placed this robots.txt file. It didn't work, so then I placed it inside the public_html file but that didn't work either.

    Even if I placed it now, I don't know how to get back to that Google page to 'verify' that I actually placed the file. Is that important or will Google figure out automatically that it's there?

    Thanks for your help, Brogan.
     
  4. Brogan

    Brogan XenForo Moderator Staff Member

  5. Core Freedom

    Core Freedom Well-Known Member

    Yes, I log into the Google analytics account every day, but what I didn't know was about the robots.txt file. The Google page asked me yesterday to verify after placing the file in my root folder, which I did, but Google said there was an error. That's why I'm confused and not sure if my site is actually getting crawled or not. :)
     
  6. Core Freedom

    Core Freedom Well-Known Member

    thanks, Brogan, really appreciate it! ;)
     
  7. MikeMpls

    MikeMpls Well-Known Member

    Cool, this probably explains why my XF forums haven't been plagues by forum spammers anywhere nearly as much as as my vB forums.

    (insert thumbs up smilie here)
     

Share This Page