Google Crawl and 'No Follow' Settings

[Divinum Fiat]

I have a question, if I may. One of my members just sent me the following message:

"XenForo tags all user posted links as "nofollow" which means their is no search engine benefit"

Yesterday I found a page in the Google webmaster tools area where it said that my site is not currently getting crawled. I downloaded and then uploaded a robot.txt file onto the 'highest level file' of my site but not knowing what the highest level file is I tried placing it into the public_html folder but that didn't work; then I tried placing it 'next to' the public_html folder but that didn't work either. And now I can't find that page on Google webmaster anymore. :-(

Obviously I do want my pages and posts get crawled by Google. And I do want to set my XF tags to be set as 'follow.' Does anyone know what I'm doing wrong?

 

[Paul B]

Only links posted by regular members are nofollow, links posted by staff aren't.

Just upload the robots.txt file to your domain root.
As an example, this is the file for XenForo.com: http://xenforo.com/robots.txt

Here's a thread on it: http://xenforo.com/community/threads/my-robots-txt-file.6752/

 

[Divinum Fiat]

oh cool, I didn't know that their links were a no-follow. Does that include their links in their signatures?

I'm not sure what the domain root is. I thought that's where I placed it, but apparently not. When I have Filezilla open in the right window I see various files, including the public_ftp, then public_html, and somewhere further below I placed this robots.txt file. It didn't work, so then I placed it inside the public_html file but that didn't work either.

Even if I placed it now, I don't know how to get back to that Google page to 'verify' that I actually placed the file. Is that important or will Google figure out automatically that it's there?

Thanks for your help, Brogan.

 

[Paul B]

You already have one in your domain root: http://www.corefreedom.com/robots.txt

Sign in to Google and access your webmaster tools account here: https://www.google.com/webmasters/tools/home?hl=en

 

[Divinum Fiat]

Yes, I log into the Google analytics account every day, but what I didn't know was about the robots.txt file. The Google page asked me yesterday to verify after placing the file in my root folder, which I did, but Google said there was an error. That's why I'm confused and not sure if my site is actually getting crawled or not.

 

[Divinum Fiat]

thanks, Brogan, really appreciate it!

 

[MikeMpls]

Cool, this probably explains why my XF forums haven't been plagues by forum spammers anywhere nearly as much as as my vB forums.

(insert thumbs up smilie here)