XF 2.2 Robots.txt is not accessible or doesn't exist

JasonBrody

Active member
Hi All,
Though I had never checked it earlier, the concern was raised after I changed my hosting.
Recently, Google Console has not been indexing my site, and its sitemap has also not been read since the last few weeks.
I am getting robots blocked and redirection error messages too.

When I checked for robots.txt file (ex: https://yourdomain.com/robots.txt ) then found that it doesn't exist/accessible. What should I do? By default, the XenForo package doesn't provide any robots.txt files.

Please suggest.
 
I am getting robots blocked and redirection error messages too.
That doesn't sound like a missing robots.txt, though. Absent robots.txt, robots should have wide open access to your site since there would be no controls on them unless you've got them set somewhere else like a firewall. The redirection errors, too, suggest something else to me but I'm not a total expert on web stuff since we have other people managing the webservers at work.
 
What if I re-install from this stage ? will I lost all data ?
As @Brogan said, the missing robot.txt isn't going to be fixed this way since the file is not part of XF. Download the one he supplied and then upload it to the root directory of your site. e.g. mine (which is blank since I don't really use it) is in public_html. cPanel File Manager or similar should allow you to do this if you have admin access to the web server.
 
Back
Top Bottom