XF 2.2 Does XF has a default robots.txt file?

[KhanTastic]

Hi friends,

Recently I got so much spam on my XF forum, I updated it to the latest version just now and everything looks fine. I deleted all the spamming posts and blocked the spam users. I just would like to know if XF has any default robots.txt file as I can not see in my root folder and I just created one by seeing the codes from a website in google search engine. I am not sure if my robots.txt is correct or not.
Because my forum is not indexing at all in search engine, the posts are not getting any views. I am thinking maybe my robots.txt is not good and blocking search engine from crawling my site.

I would appreciate if someone can confirm to me if XF really has a robots.txt file by default, if it has then can you paste the codes here for me so that I can replace it on my robots.txt file. Or should I totally get rid of robots.txt file which I created, does XF really need that robots.txt file?

Thank you so much in advance, awaiting yours.

Best regards,
Cheers!

 

[Paul B]

You just create one to suit your site and needs.

This is the one from here: https://xenforo.com/robots.txt

 

[KhanTastic]

You just create one to suit your site and needs.

This is the one from here: https://xenforo.com/robots.txt

oh thank you so much for the quick reply, yes I will use the same as this XF has.
Thanks a lot once again

Have a great evening ahead!

 

[Paul B]

Make sure you update it to reflect where your site is installed, removing or replacing community for example.

 

[KhanTastic]

Make sure you update it to reflect where your site is installed, removing or replacing community for example.

Yes I noticed that and remove that "community"

Thanks <3

 

[Old Nick]

What's the difference between sitemap.php and sitemap.xml? The content seems the same, which one i have to use for SE?

 

[bzcomputers]

What's the difference between sitemap.php and sitemap.xml? The content seems the same, which one i have to use for SE?

They are identical, you can use one or the other. Google still prefers the .xml sitemap extension.

Build and Submit a Sitemap | Google Search Central | Documentation | Google for Developers

Google supports several sitemap formats. Follow this guide to learn about formats, how to build a sitemap, and how to submit a sitemap to Google.

developers.google.com

 

[masmo]

You just create one to suit your site and needs.

This is the one from here: https://xenforo.com/robots.txt

Just to say that yesterday Bing reported the indexing problem to me because I have used the robots.txt recommended on this site and used by many others users.

XenForo IndexNow is using links with the format https://sitename.com/posts/123456/ and in the robots.txt file there is Disallow: /posts/
So be careful because this robots.txt blocks all IndexNow links!

I hope I was helpful...

 

[Old Nick]

You want each post indexed?