XF 2.0 Changes to robots.txt from Xenforo 1.5 to 2.0

JackieChun

Well-known member
Are there changes to XenForo's URL structure in the 2.0 version that warrant taking another look at your old robots.txt file? Here's what my old robots.txt looks like. I believe it was originally authored by @Brogan.

User-agent: *
Disallow: /forum/attachments/
Disallow: /forum/forums/*/?direction
Disallow: /forum/forums/*/?order
Disallow: /forum/help/
Disallow: /forum/login/
Disallow: /forum/lost-password/
Disallow: /forum/members/
Disallow: /forum/misc/
Disallow: /forum/online/
Disallow: /forum/posts/
Disallow: /forum/register/
Disallow: /forum/search/
 

SyTry

Well-known member
Hello,

XenForo Community use this one :
Code:
User-agent: *
Disallow: /community/find-new/
Disallow: /community/account/
Disallow: /community/attachments/
Disallow: /community/goto/
Disallow: /community/posts/
Disallow: /community/login/
Disallow: /community/admin.php
Allow: /

Sitemap: https://xenforo.com/community/sitemap.php

Link : https://xenforo.com/robots.txt

Regards, SyTry
 

Ozzy47

Well-known member
Hello,

XenForo Community use this one :
Code:
User-agent: *
Disallow: /community/find-new/
Disallow: /community/account/
Disallow: /community/attachments/
Disallow: /community/goto/
Disallow: /community/posts/
Disallow: /community/login/
Disallow: /community/admin.php
Allow: /

Sitemap: https://xenforo.com/community/sitemap.php

Link : https://xenforo.com/robots.txt

Regards, SyTry

That assumes the files are in the /community folder, not in the /forum folder. ;)

As for the OP sure, I believe what you already have should be fine.
 
Top