URL Redirect

tommydamic68

Well-known member
Question- perhaps directed towards @Jake Bunce - I was fiddling with an online SEO checker and it found some error, not sure if it is valid.

It said my sites www.mysite.com didn't redirect to mysite.com and vice versa= duplicate content.

Also said I am
Missing my robots.txt file and my sitemap file (the sitemap is an add on from here on the site.)

Any help or suggestion to check the validity if these findings?
 
Question- perhaps directed towards @Jake Bunce - I was fiddling with an online SEO checker and it found some error, not sure if it is valid.

It said my sites www.mysite.com didn't redirect to mysite.com and vice versa= duplicate content.
Assuming this is your cat site, then this is correct. I visited the site without www in the URL and with www in the URL. You should set up your hosting so that the www and non-www URLs direct to the same URL (you can choose whether the one they go to has www or not in it). You can see this working on my site.

Also said I am
Missing my robots.txt file and my sitemap file (the sitemap is an add on from here on the site.)
robots.txt is either missing or not accessible. If I visit www.yoursite.com/robots.txt it takes me to www.yoursite.com/community/ It could be your redirecting everything that begins with www.yoursite.com to www.yoursite.com/community and so it'll need adjusting.

I'm presuming the sitemap is sitemap.xml? If so, this should be at www.yoursite.com/sitemap.xml. Same problems here as with robots.txt
 
Assuming this is your cat site, then this is correct. I visited the site without www in the URL and with www in the URL. You should set up your hosting so that the www and non-www URLs direct to the same URL (you can choose whether the one they go to has www or not in it). You can see this working on my site.


robots.txt is either missing or not accessible. If I visit www.yoursite.com/robots.txt it takes me to www.yoursite.com/community/ It could be your redirecting everything that begins with www.yoursite.com to www.yoursite.com/community and so it'll need adjusting.

I'm presuming the sitemap is sitemap.xml? If so, this should be at www.yoursite.com/sitemap.xml. Same problems here as with robots.txt
@Martok - can my hosting service do the redirect of the URL? Or is that something I should do.
Thank you .
 
Dumb question? Is there a difference between my issue with The redirection of my URL and the code @Jake Bunce gave me for my redirects of posts and threads that is in my current .htaccess file?
 
Assuming this is your cat site, then this is correct. I visited the site without www in the URL and with www in the URL. You should set up your hosting so that the www and non-www URLs direct to the same URL (you can choose whether the one they go to has www or not in it). You can see this working on my site.


robots.txt is either missing or not accessible. If I visit www.yoursite.com/robots.txt it takes me to www.yoursite.com/community/ It could be your redirecting everything that begins with www.yoursite.com to www.yoursite.com/community and so it'll need adjusting.

I'm presuming the sitemap is sitemap.xml? If so, this should be at www.yoursite.com/sitemap.xml. Same problems here as with robots.txt
Ok - I think i fixed the site URL issue:

I used this:
Code:
RewriteCond %{HTTP_HOST} www.sphynxlair.com$
RewriteRule ^(.*)$ http://sphynxlair.com/$1 [R=301,L]

http://www.sphynxlair.com
http://sphynxlair.com

But I still get redirected to http://www.sphynxlair.com/community when i try www.mysite.com/robots.txt or www.sphynxlair.com/sitemap.xml - i am using this add on for the sitemap - http://xenforo.com/community/resources/sitemap-for-xenforo-1-2-compatible.67/
 
Did you not read the add-on discussion? Not to mention, you won't see a sitemap.xml (the actual file is archived and is called sitemap.xml.gz (which the associated search engines can read).
You also have not done a re-write for the robots.txt (actually the robots.php that runs to output the robots.txt) as required by that add-on.
You probably need something like
Code:
RewriteRule (robots\.txt)$ community/robots.php [NC,L]
in your .htaccess.
 
Last edited:
Did you not read the add-on discussion? Not to mention, you won't see a sitemap.xml (the actual file is archived and is called sitemap.xml.gz (which the associated search engines can read).
You also have not done a re-write for the robots.txt (actually the robots.php that runs to output the robots.txt) as required by that add-on.
You probably need something like
Code:
RewriteRule (robots\.txt)$ community/robots.php [NC,L]
in your .htaccess.
Here is my sitemap in webmaster tools - it appears to be working. Shouldn't the robots.txt instructions be in the core explanation of the add on, not barred in the 38 pages of the discussion? just sayin…Also, the robots.txt code above unfortunately did not work. I will se if i can get an answer on the add ons discussion are from the author. Thanks @Tracy Perry

Screen Shot 2014-01-08 at 5.03.58 AM.webp
 
and i turned off the add on and it still defaults to my site when inputting www.mysite.com/robots.txt
If you turn it off then you have to manually create a robots.txt. The re-write will depend on what web server you use (and it has to be in the .htaccess for Apache). I haven't used Apache in so long I'm not sure where/what it would need - but I provided the one for nginx that works earlier. Assistance for this add-on would be more appropriate in the discussion area for it as it is not XenForo problem/question but the add-on. I noticed that you run your site in a /community directory - so the rewrite example for nginx would need to be modified accordingly. It is based upon running your forum in the root.

screenshot.webp
 
Thanks @Tracy Perry - I have started a thread in the add on discussion area- hopefully I will get an answer. I do have a robots. Txt file In my root directory that I had created awhile ago. Not sure if it's the add on causing the issue or something either @Jake Bunce had me add (redirects for VBSEO) or even @Adam Howard had me add at one point to safeguard folders on my server.
 
So reading through some of the discussions on the sitemap add on I ran across this- apparently its just an option so i don't think the add on is the issue - and i have my own robots.txt so it must be something in my .htaccess file. The only thing I ever added was @Jake Bunce redirect suggestion. I wish someone could chime in with a solution...
Screen Shot 2014-01-08 at 8.02.10 PM.webp
Here is my .htaccess file, perhaps someone can see something:

Code:
Options -Indexes
RewriteEngine On
RewriteCond %{HTTP_HOST} www.sphynxlair.com$
RewriteRule ^(.*)$ http://sphynxlair.com/$1 [R=301,L]
RewriteRule ^[^/]+/.+-([0-9]+)/$ /community/index.php?threads/$1/ [R=301,L]
RewriteRule ^forum\.php$ /community/ [R=301,L]
RewriteRule ^(images|dir2|dir3|dir4)/.*$ - [S=1]
RewriteRule ^.*$ /community/ [R=301,L]
# Use PHP 5.3
AddType application/x-httpd-php53 .php
# Ticket PBB-18872701
<IfModule mod_suphp.c>
  suPHP_ConfigPath /home/admin/public_html
  <Files php.ini>
    order allow,deny
    deny from all
  </Files>
</IfModule>
# Ticket PBB-18872701

DirectoryIndex index.php
#RewriteEngine Off

# Comment the following line (add '#' at the beginning)
# to disable mod_rewrite functions.
# Please note: you still need to disable the hack in
# the vBSEO control panel to stop url rewrites.
RewriteEngine On

# Some servers require the Rewritebase directive to be
# enabled (remove '#' at the beginning to activate)
# Please note: when enabled, you must include the path
# to your root vB folder (i.e. RewriteBase /forums/)
#RewriteBase /

#RewriteCond %{HTTP_HOST} !^www\.sphynxlair\.com
#RewriteRule (.*) http://www.sphynxlair.com/$1 [L,R=301]

RewriteRule ^((urllist|sitemap).*\.(xml|txt)(\.gz)?)$ vbseo_sitemap/vbseo_getsitemap.php?sitemap=$1 [L]

RewriteCond %{REQUEST_URI} !(admincpvioc1/|modcpvioc2/|cron|vbseo_sitemap)
RewriteRule ^((archive/)?(.*\.php(/.*)?))$ vbseo.php [L,QSA]

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !/(admincpvioc1|modcpvioc2|clientscript|cpstyles|images)/
RewriteRule ^(.+)$ vbseo.php [L,QSA]

<Files 403.shtml>
order allow,deny
allow from all
</Files>

RewriteCond %{HTTP_USER_AGENT} ^Baiduspider.* [NC]

#RewriteCond %{HTTP_HOST} ^sphynxlair.com$
#RewriteRule ^/?$ "http\:\/\/www\.sphynxlair\.com\/" [R=301,L]

#RewriteCond %{HTTP_HOST} ^sphynxlair.com$
#RewriteRule ^forum\.+\/?(.*)$ "http\:\/\/www\.sphynxlair\.com\/forum\.php$1" [R=301,L]
# Expire images header
ExpiresActive On
ExpiresDefault A0
ExpiresByType image/gif A2592000
ExpiresByType image/png A2592000
ExpiresByType image/jpg A2592000
ExpiresByType image/jpeg A2592000
ExpiresByType image/ico A2592000
ExpiresByType text/css A2592000
ExpiresByType text/javascript A2592000
# compress text, html, javascript, css, xml:
AddOutputFilterByType DEFLATE text/plain
AddOutputFilterByType DEFLATE text/html
AddOutputFilterByType DEFLATE text/xml
AddOutputFilterByType DEFLATE text/css
AddOutputFilterByType DEFLATE application/xml
AddOutputFilterByType DEFLATE application/xhtml+xml
AddOutputFilterByType DEFLATE application/rss+xml
AddOutputFilterByType DEFLATE application/javascript
AddOutputFilterByType DEFLATE application/x-javascript

# Or, compress certain file types by extension:
<files *.html>
SetOutputFilter DEFLATE
</files>

RewriteEngine On
RewriteRule ^Microsoft-Server-ActiveSync$ hgpush/index.php
 
Top Bottom