• This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.

XF 1.4 Improve SEO - Redirection 301 to avoid duplicate content

Betclever

Active member
#1
Hello all,

How can I make a redirection to avoid duplicate content cause when I test my website with webrank, it tells me that I have to use a redirection to avoid duplicate content.

In fact, I have access with www. or without www. so how can I redirect this please?

Thanks.
 

Betclever

Active member
#2
I found a like on xenforo and added this:

User-agent: *
Disallow: /test/
Disallow: /account/
Disallow: /admin.php
Disallow: /ajax/
Disallow: /conversations/
Disallow: /events/birthdays/
Disallow: /events/monthly
Disallow: /events/weekly
Disallow: /find-new/
Disallow: /forums/-/
Disallow: /forums/tweets/
Disallow: /goto/
Disallow: /help/
Disallow: /login/
Disallow: /lost-password/
Disallow: /media/category/
Disallow: /media/keyword/
Disallow: /media/user/
Disallow: /media/service/
Disallow: /media/submit/
Disallow: /misc/style?*
Disallow: /misc/quick-navigation-menu?*
Disallow: /online/
Disallow: /pages/conduct/
Disallow: /pages/privacy/
Disallow: /posts/
Disallow: /threads/tera-tweet-from-*
Disallow: /wiki/special/
Allow: /
I just removed 3 things like test, tera-tweet and wiki but I got a question:

If "allow" is empty, is my content indexed or shall I add something?
In fact, I don't want to loose the indexation so when I see Disallow for forums and posts, what will happen then if it stays like on the list?

Thanks.
 

popowich

Active member
#3
The robots.txt above isn't for fixing your site working as both www and non-www

This is what you want in your .htaccess to force the www version of your site

# Force the site to www
RewriteEngine On
RewriteCond %{HTTP_HOST} !^www\.example\.com$
RewriteRule (.*) http://www.example.com/$1 [L,R=301]
# end force site to www

You'll need to replace exmaple.com with your domain name
 

Betclever

Active member
#4
The robots.txt above isn't for fixing your site working as both www and non-www

This is what you want in your .htaccess to force the www version of your site

# Force the site to www
RewriteEngine On
RewriteCond %{HTTP_HOST} !^www\.example\.com$
RewriteRule (.*) http://www.example.com/$1 [L,R=301]
# end force site to www

You'll need to replace exmaple.com with your domain name
Yes, I'm sorry, wrong thread. :)

I added your code but nothing changed.

I think, I'm using the second one so bet-clever.com but the www. version works too.

Please find my htaccess:

# Mod_security can interfere with uploading of content such as attachments. If you
# cannot attach files, remove the "#" from the lines below.
#<IfModule mod_security.c>
# SecFilterEngine Off
# SecFilterScanPOST Off
#</IfModule>

ErrorDocument 401 default
ErrorDocument 403 default
ErrorDocument 404 default
ErrorDocument 500 default

<IfModule mod_rewrite.c>
RewriteEngine On

# If you are having problems with the rewrite rules, remove the "#" from the
# line that begins "RewriteBase" below. You will also have to change the path
# of the rewrite to reflect the path to your XenForo installation.
#RewriteBase /xenforo

# This line may be needed to enable WebDAV editing with PHP as a CGI.
#RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]

RewriteCond %{REQUEST_FILENAME} -f [OR]
RewriteCond %{REQUEST_FILENAME} -l [OR]
RewriteCond %{REQUEST_FILENAME} -d
RewriteRule ^.*$ - [NC,L]
RewriteRule ^(data/|js/|styles/|install/|favicon\.ico|crossdomain\.xml|robots\.txt) - [NC,L]
RewriteRule ^.*$ index.php [NC,L]
</IfModule>

# Author: Final Kaoss (aka: Extreme-Gaming)
# Version 2.6
RewriteEngine on

#Deny attempts to view the Htaccess file.
<Files .htaccess>
Order allow,deny
Deny from all
</Files>
# BEGIN Browser Caching/Headers
<IfModule mod_mime.c>
AddType text/css .css
AddType text/richtext .rtf .rtx
AddType image/svg+xml .svg .svgz
AddType text/plain .txt
AddType text/xsd .xsd
AddType text/xsl .xsl
AddType video/asf .asf .asx .wax .wmv .wmx
AddType video/avi .avi
AddType image/bmp .bmp
AddType application/java .class
AddType video/divx .divx
AddType application/msword .doc .docx
AddType application/x-msdownload .exe
AddType image/gif .gif
AddType application/x-gzip .gz .gzip
AddType image/x-icon .ico
AddType image/jpeg .jpg .jpeg .jpe
AddType application/vnd.ms-access .mdb
AddType audio/midi .mid .midi
AddType video/quicktime .mov .qt
AddType audio/mpeg .mp3 .m4a
AddType video/mp4 .mp4 .m4v
AddType video/mpeg .mpeg .mpg .mpe
AddType application/vnd.ms-project .mpp
AddType application/vnd.oasis.opendocument.database .odb
AddType application/vnd.oasis.opendocument.chart .odc
AddType application/vnd.oasis.opendocument.formula .odf
AddType application/vnd.oasis.opendocument.graphics .odg
AddType application/vnd.oasis.opendocument.presentation .odp
AddType application/vnd.oasis.opendocument.spreadsheet .ods
AddType application/vnd.oasis.opendocument.text .odt
AddType audio/ogg .ogg
AddType application/pdf .pdf
AddType image/png .png
AddType application/vnd.ms-powerpoint .pot .pps .ppt .pptx
AddType audio/x-realaudio .ra .ram
AddType application/x-shockwave-flash .swf
AddType application/x-tar .tar
AddType image/tiff .tif .tiff
AddType audio/wav .wav
AddType audio/wma .wma
AddType application/vnd.ms-write .wri
AddType application/vnd.ms-excel .xla .xls .xlsx .xlt .xlw
AddType application/zip .zip
</IfModule>
<IfModule mod_expires.c>
ExpiresActive On
ExpiresByType text/css A2628000
ExpiresByType text/richtext A3600
ExpiresByType image/svg+xml A3600
ExpiresByType text/plain A3600
ExpiresByType text/xsd A3600
ExpiresByType text/xsl A3600
ExpiresByType video/asf A2628000
ExpiresByType video/avi A2628000
ExpiresByType image/bmp A2628000
ExpiresByType application/java A2628000
ExpiresByType video/divx A2628000
ExpiresByType application/msword A2628000
ExpiresByType application/x-msdownload A2628000
ExpiresByType image/gif A2628000
ExpiresByType application/x-gzip A2628000
ExpiresByType image/x-icon A2628000
ExpiresByType image/jpeg A2628000
ExpiresByType application/vnd.ms-access A2628000
ExpiresByType audio/midi A2628000
ExpiresByType video/quicktime A2628000
ExpiresByType audio/mpeg A2628000
ExpiresByType video/mp4 A2628000
ExpiresByType video/mpeg A2628000
ExpiresByType application/vnd.ms-project A2628000
ExpiresByType application/vnd.oasis.opendocument.database A2628000
ExpiresByType application/vnd.oasis.opendocument.chart A2628000
ExpiresByType application/vnd.oasis.opendocument.formula A2628000
ExpiresByType application/vnd.oasis.opendocument.graphics A2628000
ExpiresByType application/vnd.oasis.opendocument.presentation A2628000
ExpiresByType application/vnd.oasis.opendocument.spreadsheet A2628000
ExpiresByType application/vnd.oasis.opendocument.text A2628000
ExpiresByType audio/ogg A2628000
ExpiresByType application/pdf A2628000
ExpiresByType image/png A2628000
ExpiresByType application/vnd.ms-powerpoint A2628000
ExpiresByType audio/x-realaudio A2628000
ExpiresByType application/x-shockwave-flash A2628000
ExpiresByType application/x-tar A2628000
ExpiresByType image/tiff A2628000
ExpiresByType audio/wav A2628000
ExpiresByType audio/wma A2628000
ExpiresByType application/vnd.ms-write A2628000
ExpiresByType application/vnd.ms-excel A2628000
ExpiresByType application/zip A2628000
</IfModule>
<IfModule mod_deflate.c>
<IfModule mod_setenvif.c>
BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4\.0[678] no-gzip
BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
BrowserMatch \bMSI[E] !no-gzip !gzip-only-text/html
</IfModule>
<IfModule mod_headers.c>
Header append Vary User-Agent env=!dont-vary
</IfModule>
<IfModule mod_filter.c>
AddOutputFilterByType DEFLATE text/css application/x-javascript text/html text/richtext image/svg+xml text/plain text/xsd text/xsl text/xml image/x-icon
</IfModule>
</IfModule>
<FilesMatch "\.(css|CSS|)$">
<IfModule mod_headers.c>
Header set Pragma "public"
Header append Cache-Control "public, must-revalidate, proxy-revalidate"
</IfModule>
FileETag MTime Size
<IfModule mod_headers.c>
Header set X-Powered-By "Step by Step guide to speed up your VB4 (forum only)"
</IfModule>
</FilesMatch>
<FilesMatch "\.(rtf|rtx|svg|svgz|txt|xsd|xsl|RTF|RTX|SVG|SVGZ|TXT|XSD|XSL|)$">
<IfModule mod_headers.c>
Header set Pragma "public"
Header append Cache-Control "public, must-revalidate, proxy-revalidate"
</IfModule>
FileETag MTime Size
<IfModule mod_headers.c>
Header set X-Powered-By "Step by Step guide to speed up your VB4 (forum only)"
</IfModule>
</FilesMatch>
<FilesMatch "\.(asf|asx|wax|wmv|wmx|avi|bmp|class|divx|doc|docx|exe|gif|gz|gzip|ico|jpg|jpeg|jpe|mdb|mid|midi|mov|qt|mp3|m4a|mp4|m4v|mpeg|mpg|mpe|mpp|odb|odc|odf|odg|odp|ods|odt|ogg|pdf|png|pot|pps|ppt|pptx|ra|ram|swf|tar|tif|tiff|wav|wma|wri|xla|xls|xlsx|xlt|xlw|zip|ASF|ASX|WAX|WMV|WMX|AVI|BMP|CLASS|DIVX|DOC|DOCX|EXE|GIF|GZ|GZIP|ICO|JPG|JPEG|JPE|MDB|MID|MIDI|MOV|QT|MP3|M4A|MP4|M4V|MPEG|MPG|MPE|MPP|ODB|ODC|ODF|ODG|ODP|ODS|ODT|OGG|PDF|PNG|POT|PPS|PPT|PPTX|RA|RAM|SWF|TAR|TIF|TIFF|WAV|WMA|WRI|XLA|XLS|XLSX|XLT|XLW|ZIP)$">
<IfModule mod_headers.c>
Header set Pragma "public"
Header append Cache-Control "public, must-revalidate, proxy-revalidate"
</IfModule>
FileETag MTime Size
<IfModule mod_headers.c>
Header set X-Powered-By "Step by Step guide to speed up your VB4 (forum only)"
</IfModule>
</FilesMatch>
# End Browser Caching/Headers

# Force the site to www
RewriteEngine On
RewriteCond %{HTTP_HOST} !^www\.bet-clever\.com$
RewriteRule (.*) http://www.bet-clever.com/$1 [L,R=301]
# end force site to www
You see on the bottom of the code that I added your code but changed to the url of my forum but the rule doesn't work so when I go to http://bet-clever.com, it doesn't redirect to http://www.bet-clever.com

What's wrong?

PS: I'm using shorter URL by Waindigo and rewrite rules under the CP.
 

Mike

XenForo developer
Staff member
#6
You need to move the code you've added to .htaccess up, likely at the top. That said, you've added a ton of code that I can't say I'd really recommend adding unless you understand what it's doing.
 

Betclever

Active member
#7
You need to move the code you've added to .htaccess up, likely at the top. That said, you've added a ton of code that I can't say I'd really recommend adding unless you understand what it's doing.
Hello Mike, thanks for the suggestion.
In fact, I found this code on this forum where it explains that this code is necessary to add some expire dates on elements but I don't remember which ones cause it's a while I have added this code... :)

Of course, if you are able and if you have few minutes to tell me that this code is unnecessary, it will be sympa from your part so I can remove some of them.

It works for the redirection. Super :)