XF 1.4 Improve SEO - Redirection 301 to avoid duplicate content

[Betclever]

Hello all,

How can I make a redirection to avoid duplicate content cause when I test my website with webrank, it tells me that I have to use a redirection to avoid duplicate content.

In fact, I have access with www. or without www. so how can I redirect this please?

Thanks.

 

[Betclever]

I found a like on xenforo and added this:

User-agent: *
Disallow: /test/
Disallow: /account/
Disallow: /admin.php
Disallow: /ajax/
Disallow: /conversations/
Disallow: /events/birthdays/
Disallow: /events/monthly
Disallow: /events/weekly
Disallow: /find-new/
Disallow: /forums/-/
Disallow: /forums/tweets/
Disallow: /goto/
Disallow: /help/
Disallow: /login/
Disallow: /lost-password/
Disallow: /media/category/
Disallow: /media/keyword/
Disallow: /media/user/
Disallow: /media/service/
Disallow: /media/submit/
Disallow: /misc/style?*
Disallow: /misc/quick-navigation-menu?*
Disallow: /online/
Disallow: /pages/conduct/
Disallow: /pages/privacy/
Disallow: /posts/
Disallow: /threads/tera-tweet-from-*
Disallow: /wiki/special/
Allow: /

I just removed 3 things like test, tera-tweet and wiki but I got a question:

If "allow" is empty, is my content indexed or shall I add something?
In fact, I don't want to loose the indexation so when I see Disallow for forums and posts, what will happen then if it stays like on the list?

Thanks.

 

[popowich]

The robots.txt above isn't for fixing your site working as both www and non-www

This is what you want in your .htaccess to force the www version of your site

# Force the site to www
RewriteEngine On
RewriteCond %{HTTP_HOST} !^www\.example\.com$
RewriteRule (.*) http://www.example.com/$1 [L,R=301]
# end force site to www

You'll need to replace exmaple.com with your domain name

 

[Betclever]

The robots.txt above isn't for fixing your site working as both www and non-www

This is what you want in your .htaccess to force the www version of your site

# Force the site to www
RewriteEngine On
RewriteCond %{HTTP_HOST} !^www\.example\.com$
RewriteRule (.*) http://www.example.com/$1 [L,R=301]
# end force site to www

You'll need to replace exmaple.com with your domain name

Yes, I'm sorry, wrong thread.

I added your code but nothing changed.

I think, I'm using the second one so bet-clever.com but the www. version works too.

Please find my htaccess:

# Mod_security can interfere with uploading of content such as attachments. If you
# cannot attach files, remove the "#" from the lines below.
#<IfModule mod_security.c>
# SecFilterEngine Off
# SecFilterScanPOST Off
#</IfModule>

ErrorDocument 401 default
ErrorDocument 403 default
ErrorDocument 404 default
ErrorDocument 500 default

<IfModule mod_rewrite.c>
RewriteEngine On

# If you are having problems with the rewrite rules, remove the "#" from the
# line that begins "RewriteBase" below. You will also have to change the path
# of the rewrite to reflect the path to your XenForo installation.
#RewriteBase /xenforo

# This line may be needed to enable WebDAV editing with PHP as a CGI.
#RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]

RewriteCond %{REQUEST_FILENAME} -f [OR]
RewriteCond %{REQUEST_FILENAME} -l [OR]
RewriteCond %{REQUEST_FILENAME} -d
RewriteRule ^.*$ - [NC,L]
RewriteRule ^(data/|js/|styles/|install/|favicon\.ico|crossdomain\.xml|robots\.txt) - [NC,L]
RewriteRule ^.*$ index.php [NC,L]
</IfModule>

# Author: Final Kaoss (aka: Extreme-Gaming)
# Version 2.6
RewriteEngine on

#Deny attempts to view the Htaccess file.
<Files .htaccess>
Order allow,deny
Deny from all
</Files>
# BEGIN Browser Caching/Headers
<IfModule mod_mime.c>
AddType text/css .css
AddType text/richtext .rtf .rtx
AddType image/svg+xml .svg .svgz
AddType text/plain .txt
AddType text/xsd .xsd
AddType text/xsl .xsl
AddType video/asf .asf .asx .wax .wmv .wmx
AddType video/avi .avi
AddType image/bmp .bmp
AddType application/java .class
AddType video/divx .divx
AddType application/msword .doc .docx
AddType application/x-msdownload .exe
AddType image/gif .gif
AddType application/x-gzip .gz .gzip
AddType image/x-icon .ico
AddType image/jpeg .jpg .jpeg .jpe
AddType application/vnd.ms-access .mdb
AddType audio/midi .mid .midi
AddType video/quicktime .mov .qt
AddType audio/mpeg .mp3 .m4a
AddType video/mp4 .mp4 .m4v
AddType video/mpeg .mpeg .mpg .mpe
AddType application/vnd.ms-project .mpp
AddType application/vnd.oasis.opendocument.database .odb
AddType application/vnd.oasis.opendocument.chart .odc
AddType application/vnd.oasis.opendocument.formula .odf
AddType application/vnd.oasis.opendocument.graphics .odg
AddType application/vnd.oasis.opendocument.presentation .odp
AddType application/vnd.oasis.opendocument.spreadsheet .ods
AddType application/vnd.oasis.opendocument.text .odt
AddType audio/ogg .ogg
AddType application/pdf .pdf
AddType image/png .png
AddType application/vnd.ms-powerpoint .pot .pps .ppt .pptx
AddType audio/x-realaudio .ra .ram
AddType application/x-shockwave-flash .swf
AddType application/x-tar .tar
AddType image/tiff .tif .tiff
AddType audio/wav .wav
AddType audio/wma .wma
AddType application/vnd.ms-write .wri
AddType application/vnd.ms-excel .xla .xls .xlsx .xlt .xlw
AddType application/zip .zip
</IfModule>
<IfModule mod_expires.c>
ExpiresActive On
ExpiresByType text/css A2628000
ExpiresByType text/richtext A3600
ExpiresByType image/svg+xml A3600
ExpiresByType text/plain A3600
ExpiresByType text/xsd A3600
ExpiresByType text/xsl A3600
ExpiresByType video/asf A2628000
ExpiresByType video/avi A2628000
ExpiresByType image/bmp A2628000
ExpiresByType application/java A2628000
ExpiresByType video/divx A2628000
ExpiresByType application/msword A2628000
ExpiresByType application/x-msdownload A2628000
ExpiresByType image/gif A2628000
ExpiresByType application/x-gzip A2628000
ExpiresByType image/x-icon A2628000
ExpiresByType image/jpeg A2628000
ExpiresByType application/vnd.ms-access A2628000
ExpiresByType audio/midi A2628000
ExpiresByType video/quicktime A2628000
ExpiresByType audio/mpeg A2628000
ExpiresByType video/mp4 A2628000
ExpiresByType video/mpeg A2628000
ExpiresByType application/vnd.ms-project A2628000
ExpiresByType application/vnd.oasis.opendocument.database A2628000
ExpiresByType application/vnd.oasis.opendocument.chart A2628000
ExpiresByType application/vnd.oasis.opendocument.formula A2628000
ExpiresByType application/vnd.oasis.opendocument.graphics A2628000
ExpiresByType application/vnd.oasis.opendocument.presentation A2628000
ExpiresByType application/vnd.oasis.opendocument.spreadsheet A2628000
ExpiresByType application/vnd.oasis.opendocument.text A2628000
ExpiresByType audio/ogg A2628000
ExpiresByType application/pdf A2628000
ExpiresByType image/png A2628000
ExpiresByType application/vnd.ms-powerpoint A2628000
ExpiresByType audio/x-realaudio A2628000
ExpiresByType application/x-shockwave-flash A2628000
ExpiresByType application/x-tar A2628000
ExpiresByType image/tiff A2628000
ExpiresByType audio/wav A2628000
ExpiresByType audio/wma A2628000
ExpiresByType application/vnd.ms-write A2628000
ExpiresByType application/vnd.ms-excel A2628000
ExpiresByType application/zip A2628000
</IfModule>
<IfModule mod_deflate.c>
<IfModule mod_setenvif.c>
BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4\.0[678] no-gzip
BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
BrowserMatch \bMSI[E] !no-gzip !gzip-only-text/html
</IfModule>
<IfModule mod_headers.c>
Header append Vary User-Agent env=!dont-vary
</IfModule>
<IfModule mod_filter.c>
AddOutputFilterByType DEFLATE text/css application/x-javascript text/html text/richtext image/svg+xml text/plain text/xsd text/xsl text/xml image/x-icon
</IfModule>
</IfModule>
<FilesMatch "\.(css|CSS|)$">
<IfModule mod_headers.c>
Header set Pragma "public"
Header append Cache-Control "public, must-revalidate, proxy-revalidate"
</IfModule>
FileETag MTime Size
<IfModule mod_headers.c>
Header set X-Powered-By "Step by Step guide to speed up your VB4 (forum only)"
</IfModule>
</FilesMatch>
<FilesMatch "\.(rtf|rtx|svg|svgz|txt|xsd|xsl|RTF|RTX|SVG|SVGZ|TXT|XSD|XSL|)$">
<IfModule mod_headers.c>
Header set Pragma "public"
Header append Cache-Control "public, must-revalidate, proxy-revalidate"
</IfModule>
FileETag MTime Size
<IfModule mod_headers.c>
Header set X-Powered-By "Step by Step guide to speed up your VB4 (forum only)"
</IfModule>
</FilesMatch>
<FilesMatch "\.(asf|asx|wax|wmv|wmx|avi|bmp|class|divx|doc|docx|exe|gif|gz|gzip|ico|jpg|jpeg|jpe|mdb|mid|midi|mov|qt|mp3|m4a|mp4|m4v|mpeg|mpg|mpe|mpp|odb|odc|odf|odg|odp|ods|odt|ogg|pdf|png|pot|pps|ppt|pptx|ra|ram|swf|tar|tif|tiff|wav|wma|wri|xla|xls|xlsx|xlt|xlw|zip|ASF|ASX|WAX|WMV|WMX|AVI|BMP|CLASS|DIVX|DOC|DOCX|EXE|GIF|GZ|GZIP|ICO|JPG|JPEG|JPE|MDB|MID|MIDI|MOV|QT|MP3|M4A|MP4|M4V|MPEG|MPG|MPE|MPP|ODB|ODC|ODF|ODG|ODP|ODS|ODT|OGG|PDF|PNG|POT|PPS|PPT|PPTX|RA|RAM|SWF|TAR|TIF|TIFF|WAV|WMA|WRI|XLA|XLS|XLSX|XLT|XLW|ZIP)$">
<IfModule mod_headers.c>
Header set Pragma "public"
Header append Cache-Control "public, must-revalidate, proxy-revalidate"
</IfModule>
FileETag MTime Size
<IfModule mod_headers.c>
Header set X-Powered-By "Step by Step guide to speed up your VB4 (forum only)"
</IfModule>
</FilesMatch>
# End Browser Caching/Headers

# Force the site to www
RewriteEngine On
RewriteCond %{HTTP_HOST} !^www\.bet-clever\.com$
RewriteRule (.*) http://www.bet-clever.com/$1 [L,R=301]
# end force site to www

You see on the bottom of the code that I added your code but changed to the url of my forum but the rule doesn't work so when I go to http://bet-clever.com, it doesn't redirect to http://www.bet-clever.com

What's wrong?

PS: I'm using shorter URL by Waindigo and rewrite rules under the CP.

 

[Betclever]

up

 

[Mike]

You need to move the code you've added to .htaccess up, likely at the top. That said, you've added a ton of code that I can't say I'd really recommend adding unless you understand what it's doing.

 

[Betclever]

You need to move the code you've added to .htaccess up, likely at the top. That said, you've added a ton of code that I can't say I'd really recommend adding unless you understand what it's doing.

Hello Mike, thanks for the suggestion.
In fact, I found this code on this forum where it explains that this code is necessary to add some expire dates on elements but I don't remember which ones cause it's a while I have added this code...

Of course, if you are able and if you have few minutes to tell me that this code is unnecessary, it will be sympa from your part so I can remove some of them.

It works for the redirection. Super