Search engines, duplicate content and 301 redirects

[webbouk]

We moved over to xenForo 3 months ago from a Snitz based forum which at the time had many search engine results referencing a lot of content.
The Snitz forums were located in a directory called 'forum', the database for this was migrated and the xenForo files were installed in a directory named 'forums' and a large very obvious banner was placed at the top of the original forum directing and instructing people of the changeover and the location of the new, which in the main seemed to work well. The Snitz forum was then locked down to prevent new posts, replies, and registrations.

Now three months later I was going to use a Http 301 redirect to take visitors from the old 'forum' directory to the new 'forums' which would have been the end of the transition and would have aided SEO by removing links to duplicate content.
However, within the old forum directory structure there are thousands of images located in various directories so a carpet http redirect of the old 'forum' directory would break any links to these images.


For now I have modified the robots.txt file to disallow all from the previous '/forum/' directory and to allow all to the new '/forums/' directory which should in theory stop search engines from indexing the old content, but is this the best way of doing it?

 

[Mr Lucky]

For now I have modified the robots.txt file to disallow all from the previous '/forum/' directory and to allow all to the new '/forums/' directory which should in theory stop search engines from indexing the old content, but is this the best way of doing it?

I would not trust the robots.txt, instead put a robots meta direction in the page header of the old forum.

<meta name="robots" content="noindex, nofollow" />

I think I would have done this differently and just deleted the old forum files and database and installed XenForo in that directory. That way the image links would remain.

The 301 is probably best, but if you can redirect pages/threads/posts of Snitz (not sure what that is though) to XenForo but without the image folders being redirected. That may be quite complicated depending on Snitz.

 

[webbouk]

I looked at the possibility of using redirect rules to redirect the old Snitz links to xenforo but there was no similarity and it looked to be an impossible task.
Good tip re the nofollow in the header.

 

[Mr Lucky]

Is there any reason to actually keep the old files there at all? For Google to ignore them (which is what you want so it doesn't see duplicate content) I would have thought a 404 is best if you can't do specific page 301s. Or maybe even better a 410.

What happens about any external links there may be on other sites, that are linking in to the old pages though? These should be redirected or updated if possible.

 

[webbouk]

That was one of the main reasons for keeping the old forum files as we have a lot of content that is linked to and it does not seem possible to create a rule to redirect them; changing the links would be a nigh on impossible task as the site has been running since 2001

 

[Mr Lucky]

That was one of the main reasons for keeping the old forum files as we have a lot of content that is linked to and it does not seem possible to create a rule to redirect them; changing the links would be a nigh on impossible task as the site has been running since 2001

I understand the difficulty of a catchall redirect script for something unconventional such as Snitz, although I'm somebody could create a redirection script if you were prepared to pay.

If not, you could you at least find the best links and make individual redirects. Majestic SEO or similar has tools for this I think. Failing that what I would do is check in Google for the most used keyword searches, Google Analytics for most popular content and make the individual 301s for those. Damage limitation at least.

Otherwise your SEO will be harmed when those pages linked to become no index.