I recommend using a robots.txt in your site root to prevent Google from indexing those URLs. It seems you are not using any robots.txt rules right now: http://www.sportgfx.com/robots.txtI am seeing urls with /posts/ being indexed by Google instead of the actual thread url. The /posts/ is just a redirect.
Do you have your attachments viewable to guests or are you using one of the add-ons to allow them access to the images?I don't want that to happen. Google Images provides good traffic. xenForo also doesn't have the asterisk so it looks like they are just blocking the root url?
I use this: http://xenforo.com/community/resources/tinhte-image-attachment-optimization-cdn-support.909/Do you have your attachments viewable to guests or are you using one of the add-ons to allow them access to the images?
I may break down and use that one as well; I was working on some custom code as part of an existing add-on I created for a different purpose will likely say 'heck with it' and use that one from Dinh.
What is the thought on including the 'Allow: /' entry? My first reaction was what this person responded with...
We've had great success by checkingAllowfirst, and then checkingDisallow, the idea being thatAllowwas intended to be more specific thanDisallow. That's because, by default (i.e. in the absence of instructions to the contrary), all access is allowed. But then we run across something like this:
User-agent: * Disallow: /norobots/ Allow: /
The intent here is obvious, but thatAllow: /will cause a bot that checksAllowfirst to think it can crawl anything on the site.
I used to use it but if xenForo is using a correct 301 and it is still being indexed then something else needs to be figured out here. Google isn't perfect and Matt Cutts is available to talk to here. Something isn't right. These urls are always 301 redirected correct? There is no circumstance where these urls exist where there is not a 301 tagged to them? Therefore, they should not be indexed at all and the correct url, with the canonical tag on it, should be indexed.Regardless of Google hitting /posts/ or not, it should be redirecting using a 301 Moved Permanently header, which Google (and other engines) should respect and update their indexes accordingly.
Have you attempted the robots.txt entry?