1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

XF 1.1 Significant drop in traffic since vB switch - hasn't recovered

Discussion in 'Troubleshooting and Problems' started by mcadx, Nov 16, 2012.

  1. mcadx

    mcadx Member

    It's been about 9-10 months since we made the switch from vB4 to XF. We expected a significant drop in traffic since the search engines would have to re-index everything but it has not recovered. In fact, guest traffic has continued to drop. In the final days before the cutover, we had some 45,000 pages indexed with Google. Today we have less than 30,000. The site hasn't shrunk - we have over 40,000 threads and the number of posts continues to grow on a regular basis as member activity has been about the same if not a bit higher but search traffic is a mere fraction of what it used to be. We have a sitemap generator installed and there are no errors to speak of in Google Webmaster Tools and our PageRank has not dropped.

    Mods/add-ons are minimal at less than 10 so I have a feeling that we may have some settings that aren't correct. What things could we look at to make sure that everything is being indexed as it should?

  2. craigiri

    craigiri Well-Known Member

    If your site is easily crawled by a human, then google should be able to find all the pages.

    As a test - are you saying that you can grab some text from one of the 15,000 pages (not indexed) and do a google search and not find it? It may be that google has changed some of their crawling and indexing methods - and/or you have set the amount of threads per page differently (which would result in fewer pages)....

    I assume you did the basic redirects after you moved?

    My site is fairly vanilla and we lost virtually no search mojo in our move about 9 months ago - and, in fact, are gaining in visits and visitors.

    Assuming that guests can view most everything in your forums, I don't think there are too many settings on your end to fuss with.
  3. mcadx

    mcadx Member

    Yes, the redirects are in place and guests can easily navigate the site. One thing I didn't mention in the OP is that at the time of upgrade we changed the structure. The forums used to be in the root whereas Wordpress now lives in the root and the forum in /community. Either way, we placed the redirect script into the root and updated it to reflect the new location of the forums.

    As you can see, it's indexing new content that has been added but that drop is significant.

    Screen Shot 2012-11-16 at 12.52.34 PM.png
  4. hellreturn

    hellreturn Active Member

    Do you have higher 404 or 403 in your google webmaster ?
  5. mcadx

    mcadx Member

    Only have 154 404 errors. I don't see any 403. We do have some errors in the last few weeks due to my host getting hacked so I imagine those will clear up in due time.
  6. mcadx

    mcadx Member

    Just kidding. There are a lot of 403s but that's due to guests being prohibited from viewing full size images/attachments. That's no different than before with vB, however.

    EDIT: Scrolled through about 1,000 403 entries and only 2 were links to content that should've been visible to the bots. Marked them as fixed. The others were all attachment pages that guests don't have access to.
  7. psTubble27

    psTubble27 Well-Known Member

    Are the redirects (espescially from root to /community) permanent or temporary?
  8. mcadx

    mcadx Member

    Ok, as I'm digging deeper I found a bunch of 404s that seem to indicate that perhaps my redirect script isn't working properly and seems to have spiked significantly about 3 weeks ago. Looking at the URLs, they're vB URLs and not Xenforo. Though if I do a Google site search for one of the threads listed under the 404 list, it comes up with the updated page. Could there be conflicting caches of my site in Google?
  9. mcadx

    mcadx Member

    The redirect is Kier's 301 redirect script so it should be permanent. The file has been there since the day of the cutover.
  10. mcadx

    mcadx Member

    In doing some more digging, it seems that right around this time is when Apache was upgraded to 2.2.23 and screwed up my .htaccess rewrite rules that were created based on this tool: http://tools.geekpoint.net/xfseo/

    Here's what my .htaccess rewrite rules look like currently

    Safe to say that the problem is in here somewhere? Any guesses?
  11. CyclingTribe

    CyclingTribe Well-Known Member

    My CycleChat site shows a similar downward total of indexed pages but we're busier than ever. I suspect because we no longer allow user profile pages or individual post URLs to be indexed (and our separate .co.uk domain is now used as an internal CDN URL for static content) so there's less "volume" (down from 685,000 indexed pages to 190,000 or so) but more quality as the content that is indexed (mainly threads/first posts) is keyword and topic relative to our niche - cycling.

    What has been the trend for your bounce rate and visitor/return ratio over this period?

    Also, from within your XF ACP - what does your post count stats graph look like for the past 2 years?

    Shaun :D
    Digital Doctor likes this.
  12. Jarod

    Jarod Active Member


    Have you a same graph(Google analytics) with visits or page views since Vb4 switch ? .

  13. mcadx

    mcadx Member

    Bounce rate seems to be relatively steady in the same time period. Lower after the upgrade for about 2 months(I'm guessing because users were scoping out the new setup), settling off for a while and then trending higher since the above said Apache upgrade last month.

    Roughly 40% are new visitors since the upgrade, YTD.

    Post count graph seems to be slowly trending upwards, as it should.
  14. mcadx

    mcadx Member

    Screen Shot 2012-11-16 at 1.57.47 PM.png
    Jarod likes this.
  15. CyclingTribe

    CyclingTribe Well-Known Member

    What are page views for the same period?
  16. mcadx

    mcadx Member

    Used to be around 100k / month. Post migration it's about half that.
    Screen Shot 2012-11-16 at 2.55.46 PM.png
    Jarod likes this.
  17. Jake Bunce

    Jake Bunce XenForo Moderator Staff Member

    Page views always go down. XenForo is more efficient with page loads. For example, AJAX replies remove one page view. This is not a problem.
    mcadx and 0xym0r0n like this.
  18. mcadx

    mcadx Member

    That's good info, thanks.

    Any suggestions for the htaccess rewrite rules to reduce those errors?
  19. Jake Bunce

    Jake Bunce XenForo Moderator Staff Member

    Which errors for which pages specifically? If they are old vB links then they would either be redirected or turned to 404s.
    LaxmiSathy likes this.

Share This Page