Traffic Down Since VB to XF Migration

[Alfuzzy]

Site was migrated from vBulletin 4 to Xenforo 2.1 approximately 6 months ago...and traffic is down at least 50% or more. This traffic decrease was seen almost immediately after the migration (clearly something to do with the migration)...but figured things needed to settle in before investigating & getting concerned.

With the site running on vB4...current visitor numbers to the site at any given time were in the 1000's. Most of these I know were guest visitors & bots...but registered members as well. When current visitor numbers are checked after the migration to XF...rarely does the total number exceed 12-14 (members & guests).

Of course with traffic seriously decreased...site revenue is way down as well. The site previously mainly relied on the heavy guest visitor traffic for site revenue (advertising impressions). Also have an account with a company called "Skimlinks"...where you get passive income when visitors visit or purchase from any links you have on your site. Income from Skimlinks is way down as well...which obviously again reflects the large decrease in daily traffic.

When the site was migrated from vB4 to XF 2.1...the following add-on's were installed to facilitate the transition:

• Dragonbyte SEO Redirects (vB site had Dragonbyte SEO installed).
• XenForo Redirects for vBulletin

As mentioned...the large traffic decrease was noticed immediately after the vB to XF migration. Anyone have any ideas what could have happened & how things might be corrected?:

• Maybe an add-on product needs to be installed?
• Something to do with the .htaccess or robots file?
• Maybe a Xenforo setting of some sort?

Thanks much for any ideas.

 

[Ozzy47]

What are you using to gather these statistics? Is it Google Analytics

A swap to Xenforo itself shouldn't create any drop in legitimate traffic (that wasn't from senseless page views and unnecessary refreshes to begin with) as long as you did proper redirects, as all old traffic can still get to new locations, and with proper robots.txt, as well as site indices, Google and other search engines will re-index your site rather quickly - we're talking days here, not months.

 

[StarArmy]

Maybe your users are still getting used to the new layout. I suggest pumping up the content and events and give people more reasons to participate, basically create a surge of awesome new stuff on your forum. Also you should, if you haven't already, create some tutorials to show people how to use the new forum.

 

[Alfuzzy]

Using a few different sources to determine the drop in traffic:

• Google Analytics
• Advertisement view statistics (ad impressions) from account dashboards.
• "Members on Site" statistics (from the site itself)...especially guest visitors. I know this isn't always super accurate...but super big difference from vB4 to XF 2.1.

I'd be happy to take screenshots of pertinent portions of robots & .htaccess files if that would help (or PM them).

I think the biggest drop in traffic is much much fewer guest visitors from the vB4 site to the XF migrated site. With way fewer guest visitors...that means less ad impressions...thus less ad revenue/month.

Thanks

 

[Alfuzzy]

Maybe your users are still getting used to the new layout. I suggest pumping up the content and events and give people more reasons to participate, basically create a surge of awesome new stuff on your forum. Also you should, if you haven't already, create some tutorials to show people how to use the new forum.

This of course is always a great idea...to keep a site growing...keep a site interesting...and keep a site relevant.

The problem the site is having is guest visitor statistics are way down after the migration from vB4 to XF (been 6 months)...it's very easy to see. The issue is not necessarily interesting or fresh content...but a sudden drop in guest visitors.

Thanks

 

[djbaxter]

This of course is always a great idea...to keep a site growing...keep a site interesting...and keep a site relevant.

The problem the site is having is guest visitor statistics are way down after the migration from vB4 to XF (been 6 months)...it's very easy to see. The issue is not necessarily interesting or fresh content...but a sudden drop in guest visitors.

Thanks

This raises the question of how many of those guest visitors were bots that haven't yet founds your new forum and are too "dumb" to follow the redirects. In any case, guest visitors aren't usually helpful in generating revenue and obviously don't generate content on most forum configurations. I personally wouldn't worry about that metric. On many busy forums, admins look for ways to block most bots since all they do is consume bandwidth and in some cases lock database tables.

 

[Masetrix]

This of course is always a great idea...to keep a site growing...keep a site interesting...and keep a site relevant.

The problem the site is having is guest visitor statistics are way down after the migration from vB4 to XF (been 6 months)...it's very easy to see. The issue is not necessarily interesting or fresh content...but a sudden drop in guest visitors.

Thanks

Try GSC...
Have a look at:
https://search.google.com/search-console/index?resource_id=https://yourdomain.tld

 

[Alfuzzy]

This raises the question of how many of those guest visitors were bots that haven't yet founds your new forum and are too "dumb" to follow the redirects.
In any case, guest visitors aren't usually helpful in generating revenue and obviously don't generate content on most forum configurations. I personally wouldn't worry about that metric. On many busy forums, admins look for ways to block most bots since all they do is consume bandwidth and in some cases lock database tables.

Honestly...would have to be a awful lot of bots. When site was running on vB4...could look at the visitor statistics...and see anywhere from 3k-7k visitors at any one time (and that was a lower number than years gone by). That's a ton of ad impressions/day/week/month.

Someone could argue that vB4 may not have done the best job reporting visitor statistics. At this point I'd take 10% of that number!

 

[Alfuzzy]

Try GSC...
Have a look at:
https://search.google.com/search-console/index?resource_id=https://yourdomain.tld

I have a Google Search Console account...what would you suggest looking at specifically?

Thanks

 

[djbaxter]

Honestly...would have to be a awful lot of bots. When site was running on vB4...could look at the visitor statistics...and see anywhere from 3k-7k visitors at any one time (and that was a lower number than years gone by). That's a ton of ad impressions/day/week/month.

Someone could argue that vB4 may not have done the best job reporting visitor statistics. At this point I'd take 10% of that number!

I have worked on two different sites in the past year alone where large numbers of bots repeatedly crashed the database. As for ad impressions, most ads these days don't pay for impressions, only click.

 

[Alfuzzy]

As for ad impressions, most ads these days don't pay for impressions, only click.

Sure they do...I have accounts with at least 5-7 ad providers/ad networks...and all of them pay for ad impressions (Adsense being one of them). "Clicks" of course pay better than impressions...but ad impressions definitely provide site revenue (Ad RPM). Unfortunately ad revenue is down for just about everyone.

Maybe we're talking about different things.

 

[Noobz347]

You may want to browse through your console and ensure that the bots and spiders have access to the same number of forums that they had access to when you ran your site over vB.

It's been many years since I ran vB but I can tell you that one of the big reasons why we switched is because vB 'leaked' like a sieve. What I mean is: It wasn't very good (at the time) of keeping private forums away from those account groups that did not have permissions to be there.

It is conceivable that you have forums/content under XF that are 'private' to non-credentialed accounts that used to have access to under vB.

Just a thought. You may have already explored this possibility. When it occurs to me, I run dummy accounts through the forums that belong to different user groups (including those without an account) just so that I can see what is truly accessible and what is not. Just check to ensure that all of the content you intend to be visible to bots is actually still visible to bots.

 

[Alfuzzy]

You may want to browse through your console and ensure that the bots and spiders have access to the same number of forums that they had access to when you ran your site over vB.

Are you talking Google Search Console...or somewhere in the XF Admin area (I'm unsure)? Once there...how would I check for Bot & Spider access?

It's been many years since I ran vB but I can tell you that one of the big reasons why we switched is because vB 'leaked' like a sieve. What I mean is: It wasn't very good (at the time) of keeping private forums away from those account groups that did not have permissions to be there.

It is conceivable that you have forums/content under XF that are 'private' to non-credentialed accounts that used to have access to under vB.

Didn't have any issues with vBulletin in this respect. Only totally Private area was the staff area...there were a few areas that were promotions from the Registered user group...but these worked fine as well.

Just a thought. You may have already explored this possibility. When it occurs to me, I run dummy accounts through the forums that belong to different user groups (including those without an account) just so that I can see what is truly accessible and what is not. Just check to ensure that all of the content you intend to be visible to bots is actually still visible to bots.

I do the same thing. Especially handy when a member in a specific user group says they don't have access to something...then I log in as a member of that user group (with the dummy account)...to see what's up.

Thanks

 

[arn]

Site was migrated from vBulletin 4 to Xenforo 2.1 approximately 6 months ago...and traffic is down at least 50% or more. This traffic decrease was seen almost immediately after the migration (clearly something to do with the migration)...but figured things needed to settle in before investigating & getting concerned.

With the site running on vB4...current visitor numbers to the site at any given time were in the 1000's. Most of these I know were guest visitors & bots...but registered members as well. When current visitor numbers are checked after the migration to XF...rarely does the total number exceed 12-14 (members & guests).

Of course with traffic seriously decreased...site revenue is way down as well. The site previously mainly relied on the heavy guest visitor traffic for site revenue (advertising impressions). Also have an account with a company called "Skimlinks"...where you get passive income when visitors visit or purchase from any links you have on your site. Income from Skimlinks is way down as well...which obviously again reflects the large decrease in daily traffic.

When the site was migrated from vB4 to XF 2.1...the following add-on's were installed to facilitate the transition:

• Dragonbyte SEO Redirects (vB site had Dragonbyte SEO installed).
• XenForo Redirects for vBulletin

As mentioned...the large traffic decrease was noticed immediately after the vB to XF migration. Anyone have any ideas what could have happened & how things might be corrected?:

• Maybe an add-on product needs to be installed?
• Something to do with the .htaccess or robots file?
• Maybe a Xenforo setting of some sort?

Thanks much for any ideas.

An immediate drop sounds more like a config issue, or analytics measuring issue. But with the corresponding drop ad revenue and guest numbers, it's prob legit.

• Look at Google analytics. Verify that you lost primarily Organic Search traffic vs direct traffic. Graph both and see which dropped and when.
• Go to Google Search Console -> Settings -> Crawl Stats. What is the % of "By Response". Most should be 200, but you see a lot of 404, then that's a problem.

My best guess is you broke redirects so Google has dropped those results in search results.

 

[briansol]

traffic or page views?

xf uses more ajax which cuts down on total page views.

how do you uniques compare?

 

[Alfuzzy]

Thanks very much Arn for jumping in & helping with some ideas!

• Look at Google analytics. Verify that you lost primarily Organic Search traffic vs direct traffic. Graph both and see which dropped and when.

Can you please tell me where I would find this data (organic search traffic & direct traffic)...I'm not 100% sure where to find it & what value to use.

• Go to Google Search Console -> Settings -> Crawl Stats. What is the % of "By Response". Most should be 200, but you see a lot of 404, then that's a problem.

Ok...went to Google Search Console (Settings->Crawl Stats)...and here are the various "By Response" percent's. The 404 percent value is not bad at all...was wondering if any of the other values look troublesome?

By Response:

• Other Client Error (4xx) = 46%
• Ok (200) = 31%
• Moved Permanently (301) = 20%
• Not Found (404) = 2%
• Moved Temporarily (302) = 1%

Any other Google Analytics or Google Search Console values I should investigate?

My best guess is you broke redirects so Google has dropped those results in search results.

This was one theory I had...maybe there was an issue with redirects or maybe backlinks were affected when site was migrated from vB to XF.

Is there some way I can check on "broken redirects"...and if this is an issue...how to fix it?

Thanks very much for the help!

 

[Alfuzzy]

traffic or page views?

I'm usually looking at Page Views...where would I find the "traffic" number?

how do you uniques compare?

Where do I find this value? I know it's an important number...but I'm never quite sure where to look for it?

Thanks

 

[Masetrix]

@Alfuzzy

Other Client Error (4xx) = 46%

46% is a lot.
That's almost half of your page. Here you should scan the individual pages and see which 4xx codes are in the game.

HTTP Status Codes - 4xx

This group of HTTP status codes indicates that the request for the resource contains bad syntax or cannot be filled for some other reason, presumably

www.websitepulse.com

404 == broken link or page does not exist / deleted (which is almost the same)
401 /403 == These errors often arise at XenForo in connection with links that were still accessed as a guest at vB, but no longer at XF. E.g. members who restrict or have completely banned access to their profile, downloads that are only available for registered users, but are also visible to guests.

413 and 414 == such errors occur when converting links where entire text passages become links.

That is what I have experienced so far after the migration.

Hth...

 

[Alfuzzy]

@Alfuzzy
46% is a lot.

I guess that's good news & bad news. Lol

Good something found to investigate...bad that the issue exists.

404 == broken link or page does not exist / deleted (which is almost the same)
401 /403 == These errors often arise at XenForo in connection with links that were still accessed as a guest at vB, but no longer at XF. E.g. members who restrict or have completely banned access to their profile, downloads that are only available for registered users, but are also visible to guests.

413 and 414 == such errors occur when converting links where entire text passages become links.

That is what I have experienced so far after the migration.

Which specific tool from the link kindly provided should I use to scan/test with?

Thanks very much!

 

[Masetrix]

If you open a topic in the search console, e.g. cover, you will see in my language:


If you click on an error type here, further details will open. A graphic shows you how many errors occur and below it links to individual incidents. Leave the mouse on such a link and a popup will show you a magnifying glass so that you can scan the link, you can open the link in a new browser tab, or copy the link to the clipboard. If you click on such a link directly, a window with further information almost always opens on the right ...
The whole applies to each area from the main overview ...

The topic is extensive and, depending on the situation, difficult. Therefore, you should deal with the GSC. Here is a link ...

How to Use Google Search Console: GSC for Beginners

Are you ready to get serious about blogging? Then you're ready to learn how to use Google Search Console!Pinterest is an incredible tool for blogging, isn't it?But at 250 million monthly users, Pinterest receives just

nikkiblogs.com