Google Search Console Valid Pages Dropping

Alfuzzy

Active member
When I do a "site:mywebsite.com" in Chrome for my forum (for number of results)...and when I check the "Valid" indexed pages in Google Search Console...these have been slowly dropping for about the last month (when previously these numbers were steady or slowly growing).

My forum has been "normally busy" as usual...with the number of pages viewed/day steady or slightly higher...and have similar number of new threads created/day...or slightly more. Definitely no drops in site traffic or site activity.

Any ideas why these numbers might be slowly/steadily dropping? If additional info is needed...please let me know.

Thanks
 

djbaxter

Well-known member
This has been an issue for a while now:


Google, as usual, makes changes all the time to their algorithms and to their services like Search Console and Google My Business. And generally speaking, when they make a change or try to fix something, it screws up and creates other issues. One of the reasons is the complexity of their algorithms and services. Another is that different departments are responsible for the different services and they rarely or never talk to one another.

My personal opinion is that Google has become so complex in the past few years that we are going to see more and more of this going forward.

Most of the time, this sort of thing gets fixed eventually... Until the next time one of the Googlers breaks something. :)
 

JoyFreak

Well-known member
Yeah something is definitely going on with Google in the past couple of months or so. I’ve seen my ranking get worse. It’s likely to do the their recent update that rolled out around then. The ability to index your pages via the search console has been disabled due to on-going changes that I believe will go on till March 2021. I hope they fix whatever issue that’s causing this soon.
 

Alfuzzy

Active member
Thanks guys...helps me feel a little bit better.

@DJ and anyone else with thoughts/suggestions...a refresher:

A couple months ago (mid-September) I posted a "Optimizing Google Crawl Budget" question here at XF...where via suggestions from that thread...I modified my XF site map settings & modified my robots.txt file.


Before these modifications valid URL's indexed via Google Search Console for my site were slowly growing. Since that time Google Search Console valid indexed URL's have basically dropped to about 40% of what they were in mid-September. Turns out the article DJbaxter linked above was from September 30, 2020 (shortly after I made my changes).

Does it sound like I unfortunately made the changes to my XF sitemap settings & robots.txt file just before the changes Google made...and the drop in valid indexed pages is due to the changes at Google...and not the changes I made?

Thanks
 

Ozzy47

Well-known member
Does it sound like I unfortunately made the changes to my XF sitemap settings & robots.txt file just before the changes Google made...and the drop in valid indexed pages is due to the changes at Google...and not the changes I made?

That is a likely scenario.
 

djbaxter

Well-known member
I doubt that any of your changes to robots.txt were responsible. I think it's almost certainly an issue on Google's end, not on your end.

For one thing, you're certainly not alone in having pages dropped from Google Search in recent months.

Here's more:



 

Alfuzzy

Active member
Thanks for all the great info DJ. I'm just sort of concerned how low will the "Valid Indexed URL's" go before things settle.

For sites that get much of their site traffic from Google search results...if Valid Indexed URL's continue to drop...that means less traffic...and less traffic can lead to less site revenue.

If I do a "site:mywebsite.com" (no quotes) in Chrome...this number is down about 60% since mid-September. And Google Search Console "Valid Indexed URL's...is down about 18% since mid-September. And both values continue to drop.:(
 

djbaxter

Well-known member
Well, for starters when Google experiences issues like this, the reality is that there is nothing at all you can do except wait it out until Google fixes it. I learned a long time ago to set my priorities by asking this question: Is there anything I can do about this issue now, today? If the answer is no (as in the current case), I just move on to the next item on my to-do list and try to put it out of my mind.

Of course, in the case of the site:mywebsite.com search, I vaguely remember an announcement in recent weeks that Google had changed how they serve up results for those search queries. In that case, you can expect the results you get from that search query to be less accurate than what you see in the Search Console (although there have also been several glitches in the Search Console reports recently as well).

Are you seeing these numbers dropping progressively over time now? or is it more or less static after the initial decreases in late September/early November?
 

djbaxter

Well-known member
Here's some information about glitches in the Search Console:



 

Alfuzzy

Active member
Are you seeing these numbers dropping progressively over time now? or is it more or less static after the initial decreases in late September/early November?

Slow steady decrease since late September/Early October (about 60% total decrease now). During this time site traffic/day (Page Views)...new threads created/day...posts/day...have all remained mostly steady. Previous to this time (September)...these numbers were slowly increasing.

I'm guessing normally with these sort of Google modifications...changes are rather immediate & settle out quickly. This happened with the large Google algorithm change back in May (saw a big-sudden decrease in site traffic via Google Analytics). With this Google September change...doesn't seem to be sudden/rapid...but slow-steady decrease.
 

djbaxter

Well-known member
I'm guessing normally with these sort of Google modifications...changes are rather immediate & settle out quickly. This happened with the large Google algorithm change back in May (saw a big-sudden decrease in site traffic via Google Analytics). With this Google September change...doesn't seem to be sudden/rapid...but slow-steady decrease.
No not always. There have been some significant bugs in Google My Business which have been ongoing for several months now. And the complaints about the Search Console glitches go back to May 2020 or earlier.
 

Alfuzzy

Active member
As you mentioned above...if you can't do anything about it...then not much purpose in stressing. Since I wasn't aware of the September Google changes...I assumed my XF sitemap settings & robots.txt file changes were responsible for the drop in numbers (thus my concern).

But now that I know about the Google changes (which are most likely responsible)...now I can move on & concentrate on stuff that I can impact. Thanks:)
 
Top