Problem with SEO, indexing and getting google to crawl pages

pavelandreev

New member
Hey forum,
I hope this is in the right section since I consider SEO for a forum to be part of managing and growing a forum.

I`ve been having issues with getting a site to index properly.


Currently its divided into two parts.
A main site with static pages, campaign/news pages (details dont matter here) and blog pages.
And the second part a forum with threads, resources and media. (which is on the same domain, just with the /forum prefix)

Now the main site is crawled very well, every day infact based on logs, some pages even every 4 hours.
Its well indexed and appears in search results.
The forum however lags behind, even though its on the exact same domain.
I`ve double and triple checked, the pages in the forum. They are open for indexing, discover-able, crawlable and the bot has passed through them... once, about a month ago.

So what gives? Why is one part of the site indexed and shows in search results, while the other part doesnt show in search results. The pages are in a sort of limbo "Crawled - currently not indexed" or "Discovered - currently not indexed"

Is there are a tool I can use to give me clarity?
I have an active SemRush account if that helps, any tool in it that I can use to analyze?
Has anyone else had this problem with getting google to index their XenForo forum?
 

Miri

Well-known member
Hi, do non-indexed pages have more or less the same text/content as indexed ones (understood in quantity)?
Google tries not to immediately (or ever) index pages with little content.
 

pavelandreev

New member
Hi, do non-indexed pages have more or less the same text/content as indexed ones (understood in quantity)?
Google tries not to immediately (or ever) index pages with little content.
More content than the indexed pages in some cases, not less for sure.
 

AnjaC

Active member
In Google Search Console, look up Index - Pages (mine is in German, so I don`t know the correct english labeling). You see a graph with indexed and non indexed pages. Scroll down. You`ll find a list "Why pages are not indexed". Click on one of the entries. You`ll see a list of affected pages for this particular reason. Work your way through.
Maybe this may help as well:
 

Miri

Well-known member
follow this url:
Code:
https://search.google.com/search-console/index?resource_id=https://website.com/
change website.com with your site
 

pavelandreev

New member
In Google Search Console, look up Index - Pages (mine is in German, so I don`t know the correct english labeling). You see a graph with indexed and non indexed pages. Scroll down. You`ll find a list "Why pages are not indexed". Click on one of the entries. You`ll see a list of affected pages for this particular reason. Work your way through.
Maybe this may help as well:
Unfortunately the inconsistency of their report is why I`m seeking outside help.
The report is mostly useless, conflicting and doesnt provide up to date or accurate data in general.

Here are some examples for a specific page shown in the "Crawled - currently not indexed"

1669215340958.png

And then here is the report for the same page

1669215367423.png

Now granted, this page has nothing worth showing, however its both shown as Indexed and "crawled - currently not indexed"

Here is another example, which is even more difficult to understand:

1669215544224.png


This is from the forum sitemap index report.

1669215584713.png

But when you got to inspect the URL, it shows Sitemaps N/A
Even though this URL was pulled from the Sitemap index coverage report.

------

There are many many more examples like this.
 

pavelandreev

New member
follow this url:
Code:
https://search.google.com/search-console/index?resource_id=https://website.com/
change website.com with your site
Thank you for the suggestion, but I have went over the report and it does not provide any useful information as to why the pages arent indexed OR are indexed but not showing in search results.
 

Miri

Well-known member
So good news.

Let google do its job. You think about the contents. Check the url I sent you from time to time to see various errors. That's all. No one knows (unless you work for google in the correct department) that why google likes one piece of content over another.

I did a search for your page and I have to say your article is on the second page of google.

Инфаркт-симптоми-лечение-и-превенция-Cerca-con-Google.pngIf your article will be liked by readers, slowly google will raise it as high as possible.

Consider the fact that xenforo is great tool but it may not have the same seo advantage as other CMS regarding articles. But that's another story.

Greetings
 

pavelandreev

New member
So good news.

Let google do its job. You think about the contents. Check the url I sent you from time to time to see various errors. That's all. No one knows (unless you work for google in the correct department) that why google likes one piece of content over another.

I did a search for your page and I have to say your article is on the second page of google.

View attachment 276939If your article will be liked by readers, slowly google will raise it as high as possible.

Consider the fact that xenforo is great tool but it may not have the same seo advantage as other CMS regarding articles. But that's another story.

Greetings
What do you think about investing in backlinks?
Would that help with the issues and/or resolve them quicker?
 

Miri

Well-known member
It would be like building the second floor of a building without having started with the first one yet. I rather recommend investing in content for your users.
 
Top