XF 1.5 New threads not being indexed by Google?

[Amin Sabet]

For my site: https://www.mu-43.com

It used to be that if I were to Google a unique or unusual thread name a few hours after it was posted, that thread would come up in the search results every time, without fail.

These days, threads generally do not come up in a search, even if I wait a few days and the thread has many replies.

Take for example this thread: https://www.mu-43.com/threads/deet-based-bug-spray-around-camera-equipment-bags.86372/

A search for

"Deet-based bug spray around camera equipment/bags..."

doesn't return any results.

This also returns nothing, indicating that the thread is not indexed:

site:mu-43.com  Deet-based bug spray around camera equipment/bags

I've tried Googling a lot of other threads that way, and I come up with links to the forum index where threads were linked at one time or another, or page 11 of a thread which I am searching by title, but the actual threads themselves (first page) aren't coming up like they used to.

Here's what my Sitemap options look like:



There are no messages or errors showing in Google Webmaster tools, and the sitemap seems to be fine:



My site's direct traffic has held steady over time:



But Google traffic has steadily declined:




Am I missing something? Is there a better way to submit new thread URLs for indexing?

 

[Mike]

Note that sitemaps aren't necessarily generated every day (it's M/W/F by default, I believe). That said, it is still indexing some things as I can see this thread even though it was created later: https://www.mu-43.com/threads/ordered-an-e-m1-for-799€-is-it-a-good-price.86385/

Is Google showing a downward trend in the amount of crawling it's doing?

 

[Amin Sabet]

Note that sitemaps aren't necessarily generated every day (it's M/W/F by default, I believe). That said, it is still indexing some things as I can see this thread even though it was created later: https://www.mu-43.com/threads/ordered-an-e-m1-for-799€-is-it-a-good-price.86385/

Is Google showing a downward trend in the amount of crawling it's doing?

I think so:



Maybe Google just considers my content lower quality than before? Is there any other possible explanation?

 

[James]

I've not used Google webmaster tools for a while but cant you upload a site map and/or ask it to do a re-crawl?

 

[Amin Sabet]

I've not used Google webmaster tools for a while but cant you upload a site map and/or ask it to do a re-crawl?

Yes, GWT has my sitemap there and processes it 3 times a week.

I used to keep recent threads in the sidebar but stopped doing that. I'm going to try putting them back to see if that helps.

 

[Amin Sabet]

My best guess is that I've had too many affiliate links on my site. Was using an addon to automatically convert certain phrases to affiliate links. Probably incurred a Panda penalty. Have removed that addon. Fingers crossed!

 

[SuperCotilla]

The only indexation my site have after Xenforo are the forum members, maybe once in a while a very specific thread, if you run the site: etc nothing comes up but the members one by one. My site is more than 14 years old.

All I can say is that although Xenforo is a great platform it lacks presence in Google and indexation, without that we are doomed.

Not cool.

 

[Yogesh Sarkar]

I have been noticing this issue as well with our forum. Though with switch HTTPS, I guess it has been impacted even more. I am not sure why, even though we have over 39k threads visible to guests and threads even spanning several hundred pages, GWT only shows less than 39k content in submitted sitemap, even though I have the same setting as you!

 

[Robru]

doesn't return any results.

 

[Mr Lucky]

And in Duck Duck Go (my favourite search engine)