IndexNow: URLs compatibility with Bing

[masmo]

Affected version

2.3

I have been using this add-on for a long time and I've never had any problems with Bing, discussions were always indexed quickly, for the past 3 weeks I have been trying IndexNow integrated into 2.3 and the problems have started...

I didn't understand initially why they weren't indexed, I thought it was a robots.txt problem with Disallow: /posts/
https://xenforo.com/community/threa...content-in-search-engines.199574/post-1695084

After removing "Disallow: /posts/" from robots.txt I realized that there is also a redirect problem and in the Bing guideline it is written that the links sent must not be redirects.




With Indexnow integrated no discussions were indexed in 3 weeks, reactivating the previous add-on they were indexed in 48 hours.

Note: since most XenForo installations, as also suggested here, use "Disallow: /posts/" in robots.txt it would be recommended to change the link used by IndexNow.

 

[Old Nick]

I am not using this add-on anymore as IndexNow is now integrated on XF2.3 and i didn't remove Disallow: /posts/ from my robots.txt file and I have no problem reported in Bing > IndexNow.

 

[Old Nick]

And posts URLs are sent.

 

[masmo]

And posts URLs are sent.

Just because they are sent doesn't mean there isn't some problem...

 

[Old Nick]

Just because they are sent doesn't mean there isn't some problem...

and I have no problem reported in Bing > IndexNow.

 

[masmo]

i didn't remove Disallow: /posts/ from my robots.txt file and I have no problem reported in Bing > IndexNow.

But did you check here?


Before I removed the 'Disallow: /posts/' from my robots.txt I had these errors!

Then after deleting it 5xx errors started for redirects

 

[Old Nick]

Mine

 

[masmo]

Thank you for your feedback, I don't understand then the cause of all my mistakes... I didn't have these errors with the previous add-on.

 

[Old Nick]

Try without the add-on to see if that improves things, or not.

 

[Rusty Snippets]

Don’t know if it’s been mentioned, I got the error a couple times. I unchecked it, saved, then rechecked it and saved. Seems to have fixed it.

 

[BikeGremlin]

I see no value in indexing links that aren't used on the forum.
Bing indexes stuff just fine (and up to date) even without enabling the IndexNow (when it doesn't index stuff, it's not because IndexNow isn't working, but some other problem).

I would even argue that it can create more problems than help.

My article about Bing problem(s):

Microsoft Bing - site gone from SERP overnight! | BikeGremlin I/O

All the websites from my domain disappeared from Microsoft Bing’s SERP overnight!

io.bikegremlin.com

And the relevant forum discussion/feedback from other webmasters:

https://www.bikegremlin.net/threads/seo-microsoft-bing-site-gone-from-serp-overnight-article-comments.98/

 

[masmo]

Bing indexes stuff just fine (and up to date) even without enabling the IndexNow (when it doesn't index stuff, it's not because IndexNow isn't working, but some other problem).

I completely agree with you!

However since I have been using IndexNow I have noticed that new threads are indexed more faster, the problem has started since I have been using integrated IndexNow. First with the ‘Disallowed in robots.txt’ and then with the 5xx redirect errors.

I'm surprised that I'm the only one with these problems...

I now deactivate IndexNow intergrated and reactivate the add-on to see if indexing works well again.