XF 2.2 Error submitting sitemap to Bing

JoyFreak

Well-known member
Just had this error logged in the Admin CP. It only occurred once, earlier today. Any concerns or can be ignored?
Code:
Server error log
GuzzleHttp\Exception\ClientException: Error submitting sitemap to https://www.google.com/webmasters/tools/ping?sitemap=https%3A%2F%2Fwww.joyfreak.com%2Fsitemap.php: Client error: `GET https://www.google.com/webmasters/tools/ping?sitemap=https%3A%2F%2Fwww.joyfreak.com%2Fsitemap.php` resulted in a `429 Too Many Requests` response: <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html> <head><meta http-equiv="content-type" content="te (truncated...) src/vendor/guzzlehttp/guzzle/src/Exception/RequestException.php:113
Generated by: Unknown account Dec 13, 2021 at 8:22 AM
Stack trace
#0 src/vendor/guzzlehttp/guzzle/src/Middleware.php(66): GuzzleHttp\Exception\RequestException::create(Object(GuzzleHttp\Psr7\Request), Object(GuzzleHttp\Psr7\Response))
#1 src/vendor/guzzlehttp/promises/src/Promise.php(204): GuzzleHttp\Middleware::GuzzleHttp\{closure}(Object(GuzzleHttp\Psr7\Response))
#2 src/vendor/guzzlehttp/promises/src/Promise.php(153): GuzzleHttp\Promise\Promise::callHandler(1, Object(GuzzleHttp\Psr7\Response), NULL)
#3 src/vendor/guzzlehttp/promises/src/TaskQueue.php(48): GuzzleHttp\Promise\Promise::GuzzleHttp\Promise\{closure}()
#4 src/vendor/guzzlehttp/promises/src/Promise.php(248): GuzzleHttp\Promise\TaskQueue->run(true)
#5 src/vendor/guzzlehttp/promises/src/Promise.php(224): GuzzleHttp\Promise\Promise->invokeWaitFn()
#6 src/vendor/guzzlehttp/promises/src/Promise.php(269): GuzzleHttp\Promise\Promise->waitIfPending()
#7 src/vendor/guzzlehttp/promises/src/Promise.php(226): GuzzleHttp\Promise\Promise->invokeWaitList()
#8 src/vendor/guzzlehttp/promises/src/Promise.php(62): GuzzleHttp\Promise\Promise->waitIfPending()
#9 src/vendor/guzzlehttp/guzzle/src/Client.php(131): GuzzleHttp\Promise\Promise->wait()
#10 src/vendor/guzzlehttp/guzzle/src/Client.php(89): GuzzleHttp\Client->request('get', 'https://www.goo...', Array)
#11 src/XF/Sitemap/Builder.php(508): GuzzleHttp\Client->__call('get', Array)
#12 src/XF/Sitemap/Builder.php(432): XF\Sitemap\Builder->sendPing()
#13 src/XF/Sitemap/Builder.php(73): XF\Sitemap\Builder->completeBuild()
#14 src/XF/Job/Sitemap.php(27): XF\Sitemap\Builder->build(2.79395)
#15 src/XF/Job/Manager.php(260): XF\Job\Sitemap->run(8)
#16 src/XF/Job/Manager.php(202): XF\Job\Manager->runJobInternal(Array, 8)
#17 src/XF/Job/Manager.php(118): XF\Job\Manager->runJobEntry(Array, 8)
#18 src/XF/Admin/Controller/Tools.php(122): XF\Job\Manager->runByIds(Array, 8)
#19 src/XF/Mvc/Dispatcher.php(352): XF\Admin\Controller\Tools->actionRunJob(Object(XF\Mvc\ParameterBag))
#20 src/XF/Mvc/Dispatcher.php(259): XF\Mvc\Dispatcher->dispatchClass('XF:Tools', 'RunJob', Object(XF\Mvc\RouteMatch), Object(ThemeHouse\UserCriteria\XF\Admin\Controller\Tools), NULL)
#21 src/XF/Mvc/Dispatcher.php(115): XF\Mvc\Dispatcher->dispatchFromMatch(Object(XF\Mvc\RouteMatch), Object(ThemeHouse\UserCriteria\XF\Admin\Controller\Tools), NULL)
#22 src/XF/Mvc/Dispatcher.php(57): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#23 src/XF/App.php(2351): XF\Mvc\Dispatcher->run()
#24 src/XF.php(517): XF\App->run()
#25 admin.php(13): XF::runApp('XF\\Admin\\App')
#26 {main}
Request state
array(4) {
  ["url"] => string(24) "/admin.php?tools/run-job"
  ["referrer"] => string(120) "https://www.joyfreak.com/admin.php?tools/run-job&only_id=104412&_xfRedirect=%2Fadmin.php%3Ftools%2Frebuild%26success%3D1"
  ["_GET"] => array(1) {
    ["tools/run-job"] => string(0) ""
  }
  ["_POST"] => array(3) {
    ["_xfRedirect"] => string(58) "https://www.joyfreak.com/admin.php?tools/rebuild&success=1"
    ["_xfToken"] => string(8) "********"
    ["only_ids"] => string(6) "104412"
  }
}
 
Solution
So to be clear this is an issue solely on the Bing end and can be safely ignored.

I don’t know how you go about reporting issues to Bing but it’s likely that enough people need to make them aware and hopefully that will prompt them to turn it off and on again.

Hopefully this isn't a planned closure of the sitemap ping endpoint. If it is, I can't see it documented anywhere...

It seems Bing are going all-in on their IndexNow protocol but I can't see them doing away with sitemaps just yet.

Does anyone actually have a Bing Webmaster Tools account? If so, please open a ticket or send them an email regarding this or please look for any further clues about what might be going on.

Cheers :)

Chris D

XenForo developer
Staff member
Why is changing the ping URL as seen below only considered a workaround? Does it still function as intended and submit the ping for the sitemap or does it just avoid getting the server log error message?
The workaround URL is not the URL that Bing advise using in their documentation. In fact, it's the old URL before it changed to what it is at the moment. While it doesn't generate an error, we actually have no idea whether it works or not, nor whether or not Bing will eventually shut that one down.

Also, what is the $ removed on the workaround URL?
This is a mistake.

It should be:
Code:
https://bing.com/webmaster/ping.aspx?sitemap={$url}

@webbouk I think you attempted to bring this to my attention already and I dismissed it, sorry about that.
 

djbaxter

Well-known member
Still getting this error. Oddly, I don't get any error with any other site than the one using Xenforo. (I have a Vanilla, Wordpress and PHPBB site and none of them get an error submitting site maps to Bing.)
Unless you've changed it on your WordPress site, WordPress doesn't use the Xenforo default Google or Bing URLs. Wordpress uses https://rpc.pingomatic.com/.

Additionally, it appears that the Bing URL error is intentional. Bing is switching over to the new IndexNow API and trying to encourage other search engines to do the same. So far, only Baidu has followed suite.


 

Chris D

XenForo developer
Staff member
And the difference is?
What I have said since doesn't change the fact I never said it didn't work. Also it depends how you measure the success, as I was really talking about two different things in those two different comments. Changing the URL works in terms of the fact that it stops the error being logged. However, we have no evidence as to whether the sitemap ping feature continues to work as intended or not if we use the workaround URL because, while it was once the documented URL, it no longer is, so we can't be sure whether it will continue to work or not.

A: I was sworn at for posting the problem.
I am sorry you were offended by that user's comment and I will advise @gerryvz to consider a better tone and choice of words in future but many people do not consider "damn" to be a swear word, nor was it necessarily directed at you. You are only the most recent out of many people who have continued to post about the issue so it could easily be a general comment.

Additionally, it appears that the Bing URL error is intentional. Bing is switching over to the new IndexNow API and trying to encourage other search engines to do the same. So far, only Baidu has followed suite.
This is a good find. Perhaps I'm not as jaded and pessimistic as I should be by now, but going from IndexNow launching in November to turning off sitemap pings in December without documenting that officially seems highly unlikely so I think it remains to be seen until then.
 

nicodak

Well-known member
The workaround URL is not the URL that Bing advise using in their documentation. In fact, it's the old URL before it changed to what it is at the moment. While it doesn't generate an error, we actually have no idea whether it works or not, nor whether or not Bing will eventually shut that one down.


This is a mistake.

It should be:
Code:
https://bing.com/webmaster/ping.aspx?sitemap={$url}

@webbouk I think you attempted to bring this to my attention already and I dismissed it, sorry about that.
I don't understand now.
I used the URL without the $ and i don't receive error anymore since the change...

Capture web_7-1-2022_172756_regionalis.fr.jpeg
 

ActorMike

Well-known member
I think most people are aware of this, but your sitemap can also be submitted via Bing Webmaster tools and it actually just checked my sitemap automatically on 1/4/2022.




based upon @djbaxter's post, it appears removing the Bing URL from ping notifications altogether is probably a good practice.
 
Last edited:

ActorMike

Well-known member
Google is also looking into it. Indexnow API Seems like a good idea, especially for a forum where the content constantly changes.
 

BlueSquares

Member
I think most people are aware of this, but your sitemap can also be submitted via Bing Webmaster tools and it actually just checked my sitemap automatically on 1/4/2022.




based upon @djbaxter's post, it appears removing the Bing URL from ping notifications altogether is probably a good practice.
Thanks for this. I was able to submit my WordPress sitemap, but the Xenforo sitemap won't take. I press "Submit" after entering the sitemap URL The URLs https://www.site.com/sitemap.php and https://www.site.com/sitemap.xml were both attempted but failed to be picked up.

It briefly shows a spinner but the popup remains. Nothing is processed. The submit box just stays there over and over again.
 

ActorMike

Well-known member

Chris D

XenForo developer
Staff member
Unless you've changed it on your WordPress site, WordPress doesn't use the Xenforo default Google or Bing URLs. Wordpress uses https://rpc.pingomatic.com/.

Additionally, it appears that the Bing URL error is intentional. Bing is switching over to the new IndexNow API and trying to encourage other search engines to do the same. So far, only Baidu has followed suite.


Was already posted here @Jake B.

This is a good find. Perhaps I'm not as jaded and pessimistic as I should be by now, but going from IndexNow launching in November to turning off sitemap pings in December without documenting that officially seems highly unlikely so I think it remains to be seen until then.
 

nicodak

Well-known member
If you are using Cloudlare there is an article about IndexNow
 
Top