XF 2.1 Adsense Crawler

[Gon45]

Hello,

I hope I posted this in the right area. Recently I lost ads on my forum: Hallyuplus.net

The error I got was that I had blocked URL's, as show in this attachment:

Spoiler

What I can deduce from this is that the crawler cannot access those conversations. Adsense has a page all about blocked pages and how to let the crawler login, but I'm pretty new at this and I can't figure out what exactly I'm supposed to put in those fields, my admin login? Or a different login just for the crawler? I can't figure it out. Also, I'm not even sure if doing so would allow the crawler to access private messages, which is basically what it wants.

Any help is appreciated!

 

[AndyB]

I estimate 90% possibly more of your AdSense revenue is earned on the thread_view pages. So I think it's best to only show AdSense on the following pages:

forum_list
forum_view
thread_view

You might like this add-on:

https://xenforo.com/community/resources/ad-manager-responsive.7034/

 

[djbaxter]

1. What do you mean you "lost ads"?

2. That "error" is simply a statement that Adsense cannot access those pages and that you may be losing potential revenue on blocked pages. That in itself is nothing to worry about.

 

[Gon45]

I estimate 90% possibly more of your AdSense revenue is earned on the thread_view pages. So I think it's best to only show AdSense on the following pages:

forum_list
forum_view
thread_view

You might like this add-on:

https://xenforo.com/community/resources/ad-manager-responsive.7034/

Ill look into this, thanks.

1. What do you mean you "lost ads"?

2. That "error" is simply a statement that Adsense cannot access those pages and that you may be losing potential revenue on blocked pages. That in itself is nothing to worry about.

Ads stopped showing shortly after I got those crawler errors. I was reading that ads would eventually disappear if it cant crawl. I asked in the adsense community but they dont seem to have experience with Xenforo.

 

[AndyB]

Ads stopped showing shortly after I got those crawler errors.

Please be more specific, do you mean all AdSense ads stopped showing anywhere on your forum, or the AdSense ads just stopped showing in Conversations?

 

[djbaxter]

Three possibilities:

1. you have entered the code incorrectly or in the wrong place

2. AdSense doesn't like your content because it violates one of their guidelines

3. AdSense doesn't have any ads currently that match your content

 

[Gon45]

Please be more specific, do you mean all AdSense ads stopped showing anywhere on your forum, or the AdSense ads just stopped showing in Conversations?

All ads stopped showing, mobile as well. I can see that it wants access to those conversations but I'm not sure how to go about letting it do that. I assume I have to follow these steps: https://support.google.com/adsense/answer/161351?hl=en&ref_topic=1348129

However, I do not know if I need to include my admin login or what. Because the adsense crawler issues seem to stem from the fact that it cant crawl through private conversations.

 

[xydrine]

All ads stopped showing, mobile as well. I can see that it wants access to those conversations but I'm not sure how to go about letting it do that. I assume I have to follow these steps: https://support.google.com/adsense/answer/161351?hl=en&ref_topic=1348129

However, I do not know if I need to include my admin login or what. Because the adsense crawler issues seem to stem from the fact that it cant crawl through private conversations.

These areas - conversations - as you mentioned, cannot be crawled for content by AdSense's crawler (obviously), and the pages will return a 403 Forbidden error to the crawler. 403 errors will cause the crawler to crap out/error on those pages and 99.99999% is the reason for what you are seeing now in terms of your ads not working properly.

This should be a rather easy fix - just prevent ads from being shown in conversation pages/other areas of the forum where they would be forbidden for a non-logged in user/crawler from accessing (as mentioned by Andy above).

What I do is serve our adsense code in an HTML widget, and only have it appear on certain pages (using the widget location selection options) - for example here's my widget setup:






Note that I only serve advertisements to unregistered users, so depending on how you want your ads to display, you would need to change the way your widget is setup vs how mine is (in regards to a couple of the configuration options shown above), but that's a pretty easy thing to configure.

If you need help configuring this, let me know how you have your ads setup (who you want them to be displayed to - i.e. all users, etc) and I can help you out with that.

 

[bzcomputers]

Simplest solution is to update the "Prevent ads" template list that already exists within the XenForo options. No add-ons required, no template editing, no widget editing.

Setup - Options - Appearance: Prevent ads showing in these templates

account_details
contact_form
conversation_add
conversation_edit
conversation_invite
conversation_leave
conversation_list
conversation_reply
conversation_view
error
help_index
help_page
login
login_password_confirm
login_two_step
lost_password
lost_password_confirm
message_page
register_complete
register_confirm
register_connected_account
register_form

(edit your list to include at least all of the the following, to prevent being banned from ad serving entities)

 

[Gon45]

Ill try all of your suggestions and see what happens, thanks to everyone for responding!

 

[bzcomputers]

Ill try all of your suggestions and see what happens, thanks to everyone for responding!

I created a suggestion for XenForo to expand their stock advertising banned template list so this type of situation doesn't happen in the future. Hopefully it gets implemented pretty quickly.

Lack of interest - Reduce likelihood of sites being banned for serving ads on inaccessible and no content pages

After reading a recent post about an individual getting warnings from their advertising provider (Adsense) about their site displaying ads on inaccessible pages (inaccessible to the advertiser). Then said advertiser no longer serving ads to the entire site (hopefully not a permanent ban)...

xenforo.com