Google scraping concerns

[ekool]

Am I the only one that sees the potential downside to this? I've run into a ton of searches where Google is scraping someone's site for data. They answer it in a search and they structure it so that the searcher never needs to visit the original site for the content. They've gotten everything they need on the google search index and never click through. Thusly, Google has just profited from the sites content and the site doesn't get any benefit from it. Not an eyeball, pageview, the user never even needs to clickthrough. Google and the end user benefit, the content generator gets hosed.

For example, and I came up with this off the top of my head... on other searches Google does a lot more with the simple data, but for example:

top nfl teams by money - Google Search

The bulk of the question is answered in bold text, taken from the site and I don't even need to click through. It's been answered there.

 

[dethfire]

Am I the only one that sees the potential downside to this?

I've been an SEO for 20 years. You can wall yourself in, but only you will lose. Communities generally live and die with organic traffic. There is natural membership churn and the only way to get more members is to do well in Google. The more we can align with Google and have them understand our content to the fullest and then match it with queries the better.

 

[bzcomputers]

Am I the only one that sees the potential downside to this?

I was say overall it is a very small amount of Google search results that give everything a searcher is looking for in the search results alone. It does happen but I think the percentage overall is small and the potential benefits for placing well in their search results for most sites greatly outweighs the loss of a few clicks occasionally.

 

[ekool]

I've been an SEO for 20 years. You can wall yourself in, but only you will lose. Communities generally live and die with organic traffic. There is natural membership churn and the only way to get more members is to do well in Google. The more we can align with Google and have them understand our content to the fullest and then match it with queries the better.

I was say overall it is a very small amount of Google search results that give everything a searcher is looking for in the search results alone. It does happen but I think the percentage overall is small and the potential benefits for placing well in their search results for most sites greatly outweighs the loss of a few clicks occasionally.

You are both correct. But that is the problem. We are willing to give google our content, let them steal it and earn money off of it because they are so powerful. We have no choice because they have a monopoly and we live and die by their unknown black boxes. This is not a good thing.

Is it fair that they can show content from our site that someone is searching for on theirs, and thusly, prevent users from actually viewing our content? No.

But we are willing to accept that because we want the search traffic. This is unfair and why Google should be broken up as the monopoly that they so very clearly are. They own all of the ad traffic as well as the search traffic and they use this unfair advantage against normal sites and any competitor.

 

[TPerry]

But we are willing to accept that because we want the search traffic.

OR.... XF could develop a method that search engines don't get the entire content (yes, this would negatively impact discovery) but simply a segment of what the content contains.
Honestly... if the person searching is interested enough.. they will follow those links. I know I regularly do so that I can get the information in full context.

 

[dethfire]

OR.... XF could develop a method that search engines don't get the entire content (yes, this would negatively impact discovery) but simply a segment of what the content contains.
Honestly... if the person searching is interested enough.. they will follow those links. I know I regularly do so that I can get the information in full context.

I don't see how this helps at all. If you want to block Google from some of your content, then great, fill me in where you're getting your members.

 

[ekool]

OR.... XF could develop a method that search engines don't get the entire content (yes, this would negatively impact discovery) but simply a segment of what the content contains.
Honestly... if the person searching is interested enough.. they will follow those links. I know I regularly do so that I can get the information in full context.

I think Google should actually step up and set some guidelines on how much content they will use that is scraped from the websites they scan. Right now they have no incentive to limit it... every page view that doesn't leave the search engine is good for them. Ones that leave are bad. They are incentivized to steal more and more content moving forward.

They used to say "do no evil" -- They certainly have strayed far from the path.

I don't see how this helps at all. If you want to block Google from some of your content, then great, fill me in where you're getting your members.

I do believe you are missing the point.

 

[dethfire]

I do believe you are missing the point.

no I think you are missing the point. We're talking about negotiating with a trillion-dollar company. They have no incentive to do anything other than make money for their shareholders whether you like it or not or think it's right or not, you can't win, you can't punish them. You either work hard to win in the smaller and smaller piece of pie or you die.

 

[MySiteGuy]

You are both correct. But that is the problem. We are willing to give google our content, let them steal it and earn money off of it because they are so powerful. We have no choice because they have a monopoly and we live and die by their unknown black boxes. This is not a good thing.

Is it fair that they can show content from our site that someone is searching for on theirs, and thusly, prevent users from actually viewing our content? No.

But we are willing to accept that because we want the search traffic. This is unfair and why Google should be broken up as the monopoly that they so very clearly are. They own all of the ad traffic as well as the search traffic and they use this unfair advantage against normal sites and any competitor.

Go ahead and teach them a lesson, and don't complain when you get even less traffic.

 

[TPerry]

I don't see how this helps at all. If you want to block Google from some of your content, then great, fill me in where you're getting your members.

I never claimed it would help, other than maybe encouraging a user that finds it in a search engine but only gets a portion (you know, what dope dealers call "giving a taste") to actually CLICK THROUGH to your site, which CAN result in getting members... the complaint was that WHY should the Google searcher click through when what can be the entire content is DIRECTLY THERE on the Google search page. Where exactly is the encouragement to visit the site.. they already GOT the answer from the search engine page, so why should they go to the originating site that Google obtained that data from?

You DID notice where I CLEARLY stated "it would negatively impact discovery"... but I can see the point that was being made... exactly why would a user want to visit your site to see the content that is displayed already (at times) in full on the Google search engine?
My comment was about one of the few ways that issue could be prevented.... no stance on whether it is 'good/bad".

hey have no incentive to do anything other than make money for their shareholders whether you like it or not or think it's right or not, you can't win, you can't punish them

Oh yes... if everyone banded together... Google could EASILY be punished... if they can't index data, they can't provide search results... they can't provide search results, their revenue goes down.
Now, is it likely that the entire interwebz would band together and do such... is Hades scheduled to freeze over soon? I dont' think so, because their overflow heat is hitting us here in the southern US.

Go ahead and teach them a lesson, and don't complain when you get even less traffic.

Yeppers... all one can do is hope that the user found the Google search content interesting enough to pull them down the rabbit hole.

 

[dethfire]

You DID notice where I CLEARLY stated "it would negatively impact discovery"... but I can see the point that was being made... exactly why would a user want to visit your site to see the content that is displayed already (at times) in full on the Google search engine?

Oh you are totally correct. SEOs have been dealing with that ever since featured snippets came out, but there is no solution other than try to win featured snippets. You can block content, but all that's going to do is devalue your page in the eyes of Google while others get your small piece of pie.

 

[Jeremy P]

I have split these posts into a separate thread as they are a bit tangential to the the particular suggestion they were formerly posted in.

 

[Mr Lucky]

Am I the only one that sees the potential downside to this? I've run into a ton of searches where Google is scraping someone's site for data. They answer it in a search and they structure it so that the searcher never needs to visit the original site for the content

No you aren’t the only one.

What is worse IMO is that a lot of those so-called answers are AI

 

[PatriotGB]

but I can see the point that was being made... exactly why would a user want to visit your site to see the content that is displayed already (at times) in full on the Google search engine?

Because it establishes my site as the "authority." My site ranks so high, that Google QUOTES it directly in the search results.

I have been my own SEO for about 24 years (for business-related website). I think the net outcome of this practice by Google is positive.

My 18 month old forum (as of tomorrow) pretty much now ranks #1 for searches related to the primary topic. Sometimes #2. And it is definitely driving traffic. And none of it is by accident.

 

[TPerry]

Because it establishes my site as the "authority." My site ranks so high, that Google QUOTES it directly in the search results

No, ALL it does is give that user ALL the information they need without EVER going to your site.
You can be "all the authority" you want... but if folks aren't actually visiting your site to get content (and hopefully participate) from the search engine, you are just giving free meat to Google, it doesn't matter if you are at the top of page 1, or bottom of page 1000 if they aren't visiting your site.

I think the net outcome of this practice by Google is positive.

Really? Feel free to explain to ALL of us ignorant how a user getting the entire answer they need from Google's search engine and that user NEVER visiting yours is good for YOUR SEO? How exactly does that encourage participation on your site when they never even need to visit it to get the answer?

You seem to be fixated on positioning.. where the other poster was more concerned with the person doing the search never VISITING their site since the answer was already provided to them.

 

[eva2000]

For example, and I came up with this off the top of my head... on other searches Google does a lot more with the simple data, but for example:
top nfl teams by money - Google Search

FYI, you can opt out of Google Feature Snippets https://developers.google.com/search/docs/appearance/featured-snippets

How can I opt out of featured snippets?​

There are two ways that you can opt out of featured snippets:

• Block both featured and regular search snippets
• Block featured snippets only

 

[James]

FYI, you can opt out of Google Feature Snippets https://developers.google.com/search/docs/appearance/featured-snippets

Swooped in and mic dropped!

 

[badmonkey]

Interesting so many invest time, effort, and money begging Google to scrape their content.

I have a million +post forum I hope they're scraping sans such investment on my part.

 

[PatriotGB]

No, ALL it does is give that user ALL the information they need without EVER going to your site.
You can be "all the authority" you want... but if folks aren't actually visiting your site to get content (and hopefully participate) from the search engine, you are just giving free meat to Google, it doesn't matter if you are at the top of page 1, or bottom of page 1000 if they aren't visiting your site.

That's a rather pessimistic outlook. But, they ARE visiting my site. And I know because I ask new members how they found us. Mostly Google. Hooray!

Really? Feel free to explain to ALL of us ignorant how a user getting the entire answer they need from Google's search engine and that user NEVER visiting yours is good for YOUR SEO? How exactly does that encourage participation on your site when they never even need to visit it to get the answer?

Dispense with the straw man arguments and putting words in my mouth (about you being ignorant). I never even implied such things.

Again... I'm not so pessimistic. Those snippets on Google give my site VISIBILITY. Maybe they won't visit right then. Maybe they will. Maybe they'll remember the source and visit my site later.

Listen... I'm not one to defend "big tech." Hardly! Don't get me started on Big Tech's collusion with gov't as it pertains to 1st Amendment infringements (it's why I quit Facebook after MANY years of very active participation and it being my primary means of communication with friends and family).

There is no doubt my sites have benefited from Google indexing. My first website went up in 1999. It's still active. Google Snippets aren't a new thing. They have been around for many years. Like most things, it's a double-edged sword. But, in this case, I'll take the net positive. Your mileage may vary.

 

[TPerry]

That's a rather pessimistic outlook. But, they ARE visiting my site. And I know because I ask new members how they found us. Mostly Google. Hooray!

But a realistic one as the actual point was in reference to the OP's comment about the entire response being given on the Google search page... you failed to explain how the user that finds an answer there and then goes on about their other "rat killing" ever benefitted the SEO of the site that provided that answer that Google scraped and provided.
As for "hope"... well, I hope every day that a nice person will drop off a GM3000 HPS mount with a Dall Kirkham Quartz mounted on it on the front porch...

Dispense with the straw man arguments and putting words in my mouth (about you being ignorant). I never even implied such things.

No, you simply tried to change the subject from what the OP was commenting on. And you failed to give any valid SEO return that a site gets when the full answer is provided on the Google search page and that user never visits your site. You really need to learn what a straw man argument is... and I think you will find it applies more to YOUR comments (as pertaining to the OP issue) than mine.

There is no doubt my sites have benefited from Google indexing.

And nobody is denying that ANY search engine indexing your site is a "good thing" for all sites.... the point that the OP made had zero to do with Google indexing a site.. it had to do with them providing "snippets" that contained the answer and how it can discourage a user from visiting the site that originated said answer.
I just restarted the Analytics collection on my site a few days ago because the old account wasn't capturing everything... this is for the time since I restarted the stats....




So yes, organic search is good...

Rarely when I see the answer on a Google snippet do I even go to the originating page, unless I think their may be more detail that I would like to know.... but generally when I am doing a search, I am looking for the answer, and if the snippet provides that, I have no need to go to the site... I'll use the answer I found and fix the problem that I was asking about and then go about the remainder of my rat killing.