Why Trusting SEO Data to the Extent that We Do Makes No Sense: Transcript of Episode #2
Resources:
The SEO Rant on Using SEO Data without Nuance
Today we’re talking about SEO and data and why we’re addicted to it and why maybe we shouldn’t be!
Gary Sterling over at Search Engine Land posted an article talking about how the pollsters got the polling for the recent election wrong and what that means for market research. He basically says to be cautious with survey data. It’s a good article. You should read it.
My initial reaction, however, was… DUH.
But this is not as obvious to us as it should be because we love our data as SEO. We wash our data, brush its hair at night, stroke its back...we love our data.
But I think we totally misuse it.
I think the reason for that is there are two kinds of data…. I mean, I’m sure there are more than 2 kinds, but for now, there are 2 kinds:
Data that relates to simple performance or behavior. Rank goes down, impressions go down, traffic goes down, etc. It’s pretty straightforward. Often you can find the cause relatively easily (assuming the cause is not being hit by an update,etc.). Like imagine you made a change to your site and now you see rank, impressions, traffic all go down.
Then there’s data that tries to explain underlying behavior or underlying sentiment. That’s tricky stuff right there.
One, there’s the data… and then there’s you. To what extent does the data align to your specific site in your specific circumstance? Data is often an average… and that average may or may not apply to your situation.
I’ll give an example from the 2020 World Series. Yep, we’re talking baseball. Frankly, the only surprise is that I didn’t bring it up in episode #1.
Baseball is now built on analytics. I don’t know if you’ve ever seen the movie Moneyball with Brad Pitt, but you should! In a nutshell, it’s about how a real sucky player for the NY Mets (that should go without saying because that’s the only kind of player the Mets ever have) turned GM for the Oakland A’s. This GM dealt with a lack of cash by leveraging under-appreciated players by using analytics. Long story short that team did unbelievably well and everyone in baseball moved towards analytics.
Fast forward to 2020 and the Dodgers are playing the Rays in the World Series. If the Rays lose the game, they lose the series. They have one of their best players on the mound pitching. He’s doing really great. Everything is in order.
Except, the data says that after the 6th inning this pitcher doesn’t do as well. So the manager comes out and thinks to himself, “I’m a big genius, I’m using data!”. So he pulls his star pitcher and puts a new pitcher into the game...AND loses! The next pitcher lost the game for them.
Why?
Because this manager is a moron who doesn’t understand that analytics are averages. It’s a collection of data samples. The question is, right here, right now, what are you getting… what data point are you seeing? Does the situation you’re in reflect the higher-end of the average, are you getting the average, are you getting below average. In our case here, it was clear looking at the pitcher that this was an above-average performance. So basically, the manager here doesn’t understand how data works.
And that’s the problem.
Once you move out of linear equations, you have to consider the limitation and context of the data… and that is tricky.
It’s all well and good to look at some great SEO study. But applying that to your site is crazy complex.
You sort of have to use your intuition at some point. (No, not intuition!)
A teacher of mine once said 99.9% of all our decisions are made intuitively.
Having data is not a substitute for that. Having data is a way to help you uncover the story that is your situation. It helps point you in the right direction by helping you to know what questions to ask.
But personally, if you showed me data that made no intuitive sense about what I know is my site’s situation… I wouldn’t act on it. I would investigate, and if I could get some clarity then great… if not… I’m going with my instincts.
And it’s not an easy balance, because it could just be you’re being a stubborn fool who isn’t willing to change. What you might think are “instincts” is just you being too set in your ways. It could also be legit intuition. How do you know the difference? You need to know yourself. There are no shortcuts here.
I’ll give a terrible example. You read some SEO study that says reviews outside of proximity is the top factor for Local Pack ranking. And you’re looking at your situation or scenario and it just doesn’t seem like that’s what going on with you and your competitors. There just seems to be some other consideration for some reason that’s driving the way things are… I would go with that gut instinct and move on it to see what’s up instead of going nuts working on your reviews. That study doesn’t know your situation. Again, terrible example but you get what I mean.
Look, it all comes down to what my grandmother always told me… Don’t be a moron! Use your brain and get off the data train if you’re just blindly following along!