To put those findings simply, nine out of ten executives would find a way to ignore the data if it disagreed with their intuition. This is a classic example of what psychologists call “confirmation bias.”  We are wired to embrace new information that agrees with our beliefs, and reject it if it contradicts them. 
— Dooley R. (2014) Are You The One Executive Out Of Ten Who Isn’t Clueless?

Dooley is a good writer, but this time he fell in the same “confirmation bias” trap he quotes. Because yes, it’s known that managers often ignore data and make their own decisions, but this time the survey doesn’t back that experience.

Let’s see the survey results: 30% told that they’ll collect more data, and 57% old that they’ll re-analyze it. Both are correct behaviors to have when the data shows something unexpected. They demonstrate awareness of a mismatch, critical thinking and the ability to use data properly.

What about the other side? 10% trust the data blindly, and that alone is an issue. And 3% do even worse by ignoring it completely. This demonstrates poor critical thinking, no error-checking and it’s a behavior that is prone to be easily influenced by hype.

Then, of course, if after review the data still goes in the opposite direction, you must be open to change and adaptation. It’s a multi-step process.

Not just that. What does it mean to “do what the data suggest”? Data by itself doesn’t suggest anything, ever. It’s how you interpret it that counts. This is something that the “data driven” rhetoric keep missing.

And this doesn’t even count when you have a strategy designed purposefully to go against the data, like for example when you have most of the users doing A and just few doing B, but your long-term strategy needs for any reason to invest on B.

In other words: you should be data informed, not data driven.

Roger himself acknowledged this when I asked him directly:

One thing that is important is the quality of the data. It’s logical to dig deeper if the data is incomplete, sketchy, or questionable in some concrete way. It’s confirmation bias when someone doggedly refuses to accept data that is incontrovertible.

I agree that the data are open to interpretation… Loads of research shows people reject data they don’t agree with.

Ironically, this very post is an example on both the issues, interpretation and confirmation bias: I’ve interpreted the data differently, and I’d like to see more data because this one goes against my gut feeling that managers do not use data properly. QED.