One of the biggest things to happen in the last few years, is data driven decision making. Companies are spending huge amounts of money building predictive models to solve a variety of problems. Humanity has a tendency to take technologies to extremes without thinking through the consequences. What are the potential consequences with data driven decision making?

Data is collected at a constant and furious pace. In the case of you and I, the data that is collected can be intrusive and an invasion of privacy. Many sites collect everything you do while on their web page: clicks, mouse movements, scroll bars, time spent idle, and even a basic fingerprint about the computer you are browsing on. Heck, I’m collecting data on the visitors to this site myself. No need to worry, however, I’m only recording site traffic so I can see how many people, if any, visit my pages.

Data scientists are attempting to use all the data that is being collected to try and prove theories and create prediction models. In some cases, the data is being used to try and predict environmental or physical changes based on imperical data that has been collected over time. In others, they are trying to predict how and why humans behave as they do within the context of a particular situation. The behavioral models are then used to build/change websites that are better able to entice the audience into spending money.

So what is wrong with data driven decision making?

Let me put forward an analogy, by asking a question. How many of you actually know how to cook? Now I don’t mean heating a frozen meal up, or boiling up a pot of pasta. I mean actually cook an entire meal from scratch. I would put forward that many people no longer know how to cook because they have become reliant on frozen meals, fast food and resturants.

So what do we lose if we become too reliant on using big data to provide us insights, and decisions? I believe we may begin to lose our instinctive ability to find creative solutions to problems, or understand instinctively why something is. This is equivalent to understanding the why versus just being told it is. I don’t believe this will happen suddenly, but gradually over time. To be clear I also believe that using big data will benefit mankind in great ways.

In a previous blog post I talked about how we each have an inner einstein. I believe that this is fundamentally a big part of our instinct, and we should always listen to what it has to say. Just be careful to think objectively about what it says, you never know it could be wrong.

Maybe you should validate it against actual data.