A Horrible, Terrible, No Good, Just World Is Coming. Thanks, Big Data.

The arc of history certaily bends towards rationality. Big Data revolution is making sure that few decisions will have to be made based on pure opinions and hunches, a.k.a. prejudices, biases and dogmas, which are also known as bad assumptions.

Isn't that exciting that in the fast-approaching evidence-based future incompetent bosses will be quickly found out despite their self-confidence and charisma, full-of-it pundits will be exposed, hot-air-filled politicos will be voted out, and humble, talented and under-appreciated working bees will be promoted and rewarded accordingly?

The answer is easy, it's yes, which is why "Moneyball" is such a bestseller: a wiz-kid mined data to find a factor in player stats that has strongest yet non-obvious correlation with winning. The justice is served to multiple parties: lone wiz kid with proven track record has become an existential threat to the army of highly-paid scouts producing dubious output, the open-minded GM saves his career in the big way, and talented underdog players took their righteous place under the spotlight. Just world, here it comes, and although the road to a giant digital BS filter is going to be bumpy and winding, to me it's pretty clear we are heading in that general direction, and going there pretty fast.

I am also sure* that this just, rational world will turn out to be extremely unpleasant and disappointing for maybe the majority of people. See, right now, while working as a mid-level office drone or a retail sales clerk, or a barista, we all enjoy plausible deniability of being simply under-appreciated and thus we can maintain our self-esteem knowing than some stuck-up/suck-up got ahead simply because life is not fare and bad things happen to good people - us. Now, what if Big Data revolution shows that not only our mediocre bosses do not deserve telling us what to do, but we too turn out to be just as unremarkable as them? What if it turns out that there are people, who are so much more talented that they can add much more value than just about any one of us? What if majority of population is getting paid right now to simply slow down these remarkable individuals and drown their weak, nerdy but brilliant voices in the noise of millions of opinions? Well, even I turn out to be that brilliant individual, who is going to be shown objectively better at doing my job, and many others will have to rush implementing my genius ideas, I fear most people will be so permanently and irreparably depressed that I won't be able to enjoy my new-found appreciation :-). Worse is, of course, if my track record, studied under the microscope of Big Data, turns out to be just "meh" and I will have to leave behind ranks of creators and idea generator and take the spot in the swelling crowd of implementors and assistants, or worse, to become a stipend receiver paid to stay home and not break anything.

As they say, the factory of the future will have only two employees, a man and a dog: the man will be there to feed the dog, and the dog will be there to keep the man from touching the equipment. The only question is are there enough dogs?

*This post will surely one day be mined and ranked by our robot overlords for how well my prediction worked. If I turn out to be right, I hope they will consider me for the dog feeder position :-).