Big data doesn’t solve the biases in our thinking

Our intuitions are based on simplification. Intuitions help us to make quick decisions when we don’t have time to think things through thoroughly. However, these simplifications often mislead us, even if we have more time to ponder our decisions. We unconsciously search for reasons to reinforce our first impression, because it is easier to find facts supporting the intuition, rather than to look for alternative explanations.

Young Israeli psychologist Daniel Kahneman noticed in the early phase of his career that the recruitment process in the Israeli military was based on the recruiter’s intuition. He quickly discovered that the recruiters’ views did not add any value to the process. Actually, the professional recruiters often did worse than simple statistical rules would have done. This interest in the cognitive biases behind systematic errors led Daniel Kahneman to win the Nobel Memorial Prize in Economics.

The same biases that caused recruiters to choose the wrong candidates are present in strategic thinking

“A few years back, a case company acquired a company called Alfa (fictive) in order to expand into a new market segment. Later, the segment significantly slowed down, and since then, Alfa has struggled to reach its targets. Recently, the management defined a new strategy in which there is no room for Alfa. However, they are not too eager to sell Alfa, because the offers received have been lower than expected, and they were encouraged by some progress in the last quarter which gave hope that Alfa’s underperforming might be temporary. They believed the value of the company would increase if the divestment were postponed.”

The management’s argument seems logical, but one can find potential biases if one looks at the situation from the perspective of Kahneman:

1. Confirmation bias: Seeing only the facts that confirm one’s existing view.
The management wants to see progress in Alfa, as its acquisition was their decision. Thus it sees only the supporting facts.

2. Base rate bias: Overemphasizing the details and forgetting the context.
The management overvalues the company’s potential to improve its operations and forgets the base rate: the whole market is falling. Success is unlikely, even if performance improves.

3. Endowment bias: Overvaluing things merely because they own them.
The management sees more value in Alfa simply because they own it.

4. Availability bias: Taking a mental shortcut by putting too much weight on recent information and facts that are easily recalled. The management’s view of the company’s performance is too much affected by the last quarter’s performance.

Strategic decisions are all about uncertainty

Companies collect increasing amounts of data in order to decrease the level of uncertainty. But additional information only increases the complexity and hinders decision making, if the management still has the same presumptions and sees the facts selectively. More data may even increase the chance of it reinforcing our often incorrect presumptions. Kahneman came to a similar conclusion and improved the recruiters’ decisions by limiting the collected information to only six personality traits. It caused a clear improvement in the interview process. “Big data” doesn’t help if we can’t even cope with “small data”.