Someone must have checked, right?!
Correlated vs. independent sources and the case for diversifying our social interactions
Imagine the journalist who, invited by a foreign power, visits a village to learn about how peasants live. All villagers he talks to are apparently very happy with their living conditions, have abundant meals, the best tools for their jobs, plenty of free time, and amazing supervisors. Hearing the same story from different people sounds like strong evidence that life is indeed great in the village. After all, everybody seems satisfied. But life is anything but great there. The catch? It’s a Potemkin village. All villagers were instructed what to say (and, importantly, what not to say) to fool the naïve foreigner. Ultimately, there is a single source of information—the party cadre—regardless of how diverse and numerous the messengers seem.
The naïve journalist exemplifies how people can mistake correlated beliefs for independent ones. Besides the party cadre, another species in the business of intentionally manipulating people’s beliefs is the industrial spy. When spotted in a high-security area, spies want to make others believe that they had been vetted independently by many employees: the receptionist, the security guard, the floor manager, and others whose job it is to ensure that only authorized people enter certain premises.
The reality may have been very different. The spy may have simply charmed the receptionist into using her credibility with the security guard to gain entry. The floor manager, in turn, might not question anybody who is being accompanied by the security guard—and so on. Multiple layers of security are an illusion in this case. It’s only the receptionist who engaged in some form of—not very successful—vetting. Cognizant of this security risk, companies frequently try to mitigate it by having protocols in place.
Intentional manipulation attempts are alarming, but they are more the exception than the rule. Far more often, people simply fail to recognize that all their sources can be traced back to a single source.
The making of the East Grevanor expert
An evening TV program is desperate for an expert on the faraway region of East Grevanor due to some recent events. As no one is quite an expert on the territory, the producer randomly picks a charismatic guy in a suit to say some generalities that could apply to any event in any territory.
Producers of the morning program need an East Grevanor expert too. Facing the same scarcity of experts, they pick the only known expert—the random guy from the evening program. They mistakenly assume that the evening program had conducted due diligence before hiring the guy. Repeat this a few times, and every self-respecting source will only want to hire the now-consensus East Grevanor expert.
Join the herd or follow the sign?
Like the producer who outsources due diligence to others, the airport passenger at an unfamiliar airport is frequently in a similar predicament. The typical pattern is that disoriented passengers follow other passengers ahead of them to locate the exit. What initially seems like a group of 50 independent exit searchers might occasionally turn out to be an arbitrary aggregation of individuals, each sheepishly following the person in front of them. If the very first person in the crowd is looking to relieve their bladder or catch another flight, the whole group winds up at an unintended location.
The silver lining
The example of 50 passengers lost in a terminal building is arguably not very realistic. It’s unlikely you won’t find the exit if you have that many people leave the aircraft before you. This is because people can easily form an independent judgment about the exit by reading signs, and many will simply know their way around the airport. Frequently, however, forming an independent judgment is costly, so we use heuristics instead.
One of these heuristics is to associate the strength of evidence with the number of believers. This strategy works well in most situations. We save substantial time by relying on the knowledge and experience of others. If we didn’t do that, we would be spending our days independently verifying everything. This wouldn’t be feasible. But more importantly, most things just don’t matter that much as to warrant independent vetting. We accept some false beliefs as the cost we need to pay to go about our days normally.
Operation Independent Beliefs
The fact this count-the-believers heuristic works well in most cases doesn’t mean we cannot do better when it matters. We can—by focusing our attention on the number of independent believers (or by accounting for correlated beliefs in some other way). The first step in this sequence is recognizing when there is a high danger of correlated beliefs. For example, it makes little sense to go to a political rally to gauge the popularity of the politician holding the rally. Similarly, asking a large group of hairdressers about the desirability of having a stylish haircut is unlikely to lead to a correct belief. Even though both cases involve large groups of people with some relevant experience, their beliefs are highly correlated. Unfortunately, most cases are not this clear-cut, and correlated beliefs are frequently difficult to spot.
Therefore, our working assumption should be that our sources have correlated beliefs and that we are blind to what these correlated beliefs are. This mindset creates an opportunity for improvement. As we don’t know what our sources’ correlated beliefs are, we can only try to seek out people with less correlated beliefs overall. The idea is that if two people’s beliefs have a low correlation on observable issues, they might have a low correlation on the relevant (and unobserved) issue as well.
Just like it’s important to diversify one’s financial portfolio, most of us would benefit from increasing our exposure to groups currently underrepresented in our social circles. As we tend to have fairly homogeneous social circles, tapping into practically any random segment of society is likely to lead to less correlated sources. It won’t be a perfect solution by any means, but it’s a good place to start.
This post was inspired by a recent 80,000 hours podcast episode with evolutionary and cognitive psychologist Hugo Mercier.