I have been thinking a lot about probability recently. I was never interested in probability as a mathematician, but that's because probability is a tool for reasoning rather than a branch of mathematics. Mathematics is one way to reason, but so is programming or just plain old prose. Probability is a framework for understanding knowledge, especially when the environment is uncertain, noisy or simply too large. In other words:
Knowledge under uncertainty. Whether you want to predict the weather or the stock market, you want to extract structure in situations where you want to predict the future without knowing exactly how things will turn out.
Knowledge of large numbers. Given enough time and space, almost anything will happen. Probability gives us useful tools for understanding the likelihood of those events.
This week's newsletter is about the latter. I am going to illustrate the use of probabilistic thinking. Without equations. Let's take a look at a situation we are all familiar with today:
The network era was supposed to make the world flat, to make it easier for anyone to imagine and share ideas and make new products while cutting out the middle man, i.e., the industrial era firm. The early internet was certainly a time of great hope. It was the cyberwild with all the attendant lawlessness and chaos. In less than two decades, we find ourselves in a world with increasing inequality; the internet is dominated by a few platforms such as google, facebook and amazon. How did that happen?
Some of it is a conspiracy, i.e., a deliberate attempt by those with money and power to capture and tame a free space. However, I think a lot of it is a natural consequence of basic laws of probability.
Imagine a slowly biased coin toss, that starts with a fair coin but adds a tiny bias in favor of the winner each time. Perhaps the coin gathers mud on the side it lands, which makes it slightly more likely to land with the cleaner side up. In a social or business setting, the bias arises because transaction costs are lowered, or because the winner gets more publicity. There are many reasons.
Now add a "game of life" assumption, that people will play the game only if they think it's fair. A biased coin toss makes the game of life harder for losers. Suppose each time someone's win percentage goes below a certain number (let's say 40%) they quit the game. Again, we know that people tend to quit when the cards are stacked against them - behavioral game theorists have shown in the ultimatum game that people prefer walking off the table than to accept a patently unfair payoff. If we start with an unfair game, many people will choose not to play. Combine these two features, toss the coin a few thousand times and you will see the basic features of structural inequality in place.
Further, it's clear that the real winners are not the players but the platforms, i.e., those who own the playing field on which the coin is tossed. Google and amazon are not that different from vegas casinos; it doesn't matter whether you are a high-roller or a grandmother staking her retirement nest egg, the house always wins, statistically speaking.
The existence of choice is crucial - people in caste societies are forced to adopt their parents' profession, but once you add the freedom to migrate out, you get a very different dynamic. Participation in online networks is entirely driven by choice - no one is forcing you to participate, nor are they interested in making you do so.
Paradoxically, the presence of choice and perceptions of unfairness at the individual level works against the collective good. The very features that we consider central to a just society (such as choice and autonomy) can have the opposite consequences when aggregated. Networks can amplify inequality with nary a conspiracy in sight.
Sidenote - the same dynamics are at work in politics. Corrupt politicians are not-so secretly pleased when decent people are put off by corruption in politics; it makes it less likely that the decent person will enter the political arena. That absence makes it easier for the sleaze bags to stay in control. Like the house in Vegas, the real winner is the political party. As platforms, parties reap outsize benefits from aggregating products, i.e., individual MLAS and congressmen. The consequences are there for everyone to see: our elected representatives have less and less room to vote their conscience or their constituents interests over the parties'.