Mistakes to avoid when drafting millions
Draft lotteries, weak-link problems, and the power of redundancy
WWII draft lottery
Two-thirds of the 15 million men who served in the U.S. Armed Forces during WWII were draftees. The order in which they were to be called up was determined by a draft lottery, the first (and largest) of which occurred on 29 October 1940, a couple of months before the U.S. entered the war.
For the purposes of this draft, the U.S. was divided into local areas. In each local area, men between the ages of 21 and 35 were required to register and were allocated a draft number. Each number was unique within a local area but was repeated in all (or almost all) local areas across the country. A central lottery was then organized to establish the draft order in each local area by drawing the draft numbers from a glass bowl.
Great care was taken to ensure the lottery process would lead to a fair outcome. Given the level of scrutiny and preparation, one would assume there would be no scope for error. And one would be wrong.
Oops, something is amiss
The figure below shows the order in which the draft numbers were drawn. Looking at the almost empty bottom left corner of the plot, it’s clear something went wrong. Numbers in the hundreds were very unlikely to be drawn early on.
To understand what happened, we need to have a closer look at the lottery drawing—and the preparation preceding it.
The draft numbers were placed into opaque capsules, which were then stored in small paper boxes, arranged in roughly consecutive order, and grouped in units of 100. They were then poured into a glass bowl, which was a remnant of the WWI draft lottery. Shortly before the lottery started, public officials realized that the glass bowl was not large enough to accommodate all the capsules. So, they added a plastic collar as a quick fix. They then took a small wooden paddle and shuffled the capsules in the bowl.
As the numbers were introduced sequentially, low numbers went to the bottom of the bowl. Insufficient shuffling could have arisen because the paddle couldn’t reach the bottom of the bowl that was now deeper due to the last-minute plastic extension, or because some capsules broke open early on, disincentivizing proper shuffling. No matter its origin, insufficient shuffling led to the capsules not being distributed randomly before Secretary of War Henry L. Stimson reached into the bowl to draw the first number.
The non-random distribution of the capsules wouldn’t be a problem as long as the capsules were drawn randomly. The problem is—they weren’t. As the surviving visual evidence suggests, the public officials conducting the lottery mainly picked from the top of the bowl. From their perspective, the capsules were randomly distributed in the bowl, so picking from the top would be equally likely to yield a low or a high draft number. Not worried about introducing a bias, their incentive was to avoid getting elbow-deep into the glass bowl, potentially breaking capsules open or messing up something else in the process. Human drawing is non-random drawing. Even if these officials had been aware of the non-random distribution of capsules and tried to draw randomly, it’s unclear whether they would have succeeded. It’s difficult to recreate a random process without an external randomizing device.
Both stages—the lottery preparation and the lottery drawing—are weak-link problems. In cases like these, the overall outcome is determined by the performance of the weakest component in a system, regardless of how other system components behave. In the preparation stage, the weakest link was the insufficient shuffling. You could use the most opaque capsule or the highest-quality wooden spoon, but all this won’t matter if you fail to shuffle the capsules. Similarly, the weakest link in the actual lottery was the non-random drawing by public officials; other details, such as wearing blindfolds while drawing, are certainly great for show, but they won’t fix the non-random drawing.
OK, so this means that the whole drafting process was botched, right? Not so fast. Even though the organizers made a very serious mistake with the lottery, they had foresight in one crucial aspect. To see what this foresight was, recall that draft-age men had been allocated the draft numbers by their local areas. Crucially, the allocation of numbers was random, as opposed to something non-random, such as allocating numbers on a first-come-first-served basis.
In other words, there was randomization on two levels in the WWII draft: centrally (the glass bowl lottery) and locally (at the local draft boards). The first one failed, but the second one held up. As long as at least one of the randomizations held up, the draft process was fair overall.
Good morning, Vietnam
A generation later, the U.S. was at war again. This meant a new draft lottery. This time, birthdays were used to establish the draft order, displacing the WWII system of local boards allocating draft numbers to eligible men.
The lottery took place on 1 December 1969. Five days later, a report was already out, claiming that men born in December were much more likely to be drafted than those born in January. So, something went wrong again with the lottery. This time, however, there was no local randomization that would make the (embarrassing) lottery error ultimately inconsequential. Given that the dangers of a non-random lottery must have been salient in the organizers’ minds, it’s pertinent to ask: How did the same error happen again?
Preparing for this lottery, the capsules containing the birthdays were inserted into a square wooden box in monthly batches, starting with January. The newly introduced batch was counted and mixed with the existing capsules in the box. A public official then took this box, shook it a couple of times, and poured its contents into the lottery bowl.
The December capsules were much more likely to end up on top, reflecting the sequential placement of capsules and insufficient shuffling. Unsurprisingly, the human tendency to pick from the top of the bowl didn’t change either.
These two factors produced the non-random draft order, just like a generation earlier.
After this draft, the importance of having several independent randomizations was taken to heart. In the next draft, a bizarrely complicated system was in place that involved 4 independent randomizations and 2 independent rotary drums. An overkill for sure, but things at least didn’t fail that time. Given past blunders, the overkill might have been justified to restore trust in the fairness of the draft lottery.
Some lessons
The WWII and Vietnam draft lotteries offer three broad lessons. First, things can go wrong even when many people work hard to avoid failure, and the task is relatively straightforward. Furthermore, making one type of mistake in the past doesn't mean the lessons will be internalized and the same mistake won’t be made again. Second, establishing a fair draft order is a weak-link problem; if any component fails, the whole process fails. The third and final lesson is that building in redundancy (e.g., 2 independent randomizations) can protect against consequential errors as long as the two systems are uncorrelated. Sometimes, however, sources of failure that seem uncorrelated are actually correlated because they share the same hidden vulnerabilities (e.g., the same human tendencies drive insufficient shuffling and non-random drawing).