How to fix the NFL's instant replay system.

How to fix the NFL's instant replay system.

How to fix the NFL's instant replay system.

The stadium scene.
Dec. 22 2009 12:25 PM

After Further Review

The NFL's instant replay system doesn't work. Here's how to fix it.

Illustration by Mark Alan Stamaty. Click image to expand.

Every serious football fan knows the central tenet of the NFL's instant replay system: "A decision will be reversed only when the referee has indisputable visual evidence" that the call made on the field was incorrect. While we're all quick to complain when we don't agree with a replay reversal, few observers ever ask the obvious question: Why is the league's replay standard so demanding?

Earlier this month, Duke law professor Joseph Blocher asked that very question at PrawfsBlog, and since then several more bloggers have weighed in. Many start from the common legal wisdom that a fact-finder usually minimizes errors in adjudication by returning the verdict he believes most likely to be right. That's why, in a tort suit or one for breach of contract, the jury will find for the plaintiff if he proves his case by "a preponderance of the evidence"—a fancy way of saying "more likely than not." Sometimes the jury will mistakenly find for the plaintiff when the true facts favor the defendant, and sometimes it will err in the opposite direction. But, on reasonable assumptions, the "more likely than not" standard of proof is the best way to maximize correct verdicts and minimize incorrect ones.


In criminal law, our goal is not to minimize total errors. Believing, with the 18th-century English jurist William Blackstone, that it is "better that ten guilty persons escape than that one innocent suffer," the Anglo-American system deems it especially important to minimize erroneous convictions. That's why a criminal defendant must be acquitted unless the jury is persuaded of his guilt "beyond a reasonable doubt"—about the most demanding standard of proof known to law. It is designed to ensure that there will be very few erroneous convictions. However, it achieves that end at a high cost: lots of erroneous acquittals.

The NFL's "indisputable visual evidence" standard goes still further. Pro football's replay rules would seem to ensure very few erroneous reversals but at the cost of tolerating a great many erroneous initial calls. That tradeoff would make sense if we believe it "better that ten erroneous calls be allowed to stand than that one correct initial call be reversed." But that seems silly. After all, the very point of introducing instant replay was to reduce errors—"to get the play right." If we want fewer errors, our best bet is to allow the referee to reverse an initial call if he concludes that a contrary call is more likely to be right. In the legal world, that's what's known as de novo review.

Notwithstanding the written rule, isn't it possible that referees are applying something close to de novo review? After all, coaches' challenges succeed from  40 percent to 50 percent of the time. While you might think such a high success rate offers little reason to believe refs abide by the indisputable visual evidence standard, consider that the conviction rate in American criminal prosecutions stands just above 90 percent. Still, few if any scholars think that fact alone provides good evidence that juries do not adhere to the "beyond a reasonable doubt" standard of proof; the high conviction rate is consistent with the supposition that, by and large, prosecutors only bring strong cases. Similarly, a high reversal rate for instant replay is consistent with the possibility that coaches use their challenges wisely.

There are reasons to think that's so. First, because coaches have a limited number of challenges and lose a timeout if a challenge fails, they have every incentive not to make challenges that are likely to lose. Moreover, the facts that reversal rates have increased over time—coaches were successful on 29 percent of challenges in 1999—and that coach-initiated challenges succeed at twice the rate of booth-initiated challenges suggest that coaches are learning to issue challenges more judiciously. So while it appears to me—as it does to many fans—that refs don't always adhere to the strict IVE standard, it seems hard to deny that they apply a standard significantly more demanding than de novo review.

Even assuming that's right, many reasons have been proposed to keep the NFL's replay system the way it is now. None seems very convincing.