Fifteen years ago, NFL general managers spent months debating who would be a better NFL quarterback: Ryan Leaf or Peyton Manning. A majority of them sided with Leaf, which proved to be one of the great miscalculations in sports history. Peyton Manning, of course, will retire as the best quarterback to ever play, while Leaf washed out of the NFL after three years and 14 touchdown passes. Last year, Manning threw seven in one game.
NFL teams employ dozens of scouts and experts to evaluate prospects precisely to avoid these types of mistakes. Despite this, over the draft’s 77-year history, teams have learned remarkably little about the draft process. In a 2011 paper titled “The Loser’s Curse,” behavioral economists Richard Thaler (of Nudge fame) and Cade Massey show that the Leaf miscalculation was no aberration; NFL teams are universally terrible at drafting football players. In fact, since teams so fundamentally misjudge player potential, Thaler and Massey found, the 32nd pick (which goes to the Super Bowl champion) is more valuable than the first pick (which goes to the worst team). Not only that, but the chance that any given player selected will be better than the following player taken at the same position is only slightly better than a coin flip.
For most if its history, the draft worked like a marketplace without a pricing system. In 1991, Mike McCoy, then-co-owner of the Dallas Cowboys and an engineer by trade, created a draft value chart—referred to simply as “The Chart”—which assigned a numerical value to each draft pick to facilitate trading. For example, the first pick is worth 3,000 “points,” whereas the last pick is worth 0.4 “points.” When a team considers a trade, they use simple addition to decide if the trade is worthwhile. McCoy described it as akin to putting a price tag on a loaf of bread, allowing teams to determine if their loaf is worth more than the other team’s. The problem with this system is two-fold: the values were based on the NFL’s trade history, itself often the result of haphazard assumptions about draft pick values, so in a way it was a quantification of biases. Second, as The Chart spread through NFL circles, every team was using the same math to evaluate trades based on the same faulty system, compounding errors on errors. Nevertheless, few teams were concerned and welcomed a codified valuation. “A standard price list also protects you,” McCoy told Thaler and Massey. “Nobody gets skinned.”
In the draft, a risk-averse strategy would involve trading high, expensive picks for as many lower-rounders as possible. Yet we rarely see teams do this since the conventional wisdom dictates that high picks are the best. In the NFL, “risk aversion” means abiding by conventional wisdom.
McCoy’s quote is a clue as to why NFL teams draft so poorly: It’s no longer an economic problem, but a psychological one. The economics of the draft have been studied for almost two decades now: Stating that first round picks are overvalued is almost a cliché at this point, while, due to the uncertainty of translating performance in college to performance in the NFL, the best approach, generally, is to simply try to draft as many players as possible. The reason teams continue to ignore these findings has more to do with inherent biases that prevent experts from rationally assessing the draft.
AS PSYCHOLOGISTS HAVE FOUND, we all suffer from the overconfidence effect, in which we are more sure of our answers than we ought to be. But experts are even more prone to this mistake. This is precisely what we see with the NFL draft every year, as scouts insist, for instance, that a certain quarterback will or won’t succeed despite years of data and numerous failures that show there really is no way to determine what quarterbacks will go on to have successful NFL careers.
To make the overconfidence effect even more pronounced, as Thaler and Massey wrote, the more information experts have to base their decisions on, the more confident they become. This wrinkle is particularly relevant this year, with the NFL draft being held two weeks later than normal. Teddy Bridgewater, a quarterback out of Louisville, provides a good case study: Over the past month or so, Bridgewater has fallen from being viewed as the top quarterback, and possibly the top player, in the draft to someone who’s not even worth a first-round pick. The number of games he’s played during that time: zero. Teams have been able to study Bridgewater for months, so what gives? With all this extra downtime to prepare for the draft, teams have time to second-guess themselves.
General managers, equipped with all this added time, will demonstrate two seemingly paradoxical beliefs: That they have gained an upper-hand on the other teams through exhaustive research (Bridgewater is actually not good!), except they’ll then believe that other teams value players similarly to them (the news Bridgewater isn’t good becomes public knowledge). This is what is known as the false-consensus effect, in which people tend to overestimate the degree to which others share their beliefs.
On the back of the false-consensus effect, another cognitive bias gets triggered: the winner’s curse. Let’s say, as a result of a new perspective on poor Teddy Bridgewater, a team needing a quarterback decides they want Johnny Manziel. Per the false-consensus effect, if every team thinks they value Manziel similarly, then the winning bidder will presume that it has to overpay in order to get him. If a team generally thinks Manziel is worth the fifth pick, then the team who gets him is likely to draft him higher than that to ensure they get him. Since the key to a successful draft is picking as many players as possible, overinvesting in any single player is the biggest mistake a team can make.
None of these biases are crippling or even particularly detrimental to the drafting process so long as they are recognized and accounted for. But to admit a lack of confidence—or, god forbid, total ignorance—is not a part of the job description for scouts or general managers. These are exactly the types of questions they’re paid to answer.
Ironically, admitting their limitations would make GMs better at their jobs. But they risk getting fired if they admit they don’t know such things, since it is their job to draft the right players. Ultimately, GMs care most about remaining employed. Although team success is one way to make that happen, by definition, only a small minority of teams can be successful. On the other hand, abiding by conventional wisdom limits the degree to which they can be second-guessed. As Cade Massey recently noted on the Advanced Football Analytics podcast, this results in a bizarre backward kind of risk aversion. Typically, risk aversion involves diversifying, therefore reducing the degree to which any single decision can harm you. Investors do this by sticking small amounts of money in lots of different stocks. In the draft, an analogous risk-averse strategy would involve trading high, expensive picks for as many lower-rounders as possible. Yet we rarely see teams do this since the conventional wisdom dictates that high picks are the best. In the NFL, “risk aversion” means abiding by conventional wisdom.
WHEN THERE ARE SOCIAL norms, there’s safety. Once The Chart became common knowledge, GMs felt trading picks was less risky because they had widely accepted conventions to defend them against scrutiny. As a result of their increased comfort, draft-pick trades tripled. There’s no evidence any GMs seriously questioned the valuation of the chart, which has since been debunked by academics and statistics experts.
This employment-centric form of risk aversion manifests itself most clearly in one bold and illogical way: the future draft pick trade, where a team trades picks in future drafts for a pick this year. Such a trade is often a desperation move for the general manager receiving current picks, fearing for his job security—he needs his team to get better immediately or he might be fired. As a result, his desperation commands a premium. Thaler and Massey calculated a discount rate of 136 percent per year, an astonishingly high number. (Would you take out a loan with a 136 percent interest rate?) When you think about it, there’s little reason the discount rate should be higher than zero: A first round pick next year is worth the same as a first round pick this year. The difference, of course, is the general manager might be fired before he gets to use it. While cognitive biases certainly play a major role in draft day inefficiencies, few general managers have a strong incentive to break from the social norms that govern those biases. In fact, they are incentivized to abide by them and hope everyone else is doing the same.
One of the few who doesn’t have to worry about his job security is Bill Belichick, coach of the New England Patriots, who has won three Super Bowls. It’s become a bit of a running joke in NFL circles that one year Belichick collected two first-round picks and then traded one of them for a second-round pick that year and a first-rounder the following year, only to repeat the same trade every year, leaving him with an extra second-round pick every year that he essentially procures out of nowhere. Of course, it’s not really out of nowhere, just out of the space between biases and norms other coaches haven’t bothered to look. To them, it might as well be imaginary.
