Moneyball’s Lesson for Teacher Evaluations: Metrics Need Context

Moneyball’s enduring lesson may be less about replacing judgment with statistics than about asking what the numbers leave out. Billy Beane’s Oakland Athletics found overlooked baseball talent by changing how they measured players. Applying that logic to teaching raises a harder question: can a model capture the work that matters most in a classroom?

“Not everything that can be counted counts, and not everything that counts can be counted.” The saying is attributed to Albert Einstein, and it frames the tension between measurement and judgment at the heart of both debates.

Finding value in overlooked players

The 2002 Athletics had a $41 million player payroll; the New York Yankees spent $126 million. Both teams won 103 games, and Oakland reached the American League playoffs, where it nearly beat New York. The Athletics also set an American League record with a 20-game winning streak that season.

As the 2003 book Moneyball recounts, Beane questioned the scouting habits that favored visible, familiar measures of potential. Pitchers were judged on velocity, pitch selection and control; position players on tools such as hitting, speed and fielding. Oakland could develop players who looked promising by those standards, only to lose them to wealthier teams such as the Yankees, Red Sox and Phillies. By 2001, the Athletics had been left with a depleted roster.

Beane’s response was to change the question. Instead of paying for a package of star qualities, he asked analysts and scouts to find players whose less celebrated strengths could help the team win. Getting on base, for instance, mattered more to his approach than batting average alone. Several affordable players could collectively provide the contributions of one expensive star.

That strategy put Beane’s analysts in conflict with veteran scouts. In the film Moneyball, scouts object to computer printouts and percentages because they trust years of experience, observation and feel for the game. The dispute wasn’t simply between people who liked numbers and people who didn’t. It was about which evidence should guide decisions when a team couldn’t afford to follow conventional wisdom.

What teacher evaluations can measure

That scene brought to mind debates over teacher effectiveness. I’ve argued, alongside many researchers, teachers and administrators, that classroom teaching shouldn’t be reduced to a single score based partly or entirely on student test results. Some school systems have instead used algorithms combining several measures to rate teachers, make dismissal decisions and award bonuses.

A statistical model used by the New York City school system to calculate teacher effectiveness. Caption as printed in the New York Times, March 7, 2011.

Relationships with students and the ability to inspire them can shape behavior and achievement, yet those parts of teaching resist simple measurement. Test scores may reveal something about student performance, but they can’t, on their own, describe the full range of a teacher’s work. The Einstein quote points to the risk: a metric can be useful while still missing what matters.

Washington, D.C., offered a stark example in 2010. Using an algorithm based evaluation system, the district fired 241 teachers—about 4 percent of its teaching staff—and warned another 730 that they could be fired if their performance didn’t improve. Houston superintendent Mike Miles used a similar approach in Dallas from 2012 to 2015.

Better measures still need human judgment

Beane’s record complicates the idea that a new metric settles the debate. He led Oakland from 1997 to 2015, and the team reached the American League playoffs in 2003 and 2006. It hasn’t returned since.

That record doesn’t erase the value of looking beyond familiar measures. It does show why metrics should inform decisions without pretending to resolve every question. In baseball and in schools, the challenge is to recognize what the numbers reveal while staying alert to the qualities they can’t capture.