03 January 2016

MLB Demonstrates the Uselessness of Projection Systems

Major League Baseball is touting today the Steamer baseball projections for 2016. Steamer is one of several Sabermetric projection systems -- including Fangraphs' ZIPS, Baseball Prospectus's PECOTA and the Hardball Times' Oliver -- that offer a crystal ball of sorts on future player and team performance.

These systems generally work the same way, projecting the accomplishments of players by comparing their careers to similar players in history, averaging the performances of those similar players in their next season and adjusting that to the player in question.

The MLB.com story is not a particularly interesting read. It projects that Miguel Cabrera will win another batting title, Clayton Kershaw will again top the pitching charts and Giancarlo Stanton will once more outslug the field. 

Shocking.

This is what you get with projection systems, which have no ability to predict the unpredictable. The 2015 accomplishments of Jake Arrieta, Dee Gordon, AJ Pollock, Dallas Keuchel, the Kansas City Royals and New York Mets were no less a mystery to these projection systems than to you.

They also have no ability to predict outlying performances. They will never project any player will slam 58 home runs, bat .360 or accrue 21 pitching wins unless he has done that regularly in the past. 

Moreover, Steamer does not attempt to project playing time, so even though Stanton has missed 40 games a season in his career, the projection assumes he plays 150 games. (Some of these systems do take playing time into account, which means they are wildly wrong slightly less often.)

These systems gave us the Angels, Dodgers and Nationals as baseball's best teams in 2015. They got the AL East and West almost exactly wrong. They offer some mild insights that can be almost completely encapsulated in 13 simple rules.

So don't bother reading the article. (Read this one about Statcast instead.) And let's just enjoy the raging unpredictability of the game that served us so well last year.

No comments: