Fangraphs Releases Aggregate Defensive Ratings

A few weeks ago, I noted that the various freely available defensive metrics were all over the map when it came to measuring the glove-work of Robinson Cano. The variation in all of those metrics is illustrative of the still nascent nature of defensive metrics, and suggests that relying entirely on one statistic is folly. Instead, I advocated the following:

So what do we do? How do we evaluate defense? At this point, the best method is to use all of the data that we have available to us, both statistical and observational. Look at UZR, DRS, TotalZone, Fan Scouting Reports, nFRAA, fielding percentage, errors, putouts, assists, other fan opinions, the thoughts of scouts, and your own observations as well, weighing each factor as you see fit. When the composite of those inputs allows you to reach a conclusion, try to avoid presenting that conclusion as being exact or perfect, as a different weighting of the inputs would likely yield a disparate result. In sum, use the metrics, but be prudent and always be cognizant of their limitations.

Yesterday, Fangraphs released a new composite measure that should make weighing the various defensive data points we have considerably easier. I’ll let Fangraphs founder David Appleman explain:

These days it’s common place to look at multiple defensive metrics to try and get a good grasp of a player’s defensive value. On FanGraphs we even carry four different defensive metrics that include Mitchel Lichtman’s UZR, Sean Smith’s Total Zone (with Location), John Dewan’s DRS, and Tangotiger’s Fan’s Scouting Report.

All of them have different methodologies and the four rely on three different data sources.

To make comparing the four easier, there is now a new stats table on the player pages called “Aggregate Defensive Ratings” (ADR), where the UZR, DRS, TZL, and FSR are given a weighted average and there’s even a standard deviation and standard error given for the sample of four defensive metrics.

The general weighting is 1/3/3/3 for FSR/UZR/TZL/DRS. For years where FSR is not available, UZR, DRS, and TZL are weighted evenly. Only players with more than 50 innings played at a position are included in ADR.

Basically, ADR combines 3 defensive metrics to provide a composite number that should be a fairly good estimate of what the statistics show regarding a player’s defense. It also includes the Fan’s Scouting Report, which lends some weight to the “eye-test,” although the fact that it is made up of scouting reports from Fangraphs readers means it is likely skewed a bit by the defensive metrics. I would still take a look at other metrics that are not included here, as well as factor in reports from scouts, fan opinion, and personal observations, but this goes a long way towards simplifying the process of sifting through the defensive data and coming up with a number that we can feel comfortable with.

Let’s take a look at the data on Cano, whose stats got this discussion started. The numbers are for 2005-2010:

As you can see, after his awful rookie season in 2005, ADR has Cano as a pretty solid defender (+7 for 2006-2010), as his poor UZR ratings are offset by strong DRS and TZL numbers, as well as solid FSR scores for the last two seasons. Essentially, poor performance in an outlier metric is overcome by the other stats, but it is not ignored and impacts the final evaluation. Another important feature of this system is that it gives an error bar (2nd column from right) for the overall statistic, something that has been desperately needed for defensive metrics for a long time. Because these metrics are not intended to provide an exact measure of defense, it is more useful to know that Cano likely ranged from +1 to +7 in 2009 rather than to simply conclude that he was worth exactly 4 runs on defense.

I think ADR is an excellent addition to the statistical toolbox and intend to use it regularly here at TYA. Hopefully it catches on.

Follow Me On Twitter